--- title: How to deploy containerd with kata containers? url: https://devopstales.github.io/kubernetes/kata-container-containerd/ date: 2021-08-20 keywords: Kubernetes, kata-container, containerd, qvemu, kvm, nerdctl, AlmaLinux --- In this post I will show you how you can install and use kata-container engine in kubernetes. <!--more--> {{< content "/filedir/k8s-sec.html" >}} ### What is Kata container engine Kata Containers is an open source community working to build a secure container runtime with lightweight virtual machines that feel and perform like containers, but provide stronger workload isolation using hardware virtualization technology as a second layer of defense. (Source: [Kata Containers Website](https://katacontainers.io/) ) ![Kata container engine](/img/include/katacontainers.webp) ### Enable qvemu I will use Vagrant and VirtualBox for running the AlmaLinux VM so first I need to enable then Nested virtualization on the VM: ```bash VBoxManage modifyvm alma8 --nested-hw-virt on ``` After the Linux is booted test the virtualization flag in the VM: ```bash egrep --color -i "svm|vmx" /proc/cpuinfo ``` If you find one of this flags everything is ok. Now we need to enable the kvm kernel module. ```bash sudo modprobe kvm-intel ``` ### Install Kata container engine I use Almalinux 8 for this install: ```bash sudo -E dnf install -y centos-release-advanced-virtualization sudo -E dnf module disable -y virt:rhel cat <<EOF | sudo -E tee /etc/yum.repos.d/kata-containers.repo [kata-containers] name=Kata Containers baseurl=http://mirror.centos.org/centos-8/8/virt/x86_64/kata-containers enabled=1 gpgcheck=1 skip_if_unavailable=1 EOF sudo -E dnf install -y kata-containers ``` ### Install and configure containerd First I install containerd then I add Kata container as a containerd plugin to the config. ```bash sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo sudo dnf install -y containerd.io sudo mkdir -p /etc/containerd sudo containerd config default > /etc/containerd/config.toml nano /etc/containerd/config.toml ... [plugins."io.containerd.grpc.v1.cri".containerd.runtimes] [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] ... [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata] runtime_type = "io.containerd.kata.v2" # Restart containerd sudo systemctl restart containerd systemctl enable containerd.service echo "runtime-endpoint: unix:///run/containerd/containerd.sock" > /etc/crictl.yaml crictl ps ``` Now I can start a Kata container from commadnline. ```bash sudo ctr image pull docker.io/library/busybox:latest sudo ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest hello sh ``` ### Install nerdctl I like to use `nerdctl` instad of `ctr` or `crictl` cli so I will install it. ```bash wget https://github.com/containerd/nerdctl/releases/download/v0.11.0/nerdctl-0.11.0-linux-amd64.tar.gz tar -xzf nerdctl-0.11.0-linux-amd64.tar.gz mv nerdctl /usr/local/bin nerdctl ps ``` ### Install Kubernetes Configure Kernel parameters for Kubernetes. ```bash cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf overlay br_netfilter kvm-intel EOF sudo modprobe overlay sudo modprobe br_netfilter cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 EOF sysctl --system ``` Disable swap for Kubernetes. ```bash free -h swapoff -a swapoff -a sed -i.bak -r 's/(.+ swap .+)/#\1/' /etc/fstab free -h ``` The I will add the kubernetes repo and Install the packages. ```bash cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF dnf install kubelet kubeadm kubectl -y ``` Start Kubernetes with containerd engine. ```bash export IP=172.17.13.10 dnf install -y iproute-tc systemctl enable kubelet.service echo "KUBELET_EXTRA_ARGS=--cgroup-driver=systemd" | tee /etc/sysconfig/kubelet kubeadm config images pull --cri-socket=unix:///run/containerd/containerd.sock kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=$IP --cri-socket=unix:///run/containerd/containerd.sock ``` ```bash mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config kubectl get no nerdctl -n k8s.io ps crictl ps kubectl taint nodes $(hostname) node-role.kubernetes.io/master:NoSchedule- ``` ### Inincialize network ```bash wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml kubectl aplly -f kube-flannel.yml ``` OR ```bash kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml wget https://docs.projectcalico.org/manifests/custom-resources.yaml nano custom-resources.yaml ... cidr: 10.244.0.0/16 ... kubectl apply -f custom-resources.yaml ``` ### Start Deployment First I create a `RuntimeClass` for Kata then start a pod with this `RuntimeClass`. ```bash cat<<EOF | kubectl apply -f - kind: RuntimeClass apiVersion: node.k8s.io/v1 metadata: name: kata handler: kata EOF cat<<EOF | kubectl apply -f - apiVersion: v1 kind: Pod metadata: labels: app: untrusted name: www-kata spec: runtimeClassName: kata containers: - image: nginx:1.18 name: www ports: - containerPort: 80 EOF ``` ```bash $ kubectl get po NAME READY STATUS RESTARTS AGE www-kata 1/1 Running 0 2m47s $ kubectl describe po www-kata ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 2m42s default-scheduler Successfully assigned default/www-kata to alma8 Normal Pulled 2m13s kubelet Container image "nginx:1.18" already present on machine Normal Created 2m13s kubelet Created container www Normal Started 2m11s kubelet Started container www ```