---
title: Secure k3s with gVisor
url: https://devopstales.github.io/kubernetes/k3s-gvisor/
date: 2021-04-30
keywords: kubernetes deployment, kubernetes vs docker, Kubernetes Secuity, best Practices, k3s, gVisor
---


In this post I will show you how you can secure k3s with gVisor.
<!--more-->

{{< content "/filedir/k8s-sec.html" >}}

In  [previous pos](https://devopstales.github.io/kubernetes/k3s-etcd-kube-vip/) I showd you how to install a k3s Cluster. Now we modify the configuration of the containerd to use different low level container runtime.

### What is gvisor

gVisor is an application kernel, written in Go, that implements a substantial portion of the Linux system call interface. It provides an additional layer of isolation between running applications and the host operating system.

gVisor includes an Open Container Initiative (OCI) runtime called `runsc` that makes it easy to work with existing container tooling. The `runsc` runtime integrates with Docker, containerd and Kubernetes, making it simple to run sandboxed containers.

![gvisor](/img/include/gvisor2.webp)
![gvisor](/img/include/gvisor.webp)

### Bootstrap the k3s cluster

```bash
k3sup install \
  --ip=172.17.8.101 \
  --user=vagrant \
  --sudo \
  --cluster \
  --k3s-channel=stable \
  --k3s-extra-args "--no-deploy=traefik --no-deploy=servicelb --flannel-iface=enp0s8 --node-ip=172.17.8.101" \
  --merge \
  --local-path $HOME/.kube/config \
  --context=k3s-ha

k3sup join \
  --ip 172.17.8.102 \
  --user vagrant \
  --sudo \
  --k3s-channel stable \
  --server \
  --server-ip 172.17.8.101 \
  --server-user vagrant \
  --sudo \
  --k3s-extra-args "--no-deploy=traefik --no-deploy=servicelb --flannel-iface=enp0s8 --node-ip=172.17.8.102"
  
k3sup join \
  --ip 172.17.8.103 \
  --user vagrant \
  --sudo \
  --k3s-channel stable \
  --server \
  --server-ip 172.17.8.101 \
  --server-user vagrant \
  --sudo \
  --k3s-extra-args "--no-deploy=traefik --no-deploy=servicelb --flannel-iface=enp0s8 --node-ip=172.17.8.103"
```

### What is gVisor

```bash
tmux-cssh -u vagrant 172.17.8.101 172.17.8.102 172.17.8.103

sudo su -
yum install nano wget -y

nano gvisor.sh
#!/bash
(
  set -e
  URL=https://storage.googleapis.com/gvisor/releases/release/latest
  wget ${URL}/runsc ${URL}/runsc.sha512 \
    ${URL}/gvisor-containerd-shim ${URL}/gvisor-containerd-shim.sha512 \
    ${URL}/containerd-shim-runsc-v1 ${URL}/containerd-shim-runsc-v1.sha512
  sha512sum -c runsc.sha512 \
    -c gvisor-containerd-shim.sha512 \
    -c containerd-shim-runsc-v1.sha512
  rm -f *.sha512
  chmod a+rx runsc gvisor-containerd-shim containerd-shim-runsc-v1
  sudo mv runsc gvisor-containerd-shim containerd-shim-runsc-v1 /usr/local/bin
)

bash gvisor.sh

cp /var/lib/rancher/k3s/agent/etc/containerd/config.toml \
/var/lib/rancher/k3s/agent/etc/containerd/config.toml.back

cp /var/lib/rancher/k3s/agent/etc/containerd/config.toml.back \
/var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl

nano /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl
...
[plugins.cri.containerd]
  disable_snapshot_annotations = true
  snapshotter = "overlayfs"

disabled_plugins = ["restart"]

[plugins.linux]
  shim_debug = true

[plugins.cri.containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"

[plugins.cri.cni]
...

systemcl restart k3s

exit
```

```bash
cat<<EOF | kubectl apply -f -
apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
EOF
```

```bash
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: www-runc
spec:
  containers:
  - image: nginx:1.18
    name: www
    ports:
    - containerPort: 80
EOF

cat<<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: untrusted
  name: www-gvisor
spec:
  runtimeClassName: gvisor
  containers:
  - image: nginx:1.18
    name: www
    ports:
    - containerPort: 80
EOF
```

```bash
kubectl get po
NAME         READY   STATUS    RESTARTS   AGE
www-gvisor   1/1     Running   0          9s
www-runc     1/1     Running   0          1m
```