RKE2 The Secure Kubernetes Engine
In this post I will show you how you can install a secure Kubernetes Engine variant called RKE2 in a Air-Gap environment.
Parst of the K8S Security series
- Part1: Best Practices to keeping Kubernetes Clusters Secure
- Part2: RKE2 The Secure Kubernetes Engine
- Part3: RKE2 Pod Security Policy
- Part4: Kubernetes Network Policy
- Part5: Kubernetes Cluster Policy
- Part6: Using Admission Controllers
- Part7: Image security Admission Controller
What is RKE2
RKE2, also known as RKE Government, is Rancher’s next-generation Kubernetes distribution. It is a fully conformant Kubernetes distribution that focuses on security and compliance within the U.S. Federal Government sector.
Install RKE2 from rpms
Not like K3S RKE2 offers an rpm repository. Of course in an Air-Gap environment you need an internal repository to sync the packages.
cat << EOF > /etc/yum.repos.d/rancher-rke2-1-18-latest.repo
[rancher-rke2-common-latest]
name=Rancher RKE2 Common Latest
baseurl=https://rpm.rancher.io/rke2/latest/common/centos/7/noarch
enabled=1
gpgcheck=1
gpgkey=https://rpm.rancher.io/public.key
[rancher-rke2-1-18-latest]
name=Rancher RKE2 1.18 Latest
baseurl=https://rpm.rancher.io/rke2/latest/1.18/centos/7/x86_64
enabled=1
gpgcheck=1
gpgkey=https://rpm.rancher.io/public.key
EOF
yum -y install rke2-server nano
In an Air-Gap environment you cannot connect to the public internet so the containerd engine cannot connest to the registry. In this scenario yo have two options. Create an internal registry and upload all images or import images from tarball. In this demo I will use the second option.
mkdir -p /var/lib/rancher/rke2/agent/images/
scp rke2-images.linux-amd64.tar masern01:/var/lib/rancher/rke2/agent/images/
cd /var/lib/rancher/rke2/agent/images/
For RKE2 you didn’t nee docker engine. The rpms will install all the necessary binaris to run a container.
echo 'PATH=$PATH:/usr/local/bin' >> /etc/profile
echo 'PATH=$PATH:/var/lib/rancher/rke2/bin' >> /etc/profile
source /etc/profile
setenforce 1
getenforce
sed -i 's/=\(disabled\|permissive\)/=enforcing/g' /etc/sysconfig/selinux
systemctl start firewalld
systemctl enable firewalld
For the demo I will use firewalld
to block all outgoing request from the server. This is how I emulate the Air-Gap environment.
firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 0 -p tcp -m tcp --dport=443 -j DROP
firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 0 -p tcp -m tcp --dport=80 -j DROP
firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 1 -j ACCEPT
firewall-cmd --reload
Enable hardened mode.
mkdir -p /etc/rancher/rke2
cat << EOF > /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
profile: "cis-1.5"
EOF
sudo cp -f /usr/share/rke2/rke2-cis-sysctl.conf /etc/sysctl.d/60-rke2-cis.conf
sysctl -p /etc/sysctl.d/60-rke2-cis.conf
useradd -r -c "etcd user" -s /sbin/nologin -M etcd
On my VM there is multiple network interface So I will configure what to use the kubernetes engine.
mkdir -p /var/lib/rancher/rke2/server/manifests/
cat << EOF > /var/lib/rancher/rke2/server/manifests/rke2-canal-config.yml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: rke2-canal
namespace: kube-system
spec:
valuesContent: |-
flannel:
iface: "enp0s8"
EOF
systemctl enable rke2-server.service
systemctl start rke2-server.service
journalctl -u rke2-server -f
mkdir ~/.kube
ln -s /etc/rancher/rke2/rke2.yaml ~/.kube/config
ln -s /var/lib/rancher/rke2/agent/etc/crictl.yaml /etc/crictl.yaml
kubectl get node
crictl ps
crictl images
kubectl edit psp global-restricted-psp
# remove apparmor lines in annotation and save
### Autodeploy folder
/var/lib/rancher/rke2/server/manifests/