Image Signature Verification Admission Controller V2
Page content
In this post I will show you how you can deploy Connaisseur 2.0 to Image Signature Verification into a Kubernetes cluster.
Parts of the K8S Security Lab series
Container Runetime Security
- Part1: How to deploy CRI-O with Firecracker?
- Part2: How to deploy CRI-O with gVisor?
- Part3: How to deploy containerd with Firecracker?
- Part4: How to deploy containerd with gVisor?
- Part5: How to deploy containerd with kata containers?
Advanced Kernel Security
- Part6: Hardening Kubernetes with seccomp
- Part6: Linux user namespace management wit CRI-O in Kubernetes
Container Network Security
- Part4: RKE2 Install With Calico
- Part4: RKE2 Install With Cilium
Secure Kubernetes Install
- Part1: Best Practices to keeping Kubernetes Clusters Secure
- Part2: Kubernetes Hardening Guide with CIS 1.6 Benchmark
- Part5: Kubernetes Certificate Rotation
User Security
- Part5: How to create kubeconfig?
- Part5: How to create Users in Kubernetes the right way?
- Part5: Kubernetes Single Sign-on with Pinniped OpenID Connect
- Part5: Kubectl authentication with Kuberos Depricated !!
- Part5: Kubernetes authentication with Keycloak and gangway Depricated !!
- Part5: kube-openid-connect 1.0 Depricated !!
- Part10: Using Admission Controllers
- Part8: Kubernetes Network Policy
- Part8: RKE2 Pod Security Policy
- Part7b: Kubernetes Pod Security Admission
- Part7b: Kubernetes: How to migrate Pod Security Policy to Pod Security Admission?
- Part7c: Pod Security Standards using Kyverno
- Part9: Kubernetes Cluster Policy with Kyverno
- Part11a: Image security Admission Controller
- Part11b: Image security Admission Controller V2
- Part11c: Image security Admission Controller V3
- Part12: Continuous Image security
- Part12: trivy-operator 1.0
- Part12: trivy-operator 2.1: Trivy-operator is now an Admisssion controller too!!!
- Part12: trivy-operator 2.2: Patch release for Admisssion controller
- Part12: trivy-operator 2.3: Patch release for Admisssion controller
- Part15a Image Signature Verification with Connaisseur
- Part15b Image Signature Verification with Connaisseur 2.0
- Part15c Image Signature Verification with Kyverno
- Part21: How to use imagePullSecrets cluster-wide??
- Part22: Automatically change registry in pod definition
- Part15c Central authentication with oauth2-proxy
- Part15c Secure your applications with Pomerium Ingress Controller
- Part15c CrowdSec Intrusion Detection System (IDS) for Kubernetes
- Part14: Kubernetes audit logs and Falco
- Part13: K8S Logging And Monitoring
- Part13: Install Grafana Loki with Helm3
- Part16a Backup your Kubernetes Cluster
- Part16b How to Backup Kubernetes to git?
- Part17a Kubernetes and Vault integration
- Part17b Kubernetes External Vault integration
- Part18a: ArgoCD and kubeseal to encript secrets
- Part18b: Flux2 and kubeseal to encrypt secrets
- Part18c: Flux2 and Mozilla SOPS to encrypt secrets
- Part19: ArgoCD auto image updater
What is Connaisseur?
Connaisseur is an admission controller for Kubernetes that integrates Image Signature Verification into a cluster, as a means to ensure that only valid images are being deployed.
Notary
Notary is an open source signing solution for containers based on The Update Framework Notary uses TUFs’ roles and key hierarchy for signing of the images. There are five keys to sign the metadata files which lists all filenames in the collection, their sizes and respective hashes.
apt install notary
docker pull alpine
docker tag alpine:latest devopstales/testimage:unsigned
docker push devopstales/testimage:unsigned
notary -s https://notary.docker.io -d ~/.docker/trust init -p docker.io/devopstales/testimage
Root key found, using: 31579f2a034add499da6e799bc9260d08a15ab1804298218f05f78d97a669f77
Enter passphrase for root key with ID 31579f2:
Enter passphrase for new targets key with ID 42e49c6:
Repeat passphrase for new targets key with ID 42e49c6:
Enter passphrase for new snapshot key with ID 399243c:
Repeat passphrase for new snapshot key with ID 399243c:
Enter username: devopstales
Enter password:
Auto-publishing changes to docker.io/devopstales/testimage
Enter username: devopstales
Enter password:
Successfully published changes for repository docker.io/devopstales/testimage
export DOCKER_CONTENT_TRUST=1
export DOCKER_CONTENT_TRUST_SERVER=https://notary.docker.io
docker tag alpine:latest devopstales/testimage:signed
docker push devopstales/testimage:signed
$ find ~/.docker/trust/ | head
/home/devopstales/.docker/trust/
/home/devopstales/.docker/trust/private
/home/devopstales/.docker/trust/private/1f4a9a0922605b3bc19c97e180d962d530721288f4fd0845ad0aa37ba4a6f95d.key
/home/devopstales/.docker/trust/private/fe30e72f5976b2ae7d0d365f28dacfae9c71f11ad854065603ccc806900e84fa.key
/home/devopstales/.docker/trust/private/3da0d27e2d3b964d238d1d184c7578b5f2737b918ec5b8265474e22b07b2ea22.key
/home/devopstales/.docker/trust/private/root-priv.key
/home/devopstales/.docker/trust/private/root-pub.pem
/home/devopstales/.docker/trust/tuf
/home/devopstales/.docker/trust/tuf/docker.io
/home/devopstales/.docker/trust/tuf/docker.io/devopstales
notary -s https://notary.docker.io -d ~/.docker/trust list docker.io/devopstales/testimage
NAME DIGEST SIZE (BYTES) ROLE
---- ------ ------------ ----
signed 4661fb57f7890b9145907a1fe2555091d333ff3d28db86c3bb906f6a2be93c87 528 targets/devopstales
Install Connaisseur
# The installer use yq so we need to install it
wget https://github.com/mikefarah/yq/releases/download/v4.2.0/yq_linux_amd64 -O /usr/bin/yq &&\
chmod +x /usr/bin/yq
# generate the public root cert
cd ~/.docker/trust/private
sed '/^role:\sroot$/d' $(grep -iRl "role: root" .) > root-priv.key
openssl ec -in root-priv.key -pubout -out root-pub.pem
git clone https://github.com/sse-secure-systems/connaisseur.git
cd connaisseur
nano helm/values.yaml
...
validators:
...
# static validator that allows each image
- name: allow
type: static
approve: true
# pre-configured nv1 validator for public notary from Docker Hub
- name: dockerhub_basics
type: notaryv1
host: notary.docker.io
trust_roots:
# public key for official docker images (https://hub.docker.com/search?q=&type=image&image_filter=official)
# !if not needed feel free to remove the key!
- name: docker_official
key: |
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEOXYta5TgdCwXTCnLU09W5T4M4r9f
QQrqJuADP6U7g5r9ICgPSmZuRHP/1AYUfOQW3baveKsT969EfELKj1lfCA==
-----END PUBLIC KEY-----
# public key securesystemsengineering repo including Connaisseur images
# !this key is critical for Connaisseur!
- name: securesystemsengineering_official
key: |
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEsx28WV7BsQfnHF1kZmpdCTTLJaWe
d0CA+JOi8H4REuBaWSZ5zPDe468WuOJ6f71E7WFg3CVEVYHuoZt2UYbN/Q==
-----END PUBLIC KEY-----
# public key securesystemsengineering repo including devopstales images
- name: devopstales_official
key: |
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE9m6WfwViwT8lYjLF6jAs1bvd1hPp
cRUmONP49JszW1X/6Q22DygylIJGyC8IXeb3zBWVMoYDxauiqrFomHUOEA==
-----END PUBLIC KEY-----
policy:
- pattern: "*:*"
- pattern: "docker.io/library/*:*"
validator: dockerhub_basics
with:
trust_root: docker_official
- pattern: "k8s.gcr.io/*:*"
validator: allow
- pattern: "docker.io/securesystemsengineering/*:*"
validator: dockerhub_basics
with:
trust_root: securesystemsengineering_official
- pattern: "docker.io/devopstales/*:*"
validator: dockerhub_basics
with:
trust_root: devopstales_official
- the
defaultvalidator is used if no validator is specified in image policy - type: supported validators (e.g. “cosign” or “notaryv1”) notaryv2 is not yet supported
- host: url of the notary server
- key: the public part of the root key, for verifying notary’s signatures
Then deploy the helm chart. This can take a few minutes.
helm install connaisseur helm --atomic --create-namespace --namespace connaisseur
kubectl get all -n connaisseur
NAME READY STATUS RESTARTS AGE
pod/connaisseur-deployment-565d45bb74-ktbmb 1/1 Running 0 71s
pod/connaisseur-deployment-565d45bb74-pfghx 1/1 Running 0 71s
pod/connaisseur-deployment-565d45bb74-rcj44 1/1 Running 0 71s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/connaisseur-svc ClusterIP 10.43.196.6 <none> 443/TCP 71s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/connaisseur-deployment 3/3 3 3 71s
NAME DESIRED CURRENT READY AGE
replicaset.apps/connaisseur-deployment-565d45bb74 3 3 3 71s
Test the Image Signature Verification
kubens default
kubectl run unsigned --image=docker.io/devopstales/testimage:unsigned
Error from server: admission webhook "connaisseur-svc.connaisseur.svc" denied the request: Unable to find signed digest for image docker.io/devopstales/testimage:unsigned.
kubectl run signed --image=docker.io/devopstales/testimage:signed
pod/signed created
kubectl get po
Final words
Connaisseur is a grate tool and with the 2.0 it solved all of the 1.0’s shortcomings:
- There is no option to whitelist images in a specific namespace.
- Connaisseur supports only one Notary server
- Connaisseur supports only one public key
