Automatically change registry in pod definition
In this post I will show you how you can automatically change the registry part in deployed pods in Kubernetes.
Parst of the K8S Security series
- Part1: Best Practices to keeping Kubernetes Clusters Secure
- Part2: Kubernetes Hardening Guide with CIS 1.6 Benchmark
- Part3: RKE2 The Secure Kubernetes Engine
- Part4: RKE2 Install With cilium
- Part5: Kubernetes Certificate Rotation
- Part6: Hardening Kubernetes with seccomp
- Part7a: RKE2 Pod Security Policy
- Part7b: Kubernetes Pod Security Admission
- Part7c: Pod Security Standards using Kyverno
- Part8: Kubernetes Network Policy
- Part9: Kubernetes Cluster Policy with Kyverno
- Part10: Using Admission Controllers
- Part11a: Image security Admission Controller
- Part11b: Image security Admission Controller V2
- Part11c: Image security Admission Controller V3
- Part12: Continuous Image security
- Part13: K8S Logging And Monitoring
- Part14: Kubernetes audit logs and Falco
- Part15a Image Signature Verification with Connaisseur
- Part15b Image Signature Verification with Connaisseur 2.0
- Part15c Image Signature Verification with Kyverno
- Part16a Backup your Kubernetes Cluster
- Part16b How to Backup Kubernetes to git?
- Part17a Kubernetes and Vault integration
- Part17b Kubernetes External Vault integration
- Part18a: ArgoCD and kubeseal to encript secrets
- Part18b: Flux2 and kubeseal to encrypt secrets
- Part18c: Flux2 and Mozilla SOPS to encrypt secrets
- Part19: ArgoCD auto image updater
- Part20: Secure k3s with gVisor
- Part21: How to use imagePullSecrets cluster-wide??
- Part22: Automatically change registry in pod definition
ImageSwap Mutating Admission Controller for Kubernetes
The ImageSwap webhook enables you to define one or more mappings to automatically swap image definitions within Kubernetes Pods with a different registry.
Install ImageSwap:
$ kubectl apply -f https://raw.githubusercontent.com/phenixblue/imageswap-webhook/v1.4.2/deploy/install.yaml
For swapping configuration ImageSwap use a configmap to define what image should change to what:
apiVersion: v1
data:
maps: |
default:registry.example.com
#gcr.io: # This is a comment
gitlab.com:registry.example.com/gitlab
noswap_wildcards:example.com
kind: ConfigMap
metadata:
creationTimestamp: null
name: imageswap-maps
namespace: imageswap-system
Example MAPS Configs:
Disable image swapping for all registries EXCEPT gcr.io
default:
gcr.io:harbor.internal.example.com
Enable image swapping for all registries except gcr.io
With this, all images will be swapped except those that already match the harbor.internal.example.com
pattern
default:harbor.internal.example.com
noswap_wildcards:harbor.internal.example.com
Test ImageSwap
$ kubectl create ns test1
$ kubectl label ns test1 k8s.twr.io/imageswap=enabled
Then deploy a pod:
kubectl create deployment unsigned-my \
--image=docker.io/devopstales/testimage:unsigned
ImageSwap can be disabled on a per workload level by adding the k8s.twr.io/imageswap
label with a value of disabled
to the pod template.
Kyverno
Here is an example of a Kyverno policy that validates that images are only pulled from an allowed list of image registries (based on wildcard patterns):
apiVersion : kyverno.io/v1alpha1
kind: Policy
metadata:
name: check-registries
spec:
rules:
- name: check-registries
resource:
kinds:
- Deployment
- StatefulSet
validate:
message: "Registry is not allowed"
pattern:
spec:
template:
spec:
containers:
- name: "*"
# Check allowed registries
image: "*/nirmata/* | https://private.registry.io/*"
Rather than blocking Pods which come from outside registries, it is also possible to mutate them so the pulls are directed to approved registries.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: replace-image-registry
annotations:
policies.kyverno.io/title: Replace Image Registry
policies.kyverno.io/category: Sample
policies.kyverno.io/severity: medium
policies.kyverno.io/subject: Pod
policies.kyverno.io/minversion: 1.3.6
policies.kyverno.io/description: >-
Rather than blocking Pods which come from outside registries,
it is also possible to mutate them so the pulls are directed to
approved registries. In some cases, those registries may function as
pull-through proxies and can fetch the image if not cached.
This policy policy mutates all images either
in the form 'image:tag' or 'registry.corp.com/image:tag' to be prefaced
with `myregistry.corp.com/`.
spec:
background: false
rules:
- name: replace-image-registry
match:
resources:
kinds:
- Pod
mutate:
patchStrategicMerge:
spec:
containers:
- (name): "*"
image: |-
{{ regex_replace_all('^[^/]+', '{{@}}', 'myregistry.corp.com') }}