Automatically change registry in pod definition
In this post I will show you how you can automatically change the registry part in deployed pods in Kubernetes.
Parts of the K8S Security Lab series
Container Runetime Security
- Part1: How to deploy CRI-O with Firecracker?
- Part2: How to deploy CRI-O with gVisor?
- Part3: How to deploy containerd with Firecracker?
- Part4: How to deploy containerd with gVisor?
- Part5: How to deploy containerd with kata containers?
Advanced Kernel Security
- Part6: Hardening Kubernetes with seccomp
- Part6: Linux user namespace management wit CRI-O in Kubernetes
Container Network Security
- Part4: RKE2 Install With Calico
- Part4: RKE2 Install With Cilium
Secure Kubernetes Install
- Part1: Best Practices to keeping Kubernetes Clusters Secure
- Part2: Kubernetes Hardening Guide with CIS 1.6 Benchmark
- Part5: Kubernetes Certificate Rotation
User Security
- Part5: How to create kubeconfig?
- Part5: How to create Users in Kubernetes the right way?
- Part5: Kubernetes Single Sign-on with Pinniped OpenID Connect
- Part5: Kubectl authentication with Kuberos Depricated !!
- Part5: Kubernetes authentication with Keycloak and gangway Depricated !!
- Part5: kube-openid-connect 1.0 Depricated !!
- Part10: Using Admission Controllers
- Part8: Kubernetes Network Policy
- Part8: RKE2 Pod Security Policy
- Part7b: Kubernetes Pod Security Admission
- Part7b: Kubernetes: How to migrate Pod Security Policy to Pod Security Admission?
- Part7c: Pod Security Standards using Kyverno
- Part9: Kubernetes Cluster Policy with Kyverno
- Part11a: Image security Admission Controller
- Part11b: Image security Admission Controller V2
- Part11c: Image security Admission Controller V3
- Part12: Continuous Image security
- Part12: trivy-operator 1.0
- Part12: trivy-operator 2.1: Trivy-operator is now an Admisssion controller too!!!
- Part12: trivy-operator 2.2: Patch release for Admisssion controller
- Part12: trivy-operator 2.3: Patch release for Admisssion controller
- Part15a Image Signature Verification with Connaisseur
- Part15b Image Signature Verification with Connaisseur 2.0
- Part15c Image Signature Verification with Kyverno
- Part21: How to use imagePullSecrets cluster-wide??
- Part22: Automatically change registry in pod definition
- Part15c Central authentication with oauth2-proxy
- Part15c Secure your applications with Pomerium Ingress Controller
- Part15c CrowdSec Intrusion Detection System (IDS) for Kubernetes
- Part14: Kubernetes audit logs and Falco
- Part13: K8S Logging And Monitoring
- Part13: Install Grafana Loki with Helm3
- Part16a Backup your Kubernetes Cluster
- Part16b How to Backup Kubernetes to git?
- Part17a Kubernetes and Vault integration
- Part17b Kubernetes External Vault integration
- Part18a: ArgoCD and kubeseal to encript secrets
- Part18b: Flux2 and kubeseal to encrypt secrets
- Part18c: Flux2 and Mozilla SOPS to encrypt secrets
- Part19: ArgoCD auto image updater
ImageSwap Mutating Admission Controller for Kubernetes
The ImageSwap webhook enables you to define one or more mappings to automatically swap image definitions within Kubernetes Pods with a different registry.
Install ImageSwap:
$ kubectl apply -f https://raw.githubusercontent.com/phenixblue/imageswap-webhook/v1.4.2/deploy/install.yaml
For swapping configuration ImageSwap use a configmap to define what image should change to what:
apiVersion: v1
data:
maps: |
default:registry.example.com
#gcr.io: # This is a comment
gitlab.com:registry.example.com/gitlab
noswap_wildcards:example.com
kind: ConfigMap
metadata:
creationTimestamp: null
name: imageswap-maps
namespace: imageswap-system
Example MAPS Configs:
Disable image swapping for all registries EXCEPT gcr.io
default:
gcr.io:harbor.internal.example.com
Enable image swapping for all registries except gcr.io
With this, all images will be swapped except those that already match the harbor.internal.example.com
pattern
default:harbor.internal.example.com
noswap_wildcards:harbor.internal.example.com
Test ImageSwap
$ kubectl create ns test1
$ kubectl label ns test1 k8s.twr.io/imageswap=enabled
Then deploy a pod:
kubectl create deployment unsigned-my \
--image=docker.io/devopstales/testimage:unsigned
ImageSwap can be disabled on a per workload level by adding the k8s.twr.io/imageswap
label with a value of disabled
to the pod template.
Kyverno
Here is an example of a Kyverno policy that validates that images are only pulled from an allowed list of image registries (based on wildcard patterns):
apiVersion : kyverno.io/v1alpha1
kind: Policy
metadata:
name: check-registries
spec:
rules:
- name: check-registries
resource:
kinds:
- Deployment
- StatefulSet
validate:
message: "Registry is not allowed"
pattern:
spec:
template:
spec:
containers:
- name: "*"
# Check allowed registries
image: "*/nirmata/* | https://private.registry.io/*"
Rather than blocking Pods which come from outside registries, it is also possible to mutate them so the pulls are directed to approved registries.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: replace-image-registry
annotations:
policies.kyverno.io/title: Replace Image Registry
policies.kyverno.io/category: Sample
policies.kyverno.io/severity: medium
policies.kyverno.io/subject: Pod
policies.kyverno.io/minversion: 1.3.6
policies.kyverno.io/description: >-
Rather than blocking Pods which come from outside registries,
it is also possible to mutate them so the pulls are directed to
approved registries. In some cases, those registries may function as
pull-through proxies and can fetch the image if not cached.
This policy policy mutates all images either
in the form 'image:tag' or 'registry.corp.com/image:tag' to be prefaced
with `myregistry.corp.com/`.
spec:
background: false
rules:
- name: replace-image-registry
match:
resources:
kinds:
- Pod
mutate:
patchStrategicMerge:
spec:
containers:
- (name): "*"
image: |-
{{ regex_replace_all('^[^/]+', '{{@}}', 'myregistry.corp.com') }}