---
title: Kubernetes Pod Security Admission
url: https://devopstales.github.io/kubernetes/k8s-ps/
date: 2022-08-23
keywords: Kubernetes Secuity, best Practices, Pod Security Policy, Pod Security, Pod Security Admission
---


With the release of Kubernetes v1.25, Pod Security Admission has now entered to stable and PodSecurityPolicy is removed. In this article, we cover the key concepts of Pod Security Admission along with how to use it.
<!--more-->

{{< content "/filedir/k8s-sec.html" >}}

### What is a Pod Security Policy?
A Pod Security Policy is a cluster-level resource that controls security sensitive aspects of the pod specification. RBAC Controlls the usable Kubernetes objects for a user but nt the conditions of a specific ofject like allow run as root or not in a container.  PSP objects define a set of conditions that a pod must run with in order to be accepted into the system, as well as defaults for their related fields. PodSecurityPolicy is an optional admission controller that is enabled by default through the API, thus policies can be deployed without the PSP admission plugin enabled.

### What is a Pod Security Admission
Pod Security Admissionis the successor to `PodSecurityPolicy` which was deprecated in the v1.21 release, and will be removed in Kubernetes v1.25. Pod Security Admission overcomes key shortcomings of Kubernetes' existing, PodSecurityPolicy (PSP) mechanism like: challenging to deploy with controllers and teh lack of dry-run/audit capabilities made it hard to enable PodSecurityPolicy.

### Configuring Pod Security Admission
Pod Security Admission use profiles based on [Pod Security Standards](https://kubernetes.io/docs/concepts/security/pod-security-standards/). These standards define three different policy levels:

* privileged - Unrestricted policy that allows anything, including known privilege escalations. Apply this policy with caution.
* baseline - Minimally restrictive policy that prevents known privilege escalations. Allows all default values for fields in Pod specifications.
  * Disables `HostProcess` for windows
  * Disables `Host Namespaces` on linux like: `hostNetwork`, `hostPID` and `hostIPC`
  * Disables `Privileged Containers`
  * Disallow adding of `Capabilities`
  * Disallow mounting of `HostPath Volumes`
  * Disallow usage of `Host Ports`
  * On supported hosts, the `runtime/default` AppArmor profile is applied by default.
  * Setting the SELinux type is restricted, and setting a custom SELinux user or role option is forbidden.
  * Seccomp profile must not be explicitly set to `Unconfined`.
  * Disallow the configuration of `Sysctls`
* restricted - Most restrictive policy. Complies with Pod hardening best practices.
  * Disallow `Privilege Escalation`
  * Disallow running cotainer as root user, group and uid or guid as 0
  * Seccomp profile must be explicitly set to one of the allowed values. 
  * Containers must drop `ALL` capabilities, and are only permitted to add back the `NET_BIND_SERVICE` capability.

Policies are applied in a specific mode. The modes are:

* `enforce` — Any Pods that violate the policy will be rejected
* `audit` — Violations will be recorded as an annotation in the audit logs, but don't affect whether the pod is allowed.
* `warn` — Violations will send a warning message back to the user, but don't affect whether the pod is allowed.

### Demo:

Start a cluster with kind for the demo:

```bash
kind create cluster --image kindest/node:v1.23.0
kubectl cluster-info --context kind-kind
kubectx kind-kind
```

Find the enabled admission plugins:

```bash
$ kubectl -n kube-system exec kube-apiserver-kind-control-plane -it -- kube-apiserver -h | grep "default enabled ones"
...
      --enable-admission-plugins strings
admission plugins that should be enabled in addition
to default enabled ones (NamespaceLifecycle, LimitRanger,
ServiceAccount, TaintNodesByCondition, PodSecurity, Priority,
DefaultTolerationSeconds, DefaultStorageClass,
StorageObjectInUseProtection, PersistentVolumeClaimResize,
RuntimeClass, CertificateApproval, CertificateSigning,
CertificateSubjectRestriction, DefaultIngressClass,
MutatingAdmissionWebhook, ValidatingAdmissionWebhook,
ResourceQuota).
...
```

Policies are applied to a namespace via labels. These labels are as follows:

* pod-security.kubernetes.io/<MODE>: <LEVEL> (required to enable pod security Admission)
* pod-security.kubernetes.io/<MODE>-version: <VERSION> (optional, defaults to latest)

Deploy demo workload:

```bash
kubectl create ns verify-pod-security
kubens verify-pod-security

# enforces a "restricted" security policy and audits on restricted
kubectl label --overwrite ns verify-pod-security \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/audit=restricted
```

Next, try to deploy a privileged pod:

```yaml
cat <<EOF | kubectl -n verify-pod-security apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: busybox-privileged
spec:
  containers:
  - name: busybox
    image: busybox
    args:
    - sleep
    - "1000000"
    securityContext:
      allowPrivilegeEscalation: true
EOF
```

The output is similar to this:

```bash
Error from server (Forbidden): error when creating "STDIN": pods "busybox-privileged" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "busybox" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "busybox" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "busybox" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "busybox" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
```

Now let's apply the privileged Pod Security Admission level and try tp deploy again.

```bash
# enforces a "privileged" security policy and warns / audits on baseline
kubectl label --overwrite ns verify-pod-security \
  pod-security.kubernetes.io/enforce=privileged \
  pod-security.kubernetes.io/warn=baseline \
  pod-security.kubernetes.io/audit=baseline
```

Now the pod is created:

```bash
pod/busybox-privileged created
```

Let's apply the baseline Pod Security Admission level and try again.

```bash
# enforces a "baseline" security policy and warns / audits on restricted
kubectl label --overwrite ns verify-pod-security \
  pod-security.kubernetes.io/enforce=baseline \
  pod-security.kubernetes.io/warn=restricted \
  pod-security.kubernetes.io/audit=restricted
```

```yaml
cat <<EOF | kubectl -n verify-pod-security apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: busybox-baseline
spec:
  containers:
  - name: busybox
    image: busybox
    args:
    - sleep
    - "1000000"
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        add:
          - NET_BIND_SERVICE
          - CHOWN
EOF
```

The output is similar to the following. Note that the warnings match the error message from the restricted policy, but the pod is still successfully created.

```bash
Warning: would violate PodSecurity "restricted:latest": unrestricted capabilities (container "busybox" must set securityContext.capabilities.drop=["ALL"]; container "busybox" must not include "CHOWN" in securityContext.capabilities.add), runAsNonRoot != true (pod or container "busybox" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "busybox" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
pod/busybox-baseline created
```

You ken use Kyverno to autamate the creation of the labels at namespace. The usage Kyverno is out of the scope of this post but if you want to know more about about this topic check [my other post](https://devopstales.github.io/kubernetes/k8s-kyverno-cosign/).

### Applying a cluster-wide policy
In addition to applying labels to namespaces to configure policy you can also configure cluster-wide policies and exemptions using the AdmissionConfiguration resource.

First create a new kind cluster with :

```bash
kind delete cluster
```

Create a Pod Security Admission configuration that enforce and audit baseline policies while using a restricted profile to warn the end user.

```yaml
cat <<EOF > pod-security.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
  configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1beta1
    kind: PodSecurityConfiguration
    defaults:
      enforce: "baseline"
      enforce-version: "latest"
      audit: "baseline"
      audit-version: "latest"
      warn: "restricted"
      warn-version: "latest"
      audit: "restricted"
      audit-version: "latest"
    exemptions:
      # Array of authenticated usernames to exempt.
      usernames: []
      # Array of runtime class names to exempt.
      runtimeClasses: []
      # Array of namespaces to exempt.
      namespaces: [kube-system]
EOF
```

```yaml
cat <<EOF > kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    apiServer:
        # enable admission-control-config flag on the API server
        extraArgs:
          admission-control-config-file: /etc/kubernetes/policies/pod-security.yaml
        # mount new file / directories on the control plane
        extraVolumes:
          - name: policies
            hostPath: /etc/kubernetes/policies
            mountPath: /etc/kubernetes/policies
            readOnly: true
            pathType: "DirectoryOrCreate"
  # mount the local file on the control plane
  extraMounts:
  - hostPath: ./pod-security.yaml
    containerPath: /etc/kubernetes/policies/pod-security.yaml
    readOnly: true
EOF
```

```bash
kind create cluster --image kindest/node:v1.23.0 --config kind-config.yaml
kubectl cluster-info --context kind-kind
kubectx kind-kind
```

Let's create a new namespace and see if the labels apply there.

```bash
$ kubectl create namespace test-defaults
namespace/test-defaults created

$ kubectl describe namespace test-defaults
Name:         test-defaults
Labels:       kubernetes.io/metadata.name=test-defaults
Annotations:  <none>
Status:       Active

No resource quota.

No LimitRange resource.
```

Can a privileged workload be deployed?

```yaml
cat <<EOF | kubectl -n test-defaults apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: busybox-privileged
spec:
  containers:
  - name: busybox
    image: busybox
    args:
    - sleep
    - "1000000"
    securityContext:
      allowPrivilegeEscalation: true
EOF
```

The default warn level is working.

```bash
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "busybox" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "busybox" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "busybox" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "busybox" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
pod/busybox-privileged created
```

Check the API server metrics endpoint:

```bash
kubectl get --raw /metrics | grep pod_security_evaluations_total

# HELP pod_security_evaluations_total [ALPHA] Number of policy evaluations that occurred, not counting ignored or exempt requests.
# TYPE pod_security_evaluations_total counter
pod_security_evaluations_total{decision="allow",mode="enforce",policy_level="baseline",policy_version="latest",request_operation="create",resource="pod",subresource=""} 2
pod_security_evaluations_total{decision="allow",mode="enforce",policy_level="privileged",policy_version="latest",request_operation="create",resource="pod",subresource=""} 0
pod_security_evaluations_total{decision="allow",mode="enforce",policy_level="privileged",policy_version="latest",request_operation="update",resource="pod",subresource=""} 0
pod_security_evaluations_total{decision="deny",mode="audit",policy_level="baseline",policy_version="latest",request_operation="create",resource="pod",subresource=""} 1
pod_security_evaluations_total{decision="deny",mode="enforce",policy_level="baseline",policy_version="latest",request_operation="create",resource="pod",subresource=""} 1
pod_security_evaluations_total{decision="deny",mode="warn",policy_level="restricted",policy_version="latest",request_operation="create",resource="controller",subresource=""} 2
pod_security_evaluations_total{decision="deny",mode="warn",policy_level="restricted",policy_version="latest",request_operation="create",resource="pod",subresource=""} 2
```

### Auditing
Example audit-policy.yaml configuration tuned for Pod Security Admission events:

```bash
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: RequestResponse
  resources:
    - group: "" # core API group
      resources: ["pods", "pods/ephemeralcontainers", "podtemplates", "replicationcontrollers"]
    - group: "apps"
      resources: ["daemonsets", "deployments", "replicasets", "statefulsets"]
    - group: "batch"
      resources: ["cronjobs", "jobs"]
  verbs: ["create", "update"]
  omitStages:
    - "RequestReceived"
    - "ResponseStarted"
    - "Panic"
```

The enableing of the audit-policy function is out of the scope of this post but if you want to know more about about this topic check [my other post](https://devopstales.github.io/kubernetes/k8s-falco/).

### PSP migrations
If you're already using PSP, I created a guide and [published the steps to migrate off of PSP](/kubernetes/k8s-migrate-from-psp/).