Implementing Mutual TLS (mTLS) with Traefik Ingress Controller: Per-Ingress Configuration

Mutual TLS (mTLS) adds a critical security layer to your Kubernetes applications by requiring clients to authenticate with certificates—not just passwords or tokens. This guide covers per-ingress mTLS configuration with Traefik Ingress Controller, using Traefik’s native TLSOption CRD for fine-grained control over which services require client certificate authentication.

Understanding mTLS: TLS vs Mutual TLS

Standard TLS only authenticates the server to the client. mTLS extends this by requiring both parties to authenticate each other:

Comparison: TLS vs mTLS

How Traefik Handles mTLS

Traefik uses a CRD-based approach for mTLS configuration through the TLSOption resource. This provides several advantages:

• 📋 Declarative configuration – Define mTLS requirements as Kubernetes resources
• 🔗 Reusable policies – Apply the same TLSOption to multiple ingresses
• 🎯 Per-ingress control – Enable mTLS selectively per route
• 🔒 Native integration – No ConfigMap patches or deployment modifications required

Per-Ingress mTLS Configuration

This approach enables mTLS on specific ingress resources, giving you fine-grained control over which services require client certificate authentication.

Step 1: Deploy Traefik Ingress Controller

# Add the Traefik Helm repo
helm repo add traefik https://traefik.github.io/charts
helm repo update

# Install Traefik Ingress Controller
helm install traefik traefik/traefik \
  --namespace traefik \
  --create-namespace \
  --set ingressClass.enabled=true \
  --set ingressClass.isDefaultClass=true \
  --set providers.kubernetesCRD.enabled=true \
  --set providers.kubernetesIngress.enabled=true \
  --set ports.websecure.tls.enabled=true

# Verify pods are running
kubectl get pods -n traefik

Step 2: Deploy a Test Application

# Create deployment
kubectl create deployment demo-app --image=httpd --port=80

# Expose via ClusterIP service
kubectl expose deployment demo-app --name=demo-app --namespace=example

Step 3: Generate Certificate Authority (CA)

The CA signs client certificates and is used by Traefik to verify incoming client requests.

Create OpenSSL CA configuration (openssl-ca.cnf):

HOME            = .
RANDFILE        = $ENV::HOME/.rnd

[ ca ]
default_ca    = CA_default

[ CA_default ]
default_days     = 365
default_crl_days = 30
default_md       = sha256
preserve         = no
x509_extensions = ca_extensions
email_in_dn     = no
copy_extensions = copy
base_dir      = .
certificate   = $base_dir/cacert.pem
private_key   = $base_dir/cakey.pem
new_certs_dir = $base_dir
database      = $base_dir/index.txt
serial        = $base_dir/serial.txt
unique_subject = no

[ req ]
default_bits       = 4096
default_keyfile    = cakey.pem
distinguished_name = ca_distinguished_name
x509_extensions    = ca_extensions
string_mask        = utf8only

[ ca_distinguished_name ]
countryName         = Country Name (2 letter code)
countryName_default = US
stateOrProvinceName_default = California
localityName_default        = San Francisco
organizationName_default    = MyOrganization
organizationalUnitName_default = Security
commonName_default = My Certificate Authority
emailAddress_default = security@example.com

[ ca_extensions ]
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always, issuer
basicConstraints       = critical, CA:true
keyUsage               = keyCertSign, cRLSign

Generate CA certificate:

openssl req -new -x509 -config openssl-ca.cnf -days 365 \
  -newkey rsa:4096 -sha256 -noenc -out cacert.pem -outform PEM

Output files:

• cacert.pem – CA certificate (public)
• cakey.pem – CA private key (keep secure!)

Step 4: Generate Client Certificate

Generate client private key:

openssl genpkey -out client.key -algorithm RSA -pkeyopt rsa_keygen_bits:4096

Create CSR configuration (openssl-csr.cnf):

[ req ]
default_bits       = 4096
default_keyfile    = client.key
distinguished_name = client_distinguished_name
req_extensions     = client_req_extensions
string_mask        = utf8only

[ client_distinguished_name ]
countryName_default = US
stateOrProvinceName_default = California
localityName_default        = San Francisco
organizationName_default    = MyOrganization
organizationalUnitName_default = Engineering
commonName_default = authorized-client
emailAddress_default = engineering@example.com

[ client_req_extensions ]
subjectKeyIdentifier = hash
basicConstraints     = CA:FALSE
keyUsage             = digitalSignature, keyEncipherment
subjectAltName       = @alternate_names
nsComment            = "OpenSSL Generated Certificate"

[ alternate_names ]
DNS.1 = client.example.com

Create Certificate Signing Request (CSR):

openssl req -new -config openssl-csr.cnf -key client.key \
  -out client.csr -outform PEM

Initialize CA database and sign the client certificate:

touch index.txt
echo "01" > serial.txt

openssl ca -config openssl-ca.cnf \
  -policy signing_policy \
  -extensions signing_req \
  -out client.pem \
  -infiles client.csr

💡 Note: The signing_policy and signing_req sections should be added to your openssl-ca.cnf file (see full configuration in the CA generation step above).

Create PKCS#12 bundle for browser import:

PASSWORD=$(uuidgen) && echo "Export password: $PASSWORD"
openssl pkcs12 -export -inkey client.key -in client.pem \
  -name "Client Certificate" -out client.p12 -passout pass:"$PASSWORD"

Step 5: Store CA Certificate as Kubernetes Secret

kubectl create secret generic ca-mtls \
  --from-file=ca.crt=./cacert.pem \
  -n example

⚠️ Important: Traefik extracts the CA certificate from the ca.crt or tls.ca key of the secret. Use one of these exact key names.

Step 6: Create Traefik TLSOption CRD

The TLSOption resource defines the mTLS requirements for Traefik:

apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: mtlsoption
  namespace: example
spec:
  clientAuth:
    clientAuthType: RequireAndVerifyClientCert
    secretNames:
      - ca-mtls
  minVersion: VersionTLS12

Apply the configuration:

kubectl apply -f tlsoption.yaml

Key configuration options:

Step 7: Configure Ingress with TLSOption Annotation

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mtls-ingress
  namespace: example
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    # Link to TLSOption: ${NAMESPACE}-${TLSOPTION-NAME}@kubernetescrd
    traefik.ingress.kubernetes.io/router.tls.options: example-mtlsoption@kubernetescrd
spec:
  ingressClassName: traefik
  rules:
    - host: secure-app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: demo-app
                port:
                  number: 80
  tls:
    - hosts:
        - secure-app.example.com
      secretName: traefik-tls-secret

Apply the configuration:

kubectl apply -f ingress-mtls.yaml

💡 Annotation format: ${NAMESPACE}-${TLSOPTION-NAME}@kubernetescrd

Step 8: Create Server TLS Secret (Optional but Recommended)

# Generate server certificate
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout server.key -out server.crt \
  -subj "/CN=secure-app.example.com/O=MyOrganization"

# Create Kubernetes TLS secret
kubectl create secret tls traefik-tls-secret \
  --key server.key \
  --cert server.crt \
  -n example

Validation and Testing

Test Without Client Certificate (Should Fail)

curl -v https://secure-app.example.com

Expected response:

< HTTP/1.1 400 Bad Request
< Content-Type: text/html
< 
tls: failed to verify client certificate: x509: certificate signed by unknown authority

Test With Client Certificate (Should Succeed)

curl -v https://secure-app.example.com \
  --cacert cacert.pem \
  --cert client.pem \
  --key client.key

Expected response:

* SSL certificate verify ok.
* Using client certificate:
*   subject: CN=authorized-client, OU=Engineering, O=MyOrganization
> GET / HTTP/1.1
< HTTP/1.1 200 OK

Test Using OpenSSL

openssl s_client -connect secure-app.example.com:443 \
  -key client.key -cert client.pem -state

Expected output:

SSL handshake has read 2543 bytes and written 4321 bytes
Verification: OK

Browser Testing

1. Import CA certificate (cacert.pem) into your browser’s trusted root certificates
2. Import client certificate (client.p12) into your browser
3. Navigate to https://secure-app.example.com
4. Browser will prompt for certificate selection—choose your client certificate

Architecture Flow

Multiple TLSOptions for Different Security Levels

You can create multiple TLSOption resources for different security requirements:

# Strict mTLS for internal APIs
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: strict-mtls
  namespace: internal
spec:
  clientAuth:
    clientAuthType: RequireAndVerifyClientCert
    secretNames:
      - internal-ca-mtls
  minVersion: VersionTLS13
---
# Optional mTLS for partner APIs
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: optional-mtls
  namespace: partners
spec:
  clientAuth:
    clientAuthType: RequestClientCert
    secretNames:
      - partner-ca-mtls
  minVersion: VersionTLS12

Then reference them in your ingresses:

# Internal API - strict mTLS required
annotations:
  traefik.ingress.kubernetes.io/router.tls.options: internal-strict-mtls@kubernetescrd

# Partner API - client cert requested but optional
annotations:
  traefik.ingress.kubernetes.io/router.tls.options: partners-optional-mtls@kubernetescrd

Production Considerations

Certificate Management

• 🔁 Rotate certificates regularly – Implement automated rotation before expiration
• 📦 Use cert-manager – Integrate with cert-manager for automated certificate lifecycle
• 🔐 Secure CA private key – Store in HSM or secure vault, never in version control
• 📅 Monitor expiration – Set up alerts for certificates expiring within 30 days

Security Best Practices

• 🔒 Restrict namespace access – Limit RBAC permissions to the Traefik namespace
• 📋 Maintain certificate inventory – Track issued certificates and expiration dates
• 🔄 Implement revocation – Use CRL or OCSP for compromised certificates
• 🔍 Enable detailed logging – Log client certificate subjects for audit trails
• 🛡️ Use TLS 1.3 – Set minVersion: VersionTLS13 for maximum security

DNS and Networking

• 🌐 Configure DNS A records – Point your domain to the Traefik LoadBalancer IP
• 📊 Monitor failed authentications – Set up alerts for unusual 400 error rates
• 📈 Enable Traefik metrics – Monitor TLS handshake failures via Prometheus

Troubleshooting

Issue: 400 Bad Request – No Client Certificate

Symptoms:

curl -k https://secure-app.example.com
# Returns: 400 Bad Request

Solutions:

1. Verify client certificate is being sent: curl --cert client.pem --key client.key
2. Check CA secret exists in the correct namespace
3. Verify TLSOption annotation format: namespace-name@kubernetescrd
4. Check Traefik logs: kubectl logs -n traefik -l app.kubernetes.io/name=traefik

Issue: SSL Certificate Verify Failed

Symptoms:

curl: (60) SSL certificate problem: unable to get local issuer certificate

Solutions:

1. Import CA certificate: curl --cacert cacert.pem
2. Verify server certificate chain is complete
3. Check SAN matches your domain

Issue: TLSOption Not Applied

Symptoms: mTLS not enforced despite annotation

Solutions:

1. Verify annotation format: ${NAMESPACE}-${TLSOPTION-NAME}@kubernetescrd
2. Check TLSOption exists in the referenced namespace
3. Verify CA secret has ca.crt or tls.ca key
4. Ensure Traefik CRD provider is enabled

Issue: Client Certificate Not Passed to Backend

Symptoms: Backend doesn’t receive client certificate headers

Solutions:

1. Traefik passes client certificate in X-Forwarded-Tls-Client-Cert header
2. Verify backend reads the correct header
3. Check Traefik configuration for certificate forwarding

Conclusion

Implementing per-ingress mTLS with Traefik Ingress Controller provides flexible, fine-grained security for your Kubernetes applications. Using Traefik’s native TLSOption CRD offers several advantages:

• ✅ Declarative configuration – Define mTLS as Kubernetes resources
• ✅ Reusable policies – Apply TLSOptions across multiple ingresses
• ✅ Selective mTLS enforcement – Only specific services require client certificates
• ✅ Mixed public/internal services – Some endpoints public, others protected
• ✅ Native integration – No deployment patches or ConfigMap modifications

By requiring client certificates at the ingress boundary for selected services, you protect sensitive applications while maintaining accessibility for public-facing endpoints.

Found this guide helpful? Share your mTLS implementation experiences in the comments below.