Kubernetes Single Sign-on with Pinniped OpenID Connect

In this tutorial I will setup Pinniped, a Single Sign-on solution from the VMware Tanzu project.

Wtah is Pinniped

Pinniped gives you a unified login experience across all your clusters, including on-premises and managed cloud environments.

Pinniped consists of two components, Supervisor and Concierge:

  • The Pinniped Supervisor is an OIDC server which allows users to authenticate with an external identity provider (IDP), and then issues its own federation ID tokens to be passed on to clusters based on the user information from the IDP.

  • The Pinniped Concierge is a credential exchange API which takes as input a credential from an identity source (e.g., Pinniped Supervisor, proprietary IDP), authenticates the user via that credential, and returns another credential which is understood by the host Kubernetes cluster or by an impersonation proxy which acts on behalf of the user.

Pinniped Architecture

Install the Pinniped Supervisor

Run below command to install Supervisor, this will install everything to the pinniped-supervisor namespace.

kubectl apply -f https://get.pinniped.dev/latest/install-pinniped-supervisor.yaml

kubens pinniped-supervisor
kubectl get po

After installing Pinniped, create a secret with the client ID and secret from your identity provider. In below command I have named the secret oidc-client:

kubectl create secret generic oidc-client \
  --namespace pinniped-supervisor \
  --type secrets.pinniped.dev/oidc-client \
  --from-literal=clientID="<client-id-goes-here>" \

The supervisor needs to be reachable from other clusters, we need to create a service and an ingress resource.

nano pinniped-supervisor-ingress.yaml
apiVersion: v1
kind: Service
  namespace: pinniped-supervisor
  name: pinniped-supervisor
    app: pinniped-supervisor
    - port: 80
      targetPort: 8080
      protocol: TCP
apiVersion: networking.k8s.io/v1
kind: Ingress
  name: pinniped-supervisor
  namespace: pinniped-supervisor
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: ca-issuer
  - hosts:
    - supervisor.k8s.intra
    secretName: supervisor-cert
  - host: supervisor.k8s.intra
      - path: /
        pathType: Prefix
            name: pinniped-supervisor
              number: 80

Create and apply a FederationDomain resource. Give it a name and set the issuer to the URL you will be using for Supervisor.

nano FederationDomain.yaml
apiVersion: config.supervisor.pinniped.dev/v1alpha1
kind: FederationDomain
  name: keycloak
  namespace: pinniped-supervisor
  issuer: https://supervisor.k8s.intra

Create and apply a OIDCIdentityProvider resource.

nano OIDCIdentityProvider.yaml
apiVersion: idp.supervisor.pinniped.dev/v1alpha1
kind: OIDCIdentityProvider
  name: keycloak-idp
  namespace: pinniped-supervisor
  issuer: https://sso.k8s.intra/auth/realms/k8s
    username: email
    groups: groups
    additionalScopes: ['email', 'profile']
    secretName: oidc-client

Install the Pinniped Concierge

Install Concierge on any managed or unmanaged cluster you would like to use OIDC login on. This can include the cluster where Supervisor is running.

kubectl apply -f https://get.pinniped.dev/latest/install-pinniped-concierge-crds.yaml
kubectl apply -f https://get.pinniped.dev/latest/install-pinniped-concierge.yaml

kubens pinniped-concierge
kubectl get po

Create and apply a JWTAuthenticator resource, it is cluster scoped so no need for namespaces. Create a random string for the audience property using openssl rand -base64 24 command.

openssl rand -base64 24

If you ise a self signed certificate for the ingress you need o add the base64 encoded CA pem to the JWTAuthenticator.

cat k8s.pem | base64
nano JWTAuthenticator.yaml
apiVersion: authentication.concierge.pinniped.dev/v1alpha1
kind: JWTAuthenticator
   name: supervisor-jwt-authenticator
   namespace: pinniped-concierge
   issuer: https://supervisor.k8s.intra
   audience: jCIGaxMT5Yw9NvafTTVoXxGLkviPPyg6
     username: username
     groups: groups
  # tls:
  #   certificateAuthorityData: LS0tLS1CR... # optional base64 CA data if using a self-signed certificate

The supervisor.k8s.intra domain must be resolwable from the pinniped-concierge. You can add supervisor.k8s.intra as like hists file to the coredns like this.

Below is an example of a ClusterRoleBinding that binds the role cluster-admin to the Keycloak group devops-team. (In my case it came from ldap) Create your own role bindings to fit your needs and apply them to the cluster.

nano devops-team_ClusterRoleBinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
  name: cluster-admin-it-afdeling
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: devops-team

Install the Pinniped command-line tool

For osx you can use brew to install the command-line tool:

brew install vmware-tanzu/pinniped/pinniped-cli

Or find the appropriate binary for your platform from the latest release the put the command-line tool somewhere on your $PATH.

Generate the kubeconfig file wigh the default admin kubeconfig on the cluster’s master node:

pinniped get kubeconfig \
--oidc-ca-bundle /opt/k8s_sec_lab/cert/k8s.pem \
--output pinniped-kubeconfig

kubectl --kubeconfig pinniped-kubeconfig get pods -n pinniped-concierge
NAME                                                  READY   STATUS    RESTARTS   AGE
pinniped-concierge-9bc8bbdc5-qg75p                    1/1     Running   0          16m
pinniped-concierge-9bc8bbdc5-zxsjj                    1/1     Running   0          16m
pinniped-concierge-kube-cert-agent-5dcccbbb4b-56jbj   1/1     Running   0          79m

The k8s.pem file is containes the CA certificate from ca-issuer.