Kubernetes nginx ingress with helm

In this post I will show you how can you use install IngressControllert on Kubernetes with helm.

Parts of the Kubernetes series

• Part1a: Install K8S with ansible
• Part1b: Install K8S with kubeadm
• Part1c: Install K8S with kubeadm and containerd
• Part1d: Install K8S with kubeadm and allow swap
• Part1e: Install K8S with kubeadm in HA mode
• Part2: Intall metal-lb with K8S
• Part2: Intall metal-lb with BGP
• Part3: Install Nginx ingress to K8S
• Part4: Install cert-manager to K8S
• Part5a: Use local persisten volume with K8S
• Part5b: Use ceph persisten volume with K8S
• Part5c: Use ceph CSI persisten volume with K8S
• Part5d: Kubernetes CephFS volume with CSI driver
• Part5e: Use Project Longhorn as persisten volume with K8S
• Part5f: Use OpenEBS as persisten volume with K8S
• Part5f: vSphere persistent storage for K8S
• Part6: Kubernetes volume expansion with Ceph RBD CSI driver
• Part7a: Install k8s with IPVS mode
• Part7b: Install k8s with IPVS mode
• Part8: Use Helm with K8S
• Part9: Tillerless helm2 install
• Part10: Kubernetes Dashboard SSO
• Part11: Kuberos for K8S
• Part12: Gangway for K8S
• Part13a: Velero Backup for K8S
• Part13b: How to Backup Kubernetes to git?
• Part14a: K8S Logging And Monitoring
• Part14b: Install Grafana Loki with Helm3

Environment

# openshift cluster
192.168.1.41  kubernetes01 # master node
192.168.1.42  kubernetes02 # frontend node
192.168.1.43  kubernetes03 # worker node
192.168.1.44  kubernetes04 # worker node
192.168.1.45  kubernetes05 # worker node

Helm with cluster-admin permissions

at <<EOF> helm-cluster-admin.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller-admin
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tiller-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: tiller-admin
  namespace: kube-system
EOF

Init Helm

kubectl create -f helm-cluster-admin.yaml

helm init --service-account helm
kubectl get po --all-namespaces | grep tiller

Tag node for ingress

kubectl get nodes --show-labels
kubectl label nodes kubernetes02 node-role.kubernetes.io/frontend= --overwrite=true

helm install stable/nginx-ingress \
    --name nginx-ingress \
    --namespace=nginx-ingress \
    --set rbac.create=true \
    --set controller.kind=DaemonSet \
    --set controller.hostNetwork=true \
    --set controller.daemonset.useHostPort=true \
    --set controller.nodeSelector."node-role\.kubernetes\.io/frontend"= \
    --set controller.stats.enabled=true \
    --set controller.metrics.enabled=true

kubectl --namespace nginx-ingress get services -o wide -w nginx-ingress-controller
kubectl create secret tls default-ingress-tls --key /path/to/private.pem --cert /path/to/cert.pem --namespace nginx-ingress

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta1/aio/deploy/recommended.yaml

kubectl create secret tls default-ingress-tls --key /path/to/private.pem --cert /path/to/cert.pem --namespace kubernetes-dashboard

cat <<EOF> dashboard_ingress.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: kubernetes-dashboard-ingress
  namespace: kubernetes-dashboard
  annotations:
    kubernetes.io/tls-acme: 'true'
    ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/secure-backends: "true"
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - dashboard.devopstales.intra
    secretName: default-ingress-tls
  rules:
  - host: dashboard.devopstales.intra
    http:
     paths:
     - backend:
         serviceName: kubernetes-dashboard
         servicePort: 443
EOF

kubectl apply -f dashboard_ingress.yml

kubectl create serviceaccount dashboard-admin-sa
kubectl create clusterrolebinding dashboard-admin-sa --clusterrole=cluster-admin --serviceaccount=default:dashboard-admin-sa

kubectl get secrets
NAME                  TYPE                                  DATA   AGE
dashboard-admin-sa-token-XXXXX   kubernetes.io/service-account-token   3      22h

kubectl describe secret dashboard-admin-sa-token-XXXXX
Name:         dashboard-admin-sa-token-bq9cr
...
token:      XXXXXXXXXXXXXXXXXXXXXXXXXX

# use this token to login