K8s ERROR at kubectl logs

Kubernetes

I installed a kubernetes cluster in a Vagran environment. First everything was fine, but wen I try to add the command kubectl logs I got this error:

$ kubectl logs busybox-7c9687585b-12d75
Error from server (NotFound): the server could not find the requested resource ( pods/log busybox-7c9687585b-12d75)

and another for port-forward and exec:

$ kubectl exec -it busybox-7c9687585b-12d75 -- /bin/sh
error: unable to upgrade connection: pod does not exist

When I eun with -v=9 I hot HTTP Error Code 404 all around. I wondered if the connection with the kubelet is wrong because of the multiple interfaces of the VMs? So I tested it:

$ kubectl get nodes worker1 -o yaml
apiVersion: v1
kind: Node
...
status:
  addresses:
  - address: 10.0.2.15
...

The was the problem, the address is came from the NAT interface of the VM not from the bridged. But why? On the master I explicitly set the --apiserver-advertise-address for the bridged interface’s IP. I needed to add an explicit IP address of the bridged interface on the workers too. The problem is there is no option for that in kubeadm.

I looked at the man page of the kubelet and I found the following option:

--node-ip string    IP address of the node. If set, kubelet will use this IP address for the node

That is wat I need to set. So I added a no job to the end of my bootstrap scripts for master and worker nodes too.

echo KUBELET_EXTRA_ARGS=\"--node-ip=`ip addr show enp0s8 | grep inet | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}/" | tr -d '/'`\" > /etc/sysconfig/kubelet
systemctl restart kubelet