Ceph: who is mapping a RBD device

Openshift, Ceph
Page content

Main problem

I get this error in Openshift:

  MountVolume.WaitForAttach failed for volume "pvc-9fcd3d08-d14e-11e9-a958-66934f1af826" :
  rbd image k8s-prod/kubernetes-dynamic-pvc-a0029613-d14e-11e9-aaf5-8611e6b1c395 is still being used

Solution

So I Wanna know who is using this DBD device?

rbd status k8s-prod/kubernetes-dynamic-pvc-a0029613-d14e-11e9-aaf5-8611e6b1c395
Watchers:
	watcher=192.168.1.43:0/3614154426 client.165467009 cookie=18446462598732840961

To solve this problem you need to restart Openshift’s Kubernetes components.

ssh 192.168.1.43
systemctl restart docker origin-node

Check if it is still mounted:

lsblk | grep "pvc-9fcd3d08-d14e-11e9-a958-66934f1af826"
rbd12                                252:192  0  100G  0 disk /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-9fcd3d08-d14e-11e9-a958-66934f1af826/globalmount/0001-0024-e285a458-7c95-4187-8129-fbd6c370c537-0000000000000006-12154609-2d98-11ec-9b66-66a9f6ca788c

With the restart of the docker and the kubelet the container dse not running anymore, so you can unmount it. Fot theat you need the rnd client.

rbd unmap k8s-prod/kubernetes-dynamic-pvc-a0029613-d14e-11e9-aaf5-8611e6b1c395
# or 
rbd unmap -o force k8s-prod/kubernetes-dynamic-pvc-a0029613-d14e-11e9-aaf5-8611e6b1c395