Kubernetes Ceph RBD volume with CSI driver

Page content

In this post I will show you how can you use CEPH RBD with CSI driver for persistent storagi on Kubernetes.

Parst of the Openshift series

The Container Storage Interface (CSI) is a standard for exposing arbitrary block and file storage storage systems to Kubernetes. Using CSI third-party storage providers can write and deploy plugins exposing storage systems in Kubernetes. Bbefore we begin lets ensure that we have the following requirements:

  • Kubernetes cluster v1.14+
  • allow-privileged flag enabled for both kubelet and API server
  • Running Ceph cluster

    git clone https://github.com/ceph/ceph-csi.git
    cd ceph-csi/deploy/rbd/kubernetes/v1.14+/
    
    kubectl create -f csi-nodeplugin-rbac.yaml
    kubectl create -f csi-provisioner-rbac.yaml
    
    nano csi-config-map.yaml
    ---
    apiVersion: v1
    kind: ConfigMap
    data:
    config.json: |-
    [
      {
        "clusterID": "k8s-ceph",
        "monitors": [
          "192.168.1.31:6790",
          "192.168.1.32:6790",
          "192.168.1.33:6790"
        ]
      }
    ]
    metadata:
    name: ceph-csi-config
    
    
    kubectl create -fcsi-config-map.yaml
    
    kubectl create -f csi-rbdplugin-provisioner.yaml
    kubectl create -f csi-rbdplugin.yaml
    
    ceph auth get-key client.admin|base64
    QVFDTDliVmNEb21I32SHoPxXNGhmRkczTFNtcXM0ZW5VaXlTZEE977==
    
    nano csi-rbd-secret.yaml
    ---
    apiVersion: v1
    kind: Secret
    metadata:
    name: csi-rbd-secret
    namespace: default
    data:
    userID: admin
    userKey: QVFDTDliVmNEb21I32SHoPxXNGhmRkczTFNtcXM0ZW5VaXlTZEE977==
    
    nano rbd-csi-sc.yaml
    ---
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
    name: csi-rbd
    provisioner: rbd.csi.ceph.com
    parameters:
    monitors: 192.168.1.31:6790,192.168.1.32:6790,192.168.1.33:6790
    clusterID: k8s-ceph
    pool: rbd
    imageFeatures: layering
    csi.storage.k8s.io/provisioner-secret-name: csi-rbd-secret
    csi.storage.k8s.io/provisioner-secret-namespace: default
    csi.storage.k8s.io/node-publish-secret-name: csi-rbd-secret
    csi.storage.k8s.io/node-publish-secret-namespace: default
    adminid: admin
    csi.storage.k8s.io/fstype: ext4
    reclaimPolicy: Delete
    mountOptions:
    - discard
    
    kubectl create -f csi-rbd-secret.yaml
    kubectl create -f rbd-csi-sc.yaml
    
    kubectl get storageclass
    NAME      PROVISIONER        AGE
    csi-rbd   rbd.csi.ceph.com   15s
    
    nano raw-block-pvc.yaml
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
    name: raw-block-pvc
    spec:
    accessModes:
    - ReadWriteMany
    volumeMode: Block
    resources:
    requests:
      storage: 1Gi
    storageClassName: csi-rbd
    
    kubectl create -f raw-block-pvc.yaml
    
    kubectl get pvc
    NAME            STATUS    VOLUME                                  
    raw-block-pvc   Bound     pvc-fd66b4d6-757d-22e9-8f9e-4f86e2356a59
    
comments powered by Disqus