Kubernetes CephFS volume with CSI driver

Page content

In this post I will show you how can you use CephFS with CSI driver for persistent storage on Kubernetes.

Parts of the Kubernetes series

The Container Storage Interface (CSI) is a standard for exposing arbitrary block and file storage storage systems to Kubernetes. Using CSI third-party storage providers can write and deploy plugins exposing storage systems in Kubernetes. Before we begin lets ensure that we have the following requirements:

  • Kubernetes cluster v1.14+
  • allow-privileged flag enabled for both kubelet and API server
  • Running Ceph cluster
  • Created CephFS

First we need to create a namespace for the storage provider:

kubectl create ns ceph-csi-cephfs
kubens ceph-csi-cephfs

Login to the CEPH cluster and get the configurations:

ceph config generate-minimal-conf > ceph-minimal.conf

cat ceph-minimal.conf
# minimal ceph.conf for e285a458-7c95-4187-8129-fbd6c370c537
    fsid = e285a458-7c95-4187-8129-fbd6c370c537
    mon_host = [v2:,v1:] [v2:,v1:] [v2:,v1:]

ceph auth get-key client.admin
helm repo add ceph-csi https://ceph.github.io/csi-charts
helm show values ceph-csi/ceph-csi-cephfs > defaultValues.yaml

Crate helm values file:

nano values.yaml
  - clusterID: e285a458-7c95-4187-8129-fbd6c370c537
      subvolumeGroup: "csi"
  name: csi-cephfs-secret
  adminID: admin
  adminKey: QVFDWDNuVmtNV3NvSlJBQUFvazIxMCszZXFxNmF6SmpT5WJjaUE9PQ==
  create: true
  create: true
  name: k8s-cephfs
  clusterID: e285a458-7c95-4187-8129-fbd6c370c537
  # (required) CephFS filesystem name into which the volume shall be created
  fsName: k8s-etc-nvme
  reclaimPolicy: Delete
  allowVolumeExpansion: true
  volumeNamePrefix: "poc-k8s-"
  provisionerSecret: csi-cephfs-secret
  controllerExpandSecret: csi-cephfs-secret
  nodeStageSecret: csi-cephfs-secret

Deploy helm chart:

helm upgrade --install ceph-csi-cephfs ceph-csi/ceph-csi-cephfs --values ./values.yaml

Demo time

nano pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
  name: csi-cephfs-pvc
    - ReadWriteMany
      storage: 1Gi
  storageClassName: csi-cephfs-sc
kubectl apply -f pvc.yaml

kubectl get pvc
NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS    AGE
csi-cephfs-pvc   Bound    pvc-51526639-6fef-4abd-b453-c2b03c08781f   1Gi        RWX            csi-cephfs-sc   31m