Cassandra on OpenEBS in OpenShift

4 min readMay 7, 2019


Apache Cassandra is a free and open-source, distributed, wide column store, NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure


OpenEBS is the leading open-source project for container-attached and container-native storage on Kubernetes. OpenEBS adopts Container Attached Storage (CAS) approach, where each workload is provided with a dedicated storage controller. OpenEBS implements granular storage policies and isolation that enable users to optimize storage for each specific workload.


Red Hat OpenShift is a multifaceted, open source container application platform from Red Hat Inc. for the development, deployment and management of applications.

Deploying Cassandra on OpenEBS:


Storage pool:

OpenEBS provisions persistent volumes for containerized applications in its storage pool. The disks attached in a cluster can be obtained through the following command.

[root@master deployers]# oc get disksNAME AGEdisk-0cfccca1c2b46241890953a725d6e5bd 80ddisk-0e570c2917f88f553460bfed72c33c5a 19ddisk-17f034148a30ef9ae2557cffda9ae869 20ddisk-2cb335326e30192286ce250a7e9562b8 80ddisk-42157bb4ec1762a13324453f57359964 81ddisk-42b4fb20cd36896dfc2a486b977363de 82ddisk-5ebe0072a9e937b28315006adf648dc2 82ddisk-5ffac6eccd389ed98a48e4ab3327e7e9 81ddisk-619def6e7b3147a7a57e9e31d71f2906 80ddisk-61efde57f8c657e83f23a3adbacfb08c 80d

You can use above disks to create storage pool of different types using following manifest. Create cstor-spc.yml using the below manifest and populate the required disks under diskList spec.

---apiVersion: StoragePoolClaimmetadata:name: |- name: PoolResourceRequestsvalue: |-memory: 1Gicpu: 100m- name: PoolResourceLimitsvalue: |-memory: 4Gi- name: AuxResourceLimitsvalue: |-memory: 0.5Gicpu: 50mspec:name: cstor-pooltype: diskmaxPools: 5poolSpec:poolType: stripedcacheFile: /tmp/pool2.cacheoverProvisioning: falsedisks:diskList:- disk-0cfccca1c2b46241890953a725d6e5bd- disk-2cb335326e30192286ce250a7e9562b8- disk-619def6e7b3147a7a57e9e31d71f2906- disk-61efde57f8c657e83f23a3adbacfb08c- disk-99127001e6b6bc190b5169e42ebecad7

Apply the above manifest after updating disk details.

oc apply -f cstor-spc.yml

To use OpenEBS Storage by an application, a storage class should have been created in the K8s cluster updated with its provisioner and storage pool.

Create a storage class template storage-class.yml using the following configuration.

---apiVersion: StorageClassmetadata:name: |- name: StoragePoolClaimvalue: “cstor-pool”- name: ReplicaCountvalue: “1”provisioner:

Update the desired storage pool under StoragePoolClaim in above configuration and apply it to create storage class.

oc apply -f storage-class.yml

Cassandra deployment:

Lets deploy Cassandra as Kubernetes Statefulset. A statefulset in Kubernetes requires a headless service to provide network access to the pods it creates.

Create such headless service using the following manifest.

---apiVersion: v1kind: Servicemetadata:labels:app: cassandraname: cassandraspec:clusterIP: Noneports:- port: 9042selector:app: cassandra

Apply the above manifest to create headless service.

oc apply -f cassandra-service.yml

Create cassandra-statefulset.yml as shown below which can create a Statefulset for Cassandra with three replicas.

apiVersion: apps/v1beta1
kind: StatefulSetmetadata:name: cassandralabels:app: cassandraspec:serviceName: cassandrareplicas: 3selector:matchLabels:app: cassandratemplate:metadata:labels:app: cassandraspec:containers:- name: cassandraimage: Alwaysports:- containerPort: 7000name: intra-node- containerPort: 7001name: tls-intra-node- containerPort: 7199name: jmx- containerPort: 9042name: cqlresources:limits:cpu: “500m”memory: 1Girequests:cpu: “500m”memory: 1GisecurityContext:capabilities:add:- IPC_LOCKlifecycle:preStop:exec:command: [“/bin/sh”, “-c”, “PID=$(pidof java) && kill $PID && while ps -p $PID > /dev/null; do sleep 1; done”]env:- name: MAX_HEAP_SIZEvalue: 512M- name: HEAP_NEWSIZEvalue: 100M- name: CASSANDRA_SEEDSvalue: “cassandra-0.cassandra.cassandra-cstor.svc.cluster.local”- name: CASSANDRA_CLUSTER_NAMEvalue: “K8Demo”- name: CASSANDRA_DCvalue: “DC1-K8Demo”- name: CASSANDRA_RACKvalue: “Rack1-K8Demo”- name: CASSANDRA_AUTO_BOOTSTRAPvalue: “false”- name: POD_IPvalueFrom:fieldRef:fieldPath: status.podIPreadinessProbe:exec:command:- /bin/bash- -c- /ready-probe.shinitialDelaySeconds: 15timeoutSeconds: 5volumeMounts:- name: cassandra-claimmountPath: /cassandra_datavolumeClaimTemplates:- metadata:name: cstor-scspec:accessModes: [ “ReadWriteOnce” ]resources:requests:storage: 5G

Make sure to update the storage class, namespace and volume claim name in the above configuration before applying it.

Apply the above configuration to create cassandra ring.

oc apply -f cassandra-statefulset.yml -n <namespace>

You can check the statefulset status through following command.

[root@master deployers]# oc get stsNAME DESIRED CURRENT AGEcassandra 3 3 3h

Ensure that the PVC is bound to a volume through following command.

oc get pvc -n <namespace>NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEcassandra-claim-cassandra-0 Bound pvc-c0d14696–6fd8–11e9–8a2a-0050569331ce 5G RWO cstor-sc 3hcassandra-claim-cassandra-1 Bound pvc-1c040419–6fd9–11e9–8a2a-0050569331ce 5G RWO cstor-sc 3hcassandra-claim-cassandra-2 Bound pvc-969bc0a9–6fd9–11e9–8a2a-0050569331ce 5G RWO cstor-sc 3h

Ensure that the cassandra cluster is created using nodetool status command.

root@cassandra-2:/var/lib/cassandra# nodetool statusDatacenter: DC1-K8Demo======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving— Address Load Tokens Owns (effective) Host ID RackUN 166.93 KiB 32 60.8% 93b884e0-fb26–4ef3–903c-782cdde6bf74 Rack1-K8DemoUN 129.61 KiB 32 64.0% 1b7d60d2–7cbb-4445–8bcb-3389c0c7b0e4 Rack1-K8DemoUN 107.04 KiB 32 75.3% 7d37c2e5–1929–4d01-a164-ef243a575ae3 Rack1-K8Demo

Write sample data:

kubectl exec -it cassandra-0 -- bash
root@cassandra-0:/# cqlsh

Connected to K8Demo at
[cqlsh 5.0.1 | Cassandra 3.9 | CQL spec 3.4.2 | Native protocol v4]
Use HELP for help.

cqlsh> CREATE KEYSPACE openebsdb WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 2 };
cqlsh> use openebsdb;
cqlsh:openebsdb> CREATE TABLE emp(emp_id int PRIMARY KEY, emp_name text, emp_city text, emp_sal varint,emp_phone varint);
cqlsh:demodb> INSERT INTO emp (emp_id, emp_name, emp_city, emp_phone) VALUES(42,'giri', 'blr', '3278362832')