Cassandra on OpenEBS in OpenShift

Cassandra:

Apache Cassandra is a free and open-source, distributed, wide column store, NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure

OpenEBS:

OpenEBS is the leading open-source project for container-attached and container-native storage on Kubernetes. OpenEBS adopts Container Attached Storage (CAS) approach, where each workload is provided with a dedicated storage controller. OpenEBS implements granular storage policies and isolation that enable users to optimize storage for each specific workload.

OpenShift:

Red Hat OpenShift is a multifaceted, open source container application platform from Red Hat Inc. for the development, deployment and management of applications.

Deploying Cassandra on OpenEBS:

Pre-requisites:

Storage pool:

OpenEBS provisions persistent volumes for containerized applications in its storage pool. The disks attached in a cluster can be obtained through the following command.

[root@master deployers]# oc get disksNAME AGEdisk-0cfccca1c2b46241890953a725d6e5bd 80ddisk-0e570c2917f88f553460bfed72c33c5a 19ddisk-17f034148a30ef9ae2557cffda9ae869 20ddisk-2cb335326e30192286ce250a7e9562b8 80ddisk-42157bb4ec1762a13324453f57359964 81ddisk-42b4fb20cd36896dfc2a486b977363de 82ddisk-5ebe0072a9e937b28315006adf648dc2 82ddisk-5ffac6eccd389ed98a48e4ab3327e7e9 81ddisk-619def6e7b3147a7a57e9e31d71f2906 80ddisk-61efde57f8c657e83f23a3adbacfb08c 80d

You can use above disks to create storage pool of different types using following manifest. Create cstor-spc.yml using the below manifest and populate the required disks under diskList spec.

---apiVersion: openebs.io/v1alpha1kind: StoragePoolClaimmetadata:name: cstor-poolannotations:cas.openebs.io/config: |- name: PoolResourceRequestsvalue: |-memory: 1Gicpu: 100m- name: PoolResourceLimitsvalue: |-memory: 4Gi- name: AuxResourceLimitsvalue: |-memory: 0.5Gicpu: 50mspec:name: cstor-pooltype: diskmaxPools: 5poolSpec:poolType: stripedcacheFile: /tmp/pool2.cacheoverProvisioning: falsedisks:diskList:- disk-0cfccca1c2b46241890953a725d6e5bd- disk-2cb335326e30192286ce250a7e9562b8- disk-619def6e7b3147a7a57e9e31d71f2906- disk-61efde57f8c657e83f23a3adbacfb08c- disk-99127001e6b6bc190b5169e42ebecad7

Apply the above manifest after updating disk details.

oc apply -f cstor-spc.yml

To use OpenEBS Storage by an application, a storage class should have been created in the K8s cluster updated with its provisioner and storage pool.

Create a storage class template storage-class.yml using the following configuration.

---apiVersion: storage.k8s.io/v1kind: StorageClassmetadata:name: cstor-poolannotations:openebs.io/cas-type: cstorcas.openebs.io/config: |- name: StoragePoolClaimvalue: “cstor-pool”- name: ReplicaCountvalue: “1”provisioner: openebs.io/provisioner-iscsi

Update the desired storage pool under StoragePoolClaim in above configuration and apply it to create storage class.

oc apply -f storage-class.yml

Cassandra deployment:

Lets deploy Cassandra as Kubernetes Statefulset. A statefulset in Kubernetes requires a headless service to provide network access to the pods it creates.

Create such headless service using the following manifest.

---apiVersion: v1kind: Servicemetadata:labels:app: cassandraname: cassandraspec:clusterIP: Noneports:- port: 9042selector:app: cassandra

Apply the above manifest to create headless service.

oc apply -f cassandra-service.yml

Create cassandra-statefulset.yml as shown below which can create a Statefulset for Cassandra with three replicas.

---
apiVersion: apps/v1beta1
kind: StatefulSetmetadata:name: cassandralabels:app: cassandraspec:serviceName: cassandrareplicas: 3selector:matchLabels:app: cassandratemplate:metadata:labels:app: cassandraspec:containers:- name: cassandraimage: gcr.io/google-samples/cassandra:v11imagePullPolicy: Alwaysports:- containerPort: 7000name: intra-node- containerPort: 7001name: tls-intra-node- containerPort: 7199name: jmx- containerPort: 9042name: cqlresources:limits:cpu: “500m”memory: 1Girequests:cpu: “500m”memory: 1GisecurityContext:capabilities:add:- IPC_LOCKlifecycle:preStop:exec:command: [“/bin/sh”, “-c”, “PID=$(pidof java) && kill $PID && while ps -p $PID > /dev/null; do sleep 1; done”]env:- name: MAX_HEAP_SIZEvalue: 512M- name: HEAP_NEWSIZEvalue: 100M- name: CASSANDRA_SEEDSvalue: “cassandra-0.cassandra.cassandra-cstor.svc.cluster.local”- name: CASSANDRA_CLUSTER_NAMEvalue: “K8Demo”- name: CASSANDRA_DCvalue: “DC1-K8Demo”- name: CASSANDRA_RACKvalue: “Rack1-K8Demo”- name: CASSANDRA_AUTO_BOOTSTRAPvalue: “false”- name: POD_IPvalueFrom:fieldRef:fieldPath: status.podIPreadinessProbe:exec:command:- /bin/bash- -c- /ready-probe.shinitialDelaySeconds: 15timeoutSeconds: 5volumeMounts:- name: cassandra-claimmountPath: /cassandra_datavolumeClaimTemplates:- metadata:name: cassandra-claimannotations:volume.beta.kubernetes.io/storage-class: cstor-scspec:accessModes: [ “ReadWriteOnce” ]resources:requests:storage: 5G

Make sure to update the storage class, namespace and volume claim name in the above configuration before applying it.

Apply the above configuration to create cassandra ring.

oc apply -f cassandra-statefulset.yml -n <namespace>

You can check the statefulset status through following command.

[root@master deployers]# oc get stsNAME DESIRED CURRENT AGEcassandra 3 3 3h

Ensure that the PVC is bound to a volume through following command.

oc get pvc -n <namespace>NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEcassandra-claim-cassandra-0 Bound pvc-c0d14696–6fd8–11e9–8a2a-0050569331ce 5G RWO cstor-sc 3hcassandra-claim-cassandra-1 Bound pvc-1c040419–6fd9–11e9–8a2a-0050569331ce 5G RWO cstor-sc 3hcassandra-claim-cassandra-2 Bound pvc-969bc0a9–6fd9–11e9–8a2a-0050569331ce 5G RWO cstor-sc 3h

Ensure that the cassandra cluster is created using nodetool status command.

root@cassandra-2:/var/lib/cassandra# nodetool statusDatacenter: DC1-K8Demo======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving— Address Load Tokens Owns (effective) Host ID RackUN 192.30.5.71 166.93 KiB 32 60.8% 93b884e0-fb26–4ef3–903c-782cdde6bf74 Rack1-K8DemoUN 192.28.6.145 129.61 KiB 32 64.0% 1b7d60d2–7cbb-4445–8bcb-3389c0c7b0e4 Rack1-K8DemoUN 192.29.6.160 107.04 KiB 32 75.3% 7d37c2e5–1929–4d01-a164-ef243a575ae3 Rack1-K8Demo
kubectl exec -it cassandra-0 -- bash
root@cassandra-0:/# cqlsh

Connected to K8Demo at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.9 | CQL spec 3.4.2 | Native protocol v4]
Use HELP for help.

cqlsh> CREATE KEYSPACE openebsdb WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 2 };
cqlsh> use openebsdb;
cqlsh:openebsdb> CREATE TABLE emp(emp_id int PRIMARY KEY, emp_name text, emp_city text, emp_sal varint,emp_phone varint);
cqlsh:demodb> INSERT INTO emp (emp_id, emp_name, emp_city, emp_phone) VALUES(42,'giri', 'blr', '3278362832')

Software Engineer at Mayadata Inc.