posted

This blog is written by Abrar Shivani who is a software engineer in Cloud Native Applications Storage team.

This blog post will provide an overview of vSphere Cloud Provider which exposes persistent storage for containers orchestrated by Kubernetes on top of vSphere. By using a sample WordPress-MySQL application, this blog will provide a step-by-step guide on how administrators can use vSphere Cloud Provider to make application data highly available.

 

vSphere Cloud Provider

Cloud Provider is a module in Kubernetes which provides an interface for managing nodes, volumes and networking routes. VMware contributes to both vSphere and Photon Cloud Provider.

chart1

 

Containers launched using Kubernetes can be resurrected. Yet, the data stored by application running inside that container will be lost once container goes down. With vSphere Cloud Provider the data can be stored in vSphere Persistent Volume and after rescheduling of the pods containers get the data back wherever it is scheduled. vSphere Cloud Provider enables access to vSphere managed storage (vSAN, VMFS, NFS) for applications deployed in Kubernetes. This is achieved by  supporting persistent volumes and storageclass primitives in Kubernetes. Moreover, it interacts with vCenter to support various operations such as creation and deletion of volumes, attaching and detaching volumes to application pods and nodes.  vSphere Cloud Provider creates Persistent Volumes backed by VMDK and mounts to the node where pod is scheduled and available for pod to use. Later, when pod fails and rescheduled, vSphere Cloud Provider will automatically detach volume from the node and attach back to the node where new pod is scheduled. It is mounted on same location as earlier and pod gets its data back. Thus, storage failover is completely transparent to Kubernetes PODs.

 

Let’s briefly go over storage primitives in Kubernetes that will help us understand this blog much better.

  • StorageClass – StorageClass describes custom parameters that are passed to vSphere Cloud Provider for creating vmdk (example: diskformat).
  • PersistentVolume – PersistentVolume is Kubernetes API object which is associated with volume. This is created automatically if volume is provisioned dynamically.
  • PersistentVolumeClaim – PersistentVolumeClaim describes user requirements for storage. (example: volume size)
  • Service – Service is an abstraction that defines a set of pods and a policy to access them.
  • Deployment – It makes sure that pod is running and provides declarative updates for pods and replica sets.

Deploying WordPress-MySQL Application with local storage

Let’s deploy WordPress application using kubectl. For this demo, you will need a Kubernetes cluster configured with vSphere Cloud Provider to create and access vSphere volumes. Kubernetes cluster can be launched with Kubernetes-anywhere.

As shown below, the proposed deployment for the application contains two pods containing one container: one for MySQL and other is WordPress.

chart2

Deploy MySQL

1.     apiVersion: v1

2.     kind: Service

3.     metadata:

4.       name: wordpress-mysql

5.       labels:

6.         app: wordpress

7.     spec:

8.       ports:

9.         - port: 3306

10.    selector:

11.      app: wordpress

12.      tier: mysql

13.    clusterIP: None

14.  ---

15.  apiVersion: extensions/v1beta1

16.  kind: Deployment

17.  metadata:

18.    name: wordpress-mysql

19.    labels:

20.      app: wordpress

21.  spec:

22.    strategy:

23.      type: Recreate

24.    template:

25.      metadata:

26.        labels:

27.          app: wordpress

28.          tier: mysql

29.      spec:

30.        containers:

31.        - image: mysql:5.6

32.          name: mysql

33.          env:

34.          - name: MYSQL_ROOT_PASSWORD

35.            value: mysqlpassword

36.          ports:

37.          - containerPort: 3306

38.            name: mysql

 

 

Let’s go over mysql-deployment.yaml.  Lines 1 – 13 describe service object of Kubernetes. This service selects pods with label

`app: wordpress` and `tier: mysql`(Line: 10 – 12 ) and network requests are forwarded to the one of pods on port 3306 (Line: 9). Line 15 – 38 describes deployment object of Kubernetes. This deployment objects creates pod with single container that has image `mysql:5.6`(Line: 31). Moreover, MySQL credentials are passed to the container as environment variables. MySQL container exposes port 3306 where it accepts network requests.

 

Let’s deploy MySQL pod in Kubernetes. Run the following command to launch MySQL pod:

sc1

Once we execute above command, let’s verify whether MySQL pod, deployment and service are up.

sc2

Now, that we have MySQL container running, we will deploy WordPress.

Deploying WordPress

We will use following yaml for deploying WordPress.

# wordpress-deployment.yaml

1.     apiVersion: v1

2.     kind: Service

3.     metadata:

4.       name: wordpress

5.       labels:

6.         app: wordpress

7.     spec:

8.       ports:

9.         – port: 80

10.        nodePort: 30080

11.    selector:

12.      app: wordpress

13.      tier: frontend

14.    type: NodePort

15.  —

16.  apiVersion: extensions/v1beta1

17.  kind: Deployment

18.  metadata:

19.    name: wordpress

20.    labels:

21.      app: wordpress

22.  spec:

23.    strategy:

24.      type: Recreate

25.    template:

26.      metadata:

27.        labels:

28.          app: wordpress

29.          tier: frontend

30.      spec:

31.        containers:

32.        – image: wordpress:4.6.1-apache

33.          name: wordpress

34.          env:

35.          – name: WORDPRESS_DB_HOST

36.            value: wordpress-mysql

37.          – name: WORDPRESS_DB_PASSWORD

38.            value: mysqlpassword

39.          ports:

40.          – containerPort: 80

41.            name: wordpress

 

 

Let’s go over wordpress-deployment.yaml. Lines 1 – 14 describe service object of Kubernetes. This service selects pods with label

`app: wordpress` and `tier: frontend`(Line: 11 – 12 ). The incoming network requests on any node in Kubernetes cluster on port 30080 are forwarded to the one of pods with labels `app: wordpress` and `tier: frontend` on port 80 (Line: 9 – 10). Line 15 – 38 describes deployment object of Kubernetes. This deployment objects creates pod with single container that has image `wordpress:4.6.1-apache`(Line 32). Moreover, the host and MySQL credentials is passed to container as environment variables. WordPress container exposes port 80 where it accepts network requests.

 

Let’s deploy WordPress pod in Kubernetes. Run following command to launch WordPress pod:

Screen Shot 2017-04-14 at 12.15.01 PM

Once we execute the above command, let’s verify whether WordPress pod is up and running using following command:

Screen Shot 2017-04-14 at 12.15.11 PM

Let’s find the IP address and port to get access to the WordPress.

Screen Shot 2017-04-14 at 12.17.05 PM

Visit your brand new WordPress blog at http://10.160.241.61:30080

You’ll see the familiar WordPress start page:

wp1

Select your language and click Continue to configure your website.

 

Now, we have WordPress up so let’s see whether application data is accessible when MySQL pod goes down and rescheduled. Let’s try to kill MySQL pod.

Screen Shot 2017-04-14 at 12.18.14 PM

Another MySQL pod will be scheduled on available node.

Screen Shot 2017-04-14 at 12.18.54 PM

Visit the same WordPress URL now and you will find WordPress is unable to establish connection to database.

DBconn

Before we proceed to the next section to see how to persist data into vmdks using vSphere Cloud Provider in Kubernetes,

let’s clean up our setup by destroying all Kubernetes objects.

Screen Shot 2017-04-14 at 12.20.15 PM

Deploying WordPress-MySQL Application with vSphere Persistent Storage

 

In this section we will look at how we can persist data using vSphere Cloud Provider. First, we will provision the disk using storageclass and vsphere-volume provisioner. Later, we will use this storageclass to claim volumes using persistent volume claim. Once the claim is bound, we will use the volumes inside pods to store data.

 

You will need Kubernetes configured with vSphere Cloud Provider to create and access vSphere volumes. One of the ways you can have Kubernetes cluster setup with Cloud Provider configured is Kubernetes-Anywhere.

 

First, we need to define disk format for vmdk which will be created to persist MySQL and WordPress state. We can do this by creating storage class.

Let’s create storage-class using ‘vsphere-storage-class.yaml’.

# vsphere-storage-class.yaml

1.     kind: StorageClass

2.     apiVersion: storage.k8s.io/v1beta1

3.     metadata:

4.       name: fast

5.     provisioner: kubernetes.io/vsphere-volume

6.     parameters:

7.       diskformat: zeroedthick

This yaml describes that we will be using vsphere-volume provisioner (Line: 5) to provision disk with diskformat as ‘zeroedthick’ (Line: 7).

 

Run the following command to create storage class.

Screen Shot 2017-04-14 at 12.21.12 PM

Let’s validate if storage class is created.

Screen Shot 2017-04-14 at 12.21.51 PM

Deploy MySQL

Let’s deploy MySQL using mysql-deployment-vsphere.yaml

# mysql-deployment-vsphere.yaml

1.     apiVersion: v1

2.     kind: Service

3.     metadata:

4.       name: wordpress-mysql

5.       labels:

6.         app: wordpress

7.     spec:

8.       ports:

9.         – port: 3306

10.    selector:

11.      app: wordpress

12.      tier: mysql

13.    clusterIP: None

14.  —

15.  apiVersion: v1

16.  kind: PersistentVolumeClaim

17.  metadata:

18.    name: mysql-pv-claim

19.    annotations:

20.      volume.beta.kubernetes.io/storage-class: fast

21.    labels:

22.      app: wordpress

23.  spec:

24.    accessModes:

25.      – ReadWriteOnce

26.    resources:

27.      requests:

28.        storage: 20Gi

29.  —

30.  apiVersion: extensions/v1beta1

31.  kind: Deployment

32.  metadata:

33.    name: wordpress-mysql

34.    labels:

35.      app: wordpress

36.  spec:

37.    strategy:

38.      type: Recreate

39.    template:

40.      metadata:

41.        labels:

42.          app: wordpress

43.          tier: mysql

44.      spec:

45.        containers:

46.        – image: mysql:5.6

47.          name: mysql

48.          env:

49.          – name: MYSQL_ROOT_PASSWORD

50.            value: mysqlpassword

51.          ports:

52.          – containerPort: 3306

53.            name: mysql

54.          volumeMounts:

55.          – name: mysql-persistent-storage

56.            mountPath: /var/lib/mysql

57.        volumes:

58.        – name: mysql-persistent-storage

59.          persistentVolumeClaim:

60.            claimName: mysql-pv-claim

 

In above yaml, we declare service (Line: 1 – 13), persistent volume claim (Line: 15 – 28) and deployment objects of Kubernetes (Line: 30 – 60).

We have the same description for service as above. We describe persistent volume claim to use storage class which we just described above (Line: 20). Also, we mentioned that we need 20G for the volume (Line: 28). Once this claim is created it will provision vmdk of 20G with diskformat as zeroedthick. Now, let’s look at deployment description. Deployment description is same as above with additional volume information. We mention that we will use volume that is bound to claim  `mysql-pv-claim` (Line: 59 – 60) and mount it to the path ‘ /var/lib/mysql’ (Line: 56). Once MySQL container is launched vSphere Persistent Volume will be attached to the node on which it is launched and MySQL will write its data to the vmdk.

 

Run the following command to deploy MySQL,

Screen Shot 2017-04-14 at 12.22.33 PM

Let’s validate whether MySQL is deployed,

Screen Shot 2017-04-14 at 12.23.15 PM

Deploying WordPress

We will use following yaml for deploying WordPress.

# wordpress-deployment-vsphere.yaml

1.     apiVersion: v1

2.     kind: Service

3.     metadata:

4.       name: wordpress

5.       labels:

6.         app: wordpress

7.     spec:

8.       ports:

9.         – port: 80

10.        nodePort: 30080

11.    selector:

12.      app: wordpress

13.      tier: frontend

14.    type: NodePort

15.  —

16.  apiVersion: v1

17.  kind: PersistentVolumeClaim

18.  metadata:

19.    name: wp-pv-claim

20.    annotations:

21.      volume.beta.kubernetes.io/storage-class: fast

22.    labels:

23.      app: wordpress

24.  spec:

25.    accessModes:

26.      – ReadWriteOnce

27.    resources:

28.      requests:

29.        storage: 20Gi

30.  —

31.  apiVersion: extensions/v1beta1

32.  kind: Deployment

33.  metadata:

34.    name: wordpress

35.    labels:

36.      app: wordpress

37.  spec:

38.    strategy:

39.      type: Recreate

40.    template:

41.      metadata:

42.        labels:

43.          app: wordpress

44.          tier: frontend

45.      spec:

46.        containers:

47.        – image: wordpress:4.6.1-apache

48.          name: wordpress

49.          env:

50.          – name: WORDPRESS_DB_HOST

51.            value: wordpress-mysql

52.          – name: WORDPRESS_DB_PASSWORD

53.            value: mysqlpassword

54.          ports:

55.          – containerPort: 80

56.            name: wordpress

57.          volumeMounts:

58.          – name: wordpress-persistent-storage

59.            mountPath: /var/www/html

60.        volumes:

61.        – name: wordpress-persistent-storage

62.          persistentVolumeClaim:

63.            claimName: wp-pv-claim

 

Similarly, this yaml creates service, persistent volume claim and deployment objects in Kubernetes for WordPress.

 

Run following command to deploy WordPress,

Screen Shot 2017-04-14 at 12.25.32 PM

Let’s validate whether WordPress is deployed,

Screen Shot 2017-04-14 at 12.26.21 PM

Now we have WordPress and MySQL up and running let’s visit our new WordPress blog.

Enter following command to get the ip address and port to access the blog,

Screen Shot 2017-04-14 at 12.27.01 PM

Just enter this ‘10.160.241.61:30080’ in the browser to visit the WordPress blog.

Once WordPress is configured. Let’s take down the WordPress pod.

Screen Shot 2017-04-14 at 12.27.36 PM

Visit the same WordPress URL and see WordPress app is accessible with the same configuration.

Now let’s take down MySQL pod.

Screen Shot 2017-04-14 at 12.28.12 PM

Another MySQL pod will be scheduled on available node.

Screen Shot 2017-04-14 at 12.29.00 PM

Once pod is restarted by Kubernetes. The pod will pick up the same volumes and its state is intact.

 

As you can see, we can easily persist the state of containers using vSphere Cloud Provider!

 

We would love to hear your feedback! We will be at DockerCon. Please drop by at booth G9 to learn more about what we have to offer.