This blog is written by Abrar Shivani who is a software engineer in Cloud Native Applications Storage team.

This blog post will provide an overview of vSphere Cloud Provider which exposes persistent storage for containers orchestrated by Kubernetes on top of vSphere. By using a sample WordPress-MySQL application, this blog will provide a step-by-step guide on how administrators can use vSphere Cloud Provider to make application data highly available.

 

vSphere Cloud Provider

Cloud Provider is a module in Kubernetes which provides an interface for managing nodes, volumes and networking routes. VMware contributes to both vSphere and Photon Cloud Provider.

 

Containers launched using Kubernetes can be resurrected. Yet, the data stored by application running inside that container will be lost once container goes down. With vSphere Cloud Provider the data can be stored in vSphere Persistent Volume and after rescheduling of the pods containers get the data back wherever it is scheduled. vSphere Cloud Provider enables access to vSphere managed storage (vSAN, VMFS, NFS) for applications deployed in Kubernetes. This is achieved by  supporting persistent volumes and storageclass primitives in Kubernetes. Moreover, it interacts with vCenter to support various operations such as creation and deletion of volumes, attaching and detaching volumes to application pods and nodes.  vSphere Cloud Provider creates Persistent Volumes backed by VMDK and mounts to the node where pod is scheduled and available for pod to use. Later, when pod fails and rescheduled, vSphere Cloud Provider will automatically detach volume from the node and attach back to the node where new pod is scheduled. It is mounted on same location as earlier and pod gets its data back. Thus, storage failover is completely transparent to Kubernetes PODs.

 

Let’s briefly go over storage primitives in Kubernetes that will help us understand this blog much better.

• StorageClass – StorageClass describes custom parameters that are passed to vSphere Cloud Provider for creating vmdk (example: diskformat).
• PersistentVolume – PersistentVolume is Kubernetes API object which is associated with volume. This is created automatically if volume is provisioned dynamically.
• PersistentVolumeClaim – PersistentVolumeClaim describes user requirements for storage. (example: volume size)
• Service – Service is an abstraction that defines a set of pods and a policy to access them.
• Deployment – It makes sure that pod is running and provides declarative updates for pods and replica sets.

Deploying WordPress-MySQL Application with local storage

Let’s deploy WordPress application using kubectl. For this demo, you will need a Kubernetes cluster configured with vSphere Cloud Provider to create and access vSphere volumes. Kubernetes cluster can be launched with Kubernetes-anywhere.

As shown below, the proposed deployment for the application contains two pods containing one container: one for MySQL and other is WordPress.

Deploy MySQL

 

 

Let’s go over mysql-deployment.yaml.  Lines 1 – 13 describe service object of Kubernetes. This service selects pods with label

app: wordpress and tier: mysql(Line: 10 – 12 ) and network requests are forwarded to the one of pods on port 3306 (Line: 9). Line 15 – 38 describes deployment object of Kubernetes. This deployment objects creates pod with single container that has image mysql:5.6(Line: 31). Moreover, MySQL credentials are passed to the container as environment variables. MySQL container exposes port 3306 where it accepts network requests.

 

Let’s deploy MySQL pod in Kubernetes. Run the following command to launch MySQL pod:

Once we execute above command, let’s verify whether MySQL pod, deployment and service are up.

Now, that we have MySQL container running, we will deploy WordPress.

Deploying WordPress

We will use following yaml for deploying WordPress.

# wordpress-deployment.yaml 1.     apiVersion: v1 2.     kind: Service 3.     metadata: 4.       name: wordpress 5.       labels: 6.         app: wordpress 7.     spec: 8.       ports: 9.         – port: 80 10.        nodePort: 30080 11.    selector: 12.      app: wordpress 13.      tier: frontend 14.    type: NodePort 15.  — 16.  apiVersion: extensions/v1beta1 17.  kind: Deployment 18.  metadata: 19.    name: wordpress 20.    labels: 21.      app: wordpress 22.  spec: 23.    strategy: 24.      type: Recreate 25.    template: 26.      metadata: 27.        labels: 28.          app: wordpress 29.          tier: frontend 30.      spec: 31.        containers: 32.        – image: wordpress:4.6.1-apache 33.          name: wordpress 34.          env: 35.          – name: WORDPRESS_DB_HOST 36.            value: wordpress-mysql 37.          – name: WORDPRESS_DB_PASSWORD 38.            value: mysqlpassword 39.          ports: 40.          – containerPort: 80 41.            name: wordpress

 

 

Let’s go over wordpress-deployment.yaml. Lines 1 – 14 describe service object of Kubernetes. This service selects pods with label

app: wordpress and tier: frontend(Line: 11 – 12 ). The incoming network requests on any node in Kubernetes cluster on port 30080 are forwarded to the one of pods with labels app: wordpress and tier: frontend on port 80 (Line: 9 – 10). Line 15 – 38 describes deployment object of Kubernetes. This deployment objects creates pod with single container that has image wordpress:4.6.1-apache(Line 32). Moreover, the host and MySQL credentials is passed to container as environment variables. WordPress container exposes port 80 where it accepts network requests.

 

Let’s deploy WordPress pod in Kubernetes. Run following command to launch WordPress pod:

Once we execute the above command, let’s verify whether WordPress pod is up and running using following command:

Let’s find the IP address and port to get access to the WordPress.

Visit your brand new WordPress blog at http://10.160.241.61:30080

You’ll see the familiar WordPress start page:

Select your language and click Continue to configure your website.

 

Now, we have WordPress up so let’s see whether application data is accessible when MySQL pod goes down and rescheduled. Let’s try to kill MySQL pod.

Another MySQL pod will be scheduled on available node.

Visit the same WordPress URL now and you will find WordPress is unable to establish connection to database.

Before we proceed to the next section to see how to persist data into vmdks using vSphere Cloud Provider in Kubernetes,

let’s clean up our setup by destroying all Kubernetes objects.

Deploying WordPress-MySQL Application with vSphere Persistent Storage

 

In this section we will look at how we can persist data using vSphere Cloud Provider. First, we will provision the disk using storageclass and vsphere-volume provisioner. Later, we will use this storageclass to claim volumes using persistent volume claim. Once the claim is bound, we will use the volumes inside pods to store data.

 

You will need Kubernetes configured with vSphere Cloud Provider to create and access vSphere volumes. One of the ways you can have Kubernetes cluster setup with Cloud Provider configured is Kubernetes-Anywhere.

 

First, we need to define disk format for vmdk which will be created to persist MySQL and WordPress state. We can do this by creating storage class.

Let’s create storage-class using ‘vsphere-storage-class.yaml’.

# vsphere-storage-class.yaml 1.     kind: StorageClass 2.     apiVersion: storage.k8s.io/v1beta1 3.     metadata: 4.       name: fast 5.     provisioner: kubernetes.io/vsphere-volume 6.     parameters: 7.       diskformat: zeroedthick

This yaml describes that we will be using vsphere-volume provisioner (Line: 5) to provision disk with diskformat as ‘zeroedthick’ (Line: 7).

 

Run the following command to create storage class.

Let’s validate if storage class is created.

Deploy MySQL

Let’s deploy MySQL using mysql-deployment-vsphere.yaml

# mysql-deployment-vsphere.yaml 1.     apiVersion: v1 2.     kind: Service 3.     metadata: 4.       name: wordpress-mysql 5.       labels: 6.         app: wordpress 7.     spec: 8.       ports: 9.         – port: 3306 10.    selector: 11.      app: wordpress 12.      tier: mysql 13.    clusterIP: None 14.  — 15.  apiVersion: v1 16.  kind: PersistentVolumeClaim 17.  metadata: 18.    name: mysql-pv-claim 19.    annotations: 20.      volume.beta.kubernetes.io/storage-class: fast 21.    labels: 22.      app: wordpress 23.  spec: 24.    accessModes: 25.      – ReadWriteOnce 26.    resources: 27.      requests: 28.        storage: 20Gi 29.  — 30.  apiVersion: extensions/v1beta1 31.  kind: Deployment 32.  metadata: 33.    name: wordpress-mysql 34.    labels: 35.      app: wordpress 36.  spec: 37.    strategy: 38.      type: Recreate 39.    template: 40.      metadata: 41.        labels: 42.          app: wordpress 43.          tier: mysql 44.      spec: 45.        containers: 46.        – image: mysql:5.6 47.          name: mysql 48.          env: 49.          – name: MYSQL_ROOT_PASSWORD 50.            value: mysqlpassword 51.          ports: 52.          – containerPort: 3306 53.            name: mysql 54.          volumeMounts: 55.          – name: mysql-persistent-storage 56.            mountPath: /var/lib/mysql 57.        volumes: 58.        – name: mysql-persistent-storage 59.          persistentVolumeClaim: 60.            claimName: mysql-pv-claim

 

In above yaml, we declare service (Line: 1 – 13), persistent volume claim (Line: 15 – 28) and deployment objects of Kubernetes (Line: 30 – 60).

We have the same description for service as above. We describe persistent volume claim to use storage class which we just described above (Line: 20). Also, we mentioned that we need 20G for the volume (Line: 28). Once this claim is created it will provision vmdk of 20G with diskformat as zeroedthick. Now, let’s look at deployment description. Deployment description is same as above with additional volume information. We mention that we will use volume that is bound to claim  mysql-pv-claim (Line: 59 – 60) and mount it to the path ‘ /var/lib/mysql’ (Line: 56). Once MySQL container is launched vSphere Persistent Volume will be attached to the node on which it is launched and MySQL will write its data to the vmdk.

 

Run the following command to deploy MySQL,

Let’s validate whether MySQL is deployed,

Deploying WordPress

We will use following yaml for deploying WordPress.

# wordpress-deployment-vsphere.yaml 1.     apiVersion: v1 2.     kind: Service 3.     metadata: 4.       name: wordpress 5.       labels: 6.         app: wordpress 7.     spec: 8.       ports: 9.         – port: 80 10.        nodePort: 30080 11.    selector: 12.      app: wordpress 13.      tier: frontend 14.    type: NodePort 15.  — 16.  apiVersion: v1 17.  kind: PersistentVolumeClaim 18.  metadata: 19.    name: wp-pv-claim 20.    annotations: 21.      volume.beta.kubernetes.io/storage-class: fast 22.    labels: 23.      app: wordpress 24.  spec: 25.    accessModes: 26.      – ReadWriteOnce 27.    resources: 28.      requests: 29.        storage: 20Gi 30.  — 31.  apiVersion: extensions/v1beta1 32.  kind: Deployment 33.  metadata: 34.    name: wordpress 35.    labels: 36.      app: wordpress 37.  spec: 38.    strategy: 39.      type: Recreate 40.    template: 41.      metadata: 42.        labels: 43.          app: wordpress 44.          tier: frontend 45.      spec: 46.        containers: 47.        – image: wordpress:4.6.1-apache 48.          name: wordpress 49.          env: 50.          – name: WORDPRESS_DB_HOST 51.            value: wordpress-mysql 52.          – name: WORDPRESS_DB_PASSWORD 53.            value: mysqlpassword 54.          ports: 55.          – containerPort: 80 56.            name: wordpress 57.          volumeMounts: 58.          – name: wordpress-persistent-storage 59.            mountPath: /var/www/html 60.        volumes: 61.        – name: wordpress-persistent-storage 62.          persistentVolumeClaim: 63.            claimName: wp-pv-claim

 

Similarly, this yaml creates service, persistent volume claim and deployment objects in Kubernetes for WordPress.

 

Run following command to deploy WordPress,

Let’s validate whether WordPress is deployed,

Now we have WordPress and MySQL up and running let’s visit our new WordPress blog.

Enter following command to get the ip address and port to access the blog,

Just enter this ‘10.160.241.61:30080’ in the browser to visit the WordPress blog.

Once WordPress is configured. Let’s take down the WordPress pod.

Visit the same WordPress URL and see WordPress app is accessible with the same configuration.

Now let’s take down MySQL pod.

Another MySQL pod will be scheduled on available node.

Once pod is restarted by Kubernetes. The pod will pick up the same volumes and its state is intact.

 

As you can see, we can easily persist the state of containers using vSphere Cloud Provider!

 

We would love to hear your feedback! We will be at DockerCon. Please drop by at booth G9 to learn more about what we have to offer.

 

• Contact us at containers@vmware.com or kubernetes channel on VMware Code Slack
• Please visit vSphere documentation to learn about the latest features.
• Please check out vSphere Docker Volume Service that enables persistent storage for Docker Containers in vSphere environment.