VMware Cloud on AWS vSAN

Introducing Elastic vSAN

We have officially entered the next generation of storage in the VMware Cloud on AWS Service, dramatically increasing the size and scale of workload that the service can support, while simultaneously introducing flexibility into the amount of storage assigned to a cluster. We call it Elastic vSAN; a fusion of VMware vSAN and Amazons Elastic Block Store, but I’m getting ahead of myself. Before we dive into what we built, a little context.

The VMware Cloud on AWS service has been in the market for over a year and has emerged as workhorse amongst enterprise customers. The ability to rapidly spin up a production ready vSphere environment around the world has proven to be of incredible value. Our initial offering based on the I3.Metal EC2 instance has resonated with customers, so much so, that many have asked for significantly more storage and the ability to directly run larger datasets inside the service.

 

Supporting Petabyte-scale in the Cloud

When presented with the problem of dramatically increasing the amount of storage in a VMware Cloud on AWS Cluster the team took a holistic look at the issue. Some assumed we could simply ask Amazon for a new host with more internal capacity. Unfortunately, this wouldn’t have been tenable. In truth, the I3 nodes are approaching the limit for supportable capacity given current Media and Network capabilities. AWS can be a volatile environment and success requires an agile infrastructure. Our current cluster patching and host failure remediation process are network intensive. Dramatically increasing the capacity per node would require we also increase the network capacity, something that is not currently feasible. This reality caused the team to ask a different question, what if we didn’t use internal storage?

 

A Cloudy Alternative

Like so many instances over the previous year, we found the perfect solution to this problem in our Amazon Partnership.  Amazons Elastic Block Store (EBS). EBS is a unique capability within Amazon, which presents a pool of durable cloud storage as a local block device. EBS storage is allocated and connected to an EC2 Instance programmatically. This platform provided the perfect building block to construct a new class of vSAN Storage.

Elastic vSAN

 

Elastic vSAN

Using a new diskless host type Elastic vSAN enables customers to specify at cluster creation how much storage they require on a per-host basis. TheVMware Cloud on AWS Service will then dynamically build the hosts from the component services inside AWS. This flexibility empowers customers to control costs by deploying fewer total hosts where the capacity requirements outstep the compute and memory needs.

Elastic vSAN

 

Faster Recovery

Which brings us back to the beginning. If the i3.Metal instances are already near the supportable limit for capacity due to the time needed to evacuate a fully loaded host, won’t it take even longer to perform any maintenance with an Elastic vSAN host? The answer is that no it will not. Elastic vSAN will take a tiny fraction of that time. The reduced recovery time is possible because the auto-remediation services inside VMware Cloud on AWS have been optimized to minimize resync traffic by taking advantage of the enhanced availability of the EBS media. When a problem is encountered a replacement host is added into the cluster. However, instead of moving the data between the host we instead move the EBS volumes themselves. The new host is reattached the previous EBS volumes thus “moving” all the disk groups and data contained therein.

This workflow has effectively solved the data gravity challenge inside VMware Cloud on AWS. More importantly, it provides an architecture to deliver future capabilities via software; composing the underlying hardware as required based on current capabilities. If you’ve previously taken a look at VMware Cloud on AWS, it’s time to take a fresh look and reevaluate. For customers who require even more, I encourage you to watch this space. We are just getting started, and hope to prove how advantageous this architecture is as we continue to improve the VMware Cloud on AWS Service!

Availability

To view the latest status of features for VMware Cloud on AWS, visit https://cloud.vmware.com/vmc-aws/roadmap.

Resources:

 

Glenn Sizemore

Glenn Sizemore is a Senior Technical Marketing Architect in the Cloud Platform business unit at VMware. Currently, he is responsible for all things Storage in the VMware Cloud on AWS…

Related Articles