Let's start with a brief introduction. In vSphere 5.0, VMware releases a new storage appliance called the VSA. VSA is an acronym for “vSphere Storage Appliance”. This appliance is aimed at our SMB (Small and Mid-size Business) customers who may not be in a position to purchase a physical SAN or NAS array for their virtual infrastructure, and as a result, these customers do not have shared storage.
Without access to shared storage, SMB customers are unable to implement many of vSphere’s core technologies, such as vSphere HA & vMotion. Customers who decide to deploy a VSA can now benefit from many additional vSphere features without having to purchase a SAN or NAS device to provide them with shared storage.
Customers require two or three newly installed (green-field) ESXi 5.0 hosts to create a VSA cluster. The ESXi hosts must not have any virtual machines deployed & have the default logical network configuration (Management portgroup & VM portgroup). The local storage on the ESXi host should be configured into a RAID10 setup for optimal resilience. Should the server lose a disk spindle, it will not impact the VSA cluster storage. The recommendation is to use 8 local disks per ESXi; this will give optimal performance.
During the ESXi 5.0 deployment, a VMFS-5 is created on all the remaining available disk space on the ESXi servers.
Each ESXi 5.0 host will have an appliance/Virtual Machine deployed to it by the VSA cluster installer. The appliances are deployed with multiple disks (VMDKs) which allows them to use (almost) all of the VMFS-5. The appliances then present one replicated volume via NFS to each of the ESXi 5.0 hosts in the cluster. This replication of storage volumes makes the VSA very resilient to failures. All ESXi 5.0 hosts now have shared storage.
VSA Manager is the component used to manage the VSA cluster. VSA Manager is a vCenter Server 5.0 extension that you install on a vCenter Server machine. After you install it & the VSA Manager plugin is enabled, you can see the VSA Manager tab in the vSphere Client. VSA Manager will deploy & afterwards monitor the VSA cluster. Here is a screen-shot of the VSA Manager from a 3 node configuration:
The VSA manages the data replication/redundancy by dividing the local storage on the appliance into two distinct volumes, one volume becomes the replica/mirror source and the other becomes a replica/mirror destination (for a source replica on a different appliance). It then exposes the mirrored volume as an NFS volume over the network, which allows it to be mounted by all the ESXi hosts in the VSA cluster. vSphere HA & a vMotion network are also configured for all ESXi hosts in the cluster.
This is all done automatically by the installer. There is no customer intervention required to do this.
There is an obvious CAPEX saving achieved by SMB customers as there is no longer a need to purchase a dedicated SAN or NAS devices to achieve shared storage.
There is also an OPEX saving as the installation & management of the VSA may be done by the vSphere Administrator who may not have the necessary skills to manage a physical SAN or NAS array.
The installation & configuration is also much simpler than that of a physical storage array or other storage appliances.
A new vSphere Storage Appliance Technical Whitepaper is published here.
Future posts will take a closer look at the performance and resilience features of the VSA.
So, you will lose a lot of disks by using raid 5, and per volume u use 2 same replica’s which will also lose a lot of storage?
Hi Marco, yes, because of the resilience/redundancy features that we have built into the VSA, the RAID10 (not RAID5) configuration of the physical storage will indeed impact the amount of ‘business storage’ that will be made available to VMs. And the NFS replication (which is RAID1) will also have an impact.
This VSA is a good step in the right direction for VMware, but does VMware intend to address some of the concerns with flexibility.
IE, limitation to 25% storage utilization, maximum of 3 servers, 1 VSA per vCentre, adding external storage.
I find the speed of deployment very interesting but the limitations of the VSA aren’t doing it justice,
Hi Steven, this is VSA 1.0 – give us a chance
Seriously though, our primary concern is making sure everyone has a good experience with the product so we are including some tight guidelines in this release (as you’ve alluded to in the post). And yes, we are looking at lifting some of these as we go forward but no guarantees at this time. Hopefully there are still enough cool features in the VSA which will make it attractive enough for our SMB customers to at least evaluate it. I would.
I love the idea of the VSA for clusters WITH shared storage; this could help “capture” lost local storage that was originally intended for ESX implementations–i.e., a place for the service console to live, plus its swapfile, etc.–but is now unused because “traditional” shared storage is available.
Guys, thanks for getting this to us for FREE! I’m looking forward to using it as soon as I can get vSphere 5 and ESXi 5 implemented here. I think Steven’s points are valid (I would like to use it on more than 3 hosts) but this is a really cool NEW technology and I’m glad you let us use it (again for FREE).
Like to see the ability to deploy a VSA on exsisting ESX hosts using only SSDs to define a “SSD datastore”
Hoped it would work like sheepdog, http://www.itq.nl/blogs/post/Sheepdog-Project.aspx
Linear storage, easy to add new nodes
Hi Dan,
Thanks for the comment. In the 1.0 release of the VSA, the ESXi 5.0 servers must be ‘green-field’, i.e. newly installed with default settings.
Hi Marco,
When deploying the VSA 1.0, you will need to determine the number of nodes and the amount of storage you requirement in advance. There is no way to expand after deployment in this release.
Based on the comment by Chogan; There’s no scalability in VSA 1.0?
The concept overall is great for SMB but the lack of being able to expand sounds like we’re forcing SMB customers to guess at what they will need and then pay-ahead for that guess. After a few years, once they actually end up growing beyond the guess they’ve made, they will most likely be at a financial point where the negative customer experience and their added choice flexibility would drive them away from both VSA and whoever sold them the solution. Are there plans to allow for scalability in the next release? Also, will the customers of 1.0 be capable of upgrading to a release that has these added features without having to go back to bare metal?
Hi Bryan,
Unfortunately, in 1.0, we will require the customer to map out their requirements before deployment. We understand that this is not ideal, and we are looking at a scalable solution in future versions. At this time, its not possible to say when this feature will be added, nor can we comment on how the upgrade mechanism will work in future versions of the VSA. Once I know, I’ll be sure to post about it.
I have an existing vSphere Essentials+ deployment with three active servers and a dozen plus VMs. I would very much like to upgrade my ESXi hosts and transition our VMs off our low end SATA based NAS/SAN device onto high performance enterprise grade internal SAS storage / servers. Given that my Foundation Class vCenter license only allows me to manage three ESXi hosts, it seem to me that I’m going to have to take down my entire existing cluster, disconnect my existing ESXi hosts from vCenter, set up the VSA, etc. on the new servers, and then manually migrate my VM images off the old storage onto the VSA, and bring them up there. There’s no real way around this, right? I can’t leave one of the old hosts connected, and set up just two of the new ones, and then add the third new ESXi/VSA host later, right?
I know this is an old thread but you should remember that trial licenses can greatly help/improve major upgrades – 60 days is a long time.
Hi Thomas,
Unfortunately not. The hosts that participate in the VSA cluster must be installed with ESXi 5.0, and must have no running VMs. So if you want to reuse hosts that have running VMs, you will have to look at backing up/exporting the VMs and reimporting/restoring them after the VSA installation is completed.
There is also no way of adding additional hosts to the cluster in 1.0, so you must decide on a 2 node or 3 node cluster from the outset.
i’ve learned yesterday about VSA at a VMware presentation and found it very impressive.
however i have to ask a beginner’s question: if i’m assigning a data store to a virtual machine, can this volume then be formatted NTFS (from the OS in the virtual machine’s view)?
Hello Christian,
The VSA presents NFS datastores to ESXi servers, not to Virtual Machines. You then deploy your VMs on these NFS datastores. The Guest OS inside in the VM is unaware of the underlying storage, and simply see a local disk. You can create whatever filesystem is supported by the Guest OS inside in the VM. So, to answer your question, yes, you can create NTFS.
I have the following scenario :
2 ESXi, with VSA.
What is happening if all network links between the ESXi’s goes down ?
Both ESXi’s will run/start the VM machines independently from local VSA copy ?
What is happening when network link comes back online ? how does it sync ?
Hi Alex,
This is going to be one of those ‘it depends’ type answers. Which network is down? Front-End or Back-End? Can any node reach vCenter? We recommend redundancy in both NIC & physical switches for the VSA. However, It would still be interesting to document the behaviour so it probably warrants a blog article documenting the various scenarios. Watch this space.
Cormac
Hi Cormac,
Let’s assume both front-end and back-end networks are down and none of the ESXi node can reach vCenter.
Also another scenario could be with only vCenter down. Indeed, it’a a lot of various scenario and it will be great to cover this subject !
Alex
Well, if nodes can’t reach any other nodes or vCenter, then you’ll have a complete cluster outage, meaning no more shared storage. I’ll look into the other scenarios, but it’ll take a few days.
First blog on VSA network outages now posted – http://blogs.vmware.com/vsphere/2011/09/vsphere-storage-appliance-vsa-network-outage-scenario-1-back-end.html
Regarding the comment above with communication lost between the hosts and or the vCenter, when you say “complete cluster outage” and the shared storage is lost, can I assume when the communication is re-established the shared storage would be re-established?
I’m thinking vCenter maintenance
Hi KZ,
First thing to clarify is that a vCenter server outage will not affect the VSA. We only use the vCenter for management in a 3 node cluster, and for management and tie-breaker code in a 2 node cluster.
What I mean about a complete cluster outage is that when all nodes in a cluster are brought down – complete power outage at site would be an example.
And yes indeed, when the cluster comes back up, it will synchronize all of its mirrors automatically, and once sync’ed will present back the NFS datastores to the ESXi hosts.
Was told that the VSA NFS datastores can support up to estimated 25 (2 nodes) to 35 VMs (3 nodes). Besides resource limitations, can you explain why this is the case? I have a customer that wanted to deploy about 75 vm’s and if that is the case we might not be able to use this solution. Thank you appreciate your feedback
Hi Charlie,
There is no hard and fast limit to how many VMs can run on the VSA per se, but there are two consideration in particular to keep in mind.
The first is to measure the combined I/O throughput and latency expectations of the 75 VMs, and verifying that the VSA can cope. There is a WP which discusses the performance one can expect from the VSA; it should help with this:
http://www.vmware.com/files/pdf/techpaper/vsa-perf-vsphere5.pdf
The second is to ensure that you do not over commit memory on any of the ESX hosts. Since VSA participates in a vSphere HA cluster, you must also ensure that there is no memory overcomit when an ESXi host fails, and in a 3 node cluster where all the VMs move over to the remaining 2 nodes, you’ll have to ensure that all 75 VMs can power up on the remaining 2 nodes without overcommitting memory.
Hopefully this will help in your decision making process.
Cormac
Thank you Cormac for the information and for such a fast response, really appreciate it.
You mention the hosts should be in a RAID 10…but how about RAID 1 or even 0? I can’t see how ESX would care, but just want to make sure. I’m working with 2x 1U servers so I only have 2x physical disks per box. :-/
You are correct – the ESXi hosts will not care. The point of recommending a RAID level is that we do not want a single spindle failure to bring down one of the nodes in the VSA cluster. In your case, with only 2 physical disks per host, I would simply mirror them.
Hello Cormac,
the release notes shows that performance of the devices SAS/SATA is critical. Is there a way to use iodrive2 (Fursion) instead of using RAID controller+disks as storage in the VSA? Fusion makes use of Flashback which is more reliant then RAID 5 on disks. For sure the needed server is on the hcl.
Not at this time Thomas. There were some tentative discussions with FusionIO, but was not tested or certified as a solution.
VSA is great. 12x1TB boils down to 1.7TB of usable storage but it’s worth it. It’s a resilient platform. I lost my vCenter Server with the VSA manager after a disastrous SQL update. The individual ESXi hosts in the VSA cluster and their VMs continued to function. It took a couple of minutes to fire up a new vCenter Server from a pre-prepared template and then using the inbuilt VSA Recovery to the new vCenter Server worked like a charm. I think Cormac covers VSA Recovery in another blog post.