Virtual Volumes (vVols)

Virtual Volumes (vVols) now supports WSFC

One of the announcements in the vSphere 6.7 launch that caught my attention was vVols support for WSFC (Windows Server Failover Clustering) using SCSI-3 Persistent Reservations. This is great news for customers still working with RDMs. WSFC clustering solutions use SCSI-3 Persistent Reservations which enables access for multiple nodes to a device (vVol disk) and simultaneously blocks access for other nodes.

What are SCSI reservations?

SCSI reservations allow multiple nodes to access a shared disk at a storage level while restricting simultaneous access to that shared disk from other nodes to avoid data corruption. By allowing multiple nodes access to an individual disk, this allows Microsoft WSFC to manage the locking along with the storage. This is called registration and reservation, which allows one node to eject another node. Other features of SCSI-3 include persistent reservations across reboots and supporting multi-pathing to disks.

So long RDMs

WSFC is one of the very last use cases for Physical Mode Raw Device Mappings (pRDMs). Today, RDMs are required to allow SCSI reservations to be passed directly to the quorum/witness device without being handled/interpreted by the VMkernel I/O stack.

Up until vSphere 6.7 there was only one scenario where you couldn’t use vVols for SQL Server, and that was shared storage between SQL Server instances. Now, vVols support in ESXI 6.7 enables up to 5 WSFC cluster nodes to access the same shared vVol disk.

Setting up WSFC using vVols

The process for setting up WSFC on vVols is pretty straight forward.   Follow the Guidelines for setting up WSFC on vSphere 6.x keeping a few vVols specific guidelines in mind:

  • Be sure to use HW Version 13 or greater on your FCI VMs.
  • ESXi 6.7 supports vVols storage with up to 5 node WSFC.
  • The storage array must support SCSI persistent operations at the subsidiary LUN level.
  • ESXi 6.7 supports vVols Storage for Windows Server 2008 SP2 and above releases.
  • All hosts must be running ESXi 6.7 or above.
  • WSFC on vVols can work with any type of disk, “Thin” as well as “Thick”-provisioned disks.
  • The underlying transport protocol can be FC, ISCSI or FCOE.
  • Only Cluster-across-box (CAB) is supported.
  • Cluster-in-a-box (CIB) and a mixture of CAB and CIB is not supported.
  • N+1 cluster configuration, in which one ESXi host has virtual machines which are secondary nodes and one primary node is a physical box is not supported.
  • This feature enables customers to move away from using Pass-through RDM (physical compatibility mode)
  • WSFC on vVols supports HA, DRS and vMotion.

Here’s how it works

Add a SCSI Controller to the first node

  1. In the vSphere Client, select newly created virtual machine, right-click and select Edit Settings.
  2. Click the New device drop-down menu, select SCSI Controller, and click Add.
  3. Select the appropriate type of controller, depending on your operating system
  4. Set SCSI Bus Sharing to Physical and click OK

Add a new hard disk to the first node in the failover cluster 

  1. In the vSphere Client, select the newly created virtual machine, right-click and select Edit Settings.
  2. Click the New device drop-down menu, select New Hard Disk, and click Add.
  3. Select the disk size.
  4. Under Disk Provisioning, select either Thick or Thin Provision.
  5. Expand the New Hard Disk.
  6. From the Virtual Device Node drop-down menu, select the newly created SCSI controller (for example, select SCSI (1:0)) You must select a new virtual device node. You cannot use SCSI 0.
  7. Click OK. The wizard creates a new hard disk using the newly created SCSI Controller.

 

Note: Take note of the name of the DiskFile; you will need it for the next step.  You can find it by expanding the New Hard Disk and looking under Device File.

Add a SCSI Controller to each additional node

  1. In the vSphere Client, select newly created virtual machine, right-click and select Edit Settings.
  2. Click the New device drop-down menu, select SCSI Controller, and click Add.
  3. Select the appropriate type of controller, depending on your operating system
  4. Set SCSI Bus Sharing to Physical and click OK

Add existing hard disk to the additional nodes in the failover cluster

  1. In the vSphere Client, select the newly created virtual machine, right-click and select Edit Settings.
  2. Click the New device drop-down menu, select Existing Hard Disk, and click Add.
  3. In Disk File Path, browse to the location of the quorum disk specified for the first node. The name of the disk can be found by expanding the first node’s hard disk\Disk File.
  4. Select the same virtual device node you chose for the first virtual machine’s shared storage disks (for example, SCSI (1:0)), and click OK. The wizard creates a new hard disk.
  5. Click OK. The wizard adds the shared vVols disk using the newly created SCSI Controller.

That’s it. Pretty straight forward and all from within the vSphere client. And the best part of all…. no RDMs.

If you are experiencing timeouts when testing failover, be sure to check your HealthCheckTimeout setting.  The default timeout for the Failover Cluster Instance (FCI) to be considered unresponsive is 30 seconds.  You can increase this if needed by configuring the HealthCheckTimeout setting. Changes made to the timeout settings are effective immediately and do not require a restart of the SQL Server resource.

As you can see using vVols is much simpler than setting up RDMs and the shared vVols also supports HA, DRS and vMotion. If you already have RDMs and you want to get rid of them, be sure to read Cody Hosterman’s blog post on Migrating RDMs to vVols.  Now it’s your turn.  Give it a try and let me know how it goes. I’m interested in your feedback on this post as well as the new support for WSFC on vVols.

@vPedroArrow

 

Comments

14 comments have been added so far

  1. Thanks for this Article – I wasn’t aware! Do you know will VCS Clustering be supported with this configuration? Furthermore, will this work with replicated diks. Will this also allow you to replicate that vvol disk?

  2. This is part of vSphere 6.7 so any vendor that is certified to support 6.7 will work. As for replicating, yes, so long as the vendor has array-based replication built into their VVol VASA Provider.

  3. Note that sharing virtual disks backed by VVols as described above has always been supported – the work in 6.7 to enable MSCS/WSFC was primarily in the core storage stack to permit multiple SCSI-3 Persistent Reservation commands issued to a single PE on behalf of (potentially) multiple VVols to be tracked properly.

    I believe VCS Clustering actually relies on an L2 ethernet-based protocol (LLT, Low Latency Transport) as the foundation for its GAB ( Global Membership services and Atomic Broadcast Protocol ) and HAD (High Availability Daemon) services. If it doesn’t require SCSI-3 Persistent Reservations and instead, like Oracle RAC, requires only shared (virtual) hard disks it should work with VVols on vSphere 6.5 and even vSphere 6.0 as well but I don’t personally know if anyone’s tried it yet.

  4. Great post and i have been looking for this feature. Can i deploy SQL 2016 clustering on 2 * VM WSFC with vVols? is this combination supported by VMware?

    Here is my requirement,

    1. VVols + vSphere 6.7 for deploying 2 * VM WSFC + SQL clustering
    2. VVols + vSphere 6.7 for deploying 2 * VM Oracle RAC setup

  5. Kannan,

    Yes, VVols supports WSFC and Oracle RAC on vSphere 6.7. Be sure to check with your storage vendor to verify their VASA Provider is certified for vSphere 6.7.

    Pete

  6. Thanks Pete. are you aware of any recommendation to consider when we deploy SQL cluster + WSFC + vVols ?

  7. I have this working for a POC and it’s working great. Thin provisioned disks on vVol with 2 node MSCS working (for a high available file share server).
    Can you give some details on how to expand the cluster disk without having to take down the cluster please? I so far haven’t been able to figure this out.
    Can you also explain the limitations around CIB? (i.e. what would happen if the 2 VMs ended up on the same host?)

  8. This is all amazing information. What about snapshots? I have found that snapshots do not appear to work. This is HUGELY beneficial if I can use backup software that supports vVols to also backup the machines using snapshots instead of an in-guest solution.

  9. With the implementation of virtual Volumes for Failover Clustering, Is it possible to now utilise agent-less backups / snapshots to backup the Windows Server cluster nodes (e.g. Veeam Backup and Recovery 9.5) instead of agent based backups installed directly on the cluster nodes?

Leave a Reply

Your email address will not be published.