Last month, EMC became the first VMware storage partner to release a VAAI NAS plugin. VAAI, the vSphere Storage APIs for Array Integration, allows the ESXi to offload certain storage tasks to the storage array, freeing up the VMkernel to do other tasks. We had this functionality for block devices since vSphere 4.1. However support for NAS devices only came in vSphere 5.0, and unlike block, this feature required partners to create a plugin to use the feature.
Here is the EMC NAS Plugin (in the form of a VIB):
~ # esxcli software vib list Name Version Vendor Acceptance Level Install Date ——————– ———————————- —— —————- ———— EMCNasPlugin 1.0-10 EMC VMwareAccepted 2011-06-26
~ # esxcli software vib get -n EMCNasPlugin EMC_bootbank_EMCNasPlugin_1.0-10 Name: EMCNasPlugin Version: 1.0-10 Type: bootbank Vendor: EMC Acceptance Level: VMwareAccepted Summary: EMC NAS VAAI Description: EMC NAS VAAI Module for ESX Server Release Date: 2011-09-15 Depends: Conflicts: Replaces: Provides: Maintenance Mode Required: False Hardware Platforms Required: Live Install Allowed: False Live Remove Allowed: False Stateless Ready: False Overlay: False Tags: Payloads: EMCNasPlugin
I have two NFS datastores mounted from my VNX-5500. Let's see if Hardware Acceleration is supported. Hardware Acceleration is how we refer to VAAI offload primitives in the UI:
One of the new VAAI-NAS features is the ability to now pre-allocate space for VMs on NAS datastores. Up until now, admins could only ever create thin VM disks on NAS datastores. With VAAI NAS, you can now create Lazy Zero, Eager Zero or Thin VMDKs on NAS, as shown below:
Another feature of VAAI NAS is the ability to offload cloning operations to the array. Now it should be pointed out that this is only available with cold migrations (VM powered off); this feature cannot be used by Storage vMotion operations. We did a simple test in the lab with our VNX-5500 and did a clone of an 8GB VMDK. Here are the results:
Test 1: Clone of an 8GB vmdk without VAAI NAS plugin:
~ # date; vmkfstools -i Fc-w2k3-TH-2.vmdk /vmfs/volumes/NFS-test1/test/test.vmdk; date Mon Jun 27 17:56:58 UTC 2011 Destination disk format: Thin Cloning disk 'Fc-w2k3-TH-2.vmdk'… Clone: 100% done. Mon Jun 27 17:58:58 UTC 2011 /vmfs/volumes/8f9cf737-de97165b/Fc-w2k3-TH-2 #
Duration: 120 seconds
Test 2: Clone of an 8GB vmdk with VAAI NAS plugin:
~ # date; vmkfstools -i Fc-w2k3-TH-2.vmdk /vmfs/volumes/NFS-test1/test/test.vmdk; date Mon Jun 27 18:08:52 UTC 2011 Destination disk format: Thin Cloning disk 'Fc-w2k3-TH-2.vmdk'… Clone: 100% done. Mon Jun 27 18:10:32 UTC 2011 /vmfs/volumes/8f9cf737-de97165b/Fc-w2k3-TH-2 #
Duration: 100 seconds
So we can clearly see that the time to do the clone is quicker when we offload it to the array. Of course the biggest gain here is in the fact that the transfer is offloaded to the array and is not hogging any of the network stack in the VMkernel.
There is another feature in VAAI NAS that we have not mentioned, and this is the ability for VMware View desktops built on linked clones to be offloaded to the array. In this case, linked clones will be based on native array snapshots rather than VMware redo log files on the datastore. Unfortuantely, to test this feature, we need a version of VMware View which can make these API calls to offload to the array. We do not have a version of View which does this yet, but when we do, I will be sure to try it with you and share the results.
One final point to note about VAAI NAS. The offloads performed by VAAI NAS are not visible in esxtop. This is different to the VAAI offload primitives for block, where the primitives (and any failures) are readily visible in esxtop. So how can one observe calls to the VAAI NAS primtives if they are not in esxtop? Well, one way is to use the -v option when using vmkfstools. This will give you various details about calls made by the VAAI NAS plugin, in this case EMCNasPlugin, and is a good way to start troubleshooting:
~ # vmkfstools -i Fc-w2k3-TH-2.vmdk /vmfs/volumes/NFS-test1/test/test.vmdk -v verbose EMCNasPlugin: session request for 10.21.73.225 /NFS-test2 /vmfs/volumes/8f9cf737-de97165b NFS 1 EMCNasPlugin: mounted /NFS-test2 on host 10.21.73.225 DISKLIB-VMFS : "./Fc-w2k3-TH-2-flat.vmdk" : open successful (285) size = 8589934592, hd = 0. Type 3 DISKLIB-VMFS : "./Fc-w2k3-TH-2-flat.vmdk" : closed. EMCNasPlugin: session request for 10.21.73.225 /NFS-test2 /vmfs/volumes/8f9cf737-de97165b NFS 1 EMCNasPlugin: mounted /NFS-test2 on host 10.21.73.225 DISKLIB-VMFS : "./Fc-w2k3-TH-2-flat.vmdk" : open successful (14) size = 8589934592, hd = 29426731. Type 3 DISKLIB-DSCPTR: Opened [0]: "Fc-w2k3-TH-2-flat.vmdk" (0xe) DISKLIB-LINK : Opened 'Fc-w2k3-TH-2.vmdk' (0xe): vmfs, 16777216 sectors / 8 GB. DISKLIB-LIB : Opened "Fc-w2k3-TH-2.vmdk" (flags 0xe, type vmfs). Destination disk format: Thin Cloning disk 'Fc-w2k3-TH-2.vmdk'… DISKLIB-LIB : CREATE: "/vmfs/volumes/NFS-test1/test/test.vmdk" — vmfs capacity=0 (0 bytes) adapter=buslogic cowGran=0 DISKLIB-LIB : CREATE: "/vmfs/volumes/NFS-test1/test/test.vmdk" — vmfs capacity=16777216 (8 GB) adapter=buslogic cowGran=1 Clone: 9% done.DISKLIB-DSCPTR: "/vmfs/volumes/NFS-test1/test/test.vmdk" : creation successful. EMCNasPlugin: session request for 10.21.73.225 /NFS-test1 /vmfs/volumes/4978db5f-76fb8163 NFS 1 EMCNasPlugin: mounted /NFS-test1 on host 10.21.73.225 DISKLIB-VMFS : "/vmfs/volumes/NFS-test1/test/test-flat.vmdk" : open successful (17) size = 24576, hd = 0. Type 3 DISKLIB-VMFS : "/vmfs/volumes/NFS-test1/test/test-flat.vmdk" : closed. EMCNasPlugin: session request for 10.21.73.225 /NFS-test1 /vmfs/volumes/4978db5f-76fb8163 NFS 1 EMCNasPlugin: mounted /NFS-test1 on host 10.21.73.225 DISKLIB-VMFS : "/vmfs/volumes/NFS-test1/test/test-flat.vmdk" : open successful (536) size = 8589934592, hd = 7537712. Type 3 DISKLIB-DSCPTR: Opened [0]: "test-flat.vmdk" (0x218) DISKLIB-LINK : Opened '/vmfs/volumes/NFS-test1/test/test.vmdk' (0x218): vmfs, 16777216 sectors / 8 GB. DISKLIB-LIB : Opened "/vmfs/volumes/NFS-test1/test/test.vmdk" (flags 0x218, type vmfs). DISKLIB-DDB : "adapterType" = "lsilogic" (was "buslogic") DISKLIB-DDB : "longContentID" = "d8eadc90e5cdc05089e81babe7102fec" (was "6a787fcb07da9fd38a19a11afffffffe") EMCNasPlugin: session request for 10.21.73.225 /NFS-test1 /vmfs/volumes/4978db5f-76fb8163 NFS 1 EMCNasPlugin: mounted /NFS-test1 on host 10.21.73.225 EMCNasPlugin: session request for 10.21.73.225 /NFS-test2 /vmfs/volumes/8f9cf737-de97165b NFS 1 EMCNasPlugin: mounted /NFS-test2 on host 10.21.73.225 EMCNasPlugin: request to clone /vmfs/volumes/8f9cf737-de97165b/Fc-w2k3-TH-2/Fc-w2k3-TH-2-flat.vmdk to /vmfs/volumes/4978db5f-76fb8163/test/test-flat.vmdk flags 14 EMCNasPlugin: Vstorage task is in-progress… Clone: 12% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 14% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 16% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 20% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 23% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 27% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 30% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 33% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 36% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 39% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 43% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 46% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 49% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 51% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 54% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 57% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 59% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 62% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 64% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 67% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 70% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 73% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 75% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 78% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 80% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 83% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 86% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 89% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 92% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 95% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 97% done.EMCNasPlugin: Vstorage task is in-progress… Clone: 99% done.EMCNasPlugin: Vstorage task completed successfully Clone: 100% done.VAAI-NAS [EMCNasPlugin : /vmfs/volumes/4978db5f-76fb8163] : CLONE [/vmfs/volumes/4978db5f-76fb8163/test/test-flat.vmdk] succeeded. DISKLIB-LINK&#
0160; : DiskLinkNativeVmfsCloneExisting: Natively cloned from './Fc-w2k3-TH-2-flat.vmdk' to '/vmfs/volumes/NFS-test1/test/test-flat.vmdk' successfully. DISKLIB-LINK : DiskLinkNativeVmfsCloneExisting: Native clone created successfully. DISKLIB-LIB : Native clone on destination handle created successfully. DISKLIB-VMFS : "/vmfs/volumes/NFS-test1/test/test-flat.vmdk" : closed. DISKLIB-VMFS : "./Fc-w2k3-TH-2-flat.vmdk" : closed.
AIOMGR-U : stat o=1 r=0 w=0 i=1 br=0 bw=0 AIOMGR-S : stat o=6 r=12 w=0 i=2 br=196608 bw=0
Take a look at Chad's blog to find out more about the EMC VAAI NAS plugin for the VNX, and where to download it from.
I know a number of our other storage partners are actively working on their specific NAS plugins. I look forward to seeing a few more releases coming out in 2012.
Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @VMwareStorage
