When vSphere 5.0 launched last year, I did a post on the new VAAI (VMware Storage APIs for Array Integration) primitives. One of the cool new features was the the ability to reclaim stranded space on Thin Provisioned datastores, something we could not do previously, which meant large amounts of stranded space could be left on these datastores. This block reclaim primitive involved sending a SCSI UNMAP command to the array when we deleted or migrated a file from a TP datastore. Unfortunately, some performance issues were experienced with this new primitive, and the time taken to reclaim the stranded space by certain storage arrays led to some operations like Storage vMotion to time out. In the end, we had to turn off the primitive in 5.0 patch 2.
Well, the good news is that in vSphere 5.0U1, we are now reinstating the VAAI Block Reclaim/UNMAP primitive. Unfortunately, it will not be implemented as an automated process, but will be a manual process for the time being. ESXi 5.0 U1 contains a 'vmkfstools' command with a revised '-y' option so you can now once again reclaim stranded space on Thin Provisioned datastores. We do expect customers to use this primitive during their maintenance window, since running it on a datastore that is in-use by a VM can adversely affect I/O for the VM. I/O can take longer to complete, resulting in lower I/O throughput and higher I/O latency. There is no way of knowing how long an UNMAP operation will take to complete. It can be anywhere from few minutes to couple of hours depending on the size of the datastore, the amount of content that needs to be reclaimed and how well the storage array can handle the UNMAP operation.
The numeric parameter passed after -y specifies a percentage of free space to reclaim on the volume when UNMAP is issued. Some calculation is required to correctly determine what this % value should be set to, and I would urge you to read the release notes for ESXi 5.0U1 and the VAAI UNMAP KB article.
Of course, the desired objective is to have this working automatically like we had in the initial 5.0 release, and we continue to work towards this goal. We do not have an eta for this as yet however.
Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @VMwareStorage
Yay – space reclamation is in business again! Are you sharing information on stats to track the number of UNMAPs issued, UNMAP failures, etc?
Hi Satyam,
Sounds like that would make a good blog post. Watch this space.
Cormac
A Check box during XMotion will be great….
Our long term goal is definitely to get this to happen automatically, but perhaps adding a check-box during a storage vMotion/Migration/Delete operation could be a possibility in the shorter term.
I’ll take this back to our engineering folks for consideration – thanks for the suggestion.
Hi Cormac – Any update on when the UNMAP feature will be re-intruduced and automated when running storage vMotion?
Thanks, Gareth.
Hey Gareth,
We are working on it. Hopefully I’ll have some details in and around the VMworld timeframe.
Hi,
Possibly a bit of a dumb question here but would this have any effect on Change Block Tracking?
Cheers
Greg
Hi Greg,
Nope – no impact. This feature/primitive is all about the ESXi host telling the array that these blocks, which I was using previously but I am no longer using, can be placed back on your free list (reclaimed).
CBT is all about tracking which blocks have changed within a VMDK during a particular epoch.
A scenario where they might co-exist is a Storage vMotion in 5.0. CBT is used to recursively keep track of blocks changing in a VM during the migration. Eventually the number of blocks which have changed during one of the recursive copies should be small enough in number to allow us to switch over to the VM on the destination. UNMAP can then be used to tell the array that the block which the VM occupied on the source datastore may now be reclaimed.
Cormac
Hi Cormac
Thanks for this article!!
Any news on when the automated VAAI Block Reclaim/UNMAP primitive will be back or are there any changes in 5.1?
Thank you very much!!
Fabio.
No changes in 5.1. It is still manual. As soon as I hear of any changes, I’ll be sure to inform the community.
wonderful post, very informative. I’m wondering why the other experts of this sector do not understand this. You should continue your writing. I am sure, you’ve a
huge readers’ base already!
Does any body know if there has been a change to this ?
Nope – no changes recently. It still requires vmkfstools -y.
Can you give us an example of some calculations used to determine the % to use for a 3PAR SAN?