One of the hottest topics I get into when talking to customers about virtualizing Exchange is storage. Not surprising considering the number of options available when we virtualize Exchange on vSphere. If you are not familiar with the common methods for provisioning storage in vSphere a brief description of each follows:
-
VMFS based virtual disk (VMDK) – VMFS is a high performance, clustered file system that allows concurrent access by multiple hosts to files on a shared volume. VMFS offers high I/O capabilities for virtual machines and is optimized for large VMDK files. VMFS volumes can be Fibre Channel or iSCSI attached.
- Raw-device mappings (RDM) – RDM is a mapping file in a VMFS volume that acts as a proxy for a raw physical device, sometimes called a pass-thru disk. The RDM file contains metadata used to manage and redirect disk access to the physical device. RDMs can be Fibre Channel or iSCSI attached.
In early versions of ESX the virtualization overhead associated with deploying virtual disks (VMDK files) was much higher than it is today and why it was considered a best practice to place Exchange data files on physical mode raw-device mappings (RDM). As ESX and vSphere have evolved the performance difference between RDMs and virtual disks has become almost nonexistent. This leaves some questioning why we might choose to deploy RDMs for Exchange storage.
Some reasons for deploying RDMs today might include:
-
Backups are being performed using a hardware based VSS solution using array based clones or snapshots – When talking to customers I typically see backups as being the number one reason for deploying RDMs. The ability to take array based backups quickly using hardware VSS makes RDMs very attractive for large organizations with massive amounts of email data. So, if we want to take advantage of array based backups are we limited to only using RDMs? Not quite, but more on that in a minute.
-
Volumes larger than 2TB are required – With Microsoft supporting mailbox databases up to 2TB (when database resiliency is in use) volumes may need to be larger than 2TB. In vSphere 5 only physical mode RDMs support volume sizes up to 64TB, VMDK files are limited to 2TB.
-
Require the ability to swing a LUN between a native Windows host and a virtual machine – Some deployments may choose to deploy on physical mailbox servers and later migrate to virtual machines. This migration could be expedited by swinging the LUNs from the physical mailbox server and attaching them to the Exchange mailbox VM using RDMs. With database portability only the user objects would need to be updated thus avoiding the time to move mailbox data over the network.
- Management purposes – Some environments may require greater control over the relationship between LUNs and virtual machines. An RDM is assigned to a single VM (unless using a shared-disk cluster) guaranteeing that the I/O capabilities of the LUN are dedicated to a single VM.
The good news is, if you're not limited by any of the reasons above you can deploy on VMDKs with confidence. I tend to prefer VMDKs for the portability, manageability, and scalability. By portability I mean the ability to use features like Storage vMotion, Storage DRS, and vSphere Replication to provide storage load balancing and disaster recovery. Improved management comes with the native tools available in the vSphere client for working with VMDKs. Some storage vendors have very slick plug-ins for the vCenter client if you must use RDMs, but it's always nice using the native tools. From a scaling point of view larger VMFS volumes can be used to consolidate VMDKs if dedicated RDMs are pushing the 256 LUN limit in ESXi. vSphere 5 supports VMFS volumes of up to 64TB, VMDK files are limited to 2TB.
Now that we can make some better informed choices for our storage format let's get back to the backups. If you are looking to deploy a hardware based VSS backup solution it used to be that the only option was to use physical mode RDMs. Today some storage vendors have made progress in giving customers the ability to deploy on storage other than physical mode RDMs. This comes in the following forms:
-
In-guest iSCSI – Using iSCSI initiators from within the guest operating system an administrator can directly mount storage LUNs to the virtual machine. Connecting storage in this manner can still provide the ability to backup using array based snapshots and clones. This does put additional load on the virtual machine as it is now doing the storage processing, but will allow you to avoid using RDMs and can mitigate the 256 LUN limit of ESXi. At VMworld this year (both in the US and Europe) many customers shared their success stories of using in-guest iSCSI with Exchange.
- NFS based VMDKs – Some storage vendors have added the ability to perform hardware based VSS backups of VMDKs housed on NFS based networked-attached storage. I've also had many customers tell me of their success using this solution. My only comment here is that Microsoft has been pretty clear on their lack of support for housing Exchange data files (mailbox and queue databases and transaction logs) on network-attached storage (Premier customers check with your Microsoft rep). That said, I'm a huge fan of NFS based storage.
Whether to choose VMDK or RDM for your Exchange storage should be based on technical and business requirements and not on any preconceived notions of performance or supportability. All storage protocols discussed here have proven to perform well within the requirements of Exchange and support for each is well documented on Microsoft's TechNet site. I've included some helpful links below for your reading enjoyment. With that I'll wrap up this post which hopefully has given you a bit to think about and maybe presented some new options for your deployment.
As always, we look forward to hearing from you so please join the discussion!
-alex
Alex Fontana, Sr. Solutions Architect
Performance Best Practices for VMware vSphere 5: http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.0.pdf
Virtualized Exchange Server on NFS, iSCSI, and Fibre Channel: http://www.vmware.com/files/pdf/vsphere_perf_exchange-storage-protocols.pdf
Performance Characterization of VMFS and RDM: http://www.vmware.com/files/pdf/performance_char_vmfs_rdm.pdf
Exchange 2010 System Requirements: http://technet.microsoft.com/en-us/library/aa996719.aspx
