posted

37 Comments

This is an interesting question, and something which popped up in some recent discussions. This is nothing more than a math exercise considering that a VM can have 4 SCSI controllers and 15 devices per controller, but the numbers are still quite interesting. Let's look at what options you have. The following is true for all the Virtual SCSI Controllers that are supported on VMs at this time (LSILogic, BUSLogic & PVSCSI).

  1. VMDK (Virtual Machine Disks) approach. VMDKs have a maximum size of 2TB – 512 bytes. Maximum amount of storage that can be assigned to a VM using VMDKs is as follows: 4 controllers x 15 disks each x 2TB (-512 bytes) = ~120TB.
  2. Virtual (non pass-thru) RDMs approach. vRDMs also have a maximum size of 2TB – 512 bytes (same as VMDK). Therefore, the maximum amount of storage that can be assigned to a VM using vRDMs is as follows: 4 controllers x 15 disks each x (2TB – 512) = ~120TB
  3. Physical (pass-thru) RDMs approach. The maximum size of a pRDM since vSphere 5.0 is ~64TB. Hence, the maximum amount of storage that can be assigned to a VM using pRDMs (assuming vSphere 5.0) is as follows: 4 controllers x 15 disks each x 64TB = ~3.75PB

Of course, these are theoretical maximums & should be considered as such. I personally don't know of any customers who are close to this maximum size.

For completeness, there are some other options as well of course. I am aware of a number of customers implementing storage in a VM using the following techniques:

  1. iSCSI initiator in the VM (e.g. MS iSCSI Initiator – http://kb.vmware.com/kb/1010547). The amount of storage presented to a VM using this method will be down to initiator and Guest OS, but here disks can be passed directly into Guest OS. My understanding is that some of the Linux based iSCSI Initiators can have up to 1024 devices presented to them.
  2. NFS client in the VM (Linux/Unix Guest OS). Limitations on the amount of storage that can be presented to the VM will be down to NFS client and Guest OS, but shares/mounts can be passed directly into Guest OS.

Regarding RDMs (Raw Device Mappings), remember that you won’t be able to use a number of technologies if the customer goes with physical (pass-thru) RDMs, such as snapshots and the VADP APIs for backing up VMs. These technologies can still be used with virtual (non pass-thru) RDMs.

Do you have a VM with a very large storage footprint (100s of TB & greater)? I'd be interested in knowing how you've implemented it. Please send me a message in the comments.

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: Twitter @VMwareStorage

About the Author

Cormac Hogan

Cormac Hogan is a Senior Staff Engineer in the Office of the CTO in the Storage and Availability Business Unit (SABU) at VMware. He has been with VMware since April 2005 and has previously held roles in VMware’s Technical Marketing and Technical Support organizations. He has written a number of storage related white papers and have given numerous presentations on storage best practices and vSphere storage features. He is also the co-author of the “Essential Virtual SAN” book published by VMware Press.