VMware's "Proprietary" Clustered File System
There has been some discussion lately from some of our competitors on their storage architectures or lack there of. Their boast is that VMware has a “proprietary clustered file system” that forces you to change your storage management practices while they preserve your physical storage management. They’re right, they do completely preserve the complexity of physical storage management. What they fail to mention is that this is a problem, not a solution. Physical SAN management is overly complex and a headache for most customers. And in the virtual world, the status quo is compounded! Instead of managing dedicated LUNs for every physical server, if you buy their argument you’ll be managing separate dedicated LUNs for every virtual machine!
VMware doesn’t have a vested interested in preserving physical complexity. That’s why VMware invented VMFS and our hypervisor-based storage stack. Virtualization is all about abstracting the physical world to ease management burdens and reduce complexity. VMFS is very special purpose clustered file system that is largely transparent to our customers. It is used by VMware’s ESX Hypervisor to abstract and share LUNs used to store Virtual Machines and their Virtual Disks.
The result? Great customer benefits, such as:
- VMware’s instant one click provisioning, including storage. Quick, easy provisioning of a new VM, OS and application that does not require physical storage LUN provisioning.
- Mobility/Portability. i.e Vmotion and storage Vmotion. In a virtual world, workloads should be abstracted from, not beholden to, physical storage. Just like they should be abstracted from physical servers.
- Encapsulation and HW Independence. VMs should be entirely encapsulated from the physical world. This simple, but critically important facet of virtualization unleashes the power of virtual infrastructure. For an example, look at VMware’s new Site Recovery Manager that enables DR solutions that no longer require identical hardware (and software) configurations at each site.
- Reduced complexity. SAN management is hard, complicated work. Why shouldn’t it be simplified?
Now is VMFS good for absolutely every use case? No, sometimes it’s important to preserve a physical LUN format for a given workload. That’s why VMware supports RDMs (Raw Disk Maps) which allows a customer to directly map an entire LUN into the guest OS while still maintaining virtual infrastructure benefits. The trade off with RDMs is more LUNs to provision and manage, but direct access from the VM to a native physical LUN.
What our competitors fail to mention is that not only does VMware support all the use cases they claim to provide, but we also bring all the benefits of virtualization to the entire data center, storage included. And that’s something worth touting!
Well the problem we experienced today is when the proprietary file system gets corrupted, it is far more complicated to fix. In fact it all started When one of our VMware servers started doing wierd things, someone rebooted the host and voila, missing vmfs. If we had been using OCF, at least we could have booted from a live CD to salvage the disk systems.
I have also failed to find good support documents for fixing vmfs when you have problems, please could you post some links.
Posted by: Peter Norris | July 25, 2008 at 07:28 AM
Can you talk about SCSI locking that can affect the ability to hot add extents?
I can add extents to a VMFS volume but in some cases I have to power down VMs before certain nodes will see the additional space. I've got VMs from this single VMFS volume running on multiple ESX boxes so there is some kind of SCSI locking issue.
Shutting down VMs for host ESX issues isn't something we enjoy doing very often.
Posted by: SCSI Locking during extent add | August 05, 2008 at 01:50 PM
Hope to see some good stuff here. This is a topic that is overlooked and some rules of thumb with the reasons would be good. Also some discussion of VMware and iswitches
Posted by: Simon Silvie | August 07, 2008 at 02:07 AM
Have a quick question. Can
one set-up (using VMFS or
some other capability) a LUN
so that it can be shared by
multiple VMs on the same or
different physical servers?
Put another way, does VMFS
implement distributed
lock management? If not,
are there 3rd party solutions
that offer such a capability?
Posted by: Saqib Jang | November 05, 2008 at 02:52 PM
VMs and Virtual Disks stored on a VMFS volume and Virtual mode RDMs (aka non-passthru RDMs) are synchronized to only be accessible by one VM at a time within an ESX cluster. This synchronization is performed at the VMFS level. VMs and Virtual disks stored
on an NFS volume are also synchronized, but via a different mechanism.
Physical (aka pass-thru) RDMs bypass this synchronization and can therefore be mounted by more than one VM at a time. This access model is akin to having a shared, raw LUN on a SAN. To avoid data corruption with this style of RDM, some other form of out of band synchronization must be utilized by consumers of the LUN, especially if there will be more than one writer. One such method of synchronization would be a third party distributed lock manager.
Posted by: Scott Davis | November 06, 2008 at 02:12 PM