Home > Blogs > VMware vSphere Blog

Virtual Flash (vFlash) Tech Preview

This is the third and final tech preview of storage features which were shown at VMworld 2012. Already we have looked at two tech previews – Distributed Storage and Virtual Volumes (VVOLs). This next feature is vFlash or Virtual Flash, and will look at a project underway here at VMware to integrate local flash devices with vSphere. This will make significant performance improvements and reduce I/O latency for many workloads running in the Virtual Machine.

To date, VMware has done very little around flash. We only have the smartd introduced in vSphere 5.1 to monitor Solid State Drives (SSD) attributes and the swap to SSD feature. Those of you who have read the technical preview article on Distributed Storage will have read about SSD being used as a cache (for both read and write I/O). This post discusses an additional project called vFlash, the purpose of which is to integrate vSphere with local flash devices. What we are looking to do is to enable a new tier of storage for your Virtual Machines. For those of you unfamiliar with flash technology, I recently wrote a blog article on my personal blog which will give you a pretty decent overview.

Please note that since this is a tech preview article, there is no guarantee when this feature will appear (if ever), nor is there any guarantee that the end product will encompass any or all of the attributes discussed in the post. It is simply to give you an idea of what we are working on, and what features the final product might include. vFlash was discussed at VMworld 2012 & my colleague Duncan provides a nice overview of the session on his blog here.

vFlash Infrastructure

We want our customers to be able to select off the shelf flash devices, be they SSD or PCIe I/O Accelerator cards. Once the flash devices are plugged into vSphere hosts, we have a toolset available in vSphere to manage flash as a resource, just like you currently manage CPU and memory resources. Basically, you will be creating a flash pool, the contents of which will be carved up to provide flash resources to individual VMs using constructs that you are already familiar with, such as reservations, limits & shares.

vFlash is a framework. VMware will be providing our own default software to plugin (vFC) to the framework, but other flash vendors can create their own plugins (vFlash Cache Modules) containing bespoke and proprietary algorithms for utilizing their specific cache/flash devices. The vision is to publish a set of APIs to support this, and share them with potential partners.

The vFlash infrastructure does not specify which Virtual Disk data is to be cached in vFlash. That decision is left up to the vFlash Cache Module. It is envisioned that the vFlash Framework will support multiple vFlash Cache Modules per ESXi host. Different Virtual Machines running on the same host can be configured to use different vFlash Cache Modules. This permits the vFlash caching algorithm used for a given Virtual Disk to be tailored to the storage I/O behavior of the application running in the Virtual Machine using that Virtual Disk.

That caching software can come in two forms – VM-aware caching & VM-transparent caching. With VM-transparent caching, the VMs will share a pool of flash resources and are not aware that they are doing so; they will simply benefit from having flash in their I/O path. With VM-aware caching, chunks of flash can be allocated on a per VM basis, and these will show up as SCSI disks in the Guest OS.

VM-aware Caching (vFlash Memory)

This is where a flash resource is presented directly to the VM. A new virtual hardware device called vFlash Memory is added to the Virtual Machine. The interesting part of this approach is that the caching algorithm needs to be controlled by the VM, and not by the caching software on the hypervisor. This may possibly entail the installation of agents in the Guest OS, in a similar fashion to how some flash vendors currently do things. The cache appears as a disk drive to the Guest OS, which can then be formatted and used appropriately by applications in the Guest OS, and thus can benefit from having access to a flash device.

VM-transparent Caching (vFlash Cache)

This is where the VM is unaware that there is a flash cache in the I/O path, but benefits from it all the same. A new virtual hardware device called vFlash Cache is added to the VM. In this case, cache software on the hypervisor is there to provide a suitable algorithm for the I/O. Options available during the configuration of VM-transparent caching will include reservation size, the choice of vFlash Cache Module, mode of operation (write-back or write-thru), block size (tuned to Guest OS requirements) and what to do with the flash contents during a migration (migrate or drop). The user will have the option of migrating flash contents if the destination host is compatible. The default option is to attempt to migrate the content, but if the destination host is incompatible, then we will drop the flash contents and rebuild it on the destination host. Obviously this will have a performance impact as the flash contents would need to ‘warm up’ on the destination host. VMs with flash cannot be migrated to a destiantion host with insufficient vFlash resources because even if the contents are not migrated, the flash contents will need to be rebuilt at the destination host and we need to ensure that appropriate resources are available to do this. vCenter compatability checks will fail the migration if the destination host does not have sufficient cache resources.

Similarly, when considering vSphere HA, flash will need to be pre-allocated or set aside to ensure that enough resources are available for virtual machines to successfully restart on remaining hosts in the event of a host failure.

Direct attached flash storage is gaining significant momentum right now, and is an ideal reource to provide low latency for latency sensitive applications, enabling even more tier 1 applications to be virtualized. VMware wants to enable our customers to leverage flash resources through well known and well understood vSphere mechanisms.

Get notification of these blogs postings and more VMware Storage information by following me on Twitter:  @VMwareStorage

This entry was posted in vSphere and tagged , on by .
Cormac Hogan

About Cormac Hogan

Cormac Hogan is a Senior Staff Engineer in the Office of the CTO in the Storage and Availability Business Unit (SABU) at VMware. He has been with VMware since April 2005 and has previously held roles in VMware’s Technical Marketing and Technical Support organizations. He has written a number of storage related white papers and have given numerous presentations on storage best practices and vSphere storage features. He is also the co-author of the “Essential Virtual SAN” book published by VMware Press.

14 thoughts on “Virtual Flash (vFlash) Tech Preview

  1. Pingback: Technology Short Take #28 - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers

  2. Leah Schoeb

    Are you all really want to support agents in the guest OS? Several vendors that have written caching engines for vSphere are finding it more efficient not to agents in the guest. Have you looked at benefits of caching at the guest OS level vs. doing at the Hypervisor level?

    1. Cormac HoganCormac Hogan Post author

      Hi Leah,
      Indeed, I’m familiar with a number of new vendors such as Conduisiv & Proximal Data who have new caching engines. However these engines (to the best of my knowledge) still require software installed in the Guest OS, and they consume host memory rather than flash (which is more expensive). While one might argue that these engines are only using memory which is not being used by the VM, once that VM becomes busy there may be little or no memory available to do the caching. I think having a dedicated flash/cache resource which doesn’t interfere with VM resources and is cheaper than memory may be a more attractive solution to many customers. Having said that, if customers have too much memory on their hosts and can allocate more memory to a VM that it actually needs to run its apps, then a caching engine like you describe could be beneficial.

      1. Alireza Haghdoost

        I believe SanDisk FlashSoft is one of those that does not require any agent in Guest OS. I think it is possible to do the caching stuff in the hypervisor more efficiently with limited and unified processing power compared with guest OS where multiple instant of the caching agent is running concurrently.

  3. Lars Stoustrup

    First of all, great to see that VMWare is doing something in this area. Adding SSD caching to any kind of storage subsystem be it DAS, SAN or NAS is a huge improvement over anything else. I have been using Facebook flashcache (https://github.com/facebook/flashcache/) in an experimental setup for some time now and IO improvements are considerable – especially on random IO. We have a VMWare development setup with a regular RAID 10 (Adaptec) setup that provides about 4000 IO/s. Not too great. Running Ubuntu with flashcache on top of it in write-through mode, we get 18000 IO/s which is downright impressive (and cheap too). Having a flashcache like solution at VM host level would really be our dream scenario – at least IO wise :-)

    1. Alireza Haghdoost

      you mean it is not possible to install Facebook’s FlashCache on a ESX hypervisor ? Why don’t you try dm-cache ? I think STEC has also an open-source caching product , called EnhanceIO which is kind of similar to FlashCache.

  4. Ole Andre Schistad

    So now that vSphere 5.5 has launched and vSAN is in beta, can you shed some light on to which extent vFlash still exists as a separate technology?

    vSAN is interesting in some use cases, but it relies on having all the tiers on the servers which really doesn’t fit well with an Enterprise datacenter where you want data to eventually find rest on a proper SAN, so you can manage it properly.

    vFlash is a perfect complement to the SAN since it alleviates the major shortcoming of a centralized storage system, namely the propagation speed in the infrastructure which is actually a real bottleneck for flash media. So I for one am very keen for VMware to continue developing this product; in my opinion the potential for this technology is enormous.

    There are competing technologies out there, but they tend solve corner cases and are mostly just read caches which really isn’t the hard problem we need solving.

  5. Ole Andre Schistad

    Ahh right, so vFlash was replaced by vFRC.

    I did hear some hints that there might be some write-back stuff coming as well – which is what I was mostly discussing in my previous post.

    Now that is a much harder problem to solve because you need to persist a write on at least two unique devices before you can acknowledge the write, but at the same time this really is required for a transparent low-latency assist.

    I guess we’re in futures territory here, but is there anything you can share on this?


Leave a Reply

Your email address will not be published. Required fields are marked *