GPGPU Computing in a VM

Periodically on forums I see people asking whether GPGPU computing is supported by ESX, prompting me to write a brief post on the subject:

YES! VMware ESX supports GPGPU computing. Applications can access GPGPUs via CUDA or OpenCL in exactly the same way as when running natively — no changes are required to the application.

Recent versions of ESX (4.0 onwards) support a feature termed VMDirectPath IO. VMDirectPath IO allows guest operating systems to directly access PCI devices & this feature can be used to achieve GPGPU computing in a VM.  As one would expect with direct communication, the performance overheads are minimal, providing close to native performance.

The process for setting-up access to a GPU from a VM is fairly simple:

(1)    Requires a VT-d capable system (or system with AMD IOMMU)

-          Nehalem class processor or later

-          VT-d performs the remapping of I/O DMA transfers and device-generated interrupts needed for safe operation in the virtualized environment

-          Ensures guest isolation

(2)    Requires a “direct passthrough” capable GPGPU

-          Most NVIDIA Quadro cards are passthrough capable

-          Recent AMD cards are also passthrough capable

(3)    Ensure I/O MMU is enabled in system BIOS

(4)    VMDirectPath IO can be enabled via vSphere client or ESX command line

-          Enable GPGPU as a passthrough device

-          Add GPGPU to chosen VM

(5)    Install vendors latest graphics drivers in the VM

(6)    Download GPGPU SDK

(7)    Run your CUDA/OpenCL apps!

[N.B. See VMware KB1010789 for more configuration details]

I’ve personally undertaken some testing prior to this post:

-          Successfully tested with both Windows and Linux VMs (32 and 64-bit VMs)

-          Successfully tested both CUDA and OpenCL frameworks on Nvidia Quadro GPUs

-          Successfully tested OpenCL on AMD GPUs

Additionally, I spent some time investigating the performance overheads associated with performing GPGPU computing from within a VM. Given that once the data has been loaded into the GPU’s local memory and the processing operation initiated the GPU operates largely independently of the OS, if we expect to see any performance overheads, they are likely to be most significant when moving data back and forth between the VM and the GPU and initiating commands. Accordingly, I’ve spent time looking at these scenarios and found the following:

-          Data transfer back and forth between host and GPU is close to native performance

  • Found to be true even for small  data chunks (e.g. 1KB)

-          Tested a number of short-duration microkernels from a wide range of application spaces and found no noticeable performance impact

  • DSP to financial to computational fluid dynamics  (and beyond)

-           VM performance observed at typically 98%+ of native performance

Operation close to native performance is not unexpected, as the direct communication been VM and GPU provided by VMDirectPath IO ensures the hypervisor doesn’t get involved in each and every interaction (which would add overhead), allowing even short duration operations to run close to native performance.

So, in summary, ESX supports GPGPU computing in a VM, at extremely close to native performance!


11 comments have been added so far

  1. Can multiple VMs share access to the same physical GPU resources at the same time, as one can do with Hyper-V, or does VMware force the need for each VM to have their own GPU chip dedicated?

  2. This works great for most cards however it seems ESX isn’t properly mapping the dependent parents for multi-GPU cards. I have a Radeon 5970 working in KVM using VT-d but on ESXi 5.0 my 5870 works however ESXi isn’t properly stepping up to the PCIe bus, rather the local PCIe on the GPU itself so passthrough isn’t working.

  3. Can you please let me know the hardware specs of the system that you used. I used hp z800 workstation which has broadcom NIC card. So when I enable passthrough to c2050 GPU card and reboot the system vsphere server 5 hangs at cnic_registry. I even tried with vsphere4.1. I get error with NIC.

  4. how/where i can check is Geforce 9600gt is Is passthrough capable?
    Should i install some Nvidia drivers in Esxi (im with 5.0)

  5. Can VMWare take profit of GPGPU computing virtualizing more available CPU cores and running the CPU threads on the GPU?

  6. Does it mean If i have two GPUs can I assign one to ESXi and second directly to Windows 7 Guest to play games from time to time?

  7. It isn’t easy to find a free Tetris game for i – Phones, so snap this one
    up quickly while it’s still free. You can try the free version of the game and then buy the full version if
    you feel the game is as immersive as other PC and console
    zombie shooting titles. Also, scalability of a server system is a
    challenge, since real-time multiplayer gaming requires a lot of processing power and network

Leave a Reply

Your email address will not be published. Required fields are marked *