Posts Tagged ‘ esxi ’

Josh Simons

Virtualized High Performance Computing (HPC) at VMworld 2013

August 19, 2013
By

It’s been a busy few months on the HPC front here at VMware. I’ve been spending increasing amounts of time answering questions from our field people and talking with customers about the benefits of using virtualization and cloud technologies for HPC workloads. The two hottest application areas are clearly Life Sciences and Electronic Design Automation (EDA), with Financial Services as a third area where serious interest is starting to ramp. Customers in these sectors are actively evaluating or deploying HPC solutions based on VMware technology. Some of this work will be presented at VMworld 2013 August 25-29 in San Francisco. Here are the talks that will be of special interest to organizations whose job mixes include the technical, scientific or engineering...

Read more

Josh Simons

Paravirtual RDMA for Low Latency and Flexibility

December 27, 2012
By

The Office of the CTO has been exploring how to best enable application access to RDMA for those applications requiring the ultimate in high bandwidth, low-latency communication, which includes many HPC MPI applications as well as many scale-out databases and BigData approaches. Passthrough mode is the most straightforward way to enable guest-level RDMA. With passthrough (which we call VM DirectPath I/O), a physical PCI device can be made directly visible to the guest operating system running within the virtual machine. We published a research note showing that this approach delivers very good InfiniBand latencies (under 2us) and excellent bandwidths over a wide range of message sizes. There is a downside, however: Punching through the virtual machine abstraction in this way disables several...

Read more

Josh Simons

Virtualization, Italian Style

July 11, 2012
By

I’m in Fiuggi, Italy this week teaching a four-lecture course on Virtualization and High Peformance Computing as part of the Eighth International Summer School on Advanced Computer Architecture for High Performance and Embedded Systems ( ACACES 2012 ). I have about 70 students in the class, primarily from European countries.

Read more

Josh Simons

vSphere 5: vNUMA paper now available

October 27, 2011
By

I promised in the blog post vNUMA: What it is and why it matters to share the full research paper with you when it became available. It can now be downloaded here . If you find the paper useful, please click to give it a star rating on that same page. As a reminder, here is the paper abstract: A major obstacle to virtualizing HPC workloads is a concern about the performance loss due to virtualization. We will demonstrate that new features significantly enhance the performance and scalability of virtualized HPC workloads on VMware’s virtualization platform. Specifically, we will discuss VMware’s ESXi Server performance for virtual machines with up to 64 virtual CPUs as well as support for exposing virtual NUMA topology...

Read more

Josh Simons

Virtualizing Big Data

October 26, 2011
By

Analysis of large-scale, often unstructured data is becoming increasingly important within both the Enterprise and the HPC community. This is perhaps one of the most apparent areas where the convergence of HPC and Enterprise requirements can be seen as the tools and algorithmic approaches required are often the same or very similar. I imagine, for example, that the large-scale, graph-oriented “social network” analyses done by companies like Facebook are quite similar to the “anti-social network” analyses done by Homeland Security and the Intelligence community. Unsurprisingly, many VMware customers are interested in running Big Data workloads and are looking for guidance about how best to do this in a virtual environment. To help, we have published a whitepaper that examines Hadoop performance using local...

Read more

Josh Simons

vNUMA: What it is and why it matters

September 19, 2011
By
vNUMA: What it is and why it matters

In vSphere 5 , we introduced vNUMA which allows interested guest operating systems to see that they are running on a NUMA (Non Uniform Memory Architecture) topology. For those not familar, here is a one-diagram NUMA explanation. As you can see, in the UMA case, the cost of accessing a particular memory address is the same regardless of which socket your program is running on. In the NUMA case, however, it does matter. With memory attached directly to each socket there can be significant performance penalties if an application gener ates large numbers of non-local memory accesses. The ESX hypervisor has been NUMA-aware for quite some time, making memory and CPU allocation decisions based on its full understanding of the topology...

Read more