Posts Tagged ‘ infiniband ’

Josh Simons

Running HPC Applications on vSphere using InfiniBand RDMA

December 22, 2014
By
Running HPC Applications on vSphere using InfiniBand RDMA

Looking back on 2014, this was the year in which we made significant strides in assessing and addressing High Performance Computing (HPC) performance on vSphere. We've shown that throughput or task-parallel applications can be run  with only small or negligible performance degradations (usually well under 5%) when virtualized -- a fact that the HPC community seems to now generally accept. The big new thing this year, however, was the progress we are making with MPI applications -- specifically those applications that require a high-bandwith, low-latency RDMA interconnect like InfiniBand. As I mentioned in an earlier blog post , we generated a great deal of RDMA-based performance data about HPC applications and benchmarks this past summer due to the hard work...

Read more

Josh Simons

Supercomputing '14 Begins: Laissez les bon temps rouler!

November 17, 2014
By
Supercomputing '14 Begins: Laissez les bon temps rouler!

  SC'14 , the big, annual High Performance Computing (HPC) conference officially kicks off in New Orleans this evening with the opening of the exhibits. Workshops and tutorials have been held over the weekend as well: I gave a keynote presentation on Sunday at the 4th International Workshop on Network-aware Data Management ( NDM'14 ), titled RDMA in the Cloud: Enabling high-bandwidth, low-latency communication in virtual environments for HPC (slides:  NDM'14 keynote ). While I've been attending SC since the late 1980s and have attended each year I've been at VMware, this year marks the first time VMware will have an official presence on the show floor. We've partnered with EMC's Office of the CTO...

Read more

Josh Simons

High Performance Computing (HPC) Update

October 31, 2014
By
High Performance Computing (HPC) Update

Our HPC effort at VMware has been very active in recent months, and we have lots of information to share-- including new performance results. Rather than cramming all of that content into a single mega blog entry, I've decided instead to give a preview here of some of the most significant developments. I will then delve into each of these areas in more detail in a series of upcoming monthly blog posts. InfiniBand Performance This spring we installed a four-node InfiniBand HPC cluster in our lab in Cambridge, MA. The system includes four HP DL380p Gen8 servers, each with 128 GB memory and two Intel 3.0GHz E5-2667 eight-core processors and Mellanox ConnectX3 cards that support both FDR (56 Gb/s) InfiniBand and 40 Gb/s RoCE....

Read more

Josh Simons

Virtualized High Performance Computing in 2014

December 12, 2013
By
Virtualized High Performance Computing in 2014

As we complete another circuit around the sun, it's time again to make some predictions about what we'll see over the course of our next orbit. Since my work in the Office of the CTO focuses on High Performance (Technical) Computing, I'll focus there. But before looking ahead, let's take a minute to look back and see how I did with my predictions last year . 2013 Predictions Scorecard Last year, my predictions related to public cloud, private cloud, RDMA in the cloud, virtual machine evolution, and HPC acquisitions. My biggest "miss" was also my most specific prediction -- that Amazon would announce RDMA support in 2013 in order to better support latency-sensitive MPI applications. They did not,...

Read more

Josh Simons

Paravirtual RDMA for Low Latency and Flexibility

December 27, 2012
By

The Office of the CTO has been exploring how to best enable application access to RDMA for those applications requiring the ultimate in high bandwidth, low-latency communication, which includes many HPC MPI applications as well as many scale-out databases and BigData approaches. Passthrough mode is the most straightforward way to enable guest-level RDMA. With passthrough (which we call VM DirectPath I/O), a physical PCI device can be made directly visible to the guest operating system running within the virtual machine. We published a research note showing that this approach delivers very good InfiniBand latencies (under 2us) and excellent bandwidths over a wide range of message sizes. There is a downside, however: Punching through the virtual machine abstraction in this way disables several...

Read more

Josh Simons

2013 Predictions: When Worlds Collide

December 18, 2012
By
2013 Predictions: When Worlds Collide

It's time again to dust off the crystal ball, cast the yarrow sticks, read the tea leaves, and share some thoughts about where the IT world is heading over the next twelve months and beyond. Because my focus at VMware is on High Performance Computing (HPC), I'll confine my prognostications primarily to the growing area of overlap between mainstream Enterprise Computing and HPC -- places where each world can benefit from the other in important ways. Before I start, let's get one no-brainer prediction out of the way: 2012 will be remembered for Nicira , and not Nibiru . Yes, software-defined networking ( SDN ) will trump planetary collisions and the...

Read more