Virtualized Hadoop Performance with vSphere 6

A recently published whitepaper shows that not only can vSphere 6 keep up with newer high-performance servers, it thrives on their capabilities.

Two years ago, Hadoop benchmarks were run with vSphere 5.1 on a cluster of 32 dual-socket, quad-core servers. Very good performance was demonstrated, with the optimal virtualized configuration shown to be actually 2% faster than native for TeraSort (see the previous whitepaper).

These benchmarks were recently run on a cluster of the same size, but with ten-core processors, more disks and memory, dual 10GbE networking, and vSphere 6. The maximum dataset size was almost quadrupled to 30TB, to ensure that it is much bigger than the total memory in the cluster (hence qualifying the test as Big Data, by one definition).

The results, summarized in the chart below, show that the optimal virtualized configuration now delivers 12% better performance than native for TeraSort. The primary reason for this excellent performance is the ability of vSphere to map physical hardware resources to virtual hardware that is optimized for scale-out applications. The observed trend, as well as theory based on processor characteristics, indicates that the importance of being able to do this mapping correctly increases as processors become more powerful. The sub-optimal performance of one of the tests is due to the combination of very small VMs and how Hadoop does replication during data creation. On the other hand, small VMs are very advantageous for read-dominated applications, which are typically more common. Taken together with other best practices discussed in the paper, this information can be used to configure Hadoop clusters for the highest levels of performance. Despite all the hardware and software changes over the past two years, the optimal configuration was still found to be four VMs per dual-socket host.

elapsed_time_ratioPlease take a look at the whitepaper for more details on how these benchmarks were run and for analyses on why certain virtual configurations perform so well.


4 comments have been added so far

  1. This is great but leaves a few questions:

    -This paper says the design supports VMware HA. How can VMware HA work when VMs are using local storage with physical mode RDMs?

    -I can’t vmotion a hadoop node. With 4 on a host, what happens when I need to put ESXi into maintenance mode?
    Could vSAN or some other strategy be used to allow vmotion between hosts?

    1. vSphere HA is not the same thing as vMotion. Physical Mode RDM and vMotion don’t play together, but that has nothing to do with vSphere HA. Even with pRDM, vMotion is still possible in certain configurations, e.g. by enabling the “multi-writer” flag). Also, you should be aware that vMotion is supported with pRDM for Windows clustering on vSphere 6.0

  2. HA wasn’t addressed in this paper. An earlier paper discussed FT protection of the master daemons by placing the VMs that run those daemons on shared storage, and using local storage for everything else. That’s still possible. With local storage, vMotion is possible if storage vMotion is also used. But for Hadoop that may not be practical because of the volume of storage. You can always remove all the VMs on a particular host from the Hadoop cluster and put the host into maintenance mode. No data will be lost because HVE ensures that each block is replicated only once on the whole host. VSAN should work similarly, as long as the number of copies is sufficient.

  3. Good day, I was reading another fact about this on another blog. Interesting. Your perspective about it is diametrically contradicted to what I read earlier. I am still pondering with the opposite points of view, but I’m leaning to a great extent toward yours. And irrespective, that’s what is so great about modern-day democracy and the marketplace of ideas online.

Leave a Reply

Your email address will not be published.