I was looking through some slides from VMworld 2006 and came across an interesting project by one of my colleagues, Chirag Bhatt, who works on dynamic resource scheduling (DRS) performance. He was looking for a way to study the behavior of DRS under representative workloads and ended up using a variant of VMmark. He had only a single client machine available in his lab, so we suggested he omit the mailserver workload since driving multiple mailservers with a single client can be problematic due to domain controller issues. Luckily for Chirag, the VMmark harness is flexible and can easily be reconfigured for this type of test. I’ve included a couple of slides of his results below for those who couldn’t make his VMworld talk.
A two-node DRS cluster was created for these experiments using IBM eserver 336 systems. Two modified VMmark tiles, consisting of five server VMs each (file, database, java, standby, and web servers), were used. A baseline throughput measurement was first taken to determine the optimal expected throughput (Scenario 1 in the slide). Subsequent tests were started with all workloads running on a single server within the DRS cluster (Scenario 2). The DRS scheduler was then run with both moderate and aggressive settings. The resulting VM placements are shown in Scenario 2a and Scenario 2b, respectively. The aggressive setting results in the intuitive placement of essentially splitting the tiles across the two systems except for the standby server. That the standby server does not migrate is not surprising since it is consuming few resources.
Chirag’s second slide quantifies throughput performance of the server VMs. In both cases, DRS provides results close to the optimal. In fact, the nearly perfect workload division generated by the aggressive DRS setting was within 99% of the optimal case. Not only does DRS work well, but VMmark also shows good robustness along with the ability to measure cluster-level performance.
When we set out to design VMmark, we understood the need for a well-designed, representative multi-VM benchmark. However, as ESX Server has evolved into a complete datacenter infrastructure with VI3, we now see how broadly such a characterization tool is needed. VMmark displays the potential to develop along the same lines into a cluster-wide benchmark measuring much more than a single system’s performance.