I am pleased to announce that the public beta release of VMmark is now available. It can be found at http://www.vmware.com/beta/vmmark/. Before I discuss this release, I’d like to thank all of the partners who gave us such valuable feedback throughout the VMmark technology preview and private beta program. They really helped us get it into shape for a wider audience. Those of you new to VMmark might want to check out the original benchmark design document at http://www.vmware.com/vmtn/resources/573. For the public beta release, VMmark has undergone some key changes from that original specification in order to simplify benchmark setup and execution. The most important changes are:
-
Distribution of virtual appliances for the Linux-based workloads. Our good friends at Novell have graciously agreed to allow us to redistribute SUSE SLES10 VMs for the database server, file server, and web server components of VMmark. Users will still need to do some simple setup and configuration. Otherwise, the servers are ready to go.
-
Distribution of an XML-based open-source benchmarking harness for STAF/STAX appropriate for choreographing multi-workload benchmarks.
-
Reduction of the memory footprint from 7GB per tile to 5GB per tile. As I related in an earlier blog entry, this change allows us to measure a wider range of systems, including 2-socket multi-core systems with limited DIMM slots.
-
Replacement of the database workload with SysBench running against MySQL. This change allows us to distribute a fully-configured database server VM to simplify the benchmark setup.
-
Bug fixes and workload adjustments.
-
A comprehensive benchmarking guide with step-by-step instructions for benchmark setup and execution.
VMmark is intended to measure performance in an enterprise consolidation scenario. More importantly, it is a benchmark for the entire virtualization platform. Appropriately configured CPU, memory, network, and storage are recommended. In its current form, it still requires some degree of effort to set up and run effectively. Users should have an MSDN subscription (or equivalent) in order to set up and run the Windows workloads. Licensed copies of SPECweb and SPECjbb are also required. I anticipate that the primary users of VMmark will be both vendors and power users wishing to characterize various virtualization platforms. Happy benchmarking!