VMmark 4 tile
Performance Benchmarks Cassandra Database Cloud Database Performance DRS DRS DVD Store Kubernetes vCenter Virtualization VMmark vMotion vSphere Weathervane

Introducing VMmark 4: A modernized private cloud server consolidation benchmark

VMware VMmark 4.0 (available here) is a free cluster-level benchmark that measures the performance and scalability of enterprise virtualization environments.

VMmark 4 continues to leverage the design and architecture from previous generations of VMmark while providing enhanced automation and reporting. It uses a tile-based heterogeneous workload application design that includes updated versions of several VMmark 3 application workloads while also adding new modern applications that more closely represent what is running in today’s enterprise virtualization environments. VMmark 4 also contains several infrastructure workloads: vMotion, storage vMotion, cross vMotion, and clone & deploy. In addition to these, VMware vSphere DRS also runs within the cluster under test to balance the workloads across the cluster. With VMmark 4.0, we’ve continued to improve the fully automated provisioning process we introduced in VMmark 3, decreasing the time it takes to get valuable results. In most cases, you can now go from the downloadable VMmark template VM to VMmark 4 results in about two hours for a turbo run.

VMmark 4 tile

The VMmark 4 benchmark:

  • Allows accurate and repeatable benchmarking of vSphere virtual datacenter performance.
  • Uses a variety of traditional, legacy, and modern application workloads within a heterogeneous tile-based approach.
  • Facilitates the analysis and comparison of hardware, software, and configuration changes with vSphere virtualization environments.
  • Workload levels are significantly higher than previous VMmark releases to better represent today’s environments.

VMmark 4 usability updates:

  • New quick start provisioning mode: Lets you completely deploy, run, and get results from the benchmark using a single command-line request.
  • New provisioning modes: Give you more flexibility in how the VMs are distributed across a wider variety of storage.
  • New partial tile functionality: Increases the benchmark granularity through prescriptive enablement of application workloads within a final partial tile.
  • Automation first design: Makes available many of the underlying vSphere administrative operations like deleting_all_vmmark4, manual_xvmotion, and power_vmmark4_tiles that further assist the end-to-end automation of VMmark 4. See the vmmark4service for a list of the more than 20 new operations available.
  • Enhanced HTML reporting: Automatically returns enhanced HTML output for throughput, quality of service, and infrastructure operations on a per-run basis.
  • New disclosure creator application: Simplifies and automates the creation of disclosure HTML files.
  • Sustainable power collection: Collects power metrics on the systems under test and generates enhanced HTML output to help you understand both host and VM power consumption.
  • Alert integration: Embeds Slack and Google Chat alerts into VMmark 4. You can easily enable it using a single parameter.

VMmark 4 application workloads:

  • NoSQLBench: New application workload within VMmark 4 that is used to analyze the performance of a new 3-node Apache Cassandra distributed NoSQL database.
  • SocialNetwork: New application workload within VMmark that uses Docker container microservices to simulate a social graph with operations like creating posts, following users, and more.
  • DVD Store: Updated to version 3.5, which includes PostgreSQL and parallel database loading, reducing the time to provision the first tile.
  • Weathervane: Updated to version 2.0, this highly scalable web application mimics an online auction and now runs in Kubernetes containers in addition to virtual machines.
  • Standby: The standby server mimics an idle heartbeat server that is periodically pinged to ensure it’s still running and connected.

VMmark 4 infrastructure workloads:

  • vMotion: This infrastructure operation performs live migration of one of the standby VMs in a round-robin fashion to simulate modern sysadmin operations.
  • Storage vMotion: For this operation, one of the AuctionWebF VMs is migrated to a user-specified maintenance partition and then, after a period of rest, returns to the original location.
  • Cross vMotion (XvMotion): This operation simultaneously moves one of the DS3WebA VMs to an alternate host and a maintenance storage partition. Similar to the storage vMotion operation, after a period of rest, the VM will return to its original location.
  • Automated load balancing (DRS): VMmark requires that DRS be enabled and running to ensure typical rebalancing operations occur within the environment under test.