Comparing ESX Server and VMware Server Using VMmark

VMware Server is a best-in-class hosted virtualization solution and provides an ideal way for new users to start using virtualization. It also works well for small deployments as well as application test and development. Potential VMware users often ask how VMware Server differs from VMware’s ESX Server, our hypervisor-based virtualization solution. A complete response to that question would include long discourses on ESX Server’s superior feature set, manageability, and overall reliability and robustness. Since I am hardly qualified to attempt such a thorough description, I’m going to stick with what I know – performance. At my VMworld 2006 talk, I presented a performance comparison between ESX Server and VMware Server using VMmark about which I am often asked. We ran VMmark using ESX Server on an HP DL585 with four 2.2GHz dual-core processors. We then ran VMmark on VMware Server using a similar HP DL585 with four 2.4 GHz dual-core processors. The results slide from that talk is shown below. (Thanks to my colleague Lisa Roderick in our Cambridge office for collecting the VMware Server numbers.)


These results show that ESX Server not only achieves higher throughput than VMware Server for a single VMmark tile (6 workload VMs) but also exhibits better scalability when a second tile is added. This behavior is a natural consequence of the different virtualization approaches taken by the two products. VMware Server runs on top of a heavyweight, general-purpose host operating system, which manages the hardware resources. (The host OS was Windows 2003 Server in these experiments.) On the other hand, ESX Server manages the hardware resources directly and is highly tuned to efficiently support virtual machines. This optimized design reduces the overhead for individual VMs and produces higher benchmark throughput in general. It is also unsurprising that ESX Server’s highly tuned VM resource management provides superior scalability as more workloads are run.


8 comments have been added so far

  1. It’s a good chart to start thinking about vmware server vs. esx.
    I believe however that most users of vmware server don’t use 6 VMs at the same time, but rather 2 or 3 (so probably the chart doesn’t apply for these cases).

  2. I think you need to go back to the drawing board on this. As the other comment states a Server user typically only has a couple of VM’s not 6. My suggestion would be to put 1 VM on Server and 1 VM on ESX and compare. Also, you are comparing two different pieces of hardware. Run your tests using identical hardware. Finally, your scale has no reference. A score of 1, 2, 3, or n is meaningless to us.

  3. It looks like we agree that VMware Server is well suited for smaller deployments as I mentioned above. However, not everyone has such a clear understanding of VMware’s product line. Questions about the performance difference between ESX Server and VMware Server in an enterpirse environment come up often enough that it prompted the comparison I described. Although single-VM performance comparisons can some provide insight, they do not capture the complexity seen in an enterprise consolidation scenario.
    The VMmark benchmark was designed to measure virtualization performance in just such a scenario. To say that the “scale has no reference” is to misunderstand the benchmark. In fact, all VMmark scores are normalized with respect to a reference system to allow valid comparisions to be made. You can get more detail on the methodology from the VMmark tech report at http://www.vmware.com/vmtn/resources/573.
    I agree that an ideal comparison would use identical hardware. In this case, VMware Server was given a slight advantage in CPU clock speed (< 10%). I am comfortable saying that ESX Server's superior throughput was due to its better software deisgn, not the slower underlying hardware. As VMmark becomes more widely used, I would expect to see a wide range of comparions as vendors juxtapose the performance of their latest systems using the latest versions of VMware with older versions of the hardware and virtualization layer. This similar to how both SPEC and TPC benchmarks are often used.

  4. Truth is that vmware server running on a linux kernel (debian) would have no difference in speed compared to ESX server.
    Top note is only running 1 virtual at a time, ESX will manage more virtual at one time better than vmware server.

  5. serb is way off. Vmware server on top of linux would still be much slower than ESX even in the 1 vm case. This particularly applies to anything that is I/O (disk or network) bound, since ESX has a much less overhead, but even CPU bound tasks will still be slower in VMware server than ESX due to the overhead of the guest os (and about an order of magnitude more instructions being executed every world switch)

  6. Tim, you didn’t respond very well to the criticism of the VMmark scoring system. Without directing me to a lengthy paper, can you please give me more than “this one gets two points, and this one gets four”. Hard to justify 5k a box with that kind of evidence. What were the performance differences in some sort of familiar scale?

  7. A very specific comment arises when I attempt to run a database app on both platforms. On ESX it appears to run nicely. However on server I cannot access the database or start any type of broker service. I have been looking at unlinked library via ldd. etc, but cannot find a good reason yet.

  8. I have only 2 VM installed. I suppose the chart will vary with the number of VMs. 6VMs are astronomical.
    Serb is right about the speed comparison. It hardly has any difference on linux kernel. I am using Mandrake 10.1

Leave a Reply

Your email address will not be published.