VMTN Blog

Steve Wilson @ Sun talks about virtualization benchmarks:  Virtual Steve : VM Benchmarking Wars.

Despite the reports about 3x faster, what Oracle said was actually, "Oracle consistently saw much better resource utilization with an average of three times less overhead using Oracle VM."  In benchmark-speak, that's a lot different than 3x faster.  In fact, I'm guessing that someone constructed a benchmark that shows Oracle VM running at 99% of native speed, while VMware ran at 97% of native speed -- 1% overhead instead of 3%.  Yes, "three times less overhead", but ultimately trivial.

Of course, the the long term, there will be better official benchmarks of VM performance, but the real test will be customer metrics like

• How fast can I roll out a new application?
• How many system admins do I need to run my data center?
• How much energy can I save by operating my data center in a new way?

Nobody's seen Oracle's benchmark results yet, but while we're waiting, VMware's Richard McDougall also adds the following: Just
what are the important performance factors for Virtualization?

• Throughput: can the application deliver the required levels of throughput, in terms of real world transactions?
• Latency: is the latency of each transaction within tolerances, or affected by virtualization
• Scalability: does throughput/latency change as load is increased
  (often asked in the context of - “do I have enough future headroom?”)
• Efficiency: How much physical resource is used to deliver a prescribed throughput
• Price-performance: Cost of the total system
• Power-performance: Throughput relative to power, i.e. how much performance can I deliver for the power consumed
• Space-performance: How much performance can I get per rack unit
• Agility: What is the time taken to deploy a new application

Richard was the author of Ten Reasons Oracle Databases Run Best on VMware (although I don't want to slight the many folks who worked on producing those results).

Virtualization performance is a topic that is complicated, and like all benchmarks and performance metrics, has to fit into the larger context of what problems you're trying to solve. But the take-home message here is that people run Oracle and SQL Server and other databases every day in production on VMware Infrastructure, and performance is not an issue for them.