Home > Blogs > Virtual Reality

Those Darn Details… Issues in Yankee Group’s Virtualization Report

Yankee Group recently came out with a report “Virtualization Price War: VMware’s Little Big Horn?” that unfortunately contained quite a few factual errors, inaccuracies, and unsupported claims. The report was not reviewed with VMware prior to publication (as some have claimed). Therefore, VMware is listing the factual errors and inaccuracies in the report because they distort the true cost and value of VMware’s solution. Happy reading.

1. Exhibit 2: Double counting, omissions, and other issues

Exhibit 2 indiscriminately mixes VMware’s bundle and ala carte pricing, making it very difficult for readers to get an actual apples-to-apples cost comparison. If the table is meant to be a cumulative account of Microsoft and VMware pricing (i.e. add up all the rows), then there is significant double-counting in the VMware column. If the rows in the table are meant to be interpreted individually, then there are gross inconsistencies between how VMware and Microsoft’s costs are shown. Most casual readers will assume the table represents cumulative cost for both companies since they see the word “Included” throughout Microsoft’s column. The points below are written from the cumulative cost standpoint.

  1. Double counting on cost of VMware: Row 1 of Exhibit 2 calls out the list price for VMware Infrastructure 3 Enterprise Edition ($5750) in the VMware column. However, the table continues by calling out the ala carte pricing for VMware individual capabilities that are ALREADY included in the price of VI3 Enterprise Edition (i.e. VMotion, DRS, HA, vSMP). This table effectively double-counts the cost for VMware. VMware offers ala carte pricing for customers who wish to purchase one of VMware’s other editions and add features individually. No one would buy Enterprise Edition plus many of the ala carte options in Exhibit 2 because most are included in the Enterprise Edition. There is a reference to a Mr. David Dodge on page 9 of the report that points out this issue but the feedback was apparently ignored.
  2. Double counting on OS costs for VMware: The cost of a Windows Server Enterprise Edition License is counted twice for VMware – once under “Guest Licensing” and again under “Windows Server”. In this configuration, VMware products only require 1 Windows license (just like Microsoft), not 2. Again, Mr. David Dodge calls this out on page 9 of the report – “Dodge also argued that the VMware ESX3 solution can install on bare-metal and does not need an operating system to boot.” Therefore, the “Windows Server” for VMware should be $0 (the cost under “Guest Licensing” already covers it).
  3. Wrong VMware product for a “single server configuration” comparison: The top of page 7 states that Exhibit 2 is a comparison of cost for a “single server configuration.” If that is the case, then why wasn’t VMware Server (further discussion in section 2 below) or the VMware Infrastructure Foundation Acceleration Kit used? The analyst is clearly aware of the acceleration kit because it is cited on page 6 of the report. Such bundles from VMware are specifically targeted at this segment. Also, if this is a single server configuration, when why would a customer want VMotion, DRS, HA which is only useful in a multi-server configuration?
  4. Lack of clarity on management costs: Exhibit 2 fails to point out that a Microsoft System Center Enterprise Management Suite License (ESML) is $860 PER managed HOST (not including the mandatory cost of 2-years of Microsoft SA) while a VMware VirtualCenter Management Server license is a perpetual license independent of the number of managed hosts (VirtualCenter agents come with the VI3 license). By purchasing 1 VirtualCenter Management Server license, a customer can manage 5, 10, 15, 200, etc. number of ESX hosts. Also, where is meant by “Additional Charge $3,750 for Standard” (in the Management row)?
  5. Error – Microsoft Virtual Server does NOT support vSMP: Microsoft Virtual Server does not support virtual SMP. That’s one of the key differentiators that Microsoft calls out between Virtual Server and Hyper-V.
  6. Microsoft features not on-par with VMware features: Microsoft Virtual Server + SCVMM R1 offers nothing like DRS. SCVMM R1 can only recommend a physical server for initial placement when a VM is first provisioned. It has NO ability to dynamically move the VM (based on customer defined resource requirements) within a cluster of hosts to maximize resources. Also, Microsoft’s self-service provisioning is much less capable compared to VMware Lab Manager. For instance, Lab Manager handles self provisioning and management of multi-tier environments. SCVMM R1 offers nothing comparable.
  7. Omission of key features in VMware VI3 Enterprise Edition: The report does not call out many capabilities that customers get if they purchase VI3 Enterprise Edition today. If the report is going to compare VI3 Enterprise Edition to Microsoft Virtual Server, then it should show all of the core capabilities of Enterprise Edition to provide readers an accurate picture of what they are getting. Omitted VI3 Enterprise Edition features include:
    • Storage VMotion (no MS equivalent)
    • Update Manager (no MS equivalent)
    • Guided Consolidation (no MS equivalent)
    • 64-bit guest OS support (no support in MS Virtual Server)

2. Why wasn’t VMware Server compared to Microsoft Virtual Server?

On page 7, the report states the following for why VMware Server was not used in this comparison.

"Customers do have the option of deploying VMware’s free hypervisor technology to even out the cost differential between VMware and competing server operating system-based hypervisors. However, according to the Yankee Group 2006 and 2007 Global Virtualization Surveys, which polled 800 end users worldwide, only 10% of corporations are deploying the free version of VMware for production networks." (Page 7)

This rationale is a double standard – what percentage of Microsoft Virtual Server deployments are currently in “production networks?” VMware’s internal research and anecdotal evidence suggests that MS Virtual Servers’s use in product environments is low. Certainly, it is not as high as the >85% of VMware customer deploying VI3 in production. At a bare minimum, a VMware Server vs. Microsoft Virtual Server comparison should have been included for comprehensiveness sake (since this is supposed to be an “unbias” report showing readers various cost options).

Also, comparing a Type 2 hypervisor (virtualization layer running on top of a host operating system), like Microsoft Virtual Server, with a Type 1 hypervisor (hypervisor running directly on top of the hardware), like VMware ESX Server, is a flawed comparison. Microsoft’s own plan to move away from Virtual Server and replace it with Hyper-V is evidence of the gap in the products. They know that a Type 2 hypervisor is architecturally dependent on the host operating system which impacts its performance and scalability. For instance, Microsoft’s customer references for Virtual Server (see Microsoft slides from TechEd 2007 as an example) consistently show a 3:1 to 5:1 consolidation ratio. On the other hand, many of VMware’s public customer references for ESX Server (see www.vmware.com/customer) achieve 10:1 and even >20:1 consolidation ratios.

3. Inaccurate reporting in the Security section

Inaccurate reporting of security bulletin: The two sentences below, as written, are misleading. The vulnerability to break out of the guest system to the host machine is only an issue on VMware’s hosted products (ex. Workstation, VMware Server). The Yankee Group report misleads readers to believe that the vulnerability affects VMware ESX Server.

"In late September, the online security alert service Bugtraq reported that VMware issued fixes to patch 20 security vulnerabilities that affected almost every product in its portfolio, including the VMware ESX Server, the VMware Server, VMware Workstation, VMware ACE and VMware Player. These vulnerabilities could have allowed hackers to break out of the guest system in the virtual machine and to halt processes on host machines." (Page 12)

Another inaccuracy in reporting: The report makes another inaccurate claim about a recent security bulletin (see quote below). First, the 20 security flaws in sentence 1 apply to both VMware hosted and ESX products. Of the 20, 5 applied to VMware’s hosted products. Therefore, the 18 attributed to ESX Server in sentence 2 is wrong. Second, only 3 (out of the 15 flaws that are applicable to VMware’s datacenter product ESX Server) were rated as high severity or critical. The majority of the others are low criticality. This fact would have been readily apparent by clicking through to view the details of the Secunia report. (Secunia summary bulletins only show the worst threat level among all the patches.)

"VMware released a patch to correct more than 20 security flaws in its various products. According to an alert issued by Secunia, a Denmark-based security service provider that tracks security flaws in software and operating systems, 18 were described as highly critical flaws that could impact just about every version of the ESX Server." (Page 12, emphasis added)

Apples-to-oranges: Comparing a hosted to a bare-metal product: Comparing vulnerabilities in Microsoft Virtual Server 2005, which is a hosted product, to all of ESX, which is a bare-metal (hypervisor) product, is a totally apples-to-oranges. If Yankee Group wants to compare to Microsoft’s hosted product, then it should compare bugs to VMware Workstation or VMware Server. If Yankee Group is going to compare a hosted product in the datacenter to a bare-metal product, it should include all the vulnerabilities in the computing stack. For Microsoft Virtual Server 2005, it needs to run on Windows Server 2003 and uses IIS. So how many critical Windows Server 2003/IIS vulnerabilities have there been since Jan 2007?

4. Other inaccuracies and unsupported claims

Have Citrix XenServer list prices really gone down?

"And in response to the increasing competition, Citrix in January announced new pricing for its offerings that represent a 20% cut." (Page 12)

As best as we’ve been able to track, it appears that Citrix XenServer list prices have actually gone up slightly, not down. At best, they’ve stayed about the same. Here’s our understanding of how Citrix XenServer list prices have changed.

Prices below are for two-socket hosts.

Old price New price
XenServer Ent Ed Perpetual 2499 2600
XenServer Ent Ed SnS 500 400
XenServer Ent Ed Annual Sub 1599 2000
XenServer Std Ed Perpetual 750 780
XenServer Std Ed SnS N/A 120
XenServer Std Ed Annual Sub 495 600


Confusion between Microsoft Virtual Server and Hyper-V

"Microsoft leveraged its greater economies of scale by embedding Virtual Server into the Windows operating system free of charge." (Page 7)

Microsoft Virtual Server R2 SP1 is not embedded into the Windows operating system. It requires a separate download and installation. This quote confused the architectures of Microsoft Virtual Server and the upcoming Microsoft Hyper-V.


Does this pricing comparison with Red Hat make sense?

"[Red Hat] retail price tags are 35% to 70% less than the same competing products from VMware." (Page 13)

Red Hat is sold only on a subscription basis and VI3 is sold in perpetual per-socket licenses, so a comparison like this is invalid unless the users plan to expire the machines in 12 months.


Who really has broader hardware support…

"’We’re way ahead of VMware in our support for I/O virtualization, native 64-bit HA as well as the total number of CPUs supported and we support a much broader set of hardware,’ Crosby claimed." (Page 16, emphasis added)

A check of VMware’s HCL and Citrix XenServer’s HCL shows that Simon Crosby’s claim about broader hardware support is unsupported. For example, VMware 3.0.2 HCL supports over 400 server platforms. XenServer’s HCL only shows 43 servers directly tested by Citrix. Even if the server systems tested by vendors are added, the Citrix HCL grows to 55 servers. Mr. Crosby’s other 2 claims about I/O virtualization and 64-bit HA are pretty unsubstantiated as well.


Out-of-date information on VMware VI3 capabilities

"The Infrastructure Enterprise software also includes some new features… DRS… HA… VCB." (Page 15)

VMware has had DRS, HA, and VCB since June 2006. On the other hand, nowhere does the report mention the truly new features of VI3 (version 3.5) that were announced in Sept 2007 and GA’ed in Dec 2007. Readers are not getting current information from this report.


VMware VirtualCenter has had intelligent placement since VI3 in June 2006.

"He added that the functionality in VMware’s corresponding virtualization management offering falls far short of the functionality in Virtual Machine Manager. ‘VMware’s Lab Manager development tool focuses on desktops, not servers and even their new Virtual Data Center tool does not incorporate the intelligent placement feature of the Microsoft product,’ he said." (Page 11, emphasis added)

VMware VirtualCenter has had intelligent placement capabilities (for when a new VM is provisioned) since June 2006 with the launch of Virtual Infrastructure 3.

One thought on “Those Darn Details… Issues in Yankee Group’s Virtualization Report

  1. Jason Boche

    Thank you for this. I’m open to comparisons and competition but it is important the facts are straight on all sides.

Comments are closed.