Home > Blogs > Virtual Reality > Monthly Archives: February 2008

Monthly Archives: February 2008

Those Darn Details… Issues in Yankee Group’s Virtualization Report

Yankee Group recently came out with a report “Virtualization Price War: VMware’s Little Big Horn?” that unfortunately contained quite a few factual errors, inaccuracies, and unsupported claims. The report was not reviewed with VMware prior to publication (as some have claimed). Therefore, VMware is listing the factual errors and inaccuracies in the report because they distort the true cost and value of VMware’s solution. Happy reading.

1. Exhibit 2: Double counting, omissions, and other issues

Exhibit 2 indiscriminately mixes VMware’s bundle and ala carte pricing, making it very difficult for readers to get an actual apples-to-apples cost comparison. If the table is meant to be a cumulative account of Microsoft and VMware pricing (i.e. add up all the rows), then there is significant double-counting in the VMware column. If the rows in the table are meant to be interpreted individually, then there are gross inconsistencies between how VMware and Microsoft’s costs are shown. Most casual readers will assume the table represents cumulative cost for both companies since they see the word “Included” throughout Microsoft’s column. The points below are written from the cumulative cost standpoint.

  1. Double counting on cost of VMware: Row 1 of Exhibit 2 calls out the list price for VMware Infrastructure 3 Enterprise Edition ($5750) in the VMware column. However, the table continues by calling out the ala carte pricing for VMware individual capabilities that are ALREADY included in the price of VI3 Enterprise Edition (i.e. VMotion, DRS, HA, vSMP). This table effectively double-counts the cost for VMware. VMware offers ala carte pricing for customers who wish to purchase one of VMware’s other editions and add features individually. No one would buy Enterprise Edition plus many of the ala carte options in Exhibit 2 because most are included in the Enterprise Edition. There is a reference to a Mr. David Dodge on page 9 of the report that points out this issue but the feedback was apparently ignored.
  2. Double counting on OS costs for VMware: The cost of a Windows Server Enterprise Edition License is counted twice for VMware – once under “Guest Licensing” and again under “Windows Server”. In this configuration, VMware products only require 1 Windows license (just like Microsoft), not 2. Again, Mr. David Dodge calls this out on page 9 of the report – “Dodge also argued that the VMware ESX3 solution can install on bare-metal and does not need an operating system to boot.” Therefore, the “Windows Server” for VMware should be $0 (the cost under “Guest Licensing” already covers it).
  3. Wrong VMware product for a “single server configuration” comparison: The top of page 7 states that Exhibit 2 is a comparison of cost for a “single server configuration.” If that is the case, then why wasn’t VMware Server (further discussion in section 2 below) or the VMware Infrastructure Foundation Acceleration Kit used? The analyst is clearly aware of the acceleration kit because it is cited on page 6 of the report. Such bundles from VMware are specifically targeted at this segment. Also, if this is a single server configuration, when why would a customer want VMotion, DRS, HA which is only useful in a multi-server configuration?
  4. Lack of clarity on management costs: Exhibit 2 fails to point out that a Microsoft System Center Enterprise Management Suite License (ESML) is $860 PER managed HOST (not including the mandatory cost of 2-years of Microsoft SA) while a VMware VirtualCenter Management Server license is a perpetual license independent of the number of managed hosts (VirtualCenter agents come with the VI3 license). By purchasing 1 VirtualCenter Management Server license, a customer can manage 5, 10, 15, 200, etc. number of ESX hosts. Also, where is meant by “Additional Charge $3,750 for Standard” (in the Management row)?
  5. Error – Microsoft Virtual Server does NOT support vSMP: Microsoft Virtual Server does not support virtual SMP. That’s one of the key differentiators that Microsoft calls out between Virtual Server and Hyper-V.
  6. Microsoft features not on-par with VMware features: Microsoft Virtual Server + SCVMM R1 offers nothing like DRS. SCVMM R1 can only recommend a physical server for initial placement when a VM is first provisioned. It has NO ability to dynamically move the VM (based on customer defined resource requirements) within a cluster of hosts to maximize resources. Also, Microsoft’s self-service provisioning is much less capable compared to VMware Lab Manager. For instance, Lab Manager handles self provisioning and management of multi-tier environments. SCVMM R1 offers nothing comparable.
  7. Omission of key features in VMware VI3 Enterprise Edition: The report does not call out many capabilities that customers get if they purchase VI3 Enterprise Edition today. If the report is going to compare VI3 Enterprise Edition to Microsoft Virtual Server, then it should show all of the core capabilities of Enterprise Edition to provide readers an accurate picture of what they are getting. Omitted VI3 Enterprise Edition features include:
    • Storage VMotion (no MS equivalent)
    • Update Manager (no MS equivalent)
    • Guided Consolidation (no MS equivalent)
    • 64-bit guest OS support (no support in MS Virtual Server)

2. Why wasn’t VMware Server compared to Microsoft Virtual Server?

On page 7, the report states the following for why VMware Server was not used in this comparison.

"Customers do have the option of deploying VMware’s free hypervisor technology to even out the cost differential between VMware and competing server operating system-based hypervisors. However, according to the Yankee Group 2006 and 2007 Global Virtualization Surveys, which polled 800 end users worldwide, only 10% of corporations are deploying the free version of VMware for production networks." (Page 7)

This rationale is a double standard – what percentage of Microsoft Virtual Server deployments are currently in “production networks?” VMware’s internal research and anecdotal evidence suggests that MS Virtual Servers’s use in product environments is low. Certainly, it is not as high as the >85% of VMware customer deploying VI3 in production. At a bare minimum, a VMware Server vs. Microsoft Virtual Server comparison should have been included for comprehensiveness sake (since this is supposed to be an “unbias” report showing readers various cost options).

Also, comparing a Type 2 hypervisor (virtualization layer running on top of a host operating system), like Microsoft Virtual Server, with a Type 1 hypervisor (hypervisor running directly on top of the hardware), like VMware ESX Server, is a flawed comparison. Microsoft’s own plan to move away from Virtual Server and replace it with Hyper-V is evidence of the gap in the products. They know that a Type 2 hypervisor is architecturally dependent on the host operating system which impacts its performance and scalability. For instance, Microsoft’s customer references for Virtual Server (see Microsoft slides from TechEd 2007 as an example) consistently show a 3:1 to 5:1 consolidation ratio. On the other hand, many of VMware’s public customer references for ESX Server (see www.vmware.com/customer) achieve 10:1 and even >20:1 consolidation ratios.

3. Inaccurate reporting in the Security section

Inaccurate reporting of security bulletin: The two sentences below, as written, are misleading. The vulnerability to break out of the guest system to the host machine is only an issue on VMware’s hosted products (ex. Workstation, VMware Server). The Yankee Group report misleads readers to believe that the vulnerability affects VMware ESX Server.

"In late September, the online security alert service Bugtraq reported that VMware issued fixes to patch 20 security vulnerabilities that affected almost every product in its portfolio, including the VMware ESX Server, the VMware Server, VMware Workstation, VMware ACE and VMware Player. These vulnerabilities could have allowed hackers to break out of the guest system in the virtual machine and to halt processes on host machines." (Page 12)

Another inaccuracy in reporting: The report makes another inaccurate claim about a recent security bulletin (see quote below). First, the 20 security flaws in sentence 1 apply to both VMware hosted and ESX products. Of the 20, 5 applied to VMware’s hosted products. Therefore, the 18 attributed to ESX Server in sentence 2 is wrong. Second, only 3 (out of the 15 flaws that are applicable to VMware’s datacenter product ESX Server) were rated as high severity or critical. The majority of the others are low criticality. This fact would have been readily apparent by clicking through to view the details of the Secunia report. (Secunia summary bulletins only show the worst threat level among all the patches.)

"VMware released a patch to correct more than 20 security flaws in its various products. According to an alert issued by Secunia, a Denmark-based security service provider that tracks security flaws in software and operating systems, 18 were described as highly critical flaws that could impact just about every version of the ESX Server." (Page 12, emphasis added)

Apples-to-oranges: Comparing a hosted to a bare-metal product: Comparing vulnerabilities in Microsoft Virtual Server 2005, which is a hosted product, to all of ESX, which is a bare-metal (hypervisor) product, is a totally apples-to-oranges. If Yankee Group wants to compare to Microsoft’s hosted product, then it should compare bugs to VMware Workstation or VMware Server. If Yankee Group is going to compare a hosted product in the datacenter to a bare-metal product, it should include all the vulnerabilities in the computing stack. For Microsoft Virtual Server 2005, it needs to run on Windows Server 2003 and uses IIS. So how many critical Windows Server 2003/IIS vulnerabilities have there been since Jan 2007?

4. Other inaccuracies and unsupported claims

Have Citrix XenServer list prices really gone down?

"And in response to the increasing competition, Citrix in January announced new pricing for its offerings that represent a 20% cut." (Page 12)

As best as we’ve been able to track, it appears that Citrix XenServer list prices have actually gone up slightly, not down. At best, they’ve stayed about the same. Here’s our understanding of how Citrix XenServer list prices have changed.

Prices below are for two-socket hosts.

Old price New price
XenServer Ent Ed Perpetual 2499 2600
XenServer Ent Ed SnS 500 400
XenServer Ent Ed Annual Sub 1599 2000
XenServer Std Ed Perpetual 750 780
XenServer Std Ed SnS N/A 120
XenServer Std Ed Annual Sub 495 600


Confusion between Microsoft Virtual Server and Hyper-V

"Microsoft leveraged its greater economies of scale by embedding Virtual Server into the Windows operating system free of charge." (Page 7)

Microsoft Virtual Server R2 SP1 is not embedded into the Windows operating system. It requires a separate download and installation. This quote confused the architectures of Microsoft Virtual Server and the upcoming Microsoft Hyper-V.


Does this pricing comparison with Red Hat make sense?

"[Red Hat] retail price tags are 35% to 70% less than the same competing products from VMware." (Page 13)

Red Hat is sold only on a subscription basis and VI3 is sold in perpetual per-socket licenses, so a comparison like this is invalid unless the users plan to expire the machines in 12 months.


Who really has broader hardware support…

"’We’re way ahead of VMware in our support for I/O virtualization, native 64-bit HA as well as the total number of CPUs supported and we support a much broader set of hardware,’ Crosby claimed." (Page 16, emphasis added)

A check of VMware’s HCL and Citrix XenServer’s HCL shows that Simon Crosby’s claim about broader hardware support is unsupported. For example, VMware 3.0.2 HCL supports over 400 server platforms. XenServer’s HCL only shows 43 servers directly tested by Citrix. Even if the server systems tested by vendors are added, the Citrix HCL grows to 55 servers. Mr. Crosby’s other 2 claims about I/O virtualization and 64-bit HA are pretty unsubstantiated as well.


Out-of-date information on VMware VI3 capabilities

"The Infrastructure Enterprise software also includes some new features… DRS… HA… VCB." (Page 15)

VMware has had DRS, HA, and VCB since June 2006. On the other hand, nowhere does the report mention the truly new features of VI3 (version 3.5) that were announced in Sept 2007 and GA’ed in Dec 2007. Readers are not getting current information from this report.


VMware VirtualCenter has had intelligent placement since VI3 in June 2006.

"He added that the functionality in VMware’s corresponding virtualization management offering falls far short of the functionality in Virtual Machine Manager. ‘VMware’s Lab Manager development tool focuses on desktops, not servers and even their new Virtual Data Center tool does not incorporate the intelligent placement feature of the Microsoft product,’ he said." (Page 11, emphasis added)

VMware VirtualCenter has had intelligent placement capabilities (for when a new VM is provisioned) since June 2006 with the launch of Virtual Infrastructure 3.

Corrections to ZDNet blog ‘Microsoft’s Hyper-V puts VMware and Linux on Notice’

ZDNet posted a blog reviewing Microsoft Hyper-V beta “Review: Microsoft’s Hyper-V puts VMWare and Linux on notice” (February 14, 2008). The posting had some factual errors and other claims about VMware ESX Server – see below for a summary. The author of the blog has acknowledged the errors in the comments section of the blog but since most readers won’t read through all the comments, here is a summary of the correct statements about VMware ESX Server as well as counterpoints to various claims.

Claim: VMware ESX Server does not support SATA disks.

"… SATA disk drives, which are now commonplace on commodity x86 server machines, are not currently supported in VMWare ESX 3."

Correction: VMware ESX Server 3.5 supports SATA disks. The author has acknowledged this in the comments section.

Claim: VMware ESX Server always requires VMFS and a dedicated SAN-based LUN to store virtual machine images.

“ESX Server also requires a special networked clustered file system known as VMFS to store the virtual machine images, and you have to dedicate a SAN-based LUN to it.”

Correction: ESX Server provides four options for VM storage: 1) Fiber channel (FC) attached, 2) iSCSI attached, 3) NFS, and 4) local storage. Only the FC and iSCSI options require a SAN-based LUN. Also, the NFS option does not use VMFS. Again, the author has acknowledged this error in the comments section. Note that Citrix XenServer and Microsoft Hyper-V also require shared storage to perform VM migration and other high availability functions.

Claim: VMware is a closed system. Therefore, VMware development goes at a much slower pace. VMware’s lack of SATA disk support is an example of this slower pace.

“VMWare keeps its ESX hypervisor source code very close to the vest, so development goes at a much slower pace – SATA disk drives, which are now commonplace on commodity x86 server machines, are not currently supported in VMWare ESX 3.”

Counterpoint: VMware’s purported lack of SATA disk support has already been refuted (see above). In addition, VMware has delivered more industry-breaking innovations around x86 virtualization compared to anyone else. As examples, VMware was the first to deliver virtual SMP, live migration (VMotion), centralized VM management (VC), dynamic load balancing (DRS), automatic VM restart after host failure (HA), next-generation thin embedded hypervisor (ESX Server 3i with a 32MB disk footprint), and Storage VMotion. The majority of other vendors still have not delivered these capabilities. VMware has also provided ESX Server source code to partners as part of the VMware Community Source Program. One good example of the output from the Community Source Program is the Mellanox Infiniband support.

Claim: VMware ESX Server’s hardware compatibility list (HCL) severely limits a customer’s hardware choices while Microsoft and Xen do not have this issue.

"However, [ESX Server] has a much tighter environment as to what kind of hardware it can run on – the hypervisor has a limited device driver compatibility list…"

Counterpoint: Both Microsoft Hyper-V beta and Xen-based products (ex. Citrix XenServer, Virtual Iron) have HCLs as well. Click here to see Microsoft’s list of hardware platforms “suitable” for Hyper-V beta. Click here to see Citrix XenServer’s HCL. Click here to see Virtual Iron’s HCL. The Microsoft, XenServer, and Virtual Iron HCLs are actually more limited than VMware’s HCL. For instance, Microsoft has directly tested Hyper-V beta on 8 servers (TBD what the hardware requirements will be for Hyper-V GA). Citrix has directly tested 45 servers. Virtual Iron has directly certified 29 servers (for version 4). In contrast, VMware has over 400 servers on the ESX Server 3.0.2 HCL. Click here to see VMware’s HCL.

Claim: VMware ESX Server is “pure software based virtualization”

"VMWare’s ESX differs from Hyper-V and Xen in that it currently uses pure software based virtualization…”

Counterpoint: This statement can potentially mislead readers to think that ESX Server is based on software emulation technology. This conclusion would be false. For the bulk of x86 CPU instructions, ESX Server does direct execution on the physical CPU. Readers may also interpret this statement to say that VMware is somehow philosophically opposed to using CPU hardware-assist. Again, this would be a false conclusion. VMware takes a pragmatic approach and uses whichever technology delivers the best performance. Currently VMware binary translation has advantages over today’s generation of CPU hardware-assist. Click here to read a VMware whitepaper on this topic. In regards to the future, VMware is already working closely with Intel and AMD on their next-generation of CPU hardware-assist technology.