Check out this Microsoft Hyper-V ad that we found yesterday.  I have to say, that is definitely a cool robot! But, perhaps they couldn’t find real customers to spotlight in the ads? ;)

And what’s the robot drinking? Is that Microsoft Kool-Aid?  Actually, what I found really interesting was that the Robot was named, “IT 24-7”, implying continuous uptime for applications and other IT services run on Hyper-V, but did you notice the asterisk? And when you read the fine print at the very bottom of the ad… ;)

All in good fun – Happy Halloween or something...

First, Thank-you for the Feedback

I wanted to quickly say thanks to everyone who provided feedback on our most recent post comparing the installation and configuration of VMware ESXi versus Windows Server Hyper-V with Server Core -- Microsoft’s recommended deployment option.  You may not agree with all of the conclusions we presented, but we just couldn’t let Microsoft Exec Bob Muglia go unchallenged in claiming that Hyper-V is simply “The Windows You Know” and therefore an easier product to use than ESXi – it isn’t.  But thank-you, as many of you had very insightful (inciteful?) comments and we got some good, healthy debate going. For a pretty fair Microsoft response, please refer to James O’Neil’s blog .

But “Apples to Oranges”??  They’re Both Hypervisors!

But there was one piece of feedback, stated in comments by a number of Microsoft employees readers, that puzzled me. Some people cried foul because they saw our evaluation of Hyper-V with Server Core and ESXi as somehow comparing apples to oranges – I guess that was because Windows Server Hyper-V with Server Core requires a full instance of a general purpose operating system as its parent partition and ESXi does not. The comments/bloggers suggested that a more fair comparison would be ESXi vs. Hyper-V Server 2008, since Hyper-V Server is supposedly Microsoft’s ‘thin’ hypervisor that doesn’t require Windows Server OS in the parent partition - as reported by Microsoft here.  (Note: the MSFT blog linked there incorrectly states that ESXi has a Linux parent partition. That is untrue, ESXi has no parent partition.)

Well, regarding “Apples to Oranges” I am not going to dwell too much on that one, because in my opinion, ESXi and Hyper-V (all configurations) are both Hypervisors and are both aiming to serve the same purpose within a customer’s datacenter, so therefore the comparison is valid.  And, to support that notion, Microsoft compares all versions of Hyper-V to ESX/ESXi in every one of its virtualization presentations, so I think they’re in agreement with us that it is a fair comparison.  However, given that, if you want us to compare ESXi to Hyper-V Server, sure, now that the product is finally available, we can talk about that one too.

Hyper-V Server – Initial Thoughts

Ever since Microsoft first announced Hyper-V Server, almost a year ago, we’ve been speculating as to what it would look like.  It was billed as “standalone”, but until right before its release, Microsoft provided no technical details, so we were all left in the dark.   Existing Hyper-V versions were wholly dependent on Windows Server, so how “thin”, how “standalone” could it really be?

(Note: I am actually thinking that, at the time of Hyper-V server’s announcement, Microsoft itself didn’t know what the Hyper-V Server 2008 architecture would look like…)

Well, now that Hyper-V Server 2008 has finally been released - with very little fanfare considering its initial push from Microsoft - we were able to perform a preliminary evaluation.   There were two things we were initially interested in: 1) How the Hyper-V Server deployment/configuration processes compare to ESXi – gotta answer our critics, and 2) How Hyper-V Server architecture compares to ESXi – is it a more “apples to apples” comparison, or does Hyper-V Server contain Windows Server OS and is it therefore subject to all the patches, updates, vulnerabilities of the other configurations of Hyper-V?

We’ll save tackling the first issue -- comparing the install/configure processes – for another blog post.  While our initial eval tells me that the install/config process hasn’t improved with Hyper-V Server, it will still take a little time to undertake a complete analysis.  But the second item – understanding what components of Windows Server the Hyper-V Server actually contains, how the architecture compares to ESXi, and what the benefits of Hyper-V Server actually are – we can start that discussion here.

Hyper-V Server is not “Windows-less” but is merely “Windows License-less”

Our initial finding is that Hyper-V Server is not “thin”; Hyper-V Server is still ultimately Windows.  Hyper-V Server appears for the most part to be just Windows Server Hyper-V with Server Core where all other Server Core roles (except Hyper-V) have been disabled. Hyper-V Server has practically the same footprint as Windows Server Hyper-V with Server Core and is subject to the same patches, updates, attacks. It also appears to have the same restricting, indirect Windows-based driver model. In fact, it seems that the only advantage of Hyper-V Server is that one doesn’t have to buy a Windows Server license in order to deploy it – that’s it.   Hyper-V Server is not “Windows-less”, but just “Windows License-less.”

Hyper-V Server also has some significant limitations that it seems to have inherited from the Standard Edition of Windows Server 2008. It can only support a maximum of 4 sockets per host, 32GB of physical memory per host, 31GB of virtual memory per VM, and requires a rip and replace upgrade to support features like Microsoft Clustering and Quick Migration. So it seems that Hyper-V Server is more of a starter kit, meant only for very basic use cases. In comparison, ESXi is a fully functional, production ready, enterprise offering. Actually, as 1) Both ESXi and Hyper-V Server are free and 2)Only free ESXi can easily be upgraded via license key to a production solution, why would anyone ever use Hyper-V Server? What’s the advantage?

Hyper-V Server – An Overview of Our Installation Experience

For proof points supporting our above conclusions, following is a blow-by blow of Michael Hong’s experience installing Hyper-V Server:

I got my 936MB iso of Hyper-V Server downloaded. I burned it onto a CD and popped it into my brand new HP DL 360 and fired it up. After doing some recommended BIOS configurations and rebooting, I’m watching the boot sequence and getting a feeling of déjà vu. Did I just put in the wrong DVD? Because I swear this looks exactly like a full Windows Server 2008 or even a Server Core installation.

Wait…did that just say, “Installing Windows?” I thought this was Hyper-V Server that wasn’t supposed to be Windows! At this point I’m thinking, “Hey, maybe that’s not too bad. I can get a free copy of Windows without having to deal with any of their licensing nightmares.” Well let’s wait and see before I get too excited…

Hyper-V Server Disk Footprint is Similar to Hyper-V Server Core!

Okay, so after THREE more reboots I’m finally able to log in and start looking around. The first thing I check is Hyper-V Server’s disk footprint. After all, Microsoft states that one of its only three key benefits is a “small footprint.” So how “small” is it really? After plotting the numbers into my trusty byte convertor, Hyper-V Server is coming in around 2.5GB! (pagefile not included in size) WOW, that’s only a hundred megabytes less than a full-blown Windows Server Core installation! Perhaps it really just is a Windows Server Core Standard edition with one role enabled. Anyone else have any thoughts on this?

Also notice the number of files and directories. My basic install of Windows Server Core with Hyper-V enabled has:

In this install of Hyper-V Server there is actually more files and directories:

Next, let’s take a look at patching. Option number 5 in the dos-like Hyper-V Configuration menu shows an option for enabling Windows Update. Once set to automatic, it scanned for applicable patches. I didn’t expect to see any new patches since Hyper-V Server was just released yesterday. Any new patches would probably arrive next patch Tuesday right? And since this is supposed to be a light, secure hypervisor, it probably wouldn’t need as many patches as a full blown OS right? The results may surprise you:

13 applicable patches, including 2 for Internet Explorer 7? This is looking more and more like the “Windows I Know.” What in Hyper-V Server actually relies on IE7? Hyper-V Server looks like it’s a full blown Windows OS. If that’s the case, I’m sure hackers will have a field day copying over few additional files and turning it into a full working copy of Windows Server Core.

Some other things to keep in mind:

· Server Core and Hyper-V Server have the same directory structures

· Server Core and Hyper-V Server have the same command line toolset

· Server Core Standard and Hyper-V both have the same 32GB of physical memory limit and up to 4 processors

· Server Core and Hyper-V Server have the same parent partition driver model

Is Hyper-V Server really Windows Server Core Standard with only the Hyper-V role enabled? If so, will it be vulnerable to the same threats as Windows? Those 13 patches are just the starting point. What about viruses? Windows Server Core is exposed to viruses and as a result, there are anti-virus products out there today that are certified on Server Core. What about the size of Hyper-V Server’s footprint? Being only 100MB smaller than Server Core still shows it still has a very large attack surface as compared to VMware ESXi.

In Sum

We feel that, in contrast to how it is being marketed, Hyper-V Server 2008 is not “standalone”, “thin”, or  Operating System agnostic in the same manner as ESXi. Hyper-V Server is still very dependent on and subject to the limitations of Windows and therefore should not be considered an equal to ESXi.   Also, given Hyper-V Server’s restrictions and lack of upgrade path, and given that ESXI is also free and has a simple upgrade path, I question what the viable use cases for Hyper-V Server really are. Give it a try yourself and let us know what you think.  Better yet, download our free VMware ESXi and let us know how you feel it compares to Hyper-V Server.

Stay tuned for our comparison of the deployment and configuration processes for ESXi and Hyper-V Server.

Memory overcommit, aka the ability of VMware ESX to provision memory to virtual machines in excess of what is physically available on a host, has been a topic of discussion in virtualization blogs for quite some time (e.g., “More on Memory Overcommitment”) and apparently still is (e.g. “VMware vs. Microsoft: Why Memory Overcommitment is Useful in Production and Why Microsoft Denies it” and “Microsoft responds to VMware’s ability to overcommit memory” ).

Given the benefits of memory overcommit and the fact that today only VMware ESX/ESXi has it as a standard feature, it is understandable that other vendors try to downplay it by advocating that it is irrelevant, dangerous and not used in production environments. Microsoft’s position on the topic is particularly interesting….or confusing, I should say. On one side, in an interview Bob Muglia, Microsoft VP, confirmed the usefulness of memory overcommit, announcing plans to add it to their hypervisor some time in the future (have you heard this line before?), but then on the other side they don’t miss an opportunity to speak against it. James O’Neill, also from Microsoft, in his blog even challenged us to provide a reference of a customer that is actually using it in production, promising in return to make a charitable donation of $270 to an organization of our choice.

Anyway, internally at VMware we certainly have no doubts about the importance and effectiveness of memory overcommit, but we felt that after all this discussion among vendors, and after all the confusion from Microsoft as to whether it is/isn’t important and is/isn’t on the Hyper-V roadmap, that it might be more interesting for you to hear directly from our customers. Therefore, the bulk of this post will document a survey of memory overcommit usage among VMware customers. You’ll hear directly from VMware users regarding how they leverage ESX memory overcommit in their production datacenters, with no impact to performance, to increase VM density and further reduce VMware’s already low cost per application – the most relevant metric of virtualization TCO.

(Side Note: I bet MSFT will no longer question the value of overcommit once they are finally able to list it as an upcoming Hyper-V feature),

Before jumping into the survey results, I think a few clarifications are necessary.

What is memory overcommit?

Here I won’t go into all the granular, technical details of how memory overcommit works, because there is already a ton of great literature available that explains what it is and how it works (e.g., “The Role of Memory in VMware ESX Server 3” ).

However, there are a couple points that I’d like to make regarding the functionality of and requirements for Memory Overcommit.

Memory Overcommit: Required Components

Memory overcommit is the combination of three key ingredients:

1. Transparent memory page sharing
2. Balloon driver
3. Optimized algorithms in the hypervisor kernel

These 3 elements must all be present and work together seamlessly. One alone is not enough , unlike what some vendors would like people to believe (see “Ballooning is more than enough to do memory overcommit on Xen, Oracle says”). To date, only VMware ESX has all the necessary components, has had them since 2001, and has continued to improve them ever since.

Memory Overcommit: Security Impact

Transparent memory page sharing de-duplicates memory pages by sharing the identical pages among VMs. In doing so, it makes the shared pages “read-only” at the physical RAM level. If the VM tries to write to it, ESX will get a callback and it will create a private copy of the page for the VM that wants to write to it, while letting the other VMs continue to use the original shared page. Marking it read-only ensures that it is a secure technology, one VM won’t be able to affect any other VMs. However, if you need additional assurance of Memory Overcommit’s security, you should keep in mind that VMware ESX, with its Memory Page Sharing feature, is the only hypervisor in the market that has earned a Common Criteria Evaluation Assurance Level 4 (EAL4+) under the CSEC Common Criteria Evaluation and Certification Scheme (CCS). Therefore, only VMware ESX is approved for use in “sensitive, government computing environments that demand the strictest security.”

Why is memory overcommit important?

Memory overcommit enables customers to achieve higher VM density per virtual host, increasing consolidation ratios and providing a more efficient scale up - scale out model. Ultimately this translates into substantial savings and a lower cost per application than with alternative solutions, as Eric Horschman shows in his blog post.

While the declining cost of memory could suggest that hypervisors with no memory overcommit can get away without it, in reality throwing more memory at the problem is not a sustainable solution for a few reasons:

• The number of VMs deployed grows over time

• Going forward systems will be even more memory constrained than today as the number of CPU cores per server will increase considerably faster than memory capacity. As a matter of fact, in 2011 a two sockets system is expected to be capable of 64 logical CPUs and 256GB of RAM, whereas today the same system is probably capable of 8 logical CPUs and 64GB of RAM. This means that the ability of a hypervisor to efficiently manage memory will be an even more critical factor to minimize the number of servers required to run applications and ensure efficient scalability.

• Memory capacity requirements aren’t determined only by application workloads, but also by a number of valuable IT services, such as: high availability, zero downtime system maintenance, power management and rapid system provisioning. Virtualization solutions that don’t allow memory overcommit corner customers into a lose-lose situation: either reduce system utilization or don’t provide the service. Thanks to memory overcommit, our customers tell us that they were able to reduce their dependence on available physical resources, avoid unnecessary purchases, and improve infrastructure utilization. (see below for few examples on how VMware customers use memory overcommit)

Enough with the clarifications - let’s move on to the customer survey ….

We conducted an online survey of 110 VMware customers essentially asking them three questions:

1. Do you use memory overcommit?
2. Do you use memory overcommit in test/dev, production or both?
3. What is your virtual-to-physical memory ratio per ESX host (i.e., overcommit ratio)?

Here are the results:

1) 57% answered they are using memory overcommit

……so much for “nobody uses it”

2) Of the 57% who answered yes, 87% said they use it in production and test and development, 2% only in production, 11% only in test/dev

……so much for “nobody uses it in production”

3) Finally, plotting the virtual-to-physical ratios on a chart, we can see what usage looks like. Virtual-to-physical memory ratios ranged from 1.14 to 3 (average 1.8, median 1.75). 75% of the respondents use memory overcommit ratios of 1.5 or higher and 37% utilize a ratio of 2 or higher

.…..so much for “memory overcommit ratios must be low”

What the chart can’t show is that, based on our findings, companies at the low end of the memory overcommit usage spectrum tend to be recent VMware customers, while those at the high-end tend to be long standing VMware customers. This looks very similar to what we have seen happening with other VMware technologies such as VMotion: once people try it and they see how well it works, they want to extract its potential.

I believe this data clearly demonstrates that VMware customers use memory overcommit in production systems and do so with high virtual-to-physical ratios.

Finally, here is what few customers who use memory overcommit in production have to say about it:

Kadlec Medical Center - Large 188-bed hospital in southern Washington State with over 270 medical staff members and over 10,000 annual patient admissions.

“Memory overcommit is one of the unique and powerful features of VMware ESX that we leverage everyday in our production environments. Thanks to memory overcommit, we were able to increase the consolidation of production environment by over 50%, maximizing utilization without giving up on the performance of our production systems. We appreciate that VMware makes it available to customers as a standard feature of ESX” – Tim Harper, Sr. System Analyst, Kadlec Medical Center

WTC Communications - regional phone, cable, Internet provider in Kansas

"A small business like ours derived tremendous benefits from the ability of VMware ESX to overcommit memory. We cannot afford the big IT budget of a large enterprise, so we must get the most out of our production servers while guaranteeing SLAs with our customers. This is exactly what VMware ESX memory overcommit allowed us to achieve. We were able to consolidate 35 production virtual machines (both Linux and Windows) on just 3 Dell PowerEdge 2850 servers with 8GB RAM each. Typically we run our production servers at an average ratio of 1.25 virtual-to-physical memory, however during maintenance operations, the ratio increases to 1.88 as we VMotion VMs out of the host that undergoes maintenance completely transparently to the users. Memory overcommit adds unparallel flexibility to our infrastructure and saves us a lot of money not just by allowing higher consolidation, but also by eliminating the need for spare capacity to perform routine maintenance operations. Memory overcommit is a fully automated feature of ESX and it is extremely simple to use. It is really a no brainier.” -- Jim Jones, Network Administrator, WTC Communications

U.S. Department of Energy - Savannah River

"Our virtualization effort began 4 years ago, and we have made great strides in server utilization since then. After upgrading to VI3, we took advantage of VMware memory overcommit. We now routinely overcommit memory at a 2:1 ratio in our production environments and have even reached 3:1 on occasion. We even run large applications such as Lotus Domino and SQL server 2008 in VMs but this has not been an issue - no performance impact. As a result, we fully trust VMware memory overcommit in our production environments. Our IT budget is tight so in the past we have had to wait over 6 months to receive a new server. By using memory over commit, we can now deploy a system in less than 30 minutes without waiting for a new server. This keeps our internal customers very happy," - Joseph Collins, Senior Systems Engineer, U.S. Department of Energy – Savannah River

We wouldn't be doing our jobs at VMware if we didn't regularly compare our products with the competition to ensure our customers get the best technology and user experience possible.  In keeping with that practice, we recently set up Microsoft's Hyper-V to get a first hand look.  We made sure to follow Microsoft's documentation and best practices guidance to get a fair comparison and to understand exactly what a Hyper-V user experiences as he or she attempts to deploy and configure Microsoft's new product.

One key best practice we heard Microsoft's Hyper-V team stress in sessions at June's TechEd conference and again last week at VMworld was a strong recommendation to run Hyper-V using the Server Core variant of Windows Server 2008.   Using the smaller Server Core as the Hyper-V parent partition, instead of a full blown instance of Windows Server 2008, strips out Windows features and services not needed to run Hyper-V.  With Server Core, Microsoft is attempting to minimize the attack surface and patching requirements for Windows to make it a safer platform for virtual machines.  I would agree that as Hyper-V requires a general purpose operating system, you might as well make it as small as possible.  The Server Core concept seems like a good idea.  So, following Microsoft's recommendations, we deployed Hyper-V with Server Core.

Server Core -- "The Windows You Know"??

One aim in evaluating Hyper-V was to test its end-user experience, as Microsoft execs repeated over and over at their Sept. 8 virtualization event that Hyper-V would be eagerly adopted because it uses, "the Windows you know."  The insinuation is that Hyper-V is easy -- and of course that somehow VMware is not.  Microsoft is claiming that with Hyper-V there is no added learning required, no training, no classrooms, because you already know Windows, you can jump right into Hyper-V. They are also claiming that VMware ESX requires you to take the time to learn a whole new system.  But is this accurate?  Is the recommended Server Core flavor of Windows 2008 really, "the Windows you know"?  Is it easier than ESX?  We wanted to find out.

Windows Server Core = MS-DOS 2008

If you haven't seen Server Core yet, here's the UI in its entirety.  It doesn't look like the Windows I know, in fact it looks like DOS!  Are we stepping back in time?  Who knows DOS anymore?  Actually, it makes you wonder why Microsoft didn't just call it MS-DOS 2008, especially since anyone using Server Core will need to resurrect some long lost command line skills to get any work done.

Is Hyper-V with Recommended Server Core, In Fact, Easy?

So, how does the Hyper-V and DOS -- err, I mean Server Core -- combination stack up when compared to the user experience of VMware ESXi?  To try it out, we did side-by-side installations of Microsoft Server Core/Hyper-V and VMware ESXi 3.5 on identical servers.  To let you see the details of each setup process, we recorded the entire sequence in a pair of videos.

This first video shows every step required to install Hyper-V and ESXi on a fresh machine.  We kept count of the elapsed time, reboots, mouse clicks and keystrokes each product needed and it clearly shows the huge advantage the truly thin and OS-free ESXi architecture has in installation speed and simplicity.  ESXi goes from bare-metal to fully installed in one-third the time, half the mouse clicks, hundreds fewer keystrokes and just one reboot vs. seven compared to Hyper-V.  The simplicity of the ESXi wizard-driven installation is striking compared to the arduous process needed to first get the Server Core OS installed and then configure Hyper-V in a command line environment.

Our second video starts where the first left off and takes Hyper-V and ESXi through the steps needed to configure two iSCSI datastores for VM use.  iSCSI setup is a standard task for any virtualization user that wants to take advantage of shared storage for VM migration and high availability.  ESXi's Windows-based Virtual Infrastructure client makes the iSCSI setup quick and easy.  For Hyper-V, the "Windows you know" is nowhere to be seen.  Instead, working with Server Core requires you to key in a long sequence of obscure commands to configure iSCSI initiators and targets, partitions and file systems.  We generously showed the Hyper-V setup executed with no delays, although it took us hours of digging through Microsoft documents and knowledgebase articles to find the right commands to use when configuring iSCSI in Server Core.

Our Conclusion: Server Core plus Hyper-V is for Experts Only

VMware has put great effort into making ESXi the easiest and fastest hypervisor to install and configure and these videos clearly show the results.  Getting the OS out of the hypervisor plays a big part in the streamlined simplicity of ESXi as there is no general purpose OS to configure and manage and the reliability and security issues accompanying the tens of millions of lines of code an OS brings along are eliminated.  Microsoft's OS-centric Hyper-V architecture adds many steps to the setup and puts their users in a quandary: either A) they install Hyper-V on a full Windows Server 2008 OS and deal with the frequent patching and security fixes Windows requires; or, B) they follow Microsoft's best practice guidelines and suffer with the limitations of Server Core.  As the videos show, the tradeoffs with Server Core are daunting -- Windows administrators will find their familiar GUI tools are missing and they'll be left to spend a lot of quality time with search engines tracking down documentation on Microsoft's obscure command line utilities.

Take a look at the side-by-side comparison videos and let us know if you agree that ESXi provides a far faster and easier (or maybe we should say, "moister and chewier") setup experience.  Better yet, try ESXi and Hyper-V with Server Core on your own machines and tell us how it went.

The Link:   http://www.vmware.com/technology/whyvmware/

What’s Real and What’s Noise?

I think we’d all agree that there is currently a whole lot of noise in the virtualization space.  With the recent explosion of virtualization entrants, each one claiming to offer a complete, proven, production ready enterprise virtualization solution that is “good enough/better/cheaper/easier/faster/more secure/more scalable/tastes great/less filling than VMware, it is downright deafening. In one corner we’ve got Microsoft execs claiming that their first generation hypervisor is “1/3 the cost of VMware” (Note: Our response here), and even the long dormant Red Hat recently erupted to tell everyone that with Qumranet they can now run more VMs per host than VMware VI3 (Note: I didn’t see any data to support that claim, but of course someone reported on it).  With everyone shouting over everyone else, each claiming to be better than or equal to VMware , I am going deaf over here!

And I am sure it is getting very difficult for those evaluating a virtualization solution to tell the facts from the marketing.  Given all the noise, how can anyone determine who actually has a solution that can be deployed today and who is merely marketing a vision (dream?) that may or may not be realized anytime soon?  How can anyone cut through the clatter and make a confident, sound decision on a virtualization platform to support both current and future virtualization requirements?  

Here’s Our Story – One Complete Picture of Our Competitive Differentiation

To make your decision easier, we wanted to make sure that at the very least, you had a complete understanding of our side of the story. We wanted you to have a single location from which to obtain the entire picture, all the facts, on VMware’s product differentiation,  so we created the “Why Choose VMware” website.  Our goal was to clearly and succinctly articulate why we feel that that 1) VMware is the only vendor that actually does have the complete, robust, production proven virtualization solution that can meet your current virtualization requirements today and can support you in the future as your IT needs evolve and 2) All other offerings fall short of meeting even basic virtualization requirements.  How’s that for succinct? :)

Just for background on the site’s material, we met with customers, partners, and analysts to determine what they felt was important when considering a virtualization solution; on this site, we make our case for how we match up to those requirements.   I won’t repeat the whole argument here, but in summary, we articulate how VMware’s solutions meet the following conditions:

1. Is built on a robust, proven foundation
2. Delivers a platform for shared IT services
3. Provides a complete solution for virtualization management
4. Supports your entire IT infrastructure
5. Is proven across tens of thousands of customer deployments.

And yes we also address cost – and show why VMware solutions offer the lowest cost per application of any virtualization solution – even when compared to those that claim to be free.

I do apologize however…

While we did try, the next releases of VMware products will NOT be “moist and chewy like cake”. I wouldn’t recommend trying to “eat them while you work”.  I extend our sincerest apologies.  Oh well, at least we have live migration.

Walking out of the keynote at Microsoft’s “Get Virtual Now” event in Bellevue, WA this week, I found myself thinking about how this was the second time that Microsoft has pre-announced that they would have live migration for Hyper-V. (Apparently, I was not the only one -- see “Microsoft Backtracks on Live Migration, Again” on internetnews.com.)

For any readers new to virtualization, live migration is the ability to move a running virtual machine (VM) from one physical server to another without the VM end-user experiencing any disruptions. This capability enables IT admins to do things like:

• Perform planned maintenance at anytime (instead of only during evenings, weekends, or maintenance windows),
• Perform anytime dynamic load balancing to meet real-time application demands, and
• Save power by consolidating VMs to fewer servers during non-peak hours.

VMware was the first to deliver live migration in 2003 when we released VMware VMotion. Since then, others have followed VMware’s lead and delivered live migration (such as various Xen-variants in 2007). According to our customers, live migration via VMotion has become an indispensable component of their virtualized production datacenters.

Microsoft Live Migration: Take One

The first time Microsoft pre-announced live migration for Hyper-V was in 2006 when they stated that “Viridian” (codename for Hyper-V back then) would ship with live migration and other capabilities that would surpass VMware ESX. Then in May 2007, Microsoft had to retreat and drop live migration, hot-add memory and CPU, and support for 32 logical cores from Hyper-V 1.0 in order to prevent Hyper-V from slipping further. (Remember, these dropped capabilities were the ones that Microsoft previously touted as reasons Hyper-V would surpass VMware ESX.)

Microsoft Live Migration: Take Two

So now we get the second pre-announcement on live migration which says that live migration will be available in Windows Server 2008 R2. According to what Microsoft had previously told press, R2 is slated for 2010. Onstage Bob Muglia stated something to the effect of “there’s no magic in live migration, it is just a feature…” Well if it is as easy as the statement implies, why doesn't Microsoft have the functionality now? Why will it take until 2010?

Now you may say, “Yes, Microsoft is late, but I’m ok waiting.” But waiting costs your company real dollars. Look at this simple example: By using VMware VMotion for planned server maintenance in a 150-VM environment you can save almost $60,000 a year in operational costs. If we scaled to a 1000-VM environment, it results in almost $400,000 of cost savings a year. If you use VMotion for more than planned server maintenance, and use it for dynamic load balancing, distributed power management, etc, you’ll save even more!

Live Migration is a Core Virtualization Requirement

To pre-announce live migration – twice – shows that even Microsoft has realized the foundational role that live migration plays in a virtual datacenter. It’s not a “nice-to-have” but a “must-have” capability. Microsoft used to claim that Microsoft Quick Migration was “good enough” because it only caused seconds to minutes of downtime – but that downtime causes interruptions to the end user and that is unacceptable. Microsoft must have gotten enough grief from customers over this claim because I don’t see them saying it as much any more. Anyone who has actually used VMware VMotion knows that you can’t afford the downtime -- look at this demonstration of how disruptive the downtime caused by Microsoft Quick Migration is to an end-user.

Assuming no further slips, when Microsoft delivers live migration in 2010, it will be seven years behind VMware (VMotion in 2003). Today, VMware has distinct advantages over Microsoft in our VMware Infrastructure 3 product, such as Storage VMotion, logical resource pools, DRS, and in our Application and Management Infrastructure products, like VMware Lab Manager, Stage Manager, Site Recovery Manager, Lifecycle Manager. By 2010, VMware will extend our leadership even further – keep an eye on announcements coming at VMworld 2008.

Microsoft Did Demo Live Migration, but…

As a technical aside: Microsoft’s live migration demo during Bob Muglia’s keynote felt kinda weird (see for yourself - available in .mpg or .wmv) with what seemed like a streaming video player super-imposed on top of the Cluster Administrator console. During the actual migration, the video was stretched to full screen, obscuring the Cluster Administrator console, and then shrunk back down once the migration was declared as completed. I realize the video was supposed to show that the live migration did not disrupt the playing video, but it wasn’t very convincing since streaming video can be cached. Not sure Microsoft cares what I think, but I have some suggestions to make it a more compelling demo:

• Show an RDP session into the virtual machine being migrated so people can see what happens to someone logged into the virtual machine
• Show a continuous “ping” to the virtual machine being migrated so people can see any downtime of that virtual machine
• Show the management console during the entire migration so people can see what the admin will see
• Show a more complete process of how to initiate a virtual machine live migration, ie. select target host, test for compatibility, etc.

Here’s an example of how we’ve demoed VMotion.

Once again, another report recently came out predicting VMware’s doom in light of the formal market entry of Microsoft Hyper-V. This time, the predictions of fire and brimstone come from Clabby Analytics in the form of a paper titled “Six Reasons Microsoft’s Hyper-V Will Overtake VMware and Become the Major Player in the x86 Server Virtualization Marketplace.” This theme, VMware’s upcoming battle, seems to be a favorite topic among certain analysts, bloggers, and press nowadays as it makes for an eye-catching headline and probably drives up ad revenues for certain bloggers. But the reality is that 1) there is little substance behind these reports to back the dramatic headline, and 2) the authors’ conclusions are often based on incomplete analysis and generic “conventional wisdom” / thinking-of-the-masses.

(Quick Note: Microsoft also seems to like these types of reports as evidenced by how quickly the Clabby Analytics report showed up on microsoft.com for download. Or did Microsoft somehow have a hand to play in this report? Hmmm….)

Now, don’t get us wrong, Microsoft is a formidable competitor whenever they enter any market. No company should ever treat Microsoft’s entry lightly – that would be foolish. But here’s the reality: No one (including these pundits) really knows what will happen over the next 3-5 years in the fast changing virtualization market. There are lots of predictions flying around – everyone puts forth their hypotheses. It is then up to readers to judge them on the merits of the arguments. In judging the Clabby Analytics report on its merits, one quickly sees that the report’s arguments are fairly shallow (repeating generic claims that may or may not determine the final outcome) and the analysis is incomplete (conveniently leaves out much that is in VMware’s favor.)

So the way we see it, with the GA launch of Hyper-V, Microsoft has finally shipped a product that at least allows it to compete. No more excuses about Microsoft Virtual Server being a hosted product or Hyper-V being in beta. Let’s compete in the marketplace and let our respective companies’ ingenuity, innovation, and execution (and subsequent customer adoption) be the determiner of the outcome. At this point, a report declaring one company’s demise is premature at best, and a thinly veiled marketing tactic at worst.

Game on.

Six reasons cited by the Clabby Analytics report and how to interpret them:

• Price
• Packaging
• Depth
• Reach
• Control/integration
• Installed base

Reach and Installed Base Arguments

Let’s look at Reach and Installed base first.

• For Reach, the report basically sums up the argument like this: “In short, Microsoft is significantly larger than VMware.”
• The Installed base argument follows a similar line of logic: “Microsoft has a massive installed base, expected to cross 1 billion users by the end of 2008. By contrast, VMware claims to have 100,000 customers. Clearly Microsoft has a huge edge here…” (My note: how many of those 1 billion users are Windows Server users?)

The common message behind both of these arguments seems to be 1) Microsoft is bigger and has more resources, 2) Microsoft has a massive existing presence, and therefore, VMware doesn’t stand a chance. While the two statements about Microsoft’s size and presence are factually true, it’s kinda like saying “Bill Gates and Steve Ballmer are wealthy individuals” – factually true, but not terribly insightful. If one extends this argument to its logical conclusion, then all the Davids of the world, past present and future, might as well never bother getting out of bed to confront their respective Goliaths – the status quo will remain the status quo and the incumbents will remain the incumbents. Whether you buy into this line of reasoning or not determines whether you buy into 2 of the report’s 6 reasons – that’s one third of the report’s substance right there. (Incidentally, I’m sure that similar arguments were made when Microsoft first began its battles against IBM back in the 80s.)

Price Argument

Next, let’s take a look at the Price argument. The report shows a comparison of upfront software license acquisition costs between VMware and Microsoft. Some discrepancies and issues jump out right away.

1. Wrong Comparison
The report compares VMware Infrastructure (VI3) Enterprise Edition (VMware’s higher-end SKU) to Microsoft’s offering but leaves out the fact that Microsoft’s solution lacks core virtualization capabilities found in VI3 Enterprise Edition, such as:

• VMware ESXi (ultra-thin hypervisor)
• VMware VMotion (live VM migration)
• VMware Storage VMotion (live storage migration)
• VMware Dynamic Resource Scheduling (dynamic load balancing)
• VMware Distributed Power Management (automated power savings)
• VMware VMFS (distributed journaling file system built for virtualization).

A much more appropriate product comparison would have been to compare the Microsoft Hyper-V hypervisor to VMware ESXi which lists for $495 per license. And for small businesses that want a full virtual infrastructure over just a hypervisor, VMware offers the VI3 Foundation Edition and VMware SMB Acceleration Kits at very affordable price points. The report fails to mention any of these other VMware offerings.

2. VM Density Matters
Before virtualization, IT would run one application per physical server as the best practice. Therefore, for 10 hypothetical applications, people would commonly calculate the cost to run 10 servers as a proxy for the cost to run those 10 applications. The Clabby Analytics report takes this outdated approach. But with virtualization, things change dramatically – IT now runs many applications (each in its own VM) per physical server. So to figure out the cost of running those 10 applications, one must know how many VMs can run per physical server with acceptable performance – a metric called “VM density per physical host.” VMware invested in technologies to achieve very high VM density on ESX while maintain high levels of responsiveness. In fact, ESX can commonly run twice the number of VMs compared to competitors – on the same physical server. This higher density leads to less servers, licenses, power, cooling, and datacenter space. Therefore, due to VMware’s ability to achieve greater VM density per server, VMware’s cost per application, or cost per virtual machine, is much less than competing virtualization offerings. There is no mention of this important metric in the Clabby Analytics report.

One final note on Price: It’s interesting that Microsoft is pitching lower cost of upfront acquisition with Hyper-V while VMware’s focus is on total-cost-of-ownership and value. It wasn’t that long ago that Microsoft started arguing total-cost-of-ownership and value as to why people should not purchase Linux. Microsoft still has a full website dedicated to this argument. Funny how Microsoft’s position has flip-flopped when it comes to Hyper-V.

Depth Argument

Clabby Analytics argues that Depth is where Microsoft has its greatest advantage over VMware. Quote: “However, when it comes to actual management of physical and virtual servers combined, Clabby Analytics argues that Microsoft is far deeper from a management and infrastructure perspective than VMware.” Again, it is true that VMware VirtualCenter does not manage physical non-virtualized servers. Our strategy has always been to open up our management APIs so that existing systems management vendors can have deep integrations with VirtualCenter (whether it be HP, IBM, CA, BMC, Symantec, Quest, NetIQ, even Microsoft, to name a few). This way, customers can keep whatever systems management tool they’ve already purchased, and VMware integrates with it – preserving a customer’s existing management investment and best practices, and reducing disruption caused by introducing virtualization into the datacenter. With the recent expansion of Microsoft System Center to manage Linux and Unix platforms, it appears that Microsoft is really setting itself up to compete against the existing systems management vendors (like HP, IBM, CA, BMC, Symantec, Quest, NetIQ, etc.). VMware, on the other hand, partners with these vendors so the customer can leverage whatever they’ve already got for systems management.

Packaging and Control/Integration Arguments

That leaves Packaging and Control/integration as the final 2 of the 6 reasons.

On Packaging, the Clabby Analytics report states that Hyper-V’s distribution as part of Windows Server 2008 is a key advantage – it makes it easy for people to get and try Hyper-V. Yep – this tactic has always been part of Microsoft’s long-standing product strategy of leveraging Windows and distribution through PC OEMs. But the report should also have mentioned that customers can order VMware ESXi embedded from all major server OEMs. Nor does it mention how IT admins can have a new ESXi-embedded server up and running within minutes of pulling it out of the box.

Regarding Control/integration, the report states that VMware products are an “add-on” to Windows and because of this, VMware is at Microsoft’s mercy when it comes to working with Windows. First, in what way is VMware a “Windows add-on”? Is there some confusion with GSX Server here? VMware ESX is certainly no Windows add-on since it runs bare-metal on the hardware. Second, the notion that Microsoft will somehow use control of Windows to disadvantage other vendors gets into tricky territory. Isn’t that the type of behavior that got Microsoft in trouble with the DOJ and EU in the first place?

Conclusion

So judge for yourself. Do the arguments in the Clabby Analytics report make a strong case for VMware’s demise? Or is it an incomplete, one-sided analysis that basically repeats generic arguments that have been proven in the past to not be universal predictors of outcome?

Our customers have been asking us for an explanation of the key differences between the VMware ESX hypervisor architecture and the Windows-based Hyper-V architecture they've been hearing about recently from Microsoft.  We put together this summary explaining the elements of the ESX architecture that we believe set it apart from Hyper-V and Xen and the reasons behind some of our design decisions.  We thought it would be interesting material for the readers of this blog, so take a look and tell us what you think...

VMware Infrastructure - Architecture Advantages

VMware Infrastructure is a full data center infrastructure virtualization suite that provides comprehensive virtualization, management, resource optimization, application availability and operational automation capabilities in a fully integrated offering. VMware Infrastructure virtualizes the entire IT infrastructure, including servers, storage and networks and aggregates these heterogeneous resources into a simple and uniform set of computing resources in the virtual environment. With VMware Infrastructure, IT organizations can manage resources as a shared utility and dynamically provision them to different business units and projects without worrying about the underlying hardware differences and limitations.

Complete Virtual Infrastructure

As shown in the preceding figure, VMware Infrastructure can be represented in three layers:

1. The base layer or virtualization platform is VMware ESX – the highest performing, production-proven hypervisor on the market. Tens of thousands of customers deploy VMware ESX (over 85 percent in production environments) for a wide variety of workloads.

2. VMware Infrastructure’s support for pooling x86 CPU, memory, network and storage resources is the key to its advanced data center platform features. VMware Infrastructure resource pools and clusters aggregate physical resources and present them uniformly to virtual machines for dynamic load balancing, high availability and mobility of virtual machines between different physical hardware with no disruption or downtime.

3. Above the virtual infrastructure layers sits end-to-end application and infrastructure management from VMware that automates specific IT processes, ensures disaster recovery, supports virtual desktops and manages the entire software lifecycle.

VMware ESXi – The Most Advanced Hypervisor

VMware ESXi 3.5 is the latest generation of the bare-metal x86 hypervisor that VMware pioneered and introduced over seven years ago. The industry’s thinnest hypervisor, ESXi is built on the same technology as VMware ESX, so it is powerful enough to run even the most resource-intensive applications; however, it is only 32 MB in size and runs independently of a general-purpose OS.

The following table shows just how much smaller the VMware EXSi installed footprint is compared to other hypervisors. These are results from installing each product and measuring disk space consumed, less memory swap files.

Comparative Hypervisor Sizes (including management OS)

As the numbers show, ESXi has a far smaller footprint than competing hypervisors from vendors that like to label ESX as "monolithic."

The ESXi architecture contrasts sharply with the designs of Microsoft Hyper-V and Xen, which both rely on a general-purpose management OS – Windows Server 2008 for Hyper-V and Linux for Xen – that handles all management and I/O for the virtual machines.

The VMware ESX direct driver architecture avoids reliance on a heavyweight Windows or Linux management partition OS.

Advantages of the ESX Direct Driver Architecture

Our competition negatively portrays VMware ESX Server as a “monolithic” hypervisor, but our experience and testing proves it to be the best design.

The architecture for Citrix XenServer and Microsoft Hyper-V puts standard device drivers in their management partitions. Those vendors claim this structure simplifies their designs compared to the VMware architecture, which locates device drivers in the hypervisor. However, because Xen and Hyper-V virtual machine operations rely on the management partition as well as the hypervisor, any crash or exploit of the management partition affects both the physical machine and all its virtual machines. VMware ESXi has done away with all reliance on a general-purpose management OS, making it far more resistant to typical OS security and reliability issues. Additionally, our seven years of experience with enterprise customers has demonstrated the impressive reliability of our architecture. Many VMware ESX customers have achieved uptimes of more than 1,000 days without reboots.

One of our customers sent us this screenshot showing four years of continuous ESX uptime.

The VMware direct driver model scales better than the indirect driver models in the Xen and Hyper-V hypervisors.

The VMware ESX direct driver model puts certified and hardened I/O drivers directly in the VMware ESX hypervisor. These drivers must pass rigorous testing and optimization steps performed jointly by VMware and the hardware vendors before they are certified for use with VMware ESX. With the drivers in the hypervisor, VMware ESX can provide them with the special treatment, in the form of CPU scheduling and memory resources, that they need to process I/O loads from multiple virtual machines. The Xen and Microsoft architectures rely on routing all virtual machine I/O to generic drivers installed in the Linux or Windows OS in the hypervisor’s management partition. These generic drivers can be overtaxed easily by the activity of multiple virtual machines – exactly the situation a true bare-metal hypervisor, such as ESXi, can avoid. Hyper-V and Xen both use generic drivers that are not optimized for multiple virtual machine workloads.

VMware investigated the indirect driver model, now used by Xen and Hyper-V, in early versions of VMware ESX and quickly found that the direct driver model provides much better scalability and performance as the number of virtual machines on a host increases.

The scalability benefits of the VMware ESX direct driver model became clearly apparent when we tested the I/O throughput of multiple virtual machines compared to XenEnterprise, as shown in the preceding chart from a paper published here. Xen, which uses the indirect driver model, shows a severe I/O bottleneck with just three concurrent virtual machines, while VMware ESX continues to scale I/O throughput as virtual machines are added. Our customers that have compared VMware ESX with the competition regularly confirm this finding. Similar scaling issues are likely with Hyper-V, because it uses the same indirect driver model.

Better Memory Management with VMware ESX

In most virtualization scenarios, system memory is the limiting factor controlling the number of virtual machines that can be consolidated onto a single server. By more intelligently managing virtual machine memory use, VMware ESX can support more virtual machines on the same hardware than any other x86 hypervisor. Of all x86 bare-metal hypervisors, only VMware ESX supports memory overcommit, which allows the memory allocated to the virtual machines to exceed the physical memory installed on the host. VMware ESX supports memory overcommit with minimal performance impact by combining several exclusive technologies.

Memory Page Sharing

Content-based transparent memory page sharing conserves memory across virtual machines with similar guest OSs by seeking out memory pages that are identical across the multiple virtual machines and consolidating them so they are stored only once, and shared. Depending on the similarity of OSs and workloads running on a VMware ESX host, transparent page sharing can typically save anywhere from 5 to 30 percent of the server’s total memory by consolidating identical memory pages.

Transparent Page Sharing.

Memory Ballooning

VMware ESX enables virtual machines to manage their own memory swap prioritization by using memory ballooning to dynamically shift memory from idle virtual machines to active virtual machines. Memory ballooning artificially induces memory pressure within idle virtual machines as needed, forcing them to use their own paging areas and release memory for more active or higher-priority virtual machines.

Memory Ballooning.

VMware ESX handles memory ballooning by using a pre-configured swap file for temporary storage if the memory demands from virtual machines exceed the availability of physical RAM on the host server. Memory overcommitment enables great flexibility in sharing physical memory across many virtual machines, so that a subset can benefit from increased allocations of memory, when needed.

Memory Overcommit Provides Lowest Cost of Ownership

The result of this memory conservation technology in VMware ESX is that most customers can easily operate at a 2:1 memory overcommit ratio with negligible performance impact. Our customers commonly achieve much higher ratios. Compared to Xen and Microsoft Hyper-V, which do not permit memory overcommit, VMware Infrastructure customers can typically run twice as many virtual machines on a physical host, greatly reducing their cost of ownership.

TCO Benefits of VMware Infrastructure 3 and its better memory management.

The table above illustrates how a conservative 2:1 memory overcommit ratio results in a lower TCO for even our most feature-complete VMware Infrastructure 3 Enterprise edition, compared to less functional Microsoft and Xen offerings.

Storage Management Made Easy with VMFS

Virtual machines are completely encapsulated in virtual disk files that are either stored locally on the VMware ESX host or centrally managed using shared SAN, NAS or iSCSI storage. Shared storage allows virtual machines to be migrated easily across pools of hosts, and VMware Infrastructure 3 simplifies use and management of shared storage with the Virtual Machine File System (VMFS.) With VMFS, a resource pool of multiple VMware ESX servers can concurrently access the same files to boot and run virtual machines, effectively virtualizing the shared storage and greatly simplifying its management.

VMware VMFS supports and virtualizes shared storage.

While conventional file systems allow only one server to have read-write access to the file system at a given time, VMware VMFS is a high-performance cluster file system that allows concurrent read-write access by multiple VMware ESX servers to the same virtual machine storage. VMFS provides the first commercial implementation of a distributed journaling file system for shared access and rapid recovery. VMFS provides on-disk locking to ensure that multiple servers do not power on a virtual machine at the same time. Should a server fail, the on-disk lock for each virtual machine is released so that virtual machines can be restarted on other physical servers.

The VMFS cluster file system enables innovative and unique virtualization-based distributed services. These services include live migration of running virtual machines from one physical server to another, automatic restart of failed virtual machines on a different physical server, and dynamic load balancing of virtual machines across different clustered host servers. As all virtual machines see their storage as local attached SCSI disks, no changes are necessary to virtual machine storage configurations when they are migrated. For cases when direct access to storage by VMs is needed, VMFS raw device mappings give VMware ESX virtual machines the flexibility to use physical storage locations (LUNs) on storage networks for compatibility with array-based services like mirroring and replication.

Products like Xen and Microsoft Hyper-V lack an integrated cluster file system. As a result, storage provisioning is much more complex. For example, to enable independent migration and failover of virtual machines with Microsoft Hyper-V, one storage LUN must be dedicated to each virtual machine. That quickly becomes a storage administration nightmare when new VMs are provisioned. VMware Infrastructure 3 and VMFS enable the storage of multiple virtual machines on a single LUN while preserving the ability to independently migrate or failover any VM.

VMFS gives VMware Infrastructure 3 a distributed systems orientation that distinguishes it from our competition.

VMware Infrastructure 3 is the first virtualization platform that supports pooling the resources of multiple servers to offer a new array of capabilities. The revolutionary DRS and HA services rely on VMFS features to aggregate shared storage, along with the processing and network capacity of multiple hosts, into a single pool or cluster upon which virtual machines are provisioned. VMFS allows multiple hosts to share access to the virtual disk files of a virtual machine for quick VMotion migration and rapid restart while managing distributed access to prevent possible corruption. With Hyper-V, Microsoft is just now rolling out a first-generation hypervisors with a single node orientation. It lacks distributed system features like true resource pooling, and it relies on conventional clustering for virtual machine mobility and failover.

VirtualCenter – Complete Virtual Infrastructure Management

A VirtualCenter Management Server can centrally manage hundreds of VMware ESX hosts and thousands of virtual machines, delivering operational automation, resource optimization and high availability to IT environments. VirtualCenter provides a single Windows management client for all tasks called the Virtual Infrastructure client. With VirtualCenter, administrators can provision, configure, start, stop, delete, relocate and remotely access virtual machines consoles. The VirtualCenter client is also available in a web browser implementation for access from any networked device. The browser version of the client makes providing a user with access to a virtual machine as easy as sending a bookmark URL.

VMware VirtualCenter centrally manages the entire virtual data center.

VirtualCenter delivers the highest levels of simplicity, efficiency, security and reliability required to manage a virtualized IT environment of any size, with key features including:

• Centralized management
• Performance monitoring
• Operational automation
• Clustering and pooling of physical server resources
• Rapid provisioning
• Secure access control
• Full SDK support for integrations

I'll stop there for now.  All the management and automation and VDI services depicted in the top layer of the figure at the beginning of this post further set us apart from the competition.  Services like Update Manager, SRM, Lab Manager and VDM offer amazing capabilities, but we'll save that discussion for some upcoming posts.

We Reviewed Microsoft's ROI/TCO Model

Some of you may have seen Microsoft’s recently released virtualization ROI/TCO calculator. Briefly, the model purports to offer an accurate cost/benefit comparison between Microsoft’s Hyper-V offering and a “Competitive Server Virtualization Solution” – gee I wonder who the competitive solution is…?  Microsoft is beginning to advertise the calculator broadly in its partner newsletter and other email blasts – and we’ve even had customers bring it to our attention. Mainly, the VMware customers that have alerted us to the Microsoft ROI/TCO calculator were confused by many of the model’s assumptions and by the generated results - they wanted our opinion. So, we took a look.

Unfortunately We Had to Give It A Failing Grade

Of course the results were all hypothetical, because Hyper-V is not yet available, but what we found when running a realistic scenario through the model and then from reading the report’s fine print, is that like most Microsoft version 1.0 products, the initial release of this calculator has numerous errors, contains critical design mistakes, and completely misses its mark. Any results generated from this model are so unrealistic as to be completely worthless for accurately comparing costs and benefits of alternate virtualization solutions. (Maybe we all need to wait for the SP1?)

In Sum:   ROI/TCO Analysis = Good Idea ; Inaccurate Model = Bad Idea

But of course that’s what one would expect VMware to say right? Out comes a model that could show that a competitor’s product is less expensive and right away VMware is going to question its accuracy. Well I’ll soon list out the numerous issues we found with the Microsoft model – and those issues will articulate clearly why Microsoft’s model is inaccurate and why our questioning of the model is justified, but first just let me state that we at VMware fully support an accurate and realistic ROI/TCO analysis of our solutions. In fact, VMware launched an ROI/TCO calculator way back in April of 2007. Since then, our customers have generated over 30,000 reports that quantify the enormous benefits they are receiving from VMware installations. (Feel free to try it for yourself here.) However, we also believe, and I am sure you’d agree, that any ROI/TCO model must be based on 1) accurate pricing/licensing information, 2) fair product comparisons, and 3) defendable assumptions for both associated costs and realized benefits. Our analysis of the Microsoft supported model showed that it had none of those traits.

Play Along at Home

Following are some of the many mistakes and gross assumptions we found in the Microsoft model. So that you can follow along at home – please download a copy of the report we generated. I encourage you to evaluate the Microsoft calculator yourself – let us know what else you find! Also, as you read further, and as you consider all the errors in the model, keep in mind the following question:

Why Did MSFT Release Such a Misleading ROI/TCO Model?

A) Microsoft did a sloppy and hasty job with the calculator

B) Microsoft is deliberately fudging the facts

C) Both A and B

Now I don’t want to imply that Microsoft is trying to deliberately mislead anyone, but it does make me question Microsoft’s overall capacity for rigorous product testing. I hope that the same team that QA’d the Microsoft TCO model is not the same group that is currently testing the V1.0 release of Hyper-V!

Final Thought

Perhaps we at VMware could work with Microsoft and a third party to develop a standardized model for an ROI/TCO analysis of a virtualization solution? Thoughts?

Production & Dev/Test Server Virtualization – Competitive Cost Comparison
(Appendix B – page 18)

Why is a discounted Microsoft license cost being compared to VMware List Pricing??
The tool assumes the following pricing for Windows Server 2008 Editions: $719 for Standard, $2334 for Enterprise, and $2381 for Datacenter. That doesn’t match up to the list pricing that Microsoft publishes on its website, so the prices must be some sort of volume/discounted pricing. So the tool uses discounted pricing for Microsoft and list pricing for VMware ($5750 for VI3 Enterprise)? Not exactly an apples-to-apples comparison. One might argue that Microsoft does not have access to VMware volume pricing. But then again, the average reader doesn’t have access to Microsoft’s volume pricing either.

Where are the Microsoft System Center Server License Costs?
Under the “Virtualization Management Software” row, the VMware column includes the server license cost for VirtualCenter $5000. That’s fine, but to the best of our knowledge, an organization using the MSFT solution would also need server licenses, for 1) Microsoft System Center Operations Manager, 2) System Center Configuration Manager, and 3) System Center Data Protection Manager. Each component is $573 (list price) according to Microsoft’s own website. That $573 doesn’t include the SQL Server license – that’s extra. Currently, no additional server license cost is required for System Center Virtual Machine Manager 2007 - TBD whether that will remain true for Virtual Machine Manager 2008. System Center Operations Manager is required to do any performance tracking, Configuration Manager to do updates and patches, and Data Protection Manager to do backups. Most of that functionality is an integrated part of VMware VirtualCenter.

Since When Did All VMware Customers Decide to Run System Center?
The tool assumes that customers will use Microsoft System Center to manage their VMware Infrastructure deployment. Again, this little detail is buried deep in the appendices (Appendix B – “System Center Enterprise” row). Microsoft assumes that everyone who still chooses to run VMware will naturally want to manage it using Microsoft System Center. This seems like a pretty presumptuous assumption. Most companies we talk to have already invested in an enterprise management framework and want VMware to integrate with what they’ve already got. Unless they are already running Microsoft System Center, they really don’t want to rip out their existing investment and put in System Center. VMware VirtualCenter APIs are readily available through the VirtualCenter SDK making it possible for VMware partners like IBM, HP, CA, BMC, Symantec, Quest, and NetIQ (to name a few) to integrate seamless with VMware VirtualCenter. Customers can keep using what they’ve already invested in.

The Model Incorrectly Calculates Microsoft Licensing Costs
In the scenario we ran, Microsoft’s tool assumed 71 Windows Server Standard Edition licenses for 414 virtual machines running on 71 hosts. Since each Standard Edition license grants rights to run 1 VM, the model’s results leave 343 VMs in our hypothetical datacenter running out of compliance. Microsoft may claim that the TCO/ROI calculator is not a licensing calculator, but how can it calculate accurate TCO estimates using inaccurate licensing assumptions? We did find a one-line disclaimer buried in the 66-page document: “Warning – Check pricing advice and rules as the automated recommendations here may not reflect all licensing rules.” Come on, guys - licensing is such a basic component for accurate TCO estimates. The disclaimer feels pretty weak.

Microsoft Cost Savings are HUGE… if You Believe the Assumptions
We found many overly aggressive assumptions in the Microsoft calculator, which resulted in overly aggressive cost savings for the Microsoft’s solution. For instance, on page 26 “Production Server Operations and Administration Efficiency Savings”, Microsoft’s default assumption is an 83% reduction in person-hours-per-year after virtualizing with Microsoft’s solution. That translates into a VM-to-admin ratio of 250:1 after virtualization. While VMware agrees that virtualization absolutely increases administrative efficiency, 250:1 is overly aggressive.

Why are 4-way Servers the Model’s Default Assumption? Oh yeah… It Increases VMware’s Perceived Cost.
The tool assumes 4-way servers as the default configuration. Granted, 4-way server volume is growing, but IDC’s Quarterly Server Tracker published on February 26, 2008, shows only 4.27% of all servers shipped in 2007 were 4-ways, making them nowhere close to being the majority of servers virtualized today. This assumption does double the number of VMware Infrastructure 3 (VI3) Enterprise Edition licenses required since VI3 is licensed in 2-socket increments, thereby making VMware’s total deployment cost appear much higher. Most readers won’t see this point buried deep in the appendices; perhaps that was the goal? Oh, with quad-cores today and hexa/octo-cores in the near future, 2-way servers are still projected to be the majority volume virtualized server platform in 2008 and 2009.

VM Density Per Host Matters for TCO – a Fact that Microsoft Ignores
Not all virtualization platforms are created equal – some can run more virtual machines per host than others, thereby resulting in lower total cost of ownership, due to reduced hardware and related costs. In one of our recent blogs, we highlighted the lack of memory sharing technologies in Microsoft Hyper-V. This deficiency results in fewer virtual machines per host (due to inefficient memory usage), thereby increasing the number of required physical servers, thereby increasing cost. Microsoft’s tool assumes that Hyper-V will run as many VMs as VMware VI3 and deliver the same performance – we can’t wait until Hyper-V ships and prove this wrong.

VMware VI3 Enterprise Functionality is Far Greater than Hyper-V
Microsoft compares Hyper-V to VMware VI3 Enterprise Edition even though Hyper-V lacks key enabling capabilities that are prerequisites for a truly dynamic datacenter. Capabilities missing from Hyper-V include live VM migration (VMware VMotion and Storage VMotion), automated/dynamic load balancing (VMware DRS), dynamic power management (VMware DPM), integrated offline VM patching (VMware Update Manager), a next-generation ultra-thin hypervisor architecture (VMware ESXi), a clustered file system (VMware VMFS), and the ability to make VMs highly available without the storage management headache of assigning 1 LUN for each VM (VMware HA). For the basic server consolidation scenarios that Microsoft Hyper-V enables, VMware offers VI3 Foundation and VMware ESXi at very competitive price points – check out VMware’s complete server virtualization product offerings here.

Desktop and Application Virtualization – Competitive Cost Comparison

(Appendix B – page 19)

Microsoft Tool Assumes Wrong Costs for VMware VDI
To virtualize 6000 clients, Microsoft has incorrectly assumed 240 VI3 Enterprise licenses ($1,380,000) plus “Virtualization Management Software and Host Server” cost ($7000) plus 6000 connection broker client licenses ($300,000). The actual license cost of VMware VDI is 6000 x $150 = $900,000 plus virtualization management costs of $1250. This VMware VDI pricing has been in effect since Feb 2008 so the Microsoft calculator does not use up-to-date, publicly available information.

Readers Must Figure Out that Microsoft Compares Two Completely Different Technologies
By looking at the summary table on page 6, most readers would reasonably assume that the comparison is between two comparable products addressing the same business need. That’s not the case. Microsoft is comparing the cost of Terminal Services + MDOP (including Microsoft Application Virtualization) to the cost of a VMware VDI solution. VMware VDI offers full desktop VM virtualization, best in class application compatibility and true user isolation. Comparing terminal services + application virtualization to VDI is an apples-to-oranges comparison.

Microsoft Assumes Low Consolidation Ratios for VMware VDI
The consolidation ratios assumed by Microsoft for VDI are significantly lower than the consolidation ratios observed by VMware customers. On a 2-socket dual core server, VMware customers typically observe average consolidation ratios of 32-36 VMs depending upon the workloads. Microsoft only assumes 25 VMs per host, which skews VMware’s cost by another 20-30%. Of course, Microsoft assumes a higher consolidation ratio (30:1) for its own solution.

Questionable Assumptions for How Microsoft’s Desktop Solution Lowers Cost
We found some questionable assumptions on how Microsoft got some of the cost-savings estimates. For instance on page 39 “Desktop Virtualization Client Software License Cost Avoidance“, Microsoft claims that companies can reduce the number of “unused / not properly allocated licenses” by 70% using their desktop solution. Apparently, overbuying of Microsoft software licenses is a common issue. Another example is on page 37 “Desktop Virtualization Branch Office Server Cost Avoidance.” Microsoft claims that some branch office server infrastructure may be consolidated / retired and replaced with centralized virtualized servers for branch office applications. While this is one viable way to reduce IT capital cost, what about the additional IT capital costs to upgrade to low latency, highly available network connectivity to the branch offices? This network cost needs to be factored in.

Long-deferred vendor visions of agile data centers are finally coming true now that VMware virtualization with VMotion live migration has severed the ties that kept services fixed to x86 hardware.  Unfortunately, some vendors are trying to stage a revival with an inferior substitute for live migration.  Most notably, Microsoft is claiming that their "Quick Migration" feature is comparable to VMware VMotion and adequate for enterprise data centers, even though Quick Migration is not true live migration. We've even heard Microsoft tell audiences that our customers don't trust VMotion enough for production use. Don't fall for it -- VMotion is ready, proven and in heavy use today by VMware customers who are bringing true flexibility and agility to their IT operations.

Do you remember the many grand visions for IT that were trotted out by the vendors and analysts during the dot com boom times? Adaptive Enterprise Computing, Next Generation Data Centers, Organic IT, On-demand Computing, Utility Computing and more were relentlessly pitched to CIOs with PowerPoint promises of continuously available services effortlessly floating on pools of servers and storage, finding the resources they needed all by themselves and magically recovering from any faults and disasters that should arise.  CIOs put up with the daydreaming until the vendors were finally shamed into backing off on the hard sell by their noticeable inability to deliver on the promises. The technology, especially in the x86 world, just could not break the bonds that kept applications and services firmly welded to their physical hosts.

The phenomenal growth of virtualization is now reviving some of those grand IT visions. With a virtualization layer that includes live migration, x86 workloads can float free of the fixed servers and storage hardware that enterprises have in place. And, thanks to tools like VMware VMotion that live migrates servers between hosts and VMware Storage VMotion that allows transparent relocation of a VM's storage, those workloads finally can accomplish that floating without the slightest interruption to users and services. It’s not just VMware that is enabling this revolution-in-waiting; the Xen vendors are also starting to roll out their own live migration support.

It should not be surprising then, that Microsoft is using its entry into the virtualization market to bring its own grand architecture – the dormant “Dynamic Systems Initiative” – out of hibernation. Now apparently renamed as “Dynamic IT,” their vision was featured in Bob Muglia's January 21 V-day missive to hundreds of thousand of Microsoft customers and partners.  In laying out the benefits of virtualization and live migration, we couldn’t have said it better ourselves:

"In the data center, virtualization not only supports server consolidation, but it enables workloads to be added and moved automatically to precisely match real-time computing needs as demand changes. This provides greater agility, better business continuity, and more efficient use of resources."

That “moved automatically” part sounds pretty compelling. Of course, you’d only want workloads to get up and move themselves if they could do so without the inconvenience of planned maintenance windows and application downtime. That’s exactly what VMware users have been doing with VMotion since we introduced it in 2003. VMotion delivers true live migration – users and services see no interruptions when a virtual machine is moved from one host to another. VMotion has proven so liberating and reliable that 59% of VMware customers use it regularly in production; some have accumulated hundreds of thousands of perfectly transparent migrations as VMs are automatically load balanced across host clusters with DRS. You don't need to build a large-scale virtual infrastructure to benefit from VMotion. We see over and over how customers that adopt VMware Infrastructure for basic server consolidation projects quickly come to rely on the agility and freedom of VMotion as an essential element of their IT operations.  Here's what Qualcomm had to say about the flexibility provided by VMotion:

"We’ve utilized VMotion extremely heavily. It offers so many benefits: being able to deal with downtimes, being able to do maintenance on the hardware supporting ESX Server hosts, and being able to balance resources. VMotion is a must-have capability for anyone seriously thinking of deploying virtual infrastructure."

While we’re gratified to see virtualization taking the lead in reviving Microsoft's DSI story, its own virtualization tools are missing the crucial live migration support needed to pull it off. It’s important to know that Microsoft dropped plans for live migration in Hyper-V and is relying on a “not quite live” migration method it calls, “Quick Migration.” Microsoft Quick Migration works very differently than the iterative live memory transfer method used by VMware VMotion. Quick Migration fully suspends a VM, copies its memory image to disk, and then reloads and resumes the VM on a new host. That suspend/resume migration technique is far from live. In fact, Microsoft has documented (slide 47) that, even in ideal conditions, Quick Migration interrupts VMs between eight seconds and two minutes when using Gigabit speed networked storage, depending on VM memory size.

Unfortunately, that kind of downtime is more than most networked applications can tolerate. Just a few seconds of unresponsiveness will trigger TCP timeouts and application errors. We tried Quick Migration with the Hyper-V beta using Gigabit iSCSI storage connections and the results weren’t pretty, as you can see in this screen capture video:

(Clicking the screen icon switches to full screen mode, which will make the window text legible. If that doesn't work, you can go directly to this movie at blip.tv)

The Quick Migration downtime caused file copies to fail, VM console connections were severed, and database clients had to be restarted. Scheduling planned maintenance downtime and telling users their apps will be down does not fit anyone’s definition of “Dynamic IT.” In contrast, migrating the same VM with VMotion on a VMware Infrastructure platform didn’t cause even a blip in the network sessions as this video shows: