Hyper-V with Server Core -- Too Dry and Crunchy for our Taste
We wouldn't be doing our jobs at VMware if we didn't regularly compare our products with the competition to ensure our customers get the best technology and user experience possible. In keeping with that practice, we recently set up Microsoft's Hyper-V to get a first hand look. We made sure to follow Microsoft's documentation and best practices guidance to get a fair comparison and to understand exactly what a Hyper-V user experiences as he or she attempts to deploy and configure Microsoft's new product.
One key best practice we heard Microsoft's Hyper-V team stress in sessions at June's TechEd conference and again last week at VMworld was a strong recommendation to run Hyper-V using the Server Core variant of Windows Server 2008. Using the smaller Server Core as the Hyper-V parent partition, instead of a full blown instance of Windows Server 2008, strips out Windows features and services not needed to run Hyper-V. With Server Core, Microsoft is attempting to minimize the attack surface and patching requirements for Windows to make it a safer platform for virtual machines. I would agree that as Hyper-V requires a general purpose operating system, you might as well make it as small as possible. The Server Core concept seems like a good idea. So, following Microsoft's recommendations, we deployed Hyper-V with Server Core.
Server Core -- "The Windows You Know"??
One aim in evaluating Hyper-V was to test its end-user experience, as Microsoft execs repeated over and over at their Sept. 8 virtualization event that Hyper-V would be eagerly adopted because it uses, "the Windows you know." The insinuation is that Hyper-V is easy -- and of course that somehow VMware is not. Microsoft is claiming that with Hyper-V there is no added learning required, no training, no classrooms, because you already know Windows, you can jump right into Hyper-V. They are also claiming that VMware ESX requires you to take the time to learn a whole new system. But is this accurate? Is the recommended Server Core flavor of Windows 2008 really, "the Windows you know"? Is it easier than ESX? We wanted to find out.
Windows Server Core = MS-DOS 2008
If you haven't seen Server Core yet, here's the UI in its entirety. It doesn't look like the Windows I know, in fact it looks like DOS! Are we stepping back in time? Who knows DOS anymore? Actually, it makes you wonder why Microsoft didn't just call it MS-DOS 2008, especially since anyone using Server Core will need to resurrect some long lost command line skills to get any work done.
Is Hyper-V with Recommended Server Core, In Fact, Easy?
So, how does the Hyper-V and DOS -- err, I mean Server Core -- combination stack up when compared to the user experience of VMware ESXi? To try it out, we did side-by-side installations of Microsoft Server Core/Hyper-V and VMware ESXi 3.5 on identical servers. To let you see the details of each setup process, we recorded the entire sequence in a pair of videos.
This first video shows every step required to install Hyper-V and ESXi on a fresh machine. We kept count of the elapsed time, reboots, mouse clicks and keystrokes each product needed and it clearly shows the huge advantage the truly thin and OS-free ESXi architecture has in installation speed and simplicity. ESXi goes from bare-metal to fully installed in one-third the time, half the mouse clicks, hundreds fewer keystrokes and just one reboot vs. seven compared to Hyper-V. The simplicity of the ESXi wizard-driven installation is striking compared to the arduous process needed to first get the Server Core OS installed and then configure Hyper-V in a command line environment.
Our second video starts where the first left off and takes Hyper-V and ESXi through the steps needed to configure two iSCSI datastores for VM use. iSCSI setup is a standard task for any virtualization user that wants to take advantage of shared storage for VM migration and high availability. ESXi's Windows-based Virtual Infrastructure client makes the iSCSI setup quick and easy. For Hyper-V, the "Windows you know" is nowhere to be seen. Instead, working with Server Core requires you to key in a long sequence of obscure commands to configure iSCSI initiators and targets, partitions and file systems. We generously showed the Hyper-V setup executed with no delays, although it took us hours of digging through Microsoft documents and knowledgebase articles to find the right commands to use when configuring iSCSI in Server Core.
Our Conclusion: Server Core plus Hyper-V is for Experts Only
VMware has put great effort into making ESXi the easiest and fastest hypervisor to install and configure and these videos clearly show the results. Getting the OS out of the hypervisor plays a big part in the streamlined simplicity of ESXi as there is no general purpose OS to configure and manage and the reliability and security issues accompanying the tens of millions of lines of code an OS brings along are eliminated. Microsoft's OS-centric Hyper-V architecture adds many steps to the setup and puts their users in a quandary: either A) they install Hyper-V on a full Windows Server 2008 OS and deal with the frequent patching and security fixes Windows requires; or, B) they follow Microsoft's best practice guidelines and suffer with the limitations of Server Core. As the videos show, the tradeoffs with Server Core are daunting -- Windows administrators will find their familiar GUI tools are missing and they'll be left to spend a lot of quality time with search engines tracking down documentation on Microsoft's obscure command line utilities.
Take a look at the side-by-side comparison videos and let us know if you agree that ESXi provides a far faster and easier (or maybe we should say, "moister and chewier") setup experience. Better yet, try ESXi and Hyper-V with Server Core on your own machines and tell us how it went.
Posted by Eric Horschman on September 26, 2008 | Permalink |
For anyone who cannot function without pretty icons and mouse clicks, Core is definitely a PITA (“MS-DOS 2008” indeed).
“you makes yer choices and takes your chances”…
IMHO, anyone who sacrifices performance and reduced attack surface for point-n-click doesn’t deserve to know the location of; much less operate in the data center.
Posted by: JimmyJoeBobAlooba | September 26, 2008 at 04:37 PM
Jimmy -- I agree that the better trained and more experienced sysadmins will know their way around a command line, but the point I was making is that ESXi doesn't make you trade off ease of use for for a secure, thin footprint. I think that comes through pretty clear in the videos.
Posted by: Eric Horschman | September 26, 2008 at 05:55 PM
Brilliant.
Posted by: Jason Boche | September 26, 2008 at 08:12 PM
Eric,
That's my whole point; if "Server Core plus Hyper-V is for Experts Only", then the corrolary must be "ESX is for noobs"?
..you know I couldn't resist that...
Seriously; could Hyper-V manual setup and configuration be improved? Certainly; but given that Windows Core and Hyper-V both can be installed with 0 mouse or keyboard clicks via unattended setup, where does this leave ESX? ..in the back seat, FWICS...
Posted by: JimmyJoeBobAlooba | September 26, 2008 at 09:16 PM
No Admin worth their salt would install server core plus hyper-v in the so called DOS mode, but granted ESX installer is easy to install and is pitched at the right level.
Rather than take the Microsoft “snipe at your rivals product” approach, it might be better to publish an article regarding the unattended installation of ESX, as this is the information most Windows centric Admin’s would like.
Posted by: Paul Munro | September 27, 2008 at 12:52 AM
Great videos guys I like that youre calling MSFT on their rediculous claims - not that anyone would believe them anyway. In regards to the prior comment that this is a "snipe at your rivals" approach - I don't agree - Msoftie is the one who is saying that Hyper-V is the windows you know and that ESX is difficult - Eric here is refuting that point. MSFT also is sayng that thyre 30% of the cost of VMware - VMW I look forward to your response to that
Posted by: J Waters | September 27, 2008 at 12:34 PM
Eric,
You are comparing apples with peaches.
What you have to compare with ESX 3i is Hyper-V Server and not Server Core With Hyper-V.
Only doing that you will be fair with your camparison.
BR
Posted by: Mario | September 28, 2008 at 06:35 AM
Mario,
Hyper-V Server has the same interface and commands as Hyper-V with Server Core. Hyper-V Server is just Windows 2008 Standard Edition Core installation with some additional stuff stripped out that's not needed to support the Hyper-V role. Still has the same interface though.
Posted by: Mike DIPetrillo | September 28, 2008 at 06:54 AM
Hyper-V runs on a lot of different boxes. Cheap ones, big ones etc etc.
Saving a few minutes on the install of the hypervisor isn't a big deal.
Performance, huge choice of hardware and price are more important.
Its not like an SMB will be installing Hyper-V on a server every day or even once a month.
Posted by: Bruce | September 29, 2008 at 01:06 PM
Bruce,
I'll concede that the ESXi hardware compatibility list is not as broad as the HCL for ESX (the version with the console OS,) but the ESXi HCL already covers the most popular mainstream servers and is growing rapidly. Also, by not going the route of using generic drivers in a parent OS, we're able to harden and optimize the ESX device drivers and they deliver better scalability by running in the hypervisor (see http://www.vmware.com/technology/whyvmware/robust-foundation.html#c97230).
I do think the initial install experience for a user new to virtualization is important and ESXi just can't be beat in that area. The pros that want to automate ESXi installs should check out Lance Berc's tools for automating PXE booting of stateless ESXi servers here - http://communities.vmware.com/docs/DOC-7512 Under VirtualCenter management, ESXi can be controlled with PowerShell, which unfortunately is not available with Server Core, so you can't manage Hyper-V with PowerShell in that configuration.
You'll need to be more specific on the performance edge for Hyper-V. ESX 3.5 is turning in some great performance results (http://blogs.vmware.com/performance/) and we're not conceding anything in that area.
I'm not sure why you bring up price. ESXi is free and has all you need to run your VMs. Until Microsoft ships Hyper-V Server, you'll need to at least pay for Windows Server 2008 Standard Edition before you can bring up your first Hyper-V VM.
Posted by: Eric Horschman | September 29, 2008 at 02:40 PM
Since I'm running Hyper-V on my sub-500$ quad core dell PC, i'm not sure I want to pay for a box to run ESX.
Powershell can run on core, it just isn't supported yet.
http://dmitrysotnikov.wordpress.com/2008/05/15/powershell-on-server-core/
Pre-release Hyper-V loses by 4% to ESX on SQL benchmarks:
http://www.sqlmag.com/Article/ArticleID/99218/sql_server_99218.html
Server 2008 60 day trials here:
http://www.microsoft.com/windowsserver2008/en/us/trial-software.aspx
Posted by: Bruce | September 29, 2008 at 02:57 PM
Interesting points - As Microsoft’s real, primary target market for Hyper-V is an SMB, and as it is unlikely that the SMB has the resources/knowledge required to use Windows Server Core, if that organization was deploying Hyper-V it would have two unsatisfactory choices for parent partition 1) Go with Server Core and have a reduced footprint/attack surface, which is still many times greater than that of ESXi, and take your chances learning the relatively complicated, manual, somewhat undocumented configuration process or 2) Go with full blown Windows Server 2008 and enjoy an 8-10GB hypervisor footprint, which still trails ESX in functionality, performance, and stability, and enjoy rebooting all your virtual machines every week when Windows Server requires a patch for code that likely has nothing to do with virtualization.
Full Diclosure - I work for VMware.
Posted by: Timothy Stephan | September 30, 2008 at 11:07 AM
Timothy,
Your #1 choice may change with the release of Hyper-V.
As for #2:
I disagree on most points.
The foot print is kind of irrelevant when 1TB drives are so cheap. Most SMB's won't be using SANS.
Performance tests by Network World show Hyper-V performing better many scenarios.
"At six VM guests, both hypervisors are starting to struggle to deliver performance comparable to what a native operating system running directly on the server can pull off. But Microsoft kept its performance drop a bit more in check as it appears to have mastered a more linear distribution of hypervisor resources when VMs get piled on."
http://www.networkworld.com/reviews/2008/090108-test-virtualization.html?page=4
Functionality costs a lot of money in the ESX world.
Stability ... I have no metrics for Hyper-V, but we've had VM's hang on ESX because the VMware tools install/upgrade went bad.
Reboots. Not weekly. If the firewall is on, you can put monthly updates off for a long time.
Posted by: Bruce | September 30, 2008 at 11:50 AM
Bruce,
The comparison of the footprint deals with surface area of code that can be subject to compromise/exploit/attack. It has little to do with storage costs.
32MB footprint = 32MB of code subject to attack (VMware embedded ESXi)
1GB footprint = 1GB of code subject to attack (VMware ESXi installable)
1GB footprint = 1GB of code subject to attack (Microsoft Windows Server 2008 Core w/ Hyper-V)
8GB footprint = 8GB of code subject to attack (Full blown Microsoft Windows Server 2008 w/ Hyper-V)
On your previous comment, VMware ESXi (and ESX) is certainly not for everyone. I agree that both ESXi and Hyper-V serve business needs that vary from company to company. If you are running a business on a sub $500 Dell PC, then Hyper-V sounds like the right fit for you and your business.
The VMware ESXi hypervisor is intended for and must scale to fit highly available and continuously available solutions for which a sub $500 Dell PC has no correlation to. Afterall, what good is a highly available hypervisor if it's run on hardware with multiple single points of failure?
Posted by: Jason Boche | September 30, 2008 at 12:37 PM
FYI a real world example of someone trying to configure Hyper-V with Server Core.
http://joeelway.spaces.live.com/blog/cns!2095EAC3772C41DB!1152.entry
Posted by: Tim Stephan | October 20, 2008 at 12:56 PM
Someone should at least mention the opensource tool on codeplex for configuring server core, which makes the bulk of this video moot.
http://www.codeplex.com/CoreConfig
Posted by: GlenH | October 21, 2008 at 06:04 PM
The comarison is TOTALLY UNFAIR. You can also manage HyperV server remotely, in GUI, like what you do with ESX.
Posted by: flyg | November 04, 2008 at 09:26 PM
Flyg,
I'm sorry to disagree, but the side-by-side videos show a fair comparison between the free hypervisors from VMware and Microsoft (actually, you need to pay $28 extra on top of Windows 2008 to get Hyper-V as shown.) You aren't able to use the Hyper-V Manager GUI until you get through all the steps shown in the first video. In the second video, the iSCSI configuration steps for Hyper-V are not supported from Hyper-V Manager, so you need to use the Server Core command line.
Now, if you want to pay extra for System Center Virtual Machine Manager, you can use the SCVMM GUI once you have Hyper-V installed on top of Server Core, but the point of these videos was to compare the free products from both vendors.
Eric
Posted by: Eric Horschman | November 05, 2008 at 08:23 AM
As an additional note, VI Client does not run on x64 OS installs. People like me (geeks) who likely make up a vast majority of the end users of VI Client are more likely to be running x64 workstations. And it's not like x64 is new technology. I've embraced x64 and get bit all the time by software that refuses to run or install on x64. I'm on board- where's the rest of the industry?
Just my $.02
Posted by: Mark | November 13, 2008 at 09:44 PM
Never mind post above- I just found out it is supported!
Shame on me!
But the rest of the industry? Shame on you!
Posted by: Mark | November 13, 2008 at 09:47 PM
Hey all...
The hyper-v core i installed as test wasn't anytihing like what you are showing in the video.
You are are doing it wrong. Microsoft have released a version of core with hyper-v allready intalled.
allso when it starts up you get a blue "gui" menue where you can set up ip address and such just like you do it with ESCi...
to me, this video show someone either hasn't done his homework OR maybe made this before Hyper-c Core was released. But then i again i seem to remember (im not going to whatch it again to confirm) that the hyper-v rolle installed on core was a beta... and if so this makes it even more unfair since its BETA!
if the video was made before hyper-v server core was released fair enough. But i would pull this off the website.
If i were a manager at VMware this video to me would be embaressing and i would have it pulled off the site imdiatly... this is just not proffesional which were the image i had of VMware...
you going to make a comparison on a competitve product you need to get your fact streight and not like this which to me just lame
http://forums.techarena.in/operating-systems/1048628.htm
Posted by: Casper | November 14, 2008 at 05:41 PM
Casper,
The videos in this blog post show the setup of Windows Server 2008 Server Core and the Hyper-V role. What you're describing is Hyper-V Server, which Microsoft released after this post went up.
Hyper-V Server does have the character mode configuration screen you mention and that is an improvement over the Windows Server Core + Hyper-V installation experience. However, Hyper-V Server is crippled to the point that I don't think most users will bother with it. It doesn't support MSCS, so there's no VM migration or failover. It supports fewer host CPUs and less memory, and it does not come with the rights to run Windows in a VM that you get with full Windows Server. See this blog post outlining why most users will skip Hyper-V Server: http://blogs.vmware.com/virtualreality/2008/10/hyper-v-server.html
So, the videos in this post correctly show the setup experience you'll go through if you want to run Hyper-V with full Windows Server 2008 capabilities. It gives you more features, but at the cost of a more painful installation experience. We think VMware ESXi is better all around -- free, easier to install and more capable than Hyper-V.
Eric
Posted by: Eric Horschman | November 15, 2008 at 12:22 AM
Eric,
I really think Vmware doesn't need to be in a dogfight with microsoft at the moment. Vmware is way ahead of microsoft when it comes to bearmetal virtualization.
And by the way, all this can be scripted. Then it's just getting started with the installation script and your whole point is gone.
Again: please don't dogfight. My choice (at least at the moment) is Vmware (i think this will be for a very long time) but that's not becaurse this kind of nonsens.
Posted by: Arjantim | December 13, 2008 at 09:39 AM
Can ESXI run in high availability if two physical hosts are used? As in Clustering in MS core?
Posted by: John | February 16, 2009 at 07:32 PM
John,
When managed by VMware vCenter, ESXi supports our HA feature that will restart VMs on a failed ESXi host on another ESX host in a cluster. This is a cold restart.
ESXi 3.5 Update 1 and later support Microsoft Cluster Services. You can create clusters with ESXi VMs in three ways:
1. Cluster VMs in a single box (useful for dev/test and HA for a crash-prone app)
2. Cluster VMs across ESXi hosts (useful for software and hardware fault tolerance)
3. Cluster physical machines together with VMs (useful for N+1 clusters that protect an expensive physical machine with a cheap virtual machine.
Refer to the ESXi 3.5 U1 release notes here: http://www.vmware.com/support/vi3/doc/vi3_esx3i_i_35u1_vc25u1_rel_notes.html
and the ESX "Setup for Microsoft Cluster Service" guide here: http://www.vmware.com/pdf/vi3_35/esx_3/vi3_35_25_u1_mscs.pdf
Eric
Posted by: Eric Horschman | February 19, 2009 at 04:19 PM