VMTN Blog

I guess our white paper on Microsoft licensing was perceived as aggressive. Headlines included "fires a broadside," "vilifies," "attacks," "blasts," "rails," "wallops," "complains," and "criticizes." ZDNet's Mary Jo Foley speculated we were preparing for an anti-trust lawsuit. A few people (including Techworld's Manek Dubash, as linked by Microsoft's Patrick O'Rourke) thought VMware was whining, especially since we seem to be ahead. InfoWorld's Randall Kennedy called it a "breathless diatribe" and accuses us of FUD.

Microsoft responded with what the Register called "marketing bologna" -- an official response that said VMware has simply misunderstood what's going on, but rather than explain to everybody what the misunderstandings are, they will let VMware know privately how they are mistaken. Any discussions between Microsoft and VMware or EMC are way above my pay grade, so I can't give  insight there.

But it should always come down to customers and getting things done. Christian Mohn weighs in with a story:

Some time ago Microsoft approached the Gallery project, asking us why we were not actively supporting MS SQL Server as a RDBMS option. Our reply was simple; We don’t have the licenses needed for our developers to be able to develop for that platform, and the Gallery project is not in a position where we want to spend a lot of money on licenses for developers.

In the end we settled on using the Microsoft SQL Server 2005 Enterprise Edition VHD for our development purposes. That enabled us to get quite some testing and bugfixing done, so it has definitely helped the Gallery project, but the VHD is limited to a 30 day trial, which effectively means that we need to rebuild it every 30 days. The second problem with using it, is that you have to run it inside Microsofts Virtual Server software. Converting the VHD renders it useless. Again this means that every Gallery developer that wants to test their code on Microsoft SQL Server, will have to run this on a Windows based host.

Running this with VMware Server/Player, or even Xen, would have been much more flexible since we wouldn’t have to worry that much about which host OS the developers use.

Other blog reactions from Alex at vi411.org (1 and 2) and Scott Lowe (1 and 2) are also well worth reading. In general, I found blog reactions more nuanced than the trade press, who tended to focus on the fisticuffs. In the meantime, we'll keep on making good products and you folks keep on doing cool things with virtualization. Use the products that work best for your situation. If all this fuss makes you or your boss want to wait until the dust settles, go talk to your peers at other companies. There a reason for all the buzz.

--jtroyer

[Update: more from SearchServerVirtualization. Note to Virtual Iron: anyone can get the VMDK spec. Best quote from Dugie's Penseive: “Here’s the exciting burning question, how much better will Virtualization interoperability get? How aggressive is that curve going to be? I want to see that curve so steep, you can just feel the gforces kicking in!]

[Update 2: From Manek's comment, clarified that he originally used 'whinging' in Techworld, which was then linked to by Patrick @ Microsoft. I don't want to imply that Microsoft called us whiners directly. Read Manek's whole article for the complete context.]

EMC VP Chuck Hollis on how EMC's portfolio applies to VMware. Chuck is great on the high-level enterprise landscape "information infrastructure" -- how to peek into all the corners of your enterprise and manage what's going on. VMware's Virtual Infrastructure is an enabling platform, and Chuck talks about how to take advantage of what VI gives you, as well as how to manage having another layer of abstraction in your stack.

Link: Chuck's Blog: EMC Information Infrastructure for VMware.

EMC thinks (and so do I!) that people wanting to get the most out of VMware in advanced deployments will want to take a hard look at the supporting infrastructure.

• They’ll want to think about the storage and its network differently.
• They’ll want to consider how best to do backup and recovery.
• They’ll want to understand the pros and cons of different business continuity approaches.
• They’ll want storage resource management that works well with virtualized and non-virtualized environments.
• They’ll want advanced tools that help them discover their IT
  infrastructure, and use that knowledge to quickly resolve service
  deliver issues.
• And they’ll want to work with a vendor who can help them be successful with VMware.

[Note: although VMware is a subsidiary of EMC, we operate independently.]

We've posted the white paper Microsoft Virtualization Licensing and Distribution Terms, which lays out our concerns about some anti-consumer, anti-choice, and anti-ecosystem policies that Microsoft is choosing to implement. It goes into some detail around support, restrictions on virtual machines being converted or even running on other platforms, desktops, virtualization mobility, and APIs. The motivations of the players can be summed up in this intro paragraph:

In particular, Microsoft does not have key virtual infrastructure capabilities (like VMotion), and they are making those either illegal or expensive for customers; Microsoft doesn't have virtual desktop offerings, so they are denying it to customers; and Microsoft is moving to control this new layer that sits on the hardware by forcing their specifications and APIs on the industry. Included below in this document are explanations with supporting details of some of these specific areas.

The goal from Microsoft seems to be to slow down the market and downplay features they can't match, so that they have a chance to catch up. Do you think that when Viridian, Microsoft's hypervisor, eventually comes out and has "live migration" that they'll find a way to decouple licenses from physical hardware? That when Viridian can do the equivalent of DRS and HA, they'll suddenly become advocates of putting your enterprise software in virtual resource pools for manageability and reliability? Funny how that works. It's all bad for the consumer right now in 2007.

On Sunday,  Mike Niel, Microsoft's GM for virtualization strategy, posted a pre-rebuttal on the Windows Server blog. Link: Where We're Headed With Virtualization.

It's a pretty good overview of all the activity going on inside Microsoft. We should reward good behavior -- recent changes in licensing for Windows Server 2003 Datacenter Edition and SQL Server 2005 Enterprise Edition are glimpses into our virtualized resource pool future, so kudos to Mike and the team there. But I'm struck by how isolated a vision it is -- it's Microsoft's world, and we're just living in it (or as Dave Winer would say, just living in Microsoft's locked trunk). Just wait a few years and you can use Microsoft's hypervisor with Microsoft's management software, but for now just write to Microsoft's APIs and "Virtualization is a new technology for consumers, and one that isn’t mature enough yet from a security perspective for broad consumer adoption." This is not a worldview that recognizes the existing healthy ecosystem around virtualization that exists today. More on the security FUD later.

I am not a Microsoft hater, and I'm not an "open source or die" zealot. But the current situation seems like Microsoft is preventing users from realizing the very real benefits of virtualization in their homes and businesses today.

[Here's Mary Jo Foley's take. This story is bouncing around the periphery of Digg and TechMeme, but for now you can follow along at Google News.]

--jtroyer

From the Saturday, February 24, 2007 edition of the New York Times, A Software Maker Goes Up Against Microsoft. As the title implies, the story hook is competition between VMware and Microsoft. But the real issues are how customers are affected by hypervisor lock-in and licensing limits.

In a meeting with corporate customers in New York last month, Steven A. Ballmer, Microsoft’s chief executive, said, “Everybody in the operating system business wants to be the guy on the bottom,” the software that controls the hardware. ... When quizzed on Microsoft’s plans, Mr. Ballmer replied, “Our view is that virtualization is something that should be built into the operating system.” ...

VMware, however, points to license changes on Microsoft software that it says limit the ability to move virtual-machine software around data centers to automate the management of computing work. A white paper detailing VMware’s concerns will be posted Monday on its Web site (www.vmware.com), the company said.

“Microsoft is looking for any way it can to gain the upper hand,” said Diane Greene, the president of VMware.

The white paper will be available next week, but in the meantime, if you need to catch up, go check out our blog entries from last November, Freedom from OS lock-in.

Given the subject of the New York Times article, it must of course quickly bring up the ghost of Netscape. The article explains virtualization, the benefits of server consolidation, and gives the basic history of the company and the upcoming IPO. The real issues are touched on lightly -- the article explains well the relationship of virtualization and the OS (inside or underneath?), and it mentions that VMware thinks licensing changes will affect customers and prevent many people from fully utilizing their virtual infrastructure. The article ends back on competition.

Virtual Iron and XenSource take opposing views on Microsoft’s recent moves. “Microsoft sees VMware coming between them and their customers,” said John Thibault, president of Virtual Iron. “So Microsoft is manipulating its license terms to see if it can freeze the market and slow down the trend.” ...

VMware, according to Microsoft, should see the wisdom of the path XenSource chose. In his meeting with corporate customers recently, Mr. Ballmer sketched out a future in which Microsoft would put fundamental virtual-machine software in its operating systems, and “VMware builds on top.”

VMware is leery of such an accommodation, fearing it would prove to be a one-sided bargain. “We will not sign agreements that give Microsoft control of this layer,” Ms. Greene said.

See you Monday for more on the issues.

Dell has a very nice VMware Alliance center, with plenty of white papers, support documents, and case studies. Somewhere in there, and I'm not sure on what page, they point to this video:

Screencast of a VMotion demo

[warning: direct link to .wmv file. No sound. I can't figure out how to get a screenshot using Windows Media Player, but it shows a script hitting SQL Server. The VMotion takes 41 seconds and the script never loses database connectivity, and the RDP connection we're using to monitor performance never drops either. via]

Here is another video of using a LiveCD on VMware Workstation

from SecurityDistro via unix-tutorial blog.  Note you can also use this tiny LiveCD Player virtual appliance to skip some of the setup.

[Updated twice below.]

Check out Alex Bakman's VMworld 2007 presentation here as well on Top 10 Recommendations for Improving VMware ESX Security.

1. Use Firewall and Antivirus software for COS. Just as in any other operating system, this provides basic protection
2. Use VLANs to segment the physical network so only machines that are required to see each other are able to do so
3. When installing ESX, use security=high
4. Do not allow root level access over SSH and use secure commands
5. Disable all unnecessary services in console OS
6. Use VirtualCenter to help you manage granular security access
7. Stay current with ESX patches
8. Harden Guest Operating Systems
9. Control User Level Access using VirtualCenter
10. Document and monitor configuration changes in your environment, especially changes in security settings

[Update: from Schley Andrew Kutz in the comments:

I disagree about installing an AV scanner in the COS. There should be limited access to the COS to begin with, and an AV scanner provides unnecessary overhead. Remember, most ESX installations do not allocate the COS that much memory, and an AV scanner will just bog things down, and honestly is not all that helpful.

I appreciate Alex's work on this, but I work very hard to encourage people to think of VMs as physical machines. Hence, I think when designing documents like this we, as public voices for ESX, should separate guidelines and suggestions into 2 categories -- 1) topics related directly to VMs and 2) everything else. Steps 4, 5, 7, 8, 10 are all steps that should be applied to any server or operating system. They are not specific to ESX, its VMs, or guest OSs.

See also the two new papers just published on VMTN: VMware Infrastructure 3 Security Hardening and Security Design of the VMware Infrastructure 3 Architecture.]

[Update 2: From one of our internal security gurus:

Also as a general recommendation we suggest putting the Service console or Console OS (COS) on a separate network or VLan.  The  COS should only be accessible people who administer ESX.  This will significantly reduce the attack surface.  In fact if you can you should have the COS on its own private physical nic card and virtual switch.  Since you restrict permissions at the network layer the more common 'privilege escalations' are not possible.

As far as an anti-virus...  If you have a physically separate COS and you don't use the COS as a "general purpose" OS, you shouldn't need an anti-virus. If your environment requires anti-virus, you need one that will work with our glibc version.]

From VMware Forum regular Massimo re Ferre', a comparison table of the various "connection broker" options for Virtual Desktop Infrastructure. The connection broker is the software 'traffic cop' that controls session access to the server-side virtual desktops or other shared resources. Link: Brokering technologies for client consolidation.

This page is intended to be a summary for the various brokering technologies available for the VMware VDI initiative. At least this was the initial intent.

As soon as I started working on this summary two things became clear:

1. while the idea of the connection broker was initially associated to the VMware VDI initiative, the concept and values of these brokers (or "infrastructure access" packages) has expanded a bit to encompass other client consolidation models such as the more traditional "Terminal Services" model as well as the niche "PC Blades" model. Shared Services (i.e. TS / Citrix deployments are just not death and certainly hosted desktop can't be a good fit for everything either). Not to mention that these connection brokers are or will be able to broker virtual machines hosted also on non-VMware type of hypervisors.

2. connection brokering is only one piece of the functionality that these "infrastructure access" packages provide. Session brokering is the simple concept of redirecting an end-user to an available OS/application image (either from a pool or his/her own end-user image). As you can see below these packages (at least some of those) provides much more than that. That's  the reason for which referring to them as "connection brokers" is kind of limited.
   

So in a nutshell: what started to be the "VDI connection brokers" are now brokering other things than simply VDI vm's as well as these "VDI connection brokers" are doing more than "simply brokering".

[via run-virtual]

Bruce Herndon of VMware's performance team uses the VMmark benchmark to take a look at how ESX Server utilizes NUMA (non-uniform memory access) while scaling to high numbers of virtual machines. Link: Studying NUMA with VMmark.

In a NUMA system, the processors are divided into sets, also known as nodes. Each node has direct, local access to a portion of the overall system memory and must communicate via a network interconnect to access the remaining, remote memory at other NUMA nodes. The memory interconnect adds latency to remote memory accesses, making them slower than local ones. Applications that heavily utilize the faster local memory tend to perform better than those that don't. VMware ESX Server is fully NUMA-aware and exploits local memory to improve performance. ...

All in all, I am quite pleased with the results. They tell us that we need not worry about overstressing NUMA systems even as vendors make quad-core processors ubiquitous. In fact, I would say that virtual environments are a great match for commodity NUMA-based multi-core systems due to the encapsulation of memory requests within a virtual machine, which creates a largely local access pattern and limits stress on the memory subsystem. Of equal importance, these results show that the ESX scheduler exploits these types of systems well, which is good to see given how much work I know our kernel team has put into it. This type of exercise is just another area where a robust, stable, and representative virtualization benchmark like VMmark can prove invaluable.

Christian Hammond talks about Record/Replay and other features in the new Workstation beta. Link: ChipLog » VMware Workstation 6.0 beta 3.

What is this good for? Well, have you ever tried testing a program only to encounter a bug that you just can’t reproduce? Maybe there was some memory corruption that happened under some specific case that you just can’t seem to diagnose. Or maybe it’s a network packet that came in in some form that your application didn’t expect. Under normal circumstances, you’d have to do a lot of guesswork in order to find out what exactly happened. Far too often, it’s just too hard to reproduce the bug and it goes unfixed for some time.

Now imagine instead that you’re testing the program in Workstation and, before your testing, you hit Record. You attempt the test and the program crashes in some weird manner. No problem. Hit Stop and replay the recording. Just before the crash occurs, stop the playback and attach a debugger. Messed up? Didn’t find the cause? Replay that recording again.

He also mentions our free upgrade for new Workstation users -- buy Workstation 5.5 now and get a copy of Workstation 6.0 later.