VMTN Blog

Hi everyone,

I am one of the VMware vSphere product marketing managers at VMware and wanted to give you a summary of all the upcoming content related to the vSphere launch. We have developed resources for all audiences and technical levels to help you understand what is VMware vSphere, how does it work, and how to upgrade to it from VI3 or deploy it for the first time. 

Here is a quick summary of vSphere resources. More details are provided below the table.

WEBSITE:

Visit the VMware vSphere web page to learn more about the features and benefits of VMware vSphere based on your company size.  You will find datasheets, demos, and solution briefs on the vSphere product pages.

WEBCASTS / PODCASTS:

We have developed 3 webcast series:

• On-demand webcasts are pre-recorded and cover what’s new in vSphere while providing technical deep dives into vSphere features.
• Live Technical Webcasts feature live technical deep dives into various vSphere components.
• Live Overview Webcasts expand on the business value of vSphere for enterprises and SMBs.

We also have a series of podcasts for those of you who would like to learn about vSphere on the go! The podcasts will cover vSphere editions and provide technical deep dives into the new features.

UPGRADE AND EVALUATION CENTERS:

We have launched VMware vSphere Upgrade Center, a site that contains all the information you need to upgrade from VI3 to vSphere 4. The site will point you to upgrade preparation checklists, access to vSphere upgrade communities, vSphere entitlement paths for VI customers with active subscription contracts, and details on the new and improved licensing mechanism. We will add more resources to this page including upgrade best practices when vSphere becomes Generally Available.  I highly recommend that you bookmark this page if you are an existing VI customer.

[coming soon] We are creating a vSphere Evaluation Center packed with demos and technical documentation to provide a guided evaluation experience for both new and existing customers. The site will go live when vSphere becomes generally available.

QUICKSTART SERIES:

Note: This resource is NOT designed for existing VI customers

One last resource I want to point out to New Customers is the VMware vSphere QuickStart Series. This is a new FREE course that will be taught live over the web in four 2-hour modules. It is designed to teach VMware vSphere and VMware ESXi evaluators how to do a basic installation, configuration and management of either ESXi or vSphere. The course primarily consists of live product demonstrations to ensure new users gain practical experience that can be leveraged to do a basic vSphere POC or small deployment. If you are already familiar with VMware Infrastructure (VI3), you should not attend this class unless you want an 8 hour review of what you already know. 

Here is a short outline of the series:

Registration is open today and space is very limited. Visit the vSphere QuickStart landing page to view course details and register!

Re our post yesterday on ESXi, Duncan dives deeper into one of the tools that VMware is creating so that you don't have to pop the hood -- VMware Infrastructure Management Assistant (VIMA).

Link: VMotioning your Service Console?! » Yellow Bricks.

Some of you might have looked into VIMA already. Those of you that didn’t please check it out because I expect this to be the way that VMware is heading. Note, I don’t know if it really is the way VMware is heading, but a Service Console with VMotion capabilities sounds like a winner to me. A little birdie also just told me that APC, the UPS Company, is finishing their VIMA Compatible UPS software agent!

The cool thing about VIMA is that it includes the RCLI commands, the Perl toolkit and a logger daemon named vilogd. The last one will be the topic for this blog. So what does this logger daemon include? The vilogd daemon collects all the logs that are available through the DiagnosticManager VI API:

At VMworld, we talked about how we think you'll be managing your VMware Infrastructure in the future. Through programs like the VMware Ready Management Initiative and the APIs and services on the platform, the management vendors you're already using will be able to extend their tools you're already using to better manage your virtual resources.

SAP is already ahead of the game today by integrating their Adaptive Computing Controller with VirtualCenter. One of the benefits of this integration is you can manage (start, stop, resume, VMotion) your virtual machines from inside the application-aware SAP tool, so that you can see your VMs not as black-box compute workloads, but also by the SAP services they are running. Hmm, the future of virtual management is looking interesting.

Joachim gives us the details: Virtualization for SAP Solutions: News on the SAP Adaptive Computing Controller: Integration with VMware Infrastructure.

What does the integration entail? Let me start with a few customer questions that I frequently get. “With VMware ESX there is a new component in the stack. How do I do the monitoring then?”, “I am an SAP Basis Administrator and I need to see what else is running in my ESX servers. But I do not want to learn a new tool!”, or “My infrastructure manager does not want to give me access to VirtualCenter, so how can I see what is going on in my SAP landscape?” are some of the more common ones.

The upcoming integration addresses all of these points. The AC UI will feature a new tab called “Virtualization”, adding core virtual machine functionality to the existing AC functionality (which basically treats a virtual machine as a physical system). In the “Virtualization” tab you can see which application service is running on which virtual machine on which ESX host, and you get information about ESX hosts and virtual machine usage data. You can also execute core VMware commands like start/stop/suspend/resume for a VM, and trigger a migration using VMotion through the SAP UI. Very cool is also the following feature: If you click on “shutdown” for a VM, the AC Controller displays a pop-up window listing which SAP services are running in the VM and asks you whether it is still ok to proceed with the shutdown. This is clearly new stuff and a great interaction between virtual infrastructure and application. ...

If you click on a specific ESX host you get some usage statistics for that server as well as a list of all SAP services running on each virtual machine on that host.

Bernd Harzog over at Doug Brown's place (DABCC.com) has a great deep dive with B-hive's CEO and CTO. Link: Virtualization Management: VMware B-hive Detailed Product Review.

Why the Old Way No Longer Works
Before I get into how the product works, I want to spend a moment on why it is important to do things in the way that B-hive does it. There is a right way and a wrong way to do Applications Performance Management in virtualized environments. The reason for this is that when you stick a piece of software in a VM, the Windows OS (assume Windows for a moment) no longer owns the clock (the hypervisor does). This means that anything that counts time inside of a VM will do so incorrectly. This includes management agents from systems management vendors and APM vendors. This in turn means that you cannot collect resource usage information or response times from within a guest and try to use that information to infer anything about the performance of the application running in the guest. Time based metrics include CPU utilization, Page Faults per Second, Context Switches per Second, Disk I/O Reads/Writes per Second, Network Bytes Send/Received per Second, and most importantly any measure of the time elapsed between Event A (start of a transaction) and Event B (end of transaction). So, neither resource based metrics nor applications response time metrics collected from inside of a guest VM are valid. All of this is described in a VMware Whitepaper on the subject if you do not believe me. Bottom line - products that install agents to measure resource utilization and/or response time in virtualized guests do not work. So once you virtualize, a new way to do APM is needed. ...

Conclusion
By buying B-hive, VMware did not just acquire yet another product that watched resource utilization on servers. B-hive moved the ball forward in terms of how to measure performance the right way (response time), with IT Operations as the target audience. This will be a highly valuable tool to VMware customers with virtualized servers, and will significantly enhance the value of the VMware platform relative to competing platforms from Microsoft and Citrix, neither of whom have anything like this in their portfolios.

Grid Today also talks to VMware CTO Steve Herrod on the importance of this acquisition. Link: Does B-hive Acquisition Make VMware a Cloud Vendor?. [via]

To put it simply, Herrod says that Conductor is able to “think at the level that applications that at,” as opposed to thinking at an infrastructure level. Whereas most of VMware's measurement tools focus on machine metrics like CPU MHz or RAM usage, B-hive’s tool is able to, for example, recognize what it looks like to report a Web page and what it looks like when a Web page is returned to a user, and can then report on the average time to provide a page. And it is just as proficient looking into more complex, multi-tier applications, says Herrod.

This application-level insight is increasingly vital to VMware users, many of whom are implementing “VMware first” initiatives. Large companies in particular, says Herrod, are putting all of their apps in virtual environments, and they are not hesitant about requesting more support in terms of performance tracking when they migrate mission-critical applications to VMware. “From our standpoint,” he says, “we saw a way to do performance better than it’s done on physical systems, so we see it as another driver for people to bring new applications into their systems.”

“We’re 100 percent serious about making VMware the best place to run mission-critical applications,” Staten stated. “And to the extent we make those easier to manage and more available and more secure than when they’re running natively, that’s absolutely our strategy -- and this is one of the pillars in doing that.”

As you may know, VMware recently began the process of acquiring B-hive, but you may not know much about what B-hive does. From the press release, B-hive "gives infrastructure groups visibility into application performance in virtual environments such as end-user transaction response time, virtual machine utilization and cross-virtual machine dependencies."  Think of it as providing more raw data for both your existing management solutions and for automated tools like DRS, so you can "proactively resolve application performance problems by automatically triggering actions such as dynamically allocating more resources, migrating the application to a different server, provisioning additional VMs, changing transaction routing, or system re-boots."

Note also that B-hive is agentless, which means not only do you not have to install software in the guest, but that it is independent of OS, something that fits very well into VMware's philosophy and architecture.

I wanted to share this description of what B-hive does with you. It's fairly detailed yet still high level. (I don't know enough about this space to comment on Bernd's market landscape observations. Again, I don't think VMware is going in to the "application performance management" space, but we just want to give you more and better data to start implementing the dynamic cloud data center of the future. You can also go straight to the source and visit b-hive's site, where you can get a deeper understanding. Their blog is also worth checking out.

Link: Virtualization Management After VMware / B-hive, Who Wins - Who Loses | Bernd Harzog’s "Managing Virtualization" Column.

B-hive - A Quick Look at the Product
The basic idea of an APM or EUEM solution is to measure either applications performance (how fast is the application processing transactions or units of work as they arrive from users or other applications), or the end user experience, which measures how well the application is working in the eyes of its end users. B-hive is not a true end user experience solution since in order to be one of these you have to be able to measure response time from the perspective of actual end users, and B-hive does not do this. B-hive measures response time from the perspective of the presentation tier (the web server if it is a web based system) of an application system, which is exactly the right approach if you want to be a leading edge APM solution (older APM solutions tried to infer applications performance by looking at the resources used by an application, and this approach simply does not work for virtualized applications systems). So, here is a quick overview of what B-hive does:

1. B-hive attaches to a mirror (spanned) port on the switch that supports the servers closest to the users (most often the switch that supports the web servers). This allows B-hive to see all of the transactions that flow back and forth between users and the applications system.

2. B-hive's concept of response time is the time between the arrival of a request from a user, and the reply to that request on the part of the application system. By default these transactions are "atomic" or highly granular and not something that that map to what a user would view as a transaction of interest. But they are representative of the overall response time of the system, and since these atomic transactions can be supported for almost every application with no configuration, they represent the right level of detail for the IT staff supporting the virtual infrastructure.

3. B-hive also understands certain back end protocols like SQL Server and Oracle. So for two-tier client server applications or any application that talks to a database, the request/response time to and from the database server is used as a proxy for response time.

4. The level of the atomic transaction depends upon the nature of the application. For web based applications the atomic transactions are HTTP request/responses. For non-web applications they are lower level TCP/IP request/responses or database request/response times.

5. With additional effort, atomic transactions can be combined into true compound transactions or transactions of interest to applications owners and the business. In fact B-hive is sold in two flavors, one that supports just atomic transactions for the IT staff, and another one that includes the more comprehensive view of higher level transactions which are of interest to applications owners and the business analysts.

6. B-hive will try to get the user ID of the application out of the TCP/IP data stream. For web based applications this is easy. For some applications this is not possible, and the notion of who the user is defaults to the source IP address which is in many cases not unique to a user (due to NAT). This is one area where vmSight has an advantage due to its patented Connector ID technology.

7. B-hive claims to be able to automatically drive actions in the virtualized environment (for example provision a new server or move a VM) based upon its response time measurements. This takes DRS to an entirely different level, since making these kinds of decisions based upon applications response time makes much more sense than does making them based upon CPU or memory usage.

Well, once Carter spilled the beans, everybody is now waiting with bated breath for VMware to release the beta of our new curiously-named VI Toolkit (for Windows). The toolkit is powered by Windows PowerShell, a shell/scripting technology that Microsoft appears to have gotten very right indeed. The VI SDK, while extremely powerful, is not for the faint of heart. This toolkit takes that power and wraps it up in a very simple syntax which creates a compelling tool for VI admins.

I'm a Perl guy from way back, and I have to say seeing PowerShell scripts, er, cmdlets do all sorts of tricks with my VMs without breaking a sweat makes me grin like a maniac. This is going to be a real boon to VI admins.

Here's an example from the VMworld hands-on lab manual Automating VMware with PowerShell Lab Manual. How can you not like this? You don't even need a manual to understand what it does.

get-vm | get-snapshot | where { `
  $_.Created -lt (get-date).addmonths(-1) `
}

Here's a round-up of the blog reactions so far.

Dave Marshall played with it at VMworld. Link: VMware administrators find value in Microsoft PowerShell

I in fact also attended the lab during the show and found it quite interesting and compelling. For me, this was the first time I had actually used the PowerShell cmdlets to operate and manage a VMware environment. I spoke with VMware's Product Manager of API & SDK, Carter Shanklin at length. Like the people being exposed to PowerShell for the first time, Shanklin seemed very energetic about the possibilities that this scripting feature brings to VMware environments.

Eric Sloof has been working with the toolkit for a while and posting his progress. Link: VMware PowerShell - NTPRO.NL. Here are some of his posts:

• The VI3 Windows Toolkit and .net integration
• Create multiple VM’s using PowerShell
• PowerShell Snapshot Remover
• Pipe VMware PowerShell into PowerGadgets

Hal is looking for good nuts to crack with our new nutcracker. Link: Call for Script Ideas: VMware PowerShell Toolkit.

Calling all ESX admins!  I am looking for novel ideas for scripts to write for the upcoming VMware Toolkit for Windows PowerShell.  Yes–I am offering to do the writing.  I am doing research for a project (details of which to be announced in the coming weeks), and I could use some really great ideas of missing functionality or fixes to problems you have seen while working with VMware Virtual Infrastructure 3.  Pointers to something cool you have seen done with the VMware Perl Toolkit are good too.  It’ll be amusing to see how much simpler those will be in PowerShell. 

Hal and Andrew Kutz are also having a great discussion about relative complexity and our Perl vs PowerShell toolkits here: TechProsaic - VMware Perl Toolkit versus PowerShell VI Toolkit. I don't think either one really is saying my scripting language is better than yours, but more that the VMware team has done a great job of providing the right level of interface for what admins need to do. Jeffrey Snover actually puts it best on the Windows PowerShell team blog. Link: Windows PowerShell : The Semantic Gap.

Someone could read this blog and walk away thinking, "PowerShell is great and Perl is crap" - you'd be both right and wrong.  PowerShell is great but Perl is not crap.  (Hats off to superstar Larry Wall and Perl, very few people and technologies that have had the level of (positive :-) ) impact these 2 have had on the industry.  The world is a better place because that guy was born!)   The difference between the 2 examples is the semantic gap.  The PowerShell example has a very small gap between what you think and what you type. The Perl example has a very large gap.

At the end of the day, the semantic gap is "owned" by the people that provide the instrumentation.   VMWare could have just as easily provided a PowerShell Script that took just as many lines as the Perl example or they could have provide a Perl library or script which provides the semantics of the Get-VM cmdlet.

The good folks at SAPIEN Technologies have a new VI Toolkit (for Windows) book coming out. Link: Coming Soon: Managing VMWare with Windows PowerShell

We’ve a new book in the works: Managing VMWare with Windows PowerShell: TFM. Read about it at http://www.sapienpress.com/vmware.asp, where you can also read about the author and (once we have them available) download preview chapters. We’ll be looking for community reviewers before long, so if you’re an ESX Server user, stay tuned to this blog for your chance to participate and earn some cash!

Dave Stein wins the award for the best title so far: Oh No They Didn't!  VMware Getting all Uppity with Mac Daddy PowerShell

I have no doubt you or your RSS reader should stay glued to the VI PowerShell and Developer Center blogs -- the VI Toolkit (for Windows) beta is coming your way soon.

The VI Team has opened the doors on their brand spankin' new blog over at http://blogs.vmware.com/vi. They'll be posting occasional pieces on their view of virtualization, the technology, where it's headed, how it all fits together, and what they're hearing from customers, both big and small.

Product marketing manager Leena Joshi starts us off. Leena is over the jet lag from her trip to VMworld Europe in Cannes, where she presented on What's new in VI 3.5. On the new VI Team Blog, she breaks apart the different layers of VI3, and what we mean when we say "Management & Automation." Link: VMware virtualization perspectives | VI Team Blog.

Today we begin with an overview of how we think about the entire virtualization stack. The clarification of our viewpoint is necessary because the market tends to speak of two categories – 1. The hypervisor and 2. An amorphous glob called “management”. While it is relatively clear what the hypervisor is, it is far from clear what is being lumped under “management” in the virtualization context.

In our minds, there are two distinct layers of “management” on top of the hypervisor – the virtual infrastructure capabilities, and the automation capabilities.

But let’s not get ahead of ourselves, and “peel the onion” layer by layer.

Go over and say hi.

The good folks from Hyperic and Mosso have put together a nice recorded presentation on managing the operations of your virtual infrastructure -- keeping track of your virtual appliances as they are VMotioned around your pool, maintaining performance history across migrations, comparing guest and host metrics, things like that. Stacey Schneider shares the story on the Hyperic blog. Mosso (a hosting provider of over 35K web sites) also shares some war stories about firing up multiple copies of their web servers when one of their customers publishes his annual Macworld coverage on his blog.

One of the Mosso guys -- not sure if it's Jason or Jonathan -- also repeats some outdated conventional wisdom about what kinds of workloads are appropriate to virtualize. From a hosting provider's perspective, I can see keeping it simple, but be aware that you might be surprised at what is virtualizable these days. As always, the answer is 'it depends' on the characteristic of your particular workload, but many of our customers are virtualizing all workloads by default and are running their RDBMS applications happily in their virtual infrastructure.

VMware's own Mike DiPetrillo surveys the current state of patching virtualization platforms and it's not pretty. Link: VMware Patch Tuesday from Mike's blog A Little Truth.

Microsoft’s new hypervisor based product called “Hyper-V” requires a Windows operating system in the Parent Partition. Given that you’ll need to patch that Windows OS just like any other Windows OS I decided to look at the history of Microsoft patches for Windows Server 2003. ...

Virtual Iron actually uses the Xen open-source hypervisor. There are a lot of other vendors out there that use that same hypervisor (Red Hat, SUSE, SUN, Citrix/XenSource, and Oracle to name a few). While the hypervisor itself is pretty good the architecture still requires a general purpose operating system in Domain 0 (the Parent Partition in Microsoft land). What does this mean? Well, you’re back to having to patch a general purpose Linux operating system which introduces downtime for you system. ...

Last, we’ll go into a totally different architecture for the last vendor - Virtuozzo. ... you to install the patch once on the host and everyone inherits it. Guess what? It works! ... Let’s say you deploy that patch and it blows up one of the VMs on the host. If you’ve never had a patch blow up something in your environment then I want to meet you. Anyhow, something blew up so we’ll need to back that patch out. Oh wait...all of the other VMs are inheriting that same patch. ...

Mike isn't shy about sharing from his long experience in virtualization, and in my experience he knows what he's talking about. Welcome to the blogosphere, Mike. I hope we see more of you around here.

Dan Kusnetzky, who has a blog here: Virtually Speaking on ZDNet, has written a number of thought pieces with his consulting/analyst hat over here: Recent Publications from the Kusnetzky Group at his website. He's usually exploring the interface between the technology of virtualization and operationalization in a business process.

I like this recent one: Virtualization: Evolution not Revolution (pdf link). In this short 3-pager, his basic point is that things move slowly in the enterprise data center, because IT managers must be risk averse.

The Golden Rules of IT

1) If it's not broken, don't fix it. Most organizations simply don't have the
time, the resources or the funds to re-implement things that are currently
working.

I think paradoxically this has been one driver for VMware's successful adoption. It is so easy to get started with VMware -- download VMware Server or a VI3 eval, then convert [warning: sound] some necessary but little-used old servers that are just sucking up electricity, and go. You don't need a special paravirtualized kernel, just whatever you were running (Windows, Linux, Solaris, etc.); don't need to recompile your app; don't need to get special hardware; and you don't even really need a SAN or other fancy enterprise storage to get started -- just virtualize, no re-implementation needed. The key point you need to realize at this level is that you treat a virtual machine just like its physical counterpart -- although try not to have every antivirus and backup job in every virtual machine on an ESX Server fire off at the same time.

Now when that works great and you do want to see how to take more advantage of the opportunities afforded by virtual infrastructure, then you do have to do some more planning -- maybe get more storage, certainly get some expertise and evaluation of your current infrastructure, and start to figure out how this affects your processes when a new server can be provisioned in a few minutes and your DR plan is finally something more than just a fantasy.

Ultimately you end do up with a data center that looks, acts, and is managed quite differently than what you started with. So was that by evolution or revolution?

(Anyway, Dan has a lot of great stuff there; read up, then go forth and virtualize carefully but with great ultimate success.)

[Update: enterprise software is sexy when it is innovative. The relevance of this article to the current discussion is left as an exercise to the reader.]