The Console

Posted by Raghu Raghuram
Vice President of Datacenter and Desktop Platforms

It is widely acknowledged that in a couple of years, most if not all, new servers will be virtualized with the help of a hypervisor.

The average server in 2009 will overflow with compute, memory, and communication capacity - numerous CPU cores in every socket, DIMMs stocked with tens, maybe hundreds of gigabytes, and 10X increase in networking bandwidth. Virtualization is the most practical and obvious way to efficiently harness this capacity.  Especially when concurrent advances in processor and system hardware facilitate near zero-overhead virtualization, there is little reason not to virtualize.

With hypervisors on every server, the question is how do you put virtualization to best use? There are two points of view – the ‘operating system extension’ view and the ‘virtual infrastructure’ view.

Operating system vendors consider the hypervisor to be an extension to the operating system, providing mere single-machine partitioning capability while the OS continues to serve as the center of the world for such tasks as managing the hardware resources, providing system availability, governing security, and serving applications. The argument for this approach, of course, is that only incremental changes are required to existing practices for OS, hardware management and systems infrastructure in order to realize the partitioning benefits of virtualization. This is fine if you believe your current systems infrastructure is highly reliable, flexible and secure and needs only the additional benefit of higher resource utilization.

On the other hand, if you believe that today’s infrastructure is significantly complex, fragile and inflexible then you will be better served by fully exploiting additional fundamental advances that are enabled by virtualization. This is the virtual infrastructure viewpoint. Virtual infrastructure exploits the following:

Separation of the OS from the hardware resource management: For the first time in two decades, virtualization provides customers the opportunity to lift the OS cleanly off the hardware and have the OS primarily manage the application and end-users. Unlocking the hardware from the OS has already proven to simplify management of infrastructure and provides a clear separation between managing applications and managing infrastructure. Tying the OS to the application as a preconfigured, virtual appliance has proven to be remarkably powerful way of deploying and managing software faster.  In contrast, treating the hypervisor as an extension to the OS ignores this aspect of virtualization and continues the tight coupling between the OS and the hardware with all the attendant implications for complexity of change management, and makes the OS remain a single point of failure for the entire server and all its virtual machines.

Aggregation of resources and virtual resource pools: Second, these ubiquitous hypervisors distributed on every server can be orchestrated or clustered together through global resource managers to aggregate and create flexible, virtual pools of server, storage and network resources that can be freely allocated on demand. Hardware resources can be added or removed from these pools as needed. Power consumption can be orchestrated across these pools as needed. Failures of any hardware component are automatically and easily overcome using other available resources. Virtual resource pools may be dynamically and flexibly offered up to individual applications, groups of applications, business units or even discrete companies from a single shared infrastructure. This level of flexible, dynamic capacity management with an underlying shared infrastructure is only possible with virtual infrastructure because orchestrating dozens, if not hundreds of resource managers on disparate, commercial OSes is a daunting, if not impossible task.

Separation of Systems Infrastructure Services from the OS: Third, the operating system has traditionally struggled with two conflicting functions – the need to be as open and services-rich as possible in order to support all applications and the need to be simultaneously as closed as possible to deliver a high degree of reliability, availability and security.  In practice, the latter consideration has suffered at the hands of the former.  The goal of providing enabling a single OS to run all applications has resulted in unabated OS code growth from millions to tens or hundreds of millions of lines of code, mostly for application support.  As a result, customers pay a hefty price through expensive band-aids (examples: clustering, agents on every server, frequent patches for bugs and security holes, redundant identical hardware etc.) and complexity to work around this inherent design conflict.

With virtualization, there is now an opportunity to implement security, availability and reliability outside the OS, through the virtualization layer. Implementing these services outside the OS delivers significant benefits.  First, the implementation is global in scope - independent of any OS or any application. Second, implementing these capabilities once at the virtualization layer benefits every guest OS and application on every VM. You no longer have to implement and manage agents or software for availability or security or system protection per application. Third, since the implementation is not dependent on the OS, it is inherently less susceptible to attacks on the OS and therefore leads to a simpler, more robust infrastructure.

When you combine these three capabilities – applications deployed together with simplified OSes as virtual appliances; virtualization of distributed infrastructure to create virtual resource pools; and built-in, OS-independent systems infrastructure services to simplify infrastructure availability, reliability, and resource management – you get virtual infrastructure. 

Which model do you prefer? The old model of an integrated OS presiding over a monolithic systems infrastructure where the hardware, hypervisor, OS and applications are all bound together inflexibly, or the new model of virtual infrastructure that cleanly separates application requirements from hardware management and delivers shared, flexible, fault resistant system services universally to all your applications and operating systems.