EMC’s Chad Sakac follows up his multi-vendor iSCSI mega-post with the view from 50,000 feet.

Virtual Geek: So… What’s the BIG picture stuff going on under the covers?.

Customers are telling me consistently that they are looking to
transform to a new datacenter and IT model.   They use different words
when describing it.   Here are some variants:

• “I want to make IT a service back to the business – literally with an SLA model”
• “I see technologies coming together to enable something… I don’t know what to call it except ‘global datacenter optimization'”

They know they need to do it for all those reasons, but no-one wants to undertake one of those “here’s a vision – now stay with me and one day you will benefit” efforts – particularly these days.

• They need to save money at every step – constant capital expense saving.
• They need to become faster, more flexible at every step – constant refinement in both operational expense and speed.
• They need to use less power and become more green at every step
• We’ve all been around the block enough to know it’s got to be open, built on standards (Scott Lowe was totally right on that here)

Record Replay, the technology that allows you to reproduce what’s going on in a virtual machine with machine-level instructions, has been shown off at VMworlds past, but is just now coming into its own. You could experiment with it a bit in Workstation 6.0, but it is now available in a useful way in VMware Workstation 6.5, (in beta but has a new Release Candidate). Let’s let E Lewis introduce it in his new blog. Link: Better Software Development with Replay Debugging: VMware Workstation 6.5: Reverse and Replay Debugging is Here!.

I’m
proud to announce that VMware Workstation 6.5 includes new experimental
features that provide replay debugging for C/C++ developers using
Microsoft Visual Studio. Replay debugging allows developers to debug
recordings of programs running in virtual machines, and it is valuable
for finding, diagnosing, and fixing bugs that are not easily
reproduced, a particularly challenging class of bugs. Once the
manifestation of a bug has been recorded, it can be replayed (and
debugged) over and over again, and it is guaranteed to have
instruction-by-instruction identical behavior each time. In addition,
Workstation includes a feature that simulates reverse execution of the
program, making it easier to pin point the origin of a bug.

Aside from being insanely cool and perhaps the end of the heisenbug, I think this shows how VMware’s 10 years of experience manifests itself in innovation. Virtualization is about more than server consolidation, and once you are virtualized, the really interesting things can start to happen.

Here’s E demonstrating how this works. I think the UI has changed a bit since we filmed this. We’re running Visual Studio on the host, outside the VM, and attaching to a process inside the VM and putting in triggers and whatnot in the debugger as it replays until we track down the bug we’re looking for. If we go too far, we can always hit rewind.

Oh, and there’s a Lenovo laptop to be won: VMware Record and Replay Challenge

Sun’s Tim Bray has kicked off an interesting cross-blog conversation recently. He calls it the Wide Finder Project, and the basic issue is this: we’re moving toward a future of dealing with many CPUs with many cores but with (relatively) low clock rates. What are the interesting computer science and software development challenges this raises? How can we take advantage of architectures like this when dealing with parallelism is just so … painful using today’s paradigms and tools?

One thing I find fascinating about the discussion is how it’s coming from a strategic and futures-based motivation, but it’s taking place with a real roll-up-your-sleeves hacking ethic.  Tim postulated a simple, almost a toy, problem — parsing Apache log files. Tim and others are exploring this simple problem and how currently-available languages and language features affect how easy it is to take advantage of multi-CPU, multi-core architectures to rip through the file like a chainsaw through wet cardboard.

Tim started the ball rolling with Erlang (conclusion: wicked cool, but the I/O and regexp libraries aren’t up to snuff — likely a solvable problem) and others have run with it from there.

So why the Wide Finder problem on a virtualization blog? I ran across Kevin Johnson’s blog entry A Pile of Lamps.

He starts off in an earlier entry by scaring himself:

At the risk of sounding like a pessimist, I think we’ll end up with
thousands of little SOA web services engines. Each one handling a
single piece. Each one with its own HTTP stack. Each one using
PHP/Perl/Ruby/etc to implement the service functions. Each one sitting
on top of a tiny little mysql database. Eeeep!  I just scared myself – better drop this line of thought.  I’ll have nightmares for weeks.

Kevin points to Andrew Clifford’s The Dismantling of IT, which is not talking about v12n per se, but certainly fits into the picture we’re drawing here:

The most obvious change is that the new
architecture would remove technical layers, such as databases and
middleware. These capabilities would of course still exist, but they
could be standardised and hidden inside the systems. They would not
need so much management, and we would need fewer specialists.

Mark Masterson urges
him to reconcile with our future world of cooperating tiny little machines, all
busy message-passing and presumably acting somewhat autonomously to
avoid the nightmare management burden. Sounds a bit like a job for … virtualization and resource pools? Or as Kevin puts it:

Is the answer a combination of LAMP, embedded computing, cluster management, and virtualization?

A student asked a zen master "What is the value of virtualization — redshift or blueshift?" The zen master answered "Mu."

Ben Rockwood at Joyent:

I was asked to co-present with an engineer from Sun
at an upcoming conference in October. I asked him to do his slides and
then shoot me over the presentation so I could fill in my half. I
noticed that his view of virtualization and mine were very different.
To put it into jargon speak, there is a difference between Redshift
virtualization and Blueshift virtualization. …

Essentially it says that there are two different classes of business: “blueshift” companies that grow according to GDP
and are essentially over-served by Moore’s Law that computing power
doubles every two years, and “redshift” companies that grow off the
charts, and which are grossly under- served by Moore’s Law.

i.e., blueshift is server consolidation and redshift is dynamically bring new servers up quickly on your virtual infrastructure.  Zen blade master Martin MacLeod says that is the wrong answer — the value of virtualization is really in transforming your business process in both slow- and fast-growth businesses.

Martin MacLeod at Blade Watch:

But where virtualization really brings benefits is in the non-technical
arena. The ability to turn it all around, to be a real business
enabler, that the IT infrastructure can grow and adapt in line with the
business need, that we move to a system of service provisioning where
IT handle everything and provide the business with the virtual
instance, a world where I can request a server for a month to test that
.NET framework 3.0 works ok with my application, then give it back,
where I can be allocated more processing power or memory in minutes not
weeks due to the purchasing process needing sign off, processing and
parts delivery.

Link: Operating systems are old and busted | The Register.

USENIX – Operating systems aren’t so
great. They lounge like bloated monarchs on a database server — getting
far more credit than they’re worth. Clutched in their sausage fingers
are the keys to a kingdom far too vast to properly manage.

But Stanford professor Mendel Rosenblum believes virtualization may
be the guillotine that cuts the OS reign down to size. Rosenblum, who
is also a founder of VMware, called for heads to roll during his
opening keynote at the USENIX conference in Santa Clara…Virtually
roll, of course.

Massimo re Ferre’ with a long think-piece on VMware, Microsoft, market forces, value-add, and paradigm shifts in the data center: Will Microsoft sunset VMware? All his points are good, but I particularly like this one:

Maniacally focused: you need to consider
that for Microsoft this is one of the many battle-grounds. Windows
Virtualization is a line-item feature in a new OS release. This has
nothing to do with the fact that, for them, this is very important or
not. It remains a fact that their overall efforts will be diluted
across a number of markets that span from OS dominance to databases,
from mail systems to development tools etc etc. For VMware this is
"THE" market. They are laser focused to provide the best x86
virtualization experience and solutions. That’s what they do and they
can afford to run full steam towards that result. Whether they will
succeed is another matter but it’s important to notice.

And in his conclusion he invokes a paradigm shift coming in how we manage complexity in the data center. Massimo is particularly excited by Virtual Appliances (as am I), but 
his vision of the future data center isn’t dependent on that.

These are the reasons for which I don’t think Microsoft is going to sunset VMware. Clearly they will pose a challenge on them (a very tough one) but I don’t see VMware as being kicked out so easily. And the number one reason is because I really think that our Datacenters needs to be re-designed from the ground up. Let me quote myself: "This is a fascinating scenario and as you can imagine it involves more than just developing a hypervisor with a management interface: it involves creating a new culture on how we deal with IT, taking all the pieces apart and rebuild our datacenters in a much more efficient way". Now if we agree that Microsoft is making a lot of money out of this "legacy" model (this is a fact) but that we need to change it (the legacy model) to become more efficient anyway … do you think that Microsoft itself could be the agent of change in this case? If they are not pushed they will try to maintain the status-quo (well status-quo with license upgrades as new product versions come along). I remember 5 years ago I went to Microsoft asking them what they were doing about virtualization since this little company called VMware was having brilliant ideas on how to consolidate servers and they told me that they response to that was Itanium and Windows 2000 Datacenter.

See also the responses at the VMTN Forums.