If you didn’t manage to make it to the San Diego VMUG you missed a pretty good show. Darin Pendergraft from Quest was there showing off EcoShell and I had cooked up a couple of demos of my own.

One of the more popular topics was a script I wrote to determine LUN latencies and the VMs that are writing to particular LUNs. One of the trickiest challenges when adopting virtualization is the effective use and partitioning of shared resources. Technologies like VMware DRS address exactly this challenge, but it’s safe to say that we’re still nearer the beginning than the end when it comes to automatic resource allocation and leveling.

On the other hand, PowerCLI and the vSphere API give you all the tools you need to monitor your environment, allowing you to proactively move or reallocate VMs away from storage hotspots. In particular I showed off a couple of scripts that tell you what your LUN read and write latencies are, as well as listing all VMs on a given LUN. The best part, the scripts are extremely simple, requiring less than 25 lines of code for both of these things! Don’t believe me? See for yourself:

You can easily adapt Get-VMHostLunLatency to run periodically (say, every night) to tell you if you’re experiencing any storage slowness. If you are you can easily follow that up with Get-LunVM to identify VMs that should be split off to separate LUNs. PowerCLI makes it all really simple.

Note that if you want to run Get-VMHostLunLatency against vCenter you will need to have your stats level set to 2 or higher.

As for the stuff I presented at the VMUG, I’ve posted my slides to slideshare:

And you can also download all the scripts I used here:

If you’re in the San Diego area next week don’t miss the San Diego VMUG because I, your humble servant, will grace you with my presence and the vast stores of wisdom one only obtains through thousands of lonely nights in front of the computer.

I’ll be there to talk about – what else, PowerCLI and how it will help you automate your way to bliss.

As always I’ll be spending a lot of time on hands-on demos that will give you a first hand feel for what PowerCLI is and how it works. I’ll be covering reporting, provisioning and storage management, three of the most popular topics among PowerCLI users, with lots of new tips that are sure to save you lots of time. I’ll even be previewing a few new features from our upcoming release, so there’s something for everyone from beginner to expert.

Most people would call that a full day. BUT WAIT THERE”S MORE! You’ll also get the rare opportunity to see a presentation on the Virtualization Ecoshell. Now how much would you pay?

Hope to see you there.

People often want to know the last time that a VM was powered on. Unfortunately there is no completely fool-proof way to figure this out using just vCenter or ESX, but PowerCLI offers an approach that is pretty good, good enough that you can create a simple report and identify VMs that may be ready for that big hypervisor in the sky.

This is also a popular topic on the PowerCLI community, where LucD offers a solution to the problem, which relies on using events to determine when the poweron took place. Since Luc gave this solution a few things have happened:

1. The bug that caused Get-VIEvent to only return 1000 events is fixed in PowerCLI 4.0.
2. More importantly, you can pipe objects into Get-VIEvent, so you don’t have to deal with huge arrays of events. This is significant because a typical vCenter will have tens if not hundreds of thousands of events, covering everything from trivial to important things.

To take advantage of these facts I’ve written an updated PowerCLI script which generates a last-powered-on report.

One important note, ESX and vCenter actually create two events in connection with a VM poweron, one event when a user attempts to power a VM on, then another event if that power on actually succeeds. The code above actually generates a report of the last time a user attempted to power a VM on. When you attempt to power a VM on there are a lot of reasons it may fail, for instance your datastore may be full, in which case the poweron fails because ESX can’t create the memory swap file it needs (something I run into all the time).

Here’s a screen capture I made after running this against one of my ESX servers:

We can also use the always-useful, though often cryptic, select cmdlet to enhance this report by adding the total amount of space the VM uses, as follows.

This report suggests that the VM named 2k8 64 bit is a good candidate for removal, and in fact it is since I only used it to create this blog post, so it hasn’t been powered on since March 2009.

On the other hand you can see the report above is far from perfect, for example several powered on VMs have no entry for LastPoweron. Why? The reason is that, by default, ESX 4 only retains 1000 events (this is re-occurrence of the PowerCLI bug, I checked). Some of my VMs were powered on so long ago that their events no longer exist.

This brings us to another important point. ESX is not meant to store events for extended periods of time, instead vCenter will handle this for us. So the best way to generate this last powered on report is to run it against our vCenter instance, which will give us a much longer view.

Still this view does not stretch back forever, even in vCenter events will eventually expire. In vCenter 4, the event retention policy is even customizable. If you’re running vCenter 4, I’ve written this function to help you figure out what your retention policy is.

Here’s what it looks like in action:

If you know your retention policy and you know that all poweron/poweroff actions are performed through your vCenter server, you can be very confident that the information in the report is accurate, and use it to start moving unused VMs to archive storage, or even deleting them completely.

It is very important to note, though, that this report is pretty good, not perfect!! Don’t go around deleting stuff without careful human review beforehand.

I've gotten a lot of questions around when people will be able to get their hands on Onyx. We collected about 50 names at VMworld, so we're very happy to see all the different people who are interested. We haven't forgotten about you, we haven't changed our minds, we're getting things in order to preview Onyx and invites will be going out soon.

Thanks for your patience!

Do you enjoy reading huge piles of API documentation? Do you use wireshark to reverse-engineer wire protocols? Do you like using strace to figure out what the heck a program is doing?

Me neither, which is why I’m really excited about a little project we’ve been working on called “Project Onyx”. What exactly is Onyx? Quite simply, Onyx converts mouse clicks into PowerCLI scripts. Sort of like Excel’s macro record, except it’s for vSphere and doesn’t come with a stupid paper clip.

Does this sound like a tool you could use? We’re looking for a handful of dedicated people with a burning need for automation to help us get Onyx really solid. I would probably describe the tool as somewhere between alpha and pre-alpha and it’s a bit hard to set up at the moment, so we need people who are dedicated enough to work through the tough initial phases. Still it really is worth it. I’ve been working with vSphere API for about 3 years now and even I learned a lot by just toying around with Onyx.

So if you want to give Onyx a spin, here’s the deal. Come by session VM2241 at VMworld next week, and bring your business card. During the session we’ll be showing how to take scripts that Onyx generates and turn them into reusable bits of PowerShell code, that can be used in the pipeline with all the PowerCLI commands you know and love. There are two sessions, one on Tuesday at Noon and one on Tuesday at 5:30. Give us your card and we’ll pick 5 names from each session and give you first dibs on the raw power of Onyx. Enjoy the teaser video and hope to see you next week.

There’s a lot of stuff related to PowerCLI and PowerShell going on at VMworld, so it really deserves a summary of everything. It’s not enough to rename the event to PowerCLIWorld 2009 (maybe next year) but it’s still a pretty big list. Each of the sessions below either talks about PowerCLI directly and in depth, or introduces a new PowerShell tool.

1. VM2241 – Managing vSphere with PowerCLI
2. TA2650 – Take PowerCLI to the next level
3. DV2507 – VMware View – Upcoming Attractions
4. EA2526 – Sharepoint vSphere Best Practices
5. TA3521 — vNetwork Monitoring/Troubleshooting Demystification and Ecosystem Tool Demonstration

So if you’re looking to get more automated, scale out your existing environment, or just save a lot of time, be sure to check all these sessions out. Visit vmworld.com to add them to your schedule.

A lot of the magic VMware provides relies on shared storage. If a VM is on local storage, you can’t VMotion it, HA won’t restart it, and a host of other limitations. Do you know which of your VMs, if any, are on local storage? This script can make answering that question pretty easy. First, here’s a function that will locate datastores that use shared storage.

To use, launch PowerCLI, connect to your vCenter server, load the function above and run:

And you’ll wind up with a list of datastores that your hosts can share.

But that’s not what we’re after, we want datastores that can’t be shared. Not too hard,

Once we have $localOnly, we pass that to Get-VM.

This gives us the list of VMs that can’t be shared amongst hosts.

A lot of people see potential with PowerCLI, but struggle to get started
because PowerShell is a fairly new technology and not too many people know it
yet. If you’re getting started, here are my recommendations.

If you’re getting your hands on PowerShell for the very first time, I’d
recommend working through the first 3 or 4 sections of the PowerShell
owner’s manual, this will get you to the point where you can launch
PowerShell and are ready to run some commands.

After that, to master PowerShell you’ll have to master these two separate
areas:

1. The PowerShell language.
2. PowerShell cmdlets.

Know how to get help.

First, if you’re an absolute beginner, start by mastering PowerShell help.
Spending a few hours here will save you a lot of time later on. PowerShell
cmdlets are self-documenting, so the online help is accurate and definitive. It
also shows you critical facts about cmdlets, such as what arguments they accept,
what input types are needed, and what can be piped in to the cmdlet. There’s a
great intro on using help over at concentratedtech, and
you should also check out MSDN.
Use these guides, play around with the cmdlets, then try these same
principles with a few cmdlets of your own, you won’t regret it.

The key things to remember are “help <cmdlet name> –full” to get all
information about a particular cmdlet (example “help get-vm”) or “help
<cmdlet name> –examples” to get just the examples. One of the great things
about PowerShell is that help cmdlet –examples almost always yields a useful
example or two.

This works great for cmdlets, but unfortunately the situation is not so
simple for things that are part of the PowerShell language itself, like the if
statement or comparison operators. We’ll talk about how to get a handle on those
later.

The top 25 cmdlets worth knowing.

Anyway there are a lot of cmdlets and only so much time in the day. You may
be wondering where to focus your attention. There’s a great online of repository
of PowerShell code over at poshcode.org.
Recently I analyzed all the code on poshcode to count the frequencies of various
cmdlets and operators.

So here are the top 25 cmdlets used by code over at This sample is drawn from a lot of different authors all trying to accomplish different goals, so it's a good cross section of the most useful cmdlets. Note however that I would recommend that beginners not learn add-member until later, even though it is obviously a very useful cmdlet for experts.

Poshcode.org most popular cmdlets.

(*) This cmdlet ranks high due to the nature of the code on
poshcode.org, it’s a useful cmdlet to know but you should wait until you’re an
expert to really dig in to it.

If you’re starting out, you should begin by mastering most of these cmdlets
(you should skip add-member until you’ve become a master in your own right).
Remember, each of these has samples within its help that will get you on your
way. There’s one more cmdlet that VMware admins in particular get a lot of use
from, namely Measure-Object. If you’re mainly going to use PowerShell to manage
VMware, that one should be in your top 10 because our users use this cmdlet a
lot.

Operators and other language features.

As I mentioned, you’ll also need to master the PowerShell language. This
includes learning how to branch using if/then/else, how to build loops with
while and foreach, etc. For example, consider this code:

 So how, exactly, do if and else work? Help is available for all these things,
but in some cases it’s hard to track down. PowerShell’s help contains a lot of
topics in the form of about_something. For example, there is an
about_if, which talks about both the if and else statements. You can see all of
them by typing “help *about*”. The other thing to note in the statement above is
the use of “-eq”. This is a so-called comparison operator, and mastering these
is very important. Type “help about_comparison_operators” to learn all about
these. Also, here are the top comparison operators as found on poshcode.org:

 Poshcode.org most popular operators.

Lastly but not leastly, you’ll need to master the pipe. This tends to be
pretty easy because once you start piping it will seem very natural to you. Just
for reference, our trusty vertical friend appears 15491 times.

Getting really serious.

Another great little tutorial is the Top 10 Tips for Using Windows PowerShell. But if you’re really serious about upgrading your PowerShell skills,
you should consider getting a hold of either Hal Rotterberg’s Managing VMware Infrastructure
with Windows PowerShell TFM and Bruce Payette’s Windows
PowerShell in Action. Bruce’s book is the best from a theory perspective,
while Hal’s book lets you learn while you improve the way you manage VMware.

If you’re having trouble and need more help, don’t forget that help is
available in the PowerCLI
community.

Ever had a problem to solve but not sure how to proceed? This little function might help you out, it searches through all available PowerShell commands looking for a keyword you specify, and will search the full help text, including the examples.

For example, let’s say that in PowerCLI you’re trying to set up or diagnose VMotion, but not sure what cmdlets might help. All you’ve got to do is load this function, then run Search-Help vmotion, which will return this list.

CommandType     Name                            Definition
———–     —-                            ———-
Cmdlet          Move-VM                         Move-VM [[-VM] <VirtualMachi…
Cmdlet          New-Cluster                     New-Cluster [-Name] <String>…
Cmdlet          New-VMHostNetworkAdapter        New-VMHostNetworkAdapter [[-…
Cmdlet          Set-Cluster                     Set-Cluster [[-Cluster] <Clu…
Cmdlet          Set-VMHost                      Set-VMHost [[-State] <Nullab…
Cmdlet          Set-VMHostNetworkAdapter        Set-VMHostNetworkAdapter -Ph…

You can then run help on any of the results to drill down further.