Posted by
Vitali Baruh
PowerCLI QE team

Provisioning of new virtual machines often includes Operating System (OS) customization.

A typical provisioning workflow could look like this:

1.  Clone a virtual machine from a template and specify a customization specification

2.  Start the VM. This triggers the customization process

3.  Wait for the customization to complete

4.  Perform additional setup steps

Step three is quite tricky, because there’s no easy way to find out when the customization has completed within the guest operating system and if it has succeeded or has failed.

This step could be implemented in an optimistic manner with sleep for 10 minutes, but it could also be implemented in a more robust and informative way. To do this, we could create a script with the following specification:

Input

• List of virtual machines which should be monitored for the completion of a customization process
• Timeout in seconds that the script should wait for the customization to end

Output

• List of PSObjects where each object will hold a passed VM and its customization status – successful, failed, started, et.

The function is based on several virtual machine events defined by vCenter Server:

• VmStartingEvent – posted when VM is powered on
• CustomizationStartedEvent – posted when customization is started
• CustomizationSucceeded – posted when customization succeeds
• CustomizationFailed – posted when customization fails

Our script workflow is pretty simple:

• 1. For each VM we will look for the last VmStarting event, because the customization process starts after the VM has been powered on
  • 1.1. If such an event is found, change the status to “CustomizationNotStarted”
  • 1.2. If such an event is not found, change the status to “VmNotStarted”
• 2. Start a loop
  • 2.1. For each VM with status “CustomizationNotStarted”
    • 2.1.1. Check for posted CustomizationStartedEvent after the last power-on operation
      • 2.1.1.1. If such an event is found, change the status to “CustomizationStarted”
  • 2.2. For each VM with status “CustomizationStarted”
    • 2.2.1. Check for Succeded or Failed Event
      • 2.2.1.1. If such an event is found, change the status to the corresponding “CustomizationSucceeded” or “CustomizationFailed” values
  • 2.3. Check is there more VMs with status “CustomizationNotStarted” or “CustomizationStarted” (we should continue to monitor these VMs)
    • 2.3.1. If no, break the loop
  • 2.4. Check whether the specified timeout has elapsed
    • 2.4.1. If yes, break the loop
• 3. Return result

Now let see the script in action!

Step 1 – retrieve template, OS customization spec, target vmhost and target datastore.

As result we have several variables that are needed to start cloning with customization tasks.

Step 2 – start clone operations for two VMs

As result we have $vms variable holding our virtual machines.

Step 3 – start virtual machines, which triggers customization process and monitor it with our script.

The script is started with two virtual machines and also we specify to wait maximum 600 seconds for OS customization to complete.

After the start the script reports each VM which will be monitored.

The next screen shows that after about 45 second customization start events have been posted for both VMs.

The last screen shot shows the moments when the two customization succeed events have been posted and also the result of the script.

Finally the $result variable contains two custom objects. Each object contains VM and CustomizationStatus properties.

Our script could iterate over the $result variable and to perform some additional setup actions – installing software not included in the template, applying specific software configurations, etc.

Of course for these additional setup actions we can use the Invoke-VMScript and Copy-VMGuestFile cmdlets.

Now let’s see the code of WaitVmCustomization.ps1 script:

<#

.SYNOPSIS
Waits customization process for list virtual machines to completes.

.DESCRIPTION
Waits customization process for list virtual machines to completes.
The script returns if customization process ends for all virtual machines or if the specified timeout elapses.
The script returns PSObject for each specified VM.
The output object has VM and CustomizationStatus properties.

.EXAMPLE
$vm = 1..10 | foreach { New-VM -Template WindowsXPTemplate -OSCustomizationSpec WindowsXPCustomizaionSpec -Name "winxp-$_" }
.\WaitVmCustomization.ps1 -vmList $vm -timeoutSeconds 600

.NOTES
The script is based on sveral vCenter events.
* VmStarting event – this event is posted on power on operation
* CustomizationStartedEvent event – this event is posted for VM when customiztion has started
* CustomizationSucceeded event – this event is posted for VM when customization has successfully completed
* CustomizationFailed – this event is posted for VM when customization has failed

Possible CustomizationStatus values are:
* "VmNotStarted" – if it was not found VmStarting event for specific VM.
* "CustomizationNotStarted" – if it was not found CustomizationStarterdEvent for specific VM.
* "CustomizationStarted" – CustomizationStartedEvent was found, but Succeeded or Failed event were not found
* "CustomizationSucceeded" – CustomizationSucceeded event was found for this VM
* "CustomizationFailed" – CustomizationFailed event wass found for this VM

#>
[CmdletBinding()]
param(
   # VMs to monitor for OS customization completion
   [Parameter(Mandatory=$true)]
   [ValidateNotNullOrEmpty()]
   [VMware.VimAutomation.ViCore.Types.V1.Inventory.VirtualMachine[]] $vmList,
  
   # timeout in seconds to wait
   [int] $timeoutSeconds = 600
)

# constants for status
      $STATUS_VM_NOT_STARTED = "VmNotStarted"
      $STATUS_CUSTOMIZATION_NOT_STARTED = "CustomizationNotStarted"
      $STATUS_STARTED = "CustomizationStarted"
      $STATUS_SUCCEEDED = "CustomizationSucceeded"
      $STATUS_FAILED = "CustomizationFailed"
     
      $STATUS_NOT_COMPLETED_LIST = @( $STATUS_CUSTOMIZATION_NOT_STARTED, $STATUS_STARTED )
     
# constants for event types     
      $EVENT_TYPE_CUSTOMIZATION_STARTED = "VMware.Vim.CustomizationStartedEvent"
      $EVENT_TYPE_CUSTOMIZATION_SUCCEEDED = "VMware.Vim.CustomizationSucceeded"
      $EVENT_TYPE_CUSTOMIZATION_FAILED = "VMware.Vim.CustomizationFailed"
      $EVENT_TYPE_VM_START = "VMware.Vim.VmStartingEvent"

# seconds to sleep before next loop iteration
      $WAIT_INTERVAL_SECONDS = 15
     
function main($vmList, $timeoutSeconds) {
   # the moment in which the script has started
   # the maximum time to wait is measured from this moment
   $startTime = Get-Date
  
   # we will check for "start vm" events 5 minutes before current moment
   $startTimeEventFilter = $startTime.AddMinutes(-5)
  
   # initializing list of helper objects
   # each object holds VM, customization status and the last VmStarting event
   $vmDescriptors = New-Object System.Collections.ArrayList
   foreach($vm in $vmList) {
      Write-Host "Start monitoring customization process for vm '$vm'"
      $obj = "" | select VM,CustomizationStatus,StartVMEvent
      $obj.VM = $vm
      # getting all events for the $vm,
      #  filter them by type,
      #  sort them by CreatedTime,
      #  get the last one
      $obj.StartVMEvent = Get-VIEvent -Entity $vm -Start $startTimeEventFilter | `
         where { $_ -is $EVENT_TYPE_VM_START } |
         Sort CreatedTime |
         Select -Last 1
        
      if (-not $obj.StartVMEvent) {
         $obj.CustomizationStatus = $STATUS_VM_NOT_STARTED
      } else {
         $obj.CustomizationStatus = $STATUS_CUSTOMIZATION_NOT_STARTED
      }
     
      [void]($vmDescriptors.Add($obj))
   }        
  
   # declaring script block which will evaulate whether
   # to continue waiting for customization status update
   $shouldContinue = {
      # is there more virtual machines to wait for customization status update
      # we should wait for VMs with status $STATUS_STARTED or $STATUS_CUSTOMIZATION_NOT_STARTED
      $notCompletedVms = $vmDescriptors | `
         where { $STATUS_NOT_COMPLETED_LIST -contains $_.CustomizationStatus }

      # evaulating the time that has elapsed since the script is running
      $currentTime = Get-Date
      $timeElapsed = $currentTime – $startTime
     
      $timoutNotElapsed = ($timeElapsed.TotalSeconds -lt $timeoutSeconds)
     
      # returns $true if there are more virtual machines to monitor
      # and the timeout is not elapsed
      return ( ($notCompletedVms -ne $null) -and ($timoutNotElapsed) )
   }
     
   while (& $shouldContinue) {
      foreach ($vmItem in $vmDescriptors) {
         $vmName = $vmItem.VM.Name
         switch ($vmItem.CustomizationStatus) {
            $STATUS_CUSTOMIZATION_NOT_STARTED {
               # we should check for customization started event
               $vmEvents = Get-VIEvent -Entity $vmItem.VM -Start $vmItem.StartVMEvent.CreatedTime
               $startEvent = $vmEvents | where { $_ -is $EVENT_TYPE_CUSTOMIZATION_STARTED }
               if ($startEvent) {
                  $vmItem.CustomizationStatus = $STATUS_STARTED
                  Write-Host "Customization for VM '$vmName' has started"
               }
               break;
            }
            $STATUS_STARTED {
               # we should check for customization succeeded or failed event
               $vmEvents = Get-VIEvent -Entity $vmItem.VM -Start $vmItem.StartVMEvent.CreatedTime
               $succeedEvent = $vmEvents | where { $_ -is $EVENT_TYPE_CUSTOMIZATION_SUCCEEDED }
               $failedEvent = $vmEvents | where { $_ -is $EVENT_TYPE_CUSTOMIZATION_FAILED }
               if ($succeedEvent) {
                  $vmItem.CustomizationStatus = $STATUS_SUCCEEDED
                  Write-Host "Customization for VM '$vmName' has successfully completed"
               }
               if ($failedEvent) {
                  $vmItem.CustomizationStatus = $STATUS_FAILED
                  Write-Host "Customization for VM '$vmName' has failed"
               }
               break;
            }
            default {
               # in all other cases there is nothing to do
               #    $STATUS_VM_NOT_STARTED -> if VM is not started, there's no point to look for customization events
               #    $STATUS_SUCCEEDED -> customization is already succeeded
               #    $STATUS_FAILED -> customization
               break;
            }
         } # enf of switch
      } # end of the freach loop
     
      Write-Host "Sleeping for $WAIT_INTERVAL_SECONDS seconds"
      Sleep $WAIT_INTERVAL_SECONDS
   } # end of while loop
  
   # preparing result, without the helper column StartVMEvent
   $result = $vmDescriptors | select VM,CustomizationStatus
   return $result
}

#calling the main function
main $vmList $timeoutSeconds

And here is the code for the scenario script:

# prepare necesary objects for clone operations
$template = Get-Template WindowsXPTemplate
$spec = Get-OSCustomizationSpec WindowsXPSpec
$vmhost = Get-VMHost | select -First 1
$datastore = Get-Datastore -VMHost $vmhost -Name Storage1

# Start cloning operations
$vms = 1..2 | % { New-VM -Name "WinXP-$_" -ResourcePool $vmhost -Template $template -Datastore $datastore -OSCustomizationSpec $spec }

# Trigger customization and wait its completion
$vms = $vms | Start-VM
$result = .\WaitVmCustomization.ps1 -vmList $vms -timeoutSeconds 600

Several notes about the script

• It doesn’t accept pipeline input, because it checks the customization status for multiple virtual machines simultaneously. This won’t be possible if the process block of the script is executed for each object passed by the pipeline.
• It also shows how to work with specific type of events.
• Its VIEvent searches are optimized by specifying concrete entity and start time filters

Hope that this blog post is interesting and useful.

Posted by
Leni Kirilov
PowerCLI QE team

Why would I want to virtualize my test environment?

Can I do it easily?

How can I automate а virtualized environment?

What about the configuration?

Is there anything more I can automate?

In this post you will have a chance to learn directly from the team building these automation tools – the VMware PowerCLI Quality team.

We can’t imagine our test infrastructure without virtual machines and automation scripts.

Be efficient – Virtualize & automate your test environment

Recently I met with my friend Michael, a software quality engineer, who is validating the functional correctness and completeness of “Cake Wizard”. Cake Wizard is an online store, consisting of a database of items for sale, accounts, online payments, etc. It is an enterprise application developed with JEE.

In the process of validating the product, Michael needed a dedicated test environment. Michael told me he created a complete set of automation tools for executing the tests, gathering the results, and reverting the environment to its initial state. He was really proud of his accomplishment. He also integrated download and installation of product builds with a Git source control system for the test cases, Jenkins/Hudson for scheduling jobs on the test run process, and a custom test runner to execute JUnit… I was impressed! Michael had done a really great job by automating so many things. But he mentioned that he had many problems with automating the process. There were even lost test run results due to bugs.

I suggested a different approach – automation & virtualization. Michael looked confused, not to say surprised. “But virtualization is used only in production systems for better load-distribution, resource utilization and cloud services… what does it have to do with automation testing?” I smiled…

Nowadays, most testing looks like this: you get a physical environment, execute tests on it, and revert it to a clean state by using images. This approach is convenient because you don’t have to configure everything all over again. But there are some negatives as well.

• When you modify the hardware configuration, you must update the image, because it’s hardware specific;
• If the hardware configuration fails, you can lose all your data and test results, which is a potential risk for the project.
• The approach doesn’t scale well because a single hardware configuration can be at most a single test environment.

Virtualization has the same advantages as images-driven test environment setup and more:

• Better utilization – you can run more than one virtual machine (VM) on a single piece of hardware as long as there are enough resources. Provide your Quality Engineers their own environment to play with!
• Revert to a previous state – cleanup is easy because VMs are designed with that in mind. You can easily debug a problem by using snapshots. They are quite fast, too.
• Hardware failure – it’s irrelevant for the virtualized infrastructure. Since VMs are just files, you can back them up, in order to minimize loss, in case of server crashes
• Pool of resources – no need to reconfigure and recreate images when you buy new hardware. You just add it to the pool of resources.
• Maintenance & automation – with minimal effort you can replicate VMs by using virtualization and the already available automation scripts. You can even create your own automation scripts!

My friend Michael heard my arguments but was still skeptical. He had already created his environment and he asked me: “What’s the easiest way to virtualize it? What are the first steps?”

Let’s see how easy we can use VMware virtualization to:

• automate setting up a test environment,
• set up the test software
• collect test results

Set up and configure virtual test environment

First you will need a virtualization infrastructure, so let’s install VMware ESXi Hypervisor. It provides tools to run and manage virtual machines. Follow these instructions

You start your virtualized test environment by creating a empty virtual machine (VM). You install the OS (Guest OS) and applications you need just once and save the result VM. You can use it to clone and thus multiply the VM as many times as you need.

Let’s install Windows Server 2008 and a JBoss Application Server 6 on our VM and name it “Manually created VM”.

Manually configuring a VM is OK only in the process of learning the VM specifics. As Quality Engineers, we want to automate as much as possible in our work.

“How do we automate a virtual test environment and what is the best tool?”

The preferred tool for virtualization automation is VMware vSphere PowerCLI. It is closely developed with the VMware infrastructure suite (called vSphere). We need only some of the many features which PowerCLI provided, to create our test environment. The product platform is Microsoft PowerShell (running on .Net) – powerful yet very user friendly CLI (command line interpreter) which provides great console commands(called cmdlets) for managing other Windows products like SQL Server, IIS, Windows, MS Office etc. Even though PowerCLI is based on Microsoft technology, you can use it to install and configure Windows, as well as Red Hat Guest OS on your VM. Let’s see it in action:

Download and install latest PowerCLI version from here:

http://www.vmware.com/go/powercli

To start using PowerCLI you should connect to your local VMware ESXi Server (which you should have installed and configured beforehand).

Connect-VIServer 192.168.1.1 -User ‘administrator’ -Password ‘pass1234’

# For later processing get your VM and store it in a variable.

$vm = Get-VM –Name “Test_Environment_VM_1”

Start-VM $vm # you can start using your VM for configuration

It’s highly recommended to install VMware Tools on your VM – it makes some of the trickier operations possible amd improves VM’s performance…

Let’s get all network interface cards of the OS installed on the VM and then configure them.

# get all network cards. GuestUser is the username required for login to the GuestOS

$nics = Get-VMGuestNetworkInterface $vm -GuestUser ‘administrator’ -GuestPassword ‘pass1234’

We have dedicated the 192.168.2.* subnetwork for our test infrastructure, so let’s update the VMs IP and DNS addresses:

Set-VMGuestNetworkInterface $nics[0] -Ip 192.168.2.2 –Dns 192.168.1.2 ` -GuestUser ‘administrator’ -GuestPassword ‘pass1234’

If you need a more advanced configuration for your Guest OS, you can take a look at the OSCustomization cmdlets, which give you the ability to configure user accounts, active directory, workgroups as the VM is created. There are examples in the documentation.

After you finish configuring your networking, you should create a snapshot that saves the state of your VM. This is useful when you want to revert this VM to a previous state. Let’s create a snapshot for future use:

New-Snapshot –VM $vm –Name “After network configuration – 2012.02.20”

Create environment configuration files

It’s good to keep various data about your environment. We create a hashtable called $env1. It can store the names of virtual machines, the servers’ IP addresses, DB table names, etc.

You can use the NIC objects to build this configuration file for future test executions or to form a report. The $nics variable contains all the information you need such as IP, DNS, Subnetmask, etc:

$env1[“IpAddress”] = $nics[0].Ip

You can save the hashtable $env1 to a file and use it as configuration for your “execute test” script. This way you can easily switch between environments with various properties.

$env1 | Export-CliXml “d:\CakeWizard\Testing\Environments\env1_win2008.xml”

# to use the configuration you import it using this call:

$env1Config = Import-CliXml “d:\CakeWizard\Testing\Environments\env1_win2008.xml”

Now you can combine all these one-liners in a short script that configures VM’s networking and creates a report/config file with the current IPs that are in use.

Simple, huh?

“What about applications and integration with other systems like Source Control Repositories or test execution systems like Hudson/Jenkins?”

Set up tests and test framework

For even greater customization, you can execute custom scripts in the Guest OS! By copying files to the Guest OS and executing them, you can automate the following action:

• downloading and installing the software under testing;
• syncing sources with tests and frameworks;
• preparing reports;
• managing Microsoft products via their PowerShell API;
• pretty much everything you can think of!

You can copy different executable files, ZIP files with scripts, DLLs… everything you want, to the VM’s local file system. Let’s sync the tests from our source control repository. I’ve written a short script to create workspace and sync everything with it.

Copy-VMGuestFile -LocalToGuest `
-VM $vm -Source “d:\CakeWizard\tools\sources_sync.bat” –Destination “c:\CakeWizard\tools\perforce_sync.bat” `
-GuestUser “administrator” -GuestPassword ‘pass1234’

# execute the script we’ve copied
Invoke-VMScript -ScriptText “c:\CakeWizard\tools\sources_sync.bat” `
-VM $vm -GuestUser ‘administrator’ -GuestPassword ‘pass1234’

The ScriptText parameter can point to another script or be the script itself. So that makes it pretty easy to be included in Hudson/Jenkins job. 

The script you execute returns live results to your screen while working inside the VM. This is great for monitoring the progress of a long test run or analyzing test run results. The best thing about the Invoke-VMScript cmdlet is that it can execute many types of scripts: Powershell, BAT for Windows and Bash for Linux systems!

Now that we’ve prepared everything, it’s good time to create another snapshot. Let’s name it “Ready for test execution” or “Clean state”.

Install a product

Let’s upgrade to the latest build version of our product. We have a Powershell script that does the job. Let’s see how to execute it:

Invoke-VMScript -ScriptText “c:\CakeWizard\Tools\upgradeProduct.ps1” `
-VM $vm -GuestUser ‘administrator’ -GuestPassword ‘pass1234’

Create a report

You can generate a test run execution report after the test execution. Our report holds information about the test id, test status, log files and execution times.

Invoke-VMScript -ScriptText “c:\CakeWizard\Tools\generateReport.bat” `
-VM $vm -GuestUser ‘administrator’ -GuestPassword ‘pass1234’

# locally copy a report
Copy-VMGuestFile -GuestToLocal `
-VM $vm -Source “c:\Tests\Results\Report\*.*” -Destination “d:\TestResults\Report\*.*” `
-GuestUser “administrator” -GuestPassword ‘pass1234’

Clean up the environment / Revert to snapshot

When you run a test (again, you can use Invoke-VMScript to trigger execution), you might pollute your brand new Test_Environment_VM_1. Wouldn’t it be great, if you could always go back to a clean state of your test environment? This way you can be comfortable with running even the deadliest, most insane, build-crashing tests. Our new script for creating a new test environment could do the trick, but there’s an even easier way to fix the ruined VM – snapshots!

# reverting the vm to a previous state
Set-VM –VM $vm –Snapshot “Clean state”

“Set up once – test multiple versions!”

You can easily customize these one-line scripts for your own use cases. By combining all of the scripts, you will automate your virtual environment! You are just a step away from provisioning multiple test environments!

Copy VM to create a new virtual test environment

A VM is just a set of files, so we can copy a VM by copying its files. Let’s create a new environment called “Test_Environment_VM_2”.

# change directory to ‘vmstores:’. It gives you access to the datastore tree.

# navigate to your datastore and Test_Environment_VM_1 location and copy it

cd vmstores:

Copy-Item “.\Test_Environment_VM_1” -Destination “Test_Environment_VM_2” –Recurse

# now let’s register the new VM

New-VM –VMFilePath “[datastore_name] .\Test_Environment_VM_2\Test_Environment_VM_2.vmx” –Name “Test_Environment_VM_2”| Start-VM

And you are ready with your new environment and John can help you with testing “Cake Wizard”

Note – there is an easier way to clone a virtual machine – check VM templates, feature available in vCenter Server.

Virtualization makes your testing process quite agile and scalable – you can create more VMs on which to run tests, than the number of the physical machines you can buy or power up. Even your developer colleagues can use the VMs for developing the software in the first place!

You can also easily revert a VM to a clean state, without losing any time for configuration.

You can set up and configure the VM so that most of the things that you can do with a physical machine are possible (except for testing specific/special hardware devices).

You will no longer fear of hardware failures, because your VM can be automatically migrated with no downtime.

You can automate all these tasks with the highly-rated tool – VMware vSphere PowerCLI.

Well, Michael, I think that now you understand the advantages of virtualization in testing. As quirky as this concept sounds, you’ll find that the basics are not that different from the classical approach. Do you have any other questions, we can discuss over a cup of coffee?

Posted by
Alan Renouf
Technical Marketing

A question I receive all the time is:

How can I list all license keys I have for all of my Virtual Centers in my Infrastructure ?

 

Luckily PowerCLI enables us to work with multiple vCenters all at one, we can connect to many vCenters or hosts and pull back information on all of these at the same time, to do this you need to set the PowerCLI Configuration to work in multiple mode and then you can connect to more than one vCenter at the same time, to do this see the following example:

Once we are in multiple configuration mode we can then use Connect-VIserver on as many vCenters or hosts as you like, we can use this to connect to multiple vCenters with the same credentials or just list each one like below:

Connect to multiple vCenters with the same username and password:

Connect-VIServer VC1, VC2, VC3, VC4 –User Administrator –Password MyPa$$word

Connect to multiple vCenters with different credentials:

Connect-VIServer VC1 –User Administrator1 –Password MyPa$$word1

Connect-VIServer  VC2 –User Administrator2 –Password MyPa$$word2

Connect-VIServer VC3 –User Administrator3 –Password MyPa$$word3

Connect-VIServer VC4 –User Administrator4 –Password MyPa$$word4

Once connected this information is stored in the $DefaultVIServers variable, this can easily be viewed at any time:

We are also easily able to run scripts against these connections, like for example the License script at the end of this post, this lists all license keys and their information for each vCenter:

(You will excuse me if I exclude the actual key)

The Code

# Set to multiple VC Mode
if(((Get-PowerCLIConfiguration).DefaultVIServerMode) -ne "Multiple") {
    Set-PowerCLIConfiguration -DefaultVIServerMode Multiple | Out-Null
}

# Make sure you connect to your VCs here

# Get the license info from each VC in turn
$vSphereLicInfo = @()
$ServiceInstance = Get-View ServiceInstance
Foreach ($LicenseMan in Get-View ($ServiceInstance | Select -First 1).Content.LicenseManager) {
    Foreach ($License in ($LicenseMan | Select -ExpandProperty Licenses)) {
        $Details = "" |Select VC, Name, Key, Total, Used, ExpirationDate , Information
        $Details.VC = ([Uri]$LicenseMan.Client.ServiceUrl).Host
        $Details.Name= $License.Name
        $Details.Key= $License.LicenseKey
        $Details.Total= $License.Total
        $Details.Used= $License.Used
        $Details.Information= $License.Labels | Select -expand Value
        $Details.ExpirationDate = $License.Properties | Where { $_.key -eq "expirationDate" } | Select -ExpandProperty Value
        $vSphereLicInfo += $Details
    }
}
$vSphereLicInfo | Format-Table -AutoSize

Get notification of new blog postings and more by following VMware PowerCLI on Twitter: @PowerCLI

Posted by
Alan Renouf
Technical Marketing

At the end of last year, December the 9th 2011 to be exact, I was lucky enough to present at the Dutch VMUG with Luc Dekens who is known very well in the PowerCLI Community and for his excellent PowerCLI blog over at Lucd.Info

Our presentation was entitled “vSphere 5.0 Automation with PowerCLI” and was based on a presentation which was given by Raymond Epping and Jan-Willem Lammers, their presentation showed the top 10 features and enhancements which were introduced in vSphere 5.0.

Luc and I expanded on this by seeing which of these features we could automate in PowerCLI, the session was well received and we came up with some great scripts and functions which Luc and I have been sharing on our blogs over the past weeks. 

You can read more about the scripts and functions by following the links in the “Further reading and scripts” section of this post.

Presentation

The full presentation is available from the Dutch VMUG site, check it out there are some great slides and scripts in there: http://www.vmug.nl/downloads/VMUG2011/vs5AutomationwithPowercli.pdf

How long do your vMotion/SvMotion take ?

One of the scripts I wanted to highlight was a handy advanced function we created called Get-MotionDuration.

Get-MotionDuration will look back through the events of your vCenter and recall your vMotion and SvMotion history, it will then present you with a list of these events and how long they took, an example of this output can be seen below.

Obviously the timings will change based on the size and how utilized the virtual machine is but this is great for comparing vSphere 4 systems against vSphere 5 where multiple enhancements have been made to the speed of vMotion and SvMotion giving you the benefit of quickly being able to put hosts into maintenance mode or move VMs to new datastores.

Watch it in action

The following video shows this script in action:

Get-MotionDuration Script

Function Get-MotionDuration {
    $events = Get-VIEvent -Start (Get-Date).AddDays(-1)
    $relocates = $events |
        where {$_.GetType().Name -eq "TaskEvent" -and $_.Info.DescriptionId -eq "VirtualMachine.migrate" -or $_.Info.DescriptionId -eq "VirtualMachine.relocate"}
    foreach($task in $relocates){
        $tEvents = $events | where {$_.ChainId -eq $task.ChainId} |
            Sort-Object -Property CreatedTime
        if($tEvents.Count){
            New-Object PSObject -Property @{
                Name = $tEvents[0].Vm.Name
                Type = &{if($tEvents[0].Host.Name -eq $tEvents[-1].Host.Name){"svMotion"}else{"vMotion"}}
                StartTime = $tEvents[0].CreatedTime
                EndTime = $tEvents[-1].CreatedTime
                Duration = New-TimeSpan -Start $tEvents[0].CreatedTime -End $tEvents[-1].CreatedTime
            }
        }
    }
}

Connect-VIServer MyViServer –User Administrator –Password “Pa$$w0rd”

Get-MotionDuration | FT -AutoSize

Further reading and scripts

The following blog posts were created with examples from the Dutch VMUG presentation, please find time to check them out as they may save you time in the future….

• Full Dutch VMUG presentation (in English)
• vSphere 5 Top 10 – Storage DRS
• vSphere 5 Top 10 – HA
• vSphere 5 Top 10 – vMotion
• vSphere 5 Top 10 – NetIOC
• vSphere 5 Top 10 – VMFS5
• Updating your Image Profile

Posted by
Luc Dekens
PowerCLI Community Member

Introduction

The VMTN PowerCLI Community is a vast repository of PowerCLI scripts and snippets. At the time of writing this post, there were nearly 5000 threads and more than 80 documents posted.

With so many threads present, it happens quite often that users ask a question that was already answered in a different thread. The problem is that most users of the PowerCLI Community don’t really know how to search the threads and documents. Admittedly, the search functionality offered in the VMTN communities is not the easiest and most intuitive one to use. Hence this post that should give you a couple of pointers on how to use the PowerCLI Community as (very) big repository of PowerCLI goodies.

Starting a search

On any page of the community you will see a search box near the top right corner of your browser page.

Let’s type some text, for example ‘svmotion’ in the search field. The forum software will now automatically open a drop-down box with some first results.

You have 3 options to continue.

1. By clicking the magnifying glass or the ‘View All Results’ text, the Search page will open. From there you can select one of the displayed threads or documents, or you can further refine your search.

2. The results that are displayed in the drop-down box do not come from the PowerCLI Community alone. They are entries found throughout all the Communities in the VMTN forum. By clicking on the ‘Only for VMware vSphere ™ PowerCLI’ hyperlink at the top of the drop-down box, the contents of the drop-down box will change. It will contain only entries coming from the PowerCLI Community.

If you now click on the ‘View All Results’ hyperlink, you will be taken to the Search page, similar to the first option. But the Search page will only contain entries from the PowerCLI Community.

Note that clicking the magnifying glass you will be taken to the Search page, but the results will come from all VMTN communities!

You can go back to the results from all the VMTN communities by clicking the ‘All’ hyperlink at the top of the drop-down box.

3. If you see in the list a post whose title suggests that it might contain what you are looking for, you can click on that line. The selected thread will open.

The Search page

This is the page where you have a plethora of options at your disposal to refine your search.

1.  Add words to refine your selection criteria.

On the ‘Search Tips’ you can find many ways to fine-tune your search. A short extract of the more important search rules:

• When you enter multiple words, the search will look for threads and documents that contain all of the words.
• When 1 or both of the words should appear in the thread or document, use OR between the words. For example: ‘svmotion OR vmotion’
• Put the logical operators in Caps to distinguish them from your search keywords.
• When looking for an exact sequence of words, place them in double quotes. For example: ‘”background job”’
• Use the fuzzy search (~) when you are not 100% sure of the word(s) you’re looking for. For example: ‘motion~’’
• Use a weight (^n) on specific words to make them more important in your search. For example: ‘svmotion^2 vmx’. This will make the presence of ‘svmotion’ twice as important as the presence of ‘vmx’.
• Indicate in which part of the post (subject, body, tags or attachmentsText) you want to search for a word. For example: ‘subject:svmotion’
• Specify words that should not be in the results. For example: ‘svmotion NOT vmx’
• Use parenthesis to create complex search strings. For example: ‘svmotion AND (vmdk OR vmx)’

2.  Under ‘More options’ you can limit your search to a specific Community or a specific person. When you want to select a specific Community, type part of the Community name and select one of the Communities that is proposed.

Note that you will have to start with ‘vsphere’ to get to the PowerCLI Community !

The search will now be limited to the PowerCLI Community.

3. When the search returns a number of hits, you can order the results by Date, Subject or Relevance.

The options Subject and Date are self-explanatory. The Relevance option is a percentage given by the Jive search algorithm to the search results, indicating how much a specific search result corresponds with the search query.

The best results, according to the search algorithm, will appear at the top of the list.

4. With the ‘What’ options you can filter the results based on where they are found. In the PowerCLI Community the only usable options are All, Discussions or Documents. The Blog Posts and Links do not seem to be used in the PowerCLI Community.

The options correspond with the Tabs you see on the PowerCLI Community page.

5. The ‘When’ option allows you to specify in which period in the time you want to search. You can select Anytime, the past Day, Week, Month or Year.

Since the oldest threads in the PowerCLI Community date from December 2007, this can be a handy option to limit the search results to those that correspond with the actual PowerCLI version.

Remember that there are on average 2 new PowerCLI versions per year, and that each of these brings a bunch of new Cmdlets and sometimes even complete snapins. So a valid answer from 2008 is not necessarily the best answer for the latest PowerCLI version !

Some practical examples

With the knowledge from the previous section, you should now be able to perform very precise searches and get useful results. Since I’m a strong believer in the “learn by example” principle, I added a couple of practical examples in this section that will hopefully help you getting started.

Know the defaults of the Search

Remember that the Search engine by default will use AND if you enter multiple keywords.

Use OR when you want results that contain one or the other of the keywords to be present in the results.

Instead of doing ‘vmx tools copy paste’ use ‘vmx AND tools OR copy OR paste’.

Don’t use only general keywords

Try to avoid using general search terms that will produce hundreds of results.

Don’t search for ‘PowerCLI vm report’, instead use one or more specific keywords ‘vm report numcpu’. In this example the addition of the numcpu keyword will get you more specific results.

Try to use very specific keywords

This is perhaps a stupid advice, but one that could save you a lot of time.

Suppose you are looking for a thread that discusses deploying a VM from a template and that includes information on customizing the NIC of the guest.

Your first thought could be to type ‘deploy vm template nic customise’, this returned 27 threads.

If you know that you can use the Set-OSCustomizationNicMapping cmdlet for this, add the cmdlet in the keywords. That returned only 1 thread.

Use the VMware vSphere PowerCLI Overview

In the previous example the addition of the Set-OSCustomizationNicMapping keyword to the search, gave us a better search result. But how do you know that cmdlet exists if you are new to PowerCLI or had never used the cmdlet before.

In the VMware vSphere PowerCLI Overview you can use the search functionality to help you discover the Cmdlets. In this example when you for example type the word ‘nic’, the reference will return a number Cmdlets, including the Set-OSCustomizationNicMapping cmdlet.

Note that you also search for Types, Parameters and Enumerations.

Use the More and Filter Options

Do not only use keywords. Sometimes it is useful to add 1 of the More options to fine-tune your search.

For example, searching for ‘vcheck’ (the famous reporting script) will return 34 results. By adding the name of the user who posted or replied, you can limit the number of results. The same query, but with the addition of the Restrict to a specific person option, will return only 3 results.

If you want to find results that are more likely to discuss the most recent version of the vCheck script, use the When option.

This produced 3 results that all referenced vCheck v6.

Let Bookmarks help you

I don’t know if this already happened to you, but I often find a useful or interesting thread that I can’t find back after a couple of days. To avoid this use the Bookmark feature to add such a thread to your list of threads that you want to keep for later consultation.

You can find the Bookmark option on the right side under the Actions,

click it and optionally enter some notes to go with the bookmark. Note that you make your bookmarks public or private.

You can consult your Bookmarks through the My Bookmarks entry in your user accounts drop-down menu.

Many of you have probably had a communities post answered by LucD… I know I have!

He it the most active PowerCLI member and one of the most respected vExperts and PowerShell/PowerCLI users in the community.  Luc has presented at many VMworld sessions and VMUGS alike, he has a great blog with some great posts here: Lucd.info

For those of you wanting to learn PowerCLI and not knowing where to start idera recently announced he will be performing a session on PowerCLI first steps.

This is highly recommended for both beginners and experts alike.

Details

Wednesday, March 28, 2012

12:00 – 01:00 PM Central Time

Register here: https://www.vconferenceonline.com/event/regeventweb.aspx?id=560

Alan Renouf – Sr. Technical Marketing Architect

With Partner Exchange (PEX) just around the corner from Feb 11th to Feb 16th I wanted to share the PowerCLI sessions with you, there are a couple of great sessions related to PowerCLI this year as you can see from the below information.

As well as these sessions I will also be walking the floors of the Solutions Exchange chatting to the cloud providers and anyone who has anything to do with Automation or just looking to make their lives easier.

I will also be keen to talk to anyone who is thinking of creating a cloud or has a cloud at the moment using vCD, I have some cool things to show you which will help you streamline your vCD environment – Reach out to me !

In a previous video we showed you how to get started with PowerCLI 5.0.1 and specifically the vCloud Director (vCD) cmdlets, we show you how to gain a list of cmdlets which were added to PowerCLI 5.0.1 for managing vCD and how to use some of these cmdlets to gather data from your vCD infrastructure.

I received some comments from people saying this was great information and they could use this to write some scripts which would allow them to pull data from the vCD infrastructure, but how could they interact with vCD ?  At the moment the web interface gives them access to not only view the cloud data but also interact with it, stop/start vApp’s and VM’s, create new Organization’s, ProvidervDC’s, OrgvDC’s and much much more.

You may not know but this functionality is actually available in the PowerCLI 5.0.1 cmdlets.  When listing the cmdlets you will see that most start with “Get-“ which means they are mainly designed to retrieve the data.  People who have looked a little further or perhaps have slightly more PowerCLI knowledge will have noticed the cmdlet Get-CIView.

The Get-CIView cmdlet allows us to reach into the full SDK for vCloud Director and pull back the full .Net object, this basically means that we get access to perform anything in vCloud Director that you would normally do via the web interface.

Another way to retrieve the .Net View representation  of a PowerCLI object is to refer to its .ExtensionData property.

The .ExtensionData property is an “on-demand” property which gives us the same results as calling the Get-CIView cmdlet.

To see a simple example of how we might use this .Extensiondata property to interact with vCD and Start/Stop a vApp take a look at the below video example:

From this simple example we can then create PowerShell advanced functions to make this easier for us in the future as can be seen below, this will give us 2 functions which will allow us to easily stop and start vApps in the cloud using PowerCLI.

function Stop-CIVApp {
    <#
        .SYNOPSIS
            Powers Off a vApp.

        .DESCRIPTION
            Powers Off a vApp and all VMs within it.

        .PARAMETER  CIVApp
            The Name or object or collection of objects of vApps which to shutdown.
           
        .PARAMETER    Force
            Attempt to power off the VMs. Failures in undeploying the VM or associated networks are ignored.
            All references to the vApp and its VMs are removed from the database

        .EXAMPLE
            PS C:\> Get-CIVApp | Stop-CIVApp

        .EXAMPLE
            PS C:\> Get-CIVApp “MyvApp” | Stop-CIVApp
           
        .EXAMPLE
            PS C:\> Stop-CIVApp -CIVApp “MyvApp”
           
        .EXAMPLE
            PS C:\> Stop-CIVApp -CIVApp “MyvAPP” -Force
    #>
    [CmdletBinding(
        SupportsShouldProcess=$true,
        ConfirmImpact=”High”
    )]
    param(
        [parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
        $CIVApp,
        [Switch]$Force
        )
    Process {
        if ($pscmdlet.ShouldProcess($CIVAPP)) {
            If ($Force){
                $CIVApp | Foreach {
                    $_.ExtensionData.Undeploy(“force”)
                }
            } Else {
                $CIVApp | Foreach {
                    $_.ExtensionData.Undeploy(“powerOff”)
                }
            }
        }
    }
}

function Start-CIVApp {
    <#
        .SYNOPSIS
            Powers On a vApp.

        .DESCRIPTION
            Powers On a vApp and all VMs within it.

        .PARAMETER  CIVApp
            The Name or object or collection of objects of vApps which to start.

        .EXAMPLE
            PS C:\> Get-CIVApp | Start-CIVApp

        .EXAMPLE
            PS C:\> Get-CIVApp “MyvApp” | Start-CIVApp
           
        .EXAMPLE
            PS C:\> Start-CIVApp -CIVApp
    #>
    [cmdletbinding()]
    param(
        [parameter(ValueFromPipeline=$true)]
        $CIVApp
        )
    Process {
        $CIVApp | Foreach {
            $_.ExtensionData.PowerOn()
        }
    }
}