vCenter

How to run PowerCLI scripts from vCenter Alarms.

Background
vCenter provides an alarm feature which causes vCenter to automatically look for certain events or conditions within your virtual environment, and automatically take some action when conditions are met. There are quite a lot of criteria supported, you can generate alarms based on vCenter's event stream, so for example you could create an alarm any time a virtual machine is powered off. Alarms could also be created based on vCenter statistics, for example an alarm can be generated if an ESX host's memory utilization is above 90% for too long.

There are a number of actions you can take in response to an alarm. For example you can send email or generate an SNMP trap. You can also run scripts. A popular topic on the PowerCLI forum lately is how to run PowerCLI scripts in response to vCenter actions. Unfortunately the process is pretty tricky, so I've created some sample scripts and laid out a process that will hopefully give you everything you need to get going.

Note: These instructions were prepared based on vSphere 4.0. If you're using VirtualCenter, things may vary a bit (or maybe a lot), I haven't actually tried it. If you try this with VirtualCenter, please leave a note with your experiences.

Step 1: Within vCenter, decide on a well-defined location for your scripts.
Your life will be a lot easier if you put everything in one place, so you should decide where you want all your scripts and data to go. I recommend using a directory called %ALLUSERSPROFILE%\application data\VMware\VMware VirtualCenter\scripts. You'll have to create this directory but using this location has the advantage that your scripts will exist alongside a lot of other vCenter data.

One thing to note is that vCenter doesn't actually make use of environment variables, so henceforth I'm always going to use the location of C:\Documents and Settings\All Users\\application data\VMware\VMware VirtualCenter\scripts. Your drive letter and location may vary, so make the adjustment accordingly.

Step 2: Copy all the sample scripts into this location.
Getting the scripts just right is a tedious and painful process, so I've created some starter content that should make it a lot easier. After you've decided on your script location, copy all the files from the Alarms repository below into your directory.

After you've done that, your target directory will look a little something like this:

Vcenterfolder

Step 3: Decide on an authentication strategy.

If you don't want to log in to vCenter in response to an alarm, skip this section. However there's a good chance that when an alarm is fired, you want to do something to vCenter in response to it. Of course, you'll need to authenticate to vCenter to do that. PowerCLI supports SSPI and single-sign-on, but chances are that won't help you here. The reason is that vCenter runs alarm scripts using the same account that is used for the vCenter service. By default this is the local system account, and a script running as local system won't be able to use SSPI to log in the vCenter.

There are 3 ways to deal with this:

  1. Code usernames and passwords into your scripts. This is the easiest and least secure approach.
  2. Change the vCenter service to use an actual domain login. To be honest I'm not sure if this is even possible, but if it is it will be difficult to use this approach if you're not already using it. On the other hand this approach would allow use of SSPI and would not require any ongoing maintenance.
  3. Store encrypted credentials for the local system account to use when running the script. This approach is much more secure than the first approach yet is not nearly as difficult as the third approach. It's also the approach I took, so I'll spend a bit of time discussing how to do it.

Approach 3 relies on the fact that PowerShell makes it pretty easy to deal with the Windows Data Protection API. This allows us to store a credential in a file in such a way as it can only be decrypted by someone who has logged in using the same identity that was used to export the credential. If you look at the screenshot above, there's a file called systemCredentials.enc.xml. This is my exported credential.

The trick is that the alarm script is going to be running as local system, so to make it possible to read this credential, I have to export this credential as local system. If I export it as Administrator, the alarm script won't be able to use it. This means that somehow I have to run PowerShell as local system. There are tricks for starting command prompt as local system, if you're running pre-Vista it's pretty simple to launch one using the "at" command. If you're on Windows 2008 you can use the psexec utility (also described in the same link).

Once you manage to get a local system command prompt, launch PowerShell and navigat to your script directory. Then "dot source" the credentialManagement.ps1 file and run Export-PSCredential. This brings up a prompt for credentials, so type in the user name and password you want to use against vCenter. This will cache your credential and you're ready to start running alarm scripts securely. Check your directory against my screenshot above to ensure that things appear to be in order.

. .\credentialManagement.ps1

Export-PSCredential -path systemCredentials.enc.xml

The downside to this approach is that if your vCenter password ever changes, you need to go in and re-cache the credential.

The other thing to note is that my scripts assumes you use this approach. If you don't, you'll have to edit each of them accordingly.

Step 4: Associate an alarm with a sample script.
You may think that from here it's just as simple as associating an alarm with your favorite .ps1 file. Unfortunately no, vCenter will only run batch files. So we need to create an intermediate batch file that will call our PowerShell file. This is the purpose of redirect.bat, we can use this batch file to call any PowerShell file we want. Since redirect.bat takes a script as an argument, we avoid the situation of having one batch file per PowerShell file.

Let's walk through the whole process of associating an alarm with a PowerCLI script. We'll create an alarm that fires any time a host's CPU utilizatoin is less than 75% and launches a script that will create a new VM on this not-busy-enough host. Of course this is a pretty stupid thing to do, but it's a good way to prove that everything is working as it should be.

First, let's create an alarm. If you've never created an alarm you may have a hard time figuring out how to actually do it. To do it, go to the very top of your inventory tree and right click on the root node, then select Alarm > Add Alarm

Createanalarm

There are 4 tabs to fill out with the new alarm. Until you master all of this stuff I recommend just copying what I've got here.

In the first tab, within that Alarm Type group, select Hosts under the Monitor pulldown.

Screen1

In the Triggers tab, add a trigger based on Host CPU Usage (%). Set the condition to "Is below". Set the condition length for both warning and alert to 30 seconds.

Screen2

In the Reporting tab change "Repeat triggered alarm every" to 1 minute.

Screen3

In the Actions tab, within the Frequency group change "Repeat actions every" to 1 minute. Then create an action. For the action type select "Run a command". On the right-hand side there are 4 pulldowns based on the state transitions. Set the first 3 of these to Repeat. Now comes the important part: within Configuration you need to enter the path to the redirect batch script, giving it an argument of the PowerShell script to run. For our purposes enter:

"C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\scripts\redirect.bat" "C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\scripts\sampleHostAlarm.ps1"

This extremely long line needs to go in without spaces or linefeeds. If you're using a different directory from the one I suggested, you'll also need to adjust that here.

Screen4

Now we're ready to go. If you've done everything right, all of your not-too-busy hosts will start getting new VMs in less than a minute. That will continue until you disable or delete the alarm, so don't let it run too long. You can modify or delete an alarm by clicking the Alarms tab.

Step 5: Decide if you want to play it safe.
Your scripts may never exit. There are thousands of reasons why, it may enter an infinite loop, it may sit waiting for input, who knows? When vCenter runs a script, it times the operation and kills it after 5 minutes. Unfortunately it only kills the command prompt that it launches and not any child processes the command prompt may have created, so in reality vCenter is unable to reign in any runaway scripts.

One way we can cope with that is: before running any alarm script, check for old leftover scripts and kill them. If you want to play it safe and do that for yourself, I've provided a script called alarmCleanup that will do it. To use it, all you have to do is uncomment the relevant line in redirect.bat and you'll have extra protection against runaway scripts.

Step 6: Write your own scripts, using the samples as a guide.
I've provided 3 sample scripts to get you going, but there are two things worth pointing out that are not really obvious as you start writing your own scripts.

First, when your script is run, several environment variables will be defined, as this table shows.

Environment
Variable Name

Sample
Value

VMWARE_ALARM_TRIGGERINGSUMMARY

Metric Usage = 0%                              

VMWARE_ALARM_DECLARINGSUMMARY

([Yellow metric Is below 75%; Red metric Is b…

VMWARE_ALARM_OLDSTATUS

Green                                          

VMWARE_ALARM_TARGET_NAME

192.168.1.11                                   

VMWARE_ALARM_ALARMVALUE

Current values for metric/state                

VMWARE_ALARM_TARGET_ID

host-1560                                      

VMWARE_ALARM_NAME

New Alarm                                      

VMWARE_ALARM_NEWSTATUS

Red                                             

VMWARE_ALARM_EVENTDESCRIPTION

Alarm 'New Alarm' on 192.168.1.11 changed fro…

VMWARE_ALARM_ID

alarm-46                                       

If you want to get the object that actually triggered the alarm, the best thing to use is the ID. Most PowerCLI Get cmdlets support a -ID argument that lets you load objects by ID. However, the ID that PowerCLI cmdlets use are a bit different from the IDs that will be set in the environment variable. For example, in the table above we have an id of host-1560. However, we can't call Get-VMHost -ID host-1560 and expect it to work, instead we have to prepend the term HostSystem- to it. In other words we would run:

    Get-VMHost -Id HostSystem-host-1560

to get the host that triggered the alarm. The same is true of other objects, and you can see a table of prefixes below.

When
dealing with this:

Prepend
this to the ID

Virtual Machines

VirtualMachine

Hosts

HostSystem

Clusters

ClusterComputeResource

Datacenters

Datacenter

Datastores

Datastore

Since alarms can only be associated with one type of object, you can hard-code this string inside the script that will respond to the alarm. Again there are samples to get you started in the right direction.

Troubleshooting.
Unfortunately this process is pretty tricky and you're probably not going to get it right the first time. If you need to diagnose what's going on, I recommend downloading Process Explorer.Some of the key things to note, when the alarm fires you should see a new process spawn as a child of vpxd.exe, then quickly disassociate from it (this is due to the use of "start" within redirect.bat).

Procexp

Another important thing to note, if your PowerShell process isn't behaving properly, right-click on it and take a look at its environment and command line.

Procexp2

Expect the process of getting alarm scripts to take a bit of experimentation, but once you've got the first one working the rest is very easy. Good luck!

Related Articles

Comments

92 comments have been added so far

    1. DoSMM is the world’s leading provider of the best quality virtual products and services at the lowest prices. We mainly provide SMM and SEO based products and services with high security and loyalty. SMM services include Facebook, Google services, Gmail, reviews and bank cards Keyword ranking in SEO services is the best quality of all our products. We cater to the needs of our customers because our sole goal is to achieve customer satisfaction. View all our products Choose and order first to get quality products. Thanks for choosing the right place – DOSMM.
      https://dosmm.com/

      Reply

  2. Hi Carter,
    How can I create alarms by script?
    Do you have any samples?
    Thank you very much for your help.
    Thanks,
    Olegarr

    Reply

  4. Hey Carter – great post.
    But I got a wierd issue I wonder if you can hellp with…
    Im trying to run the above with a powershell script that writes events to the application event log.
    When I run this script under the account running virtualcenter it runs fine but when its initated by an alarm it fails to write events.
    I can see that the powershell process has kicked off from events in vc and in the powershell event log but nothing else happens.
    Any ideas why this would happen?
    The ps script to write events is below:
    $evt=new-object System.Diagnostics.EventLog(“Application”)
    $evt.Source=” Health Alarm”
    $infoevent=[System.Diagnostics.EventLogEntryType]::Warning
    $evt.WriteEntry(“A host in the Virtualcenter $args is reporting as having health issues. Please investigate further on this server”,$infoevent,1000)

    Reply

  5. Just a note to say the issue above was related to the 2 versions of powershell on my 64 bit server and that the execution policy on one of them was still disabled! One to note as vCenter now works on 64bit servers.

    Reply

  7. Hi,

    I have a dumb/silly question.
    How can I get in the triggered script the type of alarm trigger that happened? (if it’s from warning to green for example)
    I want to make some “if” statements in my script and take different actions depending on the status going from green to red or red to green (or intermediary states)

    Thanks!

    Reply

  9. Shibas are very smart but are somewhat mischievous and independent and can be difficult
    to train. not only can it be dangerous for your dog (for
    example, if he. Pet nutritionists are advising owners to feed good quality natural treats grown organically.

    Reply

  11. With the advancement of technology and growing use of the internet, there has been observed a great hike in the number of viruses. These viruses directly attack your computer systems, resulting in corrupting your important data and stealing your confidential information. Viruses like Ransomware can also lock your data permanently until you pay some ransom to get the access back.

    Reply

  12. Mcafee.com/activate Online Help – Step by Step guide for McAfee Activate, Download & complete installation online. We are providing independent support service if in case you face problem to activate or Activate McAfee product. Just fill the form below and will get in touch with you as quick as possible.

    Reply

  13. Norton, one of the largest security products providers, has made it quite easy to protect your computer system from the malicious online activities, viruses, Trojan horses, scams and other threats. By installing a Norton setup to your device, you can be sure of the privacy of your important files as well as confidential information. Be it a business or consumer, Norton offers a special security software to suit the needs of everyone. You can choose anyone from the following:

    Reply

  14. Students are anxiously waiting for the TS Inter Results 2018 announcement because the students know the importance of the exams and also the importance of the score at the time of further studies

    Reply

  15. Looking for Vaastu Consultant In Hyderabad? here is the right place you. The best part of home or office or in personal life is peace.

    Reply

  19. Do you want to protect your system data from malware and viruses? Then, you need first to download the Eset antivirus.Eset.com/us/activate & eset.com/ca/activate- Get steps to download, install, and activate the Eset antivirus. To enter the product key, visit eset.com/activate

    Reply

  38. Excellent work. Yours appreciate able efforts in this blog are really helpful and satisfactory for me. You mentioned quality information. I will suggest your blog to others. This article will be helpful to them. For more quality, unique information I use the (Package point)guide its also a helpful guide regarding different bundle updates like SMS or Jazz Call Package etc. keep sharing your informative and unique knowledge with us. I will wait for your more unique articles.

    Reply

  39. Treatment through ethical Ayurveda, counselling, modern medicine, exercises, and food logs. It is one of the most professional and reliable Ayurvedic clinics in New Delhi. The clinic has been treating patients across India for more than 70 years.

    Reply

  48. I have been reading your posts for quite some time. And every time, it adds to my knowledge. But could you please elaborate on the last section in your upcoming post? Keep writing.

    Reply

  53. I like this, but I hope that the VCSA provides more functionality to make the interface more accessible for programming scripting jobs. I doubt VMware wants me to be taught Python to trigger alert actions, I don’t want to keep a separate box if I can help it, I’d like PowerCLI scripts to be re-written.
    Not-so-tangentally: Do you know any dynamic logic that would be used in the configuration of resource pool shares? It’s very prone to mistakes, and I haven’t yet read an accurate blog post about how to set them up. I’ve got the possibility to set it by script (which needs PowerCLI).

    Reply

  59. Apple TV 4K combines the best of television with your favorite Apple devices and services for a special experience in your living room. To activate apple TV visit URL Activate.apple.com. In some countries and areas, you may be able to use your Apple TV to sign in with your cable or TV provider to view TV series and movies that are part of your cable or TV subscription.

    Activate.apple.com

    Reply

  60. In some countries and areas, you may be able to use your Apple TV to sign in with your cable or TV provider to view TV series and movies that are part of your cable or TV

    Reply

  67. Instagram account is generally known by IG or insta. Instagram account is created by Kevin Systrom and Mike Krieger. This is an American company that is mostly used to Photo and video sharing social services. Now its algorithm is developed by Facebook inc. the Instagram account is released on 6 October 2010. Again, Instagram can connect to any other social networking sites and also can share their photo, videos, etc. Now, in 2021, Instagram gives new features which name is “Room”. By using these features, Every user goes live together with four people. Instagram also gives Hashtags that help users generate both photos and videos.

    Reply

  69. Great post you have shared with everyone. We can get valuable knowledge from this site. Thanks for sharing this informative stuff with us.

    Reply

  76. Accounting plays an important role in any organization, be it big or small. You can keep track of all the services related to your finances and monitor your future transactions only with accounting services. Accountancy Matters is one of the best accounting Firms Melbourne offering clients, advice and appropriate solutions related to accountancy. It aims at customer service and is an exclusive accounting firm located in the suburb of Prahran in Melbourne. Further, it is focusing on lending a helping hand to all the ambitious owners of Businesses in Melbourne.

    https://businessaccountingmelbourne21.blogspot.com/2022/02/what-are-benefits-of-having-tax.html

    Reply

  77. Accounting plays an important role in any organization, be it big or small. You can keep track of all the services related to your finances and monitor your future transactions only with accounting services. Accountancy Matters is one of the best accounting Firms Melbourne offering clients, advice and appropriate solutions related to accountancy. It aims at customer service and is an exclusive accounting firm located in the suburb of Prahran in Melbourne. Further, it is focusing on lending a helping hand to all the ambitious owners of Businesses in Melbourne.

    https://businessaccountingmelbourne21.blogspot.com/2022/02/what-are-benefits-of-having-tax.html

    Reply

  88. It is really great to see your work. I work at Louis Vuitton, where there is a huge variety of OUIS VUITTON ONTHEGO MM, louis vuitton onthego bag. You can find the latest fashion trends at Louis Vuitton Neverfull Tote Enjoy the latest trends with louis vuitton onthego bag.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *