Creating an Asset Inventory for vSphere Infrastructure

We often talk about how the two biggest ways to stay secure in IT are regular patching and good account & password hygiene. However, there is a big prerequisite to both of these, one that is almost always overlooked: an asset inventory. Without a comprehensive inventory of what devices, virtual machines, and OSes are your responsibility there is no way you can secure them. You cannot patch what you don’t know about.

Some IT methodologies, like ITIL, promote tools like a Configuration Management Database, or CMDB. A full-fledged CMDB is a nice thing to have, but unless it’s updated as part of every provisioning and decommissioning process it quickly falls out of date. A CMDB is an application unto itself, and while large organizations can dedicate resources to developing and maintaining a tool like a CMDB, as well as all the process & applications around it, most IT shops will benefit from much simpler approaches. In fact, most can get away with a simple Microsoft Excel spreadsheet. Keeping it simple means that you spend more time working on solving the problem itself, versus wrestling the tools that are supposed to help you.

Let’s explore some ways to quickly and accurately inventory a virtual infrastructure. Ideally you’d use multiple methods, because not every tool will find all the assets. For example, some operating systems do not respond to ping (ICMP echo) out of the box, so if you only use a mechanism that pings IPs then you will miss something. Similarly, an inventory approach focused on vCenter Server will miss other assets, like network switches, iDRAC and iLO controllers, and the like.

We also urge caution about keeping too much information in an inventory. An inventory is a snapshot, taken at a point in time, and it will quickly diverge from reality. An inventory is also not the authoritative source for most of the information. The infrastructure and devices themselves are the authoritative source. Keep only what you need to know, be thoughtful about things you add to it, and practice the time-honored tradition of “trust but verify” when it comes to the data.

vSphere Client

Many people don’t realize that, in the lower-right corner of the inventory displays, the HTML5-based vSphere Client offers an “Export” button which will generate a CSV with a customizable amount of information in it:

Handy, right? The HTML5 client is one of the big reasons to stay current with major releases of vSphere (along with other security features like TPM support & host attestation, Secure Boot for both ESXi and guest OSes, support for Microsoft Device Guard & Credential Guard, VM Encryption, and much more).

PowerCLI

PowerCLI is a perennial favorite of vSphere admins, and it’s no question why: it allows someone to do repetitive tasks in a controlled & exact manner. In fact, a number of the tools we’ll mention later are PowerCLI scripts themselves. An inventory is a great place to start learning PowerCLI if you haven’t had time previously. Get it from code.vmware.com or simply by running “Install-Module VMware.PowerCLI” in PowerShell.

Once you’ve used “Connect-VIserver your.vcenter.server.ip.or.dns” to connect and authenticate to vCenter Server you can use the Get-VM and Get-VMGuest cmdlets to easily grab data about the VMs. In these animations I’m using the Windows PowerShell ISE, which is very helpful because it’ll suggest cmdlets and let you use the Tab key to cycle through the parameters available to you. I’m using the pipe operator (the vertical bar) to feed the output of one cmdlet into the next:

You can do the same for the hosts themselves using the Get-VMHost cmdlet:

If you’d like the data saved to a file you can pipe the output into the Out-File cmdlet, too. Give it a path like “Out-File C:\Users\username\Desktop\inventory.txt.” There are also cmdlets like Export-Csv which can be helpful for formatting data into something Excel can read. Searching the Internet for examples of these cmdlets will get you thousands of results.

nmap

Being a UNIX sysadmin I’ve thought of nmap as a close and trusted friend for a long time, and indeed it recently turned 20 years old! Nmap is a port scanner and is sometimes thought of as a “hacking tool” but it’s simply a versatile tool to help figure out what’s on the network. Most Linux distributions ship with nmap, and you can download a version for Windows from nmap.org, too. It’s very powerful and can do quite a number of scans, some more dangerous than others (see all your options with “nmap -h”). A simple “ping” scan of the network can be done with:

Add the “-v” parameter to increase verbosity. If you use a more invasive scanning setting you can also see port information, and with the “-O” setting nmap will take a guess at what OS the device is. There are a lot of examples on how to use nmap at nmap.org, too. If you are experimenting with the different scanning options I urge you to be gentle (TCP connect scans are usually a good starting place) and to scan test equipment & OSes first, working up to full environment scans.

Scanning like this generates log entries on hosts, can trigger intrusion detection and other security systems, can generate a lot of network traffic, and in extreme cases can cause denial of service situations. Don’t scan things that belong to others, for example. That said, if you are up-front with your organization about the need for accurate system inventories in order to ensure proper patching and risk management, and make sure your team and your management are on board, you’ll remove most political obstacles. Don’t forget to remind folks that devices on the network, like iDRACs, iLOs, and network switches themselves don’t appear in other inventories, so it’s important to find and secure them. After all, it just takes one rogue or unpatched device (like a fish tank IoT device) to open a door for bad actors.

RVTools

RVTools is a Windows .NET application developed by Rob de Veij that reads data from your vSphere environment and can export it in native Excel and CSV formats. It has been around a long time, is extremely comprehensive, and is straightforward to use to get information about VMs, guest OSes, and ESXi hosts:

vDocumentation

vDocumentation is a set of PowerShell cmdlets by Ariel Sanchez Mora that lets you query lots of information from your environment. It’s installable via the PowerShell gallery and needs PowerCLI to operate. It can dump information about patches, network settings, and the like so that you can see and verify the state of your hosts. It’s focused more on the infrastructure itself, rather than the guest OSes, but that makes it another useful tool when we’re trying to figure out if our inventory is missing anything.

vCheck

Last on my list is another classic tool that isn’t as much about documentation as it is about reporting and checking an environment. vCheck was originally developed by Alan Renouf and has been contributed to by many others. It’s very helpful for finding oddities in your environments, configuration mismatches, and so on. It can be scheduled and can send an email report, else it will generate an HTML version and display it locally for you:

As with vDocumentation, PowerCLI, RVTools, and the like, vCheck will not be able to see and check other parts of your infrastructure, so you’ll need to follow up separately with things you find in your nmap scans (iDRACs/iLOs, network switches, etc.). That said, a lot of what vCheck looks at is applicable to other devices, too. Is your iDRAC set to use NTP servers? Is that top of rack switch sending its logs to your syslog server? Once your inventory is complete these are things you can start auditing.

Have other thoughts or tool suggestions? Leave them in the comments. Remember that knowledge is power when it comes to patching and being secure. Good luck and happy inventory-ing!