It has been a while since vCenter 8.0u1 shipped and with it the JSON based protocol for the Virtual Infrastructure APIs aka VI/JSON. In this article we will put the VI/JSON APIs to use in a practical and complete example. We will read vCenter inventory using VI/JSON API to obtain a list of the virtual machines and hosts in a vCenter system using the PropertyCollector API and Python aiohttp library. This includes:
- Negotiate API release identifier
- Call VI/JSON APIs with Python aiohttp library
- Implement Proxies to VI/JSON APIs
- Retrieve Properties from a Set of Managed Objects with a View
You could easily do the same with PyVMOMI library except PyVMOMI still supports Python 2.7 and thus lacks asynchronous option.
“Why would you need asynchronous code?” one may ask. The simple answer is – scale. If your environment features multiple vCenter systems it is much faster to send API requests to all of them in parallel and await the results then iterating the servers one by one. Asynchronous Python allows us this flexibility without need for complex multithreaded programming. Thus VI/JSON with asynchronous Python could be great addition to an app doing complex workflows with PyVMOMI in places where it needs to optimize speed. One such scenario is to read vCenter inventory using VI/JSON API.
In this example we focus on the use of the vCenter API from Python using aiohttp library. To keep it manageable length we will not look into orchestrating multiple connections. The later may be a fun exercise for the reader to develop further the current example.
The code in the example can be ported to other programming languages such as Javascript, Rust, PHP or Dart that lack SDK support for vSphere.
Configure the environment
This article and the code with it can be downloaded as Jupyter Notebook from github.
To start with this example put the Jupiter notebook file in an empty folder where you will run the experiment. You can delete the folder at the end to clean up your system. To run the example you will need Python 3.10 or later installed on the system.
I used Visual Studio code to view and edit the Notebook. I ran the example on WSL Ubuntu 22. The same instructions should work on Linux or MacOS.
I would recommend setting up Python environment first to isolate the experiment. You can skip this step if your Jupiter Notebook application already created Python environment or other isolation for your notebook. Visual Studio Code was happy to use Python virtual environment with my notebook.
|
1 2 3 4 5 |
# Create a Python environment !python3 -m venv venv # Load the Python environment !source venv/bin/activate |
The next step is to install the necessary Python packages – “aiohttp” and “python-dotenv”. We’ll use “pip” for this.
|
1 2 3 4 5 |
# Install aiohttp %pip install aiohttp #install dotenv to read configuration file %pip install python-dotenv |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
Collecting aiohttp Obtaining dependency information for aiohttp from https://files.pythonhosted.org/packages/54/5d/4ea65eaf9a81821e2a02ba1f77644920dd0a575a2fd05557adb433db3ef6/aiohttp-3.9.1-cp311-cp311-macosx_11_0_arm64.whl.metadata Using cached aiohttp-3.9.1-cp311-cp311-macosx_11_0_arm64.whl.metadata (7.4 kB) Requirement already satisfied: attrs>=17.3.0 in ./jenv/lib/python3.11/site-packages (from aiohttp) (23.1.0) Collecting multidict<7.0,>=4.5 (from aiohttp) Using cached multidict-6.0.4-cp311-cp311-macosx_11_0_arm64.whl (29 kB) Collecting yarl<2.0,>=1.0 (from aiohttp) Obtaining dependency information for yarl<2.0,>=1.0 from https://files.pythonhosted.org/packages/20/3d/7dabf580dfc0b588e48830486b488858122b10a61f33325e0d7cf1d6180b/yarl-1.9.4-cp311-cp311-macosx_11_0_arm64.whl.metadata Using cached yarl-1.9.4-cp311-cp311-macosx_11_0_arm64.whl.metadata (31 kB) Collecting frozenlist>=1.1.1 (from aiohttp) Obtaining dependency information for frozenlist>=1.1.1 from https://files.pythonhosted.org/packages/5b/9c/f12b69997d3891ddc0d7895999a00b0c6a67f66f79498c0e30f27876435d/frozenlist-1.4.1-cp311-cp311-macosx_11_0_arm64.whl.metadata Using cached frozenlist-1.4.1-cp311-cp311-macosx_11_0_arm64.whl.metadata (12 kB) Collecting aiosignal>=1.1.2 (from aiohttp) Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Requirement already satisfied: idna>=2.0 in ./jenv/lib/python3.11/site-packages (from yarl<2.0,>=1.0->aiohttp) (3.6) Using cached aiohttp-3.9.1-cp311-cp311-macosx_11_0_arm64.whl (386 kB) Using cached frozenlist-1.4.1-cp311-cp311-macosx_11_0_arm64.whl (53 kB) Using cached yarl-1.9.4-cp311-cp311-macosx_11_0_arm64.whl (81 kB) Installing collected packages: multidict, frozenlist, yarl, aiosignal, aiohttp Successfully installed aiohttp-3.9.1 aiosignal-1.3.1 frozenlist-1.4.1 multidict-6.0.4 yarl-1.9.4 Collecting python-dotenv Using cached python_dotenv-1.0.0-py3-none-any.whl (19 kB) Installing collected packages: python-dotenv Successfully installed python-dotenv-1.0.0 |
Set up the Basics
Now we can import the Python libraries we will need:
|
1 2 3 4 5 6 7 |
import json import logging import os import sys import aiohttp from dotenv import load_dotenv |
Next we set up the connection parameters for the vCenter system. Create a .env file in the folder where you run the notebook and define the connection settings in it as follows. We will use simple user name and password authentication here. More advanced authentication options are available and we will discuss in future articles.
|
1 2 3 |
VSPHERE_USER=administrator@vsphere.local VSPHERE_PASSWORD='VMware1!' VSPHERE_SERVER=vcenter.local |
Then we load the connection parameters in Python
|
1 2 3 4 5 |
load_dotenv() server = os.environ['VSPHERE_SERVER'] user = os.environ['VSPHERE_USER'] pwd = os.environ['VSPHERE_PASSWORD'] |
Now to some less glorious details about our application like log level, an exception type we will use to convey errors, a function to convert from JSON identifier to URL path string, a utility to print shortened JSON, a “well known” reference to the root ServiceInstance object and the JSON MIME type string.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s', handlers=[logging.StreamHandler(stream=sys.stdout)]) class APIException(Exception): """Exception raised for errors in API calls.""" def __init__(self, message: str, status_code: int, response: dict) -> None: super().__init__(message) self.message = message self.status_code = status_code self.response = response def to_path(obj: dict) -> str: """Convert a vSphere ManagedObjectReference to a path string""" return f"{obj['type']}/{obj['value']}" def short_json(json_data: dict) -> str: """Shorten a JSON string to a maximum length""" str = json.dumps(json_data, indent=2) lines = str.splitlines() if len(lines) > 23: lines = lines[:20] + ["..."] + lines[-2:] str = "\n".join(lines) return str # ServiceInstance ManagedObjectReference # This is the root object of the vSphere API # It is a well-known object and is not discoverable SERVICE_INSTANCE = { "type": "ServiceInstance", "value": "ServiceInstance" } JSON_CONTENT_TYPE = "application/json" |
Negotiate API release identifier
vSphere API exposes a complex and evolving object structure. A client and server must negotiate mutually supported release identifier before APIs can be put to use.
Using a negotiated release identifier the vSphere server will coerce it’s responses to data types that the client understands. For example if the server supports new kind of ethernet adapter that the client does not know about the server will return a more generic adapter i.e. the server will upcast the network adapter to a parent class that the client understands.
Similarly clients should only rely on API features that exist in the negotiated release. Clients need to do further checks in the vSphere API as capabilities of individual objects may vary. For example depending on the virtual hardware version a virtual machine will support different use cases and configurations.
In PyVMOMI determining the release number is achieved by SmartConnect API. SmartConnect relies on internal PyVMOMI tables that combined with the vCenter server metadata allow PyVMOMI to compute a mutually supported API release identifier. To make this simpler for VI/JSON clients in vCenter 8.0.2 there is a new System::hello API that is release agnostic. Clients provide the vSphere release identifiers they have been tested with in order of preference and vCenter selects the best match.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
SUPPORTED_RELEASES = ["8.0.2.0", "8.0.1.0"] async def negotiate_release(server: str, supported_releases: list[str], session: aiohttp.ClientSession, verify: bool): """Negotiates API release identifier to use. The application author sends the list of releases they have tested with in order of priority and the server returns mutually acceptable release identifier. The API was released in 8.0.2.0 and thus if the server is 8.0.1.0 it will return HTTP status code 404. See https://developer.vmware.com/apis/vsphere-automation/latest/vcenter/api/vcenter/systemactionhello/post/ """ body = { "api_releases": supported_releases } async with session.post( f"https://{server}/api/vcenter/system?action=hello", json=body, headers={"content-type": "application/json"}, verify_ssl=verify) as r: if r.status == 404: logging.debug("System.hello is not found. Assuming 8.0.1.0") return "8.0.1.0" if r.status != 200: logging.debug("Failed to negotiate release. received status: %s", r.status) raise APIException("Failed to negotiate release", r.status, await r.json()) release = (await r.json())["api_release"] if not release: raise APIException("No mutually acceptable release found. \ Perhaps the script is very old", 0, {}) logging.debug("Negotiated release: %s", release) return release |
We can now try this to see what release can be used to talk to vCenter.
|
1 2 |
async with aiohttp.ClientSession() as session: release = await negotiate_release(server, SUPPORTED_RELEASES, session, False) |
|
1 |
2023-12-28 12:30:07,093 - DEBUG - Negotiated release: 8.0.2.0 |
Call VI/JSON APIs with Python aiohttp library
Looking a the VI/JSON Reference documentation we see a pattern. There are 2 types of HTTP requests made. GET requests retrieving properties of objects. POST requests that invoke methods with parameters. We will set up a vSphere connection class to encapsulate this concern. It will have 2 methods – fetch and invoke for the two classes of APIs and common logic to process responses as their structure is the same across both classes of APIs.
In addition we will add option to set a session key to our connection class. Session key in VI/JSON API is both moniker for the current user identity and a handle to server side state linked to the specific API client instance. A session key is obtained by successful call to one of the SessionManager::Login* APIs.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
class VSphereConnection: """vSphere VI/JSON API client connection""" def __init__(self, server: str, release: str, session: aiohttp.ClientSession, verify: bool = True) -> None: """Initialize the vSphere client Args: server: vSphere server name or IP address release: vSphere release identifier session: aiohttp session verify: Check TLS trusts or not """ self.base = f"https://{server}/sdk/vim25/{release}" self.session = session self.headers = { "Accept": "application/json"} self.verify = verify async def set_session_key(self, session_key: str) -> None: """Set session key""" self.headers["vmware-api-session-id"] = session_key async def fetch(self, obj: dict, prop: str) -> dict: """Fetch a property from a vSphere object using HTTP GET""" logging.debug("fetching %s from %s", prop, obj) async with self.session.get(f"{self.base}/{to_path(obj)}/{prop}", headers=self.headers, verify_ssl=self.verify) as r: pyaload, _ = await self._process_response(obj, prop, r) return pyaload async def invoke(self, obj: dict, method: str, body: dict) -> dict: """Invoke a method on a vSphere object using HTTP POST""" logging.debug("invoking %s on %s", method, obj) headers = { "Content-Type": "application/json" } | self.headers url = f"{self.base}/{to_path(obj)}/{method}" async with self.session.post(url, json=body, headers=headers, verify_ssl=self.verify) as r: return await self._process_response(obj, method, r) async def _process_response(self, obj: dict, operation: str, response: aiohttp.ClientResponse) \ -> (dict, dict): """Process a response from the vSphere server. Returns JSON payload and headers on success and throws APIException on error.""" content = {} if response.headers.get('content-type', '') \ .startswith(JSON_CONTENT_TYPE): content = await response.json() if response.status < 300: return content, response.headers logging.debug("Error received: %s: %s", response.status, content) raise APIException(f"Invoke {operation} operation on {obj} failed! \ {response.status}: {await response.text()}", response.status, content) |
Let’s test our code by fetching the ServiceInstance::content property.
|
1 2 3 4 5 6 7 |
async with aiohttp.ClientSession() as session: connection = VSphereConnection(server, release, session, False) # Get the service instance content content = await connection.fetch(SERVICE_INSTANCE, "content") logging.info("Service Instance Content: %s", short_json(content)) |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
2023-12-28 11:55:31,404 - DEBUG - fetching content from {'type': 'ServiceInstance', 'value': 'ServiceInstance'} 2023-12-28 11:55:31,431 - INFO - Service Instance Content: { "_typeName": "ServiceContent", "rootFolder": { "_typeName": "ManagedObjectReference", "value": "group-d1", "type": "Folder" }, "propertyCollector": { "_typeName": "ManagedObjectReference", "value": "propertyCollector", "type": "PropertyCollector" }, "viewManager": { "_typeName": "ManagedObjectReference", "value": "ViewManager", "type": "ViewManager" }, "about": { "_typeName": "AboutInfo", "name": "VMware vCenter Server", ... } } |
Implement Proxies to VI/JSON APIs
As we want to go a bit deeper than fetching miscellaneous detail about our server we would like to have objects similar to PyVMOMI library. The structure of the VI/JSON API is similar to Object Oriented programming libraries like the Java and .Net standard libraries. So objects with methods corresponding to the VI/JSON requests would come handy. We would like to recreate scenario similar to the one described in Retrieve Properties from a Set of Managed Objects with a View. We will create proxies to remote objects that expose Python friendly methods calling into the VSphereConnection we implemented above.
- First the
SessionManagerproxy will give us access to theLoginandLogoutmethods. TheLoginmethod upon success will set thevmware-api-session-idheader back in the connection for subsequent calls. ContainerViewproxy to invoke thedestroyViewAPI to release system resourcesViewManagerproxy to create a container view.PropertyCollectorto read data from the server in pages and handle errors.- Lastly the
ServiceInstancewill give us access to the other proxies and the identifier of the root folder that we need to create a Container View.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 |
class SessionManager: """vSphere SessionManager proxy""" def __init__(self, session_manager: dict, connection: VSphereConnection) -> None: self.moref = session_manager self.connection = connection async def login(self, user: str, pwd: str) -> str: """Login to vSphere server and set session key in the connection""" logging.debug("logging in") body = { "userName": user, "password": pwd } user_session, headers = await self.connection.invoke(self.moref, "Login", body) session_key = headers.get("vmware-api-session-id") if not session_key: raise APIException("Login did not return a session key", 0, {}) # Set the session key in the connection for subsequent calls await self.connection.set_session_key(session_key) logging.debug("logged in") return user_session async def logout(self) -> None: """Logout from vSphere server""" await self.connection.invoke(self.moref, "Logout", {}) await self.connection.set_session_key(None) logging.debug("logged out") class ContainerView: """vSphere ContainerView proxy""" def __init__(self, view: dict, connection: VSphereConnection) -> None: self.moref = view self.connection = connection async def destroy_view(self) -> None: """Destroy the view using DestroyView""" await self.connection.invoke(self.moref, "DestroyView", {}) logging.debug("destroyed view: %s", self.moref) class ViewManager: """vSphere ViewManager proxy""" def __init__(self, view_manager: dict, connection: VSphereConnection) -> None: self.moref = view_manager self.connection = connection async def create_container_view(self, container: dict, recursive: bool, _type: list[str]) -> ContainerView: """Create a container view using the ViewManager""" logging.debug("creating container view for %s", container) body = { "container": container, "recursive": recursive, "type": _type } view, _ = await self.connection.invoke(self.moref, "CreateContainerView", body) logging.debug("created view: %s", view) return ContainerView(view, self.connection) class PropertyCollector: """vSphere PropertyCollector proxy""" def __init__(self, property_collector: dict, connection: VSphereConnection) -> None: self.moref = property_collector self.connection = connection async def retrieve_properties_ex(self, params: dict) -> dict: """Retrieve properties using the PropertyCollector""" result, _ = await self.connection.invoke(self.moref, "RetrievePropertiesEx", params) logging.debug("retrieved properties") return result async def continue_retrieve_properties_ex(self, token: str) -> dict: """Continue retrieving properties using the PropertyCollector""" logging.debug("continuing to retrieve properties") body = { "token": token, } result, _ = await self.connection.invoke(self.moref, "ContinueRetrievePropertiesEx", body) logging.debug("continued to retrieve properties") return result async def cancel_retrieve_properties_ex(self, token: str) -> None: """Cancel retrieving properties using the PropertyCollector""" logging.debug("canceling retrieve properties on token %s", token) body = { "token": token, } await self.connection.invoke(self.moref, "CancelRetrievePropertiesEx", body) logging.debug("canceled to retrieve properties") class ServiceInstance: """vSphere ServiceInstance proxy""" def __init__(self, service_instance: dict, connection: VSphereConnection) -> None: self.moref = service_instance self.connection = connection self.content = None async def get_content(self) -> dict: """Get ServiceInstance content""" if self.content is None: self.content = await self.connection.fetch(self.moref, "content") logging.debug("retrieved service instance content") return self.content async def get_session_manager(self) -> SessionManager: """Get the identifier of the session manager from the ServiceInstance content""" return SessionManager((await self.get_content())["sessionManager"], self.connection) async def get_view_manager(self) -> ViewManager: """Get the identifier of the view manager from the ServiceInstance content""" return ViewManager((await self.get_content())["viewManager"], self.connection) async def get_property_collector(self) -> PropertyCollector: """Get the identifier of the default property collector from the ServiceInstance content""" pc_moref = (await self.get_content())["propertyCollector"] return PropertyCollector(pc_moref, self.connection) async def get_root_folder_moref(self) -> dict: """Get the identifier of the root folder from the ServiceInstance content""" return (await self.get_content())["rootFolder"] |
This was a lot. Let’s do a simple test. We will Login and Logout to see how all this works.
|
1 2 3 4 5 6 7 8 |
async with aiohttp.ClientSession() as session: connection = VSphereConnection(server, release, session, False) si = ServiceInstance(SERVICE_INSTANCE, connection) session_manager = await si.get_session_manager() user_session = await session_manager.login(user, pwd) logging.info("Session Details %s", short_json(user_session)) await session_manager.logout() |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
2023-12-28 12:30:07,146 - DEBUG - fetching content from {'type': 'ServiceInstance', 'value': 'ServiceInstance'} 2023-12-28 12:30:07,170 - DEBUG - retrieved service instance content 2023-12-28 12:30:07,170 - DEBUG - logging in 2023-12-28 12:30:07,170 - DEBUG - invoking Login on {'_typeName': 'ManagedObjectReference', 'value': 'SessionManager', 'type': 'SessionManager'} 2023-12-28 12:30:07,198 - DEBUG - logged in 2023-12-28 12:30:07,199 - INFO - Session Details { "_typeName": "UserSession", "key": "52a87a24-29a8-2aaa-5cdc-1c9b8035bd85", "userName": "VSPHERE.LOCAL\\Administrator", "fullName": "Administrator vsphere.local", "loginTime": "2023-12-28T10:30:07.269899Z", "lastActiveTime": "2023-12-28T10:30:07.269899Z", "locale": "en", "messageLocale": "en", "extensionSession": false, "ipAddress": "192.168.50.179", "userAgent": "Python/3.11 aiohttp/3.9.1", "callCount": 0 } 2023-12-28 12:30:07,199 - DEBUG - invoking Logout on {'_typeName': 'ManagedObjectReference', 'value': 'SessionManager', 'type': 'SessionManager'} 2023-12-28 12:30:07,205 - DEBUG - logged out |
Retrieve Properties from a Set of Managed Objects with a View
We can now retrieve inventory data from our vCenter server. We want to build a query to PropertyCollector that will fetch all VMs – VirtualMachine and ESXi hosts – HostSystem with their names and IP addresses for VMs. IP addresses of VMs are reported by the Guest OS Virtual Machine tools. Thus we need to navigate deeper into the virtual machine settings. We start from the summary property and then look into the guest field and last to the ipAddress within it. To ask this of the PropertyCollector we use dot delimited string summary.guest.ipAddress.
To build our code we will follow the instruction in the vSphere Web Service Programming Guide The basic steps are:
- First create a Container View for the root folder
- Use the container view with the property collector RetrievePropertiesEx method to start fetching object data
- Iterate the results using ContinueRetrievePropertiesEx
- Dismiss the results using CancelRetrievePropertiesEx in case of unexpected error.
- Dismiss the container view object using DestroyView
Build the request to RetrievePropertiesEx
I did this by first copying the example from the reference documentation. Then I copied the details from the vSphere Web Service Programming Guide. Lastly I had to look up the names of various structures I used and fill in the _typeName fields for all objects. It was not as easy as having autocompletion in PyVMOMI and also was not as bad as I feared.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
def build_retrieve_properties_ex_params(view: dict, page_size: int) -> dict: """Build the RetrievePropertiesEx parameters""" # See https://developer.vmware.com/apis/vi-json/latest/sdk/vim25/release/PropertyCollector/moId/RetrievePropertiesEx/post/ return { "options": { "_typeName": "RetrieveOptions", "maxObjects": page_size }, "specSet": [ { "_typeName": "PropertyFilterSpec", "objectSet": [ { "_typeName": "ObjectSpec", "obj": view, "selectSet": [ { "_typeName": "TraversalSpec", "name": "traverseEntities", "path": "view", "skip": False, "type": "ContainerView" } ], "skip": True } ], "propSet": [ { "_typeName": "PropertySpec", "all": False, "pathSet": [ "name", # You can traverse in depth the property # structures to select only relevant data "summary.guest.ipAddress" ], "type": "VirtualMachine" }, { "_typeName": "PropertySpec", "all": False, "pathSet": [ "name" ], "type": "HostSystem" } ], "reportMissingObjectsInResults": False } ] } |
Call the PropertyCollector
Thanks to the proxies we built above the actual code making calls to vCenter is straight forward.
It is worth highlighting the clean up code as the end. It is important to clean up and the vCenter API is stateful and the objects we interact with occupy memory and other resources on vCenter. Leaving resources behind may result in increased memory consumption and failures on vCenter side.
In our example there are two elements to cleanup. If iteration through the pages is not complete we should dismiss the server state associated with the token. When done with success or error we must release the ContainerView object that tracks the inventory for us.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
async def list_vms_and_hosts(si: ServiceInstance, page_size: int) -> list[dict]: """List all VMs and ESX hosts in the vSphere server using the PropertyCollector and ContainerView APIs. We first create a ContainerView that contains all VMs and ESX hosts. Then we use the PropertyCollector to retrieve the name and IP address of each VM and the name of each ESX host. We use the RetrievePropertiesEx API to retrieve the data in pages. We use the ContinueRetrievePropertiesEx API to retrieve the next page(s). We use the CancelRetrievePropertiesEx API to cancel the retrieval of the data if we fail to read all pages. At last we use the DestroyView API to destroy the ContainerView.""" view_mgr = await si.get_view_manager() root_folder = await si.get_root_folder_moref() pc = await si.get_property_collector() # See https://developer.vmware.com/apis/vi-json/latest/sdk/vim25/release/ViewManager/moId/CreateContainerView/post/ view = await view_mgr.create_container_view(root_folder, True, ["VirtualMachine", "HostSystem"]) token = None try: params = build_retrieve_properties_ex_params(view.moref, page_size) result = await pc.retrieve_properties_ex(params) token = result.get("token") objects = result["objects"] # Iterate result in pages with ContinueRetrievePropertiesEx while token: result = await pc.continue_retrieve_properties_ex(token) token = result.get("token") objects = objects + result["objects"] finally: # Very important to dismiss the result when we fail to read it all if token: await pc.cancel_retrieve_properties_ex(token) # Dismiss the view object we allocated earlier await view.destroy_view() return objects |
We are now ready to read vCenter Inventory using VI/JSON API. Let’s put a short script to list the VMs and ESXi hosts in our vCenter system:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
objects = None async with aiohttp.ClientSession() as session: connection = VSphereConnection(server, release, session, False) si = ServiceInstance(SERVICE_INSTANCE, connection) session_manager = await si.get_session_manager() user_session = await session_manager.login(user, pwd) try: objects = await list_vms_and_hosts(si, 5) logging.info("VMs and Hosts: %s", short_json(objects)) finally: await session_manager.logout() |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
2023-12-28 12:30:07,220 - DEBUG - fetching content from {'type': 'ServiceInstance', 'value': 'ServiceInstance'} 2023-12-28 12:30:07,241 - DEBUG - retrieved service instance content 2023-12-28 12:30:07,241 - DEBUG - logging in 2023-12-28 12:30:07,242 - DEBUG - invoking Login on {'_typeName': 'ManagedObjectReference', 'value': 'SessionManager', 'type': 'SessionManager'} 2023-12-28 12:30:07,262 - DEBUG - logged in 2023-12-28 12:30:07,263 - DEBUG - creating container view for {'_typeName': 'ManagedObjectReference', 'value': 'group-d1', 'type': 'Folder'} 2023-12-28 12:30:07,263 - DEBUG - invoking CreateContainerView on {'_typeName': 'ManagedObjectReference', 'value': 'ViewManager', 'type': 'ViewManager'} 2023-12-28 12:30:07,268 - DEBUG - created view: {'_typeName': 'ManagedObjectReference', 'value': 'session[522ef2eb-3b02-cc2c-d7ce-2b5d852a4863]5215d630-a04c-cfad-bef5-7990f6958bb5', 'type': 'ContainerView'} 2023-12-28 12:30:07,268 - DEBUG - invoking RetrievePropertiesEx on {'_typeName': 'ManagedObjectReference', 'value': 'propertyCollector', 'type': 'PropertyCollector'} 2023-12-28 12:30:07,274 - DEBUG - retrieved properties 2023-12-28 12:30:07,274 - DEBUG - continuing to retrieve properties 2023-12-28 12:30:07,274 - DEBUG - invoking ContinueRetrievePropertiesEx on {'_typeName': 'ManagedObjectReference', 'value': 'propertyCollector', 'type': 'PropertyCollector'} 2023-12-28 12:30:07,280 - DEBUG - continued to retrieve properties 2023-12-28 12:30:07,280 - DEBUG - continuing to retrieve properties 2023-12-28 12:30:07,280 - DEBUG - invoking ContinueRetrievePropertiesEx on {'_typeName': 'ManagedObjectReference', 'value': 'propertyCollector', 'type': 'PropertyCollector'} 2023-12-28 12:30:07,286 - DEBUG - continued to retrieve properties 2023-12-28 12:30:07,286 - DEBUG - continuing to retrieve properties 2023-12-28 12:30:07,286 - DEBUG - invoking ContinueRetrievePropertiesEx on {'_typeName': 'ManagedObjectReference', 'value': 'propertyCollector', 'type': 'PropertyCollector'} 2023-12-28 12:30:07,291 - DEBUG - continued to retrieve properties 2023-12-28 12:30:07,291 - DEBUG - invoking DestroyView on {'_typeName': 'ManagedObjectReference', 'value': 'session[522ef2eb-3b02-cc2c-d7ce-2b5d852a4863]5215d630-a04c-cfad-bef5-7990f6958bb5', 'type': 'ContainerView'} 2023-12-28 12:30:07,296 - DEBUG - destroyed view: {'_typeName': 'ManagedObjectReference', 'value': 'session[522ef2eb-3b02-cc2c-d7ce-2b5d852a4863]5215d630-a04c-cfad-bef5-7990f6958bb5', 'type': 'ContainerView'} 2023-12-28 12:30:07,296 - INFO - VMs and Hosts: [ { "_typeName": "ObjectContent", "obj": { "_typeName": "ManagedObjectReference", "value": "host-66", "type": "HostSystem" }, "propSet": [ { "_typeName": "DynamicProperty", "name": "name", "val": { "_typeName": "string", "_value": "192.168.50.47" } } ] }, { ... } ] 2023-12-28 12:30:07,297 - DEBUG - invoking Logout on {'_typeName': 'ManagedObjectReference', 'value': 'SessionManager', 'type': 'SessionManager'} 2023-12-28 12:30:07,312 - DEBUG - logged out |
Final touch
To make sense of the data we will output data into tables for better readability and organization. It’s the final step in the data processing workflow. We will loop over the output split it in VMs and hosts and extract properties for each data type.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
def extract_primitive_property(item, property_name): for prop in item['propSet']: if prop['name'] == property_name: return prop['val']['_value'] return 'N/A' def print_tables(data): vm_table = [] host_table = [] for item in data: if item['obj']['type'] == 'VirtualMachine': id = item['obj']['value'] name = extract_primitive_property(item, 'name') ip_address = extract_primitive_property(item, 'summary.guest.ipAddress') vm_table.append((id, name, ip_address)) elif item['obj']['type'] == 'HostSystem': id = item['obj']['value'] name = extract_primitive_property(item, 'name') host_table.append((id, name)) print("Virtual Machines:") print(f"{'ID':<15} {'Name':<25} {'IP Address':<15}") for row in vm_table: print(f"{row[0]:<15} {row[1]:<25} {row[2]:<15}") print("\nHosts:") print(f"{'ID':<15} {'Name':<25}") for row in host_table: print(f"{row[0]:<15} {row[1]:<25}") |
…and now the moment of truth.
|
1 |
print_tables(objects) |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
Virtual Machines: ID Name IP Address vm-2008 bsd2 N/A vm-77 VMware vCenter Server 192.168.50.202 vm-9009 ubuntu_test2 N/A vm-26010 vc bckp 8.9.2023 N/A vm-4010 vc backup N/A vm-33009 container N/A vm-20010 nvm N/A vm-22009 ubuntu22 N/A vm-7009 ubuntu_test N/A vm-22010 hugo N/A vm-39009 NFS server 192.168.50.87 vm-39010 gitea 192.168.50.176 vm-20009 webserver 192.168.50.156 Hosts: ID Name host-66 192.168.50.47 host-72 192.168.50.188 host-68 192.168.50.149 host-39005 192.168.50.94 |
Conclusion and next steps
In the above example we read vCenter Inventory using VI/JSON API and saw the basic mechanics of the VI/JSON API – negotiating a release identifier, using session, making basic requests to the API to fetch properties and call methods. We also built a set of proxy objects to simplify API use. Lastly we saw how to construct complex API requests by leveraging the reference documentation and API guides. The lessons from this example can be easily developed further to utilize other APIs and build powerful Asynchronous Python tools that operate scaled out vSphere environments. The example can also be ported to different language that lacks SDK support such as Javascript.
