vCenter 7.0 update 2 introduced Native Key Provider functionality. It is now very easy to set up encryption using the native key provider. A UI user simply inputs a key provider name and backups the secret data to activate the native key provider. It is similarly simple to automate vSphere native key providers through API. The big difference from past APIs is that the native key provider is part of the Automation SDK and not as one would traditionally suspect under CryptoManager from the vSphere Management SDK.
You can learn more about vSphere Native Key Providers from this video.
This blog post demonstrates simple operations with the Automation and Management APIs using Python to get you started with Native Key Providers. The vSphere Management API is modernized to support JSON there by allowing wider set of clients to access it. We will refer to the JSON documentation for both APIs in this post to allow one to replicate the workflows using only JSON and HTTP.
You will need Mac or Linux like environment with Python 3.9+ installed to use the sample code in this post.
First steps with vSphere Automation SDK Python
In this chapter we will set up vSphere Python development environment.
The vSphere Automation SDK for Python is on GitHub and VMware’s website. We will use GitHub in this post as it is easier. Some organizations prefer the added security of direct download from vmware.com to avoid supply chain attacks.
Firstly, we create a folder and set up our project.
|
1 2 |
mkdir nkp_demo cd nkp_demo |
Secondly, we configure a Python environment:
|
1 2 |
python3 -m venv venv source venv/bin/activate |
Thirdly, we download the necessary dependencies
|
1 2 3 |
pip install --upgrade pip pip install --upgrade setuptools pip install --upgrade git+https://github.com/vmware/vsphere-automation-sdk-python.git |
Lastly, to keep our script free from secrets and sensitive data we will set several environment variables. We assume vCenter accepts username and password. vCenters set up with external Identity Provider may require OAuth 2 workflow instead. Replace vCenter details from the below commands and issue them in your shell:
|
1 2 3 |
export VSPHERE_USER='vCenter_username' export VSPHERE_PASSWORD='vCenter_password' export VSPHERE_SERVER='vCenter_hostname' |
This completes the setup of the vSphere Automation SDK for Python.
Connect to vCenter and list existing native key providers
In a code editor of your choice create a new file – “main.py” inside the “nkp_demo” folder. On Mac one can use Visual Studio Code from Terminal as follows
|
1 |
/Applications/Visual\ Studio\ Code.app/Contents/MacOS/Electron main.py |
Firstly, import some packages that we will need in our demo:
|
1 2 3 4 5 6 7 8 |
import requests import urllib3 import os, sys, time from vmware.vapi.vsphere.client import create_vsphere_client, VsphereClient from com.vmware.vcenter.crypto_manager import kms_client from com.vmware.vapi.std.errors_client import AlreadyExists from pyVim.connect import SmartConnect from pyVmomi import vim |
Secondly, let’s initialize settings for our demo
|
1 2 |
provider_name="native_kms" password="$up3r$3cr3t!" |
Thirdly, we connect both the Automation SDK and pyVmomi to vCenter:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
def connect(host: str, user: str, pwd: str, insecure: bool) -> tuple[VsphereClient, vim.ServiceInstance]: session = requests.session() session.verify = not insecure urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) si = SmartConnect(host=host, user=user, pwd=pwd, disableSslCertValidation=insecure) return create_vsphere_client(host, user, pwd, session=session), si def get_kms_providers(client: VsphereClient) -> kms_client.Providers: return vsphere_client.vcenter.crypto_manager.kms.Providers vsphere_client, si = connect(os.environ["VSPHERE_SERVER"], os.environ["VSPHERE_USER"], os.environ["VSPHERE_PASSWORD"], True) kmsProviders = get_kms_providers(vsphere_client) cm = si.content.cryptoManager if not isinstance(cm, vim.encryption.CryptoManagerKmip): raise TypeError("Expected CryptoManagerKmip") |
The connect function initializes both libraries with the vCenter server address and credentials. We get back VsphereClient and ServiceInstance objects. The get_kms_providers method helps with auto-completion in editor like Visual Studio Code.
We will discuss the cryptoManager and CryptoManagerKmip logic later when we set default key provider.
Fourthly, we fetch and print the Native Key Provider data
|
1 2 3 4 5 6 7 8 9 10 11 |
def print_kms_configurations(kmsProviders: kms_client.Providers): providers = kmsProviders.list() if not providers: print("No Native Key Providers") return for provider in providers: print(f"NKP summary: {provider}") print(f"NKP details: {kmsProviders.get(provider.provider)}") print() print_kms_configurations(kmsProviders=kmsProviders) |
The kmsProvders.list() API gives a list with summary details for the Native Key providers that includes the name and overall status of the key provider. To see more wee need to call the kmsProviders.get() API with the provider name/identifier.
Lastly we save and run the program:
|
1 |
python3 main.py |
If you have no Native Key Providers set and all is well then you will see something like:
|
1 |
No Native Key Providers |
Let’s Add a Native Key Provider
Add to the end of you file the following
|
1 2 3 4 5 6 7 8 9 |
print("Create native KMS.") try: spec = kmsProviders.CreateSpec(provider_name, constraints=kmsProviders.ConstraintsSpec(tpm_required=False)) kmsProviders.create(spec) except AlreadyExists as ex: print(f"Nice KMS is already set up: {ex}") print_kms_configurations(kmsProviders=kmsProviders) |
The kmsProviders.create() API creates a new Native Key Provider. Similar to UI it needs a name and flag weather to work on TPM enabled hosts only.
If you run the program again we should now see a bit more output.
|
1 2 3 4 |
No Native Key Providers Create native KMS. NKP summary: {provider : native_kms, type : NATIVE, health : ERROR} NKP details: {health : ERROR, details : [LocalizableMessage(id='com.vmware.vim.vpxd.encryption.keyProviderNotBackedUp', default_message='Key provider native_kms requires backup.', args=[], params={'keyProviderId': LocalizationParam(s='native_kms', dt=None, i=None, d=None, l=None, format=None, precision=None)}, localized='Key provider native_kms requires backup.')], constraints : {tpm_required : False}, type : NATIVE, native_info : {key_id : 7d4322cb-69f3-451f-a179-b3cd6fb5a80e}} |
Back up the Native Key Providers
To activate the Native Key Provider it’s secure keys need to be backed up first. This is a bit tricky through the API as it requires 2 distinct steps.
Firstly, we need to initiate the export by calling kmsProviders.export() API. It requires the name identifier of the key provider and a password to encrypt the export data. The output of the API when successful contains a URL and a bearer authentication token to download the exported data.
|
1 2 3 4 5 |
print('Backup KMS') res = kmsProviders.export(kmsProviders.ExportSpec(provider=provider_name, password=password)) print(f"Backup request posted. Here are the download details: {res}") url = res.location.url token = res.location.download_token |
Secondly we need to make HTTP POST request to the URL with authentication token to fetch the exported PKCS12 data.
|
1 2 3 4 5 6 7 8 9 10 11 |
response = requests.post( url, headers={'Authorization': 'Bearer %s' % token.token}, verify=False) if not response.status_code == 200: print(f"Backup failed {response}") sys.exit(1) p12data = response.content print(f'Backup completed ok') print_kms_configurations(kmsProviders=kmsProviders) |
When all goes to plan we will see the following output at end. This indicates the Native Key Provider is now active and ready to use.
|
1 2 3 4 5 |
Backup KMS Backup request posted. Here are the download details: {type : LOCATION, location : {url : https://vc8.home/cryptomanager/kms/native_kms, download_token : {token : eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2OTA3MTEyMTMsInBhc3N3ZCI6IipQRTQxSStOQUF4Unh4bVZMMHJiMEplMkkvVEd3UUk1SFhKN3oyTm9EODhzPSIsInVybCI6Imh0dHBzOi8vdmM4LmhvbWUvY3J5cHRvbWFuYWdlci9rbXMvbmF0aXZlX2ttcyJ9.Is-66gQ-O0Iamn3o25Z_P3mbUyNEQL_jC6nPwJjap-A, expiry : 2023-07-30 10:00:13}}} Backup completed ok NKP summary: {provider : native_kms, type : NATIVE, health : OK} NKP details: {health : OK, details : [], constraints : {tpm_required : False}, type : NATIVE, native_info : {key_id : 18890e81-0d0e-415c-aec6-c4241bc15820}} |
… what do I do if my Native Key Provider is corrupted?
We backed up and activated the Native Key Provider. Let’s take a second to see how we can restore the provider from back up.
Firstly, we accidentally delete the key provider:
|
1 2 3 |
print("Delete Native KMS") kmsProviders.delete(provider=provider_name) print_kms_configurations(kmsProviders=kmsProviders) |
Secondly, we restore the provider from the backup we took before:
|
1 2 3 4 5 6 7 |
ir = kmsProviders.import_provider(kmsProviders.ImportSpec(config=p12data, password=password, constraints=kmsProviders.ConstraintsSpec(tpm_required=False))) print(f'Restored Key Provider: {ir}') # vCenter needs respite to set all hosts. Immediate read shows warnings. time.sleep(1) print_kms_configurations(kmsProviders=kmsProviders) |
How to make a Native Key Provider default?
To use the Native Key Provider we may want to set it as default for a cluster or the whole vCenter system. To get or set the default Key Provider we use GetDefaultKmsCluster and SetDefaultKmsCluster APIs. These are Management APIs available in pyVmomi library. We initialized earlier connection to vCenter using pyVmomi. We obtained a reference to the CryptoManager from the ServiceInstance content object and checked that it supports the extended CryptoManagerKmip API.
Paste the following to the end of the script to see how setting default Key Provider works.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
# Set default Key Native Provider via pyVMOMI CryptoManagerKmip defaultProvider = cm.GetDefaultKmsCluster() print(f"Default provider {defaultProvider}") providerId = vim.encryption.KeyProviderId() providerId.id = provider_name cm.SetDefaultKmsCluster(clusterId=providerId) dp = cm.GetDefaultKmsCluster() print(f"Updated default provider {dp}") cm.SetDefaultKmsCluster(clusterId=defaultProvider) dp = cm.GetDefaultKmsCluster() print(f"Restored default provider {dp}") print("Delete Native KMS") kmsProviders.delete(provider=provider_name) print("Done.") |
At the end we clean up by reverting the default key provider and delete the demo Native Key Provider.
Conclusion
We saw it is easy to automate vSphere Native Key Providers. As with other vSphere functionalities to effectively automate Native Key Providers both the vSphere Automation and vSphere Management APIs need to be used in tandem.
You can find the full source code to this article in this GitHub gist.
Follow us on Twitter @VMware_API_team and send us any feedback.
