Creating an SDDC in Python on VMC: The Hard Way

This blog shows how to create an SDDC in Python on VMC. As part of my move to the VMC product marketing team (working on developer stuff), I wanted to understand what the life of a developer is like, after 3+ years on the Cloud Economics team. So, I decided to start by coding up a simple example using the core VMC APIs. The example I picked was creating a simple 1 node SDDC. I mean, how hard could it be, right?

There are a decent number of blogs and code to use the “Automation” APIs, which is where I started my journey. These APIs allow you to anything you can do in the console, more or less, in code. Kyle Ruddy has a great example on how do to this in a PowerCLI context. He also has a foundational blog on how the VMC API works. I decided to do mine in code, specifically in Python.

Creating an SDDC in Python(ish), By DALL-E

VMC Automation APIs

VMware has pages and pages of APIs, as can be seen in the Developer Center. There are the APIs for vSphere, vRealize (now Aria), HCX and much more. In order to work with VMware Cloud, you need to be familiar with several APIs.

The first is the foundational Cloud Services Platform API, which is our cross-cutting set of APIs for things like authentication, authorization, identity, service discovery, and so on. We will need this to be able to get credentials in order to do any work on VMware Cloud.

The second set of APIs are the VMware Cloud on AWS API. This allows for all manner of manipulation of VMware Cloud on AWS, provided you have the right permissions. This includes the ability to create and delete SDDCs.

The final set of APIs, not covered here, are APIs you use us to talk to vSphere and vCenter. These remain the same, since VMware Cloud on AWS is vSphere running on AWS hardware, more or less. It should be noted that there is also a vSphere SDK available for Python, which you can find at GitHub. For the purposes of this article, I am going to focus on REST calls only, essentially doing this the hard way, as I feel it’s more instructive. (Shameless plug: I am also helping out with the Python project PyVMC, which is the Python Client for VMware Cloud on AWS. Cool code, try it.)

Creating an SDDC in Python: One Node

In this example, we are going to create and deploy a 1 Node SDDC in US-WEST-2 (which is the closest region to where I am writing this). The documentation on how to create an SDDC is sufficient to give you an idea of what to do, but it leaves a few things out. If you like to dive in like me, you’ll see there are more details that appear at first blush.

To get started, we’ll need to get an API token. To do this, login to the VMware Cloud Console. Once you are logged in, launch the VMware Cloud Service. From there, click on your name in the upper right corner, making sure you have the correct Organization selected, if you have multiple of them. From there, click the “API Tokens” tab, then click the “Generate Token” link. This will generate a token for you in a pop-up. Make sure to copy it and store it someplace safe., as we’ll need it in the next step. Tokens do expire, so you’ll have to be mindful of this.

Now we get to the code, which can be found in its entirety on GitHub.

Editor’s note: yes, some of the characters are mangled. We’re looking for solutions…


#
# return key from local file key.txt so it does not show up in code
#
def get_api_key():
    """Pull key from file so I dont have to include it in the code"""
    keytext = ''
    try:
        with open('key.txt', mode='r') as file_object:
            keytext = file_object.readline()
    except FileNotFoundError:
        msg = "Sorry, the file key.txt does not exist. You must have a key.txt file with your API code in it in order to run this sample"
        print(msg) # Sorry, the file key.txt does not exist....
        exit()

    return keytext

#
# from https://developer.vmware.com/apis/vmc/latest/
#
def vmc_login():
    """Login to VMC on AWS with previously acquired API Key"""
    #
    # Pull the Auth API Key from a file (so I don't have to store it in the code)
    #
    key = get_api_key()
    url = 'https://console.cloud.vmware.com/csp/gateway/am/api/auth/api-tokens/authorize'
    headers = {'Content-Type': 'application/json'}
    payload = {'refresh_token': key}
    r = requests.post(f'{url}', headers=headers, params=payload)
    if r.status_code != 200:
        print(f'Unsuccessful Login Attempt. Error code {r.status_code}')
        exit()
    else:
       auth_json = r.json()&#91;'access_token']
       auth_Header = {'Content-Type': 'application/json',
                       'csp-auth-token': auth_json}
        return auth_Header

# return key from local file key.txt so it does not show up in code

def get_api_key():

"""Pull key from file so I dont have to include it in the code"""

keytext = ''

try:

with open('key.txt', mode='r') as file_object:

keytext = file_object.readline()

except FileNotFoundError:

msg = "Sorry, the file key.txt does not exist. You must have a key.txt file with your API code in it in order to run this sample"

print(msg) # Sorry, the file key.txt does not exist....

exit()

return keytext

# from https://developer.vmware.com/apis/vmc/latest/

def vmc_login():

"""Login to VMC on AWS with previously acquired API Key"""

# Pull the Auth API Key from a file (so I don't have to store it in the code)

key = get_api_key()

url = 'https://console.cloud.vmware.com/csp/gateway/am/api/auth/api-tokens/authorize'

headers = {'Content-Type': 'application/json'}

payload = {'refresh_token': key}

r = requests.post(f'{url}', headers=headers, params=payload)

if r.status_code != 200:

print(f'Unsuccessful Login Attempt. Error code {r.status_code}')

exit()

else:

auth_json = r.json()['access_token']

auth_Header = {'Content-Type': 'application/json',

'csp-auth-token': auth_json}

return auth_Header

In this example, we attempt to login to VMC. We do this by pulling the API token from a file (“key.txt”) via get_api_key() and then we call the REST authentication method as described in the documentation. Don’t forget to store your key in key.txt, or you’ll get an error.

Next, we do the setup process before we call the API to create an SDDC. Here’s what it looks like:

    orgID = get_organization(auth_header,"NAME_OF_YOUR_ORG")
    connectedAccount = get_connected_account(orgID, auth_header)
    region = "US_WEST_2"
    compat_subnet = get_compatible_subnet_id(orgID, auth_header,connectedAccount, region)

    createTask = create_sddc(auth_header, orgID, "RothTest2", "AWS", region, 1, connectedAccount,compat_subnet, False)

orgID = get_organization(auth_header,"NAME_OF_YOUR_ORG")

connectedAccount = get_connected_account(orgID, auth_header)

region = "US_WEST_2"

compat_subnet = get_compatible_subnet_id(orgID, auth_header,connectedAccount, region)

createTask = create_sddc(auth_header, orgID, "RothTest2", "AWS", region, 1, connectedAccount,compat_subnet, False)

In essence, we have to do the following:

Get the object-id of the organization we’re part of. (Note: I’m scanning by the name of the org I am looking for. Not terribly sophisticated, but neither is it obscurantist).
Get the connected-account, essentially the object-id of the connected AWS account. You need this for billing purposes.
Determine a reasonable subnet for our new SDDC to connect on.

Here’s the code for each of these three steps, in order. First, let’s show how we got the organization.

#
# Docs: https://developer.vmware.com/apis/vmc/latest/vmc/api/orgs/get/
#
def get_organization(auth_header, org_name):
    """Pull a list of organizations, and return one to work on for new SDDC"""

    orgList = requests.get(‘https://vmc.vmware.com/vmc/api/orgs', headers=auth_header)
#
# Scan the list of Orgs and pick the Org we want to create an SDDC in from the list. 
#
    x=0

    for org1 in orgList.json():
        print('Num ', x,  org1&#91;'id'],'Display Name ', org1&#91;'display_name'], "user", org1&#91;'user_name'])
             
        if org1&#91;'display_name'] == org_name:
            break
        x += 1
    
    if x == len(orgList.json()):
        print("Org Not Found")
        exit(1)
#
# Return Info about Org
#
    orgID = orgList.json()&#91;x]&#91;'id']
    return orgID

# Docs: https://developer.vmware.com/apis/vmc/latest/vmc/api/orgs/get/

def get_organization(auth_header, org_name):

"""Pull a list of organizations, and return one to work on for new SDDC"""

orgList = requests.get(‘https://vmc.vmware.com/vmc/api/orgs', headers=auth_header)

# Scan the list of Orgs and pick the Org we want to create an SDDC in from the list.

x=0

for org1 in orgList.json():

print('Num ', x, org1['id'],'Display Name ', org1['display_name'], "user", org1['user_name'])

if org1['display_name'] == org_name:

break

x += 1

if x == len(orgList.json()):

print("Org Not Found")

exit(1)

# Return Info about Org

orgID = orgList.json()[x]['id']

return orgID

Next, we take the Organization ID and pull the related object with represents the connected account. For reference, we also print out the accounts with their associated ID and Owner, using PrettyTable, a table display library in Python.

#
# We get he connected account so we can get the VPC and the subnets, which are owned by the
# underlying AWS resources.
#
def get_connected_account(orgId, header):
    resp = requests.get(
        f'https://vmc.vmware.com/vmc/api/orgs/{orgId}/account-link/connected-accounts', headers=header)
    if resp.status_code != 200:
        print(f'error in connected account {resp.status_code} for org {orgId}')
        exit()
    results = json.loads(resp.text)
    print("\nConnected Accounts")
    accounts_table = PrettyTable(&#91;"ID", "Account Number", "User Name"])
    for cId in results:
        accounts_table.add_row(&#91;cId&#91;'id'],cId&#91;'account_number'],cId&#91;'user_name']])        
    print(accounts_table)    # print out the accounts table
    return results&#91;0]&#91;'id']  # pull the first account to use

# We get he connected account so we can get the VPC and the subnets, which are owned by the

# underlying AWS resources.

def get_connected_account(orgId, header):

resp = requests.get(

f'https://vmc.vmware.com/vmc/api/orgs/{orgId}/account-link/connected-accounts', headers=header)

if resp.status_code != 200:

print(f'error in connected account {resp.status_code} for org {orgId}')

exit()

results = json.loads(resp.text)

print("\nConnected Accounts")

accounts_table = PrettyTable(["ID", "Account Number", "User Name"])

for cId in results:

accounts_table.add_row([cId['id'],cId['account_number'],cId['user_name']])

print(accounts_table) # print out the accounts table

return results[0]['id'] # pull the first account to use

Here’s where things get a little convoluted when creating an SDDC in Python, since you need to pass some account and network info.

Here’s what we’ll need to do: we’ll scan the VPCs connected to our linkedAccount and pull a subnet for our SDDC to use by using the appropriate REST call from the API, whose documentation is here. Now I found this part of the architecture a little opaque, so I included pulling the VMC Map and iterating across it so users could see the full list, printed with PrettyTable. We need to do this in order to tell VMware Cloud on AWS how our newly created SDDC should get to the internet. New SDDCs do not, by default, come with connectivity to the internet. You have to configure it.

#
# Get subnet ID
#  
def get_compatible_subnet_id(orgId, header, linkedAccount, region):
    """get the first applicable compatible subnet"""

    urlParams = {'linkedAccountId' : linkedAccount,
                'region' : region}
    resp = requests.get(f'https://vmc.vmware.com/vmc/api/orgs/{orgId}/account-link/compatible-subnets',
        params=urlParams,
        headers=header)
    if resp.status_code != 200:
        print(f'\nError when getting compatible subnets {resp.status_code} : {resp.reason} with acct {linkedAccount} for org {orgId} in region {region}.')
        return None

    results = json.loads(resp.text)
    subnets = results&#91;'vpc_map']
    #
    # In my case, pull the name of the default VPC, since I know the subnets work for this
    #
    subnet_table = PrettyTable(&#91;"ID", "Description","CIDR Block","Subnet 0","Num Subnets","0 Compatible"])
    print("\nSubnet Table")
    for key in subnets:
        subnet_table.add_row(&#91;key,subnets&#91;key]&#91;'description'],subnets&#91;key]&#91;'cidr_block'],subnets&#91;key]&#91;'subnets']&#91;0]&#91;'subnet_id'],len(subnets&#91;key]&#91;'subnets']),
            subnets&#91;key]&#91;'subnets']&#91;0]&#91;'compatible']])

    print(subnet_table)
    #
    # Pull the first subnet
    #
    firstkey = list(subnets.keys())&#91;0]
    fullsubnet = subnets&#91;firstkey]&#91;'subnets']
    subnetID = fullsubnet&#91;0]&#91;'subnet_id']
    return subnetID

# Get subnet ID

def get_compatible_subnet_id(orgId, header, linkedAccount, region):

"""get the first applicable compatible subnet"""

urlParams = {'linkedAccountId' : linkedAccount,

'region' : region}

resp = requests.get(f'https://vmc.vmware.com/vmc/api/orgs/{orgId}/account-link/compatible-subnets',

params=urlParams,

headers=header)

if resp.status_code != 200:

print(f'\nError when getting compatible subnets {resp.status_code} : {resp.reason} with acct {linkedAccount} for org {orgId} in region {region}.')

return None

results = json.loads(resp.text)

subnets = results['vpc_map']

# In my case, pull the name of the default VPC, since I know the subnets work for this

subnet_table = PrettyTable(["ID", "Description","CIDR Block","Subnet 0","Num Subnets","0 Compatible"])

print("\nSubnet Table")

for key in subnets:

subnet_table.add_row([key,subnets[key]['description'],subnets[key]['cidr_block'],subnets[key]['subnets'][0]['subnet_id'],len(subnets[key]['subnets']),

subnets[key]['subnets'][0]['compatible']])

print(subnet_table)

# Pull the first subnet

firstkey = list(subnets.keys())[0]

fullsubnet = subnets[firstkey]['subnets']

subnetID = fullsubnet[0]['subnet_id']

return subnetID

Now that we have the organization’s ID, the connected account, and the subnet we want to use, as well as the region we want to create it in(US_WEST_2, see above), we can call create_sddc to call the REST API to create our SDDC.


#  Create the SDDC
# 
def create_sddc(header, org, name, provider, region, numHost, connectedAccount, subnetId, ValidateOnly):

    if subnetId == None:
        print("Error: Can Not Get Valid SubnetID")
        exit()

    data = {
        'name': name,
        'account_link_sddc_config': &#91;
            {
                'customer_subnet_ids': &#91;
                    subnetId
                ],
                'connected_account_id': connectedAccount
            }
        ],
        'provider': provider.upper(),   # make sure provider is in upper case
        'num_hosts': numHost,           # 1 host in this case
        'sddc_type': '1NODE',
        'region': region                # region where we have permissions to deploy.
    }

    resp = requests.post(f'https://vmc.vmware.com/vmc/api/orgs/{org}/sddcs', json=data, headers=header)
    json_response = resp.json()
  
    if resp.status_code in (200,202):
        print('Status Code= ' + str(resp.status_code))
        newTask = json_response&#91;'id']
        return newTask
    else:
        if "error_messages" in json_response.keys():
            print(f'Error on create {json_response&#91;"error_messages"]}')
        else:
            print("Error on create")
        exit()

# Create the SDDC

def create_sddc(header, org, name, provider, region, numHost, connectedAccount, subnetId, ValidateOnly):

if subnetId == None:

print("Error: Can Not Get Valid SubnetID")

exit()

data = {

'name': name,

'account_link_sddc_config': [

{

'customer_subnet_ids': [

subnetId

'connected_account_id': connectedAccount

}

'provider': provider.upper(), # make sure provider is in upper case

'num_hosts': numHost, # 1 host in this case

'sddc_type': '1NODE',

'region': region # region where we have permissions to deploy.

}

resp = requests.post(f'https://vmc.vmware.com/vmc/api/orgs/{org}/sddcs', json=data, headers=header)

json_response = resp.json()

if resp.status_code in (200,202):

print('Status Code= ' + str(resp.status_code))

newTask = json_response['id']

return newTask

else:

if "error_messages" in json_response.keys():

print(f'Error on create {json_response["error_messages"]}')

else:

print("Error on create")

exit()

It can take a little while for an SDDC to get created. For my account, it takes around 2 hours. As a result, if the “create” call is successful, it will return a Task object, and an HTTP status code 202. You can then poll the Task object and see the steps of the creation of the SDDC, using the code below.

def poll_sddc_until_created(orgID, newtask, header):
    notDone = True
    while notDone:
        resp = requests.get(f'https://vmc.vmware.com/vmc/api/orgs/{orgID}/tasks/{newtask}', headers=header)
        
        parsed = resp.json()

        if resp.status_code != 200:
            if "error_messages" in parsed.keys():
                print(f'Error on create {parsed&#91;"error_messages"]}')
            else:
                print("Error on create")
            return False

        print("Status Code = " + str(resp.status_code))
        print("Resp Status " + parsed&#91;'status']) # check for numeric
        print("Resp SubStatus " + parsed&#91;'sub_status'])

        if parsed&#91;'status'] == "FAILED":
            notDone = False
            print("error Message: " + parsed&#91;'error_message'])
            if 'params' in parsed:
                print("SDDC Params" + str(parsed&#91;'params']))
            return False
        time.sleep(15)  # 15 seconds, so we don’t hammer the server
    return True

def poll_sddc_until_created(orgID, newtask, header):

notDone = True

while notDone:

resp = requests.get(f'https://vmc.vmware.com/vmc/api/orgs/{orgID}/tasks/{newtask}', headers=header)

parsed = resp.json()

if resp.status_code != 200:

if "error_messages" in parsed.keys():

print(f'Error on create {parsed["error_messages"]}')

else:

print("Error on create")

return False

print("Status Code = " + str(resp.status_code))

print("Resp Status " + parsed['status']) # check for numeric

print("Resp SubStatus " + parsed['sub_status'])

if parsed['status'] == "FAILED":

notDone = False

print("error Message: " + parsed['error_message'])

if 'params' in parsed:

print("SDDC Params" + str(parsed['params']))

return False

time.sleep(15) # 15 seconds, so we don’t hammer the server

return True

This code will poll the task every 15 second, and print out the status and the substatus, which will show you the steps the automation goes through to create a new SDDC. I found the messages fairly revealing. Meanwhile, you can also see the status of your SDDC creation in the console at http://console.cloud.vmware.com/. Once the SDDC is created, you can begin setting it up, adding VMs, adding firewall rules, etc.

That’s how to create an SDDC in python on VMC. Would you like more sample code? Send your requests to @BillRothVMware on Twitter or message me on the {code} slack server.

Released Resources

VMware Cloud on AWS APIs
Developer Center for VMware Cloud on AWS (you’ll need console access)
Code from this blog on GitHub

VMC Automation APIs

Creating an SDDC in Python: One Node

Related Articles

Getting Started with the Terraform Provider for VMware Cloud on AWS

Automating VMware Cloud on AWS SDDC Cluster Lifecycle with DCLI

vSphere Client SDK - changes in the upcoming major release of vSphere