Updated Feb, 2021
In the first post, I covered how to add Cloud Accounts and view discovered compute and network resources in Cloud Assembly. If you missed the first post or need a refresher, click here – part 1 – to check it out. Each of the posts in this series assumes you have access to Cloud Assembly with the pre-reqs for provisioning handled. You will also want NSX configured and running with uplink connectivity to your gateway.
Let’s dive in. Network Profiles control which network constructs are used for placement decisions during a deploy. They also control the level of isolation a workload will have when deployed. If you recall from the first post, I added vSphere and NSX environments and will continue working with those entities. In the examples, two network profiles have been created for the NSX-T environment. Each profile widget includes a summary of the compute and networking entities associated with the profile. The options for NSX-v and NSX-T are very similar, with some differences behind the scenes, but essentially the same end results. The first network types I will cover are Existing networks and Outbound on-demand networks. I’ll cover Private on-demand security group options, public and routed networks in the third post.
Working with Infrastructure as Code Blueprints
The best place to begin learning how Cloud Assembly interacts with networking is by looking at a blueprint. Selecting the Blueprints option allows you to add and interact with blueprints. I’ll start by opening the example blueprint called Single-VM-Nat. As the blueprint name suggests, a single machine object and network with a NAT rule will be created, among other configurations, during the deployment.
To create a similar blueprint, simply drag the Cloud Agnostic machine object and NSX Network object to the canvas. I highlighted the portion of YAML which tells Code Assembly what network capabilities to look for during a deployment. In the example below, I’ve selected an NSX network entity. The networkType: outbound property will create a new network with outbound access and NAT configured. The networkType setting instructs the placement engine to look for a network profile that matches the request. But I’m getting ahead of myself, I’ll provide more detail on network profile and blueprint interactions later in the post.
Note: you could also drag a Cloud Agnostic or vSphere network onto the canvas. For this example, I chose NSX in order to display all available networkTypes in the blueprint.
Before moving forward, I removed outbound from the YAML panel by backing out outbound. Doing so displays available networkType options as shown below. We’ll go through each type and I’ll show you how to configure Cloud Assembly network profiles to accommodate each blueprint setting.
Network Type Existing
networkType: existing uses a discovered network or deployed network Origin type. I know that can be confusing so let me explain.
- Discovered origin: network objects (logical switches) created in NSX manually, using 3rd party methods, or another Cloud Assembly instance and found through the current Cloud Assembly discovery service
- Deployed origin: network objects (logical switches) provisioned by Cloud Assembly
For example, a provider network would appear as deployed. A provider network is created from a blueprint where the only object on the canvas is a network. No machines are associated. There are scenarios where an organization may want to create a static provider network for developers to use without creating an on-demand network each time a deployment occurs. The key thing to remember is both Origin types can be used for existing networkType provisioning.
When NSX is involved, deployed and discovered networks can be a switch configured for VXLAN or Geneve overlay networking. The switch could be VLAN backed too. Workloads deployed, using networkType: existing, with either Origin type, will use static or dynamically assigned IP addresses depending on your NSX, VM, and configuration management options.
Switching from my blueprint to a network profile, I’ve selected the Networks option. I added existing networks, notice the switch configured manually (outside Cloud Assembly) shows Discovered for an Origin, whereas the Cloud Assembly created switch shows Deployed.
In another example, I added existing networks that will be used for placement decisions. I chose App and DB with the constraint tags net:app and net:db for each switch. Constraint tags allow me to assign specific networks to each tier of my application in the blueprint or give users the ability to choose different networks at deployment time. The switches have their own DHCP server and assigned IP range. Remember – for Existing networks, DHCP is configured in NSX independently from Cloud Assembly. So using this configuration, App and DB will have their own distinct IP ranges.
Next, navigate to the Network Policies option and confirm the Isolation policy is set to Do not create on-demand network or on-demand security group, as shown in the screenshot. Don’t forget to confirm the External network, T-0 logical router, and Edge cluster provide the desired network connectivity for your VMs. The External network is technically not required for this network profile unless you’re adding load balancers in the blueprint.
Network Type Outbound
networkType: outbound creates on-demand networks. By default a mixed DHCP and Static IP range assignment is used for on-demand networks. When DHCP is selected for a range assignment with NSX-T, a new T-1 gateway (aka logical router), L2 switch, one-to-many SNAT rule, DNAT and Port Forwarding rule, DHCP server with IP pool, NAT route advertisement, and the proper uplinks/downlinks will be created for each blueprint deployment. Static IP and DHCP IP pools are based upon the CIDR and subnetting configuration specified in the network policies and network options. I’ll provide more detail on range assignments later in this post.
Outbound network types require that we have a network configured with a network IP range. In the example, I used a switch attached to an Overlay Transport Zone. Remember – A CIDR is required to use the switch in the External network field within Network Policies. A CIDR can be added by clicking the network name. Click the checkbox for the network and choose Manage IP Ranges (it will be available to select once you click the checkbox).
Manage IP Ranges creates a pool of IPs Cloud Assembly will allocate for use as translated external IPs. The IPs are assigned to each SNAT and DNAT rule that is created in NSX. Also these IPs are used for new load balancer virtual IPs.
Cloud Assembly requires a fully configured NSX environment with the proper routes and gateways for this scenario to work properly.
Cloud Assembly will track the allocation of IP addresses using the built-in IPAM capability. Click the IP range name to view allocation details. If you are running out of IP addresses, simply modify the start and end IP addresses to increase the pool size. Alternatively a new IP range can be added. Infoblox IPAM can be used to manage and track IP allocation as well.
Switching to the Network Policies option, for the isolation policy select Create an on-demand network. Then you must add the discovered transport zone, external switch, T-0 gateway, and Edge cluster. You will need to add a desired network CIDR, subnet size, and IP range assignment. In the example, I chose /28. Meaning a separate /28, based on the defined CIDR, will be assigned to each on-demand network that’s created.
Clicking the Subnet size dropdown presents the available subnet size options. Subnet size instructs Cloud Assembly to create a new network based on a portion of the defined CIDR, in this case part of 172.90.2.0/24. Recall with NSX-T, a T-1 gateway, switch, DHCP server and pool, route advertisement change, plus SNAT and DNAT rule is configured for each /28 network Cloud Assembly adds. The VM(s) are assigned to the switch as part of the deployment process. This is a really cool option, and is similar to how a network team might carve up private network ranges in an underlay network, but with far less effort and a much shorter time for creation.
IP range assignment controls how Cloud Assembly will assign IP addresses to VMs. I mentioned before that the default is Static and DHCP (or mixed) when a new profile is created. The Static and DHCP setting instructs Cloud Assembly to create two IP ranges. In my example, I used a /28, so two /29s will be created for IP assignment. Where applicable, one /29 will be used to assign static IP addresses to VMs and the other for dynamic assignment to VMs. All of this happens behind the scenes, there’s no need to worry about further configuration in Cloud Assembly. You can also choose either Static or DHCP from the drop down. In that case only one range is created and IPs are assigned based on either setting. Later when you add the created network in a network profile, the ranges will already be defined for you.
Two things to note, there is an exception to subnet size changes behind the scenes, when a /29 Subnet size is selected, we don’t create two /30s. Deployments will fail if a /29 is used when IP range assignment is set to Static and DHCP. Also IP range assignment is supported for outbound, routed, and private network types.
The blueprint controls whether a VM receives a static or dynamic IP. Add assignment: static to the network properties of a VM in the blueprint to assign a static address. If assignment: static isn’t present, a dynamic IP address will be used.
Update!
Gateway Resource and New NAT Resource with DNAT and Port Forwarding
vRA 8.3 and vRA Cloud offers support for a shared Gateway Object. The Gateway object can be connected, and most importantly, shared between on-demand networks within that deployment. Once you add the Gateway object, click dragging between that object and the target network connects those objects. At deployment time, a single T-1 Gateway will be created and shared among connected networks. Shared Gateways work with the Outbound and Routed network types and multiple Gateways can still be created for each deployment.
For the NAT resource, in the YAML code, the VM or Load Balancer that’s connected will appear in the natRules property value as a translatedInstance. Index controls the priority of the rule as it would appear in NSX. The DNAT rule IP will be pulled from the IP range associated in the Network Profile, just like a SNAT rule. You can also set the source IPs, in the example below ANY is set, but we could use an IP or range of IPs for the Source. Additionally we can set the source ports, also a single port or range of ports, and protocol, as well as destination and translated ports. These configurations setup a DNAT rule with port forwarding in NSX once you click Deploy
YAML portion of the Gateway and NAT resources.
After the DNAT rule and, if present, Port Forwarding rule has been provisioned, you can reconfigure NAT within the deployment topology interface. Clicking the NAT object and selecting Reconfigure, brings up the reconfiguration window for NAT rules. You have the options to add additional rules or edit an existing rule.
Clicking Edit on an existing rule, opens the NAT Rules reconfiguration window. You can change the configurations you set in the Cloud Template. Clicking Save, then Submit, puts in motion an update action for the NAT rules associated with the deployment. The updates will appear in the NAT rules portion of the Topology view once complete.
Summary and NSX Sizing
In summary, each time a blueprint with networkType: outbound is deployed, Cloud Assembly is instructing NSX to spin up networking entities and make numerous configuration changes. In fact, it’s so easy to create on-demand networks (and delete them) in NSX, you will want to provide governance over their creation and use to minimize potential network sprawl and stay below NSX-T maximums. This situation is also where Provider networks often come into play. Additionally, you can choose to control the number of T-1s that are created by sharing them within a deployment or choosing a larger subnet size. Cloud Assembly will stop provisioning networks once it can’t carve up the network any further. Instance limits and leases offer further ways to manage resource utilization. For visibility purposes, the network resources view in Cloud Assembly shows what has been deployed through Cloud Assembly to NSX too. Of course, if you want the best visibility into your NSX and vSphere (plus public cloud) networking and security configurations, there is no better product than vRealize Network Insight!
Now that you see how easy it is to create and use networks in NSX, don’t hesitate to try it out in your environment. In the next post, I’ll go through the other network types and ability to use and create NSX security groups.
Network Automation with Cloud Assembly and NSX part 1 – Network Automation with Cloud Assembly and NSX part 3 – Network Automation with Cloud Assembly and NSX part 4
