Within VMware Cloud on AWS there are 2 overlay networks – one for management and one for compute. The Management Gateway terminates the connection between your on-premises management network and the VMware Cloud on AWS SDDC management network. The Hybrid Linked Mode connection runs across this connection, and this enables the single management view of both on-premises and cloud SDDCs. This connection can occur over the internet or AWS Direct Connect (DX) if available. Direct Connect provides a dedicated network connection between your on-premises datacenter and the VPC housing your Cloud SDDC. Direct Connect is also a requirement in order to support vMotion live migrations between on-premises and cloud SDDC.
Management Gateway
Here we’re going to configure VMware Cloud on AWS to use an existing DX connection. For more information on how to sign up for a DX connection please see the AWS documentation
Login to VMware Cloud on AWS, select your SDDC and then in the left had column browse to System > Direct Connect. From here you will see all available DX interfaces associated with the linked AWS account. Select the Virtual Interface (VIF) that you wish to connect to your SDDC and click Attach. Note that once you attach the VIF to your SDDC it will no longer be available for connecting to other SDDCs.
You’ll need to check the box to confirm acceptance of the fact that doing this could incur costs on your AWS account. Check this box and click Save.
You’ll see the interfaces come up, bearing in mind that it could take a few minutes for BGP sessions to be established. Once your BGP status light turns green you’ll be able to see the routes that BGP is advertising from your SDDC as well as the routes learned via BGP from your on-premises datacenter.
We need to use this higher bandwidth, low latency connection for vMotion. To do this we need to either reconfigure the gateway of the vMotion subnet to be a device that can route traffic to the on-premises side of the DX connection (vSphere 6.5+ only). Alternatively you can remove the vMotion adapter from your hosts and create a VMkernel adapter with the vMotion TCP/IP stack. Once this is done, configure the vMotion TCP/IP stack to route to a device that can send traffic to the on-premises side of the DX connection.
Once you make these changes it’s recommended that you validate connectivity for vMotion between on-premises and cloud SDDC using vmkping.
Compute Gateway
The other gateway in use is the Compute Gateway. The Compute Gateway enables VM networking capabilities between the on-premises and Cloud SDDC.
This second gateway is only required for hybrid environments where workloads on-premises may need to communicate with the cloud SDDC. Let’s setup an IPSEC connection for this.
Login to VMware Cloud on AWS, select your SDDC and click Networking & Security > VPN > Policy Based > Add VPN
Complete the wizard. You’ll specify your remote public IP address, connection details and the remote and local networks that you want to make available over this VPN. Click Save and you should see the tunnel come up. You can check this by clicking on the Information icon on the right hand side of the screen.
Virtual Machine Networking
In order to be able to interact with each other, VMs in VMware Cloud on AWS need to be connected to a network. While VMware Cloud on AWS was originally released with NSX-V providing the network virtualization layer this is now handled by NSX-T. Individual networks are called Network Segments, and are backed by NSX-T Logical Switches. These network segments can be considered in a similar way to VLANs in a traditional network.
A single Network Segment is deployed out of the box, called sddc-cgw-network-1. This network is connected to the Compute Gateway. Let’s create some new VM networks.
Login to VMware Cloud on AWS and select your SDDC, then click Networking & Security > Network > Segments. Click Add Segments and key in the details of your first compute network segment.
Notice that it’s possible to enable DHCP on a per-segment basis. If you do this you’ll need to configure the range of addresses to assign via DHCP and also the DNS suffix.
Network segments can be routed, extended (more on this in a moment) or disconnected.
Extending Layer 2
You’re probably interested in extending those on-premises networks into the cloud. This enables you to move VMs between the on-premises and cloud SDDC without reconfiguring IP addressing. To do this, we need to stand up a Layer 2 VPN tunnel and then extend some network segments through it.
In your VMware Cloud on AWS console click Networking & Security, then browse to Networking & Security > Network > VPN > Layer 2. Select the local IP address applicable from the dropdown, then enter the remote public IP address for the other end of your tunnel. This address will be assigned to an NSX Edge – Client appliance that you deploy into your on-premises SDDC.
Click Save.
Deploy the NSX Standalone Edge Client
Open the Standalone NSX Edge link in a second browser tab, then download the Standalone Edge – Client package. Once downloaded, unpack the tar.gz archive and extract the virtual appliance files.
Back in the VMware Cloud on AWS console click Download Config. This will download a plaintext file that includes the peer code used to authenticate the L2 VPN. As this is used to establish the VPN connection make sure that you store this in a secure location!
Now that the server-side aspect of the tunnel is setup let’s create an extended network segment. Click Add Extended Segment and enter a name as well as a tunnel ID. This tunnel ID must be the configured identically for both on-premises and cloud SDDC.
Network segments can be switched from extended to routed and vice versa. This can be helpful for migrating workloads into the cloud, and then switching the segment to routed.
Login to the vSphere Client and deploy the Standalone NSX Edge appliance that you downloaded earlier.
Open the L2VPN_config file and copy the peer_code string, then paste it into the Peer Code field as shown.
Once the appliance deployment is completed power it one. Check back into the VMware Cloud on AWS console and you should see the L2VPN tunnel is now up.
You can now migrate your on-premises workloads into your cloud SDDC without changing IP address. Live (vMotion) migrations require that a DX connection is configured. If DX is unavailable the you can perform cold migrations over the IPSEC VPN connection. For further information on the requirements for live migrations please refer to the documentation.
Join me next time where I’ll be discussing the management toolset, including Hybrid Linked Mode, Content Library and more!
