VMware Transit Connect - Simplifying Networking for VMC

The release of VMware Cloud on AWS (VMC) 1.12 brings a number of exciting new capabilities to the managed service offering. A comprehensive list can be reviewed in the release notes. A key feature that is now Generally Available (GA) in all VMC commercial regions worldwide is VMware Transit Connect^TM. VMware Transit Connect enables customers to build high-speed, resilient connections between their VMware Cloud on AWS Software Defined Data Centers (SDDCs) and other resources. This capability is enabled by a feature called SDDC Groups that helps customers to logically organize SDDCs together to simplify management.

The SDDC Group construct empowers customers to quickly and easily define a collection of SDDCs, Virtual Private Clouds (VPCs) or on-premises connectivity that need to interconnect. Additionally, the SDDC Group construct provides value inside the individual SDDCs by simplifying security policy as will be shown later in this post. Behind the simplification that SDDC Groups provide is the instantiation of an VMware Managed AWS Transit Gateway, a VTGW. The VTGW is a managed service from VMware and provides the underlying connectivity between the different resources.

The initial Transit Connect service provides three primary connectivity models:

SDDC to SDDC
SDDC to Native AWS VPC
SDDC to On-premises over Direct Connect Gateway

Figure 1 – VMware Transit Connect – Sample Topology

These models solve the most commonly used connectivity requirements our customers have asked for. Over time this service will evolve and add additional options based on customer feedback and our partnership with AWS. Let’s dig into the details of how these connectivity models work.

First, we will establish some requirements to use Transit Connect.

SDDCs must be at release 1.11 or higher
SDDCs and VPCs that are members of a SDDC group MUST be in the same region
SDDC management CIDRs MUST not overlap
SDDC networks shouldn’t overlap but if they do, know that overlapping segments won’t be advertised

Next, it is important to discuss the supported flows across Transit Connect. The simple rule is that at least one endpoint in any flow MUST be a resource within a SDDC. This is a requirement because as a managed service, our support teams need to have the ability to observe at least one end of flow. This is a rule that is enforced by the Member routing domain inside the service that allow SDDCs to communicate with any resource like another SDDC, a VPC or on-premises prefix. The External routing domain only routes to member SDDCs. This can be summarized into this simple table.

SDDC to SDDC – permitted
SDDC to VPC – permitted
SDDC to on-premises – permitted
VPC to VPC – prohibited
On-premises to VPC – prohibited

With these considerations in mind, lets review the topologies that can be created.

Network Topology 1 – SDDC to SDDC

The first is SDDC to SDDC as depicted in Figure 2.

Figure 2 – SDDC to SDDC

In Figure 2 a topology with 3 SDDCs in the same AWS region is depicted. Two of the SDDCs are members of a SDDC group and can communicate through the high speed VPC attachment created with the VTGW.

This process starts in the VMC Console by clicking on the Actions button in the upper right of the screen and then selecting Create SDDC Group.

This action brings up the next 3 step process where the Transit Gateway is named, member SDDCs are selected and the acknowledgement is agreed to before clicking Create Group.

Create Name, Description and Members for SDDC

Once Create Group is selected, the backend work to instantiate the required objects and connectivity begins and the SDDCs will show in the view details page a Connectivity Status of PENDING. Once the tasks are complete the Connectivity Status will transition to CONNECTED as shown below.

We can select the routing tab across the top and review the prefixes learned from each SDDC as shown below.

Within the SDDC insight into the routing tables are visible using the new Transit Connect tab on the left navigation bar.

Get SDDC Insights via VMware Transit Connect

With the SDDCs now connected, we have the plumbing in place for them to communicate but with our focus on security at VMware, the default firewall policy will not allow resources in each data center to communicate with each other. The appropriate gateway firewall policy must be defined. This is another example of where the VMC SDDC Grouping object helps ease administrative burden. The SDDC Group maintains a list of networks advertised by each SDDC, VPC and on-premises connection and automatically populates system managed groups for the firewalls as shown below.

The groups can be used as sources and destination groups to configure firewall policy and are designed as a method to simplify customer’s experience. They are not mandatory to be used if you prefer to configure more explicit firewall policy. The image below is an example of the SDDC Group Firewall Policy objects being used. Also note the use of the “Applied To” field to select the Direct Connect interface. This is important to note because the VPC attachment between the SDDC and the VTGW goes through a NSX Edge router interface named “Direct Connect Interface.”

With this final step, we now have end to end communications enabled between these SDDCs using VMware Transit Connect.

Network Topology 2 – SDDC to VPC

The next connectivity model supported enables resources in SDDCs to communicate with resources in Native AWS VPCs. This supercharges the hybrid connectivity by reducing the reliance on VPNs to tie these environments together. When completed, a topology like the one depicted in Figure 3 is enabled.

Figure 3 – SDDC to Native AWS VPCs

Looking at Figure 3 in detail we can see the same topology depicted in Figure 2, with the addition of the Native AWS VPC to topology. Configuring this topology requires access and coordination between administrators with access to both the Native AWS VPC as well as the VMC SDDCs. Specifically, the AWS account will need the following information.

Read/write permissions to the VPC(s) that will be connected to Transit Connect
AWS Account ID(s) where the VPCs reside
VPC IDs and CIDRs that will be connected to Transit Connect

The VMC user will need to be a Cloud Admin for these operations.

To start adding a VPC to the Transit Connect configuration, navigate to the VPC Connectivity tab and click on ADD AWS ACCOUNT as illustrated below.

Add a VPC to the Transit Connect Configuration

You will be prompted to enter the AWS account number associated with the VPC or VPCs you want to connect as below.

Once the account ID is added, it will show as ASSOCIATING in the VMC console. To complete this process, the AWS Console will need to be accessed and once logged in with a user who meets the access privileges outlined above, a notification for a Resource Share will be seen in the AWS Resource Access Manager.

Click on the VMC-Group, accept the share and confirm it. The screen will show the invitation has been accepted.

After a period of time, the State in the VMC Console will change from ASSOCIATING to ASSOCIATED.

With this step complete, the next step is done in the AWS Console under the VPC tab in the Create Transit Gateway Attachment page. Select the VTGW, VPCs and subnets that will use the Transit Connect service and click on Create attachment. The screenshot below shows this process.

A confirmation window will appear and the attachment process is initiated. The AWS Console will reflect that the State is “pending acceptance” and will remain like this until the next step is completed. Now it is time to jump back to the VMC Console where the VPC that was just attached to the VTGW will be displayed. Highlight the VPC and click on ACCEPT. The screen will transition to PENDING before showing AVAILABLE. This process may take up to 15 minutes depending on the timing of the requests so be patient, it is working.

Once the Status is AVAILABLE, the initial connectivity is in place but unlike SDDC to SDDC communication, a few additional steps need to be performed. The first is to update the VPC routing table to populate the VMC connected networks as being reachable via the VTGW. This is accomplished in the AWS Console for the specific VPC and would look similar to the image below.

Edit Routes so VMC is Reachable via the VTGW

Also, don’t forget to configure the Security Groups associated with any EC2 instances to allow security policy to reflect the flows you require. The same reminder applies to the Gateway Firewall on the SDDCs. Transit Connect provides the networking but AWS Security Groups and NSX Gateway Firewalls provide the security enforcement points.

Network Topology 3 – SDDC to On-Premises

The last connectivity method we’ll review is on-premises connectivity. To provide this connectivity in the past, customers would provision Direct Connect circuits between their data centers or co-location facilities and the SDDC and frequently use a private Virtual Interface (VIF). With Transit Connect, a new type of VIF must be used, a Transit VIF and a Transit VIF can only be terminated between an AWS Direct Connect Gateway and a TGW. Direct Connect Gateways are not region based but are rather a global construct so don’t have the same considerations for regional co-location that SDDCs and VPCs require.

Figure 3 – SDDC to Native AWS VPCs

To configure a Direct Connect Gateway in Transit Connect, click on the Direct Connect Gateway tab in the VMC Console and click on ADD ACCOUNT.

Populate the fields required with a special focus on the Allowed Prefixes. Note that AWS supports 20 prefixes being advertised to the on-premises networks, so consider summarization of the networks.

Attach Your Direct Connect Gateway to the SDDC Group

The VTGW will request an association with the Direct Connect Gateway owner. This will show as REQUESTED in the VMC Console as illustrated below.

Request Association with Direct Connect Gateway Owner

In the AWS Console, accept the proposed TGW association as shown in the following screenshot.

After accepting the association, the opportunity to accept the BGP proposal is presented as shown below.

Click Accept proposal and the systems will process the requests. Please note that this may take up to 20 minutes.

Security Policy Update to Complete Connection

As with the SDDC to SDDC and the SDDC to VPC communication models, when a Direct Connect Gateway connection is established, security policy must be updated to complete the connection. One notable difference is that with an on-premises environment there may be physical firewalls that could need updated with routing and/or security policy to communicate with the Transit Connect resources.

This is the initial offering of this solution to our customers and we have a lot of great functionality available to simplify and solve customer’s hybrid cloud computing challenges.