Overview
VMware Cloud on AWS helps users to rapidly provision Software Defined Data Centers (SDDC) with just a few clicks.In order to access these SDDCs in a secure manner, a dedicated connection must be established between the on-premises site and the SDDC. While additional options may be available in the future, today the only supported option for secure connectivity is site-to-site IPSec VPN.
In the following sections we will provide a basic overview of configuring IPSec VPN within an SDDC, preparations required prior to performing this configuration, and considerations when it comes to managing and maintaining an IPSec VPN. Due to the complex nature of IPSec, it is highly recommended that you refer to the post “A Primer On IPSec VPN” in order to better understand the underlying technology.
Brief Architectural Overview
VMware Cloud on AWS is sold and operated as a service, hence clear demarcation exists between management and customer regions in VMware Cloud on AWS. VMware is responsible for the management portion and customers control the operational portion. For example, users cannot make any modifications on the Mgmt resource pool in the SDDC vCenter but can manage workloads under Compute resource pool. Similarly, two gateways provide connectivity to VMware Cloud on AWS. The Management Gateway (MGW) utilizes a VMware NSX Edge™ to enable users to connect to the management layer (vCenter, ESXi hosts, etc.) on restricted ports. The Compute Gateway (CGW) utilizes a separate NSX Edge instance and Distributed Logical Router (DLR) to enable ingress and egress of workload VM network traffic. There are no restrictions on the CGW and customers can configure firewall rules as they choose.
This also means, customers should establish separate VPN tunnels to MGW and CGW from one or more of their existing VPN gateways.
Preparation
IPSec VPN is relatively easy to configure within an SDDC, however effective planning is recommended in order to avoid common mistakes.
1. Settings – Identify the optimal remote gateway device belonging to any vendor that can implement IPsec VPN. The device should support below settings, especially those that cannot be changed in NSX Edge. Refer User Guide for the recommended settings.
Phase 1 Internet Key Exchange (IKE) Settings
Phase 2 Settings
*cannot be modified at this time
2. Connectivity – Verify the peers can reach each other. Identify whether NAT-T is used in the on-premises end i.e. whether the VPN device is sitting behind a NAT gateway. In this case both public and private gateway IP are required to be configured in the VMware Cloud on AWS portal. This also requires firewall port UDP 4500 to be opened on premises for the VPN to function
3. Remote Networks – Identify which subnets are allowed across the VPN connection, commonly referred to as the Protected Networks list or source and destination lists. The list defines the on-premises internal networks that can traverse the VPN to access your cloud workloads, along with what cloud workloads can access across the VPN for different services within your network. Multiple subnets can be added as a comma-separated list, but it is recommended that smaller consecutive subnets be summarized into fewer larger ones. This list of remote networks should match the list of allowed source networks on the peer device
4. Firewall – If the on-premises gateway is behind another firewall or if the device itself is performing a firewall function, then allow the following traffic types from VMware Cloud on AWS gateways to pass through the firewall:
5. VMware Cloud on AWS gateways do not allow traffic through the tunnel by default. Identify and open required firewall ports at both ends of the tunnel, for workloads and management traffic. Sample rule in MGW allowing access to vCenter from on-premises networks is shown below.
6. Configure the on-premises gateway device with supported cryptography settings as mentioned above. It is very important to keep the settings identical on both ends, as most of the VPN issues point back to configuration mistakes. Refer to the NSX documentation for general guidance on configuring various VPN gateway devices. Also refer William Lam’s blog post for a sample on-premises configuration with pfSense.
7. Configuration in VMware Cloud on AWS portal is very simple and the bulk of the configuration is needed at the on-premises gateway end. Make sure the on-premises environment is healthy and is configured properly. Process wise, ensure change procedures are completed in advance and appropriate resources are available to support the configuration and troubleshooting.
Configuration
NSX is the key component that provides networking features on VMware Cloud on AWS. To make things easier, network configuration options are available in a simplified mode by default in the VMware Cloud on AWS portal. The portal is used to configure VPN, firewall rules, NAT, etc., but behind the scenes the settings are applied in the NSX edge. The steps below illustrate the VPN setup in the VMware Cloud on AWS portal.
1. Log in to the VMware Cloud on AWS Console at https://vmc.vmware.com and select your SDDC
2. Navigate to the Networking tab of your SDDC
3. Under Management or Compute Gateway, click VPN and then Add VPN
4. Complete the Management or Compute Gateway VPN configuration as below
5. Open required firewall ports and initiate traffic
6. The tunnel status may be disconnected initially, click the refresh icon to update and wait for the status to change to green
And after refresh >
7. If the tunnel does not come up, click the information icon to get the relevant logging from NSX Edge and troubleshoot accordingly
8. The VPN tunnel can be disabled or deleted under VPN options
9. Monitoring – The management layer including NSX components in SDDC are monitored by VMware and Edge issues are VMware’s responsibility. At the same time, most of the VPN issues are beyond the scope of VMware unless the issue is caused by NSX components. Monitoring a ping-able IP over the tunnel using an existing on-premises monitoring tool could be a simple way to monitor the tunnel status
10. CGW MGW tunnel – VPN tunnel can also be established between CGW and MGW within the portal if required.
Operational Considerations for IPsec VPN
Below considerations are already discussed in VMware ‘Horizon Cloud networking considerations’ document and are relevant to VPN connections to VMware cloud as well. When setting up an IPsec VPN connection from a remote network to VMware Cloud on AWS, keep the following in mind:
Latency spikes – The IPsec VPN tunnel is built through the public Internet and is subject to congestion or other network-related problems common on public Internet connections that can increase latency. Latency spikes caused by the public Internet are beyond the control of both your enterprise and VMware Cloud on AWS.
Dedicated Appliance – When setting up IPsec VPNs, it is recommended that the VPNs be managed using router hardware for optimal performance. Setting up VPNs using a Windows server is not recommended. Multiple VPN connections are supported, although they must not have the same source and destination lists because the Edge Gateway cannot determine which IPsec tunnel to route traffic to.
Redundancy – Incorporating two IPsec VPNs for redundancy is an option, but bonding the VPNs is not supported. The first VPN is set as active, and the secondary VPN is disabled. VMware Cloud on AWS does not provide automated failover for VPNs. If a failure occurs, the VPN must be manually failed over.
Lifetime timers – Some IPsec VPN parameters, such as the Security Association (SA) lifetime timers, which define the lifetime that a given tunnel uses to encrypt data, cannot be changed in Edge Gateway. These parameters must be changed on the on-premises equipment to match those in Edge Gateway. The deployment process includes two phases, and both Phase 1 and Phase 2 include SA lifetime timers. When the SA timer expires, it renegotiates authentication for both sides. However, Edge Gateway does not re-authenticate on traffic, it re-authenticates only on the lifetime timer. Therefore, if the timers are not set on the on-premises side to match those on the VMware Cloud on AWS side, they can cause problems in the VPN tunnel.
Network routing – For VPN-based connections in VMware Cloud on AWS, static routing is configured during the VPN peering process. Static routes must be maintained on the customer side in order to reach IP subnets delegated to the SDDC, and these static routes need to point to the corresponding VPN tunnel.
Summary
Preparation is a key step in keeping VPN setup in VMware Cloud on AWS simple and without issue. VMware teams are working to make the VPN setup simpler by making available the resources around troubleshooting and configuration. Watch out for more on this topic!
