VMware Cloud Foundation on Dell EMC VxRail Blog Series – part 6: NSX-T and L3 Stretch Cluster updates

Author:

Amitrajit Chatterjee, Field Solutions Architect

In the previous blog, we discussed how to deploy NSX-T with VMware Cloud Foundation (VCF) on Dell EMC VxRail. With the latest release of Cloud Foundation 3.9.1, customers can now deploy NSX-T and select the physical network interface cards (pNICs) more easily.

If you recall in the previous blog, there was a pNIC selection algorithm that you had to take into consideration before deploying NSX-T. During the “Add Cluster” workflow, VxRail would select the first two free pNICs (of similar model and speed) and use them for creation of N-VDS. With VCF 3.9.1, there is now a new user interface (UI) that will allow you to choose specific pNICs for use with NSX-T – this dramatically simplifies the deployment process for NSX-T.

The team has also made it easier to deploy L3 stretch clusters for those customers wishing to deploy High Availability (HA) services to protect against Availability Zone (AZ) failure. In order to deploy L3 stretched clusters, customers must be running a minimum of VxRail release 4.7.300, which introduced L3 networking capabilities.

The L3 stretched cluster capability was first introduced in Cloud Foundation 3.9 and was accomplished via manual guidance. In Cloud Foundation 3.9.1, this has now been automated with the help of the SOS (Supportability and Serviceability) utility integrated into VCF. It is a command-line Python tool that is use for running health checks, collect logs and perform vsan stretch clusters.

The pre-requisites for preparing the cluster for stretching

• Ensure that you have a vSAN Enterprise license, which is required for stretching a cluster.
• In a L3 environment, for the management domain the management VLAN needs to be L2 stretched. Else in case of a failover the management vms will need to be re- configured.
• All VMs on an external network must be on a virtual wire. If they are on a VLAN, that VLAN must be stretched as well.
• Each availability zone must have its own vMotion, vSAN, and VXLAN networks.
• Each stretched cluster requires a vSAN witness host in a third party location. The maximum RTT on the witness is 200ms.
• If you are stretching a cluster in a VI workload domain, you must stretch the management domain cluster first. The vCenter Servers for all workload domains are in the management domain. Hence, you must protect the management domain to ensure that you can access and manage the workload domains.
• Ensure that you have enough hosts such that there is an equal number of hosts on each availability zone. This is to ensure that there are enough resources in case an availability zone goes down completely.

The details for the Cloud Foundation networks for Layer 3 are as follows:

The Dell-EMC VxRail now allows for adding hosts over L3 by utilising a “node proxy” mechanism. After the VCF-VxRail AZ1 rack has been built out, user can add a rack in AZ2 over L3. From the VxRail Manager of AZ1 rack, user manually selects a new node from the AZ2 rack and configures the management network for that node. The node will be performing as a proxy node. Subsequently VxRail Manager can display the discovered nodes in that rack and let user select which nodes need be added. The management network between AZ1 and AZ2 needs to be on the same subnet.

Once AZ1 is up and running and AZ2 hosts are available via proxy then the SOS tool can be used to stretch the cluster. For stretching the Management Domain the following document can be used – VMware Cloud Foundation on Dell EMC VxRail Admin Guide – 3.9.1

For stretching the NSX-T workload domain, using SSH, log in to the SDDC Manager VM and run the following command to prepare the environment –

/opt/vmware/sddc-support/sos –prepare-stretch –sc-domain <SDDC-valid-domain-name> –sc-cluster <valid cluster name which is a part of the domain to be stretched>

Once the workflow is triggered, track the task status in the SDDC Manager UI. Once it finishes successfully, run the following command to stretch the cluster for Layer 3 (L3) networks:

/opt/vmware/sddc-support/sos –l3-stretch –stretch-vsan –sc-domain <SDDC-valid-domain-name> –sc-cluster <valid cluster name which is a part of the domain to be stretched> –sc-hosts <valid host names> –witness-host-fqdn <witness host/appliance IP or fqdn> –witness-vsan-ip <witness vsan IP address> –witness-vsan-cidr <witness-vsan-network-IP-address-with-mask>

Enter the inputs for the following:

• ESXi host passwords
• VSAN gateway IP/CIDR for the preferred(primary) and non-preferred(secondary) site

Once the tasks complete successfully the AZ1 and AZ2 will be up and running in VSAN stretched cluster mode. Subsequently the customer can expand/shrink existing stretched clusters or choose to stretch new domains/clusters.

For more details follow the VMware Cloud Foundation on Dell EMC VxRail Admin Guide – 3.9.1