Introduction to Edge Clusters in VMware vCloud Director 9.7 (Part 1 of 3)

Introduction – What is an Edge Cluster?

In this blog post, we will discuss the importance of Edge Cluster capability in VMware vCloud Director (vCD) 9.7. With this new addition, network services for multi-tenant architectures are greatly enhanced and we will be covering these considerations in further detail.

Today in vCloud Director, NSX-V Edge Gateways VMs, or Edge Services Gateways (ESGs), are deployed in a System Resource Pool belonging to an Organization Virtual Data Center (oVDC). These ESGs are the essential function for all virtual network services such as NAT, Edge Firewall, Load Balancing, VPN services, and so forth. In a multi-tenant provider architecture, one strives to ensure all services are highly available to meet stated service level agreements. Edge Cluster capability in 9.7 presents new availability, security, and network hygiene enhancements.

By our definition, an Edge Cluster is a logical construct driving a placement decision of oVDC Edge Services Gateways which consist of a resource pool and a desired storage policy. The Edge Cluster is created by the provider administrator and then assigned to Org VDCs as a primary or secondary (for high availability) instance to the Org VDC Network Profile. Therefore, the deployment of ESGs to Edge Clusters are transparent by nature and analogous to deploying a network from a network pool today.

So why are Edge Clusters important in vCloud Director and why should a provider configure them?

1. Consumption of dedicated Edge Clusters for North/South traffic – optimized traffic flow while minimizing the span of Layer 2 broadcast traffic. In essence, a provider can control and streamline tenant traffic by utilizing Edge Clusters.
2. Provide a higher level of availability to Edge nodes that can distinctly fail between two clusters.
3. Ability to balance organization Edge services between multiple Edge Clusters – I do not have to use the “same” Primary and Secondary Edge Cluster for every org VDC. This can be configured on a per orgVDC basis.

Pre-9.7 Edge Capabilities

Previously, starting with version 9.0 of vCD, one could establish an Edge Cluster by configuring metadata on the Provider Virtual Data Center (pVDC) object. Another colleague of mine, Timo Sugliani, did a great overview of the metadata capabilities located here.  However, this presented a few considerations and limitations:

1. Lack of ability of a custom storage profile – default storage profile from the oVDC was utilized.
2. Single cluster availability – in the event of cluster failure, no automated failover to a secondary Edge Cluster.
3. vCD has to use affinity rules to achieve placement while maintaining distinct entities, adding overall complexity to the system and vCD placement.

In 9.7, Edge Cluster is the desired design configuration over placement by metadata configuration. If both solutions are utilized, the Edge Cluster implementation takes precedence over the metadata placement. This is important to note when transitioning from metadata placement to Edge Cluster placement – since this configuration is non-disruptive, we can apply this on a per-oVDC basis and redeploy the tenant Edge(s) during a maintenance window.

While we will continue to support this legacy configuration, a dedicated Edge Cluster presents many new opportunities for a provider.

9.7 – Edge Cluster Capabilities

The following are the key functions of Edge Cluster inside of vCD 9.7 –

1. Management of Edge Clusters – the ability to create and manage Edge Clusters as it pertains to vCD constructs. Edge Clusters are an entirely new construct that is defined inside of the vCD API.
2. Definition of Primary and Secondary Edge Clusters for oVDCs – we now can stipulate the desired placement of org VDC edges to an Edge Cluster while meeting availability metrics with a secondary location.
3. Custom storage policy selection – each Edge Cluster can consume a specific storage policy that is originated by the provider administrator inside of vCenter and vCD. The distinct advantage is discrete control of which storage system is utilized by ESG’s.
4. Isolation of Edge workloads – since ESGs are on dedicated computing infrastructure, we can minimize northbound connectivity to only these specific Edge Clusters. More on that shortly.

In definition, an Edge Cluster constitutes the following:

1. vCenter Resource Pool Object
2. Storage Policy (Profile) – from vCenter/vCD
3. Name and Description

Currently, Edge Cluster instantiation and configuration must be done via the vCD API.

New Network Construct in vCloud Director – VDC Network Profile

In vCD 9.7, we are introducing a new organization VDC specific construct – VDC Network Profile.

Within a VDC Network Profile, one has the ability to establish provider networking resources within this organization VDC (oVDC). Going forward, every oVDC will have a default Network Profile attached to it. This is where we configure the use and consumption of distinct Edge Clusters.

In the below screenshot, we can see a VDC that has specific network properties available – “primaryEdgeCluster” and “secondaryEdgeCluster.”

With 9.7, a VDC Network Profile has the ability to utilize a Primary and Secondary Edge Cluster in an NSX-V backed oVDC object. More on the distinct differences between Primary and Secondary Edge Clusters.

The end goal is the ability to continue to advance Network Profile services available on a per oVDC basis.

Establishing Edge Clusters and Applicability to Organization VDCs

Above a high-level view of the steps for instantiating an Edge Cluster inside of a vCD instance. Our next blog post will review this in a step by step configuration. However, we wanted to provide a view on how this looks from a relational mapping perspective.

Below, we can see how the JSON body is formulated on each Edge Cluster (under the /cloudapi/1.0.0/edgeCluster API path) while each Tenant oVDC has a “/networkProfile” now available for configuration.

While this provides a provider the flexibility to specify a primary and secondary Edge Cluster configuration on a per-oVDC basis, this does not necessarily mean it must be the *same* primary and secondary instance each time. Therefore, one could have something like the following –

In the above example, Tenant 3 oVDC has its primaryEdgeCluster associated to “RegionA01-EDGE02” while the secondaryEdgeCluster is associated to “RegionA01-EDGE01.” What’s unique is the ability to essentially load balance Edge resources at a very granular level. Moreover, we could have up to ten (10) Edge Clusters registered to a vCD instance, so many more permutations could be constructed from this overall approach.

What’s Next?

In the next blog post, we will review a step by step guide on registering two Edge Clusters to a vCD instance while configuring a oVDC to consume these new Edge Cluster resources.

Part 2 – Setting up Edge Clusters in VMware vCloud Director 9.7 (Part 2 of 3)

Part 3 – Edge Cluster Design and Migration Considerations for VMware vCloud Director 9.7 (Part 3 of 3)

By Daniel Paluszek and Abhinav Mishra