SAP HANA on vSphere 7 Update 1 – vSphere Cluster Service (vCLS)

Overview

Starting with vSphere 7.0 Update 1, vSphere Cluster Service (vCLS) is enabled by default and runs in all vSphere clusters. VMware would like to make critical cluster services like vSphere High Availability (HA) and vSphere Distributed Resource Scheduler (DRS) always available and vCLS is an initiative to reach the vision.

SAP HANA as the foundation of most SAP business applications is a very critical asset of all companies using SAP solutions for their business. Due to the criticality of these applications for a business it is important to protect and optimally operate SAP HANA.

Running SAP HANA on VMware vSphere provides an easy way to protect and operate it by leveraging vSphere cluster services like vSphere High Availability (HA) or vSphere Distributed Resource Scheduler (DRS), which depends on vCenter Server availability for its configuration and operation.

The dependency of these cluster services on vCenter is not ideal and VMware introduced in the vSphere 7 Update 1 release the vSphere Clustering Service (vCLS), which is the first step to decouple and distribute the control plane for clustering services in vSphere and to remove the vCenter dependency. If vCenter Server becomes unavailable, in future, vCLS will ensure that the cluster services remain available to maintain the resources and health of the workloads that run in the clusters.

vCLS is enabled when you upgrade to vSphere 7.0 Update 1 or when you have a new vSphere 7.0 Update 1 deployment. vCLS VM(s) get automatically deployed as part of vCenter Server upgrade, regardless which ESXi version gets used.

Architecture

vCLS uses agent virtual machines to maintain cluster services health. Up to maximal three vCLS agent virtual machines (vCLS VMs) are created when you add hosts to clusters. These vCLS VMs, which build the cluster control plane, are lightweight agent VMs.

vCLS VMs are required to run in each vSphere cluster, distributed within a cluster. vCLS is also enabled on clusters which contain only one or two hosts. In these clusters the number of vCLS VMs is one and two, respectively.

Below figure shows the high-level architecture with the new cluster control plane.

A cluster enabled with vCLS can contain ESXi hosts of different versions if the ESXi versions are compatible with vCenter Server 7.0 Update 1. vCLS works with both vLCM and VUM managed clusters and runs in all vSphere license SKU clusters.

vCLS VM details

vCLS VMs run in every cluster, even if cluster services like vSphere DRS or vSphere HA are not enabled on the cluster.

Each vCLS VM has 100MHz and 100MB capacity reserved in the cluster. For details check following document.

In the normal use case these VMs are nearly not noticeable in terms of resource consumption and users are not expected to maintain the lifecycle or state for the agent VMs, they should not be treated like the typical workload VMs.

Number of vCLS Agent VMs in Clusters:

vCLS Deployment Guidelines for SAP HANA Landscapes

As of SAP note 2937606, SAP HANA on VMware vSphere 7.0 in production, which includes the support for vSphere 7.0 Update 1 following restriction got defined:

“SAP HANA VMs can get co-deployed with SAP non-production HANA or any other workload VMs on the same vSphere ESXi host, as long as the production SAP HANA VMs are not negatively impacted by the co-deployed VMs. In case of negative impact on SAP HANA, SAP may ask to remove any other workload”. Also, “no NUMA node sharing between SAP HANA and non-HANA allowed”.

Due to the mandatory and automated installation process of vCLS VMs, when upgrading to vCenter 7.0 Update 1, it is necessary, because of above guidelines, to check if vCLS VMs got co-deployed on vSphere ESXi hosts that run SAP HANA production level VMs. If this is the case, then these VMs must get migrated to hosts that do not run SAP HANA production level VMs.

This can get achieved by configuring vCLS VM Anti-Affinity Policies. A vCLS VM anti-affinity policy describes a relationship between VMs that have been assigned a special anti-affinity tag (e.g. tag name SAP HANA) and vCLS system VMs.

If this tag is assigned to SAP HANA VMs, the vCLS VM anti-affinity policy discourages placement of vCLS VMs and SAP HANA VMs on the same host. With such a policy it can get assured that vCLS VMs and SAP HANA VMs do not get co-deployed.

After the policy is created and tags were assigned, the placement engine attempts to place vCLS VMs on the hosts where tagged VMs are not running.

vCLS Deployment Examples for SAP HANA Landscapes

Customers deploy SAP HANA typically on dedicated ESXi hosts. These hosts can be part of small or large clusters (in terms of number of hosts). They can be mixed with hosts running non-SAP HANA workload VMs or can be part of a dedicated SAP HANA only cluster.

Following section provides examples of typical SAP Landscape Clusters and provide some guidelines where to place the up to three lightweight vCLS VMs.

Mixed SAP HANA and non-SAP HANA VM vSphere Cluster

A mixed cluster should be the typical scenario for most customers. In this case check the vCLS VMs if they got deployed on ESXi hosts that run production level SAP HANA VMs. If yes, then the vCLS VM may run on the same CPU socket as a SAP HANA VM. To avoid this, configure  vCLS Anti-Affinity rules as described in following document.

Below figure shows the initial deployed vCLS VMs and how these VMs will get automatically migrated (green arrows) when the Anti-Affinity rules are activated to comply with SAP notes 2937606 and 3102813. Not shown in this figure are the HA host / HA capacity reserved for HA failover situations, which could get used to run the vCLS VMs.

Dedicated SAP HANA VM vSphere Cluster

Customers may have deployed an SAP HANA cluster with dedicated hosts that run only SAP HANA workload VMs. In this case automatically deployed vCLS VMs cannot get migrated easily to hosts that do not run SAP HANA VMs. Solution is to add existing hosts with non-SAP HANA workload VMs to this cluster, or to have non-tagged SAP HANA non-production VMs running on at least one host. These existing hosts may run any workload like SAP application server VM or infrastructure workload VMs. It is not required to buy new host for this!

Below figure shows the initial deployed vCLS VMs and how these VMs will get moved, when the SAP HANA vCLS Anti-Affinity rule gets executed, to cluster added hosts. As before, not shown in this figure are the HA host / HA capacity reserved for HA failover situations, which could get used to run the vCLS VMs.

SAP HANA HCI vSphere Cluster

Just as with the dedicated SAP HANA cluster an SAP HANA HCI cluster may only run SAP HANA workload VMs. As with SAP HANA running on traditional storage, SAP HANA HCI (SAP note 2718982) supports the co-deployment with non-SAP HANA VMs as outlined in SAP note 2937606 (vSphere 7.0) and 2393917 (vSphere 6.5/6.7).

If vCenter gets upgraded to 7.0 U1 then the vCLS VMs will get automatically deployed on SAP HANA HCI nodes. If these nodes are exclusively used for SAP HANA, then these vCLS must get removed and migrated to vSphere hosts hat do not run SAP HANA VMs. If all hosts get exclusively used for SAP HANA VMs then the vCLS VMs can get alternatively migrated to the HA host(s) (e. g. n+1 cluster).

vSphere ESXi hosts that could get added are for instance the host that supports vCenter or SAP application server workload VMs.

In the case of an SAP HANA HCI partner system validation or if an additional non-SAP HANA ESXi host cannot get added to the cluster then the Retreat Mode can get used to remove the vCLS VMs from this cluster. Please note the impacted cluster services due to the enablement of Retreat Mode on a cluster.

Note: To allow a vCLS VM to run on a mixed-host, as shown in above picture, no “SAP HANA” tagged VM can run on this host. Not shown in the picture is the HA host(s), which could also run the vCLS VMs.

Adding an additional non-SAP HANA host or leveraging the HA host for the vCLS VMs is also possible in the case that the SAP HANA HCI systems get connected as a workload domain to a VCF environment..

Summary

By introducing the vSphere Clustering Service (vCLS), VMware is embarking on a journey to remove the vCenter dependency and possible related issues when vCenter server is not available and provides a scalable platform for larger vSphere hosts deployments.

Resources and Additional Readings

• vSphere Cluster Services (vCLS)
• vCLS VM Anti-Affinity Policies
• VMware KB 80472, vSphere Cluster Services (vCLS) in vSphere 7.0 Update 1
• Blog: vSphere 7 Update 1 – vSphere Clustering Service (vCLS)
• SAP HANA on VMware vSphere
• SAP note 2937606 – SAP HANA on VMware vSphere 7.0 in production
• SAP HANA HCI on vSphere SAP note 2718982