SAP Business Suite 4 SAP HANA (or SAP S/4HANA) is the SAP Business Suite that is built on SAP’s in memory columnar database platform SAP HANA. SAP HANA®, the in-memory real-time platform, was initially introduced as a physical appliance and has steadily evolved to include support for virtualization with VMware vSphere® and SAP HANA tailored data center integration (TDI). Virtualized SAP HANA is now supported in scale-up and scale-out configurations in VMware® environments. Running SAP HANA on vSphere offers customers agility, resource optimization, and ease of provisioning. This solution enables SAP customers to provision instances of SAP HANA more quickly and effectively by using vSphere virtual machines (VM). Using the SAP HANA platform with the vSphere virtualization infrastructure constitutes an optimized environment for achieving a unique, cost-effective solution. VMware capabilities such as VMware vSphere vMotion®, VMware vSphere Distributed Resource Scheduler™ (vSphere DRS), and VMware vSphere High Availability (vSphere HA) are inherent components of the virtualized SAP HANA platform.
The need exists for a comprehensive, “end-to-end” solution that describes the implementation of a virtual SAP HANA deployment. VMware Solutions Labs was chosen to first develop a robust validated end to end solution. This is the first solution that takes the VMware Validated Design concept and then uses its components to run an Enterprise application like SAP on top of it. This prescriptive approach called Application Workload Guidance Design applies the VMware Validated Design (VVD) to SAP S/4HANA on the vSphere platform.
Application Workload Guidance (AWG):
Application Workload Guidance provide a comprehensive and extensively tested set of blueprints for building and operating an SDDC. Each design is customized for a chosen functionality and all applications running in that environment. Each holistic and standardized data center–level design spans compute, storage, networking, and management, providing a proven framework for how to deploy, configure, and operate an SDDC-based private cloud in support of a wide range of use cases.
Every VMware AWG solution includes the following:
- A solution overview that details design objectives and software components
- An example design that is based on a customer scenario
AWG for SAP S4/HANA
This Application Workload Guidance focuses on virtualized SAP HANA with tailored data center integration (TDI). Best practices for virtualizing SAP HANA for scale-up and scale-out configurations are leveraged in this design. The design is sized for various use cases, is tested on SAP-certified partner hardware, and is then validated. This reference architecture provides information specific to SAP HANA and leverages existing VVDs for general-purpose guidelines.
The physical architecture includes Dell R630 and R730 PowerEdge Servers with up to 1TB memory, a Pure Storage TDI array, and Brocade SAN and network components. The components are exemplary but are interchangeable. For example, the Pure Storage array is used for the testing, but any flash storage array can be substituted. The functional testing is not dependent on the storage type, so a VMware vSAN architecture can also be utilized. This work involves some performance testing, but it validates earlier vSphere 6.5 certification testing.
Use Cases
This design is targeted for the following use cases:
- Rapid provisioning to increase timelines for testing, training, and SAP upgrades; clones SAP systems quickly and easily
- Development and Test refresh from production
- Enhanced application availability
- High out-of-the-box availability, without complex configuration and setup, to protect against VMware ESXi™ server failure; no downtime due to ESXi server maintenance
- Continuous utilization monitoring across resource pools and intelligent allocation of available resources among VMs, based on predefined rules that reflect customer business needs and changing priorities
- VMware vSphere Fault Tolerance for SAP Central Services
- Server consolidation and data center energy cost reduction; runs SAP in VMs consolidated onto fewer physical servers that use less energy overall
- Unified performance monitoring of the virtualized SAP environment; coordinates performance reporting and analytics across the solution infrastructure stack, including the SAP and database tier, the guest operating system (OS), the hypervisor layer, and storage
- Enhanced security that leverages VMware NSX; enables restricted communication between application components
Workload Design and Sizing
The SAP HANA AWG is based on a greenfield implementation using the SAP Quick Sizer tool for SAP HANA. Quick Sizer is a Web-based tool designed to facilitate a faster and easier way to size the SAP Business Suite. It has been developed by SAP in close cooperation with all platform partners and is free of cost. For more details, see SAP Quick Sizer (requires SAP login).
The Quick Sizer tool is used in the same way for virtualized deployments as for physical environments. Based on the input business requirements, the output generated by the SAP HANA Quick Sizer tool for the VVD is shown in Table below.
Table 1: Quick Sizer based SAPS by module
Physical Design
This section details the ESXi hosts proposed for the vSphere infrastructure design. The hosts chosen are based on the customers’ preferred hardware vendors and the standard models in use for database and application needs.
Database Host Design Specifications
The database hosts require high-memory systems with many cores to meet the requirements of the SAP S/4HANA database tier. The Dell R730 PowerEdge Server with two processors with 44 cores and 1TB RAM is a good building block for the database tier. Each node has four 10GBps network adapters for networking and two 16GBps Fibre Channel ports for high-performance SAN access.
Application Host Design Specifications:
The application hosts do not require significant amounts of lot memory. Lower-priced hosts can be used to meet the requirements of the SAP S/4HANA application tier. The Dell R630 PowerEdge Server with two processors with 20 cores and 256GB RAM is the customer standard for application tier and is a good building block for this tier. Each node has four 10GBps network adapters for networking and two 16GBps Fibre Channel ports for high-performance SAN access.
The configuration and assembly process for each system is standardized, with all components installed the same on each host. Standardizing not only the model but also the physical configuration of the ESXi hosts is critical to providing a manageable and supportable infrastructure, because it eliminates variability. Consistent PCI card slot location, especially for network controllers, is essential for accurate alignment of physical-to-virtual I/O resources. The servers are described in Table 10. Both servers are on the VMware Hardware Compatibility List.
Table 2: Target Server SAPS Rating
The preceding SPECint ratings are available at https://www.spec.org/cpu2006/results/res2016q2/cpu2006-20160321-39711.html and https://www.spec.org/cpu2006/results/res2016q2/cpu2006-20160307-39256.html.
The SAP benchmark certification is available at http://download.sap.com/download.epd?context=40E2D9D5E00EEF7CEE07089B1A1E0B4D782C09749CEA51D4E8F4C50F3C1581DB.
The Dell R730 PowerEdge Server is used to run SAP HANA VMs and is an entry-level server supported by SAP to run SAP HANA. See http://global.sap.com/community/ebook/2014-09-02-hana-hardware/enen/entry-level-systems.html#categories=Dell&recordid=1810.
When sizing SAP on VMware with hyper-threaded Intel servers, there are two options: with one vCPU scheduled per core or with two vCPUs scheduled per core. In the latter case, the hypervisor leverages hyper-threading, which provides an extra boost in transaction throughput. This concept is explained in section 3.2.4 of http://www.vmware.com/files/pdf/business-critical-apps/sap-on-vmware-best-practices.pdf.
In this VVD design, we size based on the following requirements and assumptions:
- One vCPU per core, because it generates VM sizes with fewer vCPUs
- Hyper-threading performance benefit of 15 percent as documented in http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/whitepaper/sap_hana_on_vmware_vsphere_best_practices_guide-white-paper.pdf.
- CPU utilization of 65 percent
- SAPS server ratings based on logical CPUs being utilized.
Based on the preceding, Table 3 shows the SAPS per vCPU calculated for the two Dell servers
Table 3: Calculate SAPS per vCPU for Each Target Server
NOTE: The terminology used here in regard to the ESXi CPU scheduler, which impacts SAPS throughput, assumes the following:
- A vCPU and a world are interchangeably used to refer to schedulable CPU context, corresponding to a process in conventional OSs.
- On hyper-threaded systems, a physical core has two logical processors; a vCPU, or world, is scheduled on a logical processor.
- A vCPU can be scheduled on a logical processor on a core while the other logical processor of the core is idle. This is referred to as one vCPU scheduled per core.
- Two vCPUs can be scheduled on the two logical processors of the same core. This is referred to as two vCPUs scheduled per core.
- This terminology is consistent with the following guide: The CPU Scheduler in VMware vSphere 5.1.
Application Tier Sizing
Given the preceding guidelines, the following application server VMs were chosen.
| SAP Component | Application Tier VM vCPU | vCPU Oversizing (VM vCPU count Minus the vCPUs Calculated Based on SAPS) | Application Tier Minimum VM Memory (GB)
(Source: Quick Sizer Output) |
| BW/4HA_SRV | 2 x 10-way | 20-13 = 7 | 42/2 = 21 |
| CRM | 7 x 10-way | 70-68 = 2 | 318/7 = 46 |
| MDG | 2 x 2-way | 4-2 = 2 | 10/2 = 5 |
| S/4 | 4 x 10-way | 40-34 = 6 | 133/4 = 34 |
| SRM | 10 x 10-way | 100-92 = 8 | 327/10 = 33 |
| TM | 2 x 4-way | 8-6 = 2 | 25/2 = 13 |
| Total Application Server vCPUs | 242 | ||
| Total vCPUs for ASCS + vSphere FT Shadow VMs | 24 |
Table 4. Determine Final Application Server VM Sizes for Each SAP Component
The VMs listed in Table 4 are deployed on a 13-host ESXi cluster with a total of 260 cores; that is 13 x 20. The total vCPU count from Table 13 is 242 + 24 = 266. This indicates a vCPU overcommit situation; that is, the total number of vCPUs is greater than the total number of cores. This is not a concern, however, because there is an oversizing of vCPUs equal to 27 vCPUs, as is shown in the third column of the table. In the event of a single host failure, after all the impacted VMs have restarted on the remaining 12 hosts, there will be an even greater vCPU overcommitment. This will not create a serious CPU bottleneck for the following reasons:
There is a total vCPU oversizing due to the rounding up of the VM vCPUs to a standardized size; that is, 27 vCPUs. Sizing has been calculated at 65-percent CPU utilization, so there is headroom.
The hyper-threading benefit adds an additional buffer.
In practice, not all workloads across the VMs peak at the same time.
Given the preceding specifications, after commencement of production workloads, real-time monitoring of the workload will most likely generate additional consolidation opportunities.
SAP HANA Database Tier Sizing
Table 14 shows the estimated SAP HANA VM sizes based on the Quick Sizer business requirements.
| SAP Component | Database Memory Requirement (GB) | Database Tier SAPS Requirement | # of vCPUs Required | Proposed VM Size (target host: 2 sockets, 44 cores, 1 TB) |
| BW/4HA_SRV | 607 | n/a | 2 sockets | 650GB, 44 vCPU |
| CRM | 197 | 42,000 | 31 | 250GB, 44 vCPU |
| MDG | 227 | 10,000 | 8 | 250GB, 11 vCPU |
| S/4 | 774 | 17,000 | 13 | 800GB, 44 vCPU |
| SRM | 197 | 21,000 | 16 | 250GB, 22 vCPU |
| TM | 197 | 10,000 | 8 | 250GB, 11 vCPU |
| Total Memory
2,450 GB |
Total CPU
198 vCPU |
Table 5. Estimated SAP HANA Virtual Machine Size
The vCPUs for the SAP HANA VMs in Table 14 have been rounded up in the far-right column, “Proposed VM Size,” to be in compliance with the guidelines specified in SAP note 2393917, “SAP HANA Virtual Machines on VMware vSphere 6.5 in Production”: “no odd multiples of half sockets like 1.5 socket VMs, 2.5 socket VMs, etc.; minimum VM size of half a socket.”
Data Center Design
In vSphere, a data center is the highest-level logical boundary. It can delineate separate physical sites or locations as well as vSphere infrastructures with completely independent purposes.
Within vSphere data centers, ESXi hosts typically are organized into clusters. Clusters group similar hosts into a logical unit of virtual resources, enabling the following technologies:
- vSphere vMotion
- vSphere HA
- vSphere DRS
- vSphere FT
vSphere Clusters
As part of this logical design, vSphere clusters are created to aggregate hosts. Due to the high cost of the database cluster, an N+1 cluster model is used. Having two additional hosts means 50-percent over-provisioning for high availability and maintenance. If the SLA warrants it, this should be made an N+2 cluster. For the application cluster, an N+2 model is used to enhance redundancy for the large number of nodes in the cluster and to enable availability during maintenance.
| Database Cluster | CPU | Memory |
| Total | 112 | 2,450 |
| Per node | 44 | 1,024 |
| Number of Nodes | 2.55 | 2.39 |
| Cluster Size (N+1) | 4 | 4 |
Table 6: Cluster Size Calculation DB Tier
Cluster Size Calculation: (Application Tier)
| Database Cluster | CPU | Memory |
| Total | 266 | 814 |
| Per node | 20 | 256 |
| Number of Nodes | 13.30 | 3.18 |
| Cluster Size (N+1) | 15 | 5 |
Table 7: Cluster Size Calculation App Tier
| Cluster | Database | Application |
| vSphere cluster size | 4 hosts | 15 hosts |
| Capacity for host failures per cluster | 1 host | 1 host |
| Capacity for maintenance | None | 1 host |
Table 8: Total Number of Hosts and Clusters Required
Now that we have done the cluster design based customer SAP S4/HANA business requirements for compute and memory, we will proceed to designing the storage and networking. Part 2 of this blog series will focus on storage networking and security.
More details from this blog series can be found in this comprehensive Whitepaper. In part 2 we will look at storage, network and security design for the proposed customer environment.
