This is the second blog of my SAP HANA on vSphere blog series and will provide information on SAP HANA on VMware vSphere compute resource sizing. This blog got updated on November 23rd 2016 to include the Intel Xeon Ex-v4 CPUs (Broadwell), which are now supported.
Sizing SAP HANA database starts with the determination of the database size. This can get done either by using the SAP HANA Quick Sizer application (new system), or by running a specific SAP HANA sizing report (existing systems), as documented in SAP note 1872170 (S4/HANA) and SAP note 2296290 (BW).
Beside determining the database size of HANA that will get operated in RAM, it is also necessary to size the needed resources for the application server stack. This can get done by using he SAP HANA Quick Sizer as this tool will provide the necessary system configuration required for the application part of a SAP HANA based business application.
This blog does not describe how to perform the actual SAP HANA system RAM sizing, for this use above tools and methodes. For details read the sizing SAP HANA Sizing Approaches presentation, which describes the SAP HANA QuickSizer and ABAP SAP HANA sizing reports available for sizing SAP HANA systems. Also a good start is the Sizing Approaches for SAP HANA document. (Attention you may require a SAP SDN account to access the SAP HANA sizing documents).
CPU and RAM Sizing SAP HANA on VMware vSphere
Sizing SAP HANA is different from sizing a classical SAP application. Instead focusing on CPU resources, for SAP HANA memory is the focus and how many CPU resources are required to support a specific amount of RAM. Beside this, we have to consider the different CPU and RAM maximums of vSphere 5.5 and 6, as it defines the maximal size of a virtual SAP HANA VM. To make it even more complex, we have to take into account which workload (SoH or BWoH) will run on the selected SAP HANA system, which CPU and which HANA SPS version will get used, as different CPU socket to RAM ratios exist for all of these combinations.
In the soon to get published new “Architecture Guidelines and Best Practices for Deployments of SAP® HANA® on VMware vSphere®” document, which will be available on www.vmware.com/go/sap-hana, I have detailed sizing formulas specified, which I have defined jointly with SAP. The formula respects hyperthreading with a performance gain of 15% and the used CPU type (amount of available cores), virtualization memory costs and can get used also to calculate the needed vCPUs to support a certain amount of HANA RAM.
Instead of using these sizing formulas, I will explain a simplified way how to size production level ready SAP HANA VMs that respect the SAP defined CPU socket / RAM ratios.
Since we want to leverage hyperthreading, it is required to enable it on the host and that the VMs get configured with the parameter Numa.PreferHT=1. Using this parameter ensures NUMA node locality and forces the ESXi scheduler to use hyperthreads instead of potentially idle cores on other NUMA nodes.
More detailed explanations will be available in the referred VMware SAP HANA guide. As well as storage and network sizing and configuration information.
VMware vSphere SAP HANA relevant Sizing figures
Before we start to size SAP HANA VMs, we have to discuss the current vSphere compute resource maximal sizes and define the current sizing limitation for memory. This limitation is important to understand when sizing a BWoH system on vSphere, as the CPU VM limitation of vSphere 6 of 128 vCPUs will not allow to utilize the theoretical maximum of 4080 GB per VM. Reason is that SAP has defined specific CPU socket to RAM ratios, which are more demanding for an analytical workload like BW. For OLTP like applications like SoH this ratio is higher and here we could size larger memory sizes as even vSphere supports (see below table)!
The table enumerates the current maximal sizing figures for virtualized SAP HANA systems on vSphere. Please note that these figures represent the theoretical sizing maximal sizes and should get aligned after some time to the real live SAP HANA configuration, as you may have less CPU resource need and would therefore be able to increase the memory beyond this theoretical sizing figures. If more VMs get installed on a server or when a single VM will consume the complete available RAM, then also RAM for vSphere has to get reserved. We calculate 1 percent up to 3 percent for vSphere. This depends on the actual VM configuration.
| CPU | HANA Workload | HANA SPS Version | vSphere Version | Maximal “theoretical sizing” RAM Size1 | Theoretical maximal VM size and configuration2 |
| Intel “Brodwell” Xeon E7-v4 | SoH | SPS >= 12 | 6.0 | 4748.99 GB | 128 vCPUs, 4080 GB vRAM |
| BWoH | 3165,99 GB | 128 vCPUs, 3166 GB vRAM | |||
| Intel “Haswell” Xeon E7-v3 | SoH | SPS <= 10 | 5.5 | 2374.49 GB | 64 vCPUs, 1024 GB vRAM |
| SPS >= 11 | 6.0 | 4748.99 GB | 128 vCPUs, 4080 GB vRAM | ||
| BWoH | SPS <= 10 | 5.5 | 1582,99 GB | 64 vCPUs, 1024 GB vRAM | |
| SPS >= 11 | 6.0 | 3165,99 GB | 128 vCPUs, 3166 GB vRAM | ||
| Intel “Ivy-Bridge” Xeon E7-v2 | SoH | SPS <= 10 | 5.5 | 2374.49 GB | 64 vCPUs, 1024 GB vRAM |
| SPS >= 11 | 6.0 | 4748.99 GB | 120 vCPUs, 4080 GB vRAM | ||
| BWoH | SPS <= 10 | 5.5 | 1582,99 GB | 64 vCPUs, 1024 GB vRAM | |
| SPS >= 11 | 6.0 | 3165,99 GB | 120 vCPUs, 2048 GB vRAM |
1Theoretical sizing maximal figure for single large SAP HANA VMs when applying the SAP HANA core to ram ratio. RAM configuration may get modified after initial deployment and real CPU utilization based on SAP early watch reports and SAP approval.
2The maximal VM CPU and RAM size depends beside the vSphere limits also and the current certification status on the underlying HW.
The figures in above table are relevant when for instance an 8-socket server should get used for a maximal sized virtualized SAP HANA BW system. The RAM sizing limitation should get seen as the first start to size the VM and the RAM may get aligned, after consulting SAP HANA support and sizing teams, to the actual server usage and CPU utilization.
Assumption: VMs for two SAP ERP and one existing BW system needs to get sized. The SAP sizing reports provide following RAM sizes including data growth as defined in the report the next years.
- ERP SID1 – 444 GIB
- ERP SID2 – 165.2 GIB
- BI – SID3 – 1009 GIB
The total RAM needed would around 1620 GIB + the RAM needed for vSphere (+1.5% = total 1644 GIB). Since SoH and BWoH should get consolidated on the same server, the selected server has to be certified or supported to run BWoH. The RAM requirements (512 GB per socket max.) of this SAP HANA BW application type will be the leading factor for this configuration.
SAP HANA CPU socket to RAM Ratio
The CPU socket to RAM ratio depends on the used CPU generation, the HANA SPS level and the SAP HANA workload type that will get deployed. This ratio gets defined by SAP and must get used for the initial sizing of a SAP system.
Finding out the CPU socket to RAM Ratio is relatively easy as it just dividing the maximal RAM supported in a SAP HANA certified server for BWoH or SoH and a specific SPS level.
The following table provides some examples on Core to RAM ratios and the average SAPS figures of supported HW configurations.
| CPU | Average SAPS per CPU / Socket | Average SAPS per CPU Core | HANA Workload | HANA SPS Version | CPU Socket to RAM ratio |
| Intel Xeon EX E7-v4 (Broadwell) | SoH | SPS <= 11
SPS >= 12 |
<=768 GB / socket
<=1024 GB / socket |
||
| 56575 | 2357 | BWoH | SPS >= 11 | <=512 GB / socket | |
| Intel Xeon EX E7-v3
(Haswell) |
SoH | SPS >= 11 | <=768 GB / socket | ||
| 41312 | 2295 | BWoH | SPS <= 10 | <=384 GB / socket | |
| SPS >= 11 | <=512 GB / socket | ||||
| Intel Xeon EX E7-v2
(Ivy-Bridge) |
SoH | SPS >= 10 | <=768 GB / socket | ||
| 33894 | 2260 | BWoH | SPS >= 10 | <=256 GB / socket | |
| Intel Xeon EP E5 26xx v4 (Broadwell)
maximal 2 socket supported |
SoH | SPS <= 11
SPS >= 12 |
<=768 GB / socket
<=1024 GB / socket |
||
| 57025 | 2592 | BWoH | SPS >= 10 | <=384 GB / socket |
With above CPU Socket to RAM ratios we can now calculate how many sockets a server needs to have and how many NUMA nodes / CPU sockets a VM will have to get assigned with.
The SAPS figures in the table are based on published 2-tier SD benchmarks of SAP enhancement package 5 for SAP ERP 6.0 on Sybase DB and Linux OS configurations. The Socket to RAM rations are based on the SAP HANA certified appliance configurations.
Which server needs to get purchased?
Total needed RAM including vSphere RAM need divided by the BWoH CPU Socket to RAM ratio, as one of the HANA VMs will be a BW HANA VM. –> 1644 GB RAM / 512 GB RAM = 3.21.
This means a 4 socket BWoH HANA server with a total of 2 TB RAM installed will be enough to support the planned HANA VMs. Attention, the 512 GB RAM per socket are only available with a Haswell server and SPS11. If SPS11 can not get used, then a different Socket to RAM ratio will need to get applied.
Select now a certified server configuration listed on the Certified SAP HANA Hardware Directory can get selected. We also know now; that vSphere 5.5 would be enough to support this configuration.
How does the SAP HANA VM configuration look like?
Same calculation as before, we divide the sized SAP HANA RAM by the RAM installed in a CPU socket. ->
- ERP SID1 – 444 GIB / 512 GB RAM = 0.87 x RAM of CPU socket
- ERP SID2 – 165.2 GIB / 512 GB RAM = 032 x RAM of CPU socket
- BI – SID3 – 1009 GIB / 512 GB RAM = 1.97 x RAM of CPU socket
Since we have production level SAP HANA VMs, we have to round-up the calculated CPU sockets and will assign all available CPU and RAM resources to the VMs that are available in the assigned sockets, minus the RAM needed for vSphere.
- ERP SID1 – VM = 500 GB RAM and 36 vCPUs
- ERP SID2 – VM = 500 GB RAM and 36 vCPUs
- BI – SID3 – VM = 1024 GB RAM and 64 vCPUs (CPU limit)
Below figure shows the VMs running on a single SAP HANA supported 4 socket, 144 thread, 2 TB RAM HANA server. When deployed as appliance systems, then three physical Appliance server systems have to get deployed for production SAP HANA instances instead of only own virtualized server system as shown, when the customer does not want to leverage the multi-tenancy SAP HANA feature.
When configured for non-prod VMs then the calculated, smaller, VM configuration as selected for the production environment would be possible. Here is to mention that the smallest vCPU count of a SAP HANA VM is 10.
Example for non-production SAP HANA VM when using the formulas published in the the architecture guide. For details please refer to this document once available.
- ERP SID1 – VM – 18 vCPUs and 444 GIB vRAM
- ERP SID2 – VM – 10 vCPUs and 165 GIB vRAM
- BI – SID3 – VM – 42 vCPUs and 1009 GIB vRAM
- Free resources: 74 CPU threads and 430 GB RAM for other VM(s)
Sizing the non-prod VMs with the calculated figures, would allow to use the non-utilized 74 CPU threads and 430 GB RAM (-1.5% = 420 GB) for other workloads, like to other SAP HANA VMs or to reserve these free resources to provide VMware HA resources for potentially failing VMs of other host systems that now may get restarted on this host. Below example shows how use some of the free resources by sharing NUMA node 3 and 4 with other non-prod HANA VMs.
Conclusion
The examples show a vSphere 5.5 based server configuration, but the information published above are also valid for calculating vSphere 6 based VMs. The only difference is that the CPU limitation is not 64, but 128 vCPUs and instead of 1 TB vRAM, it would be 4 TB vRAM. It is also important to note that not only the vSphere VM sizes are defining configuration limitations, but also the SAP HANA supported hardware configurations and SAP HANA SPS levels! Over the time SAP may change the CPU to RAM ratios to the positive or, if new developed SAP HANA native applications require more CPU resources, to the negative, as more CPU power maybe required to process the configured RAM of these possible new applications.
More details in the new SAP HANA architecture guide. As SAP writes in its document Sizing Approaches for SAP HANA about SAP HANA sizing:
“Sizing SAP HANA as a database can be quite simple, actually. If you already have a NetWeaver-based system you can execute a sizing report which gives you a memory value and based on this you can choose a fitting hardware from the catalog of certified hardware, which will be delivered as an appliance and per se will have the required capacity for the different components (CPU, memory, disk). The same applies if you are new to SAP. There is a sizing tool called Quick Sizer which helps you to initially calculate sizing requirements for SAP HANA based on your inputs of transactional data.“, page 4
