Does XenApp (or VDI) on vSphere 5.x Still Require a HIMP Adjustment?
Short Answer: No – This adjustment is only required under vSphere 4.x.
Background:
Some time ago, a 3rd party project compared hosting platforms for XenApp and found vSphere 4.x to be deficient. After some analysis with the 3rd party, the following guidance was generated:
ESX provides fine-grained CPU scheduling that, among other things, enforces resource settings like CPU reservations, limits, and shares. Based on those settings, the scheduler determines if a VM has fallen behind in the amount of CPU time it should have received. Should the VM indeed be behind, attempts are made to help it catch up, usually by scheduling it more frequently.
If the processor has hyper-threading, however, scheduling more frequently does not guarantee that the VM will catch up. This is because the amount of CPU resources received by the vCPU is affected by the activity on the other logical processor on the same physical core. To guarantee the vCPU that is behind can catch up, ESX will sometimes not schedule VMs on the other logical processor, effectively leaving it idle.
In the case of the Intel Xeon 5500 series (and other modern hyper-threaded) processors, not scheduling on a logical processor may have a measurable impact on the overall throughput of the system. As a result, in systems that:
have more than 50% CPU utilization, and
are very close to exactly committed (number of vCPUs = number of pCPUs +/- 25%), and
and have particular kinds of bursty CPU usage patterns
we have observed a throughput drop of up to 15% when this fairness algorithm takes effect.
So altering the default HaltingIdleMsecPenalty (aka HIMP) became a practice when those ESXi instance were hosting XenApp.
You notice a small entry on the above KB article that now says:
Note: This issue does not affect ESXi 5.0.
So the question becomes – why? What’s changed under ESXi 5.x?
Two major changes (thanks to Seongboem in our Perf-Eng team for this explanation):
The VMware scheduler now detects the situation where utilizing both HT threads is harmful to fairness and in response lets one HT thread idle. This is good for fairness but bad for CPU utilization. As such, the detection mechanism has becomes more accurate in vSphere 5.x, especially for workloads like XenApp and VDI.
The charging model has changed in vSphere 5.x such that utilizing both HT threads is favored in terms of fairness. This reflects recent HT improvement where utilizing both HT shows meaningful benefit. Older generations of HT architecture showed little or no improvement for industry standard workloads but Nehalem and later showed measurable benefit.
So while our solution team’s best practice guide is not yet updated to reflect this change for vSphere 5.x, all the other practices and considerations should still be adhered to.
