Recently, we have published a whitepaper about how ESX server manages the host memory resource. This paper not only presents the basic memory resource management concepts but also shows experiment results explaining the performance impact of three different memory reclamation techniques:
Page sharing, ballooning, and host swapping used in ESX sever. The experiment results show that:
1) Page sharing introduces negligible performance overhead;
2) Compared to host swapping, ballooning will cause much smaller performance degradation when reclaiming memory. In some cases, ballooning even brings zero performance overhead.
The following is the brief summary of the paper.
In general, ESX server uses high-level resource management policies to compute a target memory allocation for every virtual machine based on the current system load and parameter settings for the virtual machine (shares, reservation, and limit, etc). The computed target allocation is used to guide the dynamic adjustment of the memory allocation for each virtual machine. In the cases where host memory is overcommitted, the target allocations are still achieved by invoking several lower-level memory reclamation techniques to reclaim memory from virtual machines.
In this paper, we start from introducing the basic memory virtualization concepts. Then, we discuss the reason why supporting memory overcommitment is necessary in ESX server. Three memory reclamation techniques are currently used in ESX server: Transparent Page Sharing (TPS), Ballooning and Host Swapping. We illustrate the mechanism of these three techniques and analysis the Pros and Cons of each technique from performance perspective. In addition, we present how ESX memory scheduler uses a share-based allocation algorithm to allocate memory for multiple Virtual machines when host memory is overcommitted.
Beyond the technique discussion, we conduct experiments to help user understand how individual memory reclamation techniques impact the performance of various applications. In these experiments, we choose to use SPECjbb, Kernel Compile, Swingbench and Exchange benchmarks to evaluate different techniques.
Finally, based on the memory management concepts and performance evaluation results, we present some best practices for host and guest memory usage.
For more details, please read the full paper from here.