We have published a whitepaper about how ESX server 4.1 manages the host memory resource. This paper not only presents the basic memory resource management concepts but also shows experiment results explaining the performance impact for four different memory reclamation techniques: Page sharing, ballooning, memory compression and host swapping used in ESX server 4.1. The experiment results show that:
1) Page sharing introduces negligible performance overhead;
2) Compared to host swapping, ballooning will cause much smaller performance degradation when reclaiming memory. In some cases, ballooning even brings zero performance overhead.
3) Memory compression can significantly reduce the amount of the swapped out pages and hence greatly improve the overall performance in high memory overcommitment scenario.
The following is the brief summary of the paper.
In this paper, we start from introducing the basic memory virtualization concepts. Then, we discuss the reason why supporting memory overcommitment is necessary in ESX server. Four memory reclamation techniques are currently used in ESX server: Transparent Page Sharing (TPS), Ballooning, Memory Compression and Host Swapping. We illustrate the mechanism of these techniques and analysis the Pros and Cons of each technique from performance perspective. In addition, we present how ESX memory scheduler uses a share-based allocation algorithm to allocate memory for multiple Virtual machines when host memory is overcommitted.
Beyond the technique discussion, we conduct experiments to help user understand how individual memory reclamation techniques impact the performance of various applications. In these experiments, we choose to use SPECjbb, Kernel Compile, Swingbench, SharePoint and Exchange benchmarks to evaluate different techniques.
Finally, based on the memory management concepts and performance evaluation results, we present some best practices for host and guest memory usage.
The paper can be found in
http://www.vmware.com/files/pdf/techpaper/vsp_41_perf_memory_mgmt.pdf
Note that this paper is written based on ESX4.0 memory management paper. Besides the new content introduced in ESX4.1, e.g., memory compression, quite a few places have been updated to represent the state of the art of ESX memory management.