This is the third part in a series of blog posts on Exchange 2010 performance on vSphere 4. In the first post, tests were done to show the scale-up performance of a single Mailbox Server VM hosting up to 8000 users. The second post was about how increasing the amount of RAM would reduce the number of IOPS, resulting in better performance. This article is about scale-out performance with multiple Exchange 2010 Mailbox Server VMs. For a variety of reasons, it often makes sense to have more than one Mailbox Server in an Exchange environment. Testing was conducted in VMware labs to determine the performance impact of spreading the same number of users across multiple VMs as opposed to running them in a single VM. The results showed that great performance was maintained with only a slight increase in latency when scaling out with either of the 4000 or 8000 user scenarios.
Configuration
The same configuration was used as in the previous two blogs. A Dell PowerEdge R710 with dual quad-core Xeon X5570 processors and 96GB of RAM was installed with a development version of vSphere 4 (build 235768). In the previous tests, there were three Exchange VMs running on the vSphere server: Mailbox, CAS, and Hub. For these scale-out tests, the number of Mailbox Server VMs was increased to 4 and the CAS and Hub Server VMs were moved to another vSphere server. This resulted in only four Mailbox Server VMs running on the vSphere server being tested.
Storage for the VMs was increased to provide additional space for the new Mailbox Server VMs. The number of data LUNs was doubled, resulting in four 9-disk RAID 5 LUNs with each VM having virtual disks on all of the LUNs.
Microsoft Exchange Load Generator 2010 Beta was used to simulate the users for the scale-out tests. The Online Outlook 2007 Very Heavy profile with 100 MB mailboxes was used. The option to have all LoadGen Users pre-logon was set, and the rest of the settings for LoadGen 2010 beta were left at default values.
Testing
In order to test the scale-out performance of Exchange 2010 on vSphere, the same number of users were run on a single Mailbox Server VM, two Mailbox Server VMs, and four Mailbox Server VMs. The total amount of memory and vCPUs was kept constant and the users were evenly divided across the VMs. The charts below show the results for 4000 and 8000 user tests across one, two, and four VMs.
The performance in the 4000 user tests shows a rise of only 30ms in the 95th percentile SendMail response time between a single 4-vCPU VM and four 1-vCPU VMs. The 8000 user tests show an increase of approximately 140ms in the same metric when comparing the single 8-vCPU VM with four 2-vCPU VMs. Even though this is a significant percent increase, the absolute increase is still relatively small in comparison to the 1 second threshold which is where users will begin to perceive a difference in performance.
The reason for the increase in response time is due to an increase in IOPS, which puts additional load on the storage array and disk latency increases slightly. IOPS increases as VMs are added, even though the number of users stays the same, because using four smaller caches is less efficient than one large cache. This shows that Exchange 2010 does a good job of managing large caches when the RAM is available.
Another way to look at the data is to compare the same size VMs being used to support an increasing number of users. A single 4-vCPU VM supporting 4000 users had a 95th percentile SendMail latency of 234ms and two 4-vCPU VMs supporting 8000 users had 314ms. Doubling the workload on the server with an additional Mailbox Server VM resulted in an increase of only 80ms.
Conclusion
Performance of Exchange 2010 in the scale-out tests on vSphere was great with response times well below 1 second. vSphere allows Exchange 2010 architects to have the flexibility to use multiple mailbox server VMs to create and manage their Exchange 2010 environment as their needs dictate and get great performance. At the same time testing indicated there is the opportunity to reduce IOPS by using fewer mailbox server VMs with larger RAM sizes instead of using a greater number of mailbox server VMs with less RAM. This aspect of Exchange 2010 performance is due to Exchange 2010’s ability to efficiently manage the larger memory size.