Home > Blogs > Virtual Reality

Considering XenServer for XenApp? It might be time for a “Virtual Reality Check”.

If you haven't already done so, now might be a good time to surf over to the Project Virtual Reality Check site (registration required).  No, it's not a web page dedicated to proofreading this blog.  It's run by a team of virtualization consultants that have been publishing comparisons of VDI and Terminal Services performance in various hypervisors.  They made a big splash back in January with their first set of reports comparing ESX 3.5, XenServer 5.0 and Hyper-V R1.  Our exclusive ability to overcommit memory made ESX the clear winner in the VDI tests with support for more than twice as many VMs as the others.  On the Terminal Services tests, it was XenServer that came out ahead in the number of user sessions it could handle.  It wasn't good news for VMware, but the story doesn't end there.

Nothing gets us more worked up than finishing second in a product comparison, so our performance team started digging into the Project VRC benchmark to see what might be going on.  Right off the bat, we saw that ESX wasn't configured to use the hardware assist for memory management (AMD RVI) that was present on Project VRC's servers.  ESX 3.5 was the first hypervisor to support hardware assist for memory management and we certainly wanted ESX to look its best.  The Project VRC guys updated their ESX test report in March and the findings showed that ESX narrowed the XenServer lead in Terminal Services user sessions by half once RVI was enabled.  It was an improvement, but we still weren't ready to accept second place.

The Project VRC results prompted us to do some rigorous testing of ESX performance with XenApp — a workload based closely on Terminal Services — and our findings convinced us that we could support more user sessions, so what was going on with the Project VRC tests?

It turns out that the Project VRC team was also taking a second look at their benchmark.  Our performance team (and also the experts at Citrix) collaborated with Project VRC's test architects and all groups came to the conclusion that timing issues were skewing the results.  Use of in-guest timers, one of the classic demons in virtualization benchmarking, was identified as a problem area.  You can see our take on it here.

The Project VRC team dug in and developed a completely new Terminal Services benchmark that reduced use of in-guest timing, when compared to the first version.  The new workload created by Project VRC still depends on in-guest timing, but is a big improvement over the first version and we're grateful for Project VRC's efforts and flexibility.  VMware was also hard at work since the first Project VRC tests improving performance in vSphere.  Our own testing demonstrated a 30% improvement in XenApp throughput between ESX 3.5 and 4.0, so we expected a strong showing in new test runs.

When the first Project VRC comparisons came out, we saw Citrix promote them heavily, especially with customers that were deciding on a virtualization platform for their XenApp servers.  XenApp and Presentation Server customers have been rapidly moving those servers into VMware virtual machines and it's understandable that Citrix wanted some independent test results that might slow down what they saw as defections and keep those XenApp servers on their hypervisor platform.

If you're a XenApp user making a virtualization platform choice you need to read the new Project VRC paper that came out last week NOW.  The paper is titled, "VRC, VSI and Clocks Reviewed".  The results have shifted dramatically since the January findings.  Instead of trailing, ESX 4.0 now has a 4% advantage over XenServer 5.5 in the number of Terminal Services users it can support on a server. 

We attribute the turnaround for vSphere in the latest tests to the Terminal Services/XenApp performance enhancements we made in ESX 4.0 and improvements made by Project VRC to their benchmark.  These new results will come as big relief to XenApp users that felt pressured to virtualize on XenServer after seeing the old Project VRC numbers.  When presented with test findings that seemed to show XenServer could support more users per host, it was harder to choose the proven reliability and richer feature set provided by VMware vSphere.  We certainly hope no customers were lured into making the wrong choice based on test results now known to be flawed and outdated, but those of you yet to make that decision will be happy to know that the latest Project VRC comparisons make the selection of vSphere a no-brainer.

This entry was posted in Uncategorized on by .

About Eric Horschman

Eric Horschman, Product Marketing Director at VMware Eric joined VMware in 1999 where, as product manager, he brought the first x86 server virtualization product to market and helped establish VMware as the leader in enterprise virtual infrastructure. Since then, he’s worked to launch and market a broad set of VMware enterprise platforms. Currently, Eric leads product marketing for Azure VMware Solution as part of the VMware Cloud Infrastructure Business Group.

5 thoughts on “Considering XenServer for XenApp? It might be time for a “Virtual Reality Check”.

  1. Jeremy

    So where exactly is View for vSphere 4? My conspiracy hat is making me think that vSphere was pushed out the door early in order to combat the Citrix free xenserver offering. Why in the world would you release an update for your server without making sure you support your own software. Were in 4th quarter. I need to upgrade but AM STUCK because of view.
    Any idea when we will see any indication of a release?

  2. Dennis Brown

    I know this is a bit late, but I can second problems with XenServer. In April of 2009, we decided we wanted to simplify computer management by moving around 300 PC’s to terminals. To minimize the impact on the amount of servers required, we looked for various benchmarks. The ProjectVRC tests convinced me that XenServer was a good choice to build a virtual farm, better than even bare metal. The project became a nightmare of dead servers, mysterious power-downs and lockups. In December of 2009, I rebuilt all the terminal servers on ESX 3.5 and it has been happy ever since, including a recent 4.0 upgrade. I believe the single greatest problem in this equation was the shared storage; it seems XenServer did not know quite what to do with fibre channel luns and frequently lost track of them or locked them. This was apparently related to a known bug; after the second set of VRC tests, I was gratified in saying that the extra 1000$ / server was worth it for the stability ESX provided with little to no perfromance decrease. So it turns out in the end that if you want one physical box with local storage, XenServer would likely be fine; if you want a real, full-on dynamic virtual terminal cluster, stick with VMWare ESX. There’s a definite reason VMWare is still the biggest and best in the Virtualization Space…..

Comments are closed.