News

Lab Experiment: Hypervisors

Is there really a performance difference between hypervisors from VMware, Microsoft and Citrix? We put them under a microscope and stress-tested them for days to find out the answer.

• By Rick Vanover
• April 27, 2009

When it comes to hypervisors, there are many choices now. That's good news for consumers, who were much more limited even one year ago. Still, when administrators go shopping, they look first at hypervisors from VMware Inc. (ESX), Microsoft (Hyper-V) and Citrix Systems Inc. (XenServer). Given that situation, it makes sense to do a side-by-side, exact comparison under identical conditions to determine which of these is the best-performing product. After all, the hypervisor is still the engine that drives virtualization, even with all the management and third-party products built on top.

To that end, RCP's sister publication, Virtualization Review, recently put the three challengers through a battery of tests to find out how they performed a variety of virtualization functions.

Test Objectives
All the hypervisors offer essentially the same base functionality. In this series of tests, the objective was to put the same workloads on each one and see how they stacked up. The types of workloads tested varied to simulate a typical environment in which some virtual machines (VMs) are stressed, and some aren't. Each platform was subjected to the same test plan parameters to give a fair accounting of its performance.

Microsoft Hyper-V
Hyper-V performed quite differently from expectations. It's been a focus of Microsoft development efforts, and it shows: Overall, Hyper-V did well in this comparison and proved itself a worthy product. What's even more impressive is that it's the youngest of these hypervisors, by a wide margin. For a version 1 offering, Hyper-V stands out in Microsoft's product line.

In our tests, Hyper-V did well in all categories -- in fact, it outperformed ESX in most. It's a real, viable competitor in the hypervisor space.

Citrix XenServer
XenServer may not have ESX's reputation or Microsoft's marketing muscle, but the hypervisor definitely held its own in this comparison. In fact, it did better than that: XenServer performed best over the largest range of categories in our tests.

XenServer's test results are impressive, but are they enough to justify a replacement of your current hypervisor? For environments with virtualized systems that have a large number of CPUs and memory-intensive workloads, it may be a good choice. The caution is that those high I/O workloads flirt with not being good virtualization candidates, so most admins would instinctively place these workloads on physical systems. Make no mistake, however: XenServer did extremely well, posting excellent performance numbers.

VMware ESX 3.5
Because VMware is the virtualization market leader, one tacit test objective was to determine if ESX is the right hypervisor in an environment with a large number of VMs doing less work, which is typical for many virtualization environments.

For the first two tests of heavy workloads, VMware underperformed both XenServer and Hyper-V. For the lighter workloads on the third test, the results were almost indistinguishable across the platforms-but ESX had the best results in three of the four categories.

Lessons Learned
After doing these comparisons of ESX to Hyper-V and XenServer, it's clear that at the hypervisor level, ESX is optimized for a large number of less-intensive workload VMs. For intensive workloads that may not be optimized for memory-overcommit applications, Hyper-V and XenServer should definitely be considered-even if that means adding another hypervisor to the data center.

All three hypervisors are legitimate virtualization platforms, and that no one company has a monopoly on virtualization any longer.