What's the best virtualisation suite?

Over the past few years, virtualisation has become a viable option to thin down your datacentres in physical size, heat and cost.

At its core, there are two major types of virtualisation: Type 1 and Type 2. There are complaints attached to these labels, and there are hybrids that defy them, but by and large they serve their purpose in helping us to categorise virtualisation.

Type 1 is known as a native, or bare metal hypervisor. It requires no operating system to be pre-installed — it is one, just in a very stripped down form. If your intent is virtualisation alone, this is the most efficient type, allowing access to the hardware through the hypervisor.

Type 2 on the other hand, is run on a pre-installed operating system. It is also known as hosted virtualisation, or a virtual machine monitor (VMM). You'd likely run this when you need the host OS's features to have access to your hardware. For example, you may want to run on Solaris to give direct access to creating a RAID-Z array. Or perhaps you need to run on Windows, to give a virtualised operating system access to a hardware device that usually only Windows can see. Or maybe you just want to test out a new OS before doing a roll-out — either way, different virtualisation suits different needs.

In this round-up we'll be looking at Type 1 virtualisation solutions from the market leaders: Microsoft, Citrix and VMware. In the coming weeks we'll look into Type 2 competitors Parallels, VirtualBox, VMware Workstation and a few odd tools that aren't quite virtualisation, but do get binaries running across different OSes.

Bare metal hypervisors

In today's market, VMware still retains dominant share, but others are starting to move in on its lead, including XenServer from Citrix and Hyper-V from Microsoft. Both have received a large amount of press and also support from the industry. Of course, there are other Xen-based solutions such as xVM Server from Sun and Red Hat Linux Enterprise (RHEL) Server, which currently include virtualisation (although RHEL will be moving to KVM later in the year, as Ubuntu did last year). For this review our focus was primarily on VMware ESXi 3.5, XenServer 5, and Hyper-V R2 beta (more on why later).

How we tested

Our objective was to perform a set of (repeatable) tasks on virtual machines running on top of the various hypervisors and measure the performance of these tasks.

The server we used for this testing was kindly provided by Xenon. Featuring a dual-Intel Xeon E5462 processors, 8GB RA, and a 750GB HDD, it runs on a Super Micro X7DWU motherboard and is complemented with ATI ES1000 graphics and an Adaptec 3405 RAID controller.

Xenon's physical server was first tested with a native install of Windows Server 2003 SP1 to establish a performance baseline using the eight cores and 8GB RAM available. These hypervisors were configured with four virtual CPUs and 3.5GB of RAM (leaving 1GB free for the hypervisor). This is the maximum configuration possible for all the servers, except for the XenServer, which supports up to eight virtual CPUs. A fresh install of Windows Server 2003 SP1 was then installed on the configured virtual machine (VM) on each of the hypervisors in turn, and the following tests were run:

• The OS was upgraded from SP1 to SP2, the time taken was recorded
• Sungard was run, the result was recorded
• Cinebench single CPU was run, result recorded
• Cinebench xCPU was run and result recorded
• A 600MB file was copied up and down from a network share
• A 4GB file was copied up and down from the network share

VMware Server 2, a Type 2 virtualisation product, was then tested as another control to see how hypervisors performed.

Testing procedure

The testing procedure was divided up into the following categories:

• Product features and function — maximum configuration sizes
• Product accessibility — how easy it is to download and install
• Manageability — how easy it is to create and manage virtual machines

Product features and functions
The following table summarises each of the products' configuration limits:

Product accessibility

Each product was assessed based on how easy it is to obtain and install.

Citrix XenServer
Downloading XenServer from Citrix is simple. After navigating to the XenServer products page, it is a simple matter of filling in some details before being directed to the XenServer download page. Two files are required, an INSTALL and LINUX ISO, totalling a little over a gigabyte.

Of all the hypervisor installations, XenServer was the most simple. It merely required booting the server from the install CD and following its prompts to enter details such as its IP address and server name. It should be very familiar to anyone who has installed Red Hat since the Domain 0 kernel is based on CENTOS. The install CD is then also used in a Windows machine to install the XenCenter management utility (not web-based). Within 10 minutes you are ready to start creating virtual machines.

VMware ESXi 3.5
VMware ESXi 3.5 is also extremely simple to download from the VMware website. An install CD was created and the installation we undertook was straightforward. Screens are a little more polished than XenServer, reflecting this product's maturity. A web-based management console is then installed on a windows machine from a separate download, the overall process takes a little bit longer than XenServer but was still easy for a novice to get through.

Microsoft Hyper-V Server 2008
Hyper-V Beta R2 release was used for this test since the release version was unable to recognise the latest Intel PRO Gigabit on-board NICs in the server, reporting "No Network on Boot". Unlike the previous two hypervisors that use a Linux-based kernel, Hyper-V is wrapped within an appliance version of Windows Server 2008 (in fact, Windows 7 Server for the Beta R2).

The download is easy, and installation of the Hypervisor onto the server is also fairly straightforward. Configuration is done via a terminal screen, which launches once you're logged in. We found this a little tedious with lots of options to understand and make selections from.

The Windows Hyper-V management console is an MMC snap-in, which needs to be installed on either a Windows 2008 Server or a Windows Vista machine running the administration tools (x86 or x64, along with RSAT) — XP is not supported. It took us some time to understand what was required to get this working. Updates had to be applied, and the TestLab staff were unable to get the virtual machine manager running on a Vista machine, which requires a significant amount of voodoo to get working, and is nothing short of painful.

We ended up installing Windows 2008 server in order to manage the Hyper-V server. Management of the Hyper-V server is not possible until the firewall is turned off, or several services cleared (either via remote MMC snap-in once you've set up remote administration properly, or the appropriate netsh command at the command line). Once we were finally able to manage the Hyper-V server, it was possible to start creating virtual machines. Windows Server 2003 SP1 requires an update to SP2 followed by another (provided) update before Integration Services can be installed (Hyper-V's equivalent to Xen and VMtools).

Manageability

Each product was assessed to determine the ease of host server management (using the console provided).

Citrix XenServer
Once again, XenServer has the most features available, but it is only a Windows-based application (as opposed to web-based). A web-based console is available, however, it is not supported. The console makes it easy to create, backup and even copy virtual machines (this is the only console that allowed us to copy a VM). Performance statistics and VM configuration are also easy to view and edit.

Another advantage of the XenCenter console is that it provides all the features needed if you're upgrading your licence key to a full featured edition of XenServer. The same console will allow the live migration of virtual machines, creation of resource pools and set up high availability. There is no need to upgrade your console.

XenServer Management Console (Credit: Enex Testlab)

VMware ESXi 3.5
VMware has a web-based console. This is also quite easy to use and navigate, although it has fewer features than XenCenter. To access its complete capability you'll need to use VMware's vCenter Server; this server is recommended if you wish to move virtual machines or automate high availability, but comes at a cost. It also requires the installation of SQL server express and will not install onto a Vista machine. This, of course, adds significantly to the complexity of the environment because the vCenter Server console then becomes a critical part of your infrastructure.

VMware Management Console (Credit: Enex Testlab)

Microsoft Hyper-V Server 2008
Hyper-V requires the user to install its Server Management tools for Vista, then find the relevant update package on the Microsoft website in order to manage the Hyper-V server. Information available is very sketchy, and the required software is hard to find, let alone understand. As a result, the TestLab staff were unable to get a Vista client working in the time available. We compromised by installing a Windows 2008 server in order to be able to manage the Hyper-V server. Eventually after downloading and installing an update, a connection was made to the Hyper-V server and the creation of virtual machines was possible. Even so, the console disconnected a number of times.

To exit the console screen, it is necessary to enter CTRL+ALT+Left arrow; this was extremely tedious after a server reboot or if we wanted to view in full screen. A VM with no network configured was created. To this, a network was added, but the VM did not detect this new network. To get the network attached to the VM it became necessary to reinstall Windows — although we put this down to the tested Hyper-V software being in beta.

After the ease of setting up XenServer and VMware ESXi, Hyper-V was a task. The Hyper-V console doesn't have any of the nice performance graphs that VMware and XenServer provide, although CPU usage is available. For full features you will need to use System Center Virtual Machine Manager (SCVMM).

Hyper-V Management Screen (Credit: Enex Testlab)

The test results

The following table displays the test results, the first column shows results for the baseline physical server, the last shows the VMware Server 2 host-based virtualisation environment, and all the bare metal hypervisors are in between.

As you can see, all the bare metal hypervisors performed well and the results were quite close, with one exception. XenServer, unlike VMware and Hyper-V, has the ability to support up to eight vCPUs per VM, and as you can see from the Sungard test with eight vCPUs, XenServer performed exceptionally well. It is, however, worth noting that its vCPUs present themselves to the Windows OS as physical CPUs, which means for eight CPUs to be selected you will need Windows Server Enterprise Edition running on your VM.

With the results table in mind, some interesting conclusions can be drawn:

• VMware Server 2 hosted environment only supports up to two vCPUs based on performance results. Bare metal hypervisors outperform host-based hypervisors significantly
• XenServer, with its support for eight-vCPU support, performs very similarly to bare metal, the hypervisor overhead is minimal
• If you look at the Sungard test, XenServer with four vCPUs marginally outperforms the others, followed by Hyper-V then VMware.
• XenServer seems to have an issue with the latest Intel NIC installed in the server, copy-up times were unacceptably long. This was also tested on another server in-house with a different NIC (still Intel) and it worked perfectly.
• Two VMs running concurrently had minimal impact on the other VM for all the tested platforms

Verdict

As can be seen from the results there is little difference between each of the bare metal hypervisors. The host-based version tested is considerably slower on all tests, so choosing a bare metal hypervisor over a host-based makes a lot of sense when hosting a server environment.

The performance difference is slightly in favour of XenServer, followed closely by Hyper-V and then VMware. However, if you wish to, for example, virtualise Citrix XenApp (presentation server) then XenServer would be the bare metal hypervisor of choice.

When choosing a bare metal hypervisor you should first look at the workloads you are planning to run on the virtualised server, what your current environment is using, and what expertise will be available. For those with strong in-house Microsoft skills, Hyper-V will make the most sense and similarly, if your IT department has Linux skills then Red Hat or Xen make more sense.

Forced to choose, we'd have to put VMware in the runner's up position, with Citrix XenServer getting our vote. It is easy to acquire; the Windows console provides good management that will support the user when deciding to migrate to the fully-paid edition of the product, without the need to introduce any other consoles.

Summary

XenServer
Pros:

• Easy to install: single disc to install XenServer on the host, and XenCenter on the Windows console.
• Single console used for the free version right through to platinum editions of the product, no requirement to change or use another console when live migrating or using HA features in the full product, only console that provided a copy VM function.
• Supports up to eight vCPUs making it the clear performer in this configuration with minimal overhead.
• Product support via forums and the XenSource community is readily available.

Cons:

• XenCenter management console is a windows application and not a web console.
• XenServer does not have features such as memory-over-commit and consolidated backup in its full products, although these are coming in future releases.
• Tool support such as those for converting physical to virtual are not as advanced as VMware, but are improving with each release.

Bottom line
Has the most features of any free hypervisor, easiest to install and manage, performance is excellent and it is the only hypervisor to support up to eight vCPUs

VMware
Pros:

• ESXi 3.5 is easy to install and manage from the web console, and is the most polished of the hypervisors.
• Has many advanced features in its full product that highlights its maturity, best upgrade path and roadmap is strong.
• Is the market leader, which translates to good support via forums and the many certified engineers available in the workforce, many tools such as capacity planner are available to assist in the migration from physical to virtual.

Cons:

• Web console limited in terms of managing the virtual machine, requires upgrade to vCenter server for advanced features such as Live Migration and HA.
• Upgrade from free to fully-featured version requires upgrade of console to vCenter Server, adding a greater level of complexity.

Bottom line
It is the market leader, which shows in the maturity of its product, the polish of its console and the vast number of support tools available, but comes at a cost.

Hyper-V
Pros:

• Best integration with Microsoft infrastructure, via MMC snap-in.
• Strong development focus from Microsoft.
• Microsoft training and development, which translates into a large number of individuals trained and certified on the product.

Cons:

• Released version does not recognise the Intel Pro1000 NICs installed in the test server, whereas the R2 Beta did.
• Clunky text-based console to configure the Hyper-V server once up and running, command line interface required to turn off firewall to enable the management console to communicate with the server.
• Integration tools will not install onto our Windows Server 2003 SP1 test server, which required us to add a legacy network connection, connecting to a share to run the SP2 update, then an update before integration services would install.
• Lacking in enterprise features right now; however, extremely aggressive and strong roadmap will see this catch up very quickly.

Bottom line
The product is not as mature as VMware or XenServer, but has a lot of momentum and integration in a Windows environment will make this the hypervisor of choice for those running mainly Microsoft. Watch this product.