What's the best hosted virtualisation suite?

A lot of the fuss behind virtualisation is focused around the datacentre. That's all well and good, but there is a whole world of virtualisation for workstations where competition for the best suite is red-hot and constantly improving.

Previously we took a look at the bare-metal hypervisors that are favoured in datacentres. Today we look at Type 2 virtualisation — where one operating system needs to be running to host another operating system.

The line-up for this round-up is: VMware Workstation, Virtualbox, Parallels Workstation and KVM. Wine is also tested to provide some control to the test and because it can achieve some of the benefits of a hypervisor for running Windows applications.

Hardware

The workstation we used for testing was an HP Z800 workstation; this system features dual-Xeon W5580 processors running at 3.2GHz, 2GB RAM and a 1TB HDD. A monstrous Nvidia Quadro FX 5800 with 4GB RAM is also part of the mix.

Ubuntu 9.04 was chosen as the base operating system as it provided the widest range of hypervisors to examine and allowed for testing of KVM and Wine, which are not available on any Windows operating systems.

Windows XP SP2 was installed and the following tests were performed:

• Cinebench single CPU
• Cinebench multiple CPU
• Cinebench openGL
• Start-up time

Windows XP was chosen over its Windows Vista or Windows 7 brethren as it is stable, lighter and a well-known quantity. We felt that this would allow each of the hypervisors to show off their potential with the additional extra packages and tools included for XP. KVM was the only hypervisor to not provide an additional package to install to accelerate Windows XP's performance.

VMware Workstation
Installation for VMware Workstation is a breeze. After downloading the product from its product page, it was a simple case of installing the packages needed in Ubuntu and executing the downloaded software bundle and following the prompts. The bundle is an easier alternative to converting the rpm file for non-rpm-based distributions such as Ubuntu.

The hardest part of this install was having to sign up to the VMware site and receive a trial key, which is a sign of the ease of installation for this mature product.

It's a snap to get Windows XP running with the unattended install process.
(Credit: Chris Duckett/ZDNet.com.au)

VirtualBox
Two options exist to get VirtualBox onto a Linux system: via the distribution's package manager or via the VirtualBox site itself.

Based on previous experience with a Gentoo box, the VirtualBox OSE package can be more hassle than it is worth. The pre-packaged version available from the VirtualBox site worked liked a charm. Simply download the appropriate file, click to install it and you will have VirtualBox up and running in no time.

VirtualBox's sheer pace is hard to beat.
(Credit: Chris Duckett/ZDNet.com.au)

Parallels Workstation
Parallels' traditional strength has been on the OS X operating system, and one would truly hope so after the drama needed to get Parallels working properly on Ubuntu. It's easy enough to acquire a copy of Parallels and the necessary trial key from its product page, but careful reading of said page should flash some warning lights.

For instance, the site says "New! Parallels Workstation now installs on new Linux kernel (2.6.23)", a kernel that has its origins in 2007 and has been superseded by six stable releases of the kernel in that time.

Also "new" to Parallels Workstation according to its product page is support for Ubuntu 7.10 and Fedora 8, yet again the state-of-the-art Linux circa late 2007.

The real pain with installing Parallels came after installing the deb archive off the site and finding it impossible to complete the parallels-config command. This was due to using the latest Ubuntu distribution. Thanks to the instructions available here installation was able to be completed, but this process easily made Parallels Workstation the hardest product to get up and running.

It's a snap to get Windows XP running with the unattended install process.
(Credit: Chris Duckett/ZDNet.com.au)

Wine
Wine installation is handled by Ubuntu's apt package management system. An apt-get install wine is all that is needed to install it, followed by running the winecfg command. Once these two commands are completed, you are ready to roll.

Wine integrates fully with the Ubuntu environment.
(Credit: Chris Duckett/ZDNet.com.au)

KVM
KVM's installation is also handled by apt, however unlike Wine, there are a number of other steps needed to complete the installation. Some of the steps require editing of XML files, but the instructions are straightforward and should not create any difficulty.

It is also recommended to install a graphical tool for KVM to make managing of virtual machines much more easier.

In typical *nix fashion, the Virtual Machine Manager shown here does not come bundled with KVM.
(Credit: Chris Duckett/ZDNet.com.au)

VMware Workstation
Upon creating a new virtual machine for our Windows XP test, we were greeted by a unique VMware feature: an unattended install. This is a great time-saver, as XP has a number of prompts that require input throughout the install — just enter your Windows product key, a username and password and VMWare takes care of the rest.

The control panel for virtual machines is well laid-out and easy to use; once again the maturity of this product shines through. It is also the only hypervisor, besides KVM, to explicitly allow editing of the number of cores that a virtual machine can use.

VMware Workstation has a record/playback feature that allows the reproduction of the machine's state, although this did involve dropping the machine back to executing on one core only. It could be useful for debuggers and testers. It also has the ability to run virtual machines' execution in the background and drop the viewing window.

VMware Workstation's management window.
(Credit: Chris Duckett/ZDNet.com.au)

VirtualBox
VirtualBox focuses on one thing and does it very well: running desktop-focused virtual machines. It's a no-frills approach that keeps managing VMs dead simple but removes the flexibility present in other systems.

One problem we encountered was VirtualBox's inability to boot from a NetBSD 5.0 CD. This was the only hypervisor to fail, and was the only outstanding tarnish on VirtualBox's record.

All the options for each VM is kept in a separate window that keeps the interface clean.
(Credit: Chris Duckett/ZDNet.com.au)

Parallels Workstation
While the other hypervisors tend to display a list of virtual machines created within them, Parallels works with its VMs via files. This means that you have to navigate the filesystem and open the file of the VM you wish to use in the same manner as a word processor application. This makes the interface clunky and feel dated in comparison.

Another problem we had with Parallels is that our Windows XP VM would need to be reset after powering on to boot properly. A minor annoyance, but one that can be quite aggravating over time.

The information on each VM is well presented, but it lacks the flexibility of the VMware Workstation or KVM.

Parallels Workstation does a nice job of showing the important configuration settings.
(Credit: Chris Duckett/ZDNet.com.au)

KVM
KVM takes a traditional Unix approach and separates the execution of the virtual machines from their viewer completely — for instance, you can have Windows XP running without having to have a window open showing the desktop; it will just run quietly in the background.

Clearly KVM is aimed at a higher level of use than just workstations, and the design reflects this. Although making use of the available graphical tools is recommended, you can revert to XML or a console interface should you wish.

As the KVM viewers are separate from the server, this gives a viewer the ability to connect to KVM hosts that may be running on other machines.

Creation and management of virtual machines is easy with the graphical tools and the ability to drop back into text files gives KVM lots of flexibility and under-the-hood power. With our set-up, we could easily create and deploy many Ubuntu virtual machines, an ability which is more at home in the server room.

Along with VMware Workstation, KVM is the only other product that allows you to specify the number of cores to show to the virtual machine, showing its "beyond workstation" pedigree.

KVM has a tabbed interface, where the VM is able to be viewed in the "Console" tab.
(Credit: Chris Duckett/ZDNet.com.au)

Wine
As a program that simply executes programs for Windows, the management is left to each user — that means the program can live wherever and Wine will happily execute them.

The flexibility of the system is inherent on which program the user wishes to run and its status in the Wine Application Database. Unless the program has platinum status, expect to have problems and minor annoyances as a bare minimum.

There's no question that Wine is meant to imitate Windows.
(Credit: Chris Duckett/ZDNet.com.au)

When performing benchmarks of the hypervisors, only VMware Workstation was able to run every test we required. This speaks volumes for the stability and flexibility of the product.

The results of the tests we performed were:

The key rows to look at are the start-up time and OpenGL score in Cinebench.

VirtualBox and Parallels appear to blitz the competition in the one CPU test, but everything is not as it seems. With those two products unable to be configured to restrict the number of cores used, I suspect that they are both using as many cores as they can in an effort to boost performance. This is backed up by the fact that Parallels and VirtualBox are able to appear faster than Wine on a single CPU, which is without the additional hypervisor and OS overhead, yet Wine smashes their scores on the multiple CPU score — what appears as one CPU to the guest OS may in fact be many processes and/or threads on the host OS. Therefore I put more credence into the start-up time as a better reflection of performance.

In general usage, each of these hypervisors feel quick within Windows XP. VirtualBox does stand above the competition though, and is blisteringly quick with OpenGL compared to the other hypervisors.

Cost

The following table summarises the cost of each product (does not include cost of support):

Based on the tests and interaction with the product, the only option that we would rule out is Parallels Workstation. Even though it is the cheapest of the non-free options, the trouble that it gave and the continual need to reset the VM immediately after powering it struck it off the list. Some of these issues would immediately be ruled out if you're using Parallels on the Mac OS X platform, where it is much more mature.

If you are a user of a Linux workstation and have a need for one or two Windows applications and do not need a full-blown virtualisation solution, then Wine is the best way to go. The applications are much quicker than they would be in a hypervisor, but do have the caveat that they may not work 100 per cent as you expect in Windows.

KVM finds its way in here as a hosted virtualisation suite, but is generally up against workstation-focused software rather than server software, which is its natural terrain. Due to its lack of performance and problems with graphics rendering it is not recommended for pure workstation purposes. However, if you don't mind tinkering with Linux and have a need to deploy multiple copies of the host operating system, for example in multiple Ubuntu instances, then KVM could work for you.

In the server room, KVM does find itself up against products more in line with what we covered previously, though considering that it is free, easily available on Linux and can replicate its host OS easily, it definitely has a place at the table, but is not the winner.

That only leaves two products in line for the title: VMware Workstation and VirtualBox.

The clear tear-away from the pack in terms of performance was VirtualBox. When VirtualBox works it is very good and only has issues on the rarest of occasions. The new 3D acceleration built into the product gives it a leg-up on the competition and the fact that it is free only sweetens the deal. A minor grey cloud on VirtualBox's horizon is the purchase of Sun Microsystems, the current developer of VirtualBox, by Oracle. It is unlikely that Oracle will kill the product, but could be tempted to merge it with its own virtualisation solutions or make it more enterprise-focused.

VMware Workstation is highly configurable and built by a company that knows virtualisation. During testing, it was faultless and the management systems were top notch. Although its performance stats were not the best, the feel of the system is not slow. Being able to record a VM session will definitely find a niche.

What is our final recommendation?

If you must have 100 per cent compatibility, certainty and would like support, then VMware would be your choice.

But provided you are not trying to boot NetBSD 5.0, we would recommend VirtualBox. It's fast, easy to use and has that sweet price point: free.