Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

For the purposes of testing the two software packages, we used a dual quad-core 2.7Ghz AMD Opteron workstation with 16GB of RAM, a GeForce 9800 1GB DDR3 graphics card (using the latest production level nVidia proprietary drivers) and a 500GB SATA-2 hard disk. Our OS environment was Ubuntu LTS 10.04, 64-bit edition. Internet connectivity was an Optimum Ultra 100Mb cable modem link.

For the purposes of our tests and for limiting the scope of the review of both VirtualBox 3.20 and VMware Workstation 7.1, I decided to concentrate on Windows XP and Windows 7 64-bit and 32-bit performance and usability under 64-bit Linux, otherwise there are far too many permutations to consider for host and guest operating systems.

For 32-bit VMs, we used 2 vCPUs and 4GB of RAM. For 64-bit VMs, we used 4 vCPUs and 4GB of RAM. For all benchmarks, we ran at 1024x768 resolution unless the benchmark software forced it down to a lower resolution.

This desire to test Windows virtualization on Linux exclusively was motivated by my recent switch to Linux as my primary operating environment, so be aware that performance of other host and guest OSes under your specific system may vary considerably.

Note that the requirements of both of VMware Workstation and VirtualBox are well below that of the test system. You can comfortably run each software package on a dual-core 32-bit processor with as little as 2GB of system memory. However, it is recommended that you have at least a 4GB system and a 64-bit CPU if you are going to be virtualizing desktop operating systems with full GUIs.

VMWare Workstation 7.1

I actually wrote the very first review of the product, way back in October 1999, long before the company ever expanded into server virtualization with ESX Server and vSphere 4 and became the enterprise virtualization powerhouse that it is today.

VMware Workstation 7.1 supports both Linux and Windows host operating systems, and costs $189.00. A similar product for Mac, VMware Fusion, also is sold by the company.

VMware Workstation 7.1 supports many 32-Bit and 64-bit guest OSes, and provides full synthetic paravirtualized driver support for both Windows and Linux.

Being the mature product that VMware Workstation 7.1 is, its list of features is extremely comprehensive, and not exclusive to the list below:

* Full Paravirtualized "VMware Tools" for Windows and Linux and partial synthetic driver support for Solaris and FreeBSD. These are synthetic device drivers which help improve video, mouse, I/O and networking performance while virtualized.

* Shared folder support. This allows for easy data exchange between the host OS (the OS running VMware Workstation) and the guest operating system. Shared folders from the host OS appear as a mapped network drive in Windows guest OSes.

* Virtual USB controllers. VMware implements a virtual USB 2.0/USB 1.1 controller and allows you to connect arbitrary USB devices to your virtual machines without having to install device specific drivers on the host.

* Sound driver support. VMware can emulate an Intel AC'97 or SoundBlaster 16.

* Hardware virtualization support with VT-X and AMD-V(supported 64-Bit CPU required)

* Support for up to 4 vCPU's per VM with 32GB of memory each.

* Accelerated 2D graphics and comprehensive 3D graphics support for select guest OSes. VMware can allocate up to 256MB of virtualized video RAM and has full support for Windows Direct3D, OpenGL 2.13D and DirectX9.0c with Shader Model 3 support.

* Seamless Desktop Mode Windowed apps from the guest OS can be displayed on the host OS without displaying the entire guest desktop. i.e, Microsoft Word can run on Linux without showing the entire Windows UI.

* Support for Encrypted Virtual Machines.

So how did VMware Workstation 7.1 perform? Exceptionally well. In virtually all of our tests, it matched or exceeded the performance of Oracle VM VirtualBox. Windows XP and Windows 7 32-bit and 64-bit performance was extremely snappy, and close enough to native that when we were running it in full screen mode, we couldn't perceive the difference between "On the metal" and virtualized on our test system.

Benchmark Score for Novabench using VMWare Workstation 7.1 on 32-bit Windows XP Guest (above)

Benchmark Score for Novabench using VMware Workstation 7.1 on 32-bit Windows 7 Guest (above)

Benchmark Score for Novabench using VMware Workstation 7.1 on 64-bit Windows 7 Guest (above)

On Novabench, which is a 32-bit Windows benchmark, we saw some particularly interesting results with VMware 7.1. For starters, we noticed that when using 32-bit OSes, applications were only able to see 3GB out of the 4GB of total system RAM installed. This didn't matter whether we set the RAM to 3584 (3.5GB, the actual limit for 32-bit Windows XP) or 4096MB.

Apparently, in 32-bit Windows 7, this has something to do with Memory-Mapped (MMIO), but we didn't see this behavior occur in VirtualBox. In VirtualBox, if we allocated 3584MB in Windows XP, we saw the full 3584MB.

3DMark03 Running under VMware 7.1 using Direct3D and DirectX9 (1024x768)

3DMark03 Results for VMware Workstation 7.1 under Windows 7 32-Bit

Overall, the 64-Bit version of Windows 7 performed about the same as the 32-bit version on VMware Workstation 7.1 when it came to pure computational, 2D/3D graphics and I/O-based benchmarks.

However Windows 7 64-Bit was clearly the better OS in terms of overall efficiency when being virtualized on VMWare when being compared with Windows XP and the 32-bit version of Windows 7, especially when considering the MMIO RAM bottleneck. If you want to allocate 4GB or more of RAM to a Windows 7 VM, you definitely want to use the 64-bit version.

Oracle VM VirtualBox 3.20

VirtualBox also supports a wide array of 32-bit and 64-bit guest OSes, many of which have full paravirtualized synthetic device drivers supported.

I admit to being a bit partial and biased when it comes to VirtualBox, as I have been using it as my primary desktop virtualization environment since 2007. I've used it many times on ZDNet for doing OS demonstrations, such as with my past video reviews of Ubuntu Lucid Lynx 10.04 and OpenSUSE 11.2. I also use the product on a daily basis to run my legacy Windows XP environment for work.

There's an awful lot to like about the product, the first of which is that it is Free -- as in the strictest GNU/FSF sense, because it's licensed under GPL. Oracle actually distributes two versions of VirtualBox, the first of which is VirtualBox Open Source Edition (OSE) and the other, the commercial version, which is also free of cost. The difference between the two versions is that OSE lacks the built in RDP server for remote desktop sessions as well as USB support.

Unless you are the strictest of Free Software/GNU adherents, I strongly suggest you download the commercially licensed version, which is the default version available on the VirtualBox site.

On paper, Oracle VM VirtualBox supports the following features, including the following:

* Full Paravirtualized "Guest Additions" for Windows, Linux and Solaris guests. These are synthetic device drivers which help improve video, mouse, I/O and networking performance while virtualized.

* Shared folder support. This allows for easy data exchange between the host OS (the OS running VirtualBox) and the guest operating system. Shared folders from the host OS appear as a mapped network drive in Windows guest OSes.

* Virtual USB controllers. VirtualBox implements a virtual USB 2.0/USB 1.1 controller and allows you to connect arbitrary USB devices to your virtual machines without having to install device specific drivers on the host.

* Broad virtual network driver support. VirtualBox can emulate several common legacy Ethernet cards (AMD PCNet series) and several series of Intel Pro/1000 chipsets for maximum OS compatibility.

* Remote Desktop Protocol. Unlike any other virtualization software, VirtualBox fully supports the standard Remote Desktop Protocol (RDP). A virtual machine can act as an RDP server, allowing you to "run" the virtual machine remotely on some thin client that merely displays the RDP data.

* USB over RDP. With this unique feature, a virtual machine that acts as an RDP server can still access arbitrary USB devices that are connected on the RDP client. This way, a powerful server machine can virtualize a lot of thin clients that merely need to display RDP data and have USB devices plugged in.

* Sound driver support. VirtualBox can emulate an Intel AC'97 or SoundBlaster 16.

* Hardware virtualization support with VT-X and AMD-V(supported 64-Bit CPU required) including nested paging and PAE/NX bit support.

* Support for up to 16 vCPU's per VM with 16GB of RAM each, and 32 virtual cores per host OS.

* Accelerated 2D graphics and "experimental" 3D graphics support for select guest OSes. VirtualBox can allocate up to 128MB of virtualized video RAM if desired.

* Seamless Desktop Mode Windowed apps from the guest OS can be displayed on the host OS without displaying the entire guest desktop. i.e, Microsoft Word can run on Linux without showing the entire Windows UI.

* Support for competing Virtual Disk formats from VMware and Microsoft.

* VM "Teleportation" (Aka Live Migration) between VirtualBox hosts.

* Experimental EFI Support, which permits installation of Mac OS X on standard PC hardware running VirtualBox, unmodified.

So with all of these features, how does the free Oracle VM VirtualBox stack up against its competition, VMware Workstation 7.1 and VMWare Fusion for Mac?

From the perspective of basic Windows performance,  VirtualBox is definitely more than "Good Enough" to serve as the basis for providing 2D Windows application compatibility under Linux, such as for Microsoft Office and effectively any standard 2D business productivity application, which accounts for about 80 percent or more than what your average business computing user is going to use Windows for.

Benchmark Score for Novabench using VirtualBox on 32-bit Windows XP Guest (above)

Benchmark Score for Novabench using VirtualBox on 64-bit Windows 7 (above)

However, where VirtualBox fails miserably is in virtualized 3D application performance on Windows.

For starters, VirtualBox currently only supports 3D on 32-bit Windows OSes. That means that in the case of 64-bit Windows Vista and Windows 7, the Aero Glass effects won't work. It also only supports  basic Direct3D functionality with DirectX8 and DirectX9 and shouldn't be expected to be used for serious gaming.

Windows Experience Index Results for 64-Bit Windows 7 on Oracle VM VirtualBox 3.2.

We also tried to perform a 32-bit Windows Experience Index (WEI) test on VirtualBox using the supported experimental Direct3D driver, but at the time of this writing, VirtualBox 3.2 was released with a bug that prevented Windows 7 32-bit from being installed on my test system. We hope to update this with new results in the near future.

To do 3D tests on VirtualBox in virtualized Windows XP we installed the free version of Futuremark's 3DMark03, which tests against DirectX8 and DirectX9 compatible graphics cards.

[NOTE: In the initial draft of this article we were unable to get 3D acceleration to work in VirtualBox on Windows XP, but we discovered that the 3D driver does not get enabled unless you install the Guest Additions in Windows XP "Safe Mode". Apparently, this is also true for the 32-bit version of Windows 7.]

"Mother Nature" demo in 3DMark03 under VirtualBox 3.20 (800x600)

3DMark03 benchmark results with VirtualBox 3.20 on 32-bit XP guest.

Note that while we had a few issues with getting 3D graphics working on Windows with VirtualBox, we had no problems whatsoever with 3D virtualized Linux OSes, as we've demonstrated in the past. This issue appears to be specific to virtualized Windows OSes at this time. However, as we said earlier, testing virtualized Linux 3D performance was not within the scope of this particular review for time constraint.

The Verdict

Where they differ is in the area of 3D graphics support for Windows guests. Clearly, in this area, VMware Workstation 7.1 is the superior product, as its virtual 3D synthetic driver is able to support up to DirectX9 levels of compatibility, which make even running demanding 3D Windows games and 3D applications in a virtualized environment possible. Based on our 3DMark results, the 3D support in VMware yields almost twice the 3D performance of a comparable VirtualBox system.

Oracle VM VirtualBox satisfies the basic requirements of 3DBench03 in Windows XP with its experimental support, but be sure to install the drivers in "Safe Mode" or as we found out, they won't work.

Arguably, due to a bug during product launch, we had some issues installing Windows 7 32-Bit on VirtualBox 3.20, and we hope to get this rectified shortly so we can see if Windows 7 3D graphics works properly as promised on that software.

In VMware, we were successfully able to run 32-bit DirectX9 benchmarks with 3DBench03 and were even able to run the newer 3DBench06 in "Demo" mode. 3DBench Vantage, which supports the newer DirectX10 and DirectX11 video cards was unable to run, but mainstream games and apps which support these cards are currently few and far between.

If you're going to use Windows XP and Windows 7 mainstream productivity apps, Oracle VM VirtualBox 3.20 would seem to be a wise choice, as you can't beat the price and regular 2D application performance is excellent.

However, if you have a need for more demanding 2D and 3D games, visualization applications as well as a need to use those apps in the 64-Bit versions of the Windows OS, you're probably going to want to use VMware and shell out the $189.00 for it, or at the very least, download the eval and then continue to use the "Player" app after your Windows VM is created.

Have you done some VMWare Workstation 7.1 vs. VirtualBox tests of your own? Talk Back and Let Me Know.