Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

Summary: Two titans battle for desktop virtualization supremacy. Who will reign supreme?

SHARE:

Two titans battle for desktop virtualization supremacy. Who will reign supreme?

Welcome, True Believers.

Two years ago, we visited the subject of free desktop virtualization hypervisors. At the time, the two preferred free solutions were Sun's xVM VirtualBox and and VMware Server 2.0.

Things have changed quite a bit in the last two years. For starters, Sun no longer exists as an independent company -- it's now owned by Oracle.

VMware, a division of EMC Corporation, decided to make some strategic changes along the lines of its free virtualization solutions by offering its enterprise-level hypervisor, ESX Server, in a free version, ESXi. This complicates things somewhat for the desktop virtualization user seeking a free solution, as ESX 3i requires a dedicated system.

While the company continues to maintain VMware Server 2.0, its free hosted virtualization product, it has concentrated its desktop virtualization development efforts on VMWare Workstation, a $189.00 desktop virtualization software package.

Version 7.1 of VMware Workstation was released very recently, on May 25, 2010.

VMware Workstation is also complemented by VMware Player, a stand-alone executable that allows you to "Play" operating systems created with the VMware Workstation product. Therefore, you can download the evaluation version of VMware Workstation 7.1 and after the evaluation period expires, you can continue to use your virtual machines on the "Player".

[EDIT: It has been pointed out to me by VMware's Public Relations firm, Outcast, that the latest version of Player, version 3.1, uses the same virtualization engine as Workstation 7.1 and now has the ability to create Virtual Machines. However, we did not test Player as part of this review.]

Oracle, having recently completed its acquisition of Sun, has continued to develop and introduce significant improvements in VirtualBox. Version 3.20, the first version to bear the Oracle logo, was also released this week.

With the latest releases of the two major desktop virtualization apps released at around the same time, which one reigns supreme? We thoroughly tested both on our Linux system and observed them for performance and usability. Here's how they fared against each other.

What we tested on

For the purposes of testing the two software packages, we used a dual quad-core 2.7Ghz AMD Opteron workstation with 16GB of RAM, a GeForce 9800 1GB DDR3 graphics card (using the latest production level nVidia proprietary drivers) and a 500GB SATA-2 hard disk. Our OS environment was Ubuntu LTS 10.04, 64-bit edition. Internet connectivity was an Optimum Ultra 100Mb cable modem link.

For the purposes of our tests and for limiting the scope of the review of both VirtualBox 3.20 and VMware Workstation 7.1, I decided to concentrate on Windows XP and Windows 7 64-bit and 32-bit performance and usability under 64-bit Linux, otherwise there are far too many permutations to consider for host and guest operating systems.

For 32-bit VMs, we used 2 vCPUs and 4GB of RAM. For 64-bit VMs, we used 4 vCPUs and 4GB of RAM. For all benchmarks, we ran at 1024x768 resolution unless the benchmark software forced it down to a lower resolution.

This desire to test Windows virtualization on Linux exclusively was motivated by my recent switch to Linux as my primary operating environment, so be aware that performance of other host and guest OSes under your specific system may vary considerably.

Note that the requirements of both of VMware Workstation and VirtualBox are well below that of the test system. You can comfortably run each software package on a dual-core 32-bit processor with as little as 2GB of system memory. However, it is recommended that you have at least a 4GB system and a 64-bit CPU if you are going to be virtualizing desktop operating systems with full GUIs.

VMWare Workstation 7.1

VMware Workstation 7.1 is the most mature desktop/workstation virtualization product on the market -- in fact, it is the first product that VMware ever introduced, back in 1999, so it has undergone 10 years of intense product development.

I actually wrote the very first review of the product, way back in October 1999, long before the company ever expanded into server virtualization with ESX Server and vSphere 4 and became the enterprise virtualization powerhouse that it is today.

VMware Workstation 7.1 supports both Linux and Windows host operating systems, and costs $189.00. A similar product for Mac, VMware Fusion, also is sold by the company.

VMware Workstation 7.1 supports many 32-Bit and 64-bit guest OSes, and provides full synthetic paravirtualized driver support for both Windows and Linux.

Being the mature product that VMware Workstation 7.1 is, its list of features is extremely comprehensive, and not exclusive to the list below:

* Full Paravirtualized "VMware Tools" for Windows and Linux and partial synthetic driver support for Solaris and FreeBSD. These are synthetic device drivers which help improve video, mouse, I/O and networking performance while virtualized.

* Shared folder support. This allows for easy data exchange between the host OS (the OS running VMware Workstation) and the guest operating system. Shared folders from the host OS appear as a mapped network drive in Windows guest OSes.

* Virtual USB controllers. VMware implements a virtual USB 2.0/USB 1.1 controller and allows you to connect arbitrary USB devices to your virtual machines without having to install device specific drivers on the host.

* Sound driver support. VMware can emulate an Intel AC'97 or SoundBlaster 16.

* Hardware virtualization support with VT-X and AMD-V (supported 64-Bit CPU required)

* Support for up to 4 vCPU's per VM with 32GB of memory each.

* Accelerated 2D graphics and comprehensive 3D graphics support for select guest OSes. VMware can allocate up to 256MB of virtualized video RAM and has full support for Windows Direct3D, OpenGL 2.13D and DirectX9.0c with Shader Model 3 support.

* Seamless Desktop Mode Windowed apps from the guest OS can be displayed on the host OS without displaying the entire guest desktop. i.e, Microsoft Word can run on Linux without showing the entire Windows UI.

* Support for Encrypted Virtual Machines.

So how did VMware Workstation 7.1 perform? Exceptionally well. In virtually all of our tests, it matched or exceeded the performance of Oracle VM VirtualBox. Windows XP and Windows 7 32-bit and 64-bit performance was extremely snappy, and close enough to native that when we were running it in full screen mode, we couldn't perceive the difference between "On the metal" and virtualized on our test system.

Benchmark Score for Novabench using VMWare Workstation 7.1 on 32-bit Windows XP Guest (above)

Benchmark Score for Novabench using VMware Workstation 7.1 on 32-bit Windows 7 Guest (above)

Benchmark Score for Novabench using VMware Workstation 7.1 on 64-bit Windows 7 Guest (above)

On Novabench, which is a 32-bit Windows benchmark, we saw some particularly interesting results with VMware 7.1. For starters, we noticed that when using 32-bit OSes, applications were only able to see 3GB out of the 4GB of total system RAM installed. This didn't matter whether we set the RAM to 3584 (3.5GB, the actual limit for 32-bit Windows XP) or 4096MB.

Apparently, in 32-bit Windows 7, this has something to do with Memory-Mapped (MMIO), but we didn't see this behavior occur in VirtualBox. In VirtualBox, if we allocated 3584MB in Windows XP, we saw the full 3584MB.

Windows Experience Index Results for 32-Bit Windows 7 on VMware Workstation 7.1

Windows Experience Index Results for 64-Bit Windows 7 on VMware Workstation 7.1

3DMark03 Running under VMware 7.1 using Direct3D and DirectX9 (1024x768)

3DMark03 Results for VMware Workstation 7.1 under Windows 7 32-Bit

Overall, the 64-Bit version of Windows 7 performed about the same as the 32-bit version on VMware Workstation 7.1 when it came to pure computational, 2D/3D graphics and I/O-based benchmarks.

However Windows 7 64-Bit was clearly the better OS in terms of overall efficiency when being virtualized on VMWare when being compared with Windows XP and the 32-bit version of Windows 7, especially when considering the MMIO RAM bottleneck. If you want to allocate 4GB or more of RAM to a Windows 7 VM, you definitely want to use the 64-bit version.

Oracle VM VirtualBox 3.20

Oracle VM VirtualBox, formerly Sun xVM VirtualBox, is an Open Source project based upon the work of Innotek GmbH, a German software firm that was purchased by Sun Microsystems in 2007. It supports a wide array of host 32-bit and 64-bit operating systems, including Linux, Mac OS X, FreeBSD and of course, Windows.

VirtualBox also supports a wide array of 32-bit and 64-bit guest OSes, many of which have full paravirtualized synthetic device drivers supported.

I admit to being a bit partial and biased when it comes to VirtualBox, as I have been using it as my primary desktop virtualization environment since 2007. I've used it many times on ZDNet for doing OS demonstrations, such as with my past video reviews of Ubuntu Lucid Lynx 10.04 and OpenSUSE 11.2. I also use the product on a daily basis to run my legacy Windows XP environment for work.

There's an awful lot to like about the product, the first of which is that it is Free -- as in the strictest GNU/FSF sense, because it's licensed under GPL. Oracle actually distributes two versions of VirtualBox, the first of which is VirtualBox Open Source Edition (OSE) and the other, the commercial version, which is also free of cost. The difference between the two versions is that OSE lacks the built in RDP server for remote desktop sessions as well as USB support.

Unless you are the strictest of Free Software/GNU adherents, I strongly suggest you download the commercially licensed version, which is the default version available on the VirtualBox site.

On paper, Oracle VM VirtualBox supports the following features, including the following:

* Full Paravirtualized "Guest Additions" for Windows, Linux and Solaris guests. These are synthetic device drivers which help improve video, mouse, I/O and networking performance while virtualized.

* Shared folder support. This allows for easy data exchange between the host OS (the OS running VirtualBox) and the guest operating system. Shared folders from the host OS appear as a mapped network drive in Windows guest OSes.

* Virtual USB controllers. VirtualBox implements a virtual USB 2.0/USB 1.1 controller and allows you to connect arbitrary USB devices to your virtual machines without having to install device specific drivers on the host.

* Broad virtual network driver support. VirtualBox can emulate several common legacy Ethernet cards (AMD PCNet series) and several series of Intel Pro/1000 chipsets for maximum OS compatibility.

* Remote Desktop Protocol. Unlike any other virtualization software, VirtualBox fully supports the standard Remote Desktop Protocol (RDP). A virtual machine can act as an RDP server, allowing you to "run" the virtual machine remotely on some thin client that merely displays the RDP data.

* USB over RDP. With this unique feature, a virtual machine that acts as an RDP server can still access arbitrary USB devices that are connected on the RDP client. This way, a powerful server machine can virtualize a lot of thin clients that merely need to display RDP data and have USB devices plugged in.

* Sound driver support. VirtualBox can emulate an Intel AC'97 or SoundBlaster 16.

* Hardware virtualization support with VT-X and AMD-V (supported 64-Bit CPU required) including nested paging and PAE/NX bit support.

* Support for up to 16 vCPU's per VM with 16GB of RAM each, and 32 virtual cores per host OS.

* Accelerated 2D graphics and "experimental" 3D graphics support for select guest OSes. VirtualBox can allocate up to 128MB of virtualized video RAM if desired.

* Seamless Desktop Mode Windowed apps from the guest OS can be displayed on the host OS without displaying the entire guest desktop. i.e, Microsoft Word can run on Linux without showing the entire Windows UI.

* Support for competing Virtual Disk formats from VMware and Microsoft.

* VM "Teleportation" (Aka Live Migration) between VirtualBox hosts.

* Experimental EFI Support, which permits installation of Mac OS X on standard PC hardware running VirtualBox, unmodified.

So with all of these features, how does the free Oracle VM VirtualBox stack up against its competition, VMware Workstation 7.1 and VMWare Fusion for Mac?

From the perspective of basic Windows performance,  VirtualBox is definitely more than "Good Enough" to serve as the basis for providing 2D Windows application compatibility under Linux, such as for Microsoft Office and effectively any standard 2D business productivity application, which accounts for about 80 percent or more than what your average business computing user is going to use Windows for.

Benchmark Score for Novabench using VirtualBox on 32-bit Windows XP Guest (above)

Benchmark Score for Novabench using VirtualBox on 64-bit Windows 7 (above)

However, where VirtualBox fails miserably is in virtualized 3D application performance on Windows.

For starters, VirtualBox currently only supports 3D on 32-bit Windows OSes. That means that in the case of 64-bit Windows Vista and Windows 7, the Aero Glass effects won't work. It also only supports  basic Direct3D functionality with DirectX8 and DirectX9 and shouldn't be expected to be used for serious gaming.

Windows Experience Index Results for 64-Bit Windows 7 on Oracle VM VirtualBox 3.2.

We also tried to perform a 32-bit Windows Experience Index (WEI) test on VirtualBox using the supported experimental Direct3D driver, but at the time of this writing, VirtualBox 3.2 was released with a bug that prevented Windows 7 32-bit from being installed on my test system. We hope to update this with new results in the near future.

To do 3D tests on VirtualBox in virtualized Windows XP we installed the free version of Futuremark's 3DMark03, which tests against DirectX8 and DirectX9 compatible graphics cards.

[NOTE: In the initial draft of this article we were unable to get 3D acceleration to work in VirtualBox on Windows XP, but we discovered that the 3D driver does not get enabled unless you install the Guest Additions in Windows XP "Safe Mode". Apparently, this is also true for the 32-bit version of Windows 7.]

"Mother Nature" demo in 3DMark03 under VirtualBox 3.20 (800x600)

3DMark03 benchmark results with VirtualBox 3.20 on 32-bit XP guest.

Note that while we had a few issues with getting 3D graphics working on Windows with VirtualBox, we had no problems whatsoever with 3D virtualized Linux OSes, as we've demonstrated in the past. This issue appears to be specific to virtualized Windows OSes at this time. However, as we said earlier, testing virtualized Linux 3D performance was not within the scope of this particular review for time constraint.

The Verdict

Both VMware Workstation 7.1 and Oracle VM VirtualBox 3.2 are excellent desktop virtualization products -- in terms of general Windows application usage, both perform extremely well and are comparable from a purely computational, I/O and networking performance perspective.

Where they differ is in the area of 3D graphics support for Windows guests. Clearly, in this area, VMware Workstation 7.1 is the superior product, as its virtual 3D synthetic driver is able to support up to DirectX9 levels of compatibility, which make even running demanding 3D Windows games and 3D applications in a virtualized environment possible. Based on our 3DMark results, the 3D support in VMware yields almost twice the 3D performance of a comparable VirtualBox system.

Oracle VM VirtualBox satisfies the basic requirements of 3DBench03 in Windows XP with its experimental support, but be sure to install the drivers in "Safe Mode" or as we found out, they won't work.

Arguably, due to a bug during product launch, we had some issues installing Windows 7 32-Bit on VirtualBox 3.20, and we hope to get this rectified shortly so we can see if Windows 7 3D graphics works properly as promised on that software.

In VMware, we were successfully able to run 32-bit DirectX9 benchmarks with 3DBench03 and were even able to run the newer 3DBench06 in "Demo" mode. 3DBench Vantage, which supports the newer DirectX10 and DirectX11 video cards was unable to run, but mainstream games and apps which support these cards are currently few and far between.

If you're going to use Windows XP and Windows 7 mainstream productivity apps, Oracle VM VirtualBox 3.20 would seem to be a wise choice, as you can't beat the price and regular 2D application performance is excellent.

However, if you have a need for more demanding 2D and 3D games, visualization applications as well as a need to use those apps in the 64-Bit versions of the Windows OS, you're probably going to want to use VMware and shell out the $189.00 for it, or at the very least, download the eval and then continue to use the "Player" app after your Windows VM is created.

Have you done some VMWare Workstation 7.1 vs. VirtualBox tests of your own? Talk Back and Let Me Know.

Topics: Virtualization, Hardware, Oracle, VMware

About

Jason Perlow, Sr. Technology Editor at ZDNet, is a technologist with over two decades of experience integrating large heterogeneous multi-vendor computing environments in Fortune 500 companies. Jason is currently a Partner Technology Strategist with Microsoft Corp. His expressed views do not necessarily represent those of his employer.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

26 comments
Log in or register to join the discussion
  • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

    This virtualization should be great. I like it<a href="http://ipadbagblog.com/"><font color="white"> k</font></a><a href="http://www.sutudeg.org/"><font color="white"> l</font></a><a href="http://www.short-domain-names.com/"><font color="white"> m</font></a>
    Bonek suroboyo
    • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

      As I?ve pointed out in other posts, the number of downloads doesn?t directly equal the number of copies of software in actual use. Andy must have read that post.
      Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        Although he was polite enough not to say anything about that post, he do go on to say that over 2 million people gone to the effort of registering their copies. That figure is a better measure of software that either is involved with a pilot project or is in use in a production system.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        like Google Earth and CAM-based software that are popular among heavy users of imagery like industrial designers, automotive and robotics engineers, architects, etc.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        Improved Network Performance: Makes network intensive applications like rich media (video, audio, interactive media, etc.) even faster.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        Storage Support: Comes with built-in iSCSI support to connect to storage systems, such as Sun?s newly announced Open Storage appliances, the Sun Storage 7000 family,
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        also known as ?Amber Road.? This feature enables easier management and sharing of virtual disk images.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        In addition, xVM VirtualBox 2.1 software offers improved support for:

        Mac OS X on Intel Virtualization Technology (VT-x): Provides better support for Mac OS X as a host OS utilizing hardware-assisted Intel VT-x for better performance.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        Intel Core i7 processor: Enables extremely fast performance on leading-edge hardware with support for the new Intel Core microarchitecture in the Intel Core i7 processor (codenamed Nehalem).
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        64-bit guest OS on 32-bit host platforms: Allows users to run powerful 64-bit guest OS on 32-bit host platforms without the need to upgrade the host OS while taking advantage of multi-thread applications on powerful hardware.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        There are many entries in the virtual machine software market. Without thinking about it too hard, the list of supplies of this sort of technology includes:

        VMware
        Citrix and the other members of the Xen community (includes Linux distributors, such as Red Hat, SUSE, etc., Sun, Oracle and Virtual Iron)
        Microsoft
        Parallels
        The KVM community (this includes all of the Linux distributors as well)
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        This, of course, demonstrates that Sun is facing significant challenges in offering something so distinctive that it will rise above all of the other challengers.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        Several have already found ways to support 64-bit operating systems within virtual machines on hardware that supports this. Sun can point to the fact that their product supports all of the major virtual machine file formats. They can also point to support of accelerated graphics and other performance enhancing features.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        While this appears to be an interesting offering, Sun still faces a great deal of work to get a typical IT decision maker to understand the benefits of the Sun product when there are others making a great deal of noise about the wonders of their products and their technology.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        Suns tactic to demonstrate a large and enthusiastic market response typically centers on presenting a large and ever growing number of downloads of the software from Suns servers. This metric, of course, can not be verified externally. A more important consideration is that downloads do not directly or equal product usage. An organization could download software to evaluate it and delete it after using it only<a href="http://ipadbagblog.com/"><font color="light&height"> ipad bag blog</font></a><a href="http://www.sutudeg.org/"><font color="light&height"> sutudeg </font></a> <a href="http://wposfv.com/"><font color="light&amp;height">education news</font></a> and once.
        Linux Love
      • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

        @Linux Love That is really a big question. Google's servers are the heart of Google's business. And it has long been a FEATURE, a FEATURE, not a LOOPHOLE, that one could privately modify the GPL code they use to run their business. Of course web applications are obviously SaaS. But where does one draw the line between those applications and the servers that host them? For example, take an insurance company running open source on their back end servers. At some point they decide to put a customer facing front end on those servers so that customers can access their accounts over the Net. Does that suddenly make that whole kaboodle Saas? If so, I am not sure I am comfortable with AGPL. In fact, I am not sure I am comfortable with this concept anyway since it undercuts one of the few provisions that make GPL software highly attractive to businesses that are not engaged in reselling the software itself. It really compromises the spirit of the GPL in some ways.
        arabaoyunlari@...
      • Good

        Hi, i have visited this site a couple of times now and i have to say that i find it quite exeptional actually. it'll be nice to read more in the future.
        <a rel="follow" href="http://www.1800duilaws.com/florida-dui-attorney/">Florida Dui Lawyers</a>
        Robot34
    • RE: Virtualization Smackdown 2: Oracle VM VirtualBox 3.2 vs. VMware Workstation 7.1

      @Bonek suroboyo After all these years you should know that M$ promises high and delivers low, if it bothers to deliver at all. This is yet another contemptible decision by M$ that demonstrates that its supposed commitment to customers is barely even words.
      Arabalar
    • NICE

      Interesting blog. It would be great if you can provide more details about it. Thanks you.
      <a rel="follow" href="http://www.philadelphiaaccidentlawyer.com/">Philadelphia accident lawyer</a>
      <a rel="follow" href="http://www.mystopblushingcure.com/">blushing cure</a>
      sabir12
    • nice one

      Thanks for such an interesting article here. I was searching for something like that for quite a long time and at last I have found it here.
      <a rel="follow" href="http://www.mystopblushingcure.com/">blushing cure</a>
      Robot34