VMWare's filesystem clustering cracked open

VMWare's filesystem clustering cracked open

Summary: Late last week, an important plate in VMWare's proprietary battle armor was cracked open when Fluid Operations, the company behind the eCloudmanager storage virtualization product open-sourced on the community Google Code site their clean room implementation of VMFS, the clustered locking file system used by VMWare's ESX Server hypervisor product.

SHARE:

crosshairs-vm.jpg Late last week, an important plate in VMWare's proprietary battle armor was cracked open when Fluid Operations, the company behind the eCloudmanager storage virtualization product open-sourced on the community Google Code site their clean room implementation of VMFS, the clustered locking file system used by VMWare's ESX Server hypervisor product.

While Fluid Operations' Java-based driver is only in the early stages, and is currently only read-only capable at this point, this represents some of the very first significant efforts by the independent Open Source community to provide interoperability with VMWare's standards.

Click on the "Read the rest of this entry" link below for more.

VMWare itself has only had a few exploratory forays into Open Source. In 2006 they opened their Virtual Machine Disk Format (VMDK) which is used as the file container for all VMWare VMs. About a year ago, the company released Open-VM-Tools, the paravirtualization drivers for optimizing video, I/O, network and mouse operations within a VMWare virtual machine. VMWare recently announced  VMWare View Open Client which enables Linux-based systems to connect to remote desktops. VMWare has also released the modifications to  GPL/LGPL-licensed components used in their products, but has fallen short of actually open sourcing their ESX hypervisor, the crown jewel of the VMWare stack.

Also Read -- VMWare: Time to Pay the Open Source Piper

Why is Fluid Operations' VMFS driver important? Well, for starters, even with it being read-only capable at this point, it brings in the near-future possibility of no longer needing to purchase VMWare's Consolidated Backup software, which is essentially a read-only VMFS software driver for Windows that is used as a gateway for enterprise backup software such as IBM's Tivoli Storage Manager, Symantec's Netbackup, CA ArcServe and EMC's Legato to directly access VMFS volumes on a Storage Area Network (SAN). This eliminates the need to install network-based backup agents directly on Virtual Machines and use the LAN for backup traffic -- a backup job can be consolidated down to backing up the VMDK files on the VMFS volumes to disk or to tape. With the Fluid Operations driver, you could potentially build VMFS functionality directly into a Linux or UNIX-based media server for no extra cost.

The holy grail of Fluid Operations' efforts, of course would be to build a native read-write VMFS driver, with full clustering capabilities, and the ability to natively build fresh VMFS volumes under Linux, UNIX and Microsoft Windows. This would allow VMFS to become the de-facto VM cluster file system standard for all hypervisors, not just VMWare's ESX. It should be noted that Microsoft will be releasing its own clustered version of their NTFS file system with Windows Server 2008 R2 to support VM clustering in their own Hyper-V virtualization layer, but to date has never indicated that the specifications for NTFS will ever be released into Open Source.

[poll id="10"]

This begs the question -- should VMWare itself open its VMFS specification, as it did with VMDK? And should VMWare Open Source it's ESX hypervisor now that Citrix has released its competing enterprise hypervisor product, XenServer 5 for free? Talk back and let me know.

[poll id="11"]

Topics: Hardware, Virtualization, VMware

About

Jason Perlow, Sr. Technology Editor at ZDNet, is a technologist with over two decades of experience integrating large heterogeneous multi-vendor computing environments in Fortune 500 companies. Jason is currently a Partner Technology Strategist with Microsoft Corp. His expressed views do not necessarily represent those of his employer.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

5 comments
Log in or register to join the discussion
  • what customers want

    Customers (should) want open standards and non-proprietary flexibility. They should always have the option of switching vendors with as little stress as possible. Why would any company want to lock itself in? It takes away any incentive for the vendors to provide good service and competitive pricing after the initial sale.
    jcschweitzer
    • If they want to keep software proprietary, thats fine, but...

      they should atleast allow interfaces with their product for reasons like backups, etc that are not closed source.

      Its important to allow portability of your product or you end up like novell and apple.
      Been_Done_Before
  • RE: VMWare's filesystem clustering cracked open

    VMFS should be open sourced, the more folks that adopt VMFS as a clustering filesystem the larger market share and storage dominance VMware gets as a result. VMFS is actually quite simple and could be reverse engineered with relative ease; digging into the guts of the VMkernel is much more complicated and would take months if not years of full time specialized analysis to expose its core functionality.

    Open sourcing the ESX/ESXi Hypervisor/VMkernel would be absolute suicide, VMware's competitive advantage is having developed the highest performing hypervisor on the market without exception. Giving away the secret sauce to their advanced memory management techniques such as transparent page sharing, memory ballooning, page swapping or the magic behind virtual switching doesn't make any sense whatsoever - those are the crown jewels of their technology.

    My $.02

    Gregory Perry

    http://GoVirtual.tv
    liveammo
  • RE: VMWare's filesystem clustering cracked open

    Of course they should give it all away free. Their software
    engineers will all happily work for nothing, right? Probably
    all in their spare time, too...

    (I'm sure the effort involved in 'opening' VMFS, however
    simple it may be, will take away valuable engineering time
    from their profitable products, so realistically they have to
    prioritise. Just look how long it's taken Microsoft to start
    openly documenting some of their protocols. Then ask the
    Samba Team how accurate they turn out to be.)
    jrguk
  • RE: VMWare's filesystem clustering cracked open

    What's with the headlines?

    As the FAQ says, "The clustered sync/lock algorithm is mostly unknown," and they need help analyzing it. (See here: http://code.google.com/p/vmfs/wiki/HowCanIHelp) So, NO, the file system clustering has not been cracked open. A good chunk of the file system has been reverse engineered, that's about it.

    There are some security implications with this, so now it's very important to keep your VMFS volumes protected.
    packetracer