Apple kicks ZFS in the butt

Apple kicks ZFS in the butt

Summary: Apple promised ZFS on Snow Leopard - and didn't deliver. A Mac's data integrity is now no better than Windows. And that's not good enough.

SHARE:
TOPICS: Apple
46

It's official: ZFS - a kick-butt file system - is nowhere to be seen in the latest release of Mac OS X, Snow Leopard. Even though it appeared in 10.5 Server, and was expected to become the default file system at some point, Apple has abandoned the Sun-developed ZFS, the first 21st century file system.

A bummer for anyone who stores data on their computer.

Why should I care? Apple is hoping you don't - and they're probably right. None of the mainstream press have mentioned dropped feature, even though it is right up there with parallel processing support as a winner for users.

ZFS combines a file system and a volume manager, along with some cool architectural features, to create an easily managed and highly reliable file system. Advanced features that just work.

Some cool features.

  • Manage storage, not disks. You can put all your disks in a pool and specify the redundancy level. ZFS takes care of the rest.
  • No more silent data corruption.Wonky things can happen to your data to and from a disk. ZFS checksums every file before it is written and stores the checksum on the parent. When the file is read, the checksum tells the filesystem if that is the block it wrote.
  • Easy snapshots. Ever wish you could roll back to a known good state? Snapshots make that easy and ZFS makes snapshots easy.
  • High performance software RAID built-in. Worried about protecting your data. ZFS provides strong RAID capabilities without adding hardware.
  • Transparent compression on the fly. Save capacity by compressing old and/or large files automagically.

What happened? 2 years ago it looked like ZFS was locked in to Snow Leopard. The Apple team was working with the Sun ZFS team. It was enabled as a read-only file system on 10.5 server. Apple even freakin' announced ZFS on Snow Leopard. The advantages - to storage geeks - were obvious.

Plus the opportunity to put daylight between OS X and Windows 7. Microsoft's ambitions for something called WinFS crashed to earth 3 years ago (see Bring me the head of WinFS.

But Apple started walking back ZFS about 9 months ago. Newer builds of Snow Leopard had less and less ZFS content until today's official release - which has none.

Maybe some insight will emerge from secretive Apple, but don't count on it. Removing ZFS from the server edition, where it makes even more obvious sense, suggests it is gone for good.

What did it in? Maybe it was a schedule problem - file systems require a lot of testing - and rewriting all the other bits took precedence. NIH - Not Invented Here - syndrome is another possibility. Or perhaps the uncertainty of Sun's future led Apple to pull back.

Or maybe they just decided customers wouldn't know enough to care, so why bother? Whatever the reason it is a major step backwards for the PC industry.

The Storage Bits take File systems are essential but unsexy plumbing. Whether it's a missing or corrupted file or a system slowing to a crawl because the directory is bloated, there is no error message that says "Your FS is screwed up."

And as noted in How Microsoft puts your data at risk - which indicted Apple's HFS+ as well -

. . . more than half of all data loss is caused by system and hardware problems. A high quality file system that took better care of our data could eliminate many of those failures.

The industry knows how to fix the problems. The question is when. With a resurgent Mac pushing ZFS maybe Redmond will see the light sooner, rather than later, and dramatically increase the reliability of all our systems.

With Apple's retreat from ZFS everyone who uses a personal computer is the loser. Maybe the Microsoft team working to improve NTFS will now take the lead in file system quality and feature.

Comments welcome, of course. Update: I got some more theories over the weekend on why the ZFS deal fell through. Check them out on StorageMojo. The short answer: licensing; GPL vs CDDL. End update.

Topic: Apple

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

46 comments
Log in or register to join the discussion
  • Curious

    What features does ZFS sport that NTFS does
    not? Which of those are important to data
    integrity?

    Reading the <i>IRON File Systems</i> I couldn't
    help notice that NTFS fared quite well. Do you
    have any data to suggest that ZFS would fare
    better than these other file systems?

    The file systems from the study still evolve.
    In particular NTFS has received updates and
    numerous new features with each revision. NTFS
    now has background recovery of files, i.e. it
    will attempt to repair corrupted files in the
    background while the system stays running (only
    the suspected file will be unavailable for the
    duration).

    The file systems and operating systems may also
    divide the tasks differently. Since Windows
    2003, Windows has had the volume shadow copy
    service for creating snapshots. While not part
    of the file system per se, it meets the exact
    same purpose.
    honeymonster
    • ZFS is a decade ahead of NTFS, even with updates

      Comparing the two is like comparing fuel injection to carburetors. ZFS does CRC down to the block level which protects you from silent data corruption, a phenomenon that Microsoft hasn't really even acknowledged exists. SDC happens when a bit on a disl platter is "flipped" without the disk's knowledge, and the larger the disk capacity the more likely it becomes. This is due to the smaller particle sizes required for the density of larger disks. E. Schlesinger from Carnegie Mellon has an interesting description of this phenomenon which was first pointed out in an IEEE paper in the late 90ies.

      Since the disk isn't aware of the flip, the OS isn't either unless you are running a "smart" filesystem like ZFS or an array that scrubs it's disks without the OS knowledge. The weakness of scrubbing is that depending on how busy the array is it may not get to the bad data before a second bit is flipped and the parity hides the corruption. I am unaware of any array that uses CRC for scrubbing, if anyone knows of one please post it here, I'd be interested.

      Also, have you ever seen a Windows Server crash while writing to a 16TB (16,000 GB) volume? It can take a week to restart while Windows checks the NTFS filesystem. ZFS does not need to do this because of it's intent logs. A very busy server might take a few minutes to restart.

      If you want to ensure your data is safe, one thing you can do (until Microsoft ports ZFS to Windows) is run Solaris 10 as your host OS (on your HP Proliant or IBM X series) with ZFS as the filesystem, then run Win2k3 and/or Win2k8 in VMs under xVM. This will not only protect you from SDC (and give you the other benefits of ZFS) all in a package [i]that is fully blessed by Microsoft![/i].
      Cool huh?
      914four
      • Disks already do ECC natively

        Disks already natively do ECC on a block-by-block basis, and correct several bits and detect errors in even more bits. They have been doing this in one form or another since at least the early '80s, even in the cheapest PC disks.

        I'm not sure what ZFS's CRC offers, except maybe an extra layer of protection. It might also standardize the level of correction, since it's my understanding that ZFS creates an unified file system across multiple heterogeneous devices, each of which might have a different error correction level.

        SDC is a real problem. It reflects the fact that bit error rates have not gone down as fast as data densities have gone up. If this trend continues disks will at some point become an unreliable storage medium that even the most sophisticated redundancy measures cannot overcome without severely compromising performance.
        zackers
  • Thank you Robin Harris, for giving Windows folks some ammunition.

    Robin,

    Perhaps ZFS blewup in Apple's face much in the way that WinFS blewup in Microsoft's face. With Sun's new direction and having to fix all of Leopard's problem, Apple couldn't get it to work. Or being at a mercy of a company just sold to Oracle, I would be leary too. No better or worse than Microsoft with WinFS.

    However, without a revolutionary file system, adding/copying features that has in Windows for years, and no truely revolutionary features. Snow Leopard is just a glorified service pack. A minor update to smooth out the mess in 10.5. Windows 7 goes so far above and beyond a mere service pack that its truely an evolutionary step foward for Windows users. So what if it borrows from Apple?

    Even Steve Jobs himself said, "Good artists copy, great artists steal."

    For all the features Snow Leopard takes from Windows, turnabout is fair play.
    Solid Jedi Knight
    • I think it was the Oracle thing.

      And Windows copies from OS X. Name something Snow Leopard took from Windows Vista that did not exist in Leopard. Remember, Snow Leopard is just Leopard Plus.
      No More Microsoft Software Ever!
      • I agree

        I personally believe the Oracle purchase is making folks nervous about the future of Sun.
        914four
  • Or Maybe...

    ZFS isn't the wunderkind everybody thought it was. Maybe Apple discovered a major problem with ZFS, one that's inherent to its design. Apple being Apple, we'll probably never know.

    Most likely tho, Sun's ingestion by Oracle was the culprit. Who wants an orphaned file system? Especially with Larry "Eat 'em all" Ellison in control of its destiny?
    wolf_z
    • ZFS is in large scale production now

      A show-stopper bug is unlikely as ZFS is in large-scale production now
      on Solaris and OpenSolaris. As an open source product Apple is free to
      work on the code no matter what happens with Oracle.

      If it wasn't purely NIH the problem of migrating millions of HFS+ users
      was the likely deal breaker.

      What Apple - or Microsoft - could do instead will be the topic of another
      post.

      Robin
      Robin Harris
      • And FAT32 and NTFS are still in production

        right now, even with all the issues you claim are there.

        So just because something is in production doesn't mean its 100% bug free.

        It makes more sense that Apple found a bug that Solaris and OpenSolaris may have found, but still have to continue on with in order to "validate" ZFS.

        Or maybe they missed it themselves, and are continuing on with a broken file system.
        John Zern
      • Who owns Sun Microsystems now?

        What are they developing? BtrFS...
        UsernameRequired
      • Licensing Problem?

        Robin:

        The latest scuttlebutt on this subject is the licensing problem Apple had with Sun. For more info, check out this link:

        http://www.macrumors.com/2009/09/01/lack-of-zfs-file-system-support-in-snow-leopard-due-to-licensing-issues/
        aj.redmond@...
    • Aren't Steve and Larry tight?

      As close to BFFs as egomaniacal CEOs can get from what I've heard. So to
      me that would be a plus as opposed to a negative, especially if Jonathan
      "My Little Pony" Schwartz pissed off [i]El Jobso[/i] at some point.
      Conspiracy theories aside, no one should discount Apple's tendency to
      abandon technologies it sees as unnecessary, obsolete or just plain
      clunky. Whether or not that's truly the case.
      matthew_maurice
  • RE: Apple kicks ZFS in the butt

    I went to the OpenSolaris blogs for zfs and they stopped
    right around the time Apple dropped support for zfs.
    http://www.opensolaris.org/os/community/zfs/blogs/

    rickdees@...
  • Does it makes sense for an "install in place" OS?

    Does it make sense for Apple's jive for Snow Leopard? They
    want you to "upgrade" Leopard - not a clean install, so how
    do you introduce ZFS?

    I do think it has a lot to do with the Sun/Oracle situation. I
    also wonder if Apple will brew their own filesystem inspired
    by ZFS, this would make a lot of sense as Apple's
    requirements for a filesystem are a little different (given their
    HFS past).
    jeremychappell
  • LOL, check out this previous post on ZDNet

    [i]Looks like the once much vaunted WinFS is still a no show.

    Maybe Microsoft should bit the bullet and licence ZFS from Sun, like Apple, and be done with it. We could then have a decent File System.
    [b]neil.postlethwaite@...[/b][/i]

    http://talkback.zdnet.com/5208-12558-0.html?forumID=1&threadID=50946&messageID=956811&start=0

    Oops...

    Just your typical double standards you'll find from your typical ZDNet reader.
    Qbt
    • Are you suggesting Microsoft would have dumped ZFS too?

      Sounds like Apple did the right thing rather than invest millions/billions only to end up with a turd (read XBox and Zune and Windows Mobile).
      No More Microsoft Software Ever!
      • LOL, no

        And you got the rest of your post wrong too.

        But thanks for playing...
        Qbt
  • Ext4 file system is why.

    The Ext4 file system that comes with the Karmic Koala version of Ubuntu is why. Look here to see what it offers.

    http://ext4.wiki.kernel.org/index.php/New_ext4_features

    Anything Sun touches it kills and I thought I would see ZFS out in the Linux world but Ext4 is much, much more powerful. The feature set of Ext4 is available now. The next version of Ubuntu has it built in and it is incredibly fast. The next generation server Ubuntu also uses this as its default. Apple, as usual, has no clue about technology and is waiting to have this OS instead of the ancient crap from Sun.
    progon
    • err..Ext4 is not an OS (NT)

      NT
      No More Microsoft Software Ever!
    • ext4 is nothing like ZFS, btrfs is though.

      ext4 is a Linux based file system, MacOSX is based upon the BSD operating system (it is actually a derivative of NextStep which ran on the Mach kernel). <a href=http://en.wikipedia.org/wiki/Ext4>ext4</a> does not have the capabilities of ZFS (created by Sun Microsystems, now owned by Oracle) or <a href=http://en.wikipedia.org/wiki/Btrfs>Btrfs</a> (also from Oracle, a closer equivalent in capabilities to <a href=http://www.opensolaris.org/os/community/zfs/whatis/>ZFS</a>), but ext3 ext4 and btrfs are all Linux based file systems. Further, ZFS is a 128-bit file system capable of handling storage up in excess of 16 exabytes.

      If you are going to comment of file systems, please at least learn about what they are and the OSs that they are associated with. Apple OSx is based upon BSD, not Linux!
      B.O.F.H.