Mac ZFS is dead: RIP.

Mac ZFS is dead: RIP.

Summary: Apple made it official this week: the innovative ZFS file system for Mac OS X is dead. Who killed it?

SHARE:
47

PC file system progress stalled this week with the news on MacOSforge that Apple's ZFS project is dead.

ZFS Project Shutdown 2009-10-23 The ZFS project has been discontinued. The mailing list and repository will also be removed shortly.

ZFS, developed by Sun engineers, is the first 21st century file system. NTFS and HFS+ are firmly rooted in the 1980s. ZFS has a lot of cool features:

  • End-to-end data integrity. Current file systems are prone to many problems - ranging from phantom writes to inconsistent error-handling - that mess up your data. The ZFS architecture eliminated them with parent block checksums.
  • Pooled storage. Add a drive and it adds extra capacity, not another volume. Less management.
  • No need for journaling. Which is one problem Solid State Drives don't handle well. Get rid of it and SSDs work better.
  • Built-in RAID that is as fast as hardware RAID. Get data protection for a lower cost.
  • Low-cost snapshot copy. As a copy-on-write system, ZFS can create new snapshots - once an hour, minute or second - with low CPU and storage overhead. Cruise back in time to just before the virus hit, recover, and life is good.

Apple announced in June '08 that Snow Leopard server would support ZFS. But things came apart early this year.

What happened? Jeff Bonwick, ZFS architect, posted Saturday on an earlier quoted comment:

> Apple can currently just take the ZFS CDDL code and incorporate it > (like they did with DTrace), but it may be that they wanted a "private > license" from Sun (with appropriate technical support and > indemnification), and the two entities couldn't come to mutually > agreeable terms.

I cannot disclose details, but that is the essence of it.

Jeff

Indemnification? Sun is being sued by NetApp, a $3B enterprise storage company, claiming that ZFS infringes on NetApp patents. If NetApp won, Apple would find itself in a tough position unless Sun shouldered the financial damage. That's indemnification.

Sun has made a (IMHO) strong case that NetApp's patents should be invalidated by prior art. But with all their other problems and the Oracle purchase it was a headache they and Oracle didn't need.

Where does Apple go from here? Apple has hired some smart file system engineers and wants to hire more to work on "state-of-the-art file system technologies for Mac OS X."

But writing new file systems isn't easy. It takes 5-7 years for a new file system to achieve the maturity needed to support large-scale deployment.

So if Apple is starting from scratch we have a long wait for real innovation to appear. Like Mac OS XII.

What about Microsoft? Redmond's file system gurus are well aware of NTFS issues. And under the covers they are making stepwise enhancements to the architecture and implementation.

But as the NTFS and HFS+ architectures age and the pace of storage innovation increases the gap between what is and what could be grows. It's like putting a 1001 hp Bugatti engine in a Model T: the power is there but you can't use it.

The Storage Bits take This kind of cock-up makes me hate software patents - but that's another post. As long as law allows companies will try to enforce them.

NetApp missed a golden opportunity to raise their visibility in the consumer market by cutting a deal with Apple directly. "NetApp is powering Apple's advanced storage technologies" would make the company a lot more visible outside the enterprise market.

NetApp is a good company, but they've lost their way lately. Note to new CEO Tom Georgens: with EMC moving aggressively into the consumer space you don't have forever to reposition NetApp for a consumer-driven world.

Steve Jobs doesn't get storage. Consumers are generating masses of video and photos at an accelerating pace - and they'll need reliable, available and dirt-easy storage. Lots of it.

Until the Next New Thing in file systems rolls out of Cupertino, Redmond or, maybe, Redwood City, consumers will stuck with too many BSODs, missing and corrupted files and app crashes. Let's hope we don't have to wait too many more years.

Comments welcome, of course. Update:There's now a Google Code page for MacZFS.

Topics: Apple, Hardware, Oracle, Storage

About

Robin Harris has been a computer buff for over 35 years and selling and marketing data storage for over 30 years in companies large and small.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

47 comments
Log in or register to join the discussion
  • fortunately

    while zfs is cool, there are enough other file systems, so we users won't miss it much
    ljenux-23043766007667558234416105604265
    • NTFS

      Actualy has similar features to most of these but maybe its not implemented as well.

      jdbukis@...
      • Please, NTFS isn't even close.

        I get that it can be difficult for civilians to appreciate the differences, but
        end-to-end data integrity, pooling and low-cost snapshots are major
        plusses.

        NTFS and HFS+ aren't even close.

        Robin
        R Harris
      • NTFS is impressive, but indeed too old

        Its design is sound, and 7 doesn't even allow you to make use of all its capabilities (note: 7's NTFS version is unchanged from XP's; it's merely making use of more features).

        But I have to agree with other posters: NTFS isn't well optimized for large storage jobs (larger storage = more CPU use), can't guarantee data integrity, has no redundant storage capabilities, no copy-on-write capabilities...

        So, ZFS is going down. Well, that makes Btrfs on Linux ever more tempting...
        Mitch 74
        • Nonsense!

          A filesystem is as good as the driver implementing it. For example, NTFS doesn't support case sensitivity... well, actually, it does, but Win32 does not, and hence the usual APIs for file and directory access do not enforce case sensitivity. Ah, but run Interix and suddenly case sensitivity works just fine.

          In this instance, saying that ZFS supports this and that and the other thing is true [i]so long as the underlying driver and APIs expose that support[/i]. Most Ext2 implementations on Win32 don't enforce ACLs, for instance, and none of them use Ext3 journaling. NTFS can also support any number of ad hoc and arbitrary 'features' in this manner. Case in point, VSS was not really supported until Vista.

          I would argue that guaranteeing data integrity, redundant storage and copy-on-write are not functions of the filesystem per se, and should be enforced/implemented at the driver level. Now, you can argue that they should be part of the filesystem [i]specification[/i]... I'd agree. But seeing as NTFS is a black box, who knows what's really part of the spec and what isn't?
          Da-G
          • Data integrity IS a function of a file system

            I can't agree that "... guaranteeing data integrity, redundant storage
            and copy-on-write are not functions of the filesystem per se ...."

            Really? The first is a basic function of a file system, though not well
            implemented in many of today's offerings.

            The second is what people pay big bucks for in a RAID system. If the
            file system can do it as fast, better and cheaper, why not?

            The third is both a performance and a data integrity tool.

            Bottom line: many of the structures we live with today, such as
            separate file system and volume management, are artifacts of history,
            not well-considered engineering decisions. They make our systems
            more complex and less reliable. Lose 'em!

            Robin
            R Harris
          • Data integrity starts at the lowest level

            As someone who has written firmware for storage devices, I can assure you that data integrity starts at the lowest levels, down to the physical device. While the actual device can't provide the redundancy needed by modern file systems, if we didn't care about returning as accurate data as possible all file system schemes would be impossible.

            Just look at the looming crisis caused by error rates on disk media not decreasing as fast as data density is increasing. If nothing is done to solve this problem, even the most sophisticated file systems will be prone to errors or severely degraded performance.
            zackers
      • Yes, just like a 1932 Ford has a lot of common features with a 2009 Ferrari

        Not even in the same Century my friend. Read before you post such things.
        914four
        • knee-jerk Redmond fanbuism

          It's a disease on zdnet. No easy cure except extensive deprogramming...
          Wintel BSOD
    • Re: fortunately

      Spoken like someone that has never used ZFS. I've been using it since build 46 of OpenSolaris. It has saved data loss effortlessly through several catastrophic UPS, controller, and disk failures.

      Slapping a new disk in and not worry about partitioning the file system, SSD cache support, the merits of ZFS goes on and on. I was hoping to use some OS/X servers in the future. I've scrapped that idea as soon as I heard they scrapped ZFS.
      GAGendel
    • Heh

      Well "users" might not miss it, my Granny always said: "You can't miss
      what you never had" - in that sense you're right.

      But in every other sense, you're just flat out wrong. ZFS is a wonderful
      filesystem, however it's not ideal for Apple.

      ZFS is tuned for two things that make it a less than ideal fit for Apple:

      1) Big deployments
      2) Spinning Disks

      Macs mostly have one honking big disk (SLED) ZFS is meant for lots of
      disks, that's where it really makes sense. I can't see an iMac with ten
      to twenty disks. I can see Apple putting in a pair - but that's hardly
      ZFS territory. ZFS is a product from a "big iron" company (Sun) Apple
      don't make "big iron"... This is a bad fit.

      ZFS was born on spinning disks, the world is moving to SSDs. Sure
      we're still a little way off an iMac with SSDs - but it'll happen (if I were
      a betting man I'd say by the end of 2012 - at least as an option). Now
      ZFS can use Flash based storage - but that's "added in" as an
      accelerator for spinning disks, ZFS isn't tuned for it. If Apple are going
      to build one (or adapt something "like" ZFS) then it would make sense
      to bake in SSD support, essentially assume it'll be deployed on SSD
      not spinning disks.

      So is this bad news for end users of the Mac platform? Yes - short
      term. What about long term? Depends, can Apple do a good job of
      their brave new replacement to HFS+? Who'd bet against Apple? Not
      me, but success is far from certain.
      jeremychappell
  • How about EXT4 and BTRFS?

    What do you think of EXT4 and it's destined successor, BTRFS?
    pjotr123
    • Both have interesting properties

      and I haven't done a deep dive on either. But neither is as mature as ZFS
      is today and won't be for some time.

      ZFS is used in production in a number of shops - including some large
      scale ones. That is valuable experience and validation.

      Robin
      R Harris
      • The coming period will be interesting

        BTRFS is still heavily in development, of course. And will be for years.

        But EXT4 is going mainstream now, as it's the default file system for Ubuntu 9.10 (which will be released in two days). That'll put it to the test on millions of computers.

        And in six months, EXT4 will be put to the test in corporate use as well: then Ubuntu 10.04 LTS will be released, which is aiming at businesses too (being Long Term Support).

        We'll see. It'll be worthwhile to follow this development closely. :-)

        Regards, Pjotr.
        pjotr123
        • Yes, bring on Ubuntu 9.10

          Ubuntu 9.10 and other distros with ext 4 is an
          improvement over ext 3. I am hoping for btrfs
          before Microsoft updates their file system.

          root12
        • Ditto for Mandriva 2010 ...

          Ext4 is default file system format.
          George Mitchell
      • Ext4 is a stable production file system at this point ...

        Ext4 is rock solid at this point. I have it running on multiple systems and it is flat out reliable and probably one of the best file systems out there at this point. But, granted, it doesn't have many of the advanced features of ZFS. Btrfs, on the other hand, is likely going to be more advanced than ZFS, but indeed has a long way to go before it can be considered anywhere near stable. Linux distros are rapidly moving to ext4 now. It is the default on Mandriva 2010 for example. And btrfs is the planned replacement for ext4.
        George Mitchell
  • Apple picks NetApp WAFL over Sun ZFS

    If you walk around Apple's Data Centers, you will see ton's of NetApp filers. PetaBytes & PetaBytes of them.

    iTunes, MobileMe and all sorts of Apple services are powered by NetApp.

    Apple has already voted with their dollars. They chose NetApp for practical cost-effective reasons, instead of free-love open-source ideology.
    Storage101
    • Means nothing...

      Unless Apple is adapting NetApp to run in OSX, it means nothing. What they run in their server farms has nothing to do with the core OS that ships on Macs. ZFS for Mac (and all forms of ZFS) may be dead, but wouldn't be for lack of trying (unless Jobs finally killed it after the ZFS-announcement fiasco that occurred before Leopard was launched).

      Changing the file system is like giving someone a new heart: You have to nurse it along and you probably won't be able to be as active as you previously were with the old one and one wrong move can kill you.

      I didn't know HFS+ was broken. Apple's not moving to NTFS and a lot of computer core technologies are based on 80s concepts. Being old and mature isn't the problem. Not adapting to handle new technologies (like multi-TB HDDs) is a problem.

      So what's next?
      Eriamjh
    • Hardly

      Which is why they announced WAFL on Snow Leopard Server, right?

      Tell me another!

      Robin
      R Harris