Mac ZFS is dead: RIP.

PC file system progress stalled this week with the news on MacOSforge that Apple's ZFS project is dead.

ZFS Project Shutdown 2009-10-23 The ZFS project has been discontinued. The mailing list and repository will also be removed shortly.

ZFS, developed by Sun engineers, is the first 21st century file system. NTFS and HFS+ are firmly rooted in the 1980s. ZFS has a lot of cool features:

• End-to-end data integrity. Current file systems are prone to many problems - ranging from phantom writes to inconsistent error-handling - that mess up your data. The ZFS architecture eliminated them with parent block checksums.
• Pooled storage. Add a drive and it adds extra capacity, not another volume. Less management.
• No need for journaling. Which is one problem Solid State Drives don't handle well. Get rid of it and SSDs work better.
• Built-in RAID that is as fast as hardware RAID. Get data protection for a lower cost.
• Low-cost snapshot copy. As a copy-on-write system, ZFS can create new snapshots - once an hour, minute or second - with low CPU and storage overhead. Cruise back in time to just before the virus hit, recover, and life is good.

Apple announced in June '08 that Snow Leopard server would support ZFS. But things came apart early this year.

What happened? Jeff Bonwick, ZFS architect, posted Saturday on an earlier quoted comment:

> Apple can currently just take the ZFS CDDL code and incorporate it > (like they did with DTrace), but it may be that they wanted a "private > license" from Sun (with appropriate technical support and > indemnification), and the two entities couldn't come to mutually > agreeable terms.

I cannot disclose details, but that is the essence of it.

Jeff

Indemnification? Sun is being sued by NetApp, a $3B enterprise storage company, claiming that ZFS infringes on NetApp patents. If NetApp won, Apple would find itself in a tough position unless Sun shouldered the financial damage. That's indemnification.

Sun has made a (IMHO) strong case that NetApp's patents should be invalidated by prior art. But with all their other problems and the Oracle purchase it was a headache they and Oracle didn't need.

Where does Apple go from here? Apple has hired some smart file system engineers and wants to hire more to work on "state-of-the-art file system technologies for Mac OS X."

But writing new file systems isn't easy. It takes 5-7 years for a new file system to achieve the maturity needed to support large-scale deployment.

So if Apple is starting from scratch we have a long wait for real innovation to appear. Like Mac OS XII.

What about Microsoft? Redmond's file system gurus are well aware of NTFS issues. And under the covers they are making stepwise enhancements to the architecture and implementation.

But as the NTFS and HFS+ architectures age and the pace of storage innovation increases the gap between what is and what could be grows. It's like putting a 1001 hp Bugatti engine in a Model T: the power is there but you can't use it.

The Storage Bits take This kind of cock-up makes me hate software patents - but that's another post. As long as law allows companies will try to enforce them.

NetApp missed a golden opportunity to raise their visibility in the consumer market by cutting a deal with Apple directly. "NetApp is powering Apple's advanced storage technologies" would make the company a lot more visible outside the enterprise market.

NetApp is a good company, but they've lost their way lately. Note to new CEO Tom Georgens: with EMC moving aggressively into the consumer space you don't have forever to reposition NetApp for a consumer-driven world.

Steve Jobs doesn't get storage. Consumers are generating masses of video and photos at an accelerating pace - and they'll need reliable, available and dirt-easy storage. Lots of it.

Until the Next New Thing in file systems rolls out of Cupertino, Redmond or, maybe, Redwood City, consumers will stuck with too many BSODs, missing and corrupted files and app crashes. Let's hope we don't have to wait too many more years.

Comments welcome, of course. Update:There's now a Google Code page for MacZFS.