Apple's new kick-butt file system pushes on

Fixing data corruption before it starts Apple is moving quickly ahead with their plan to give Mac users the most reliable data storage on any desktop with the new file system ZFS. According to MacRumors the latest developers build of OS X Leopard includes V1.1 of ZFS.

ZFS is the first desktop file system with true end-to-end data integrity. Thanks to sophisticated tree-based checksums it detects and corrects silent data corruption anywhere in the data path: disks, cables, interfaces and more.

Apple's Mac OS 10.5.0 - Leopard - is expected to be released late this month. It will include a read-only version of ZFS. The V1.1 ZFS in the developer's preview includes full read/write capability. This suggests that by the end of February, ZFS will be available (in 10.5.1) for users with large data pools.

While there is no word yet on when Mac users will be able to boot from ZFS - thus enabling its use on single disk systems such as notebooks and small desktops - I predict that in summer '08.

Why ZFS is important If you've ever had to reinstall an application, operating system or a driver that suddenly wouldn't work or had a "file not found" error, you've probably been a victim of silent data corruption. It is hard to tell since no operating system has an error message for it.

As I've documented (see Data corruption is worse than you know, How Microsoft puts your data at risk, 50 ways to lose your data and How data gets lost) silent data corruption is a real problem with today's desktop and server file systems. And it's getting worse as disk drives capacities grow.

Microsoft's aging NTFS can't compete Once ZFS becomes the default file system for Mac OS X, all the average user will notice is that their Mac is even more reliable. Other ZFS features, such as RAID Z, will mostly be of interest to media and scientific professionals with very large data sets.

In the server world though, all the Windows buyers will have to explain why they prefer a system with more data corruption to one that is provably more reliable. And big corporate customers will have another cudgel to beat price reductions out of Microsoft.

Ouch!

The Storage Bits take ZFS, an open-source product developed by a small team of smart Sun engineers - I know, I've met them - is a game changer for the industry. All popular file systems have had roughly the same level of data integrity so no one mentioned the problem.

Now, however, Sun and Apple will soon have the best data integrity story in the industry. Apple's Time Machine will give Mac the best back up story as well. Microsoft and the Linux server folks will have to respond. We, the users, will win, no matter which platform we prefer.

In the meantime, Apple and Sun will make hay while competitors play catch up.

Update: Microsoft fans, I presume, having forgotten how enthusiasm for new software feels, have intimated that I must have a financial or business relationship with Sun, Apple, ZFS or something. Sadly, I don't. My condolences to those whose software updates inspire fear and dread.

Update II: If you want to learn more from a "dirt-under-the-fingernails" engineer about ZFS, check out this post from file system engineer Drew Thaler Don't be a ZFS hater.

Comments welcome, as always.