ZFS/Mac coming soon
Summary: Unix and Mac OS are designed to support different file systems - Mac folks may use 3 a day without knowing it - which is why ZFS on Mac is not only possible, but practical. And not only practical, but a giant step forward for data integrity and performance.
Unix and Mac OS are designed to support different file systems - Mac folks may use 3 a day without knowing it - which is why ZFS on Mac is not only possible, but practical. And not only practical, but a giant step forward for data integrity and performance. Here's what to expect.
A new company Ten's Complement is working to release ZFS for Mac. They're starting an engineering beta and plan to release a server-grade product later this year.
What is ZFS? ZFS is designed to protect, store and access data in the most demanding enterprise environments. Using standard, low-cost components: disk drives, enclosures, adapters, cables. No RAID arrays, volume managers, CDP, fsck, partitions, or volumes.
Almost makes you nostalgic for the good old days, doesn’t it? Like before Novocaine.
It is a 21st century open source file system developed at Sun with multiple cool features:
- Its tree structured checksums eliminates most of the bit rot that afflicts Macs and PCs. When ZFS retrieves your data, you can be sure it is your data, and not the misbegotten spawn of a driver burp.
- Add a disk drive to ZFS and it simply joins the pool of blocks available for storage. You don’t have to manage another disk.
- Cheap snapshots: roll your file system back to any point in time - like before you downloaded a malicious pdf - with almost no overhead.
- Fast, cheap RAID. ZFS implements very fast RAID that fixes the performance knock-off against software RAID. In ZFS all writes are the fastest kind: full stripe writes, running on the fastest processor in your system (your Mac), rather than some 3 year old microcontroller.
- Every time you add a disk to your Mac you see another disk volume on the desktop. ZFS eliminates the whole volume concept. Add a disk or five to your system and it joins your storage pool. More capacity. Not more management.
Apple's ZFS history Apple announced ZFS on Mac Server 10.6 in 2007, but Sun - whose engineers developed the open-source file system - put itself up for sale before license negotiations were concluded and Apple had to back down. NetApp's patent suit against Sun over some of the ZFS technology slowed things down as well.
So Apple de-committed from ZFS. Since then they've been hacking the ancient HFS+ like crazy to make it look cool. But it isn't cool - which is why we need ZFS on the Mac.
How will this work? Ten's Complement plans bring an enterprise-grade ZFS to Mac OS X. The founder, Don Brady, is the ex-Apple engineer who led the port of ZFS to Mac OS.
After 20 years at Apple he knows Mac OS and how Apple products should work.
The beta is fully subscribed with more volunteers than Ten's Complement could handle. The plan is to release a command line interface version - you'll need a sysadmin's comfort with the OS X CLI - with limited GUI support later this year.
After that? Well, there's no reason a slick Apple-style GUI couldn't be added. We'll have to wait and see.
Can you really add a file system to OS X? Sure. OS X is already plug & play with FAT 16/32, ExFAT, ISO 9660 on CDs, UDF on DVDs, as well as HFS+. You also used to be able to configure a Unix FS from Disk Utility, but no more.
NTFS could be on the list, but it's a moving target with a raft of improvements due out in Q4 (see How Microsoft puts your data at risk for why). And, of course, Apple offers Quantum's StorNext cluster FS as XSAN.
Yes, you can add file systems to OS X.
The Storage Bits take Once Ten's Complement gets a consumer-friendly product to market I'll try it. I've lost hard-to-replace files due to HFS+ data corruption and I'm not happy about it.
The Mac software group should re-think their reliance on HFS+: the Microsoft NTFS team - some very smart guys there - will be rolling out improvements later this year. As data stores continue to grow, file system failures will become more obvious and more irritating.
If your Mac is business critical, ZFS on OS X will help keep you up and running. It's too bad Apple dropped the ball, but I'm glad Ten's Complement has picked it up.
Comments welcome, of course. Learn more about ZFS: ZFS: Threat or Menace? Apple's new kick-butt file system ZFS data integrity tested And thanks to David Morgenstern for alerting me that ZFS returns to the Mac.
Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.
Talkback
The problem with NTFS is that Microsoft doesn't license
RE: ZFS/Mac coming soon
Check out tuxera.com and their NTFS for Mac driver. I don't think Microsoft would mind licensing Apple directly. What do they have to lose?
Robin
RE: ZFS/Mac coming soon
Microsoft making money from Apple. What's the problem there? Seems like a win-win for MS if you ask me.
How about performance?
RE: ZFS/Mac coming soon
Good question. I asked Don to respond:
"If he's been using HFS+, using multiple disks is essential for parallel metadata access scalability (server). HFS+ has a global lock for the journal and catalog b-tree ? only one metadata updater at a time. If he is streaming to already open, already allocated files (more likely the case), then splitting them across devices is likely just helping mitigate the seek penalties since the metadata updates would be minimal in that scenario. If you used a striped disk array with a single HFS+ file system, you wouldn't be able to leverage all the aggregate bandwidth since HFS+ doesn't know about the lower I/O layers -- it can issue asynchronous I/O but only to a certain point.
So does he have to split HFS+ across devices to get acceptable bandwidth? If so, perhaps his file system isn't up to the task. =)
On the other hand, ZFS is built to optimize the writes across all the devices (vdevs). So you wouldn't need to segment your file systems to utilize the bandwidth of the devices in your pool. It will be a simpler config (no up front guessing on how to provision the space across multiple discrete file systems and disks) - it's one pool that the ZFS I/O stack can optimize for.
There's nothing to prevent you for creating a separate pool for each device, but that would trade administration complexity for a minimal bandwidth gain.
One advantage that HFS+ does have WRT performance, is in direct/in-place I/O. Since it is writing data in place you can stream directly to the disk (DMA) and avoid the buffering overhead in the file system layer. If speed trumps data integrity, then that might be OK. There will always be working sets where direct I/O will be faster on HFS+, so if speed is king, then stick with HFS+. If you care about the integrity of your data however, then it's a no brainer to trade up to ZFS. I can design a file system that is blazingly fast if data integrity is unimportant."
In RAID terms, ZFS RAIDZ write full stripe writes - the fastest kind - all the time. Bing "ZFS Performance Versus Hardware RAID" to learn more.
RE: ZFS/Mac coming soon
Thanks for looking into this more deeply, Robin. In many media authoring situations, it's not so much about write efficiency as it is read efficiency. A musical sampling synthesizer may be reading from any of thousands of open sample files, based on the keys that are struck and the intensity with which they are struck. Same is true when assembling video effects. The write portion is actually much simplified, since all of this input is typically combined into just a few channels. So the majority of file system optimization in these cases has more to do with seek latency than any other factors. And of course data integrity remains important. Thanks again.
Curious
Sounds like another opportunity to outflank M$ to me.
Message has been deleted.
Message has been deleted.
RE: ZFS/Mac coming soon
Thanks for the article!
Add/Remove
Drives are not automatically added to existing pools
Based on my experience using ZFS with OpenSolaris, physical drives are not automatically added into ZFS pools. For removable media like USB thumbdrives, those drives will remain independent.
For what it is worth, you *could* build a ZFS pool using USB thumbdrives, but I can't think of any worthwhile reason. (But I like the silliness of putting those numerous promotional drives that I've got laying about to some use, even though I'd need to add some 50+ USB ports!)
RE: ZFS/Mac coming soon
Good question. Also what happens when a hard drive in the pool goes out and has to be replaced? How do you know what needs to be restored?
And backup. What about time capsule? If you have several hard drives, it can easily create a pool greater than time capsule hard drive capacity making the time capsule unable to backup that much data because the data exceeds its capacity.
RE: ZFS/Mac coming soon
If you're using RAIDZ you don't need to know what to restore because the file system will do that for you, just as a regular RAID rebuild.
Building a system disk bigger than your backup capacity would be silly, so don't do it. As it is, Time Capsule is a non-redundant and easily corrupted tool, so I don't recommend its use.
Robin
Kudos to Apple Development Engineers
Add to that, as pointed out by RH, that (by design) it is not prone (..or far less prone) to the bit-rot inherent in FAT* and NTFS file systems and its benefits become a real selling point.
This can only be the beginning of a great leap forward / boon for the hardware architecture in Macs.
... now if only Apple would take software integrity / security more seriously. ;)
RE: ZFS/Mac coming soon
RE: ZFS/Mac coming soon
Hilariously insightful wording...
I use zfs
RE: ZFS/Mac coming soon
RE: ZFS/Mac coming soon