ZFS/Mac coming soon

ZFS/Mac coming soon

Summary: Unix and Mac OS are designed to support different file systems - Mac folks may use 3 a day without knowing it - which is why ZFS on Mac is not only possible, but practical. And not only practical, but a giant step forward for data integrity and performance.

SHARE:
TOPICS: Apple, Hardware
18

Unix and Mac OS are designed to support different file systems - Mac folks may use 3 a day without knowing it - which is why ZFS on Mac is not only possible, but practical. And not only practical, but a giant step forward for data integrity and performance. Here's what to expect.

A new company Ten's Complement is working to release ZFS for Mac. They're starting an engineering beta and plan to release a server-grade product later this year.

What is ZFS? ZFS is designed to protect, store and access data in the most demanding enterprise environments. Using standard, low-cost components: disk drives, enclosures, adapters, cables. No RAID arrays, volume managers, CDP, fsck, partitions, or volumes.

Almost makes you nostalgic for the good old days, doesn’t it? Like before Novocaine.

It is a 21st century open source file system developed at Sun with multiple cool features:

  • Its tree structured checksums eliminates most of the bit rot that afflicts Macs and PCs. When ZFS retrieves your data, you can be sure it is your data, and not the misbegotten spawn of a driver burp.
  • Add a disk drive to ZFS and it simply joins the pool of blocks available for storage. You don’t have to manage another disk.
  • Cheap snapshots: roll your file system back to any point in time - like before you downloaded a malicious pdf - with almost no overhead.
  • Fast, cheap RAID. ZFS implements very fast RAID that fixes the performance knock-off against software RAID. In ZFS all writes are the fastest kind: full stripe writes, running on the fastest processor in your system (your Mac), rather than some 3 year old microcontroller.
  • Every time you add a disk to your Mac you see another disk volume on the desktop. ZFS eliminates the whole volume concept. Add a disk or five to your system and it joins your storage pool. More capacity. Not more management.

Apple's ZFS history Apple announced ZFS on Mac Server 10.6 in 2007, but Sun - whose engineers developed the open-source file system - put itself up for sale before license negotiations were concluded and Apple had to back down. NetApp's patent suit against Sun over some of the ZFS technology slowed things down as well.

So Apple de-committed from ZFS. Since then they've been hacking the ancient HFS+ like crazy to make it look cool. But it isn't cool - which is why we need ZFS on the Mac.

How will this work? Ten's Complement plans bring an enterprise-grade ZFS to Mac OS X. The founder, Don Brady, is the ex-Apple engineer who led the port of ZFS to Mac OS.

After 20 years at Apple he knows Mac OS and how Apple products should work.

The beta is fully subscribed with more volunteers than Ten's Complement could handle. The plan is to release a command line interface version - you'll need a sysadmin's comfort with the OS X CLI - with limited GUI support later this year.

After that? Well, there's no reason a slick Apple-style GUI couldn't be added. We'll have to wait and see.

Can you really add a file system to OS X? Sure. OS X is already plug & play with FAT 16/32, ExFAT, ISO 9660 on CDs, UDF on DVDs, as well as HFS+. You also used to be able to configure a Unix FS from Disk Utility, but no more.

NTFS could be on the list, but it's a moving target with a raft of improvements due out in Q4 (see How Microsoft puts your data at risk for why). And, of course, Apple offers Quantum's StorNext cluster FS as XSAN.

Yes, you can add file systems to OS X.

The Storage Bits take Once Ten's Complement gets a consumer-friendly product to market I'll try it. I've lost hard-to-replace files due to HFS+ data corruption and I'm not happy about it.

The Mac software group should re-think their reliance on HFS+: the Microsoft NTFS team - some very smart guys there - will be rolling out improvements later this year. As data stores continue to grow, file system failures will become more obvious and more irritating.

If your Mac is business critical, ZFS on OS X will help keep you up and running. It's too bad Apple dropped the ball, but I'm glad Ten's Complement has picked it up.

Comments welcome, of course. Learn more about ZFS: ZFS: Threat or Menace? Apple's new kick-butt file system ZFS data integrity tested And thanks to David Morgenstern for alerting me that ZFS returns to the Mac.

Topics: Apple, Hardware

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

18 comments
Log in or register to join the discussion
  • The problem with NTFS is that Microsoft doesn't license

    it for write access. Do you really think they're going to sell Apple a license?
    fr_gough
    • RE: ZFS/Mac coming soon

      @frgough@...
      Check out tuxera.com and their NTFS for Mac driver. I don't think Microsoft would mind licensing Apple directly. What do they have to lose?

      Robin
      Robin Harris
    • RE: ZFS/Mac coming soon

      @frgough@...

      Microsoft making money from Apple. What's the problem there? Seems like a win-win for MS if you ask me.
      Bates_
  • How about performance?

    ZFS is intriguing, but I wonder if you can set it up for high performance. In media-authoring, you often intentionally put data on different disks to increase throughput. Source video here, source audio there, result product elsewhere. Can ZFS be put together in such a way?
    MC_z
    • RE: ZFS/Mac coming soon

      @MC_z
      Good question. I asked Don to respond:

      "If he's been using HFS+, using multiple disks is essential for parallel metadata access scalability (server). HFS+ has a global lock for the journal and catalog b-tree ? only one metadata updater at a time. If he is streaming to already open, already allocated files (more likely the case), then splitting them across devices is likely just helping mitigate the seek penalties since the metadata updates would be minimal in that scenario. If you used a striped disk array with a single HFS+ file system, you wouldn't be able to leverage all the aggregate bandwidth since HFS+ doesn't know about the lower I/O layers -- it can issue asynchronous I/O but only to a certain point.

      So does he have to split HFS+ across devices to get acceptable bandwidth? If so, perhaps his file system isn't up to the task. =)

      On the other hand, ZFS is built to optimize the writes across all the devices (vdevs). So you wouldn't need to segment your file systems to utilize the bandwidth of the devices in your pool. It will be a simpler config (no up front guessing on how to provision the space across multiple discrete file systems and disks) - it's one pool that the ZFS I/O stack can optimize for.

      There's nothing to prevent you for creating a separate pool for each device, but that would trade administration complexity for a minimal bandwidth gain.

      One advantage that HFS+ does have WRT performance, is in direct/in-place I/O. Since it is writing data in place you can stream directly to the disk (DMA) and avoid the buffering overhead in the file system layer. If speed trumps data integrity, then that might be OK. There will always be working sets where direct I/O will be faster on HFS+, so if speed is king, then stick with HFS+. If you care about the integrity of your data however, then it's a no brainer to trade up to ZFS. I can design a file system that is blazingly fast if data integrity is unimportant."

      In RAID terms, ZFS RAIDZ write full stripe writes - the fastest kind - all the time. Bing "ZFS Performance Versus Hardware RAID" to learn more.
      Robin Harris
      • RE: ZFS/Mac coming soon

        @Robin Harris
        Thanks for looking into this more deeply, Robin. In many media authoring situations, it's not so much about write efficiency as it is read efficiency. A musical sampling synthesizer may be reading from any of thousands of open sample files, based on the keys that are struck and the intensity with which they are struck. Same is true when assembling video effects. The write portion is actually much simplified, since all of this input is typically combined into just a few channels. So the majority of file system optimization in these cases has more to do with seek latency than any other factors. And of course data integrity remains important. Thanks again.
        MC_z
  • Curious

    A suitable configuration of Lightpeak fabric, ZFS and commodity disks would surely blow the Enterprise Data Storage market apart ... yet did I not read that Apple have withdrawn their Xserve product?

    Sounds like another opportunity to outflank M$ to me.
    jacksonjohn
  • Message has been deleted.

    dfhjtykyk
  • Message has been deleted.

    Wholesale Supplier
  • RE: ZFS/Mac coming soon

    Absolutely wonderful. Apple has smart people. They knew that ZFS is better but couldn't bring it on board for obvious reasons. I run ZFS on Sun servers and nothing else will do. Will be re-doing the root partitions soon with ZFS root in an OS upgrade. The boost in speed, integrity and ease of management is going to be worth every effort put into it. I wonder how much it will cost if anything?
    Thanks for the article!
    rpollard@...
  • Add/Remove

    A lot of Mac users use different disks for different purposes, at different times; a USB external for backup, e-sata external RAID for video editing or *big* Photoshop/LR/Aperture/Capture One jobs. Disks come and go; USB home for backup storage, another to the bureau for printing or duplicating, or a 4GB USB key to give to a client whose mail system doesnt accept large file transfers. Never mind the external startup disks that can be moved from machine to machine/site to site. And Thunderbolt will add to the connection/storage options<br><br>What happens when you want to eject that 4GB drive? It?s been made part of the ZFS pool, so how do you tell the OS to eject *it* rather than another disk, and how long does it take to write that data to elsewhere in the pool? [I understand there is some redundancy/spanning inherent in ZFS, but what does it mean in practice?].<br><br>I believe ZFS is a supremely capable and -featured file system, but it seems to have its best application in an enterprise system with relatively static disk utilisation. Prove me wrong, though!
    mediumcool
    • Drives are not automatically added to existing pools

      @mediumcool
      Based on my experience using ZFS with OpenSolaris, physical drives are not automatically added into ZFS pools. For removable media like USB thumbdrives, those drives will remain independent.

      For what it is worth, you *could* build a ZFS pool using USB thumbdrives, but I can't think of any worthwhile reason. (But I like the silliness of putting those numerous promotional drives that I've got laying about to some use, even though I'd need to add some 50+ USB ports!)
      R_Connelie@...
    • RE: ZFS/Mac coming soon

      @mediumcool
      Good question. Also what happens when a hard drive in the pool goes out and has to be replaced? How do you know what needs to be restored?

      And backup. What about time capsule? If you have several hard drives, it can easily create a pool greater than time capsule hard drive capacity making the time capsule unable to backup that much data because the data exceeds its capacity.
      Dalece
      • RE: ZFS/Mac coming soon

        @Dalece
        If you're using RAIDZ you don't need to know what to restore because the file system will do that for you, just as a regular RAID rebuild.

        Building a system disk bigger than your backup capacity would be silly, so don't do it. As it is, Time Capsule is a non-redundant and easily corrupted tool, so I don't recommend its use.

        Robin
        Robin Harris
  • Kudos to Apple Development Engineers

    for recognizing another great idea. But obviously, ZFS goes beyond being simply 'a good idea' it is clearly an excellent and far more integrity-based file system than most others available.

    Add to that, as pointed out by RH, that (by design) it is not prone (..or far less prone) to the bit-rot inherent in FAT* and NTFS file systems and its benefits become a real selling point.

    This can only be the beginning of a great leap forward / boon for the hardware architecture in Macs.

    ... now if only Apple would take software integrity / security more seriously. ;)
    thx-1138_
  • RE: ZFS/Mac coming soon

    @Robin Harris <br>You note that "..NTFS could be on the list, but its a moving target with a raft of improvements due out in Q4.."<br><br>Is there any available and current documentation that describes this in more detail? The blog you reference is dated August 2007.. Quite sometime ago..
    PJDuro
  • RE: ZFS/Mac coming soon

    "you can be sure it is your data, and not the misbegotten spawn of a driver burp."

    Hilariously insightful wording...
    flurrball
  • I use zfs

    I use zfs with openindiana as a time machine backup of my Mac.. Data is mirrored between two drives. One was a WD. It died (surprise). I replaced it with another drive and it resilvered (replicated) perfectly without having to turn off the machine or even reboot and didn't loose any data in the process. Go ZFS!!!
    Matt Connolly