ie8 fix

ZFS/Mac coming soon

By | March 17, 2011, 4:32pm PDT

Unix and Mac OS are designed to support different file systems - Mac folks may use 3 a day without knowing it - which is why ZFS on Mac is not only possible, but practical. And not only practical, but a giant step forward for data integrity and performance. Here’s what to expect.

A new company
Ten’s Complement is working to release ZFS for Mac. They’re starting an engineering beta and plan to release a server-grade product later this year.

What is ZFS?
ZFS is designed to protect, store and access data in the most demanding enterprise environments. Using standard, low-cost components: disk drives, enclosures, adapters, cables. No RAID arrays, volume managers, CDP, fsck, partitions, or volumes.

Almost makes you nostalgic for the good old days, doesn’t it? Like before Novocaine.

It is a 21st century open source file system developed at Sun with multiple cool features:

  • Its tree structured checksums eliminates most of the bit rot that afflicts Macs and PCs. When ZFS retrieves your data, you can be sure it is your data, and not the misbegotten spawn of a driver burp.
  • Add a disk drive to ZFS and it simply joins the pool of blocks available for storage. You don’t have to manage another disk.
  • Cheap snapshots: roll your file system back to any point in time - like before you downloaded a malicious pdf - with almost no overhead.
  • Fast, cheap RAID. ZFS implements very fast RAID that fixes the performance knock-off against software RAID. In ZFS all writes are the fastest kind: full stripe writes, running on the fastest processor in your system (your Mac), rather than some 3 year old microcontroller.
  • Every time you add a disk to your Mac you see another disk volume on the desktop. ZFS eliminates the whole volume concept. Add a disk or five to your system and it joins your storage pool. More capacity. Not more management.

Apple’s ZFS history
Apple announced ZFS on Mac Server 10.6 in 2007, but Sun - whose engineers developed the open-source file system - put itself up for sale before license negotiations were concluded and Apple had to back down. NetApp’s patent suit against Sun over some of the ZFS technology slowed things down as well.

So Apple de-committed from ZFS. Since then they’ve been hacking the ancient HFS+ like crazy to make it look cool. But it isn’t cool - which is why we need ZFS on the Mac.

How will this work?
Ten’s Complement plans bring an enterprise-grade ZFS to Mac OS X. The founder, Don Brady, is the ex-Apple engineer who led the port of ZFS to Mac OS.

After 20 years at Apple he knows Mac OS and how Apple products should work.

The beta is fully subscribed with more volunteers than Ten’s Complement could handle. The plan is to release a command line interface version - you’ll need a sysadmin’s comfort with the OS X CLI - with limited GUI support later this year.

After that? Well, there’s no reason a slick Apple-style GUI couldn’t be added. We’ll have to wait and see.

Can you really add a file system to OS X?
Sure. OS X is already plug & play with FAT 16/32, ExFAT, ISO 9660 on CDs, UDF on DVDs, as well as HFS+. You also used to be able to configure a Unix FS from Disk Utility, but no more.

NTFS could be on the list, but it’s a moving target with a raft of improvements due out in Q4 (see How Microsoft puts your data at risk for why). And, of course, Apple offers Quantum’s StorNext cluster FS as XSAN.

Yes, you can add file systems to OS X.

The Storage Bits take
Once Ten’s Complement gets a consumer-friendly product to market I’ll try it. I’ve lost hard-to-replace files due to HFS+ data corruption and I’m not happy about it.

The Mac software group should re-think their reliance on HFS+: the Microsoft NTFS team - some very smart guys there - will be rolling out improvements later this year. As data stores continue to grow, file system failures will become more obvious and more irritating.

If your Mac is business critical, ZFS on OS X will help keep you up and running. It’s too bad Apple dropped the ball, but I’m glad Ten’s Complement has picked it up.

Comments welcome, of course. Learn more about ZFS:
ZFS: Threat or Menace?
Apple’s new kick-butt file system
ZFS data integrity tested
And thanks to David Morgenstern for alerting me that ZFS returns to the Mac.

Kick off your day with ZDNet's daily e-mail newsletter. It's the freshest tech news and opinion, served hot. Get it.

Topics

Robin Harris has been messing with computers for over 30 years and selling and marketing data storage for over 20 in companies large and small.

Disclosure

Robin Harris

Robin Harris is a president of TechnoQWAN, a consulting and analyst firm in northern Arizona. He also writes StorageMojo.com, a blog which accepts advertising from companies in the storage industry, and has a 25 year history with IT vendors. He has many industry contacts, many of whom are friends and all of whom he has opinions about. Robin has relationships with many companies in the technology industry. Every company he writes about may have sought to influence his opinion through carefully-crafted marketing messages and self-serving white papers, gifts ranging from desk calendars, t-shirts, lunches and trips as well as analyst or consulting assignments. He also invests in some technology companies. He may accept payment for services in stock as well. Robin discloses financial investments in or client relationships with companies named in Storage Bits. To help readers sort out the gold from the dross in his writings, Robin tries to communicate his reasons as clearly as he can. If you agree, you are intelligent and discerning. If you disagree, well, you disagree. In all cases, Robin encourages readers to subject everything they read, see or hear on the internet or from politicians to some simple questions: * What assumptions are implicit in the world view and judgments of the author? * What, if any, is the factual basis for the opinions the author expresses? * Is it reasonable, logical and clear? Your critical faculties: use ‘em or lose ‘em!

Biography

Robin Harris

Harris has been messing with computers for over 30 years and selling and marketing data storage for over 20 in companies large and small. He introduced a couple of multi-billion dollar storage products (DLT, the first Fibre Channel array) to market, as well as a many smaller ones. Earlier he spent 10 years marketing servers and networks. After leaving corporate life he founded TechnoQWAN, a consulting and analyst firm. He also developed StorageMojo into one of the top storage industry blogs.

Robin writes, consults, coaches and lives among the mountains of northern Arizona.

25
Comments

Join the conversation!

Just In

RE: ZFS/Mac coming soon
FAULKNE 13th Oct
Good day to confirm this comment I would appreciate T h e b e s t o f Z D N e t d e l i v e r e d your website very nice to everyone Yes, Oracle is the only one with shared-disk architecture, but that is there advantage. It means you can add or remove nodes and the database lives on. In a shared nothing architecture, if you lose a node, you lose the system. I'm sure Oracle appreciates EMC highlighting their advantage.I also desire to signal in your RSS feeds. Thank you as soon as once again and maintain up the great operate Awesome post! Thank you very much || thanks for nice content this is really benefit to me.
it for write access. Do you really think they're going to sell Apple a license?
0 Votes
+ -
RE: ZFS/Mac coming soon
Robin Harris 17th Mar 2011
@frgough@...
Check out tuxera.com and their NTFS for Mac driver. I don't think Microsoft would mind licensing Apple directly. What do they have to lose?

Robin
0 Votes
+ -
RE: ZFS/Mac coming soon
Bates_ 18th Mar 2011
@frgough@...

Microsoft making money from Apple. What's the problem there? Seems like a win-win for MS if you ask me.
0 Votes
+ -
How about performance?
MC_z 17th Mar 2011
ZFS is intriguing, but I wonder if you can set it up for high performance. In media-authoring, you often intentionally put data on different disks to increase throughput. Source video here, source audio there, result product elsewhere. Can ZFS be put together in such a way?
0 Votes
+ -
RE: ZFS/Mac coming soon
Robin Harris 17th Mar 2011
@MC_z
Good question. I asked Don to respond:

"If he's been using HFS+, using multiple disks is essential for parallel metadata access scalability (server). HFS+ has a global lock for the journal and catalog b-tree ? only one metadata updater at a time. If he is streaming to already open, already allocated files (more likely the case), then splitting them across devices is likely just helping mitigate the seek penalties since the metadata updates would be minimal in that scenario. If you used a striped disk array with a single HFS+ file system, you wouldn't be able to leverage all the aggregate bandwidth since HFS+ doesn't know about the lower I/O layers -- it can issue asynchronous I/O but only to a certain point.

So does he have to split HFS+ across devices to get acceptable bandwidth? If so, perhaps his file system isn't up to the task. =)

On the other hand, ZFS is built to optimize the writes across all the devices (vdevs). So you wouldn't need to segment your file systems to utilize the bandwidth of the devices in your pool. It will be a simpler config (no up front guessing on how to provision the space across multiple discrete file systems and disks) - it's one pool that the ZFS I/O stack can optimize for.

There's nothing to prevent you for creating a separate pool for each device, but that would trade administration complexity for a minimal bandwidth gain.

One advantage that HFS+ does have WRT performance, is in direct/in-place I/O. Since it is writing data in place you can stream directly to the disk (DMA) and avoid the buffering overhead in the file system layer. If speed trumps data integrity, then that might be OK. There will always be working sets where direct I/O will be faster on HFS+, so if speed is king, then stick with HFS+. If you care about the integrity of your data however, then it's a no brainer to trade up to ZFS. I can design a file system that is blazingly fast if data integrity is unimportant."

In RAID terms, ZFS RAIDZ write full stripe writes - the fastest kind - all the time. Bing "ZFS Performance Versus Hardware RAID" to learn more.
0 Votes
+ -
RE: ZFS/Mac coming soon
MC_z 18th Mar 2011
@Robin Harris
Thanks for looking into this more deeply, Robin. In many media authoring situations, it's not so much about write efficiency as it is read efficiency. A musical sampling synthesizer may be reading from any of thousands of open sample files, based on the keys that are struck and the intensity with which they are struck. Same is true when assembling video effects. The write portion is actually much simplified, since all of this input is typically combined into just a few channels. So the majority of file system optimization in these cases has more to do with seek latency than any other factors. And of course data integrity remains important. Thanks again.
0 Votes
+ -
Curious
johnfenjackson@... 18th Mar 2011
A suitable configuration of Lightpeak fabric, ZFS and commodity disks would surely blow the Enterprise Data Storage market apart ... yet did I not read that Apple have withdrawn their Xserve product?

Sounds like another opportunity to outflank M$ to me.
0 Votes
+ -
Message has been deleted.
dfhjtykyk Updated - 18th Mar 2011
0 Votes
+ -
Message has been deleted.
Wholesale Supplier Updated - 18th Mar 2011
0 Votes
+ -
RE: ZFS/Mac coming soon
rpollard@... 18th Mar 2011
Absolutely wonderful. Apple has smart people. They knew that ZFS is better but couldn't bring it on board for obvious reasons. I run ZFS on Sun servers and nothing else will do. Will be re-doing the root partitions soon with ZFS root in an OS upgrade. The boost in speed, integrity and ease of management is going to be worth every effort put into it. I wonder how much it will cost if anything?
Thanks for the article!
0 Votes
+ -
Add/Remove
mediumcool Updated - 18th Mar 2011
A lot of Mac users use different disks for different purposes, at different times; a USB external for backup, e-sata external RAID for video editing or *big* Photoshop/LR/Aperture/Capture One jobs. Disks come and go; USB home for backup storage, another to the bureau for printing or duplicating, or a 4GB USB key to give to a client whose mail system doesnt accept large file transfers. Never mind the external startup disks that can be moved from machine to machine/site to site. And Thunderbolt will add to the connection/storage options

What happens when you want to eject that 4GB drive? It?s been made part of the ZFS pool, so how do you tell the OS to eject *it* rather than another disk, and how long does it take to write that data to elsewhere in the pool? [I understand there is some redundancy/spanning inherent in ZFS, but what does it mean in practice?].

I believe ZFS is a supremely capable and -featured file system, but it seems to have its best application in an enterprise system with relatively static disk utilisation. Prove me wrong, though!
0 Votes
+ -
@mediumcool
Based on my experience using ZFS with OpenSolaris, physical drives are not automatically added into ZFS pools. For removable media like USB thumbdrives, those drives will remain independent.

For what it is worth, you *could* build a ZFS pool using USB thumbdrives, but I can't think of any worthwhile reason. (But I like the silliness of putting those numerous promotional drives that I've got laying about to some use, even though I'd need to add some 50+ USB ports!)
0 Votes
+ -
RE: ZFS/Mac coming soon
Dalece 18th Mar 2011
@mediumcool
Good question. Also what happens when a hard drive in the pool goes out and has to be replaced? How do you know what needs to be restored?

And backup. What about time capsule? If you have several hard drives, it can easily create a pool greater than time capsule hard drive capacity making the time capsule unable to backup that much data because the data exceeds its capacity.
0 Votes
+ -
RE: ZFS/Mac coming soon
Robin Harris 18th Mar 2011
@Dalece
If you're using RAIDZ you don't need to know what to restore because the file system will do that for you, just as a regular RAID rebuild.

Building a system disk bigger than your backup capacity would be silly, so don't do it. As it is, Time Capsule is a non-redundant and easily corrupted tool, so I don't recommend its use.

Robin
0 Votes
+ -
Kudos to Apple Development Engineers
thx-1138_@... 18th Mar 2011
for recognizing another great idea. But obviously, ZFS goes beyond being simply 'a good idea' it is clearly an excellent and far more integrity-based file system than most others available.

Add to that, as pointed out by RH, that (by design) it is not prone (..or far less prone) to the bit-rot inherent in FAT* and NTFS file systems and its benefits become a real selling point.

This can only be the beginning of a great leap forward / boon for the hardware architecture in Macs.

... now if only Apple would take software integrity / security more seriously. wink
0 Votes
+ -
RE: ZFS/Mac coming soon
PJDuro Updated - 20th Mar 2011
@Robin Harris
You note that "..NTFS could be on the list, but its a moving target with a raft of improvements due out in Q4.."

Is there any available and current documentation that describes this in more detail? The blog you reference is dated August 2007.. Quite sometime ago..
0 Votes
+ -
RE: ZFS/Mac coming soon
flurrball 1st Apr 2011
"you can be sure it is your data, and not the misbegotten spawn of a driver burp."

Hilariously insightful wording...
0 Votes
+ -
I use zfs
Matt Connolly 24th May 2011
I use zfs with openindiana as a time machine backup of my Mac.. Data is mirrored between two drives. One was a WD. It died (surprise). I replaced it with another drive and it resilvered (replicated) perfectly without having to turn off the machine or even reboot and didn't loose any data in the process. Go ZFS!!!
0 Votes
+ -
RE: ZFS/Mac coming soon
MACKENZI 11th Sep
I also desire to signal in your RSS feeds. Thank you as soon as once again and maintain up the great operate! nccma cooler
0 Votes
+ -
RE: ZFS/Mac coming soon
PEARLINEI 12th Sep
I used to be more than happy to seek out this internet-site.I wanted to thanks in your time for this glorious read!! I positively enjoying each little bit of it and I have you bookmarked to check out new stuff you weblog post. this thread is amazing i like your work and i appreciate you that you have share a useful stuff thanks for sharing the i shop abatwa
0 Votes
+ -
RE: ZFS/Mac coming soon
SATURNINA 14th Sep
I think the representation of this article is actually superb one. This is my first visit to your site. Thanks a lot and keep sharing the information. Keep updating the information for all of us. Thanks ZDNet Government was launched as the brand's first industry vertical, with a mission to cater to IT professionals in the public secto I agree with your post. However, do you have any sources I can cite for my paper wheel car com bury
0 Votes
+ -
RE: ZFS/Mac coming soon
TOCCAR 25th Sep
Well welcome, hopefully you can become a vital member of the community and really help to push far ahead of google. Which Im sure the development team would love. This will of course earn you alot points too and get you on the leaders board. z d n e t t h a n k Im not sure i come to an agreement with you on every level, howevor it absolutely was a good posting, many thanks for taking the time to put up your ideas.
0 Votes
+ -
RE: ZFS/Mac coming soon
MCKNIGH 26th Sep
Thanks nice info z d n e t I really liked your current article write more..let me add you to its favorite The articles you have on zdnet s i t e are always so enjoyable to read. Good work and I bookmarked it.
0 Votes
+ -
RE: ZFS/Mac coming soon
MEJIAHA 30th Sep
Fantastic news about the new release.I positively enjoying each little bit of it and I have you b o o k m a r k e d to check out new stuff you weblog post.Im not sure i come to an agreement with you on every level, howevor it absolutely was a good posting, many thanks for taking the time to put up your ideas
0 Votes
+ -
RE: ZFS/Mac coming soon
FAULKNE 13th Oct
Good day to confirm this comment I would appreciate T h e b e s t o f Z D N e t d e l i v e r e d your website very nice to everyone Yes, Oracle is the only one with shared-disk architecture, but that is there advantage. It means you can add or remove nodes and the database lives on. In a shared nothing architecture, if you lose a node, you lose the system. I'm sure Oracle appreciates EMC highlighting their advantage.I also desire to signal in your RSS feeds. Thank you as soon as once again and maintain up the great operate Awesome post! Thank you very much || thanks for nice content this is really benefit to me.

Join the conversation!

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]
ie8 fix

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity

White Papers, Webcasts, & Resources
ie8 fix