Mac storage boldly enters the 1990s

Mac storage boldly enters the 1990s

Summary: Apple's set the Wayback Machine to warp factor 9: OS X Lion now has a Logical Volume Manager - great technology in 1990. Today? Not so much.

TOPICS: Storage, Apple, Hardware

What is an LVM? PC users know that system disk drives show up as a C: drive or a Macintosh HD - or whatever you choose to call it - on their system. These are physical drives or physical volumes.

Logical Volume Managers virtualize these physical drives. The LVM takes the attached physical blocks on 1 or more disks and presents a virtual disk to the OS. The virtual disk walks like a disk and quacks like a disk, but the underlying storage is hidden from the OS.

So what's hiding underneath? It can be a physical disk, a disk partition or a RAID stripe of multiple disks.

LVMs can be handy tools for storage administrators, but like any paradigm that relies on managing discrete units, LVMs don't scale. Ask any sysadmin struggling to update an Excel spreadsheet of all the LVs on a SAN.

Enter the new millennium Around 2001 architects saw the scale problem and asked: why pretend we have physical or virtual disks at all? Why not treat the underlying storage as a big pool of blocks?

That's what architects of Google's BigTable, Amazon's Dynamo, Sun's ZFS, Oracle's BTRFS and Basho's Bitcask, among others, all decided to do. Which is why LVMs are so last millennium.

These newer storage tools treat the underlying disks as a pool of blocks. Set a policy for replication - there's no RAID in the common sense - and you can add drives as needed.

That's how the 21st century does it. Too bad the Mac file system guys didn't get the memo.

The Storage Bits take For the average user with 1 or 2 disks managing the physical volumes isn't a big deal. But the era of Big Data isn't only in data centers: I have 16 external disks attached to my new iMac - plus several more that plug into a drive dock.

But the new LVM in Mac OS isn't the biggest problem: the lack of data integrity in HFS+, the current Mac filesystem, is more worrisome. Video editors worry about not being able to transfer clips to the latest version of Final Cut Pro. How about losing your clips to silent data corruption?

If you're wondering what the post-Steve Apple looks like, the continuing hacking of HFS+ is a sorry case in point. Steve doesn't care about file systems - they're plumbing - and thus the normal corporate sins of sloth, inertia and risk-avoidance kick in.

Apple was moving towards the modern ZFS 3 years ago - see Apple announces ZFS on Snow Leopard - but didn't move fast enough to nail down a license before Sun went up for sale.

After the dust cleared Apple had no license for ZFS and discontinued the project. So the hacking of HFS+ has continued long past its expected demise.

Data integrity is one area where Microsoft has the chance to whack Mac OS. While NTFS isn't much better than HFS+ - see How Microsoft puts your data at risk - the NTFS team has been hard at work to fix the biggest problems. I expect to see announcements from them by the end of the year.

Comments welcome, of course. Data integrity should be Job 1 for storage folks, just as computational correctness should be for CPU vendors. But in the PC world, "good enough" has been good enough.

Topics: Storage, Apple, Hardware

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • RE: Mac storage boldly enters the 1990s


    Once again, Apple playing catch up.
    • With whom?

      LVM on a consumer desktop is following whom.

      We've been using this technology on servers for sometime, and as robin points out it is useful for a number of tech savy end user but it's introduction to consumer OS isn't 1990s.

      Data integrity is very important. It's always a trade off between demands of the market: ease of use, cost, performance, etc. apple missed their chance with ZFS, what's next?
      Richard Flude
    • RE: Mac storage boldly enters the 1990s

      @Droid101 Another troll... and look, it worked!
  • RE: Mac storage boldly enters the 1990s

    You're right Robin, but at the same time consider that Macs are very much "consumer" focused which isn't the usual purview of LVM deployments. While there are exceptions, no one is going to roll a massive fleet of Mac OS X servers were the need for an LVM critical. Yes, there's some exception somewhere but I'm talking about the mainstream. Not even Apple with its new massive data center in North Carolina is "Mac OS X only". Hell, Apple canned its Xserve line of servers. Yes you have a lot of drives but you aren't the average user of Mac OS X. Not to mention that a lay person likely doesn't have the inclination and wherewithal to bother. For the consumer I would suggest a Drobo.

    • RE: Mac storage boldly enters the 1990s

      @betelgeuse68 - I'm very interested in seeing how the 3rd party ZFS on Mac project develops. It could be better than Drobo since ZFS doesn't require a hardware controller.
      R Harris
      • By the way, Robin, why no mension of Linus' Ext4 file system in your post?

        @R Harris: is that because Linus ridiculously bashed HFS+ while his own file system, as Ken Hess mentioned, is dramatically behind in the most of key aspects.
      • No mention ox Ext4

        Because this is an article about LVM in OSX.

        It has nothing to do with Linux.
        Michael Alan Goff
      • Really? Then why four other file systems were mentioned?

      • RE: Mac storage boldly enters the 1990s

        Likely for a comparison of what a few other people do and why LVM isn't so good.
        Michael Alan Goff
      • LVM is in Ext4, where are no fancy improvements unlike other file systems

        @goff256: ... mentioned, yet only HFS+ is critiqued.

        Do you think this is coincidental?
      • RE: Mac storage boldly enters the 1990s

        It isn't a coincidence, it's an article of criticism about that particular thing.
        Michael Alan Goff
      • So you think it is fair that Robin critiques HFS for the same reason which

        @goff256: ... related to Ext4, but somehow Ext4 never gets critiqued?
    • RE: Mac storage boldly enters the 1990s

      @betelgeuse68 " no one is going to roll a massive fleet of Mac OS X servers"

      Yep, your 100% right on that one! LOL
  • RE: Mac storage boldly enters the 1990s

    Robin - you forgot Netapp (WAFL) file system - by far the first ones to virtualize pooled storage and commercially more successful (by far) than any of the names you drop :-)
    • RE: Mac storage boldly enters the 1990s

      @jsensarma Didn't NetApp buy Spinnaker? Why was that?
      R Harris
    • WAFL came first - I think not....

      I think you'll find that virtualized storage on physical media has been around since at least the 1960's - long before PC's or WAFL (patented in 1998). IBM's VM operating system was built around the concept of virtualized machines, virtualized peripherals and virtualized (logical) storage.
  • You have been beating this asinine silent data corruption

    horse for years. Apparently, you don't understand mathematics and things like checksums and error correction logarithms.

    Here's a fun little experiment. Take a kilobyte text document. Write a tool that reads and writes it randomly across your disk continuously. Let it run for 3 months 24x7. Get back to us on how many characters get garbled.
    • Data Corruption is Real and Scary

      Today's 3TB consumer drives have a 1 in 4 chance of a non-recoverable bit error on a full drive. So if you put eight of those in a RAID array, you need dual parity (RAID6) just to compensate for the manufacturer's claimed error rate on the drive. Enterprise drives claim one tenth that error rate but that's not what most people are using for personal or small business RAID systems. Most Drobos are stuffed with Western Digital consumer drives. This is closer to the ragged edge of disaster than anybody realizes.
      Jeff X Brown
      • RE: Mac storage boldly enters the 1990s

        @Jeff X Brown Yes, and that's a conscious choice by the industry and consumers to choose storage density and capacity over reliability. If we didn't make disks at such high densities the error rate would go down to a point where RAID5 would be sufficient again.

        Of course, there exist technologies such as RAID6 and ZFS which build in ever greater redundancies, and that's how the industry is tackling the issue. RAID5 is now essentially obsolete.
    • RE: Mac storage boldly enters the 1990s

      Oh. And you think HFS+ has checksums?
      R Harris