Why dumb-downed no-RAID storage is bad for consumers

Why dumb-downed no-RAID storage is bad for consumers

Summary: Data storage is a rapidly expanding area of consumer electronics because people have growing amounts of data at home along with the fact that they have more than one computer and devices that need to access the same data.  Why should consumers have to choose between losing Desperate House Wives or the Final Four games and risk marital stability?

TOPICS: Hardware, Storage

Data storage is a rapidly expanding area of consumer electronics because people have growing amounts of data at home along with the fact that they have more than one computer and devices that need to access the same data.  Why should consumers have to choose between losing Desperate House Wives or the Final Four games and risk marital stability?Unfortunately, there is a new trend to dumb-down storage for the consumer electronics market by shunning formal RAID technology which I feel is to the detriment of consumers.  Consumers often get intimidated by technical terms like RAID storage so I'm going to give the simplified explanation of RAID and explain why it's the perfect technology for consumers.

Some companies have recently begun capitalizing on the fear of RAID complexity that they are advertising the fact that their products don't use RAID.  This actually goes against traditional marketing which tends to advertise the fact that a certain product has advanced features, but this new trend advertises the fact that a certain advanced feature is missing.  For example, two major products being launched by Data Robotics and Microsoft in the very near term are boasting the fact that RAID (in the formal sense) isn't being used.  But at the same time we have new storage products that do advertise RAID capability and the consumer will become more confused than ever whether they need RAID or not.

Data Robotics does a form of proprietary RAID in their Drobo product which seamlessly transitions from data mirroring to data striping with parity and even back to mirroring mode if the device is mostly empty.  Mirroring mode is used when there are two hard drives in the device and both hard drives are being used to store the same data twice.  That effectively cuts your usable drive capacity in half.  When you start adding additional hard drives beyond two, the Drobo device switches to a striping with parity mode though it's not one of the formal RAID Levels.  Striping with parity means that the storage device can tolerate a failure in any of the hard drives and still be able to recover the data and repair itself when the bad hard drive is replaced.  While the ability to seamlessly flip from data mirroring mode to striping with parity is new and a welcome feature, what's not clear is whether this feature is always useful and it's not clear how well the RAID mechanism performs.  The claims of Drobo being able to mix and match drive sizes is also dubious since any difference in capacity between the hard drives in the array is usually unused and left idle.  Later on today I'm scheduled to go to Data Robotics' office and check out the device first hand so I'll be updating my review of their product.

Microsoft's new Windows Home Server will use a new feature called "Drive Extender" that seamlessly merges multiple hard drives by doing file-level data striping with the user-configurable option for data mirroring.  Even though Windows Home Server has the ability to support software-level RAID from the Windows Server OS and there's no reason it can't support hardware- or firmware-level RAID, those features are hidden to the user and depreciated in favor of Drive Extender technology because Microsoft is afraid that RAID will scare off consumer electronics buyers.  Drive Extender has the ability to virtually merge multiple independent hard drives and add internal or external hard drives on the fly so this is definitely a welcome feature for ease of manageability.  The problem is that it doesn't use formal RAID technology so it lacks performance and scalability of capacity.

A Windows Home Server user is expected to designate the folders they wish to be fault tolerant and only those folders are mirrored across two hard drives and everything else is only stored once with no fault tolerance.  If a user chooses to make all of their folders redundant, then that effectively chops the capacity of the Windows Home Server in half which is not desirable at all and I doubt that any consumer will be happy when they buy two 500 GB hard drives and find out they only have 500 GBs of storage.  Microsoft argues that some files aren't important to users and they don't need to be redundant, but why should consumers have to choose between losing Desperate House Wives or the Final Four games and risk marital stability?

The good news about Windows Home Server (or any other custom server running Linux or FreeBSD) is that nothing prevents an appliance maker from using hardware- or software-based RAID Level 5.  Nothing prevents them from hiding RAID complexity by implementing seamless RAID expansion with simple wizards that ask the user if they wish to expand their fault tolerant RAID volume whenever a new internal drive is added.  When RAID Level 6 technology gets cheaper and more commoditized in the near future, that will provide the ultimate form of fault tolerance because it can tolerate a failure in any two hard drives in the array.  RAID Level 5 is the most economic form of redundant fault tolerant storage because you only lose one drive in capacity.  If you had 4 hard drives in a RAID Level 5 configuration, your effective capacity is the total capacity of 3 hard drives.  If you had 6 hard drives in RAID Level 5 configuration, your effective capacity is equal to the total capacity of 5 hard drives.  If one hard drive dies, you can just replace that one hard drive and all of your files are safe.  Even though this results in a loss of one hard drive in capacity, it's a small insurance to pay to insure the integrity of all the hard drives in the array.  If we were using Drive Extender technology configured to have everything redundant instead of RAID Level 5, we would lose half of the hard drives to redundancy which is a huge waste of storage capacity.

Slower external USB 2.0 devices could be added to the pool of storage using something like Drive Extender and not the RAID volume since it would degrade the performance of an internal RAID, but Windows Home Server would need to be able to distinguish between redundant RAID volumes and non-redundant external drives.  The redundant volumes should be used for unique data and the external non-redundant Drive Extender volumes should be used for dumping backup sets from client PCs or used for non-critical data.  At this point in time I do not believe Windows Home Server is configured this way but I believe they should consider it.  I will check with Microsoft and update this section with Microsoft's response.

Ultimately, most consumers won't understand all these nuances in storage technology and they'll just eat up the marketing and whatever it says on the retail box.  I'm going to do my best to educate the public on these things and hopefully influence other technology reviewers to take product vendors to task.  Consumers deserve a good experience with technology especially when they're being asked to spend so much of their hard earned money and companies have a responsibility to produce good products.

Topics: Hardware, Storage

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • In defense of WHS

    WHS allows people to use drives of various sizes unlike the traditional RAID. This one feature is of great value. Since most people A) do not back up and B) do not want to upgrade their (non-existent) server hard drives in pairs (or worse), MS has given people a flexible option that handles both. Certainly, it isn't perfect, but it is cost effective and reasonable.

    That is the point.
    • You can use different size drives in RAID arrays..

      • Sort of

        If you need to replace a drive you can use a larger drive but you can't use a smaller one.

        But say you have a mirror set of 160 GB hard drives. You want to upgrade to 300 GB hard drives. So instead buying 1 300 GB hard drive you have to buy two. While you are correct you could buy one you'd still only have 160 GB in the mirror set.
        • In addition......

          Even if you did get 2 300Gig drives, you will still have the partitioned data set for 160 Gig.

          You have two options. Back up the data and recreate the array to use all 300 gig, or use the extra space as another volume.

          There may be a utility to expand an array partition, similar to Partition Magic, but I currently don't know of any.
          linux for me
          • A tool like that would be nice (nt)

          • That's the App we need to design

            It will be the ZDnet Talkback's Killer App.
          • WHS - RAID - Resizing

            Assuming Windows Home Server is going to do things correctly, it will be setting up any data disks as Dynamic Disks (and forcing the use of a different disk for Operating System).

            For those users familiar with Windows Server and Dynamic Disks, you can resize the volumes on a dynamic disk from within Disk Management (which I'm sure they could create a nice fancy GUI-based wizard for) so long as it does not contain the Operating System.
            Angel Of Death
    • You can do the same thing since Windows NT

      You can do the same thing since Windows NT by merging multiple hard drives in to one continuous volume. But like I said in the blog, Drive Extender is a nice NEW feature. My problem is that other features like RAID are being depreciated.
  • Reasoning with Fear of RAID

    The only questioning of RAID that I can see is cost. I remember not so long ago when RAID was changed from "Redundant Array of Inexpensive Disks" to "Redundant Array of Independant Disks." This was because at the time, RAID had an overhead.

    Then I also remember not so long ago when you were even advertising that RAID 5 was slow. And in the case of devices that don't have certain controllers or processing power behind an array, they can be slow.

    Do I skoff at those who refuse to use RAID? Not really. But would I choose it over a RAIDless environment. Only if Price was not much of a concern.
    • Price is no longer a concern

      Price is no longer a concern and performance with the built in free RAID controllers using software is extremely fast.

      That's why I focused on RAID Level 5 in this blog; it's extremely cost effective since you only lose one drive in capacity.
      • Now it is

        SATA being one of the key technologies that make RAID 5 at the consumer level a
        possibility. Even so, how many people want to spend $$$ on storage alone.
        Granted, I remember when a simple NAS would still run $5000. For low end
        commercial users, this would be ideal. For home users, this would depend on
        how much you value your data.

        As for consumers advertising against it, that would be like a car maker telling you
        that they are proud they don't have Anti-lock brakes or Airbags. Sure a few
        injuries could be avoided but you miss the point on the greater good.

        However, this goes back to the market economy and their desire to sell consumers
        junk with a high mark up. Only so they can sell slightly better junk when they
        return in a year or two.
        • I can build a gigabit NAS with hot-swap for $700

          I can build a dual-core gigabit server with 5 hot-swap SATA bays for $700 not including drives that performs more than twice as fast as an off-the-shelf NAS you can get for the same price.
  • You have it backwards

    Most PC users are not technical. They don't do backups, and would not know how to restore a full-volume backup if their life depended on it. Sad, but true. Enter raid with multiple drives, and ask a typical user to figure out what whent wrong when it does. And it will. Industry experts indicate that RAID in consumer hands represents a lot more trouble than its worth! The number of issues reported are not just related to hardware failures such as failed drives, but also to glitches in software RAID and cheap on-motherboard hardware RAID controllers. The experts know its bad news. So do consumers who have bought into it, and had issues.

    If a consumer wants RAID, they'd be smart to first do their homework. They'll realize that they have to pay MORE money, starting with a good RAID controller card, and including a higher Total Cost of Ownership (TCO), and higher frequency of failure.

    They should also know that RAID configurations that are geared towards performance will only deliver noticeable performance under *some* conditions, like reading or writing huge files. Most consumers won't experience any performance gains.

    And for those consumers who want RAID for security purposes, such as a redundant drive, they need to be informed that there is a performance penalty for that security (which may not matter). But what *will* matter, is that RAID will not protect them from viruses, accidental deletions or corruptions, malicious damage, etc.

    The cheapest, most effective solution for consumers, is making reliable backups, knowing how to validate the backup's integrity, and knowing how to do a full-volume restore, should the need arise. But this approach requires discipline and education, two things that common PC users lack. So that's why RAID is so appealing. It is the Magic Amulet, which protects the user from all harm, just by wearing it.
    • Yeah

      All RAID really protects you from is hard drive crashes. From my own experience, I would say that consumers would be much better off getting a home network server, maybe something like the Buffalo Technology Linkstation. And setup the computers to automatically back themselves up to the network storage.

      I have my own computers setup like this, using Norton Ghost to image the partition containing the Documents and Settings folder and other crucial personal files every day, and it images the system partitions once a month. These backups are well worth it, and have saved my butt several times.
    • Backups are Better for Home Users

      Give me timed backups any day, like Drive Image for Windows that can do a timed backup back up my OS partition while its running (do it nightly via Task Manager), compress the backup data in the process (conserve hard drive space) and can be restored to any similar or larger size hard drive partition. And its no more difficult (probaly easier) for the average home user to do that than it is to figure out how Raid should work and how to fix it when it doesn't.
      • Windows Home Server does incremental and single-instance storage

        Windows Home Server does incremental and single-instance storage and it gives you bare metal recovery features.
    • That's a different topic

      RAID is hardware redundancy. It does not protect users from stupid human tricks or malicious software tricks. Both issues have to be considered, but it's not within the scope of this particular blog. I agree they're related, but I wrote this blog to address the hardware redundancy issue.
  • RAID Whatever!

    I do think that dropping RAID is an industry wide thing in the happening. There is a reason Google doesn't use RAID!

    I do think the number one reason why RAID is not the solution for consumers is the fact that RAID systems needs to be planned before hand and it is very difficult to grow organically. The path Microsoft has taken can be classified as a more intelligent more flexible alternative - because the management of storage moved up in the stack smarter things can be done. Things that can be done in future with systems like these:

    1) Any type of drive can be added at any time. In future I think that it would be possible to replace any single drive with another type of drive without losing data in the process or without going through a complicated back-up restore process!

    2) Storage management moved up the stack - why should all the storage in the future be local? What if one drive sits on the other side of the world? What about a peer-to-peer storage mesh in the home that optimize performance and redundancy by using unused space on desktop hard drives and your PVR?

    3) Performance - RAID can improve performance only in a very dumb fashion using striping. These new technologies will in future definitely cater for moving highly used objects to faster storage!

    4) Power management - RAID requires all drives the be on all the time. Smart technology will enable redundant systems good enough for the consumer without having 5 disks spinning all the time!
    • Google uses redundant array of inexpensive servers

      Google uses redundant array of inexpensive servers. They don't care if the entire 1U box dies.

      Clearly you haven't see the kind of performance we're getting with the latest RAID technology. http://blogs.zdnet.com/Ou/?p=484

      You talk about power efficiency in shutting down hard drives and then you suggest using a storage mesh with multiple PCs running in the home. Do you know how ridiculous you sound? A PC uses 100 watts (maybe more), a hard drive spins at 9 watts.
      • Umm...

        1) Google uses redundent servers. Yes - they manage storage from higher up the stack. They don't use a fixed hardware based storage scheme.

        2) RAID requires all disks spinning when writing data. Intelligent mesh networks can have 1 drive spinning on the server, and another on a PC when that PC is on and connected. Hence the word "intelligent" in intelligent mesh networks. Granted - if you have a dumbed down mesh network you are going to use more power. But if the person is using his PC anyway or that PC is busy with a background job of some kind, you might as well use it in the network with negligible extra power usage.

        3) Performance is not such a big issue on home networks. If you can stream two HD movies of a hard drive you probably are fast enough. Furthermore - I'll bet intelligent non-RAID storage will beat RAID performance figures within a few years - by using flash based buffers, and doing some usage based optimizations.