Building our massive storage Media Tank

Building our massive storage Media Tank

Summary: We continue our massive Media Tank story by answering the questions almost everyone asked: what's inside? Read on, and we'll tell you all about it.

SHARE:
TOPICS: Storage
14

Welcome to Part 2 of our Media Tank, Mark I project. If you missed it, last week's article covered the specs behind the Tank, why my wife and I built it, and what we use it for. Today is all about the build itself.

The OS

Our fundamental goal was to build an array that could handle as many hard drives as possible. Because I had a bunch of old, spare, Windows XP licenses floating around, I decided to base the machine on XP. I wasn’t terribly concerned about Web security (a big problem for XP), since I had no intention of surfing the Web from this machine and, once built, it would live behind an enterprise-grade firewall. The Tank was going to be headless (no monitor), and that meant all I needed was the ability to create Windows shares.

Media Tank
Media Tank, Mark I (courtesy David Gewirtz media archive)

We work from home, so the network here is more than just something for family fun. It’s a serious professional tool. My wife co-founded our publishing business with me way back in the 1990s, and we worked together for more than a decade before we got married.

I’m telling you this because our network has been built to run our office, even though that office is located in a house – a house that’s been pretty much optimized to function as a network operations center as well as a family dwelling (although my mom thinks we should have a rarely-if-ever-used sitting area for guests instead of what looks to her like a computer lab).

Here at Camp David, we have a bunch of Windows machines, as well as four Macs, and a few Linux VMs. Because most of our day-to-day work is on Windows, we optimized for Windows shares, and XP did the trick. The Linux VMs and Macs understand SMB moderately well, so they’re playing along nicely.

And yes, I could have set up the Tank as a Linux box, but I didn’t want to. Linux regularly finds a way to annoy me or ruin my week. XP was easy, cooperative, and available. That’s all there is to that story.

Drive boundaries

You may have noticed that this project is about the Media Tank, Mark I. That’s because this is a first iteration implementation. This build is logically drive-centric. Each share lives on a drive and if that drive runs out of space, that share gets full – even if another drive has a ton of spare space.

This works, but storage pooling is a viable technology. At some point, we may embark on a Mark II project that pools storage across all the drives. But for now, shares don’t cross drives.

Before I move on to tell you about the gear, I want to post a disclaimer. When possible, I’m sharing a link to the item I bought. In most cases, those links are on Newegg, because I generally have pretty good luck buying from them. So, when you see the link and it takes to you Newegg, that’s because that’s the item I bought, not necessarily a pitch for you to go there and buy. I’m also using images from Newegg, so it seems fair to throw them some link love in return.

Boot drive

The Media Tank has the capacity to run ten drives: nine shared data drives and one boot drive. For the boot drive, I used an old spare drive. I loaded XP on it, and it runs. Since very little needs to live on the boot drive, as long as it’s not dog slow, it works for my purposes here. As I recall, it’s an old 300GB 7200 RPM platter.

Next up, choosing the components...

Topic: Storage

About

David Gewirtz, Distinguished Lecturer at CBS Interactive, is an author, U.S. policy advisor, and computer scientist. He is featured in the History Channel special The President's Book of Secrets and is a member of the National Press Club.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

14 comments
Log in or register to join the discussion
  • Keep 'em coming

    Most interesting stuff I've read on ZDNet in a while. I have an Aspire Easystor H340 that does the trick for now, but I'll eventually outgrow it (or it will die at some point), so I'm taking notes here.
    MichelR666
  • Awesome!

    I want to build one kinda like yours David. Is it really that hard to just take the case cover off and snap a few pics? Nothing too extreme depending on where the tank sits.
    Please?
    JustWow2000
  • I will keep my advice short...

    For storage, never, ever use anything but ZFS!
    If you value your data, that is. :)
    danbi
    • That is poor advice

      So we are fortunate that is was short.
      John Zern
  • XP, David?

    I'm generally not one of the people around here who participate in OS wars or anything, but if you're making a huge dedicated storage array whose life's purpose is to share folders, why not use one of the many excellent appliance operating systems?

    My home storage server runs NAS4Free, and runs it very well. FreeNAS, OpenFiler, NexentaStor, and OpenMediaVault are all explicitly designed for the very purpose for which you're assembling the hardware. Additionally, some of the distributions above run on ZFS, which is among the best file systems available for highly resilient storage.

    I know you said that Linux finds its way to annoy you in one form or another, and I completely concur with your decision to not install Ubuntu/Fedora/Cent and try to coax it into not making a mess. However, purpose-built Linux appliances are generally more stable and user friendly, and in nearly every case uses a browser based control panel instead of requiring the use of a command prompt. I've never used it out of necessity on my box.

    Check 'em out before you opt for XP. Windows may be familiar, but there is decisive optimization for storage in NAS4Free/FreeNAS/OpenFiler/NexentaStor/OpenMediaVault. Truth be told, the amount of work it would take to mold XP into a storage server is roughly the same amount of work it would take to take a lap around the UIs of these projects.
    Joey
    voyager529@...
    • Nope

      I have never experienced Linux to be more stable than NTFS-based machines. I've had EXT3-based Linux machines (which is what, years ago, we started with as a file server) crash to unrecoverability, where the NT server we ran ran for something like 5 years without a restart.

      ZFS may be well done, but it's complex and persnickety. NTFS just works. I've never, ever lost a system to NTFS, but had a variety of Linux machines fail in completely unrecoverable and nasty ways.

      The Tank is a good example. Even though it's grown, it's been rock-solid robust for the last three years. That's because of NTFS. XP, for this purpose, is merely a shell around the NTFS and SMB environment.
      David Gewirtz
      • ZFS

        Let's just say, that ZFS is not an Linux thing. Linux might have some good spots, but filesystems is not one of these.

        ZFS was designed by SUN in Solaris and is best available in Solaris, it's derivatives, like OpenIndiana etc and, FreeBSD. There is Linux port of ZFS but it is still not very well integrated. There is also an OS X port of ZFS, which too, is not very well integrated. All of these platforms will do fine with ZFS for storage (even Linux and OS X). To my knowledge, these is no usable ZFS port for anything Windows.

        ZFS is by far the best possible platform for storage. It integrated both device manager, caching, volume manager, block device and POSIX filesystem layers. ZFS ensures reliability by using end-to-end cryptographic checksums for every block it writes, plus plenty of redundancy of metadata to recover from many different data corruption scenarios. In fact, if your "computer" hardware, such as CPU, memory is "good", ZFS will take care of your data consistency.

        One of the original design goals of ZFS was to take care of the ever increasing size of disk drives and the more or less constant amount of error correction that is provided. For example, consumer disks provide sort of 10^14 error rate. This means, that for every 12.5TB of data you read, you will inevitably have at least one bit wrong. That is, with any other file system, reading your entire "tank" data will provide you with at least one file that is read wrong. Now, consider you will process that file and write it back.. with the wrong data. And hopefully you get the idea.
        Going with the much more expensive enterprise drives gives you just 10^15 error rate, which means you will garble your data after just 10 full reads...

        Doesn't matter if it's NTFS, FAT32, EXT3 or UFS. Or HFS+. As long as your huge file storage doesn't do integrated end-to-end cryptographic checking for *everything* your data is silently corrupt. You should have kept your paper receipts :)

        You think it's rock solid, only because you have't yet discovered the data corruption.
        Been there, done that. Never be naive with storage again.
        danbi
      • In Agreement

        I'm in agreement with you David. For a while, I drank the purple, sweet Linux juice only to find that it would let me down. Granted, most of my experience is with Red Hat and all of it's variants such as CentOS and Fedora.

        Starting with Windows XP/Server 2003, my issues have been very small. With the advent of Vista/Server 2008 and all later versions (OS versions 6.0, 6.1, and recently 6.2) losing data to a NTFS Windows' share has been a nonissue--even on a failed Dell PowerVault MD1000/3000.

        If I ever take on a personal project--and even a professional one in which I have controlling interest--I will use Windows Server backends for the storage and let OSX and Linux clients and servers play nicely with it.
        DarienHawk67
      • I agree

        When building my home storage server I really wanted to go with ZFS. Since my NIC was not supported by OpenIndiana I got cornered into Linux and went with Ubuntu Server.

        It works great for me, but I fully agree, with you David, that for an all Windows network it would be easier to get XP fully optimized than all of the fun that I've had with Samba.
        swoarrior
    • Those appliance OS's.....

      Such as OpenFiler, are usually Red Hat based systems. I have used OpenFiler many times and it has never failed on any server I installed. Using a good raid controller, David's Tank can be built with the same costs and would be much more reliable too.

      But XP??? Come on David......
      linux for me
  • Were These Full Retail Copies Of XP?

    Because you're not allowed to install OEM XP licences on a machine for your own use.
    ldo17
    • Fully compliant

      Over the course of the years, we bought a tremendous number of XP licenses, which were on machines taken out of service. It is entirely within licensing parameters to move a license purchased from one hardware machine to another.

      I very rarely bought machines with OEM XP licenses on them. During the XP era, with the exception of a few laptops, we built most of our machines and bought separate XP licenses. We've done the same with Win7, including my super-honker laptop, which I bought without an OS and then bought and installed my own.
      David Gewirtz
  • Use Server 2008 R2/2012 Eval

    As I am pretty sure you already know, you can legitimately use Widows Server 2012 (or 2008 R2) 180-day eval edition. With a single, fully MSFT supported command line, you can have your fully authorized and activated "server" running for a very long time. Note that I am not talking about any hacks, cracks, etc. The server can be fully patched and updated with all of the latest WGA technologies.

    The Eval server should still be very active by the time you decide to build the Mark [x].
    DarienHawk67
  • Time for an upgrade ?

    Nice article, but you might want to start thinking about an upgrade. XP is old, and it's SMB stack is old and slow. If you were to upgrade to 2008, or 2012, you would get SMB2, and some serious networking improvements. I would also recommend some level of RAID protection, just in case one of those drives die. Also, a backup strategy for this beast might be in order, as a nifty flood, or fire, could eradicate your known universe.
    Another thought... external 8 bay tower, with internal hardware RAID in the tower, connected over USB3, stuffed with 2 or 3TB drives... Then you could use any stock PC as the NAS gateway.
    Iozone