Getting started with our massive Media Tank home storage project

Getting started with our massive Media Tank home storage project

Summary: Over the course of the next few articles, we're going to take you through the full Media Tank project, including all the tricks that were necessary to cram so many inexpensive drives inside one box and make it all work.

SHARE:
TOPICS: Storage
25

It’s been a while since I’ve introduced you to a completely new project, so I figured it was time to open the sliding trays of one of my favorite (and most useful projects): the Media Tank, Mark I. Here’s the most important spec: 20 terabytes of storage across 10 data drives, in one 20-inch mid-tower case.

Media Tank
Media Tank, Mark I (courtesy David Gewirtz media archive)

The whole Media Tank project is actually more than just storage. We ran gigabit Ethernet to every wall of the house (include inside some closets), there’s a second tank (a clone of the first) that’s used for local video backup, and we have an additional three tiers of backup for other types of media.

Over the course of the next few articles, I’m going to take you through the full Media Tank project, including all the tricks that were necessary to cram so many inexpensive drives inside one box and make it all work.

In this article, though, I’m going to spec out the project and explain why we built it.

It all goes back to 2005. My wife and I had just gotten married. We’d decided to move from chilly New Jersey to warm Florida. When we started looking for a house, we realized we’d somehow have to fit the entire contents of my wife’s pre-marriage home, my pre-marriage collection of stuff, and the office for what was then my online publishing company.

We were gonna need a lot of space. A lot of space.

So we rented a big house in Central Florida. We filled a moving truck with 19,280 pounds of stuff. Her clothes, my T-shirts and jeans, her girl stuff, all my gear, her huge music collection, my very large video collection, and both of our book collections. Together we had thousands of CDs, close to a thousand DVDs, and wall upon wall upon wall of books.

Even though the house was a monster, it was still less expensive to rent than either of us spent on rent in New Jersey before we moved in together. Florida isn’t as inexpensive as it once was, but it’s still a lot less expensive than living in the metropolitan New York area.

We managed to combine it all, threw some stuff out, and got some new stuff. After about six years, our way-too-big-to-clean house was even more full of stuff than when we came down here. Our lease renewal was about to come due, and we realized it was time to buy a house of our own.

But we wanted something more manageable. We didn’t want to turn into hoarders. We’d each known a few hoarders in our lives, and we were not about to let ourselves become another hoarder couple. We were committed to cutting back on all that stuff.

Looking around the big rental house, it became apparent that a huge percentage of the space was given to warehousing our books, music, files, videos – our media. What if we could reduce that?

What if we went all digital?

Having the Media Tank made it possible to very comfortably downsize our home.

At first, my wife tried to scan in all our books. We bought an industrial paper cutter and a great scanner called a ScanSnap. Over the course of a few months, she made a valiant effort to scan in all our books.

As it turns out, that project – at the time – proved somewhat impractical, merely because of the physical effort of scanning in the books and keeping the scanner running. She did manage to scan in a hundred or so of our favorite books, plus most of our important manuals (if they weren't available for download).

In the middle of this, the first iPad came out, and we confirmed that reading scanned PDFs on a tablet was just as practical as reading a physical book. The concept worked, at least in theory.

In practice, we wound up getting rid of a lot of our books. This was the third or fourth major book purge I'd done in my life, and each time is as difficult. Each book is a friend. But we donated as many as we could (fewer people and organizations are willing to accept book donations these days). We realized we'd read many of them and were unlikely to read them again. Many of the reference books could virtually be replaced by the Internet. Many others are available on Kindle format if we feel we must reread them at some future time, or available for pennies used from Amazon.

We’re still (almost three years later) trying to get the remainder of our books digitized, though we're nearing the end of our mass scanning days. That’s a story for another article series.

We came to realize if we were going to convert all that physical media (not just books, but files, movies, videos, manuals, even software disks) to digital form, we’d need a central server to store it all.

Next up: what we store on the tank now...

Topic: Storage

About

David Gewirtz, Distinguished Lecturer at CBS Interactive, is an author, U.S. policy advisor, and computer scientist. He is featured in the History Channel special The President's Book of Secrets and is a member of the National Press Club.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

25 comments
Log in or register to join the discussion
  • wow david, your storage device looks very impressive!

    Can't wait to hear your experiences on it!

    I have something similar. I have a 12tb porn machine (yep, I admit it, its pretty much mainly porn.

    though its not nearly as impressive as yours! I just built a cheap computer and started stuffing drives in.....
    the_tyrant
  • I can't wait for the details!

    That is quite an awesome undertaking and I personally can't wait to see how you did it. That is exactly the sort of setup I have been dreaming about doing, but I'm not even sure where to start. All I can say is wow, that's cool!
    JustWow2000
    • Me, too.

      I'm planning a build out within the next year, so this article is one of the few I actually look forward to reading from ZDNet. I wish they'd do more like this.
      BillDem
    • Me Three!!

      Looking forward to hearing more of the story. Hope you'll include some info for us folks who just don't know where to start on a project like this. In other words, after you build the tank, how start digitizing your life!

      thanks!
      zzzmoosetech
    • Me four!

      So many drives around (13 TB) but having everything in one place it´s much better.
      I´ll look forward your article, thanks!
      cuitla68
  • Similar Experience

    Have a similar experience.

    Although we didn't digitize books, we did go digital with all our DVDs. We now barely have any, and everything's stored on a nice 16TB central server in a little closet below our stairs. Wires run throughout the house, everything's reachable from everywhere else. 4 fixed XBMC installations for viewing all the material. All our files are stored there. The most expensive components are the HDs, everything else was "repurposed". Periodic backups occur of our most important stuff, and they're stored offsite.

    Having everything centralized is wonderful.

    Look forward to more. In specific, I'm curious what sort of hardware failure you can survive, and associated costs.
    hades_2100@...
  • about the same experience

    We stopped buying physical media and paper books when we moved in our current house about nine years ago. All existing DVDs, CDs and tape cassettes were digitized but we keep the paper books (trick: part of this is actually stored in our parents houses). Our difference is that we are completely wireless.

    As we are about to move to a new much larger house, we intend to build few more storage servers, mainly for redundancy and of course put wires everywhere.

    But I wonder about the service provider you used.. How are you able to trust them they actually shredded your documents and that nothing was lost in the conversion process?
    danbi
  • Backup?

    A backup unit in the closet of the same house won't help if you have a fire!
    I bet "Carbonite" hates you!! LOL

    WHY does anybody NEED thousands of movies or songs......there might be a dozen movies I would watch more then one time.
    jimbritttn
    • WHY does everyone ASSUME

      that everyone else's needs must be the same as their own?
      frylock
  • Give us some of your music tracks to listen

    Put some samples on sound cloud :)
    Tomas M.
  • Bring it on!

    Sounds like you've gone a different way to me: I've plumped for a fast workstation with SSD's doing ... you know, work (in progress) ... and slow data servers doing ... storage, backup and cloud interface for all the stuff I never intend to look at again ;-).

    Areas I'm always looking for ideas on:

    1. GBE is fine but INFINIBAND would be better. The cost of an INFINIBAND switch is still too high though. (Robin Harris wrote about this a few years ago.)

    2. Despite all the ZDNET bloggers telling me how wonderful the cloud is and how generous MSFT, APPL and AMZN are going to be forever ... I don't trust them as far as I can throw a datacentre. So I'm buying ONE copy of Office 2103 and making it available via remote desktop. Not so good if one is off-site: any ideas about a cheap REMOTEFX or TERADICI product to speed up WAN access?

    3. Availability at home.

    a. UPS to protect from power outages. 6-8 hours. Which, how?
    b. Dual ISP connections. I'm thinking about CONNECTIFY http://www.connectify.me/dispatch/. Fibre + 3G backup?

    As long as I don't have to listen to Gewirtz-musik I'll be tuning in ;-)
    jacksonjohn
  • Possible copyright infringement!

    Technically, once you dispose of the hard copy you have no proof you ever bought it. This makes the digital copy illegal. So, all your digitized media could be considered copyright infringement.
    cuulblu
    • Very true...

      EULAs and copyright laws specifically mention that copying or "backing up" stuff is for archival purposes only. You are basically multiplying your copies of items, i.e. copying a movie instead of buying the hard copy and a digital copy. Of course newer movies like Blu-Rays that include hard & digital copies wouldn't apply.
      thomascwhitfield@...
    • Very true.

      When I build out my media server, I plan to pack all the discs away in boxes to store in the garage or attic. I never digitize something without keeping the original as proof that I own it. Storing my media in boxes is fine for that function and removes MANY shelf units full of media taking up floor space. I only have around 700 CDs, but I have over 1200 Blu-rays and DVDs (and growing.) My servers are going to be 24 drives each initially because of all the Blu-rays. My hundreds of books will be replaced over time by Kindle versions, rather than local storage. Digitizing books is just too much of a pain.

      I look forward to reading your entire series on this project. I want to see how your parts list compares to what I've come up with. I also want to hear what media server software you decide to use.
      BillDem
      • Re: I never digitize something without keeping the original as proof that I

        Makes no difference. It is the act of digitization that is illegal.
        ldo17
  • Good point!

    Never thought about this way, but it is true... unless you keep those scanned receipts.

    No one keeps them, but Gewirtz family do :)
    Tomas M.
    • Back to the 1990s

      We have all our paperwork, for everything, going back to the early 1990s (and some going back even further). Now, most of it is in PDF, but you begin to see what that took so much space.
      David Gewirtz
  • looking forward to hearing more

    My young man and I are looking for a house together and we've got something like 70 feet of bookcases. Each. Just finding a house with that much blank wall space is a challenge. We are starting to digitize the music and movies, but I have to admit I'm a purist and like opening the cases and playing Albums and so on. I'm also looking forward to hearing about installing the broadband.
    slfisher@...
  • This blows me away!

    And, has given me some ideas! I am looking forward to future installments. I follow you on Twitter, so I hope you will post a notification there when the next installment is ready!
    Rodo1
  • Just go all in - once!

    http://blog.backblaze.com/2013/02/20/180tb-of-good-vibrations-storage-pod-3-0/

    will likely take care of you for quite some time--and in a cost effective way.
    Bruce_B2