Getting started with our massive Media Tank home storage project

Getting started with our massive Media Tank home storage project

Summary: Over the course of the next few articles, we're going to take you through the full Media Tank project, including all the tricks that were necessary to cram so many inexpensive drives inside one box and make it all work.

SHARE:
TOPICS: Storage
25

That server would become the Media Tank.

Here’s some of what we store on the tank now:

  • Audio, particularly audio we’ve created, along with audio books. We have a complete share dedicated to just non-album audio
  • All our music, nearly 1,500 albums. We keep both lossless (flac) and lossy copies (MP3)
  • All the books we’ve managed to scan, along with all the manuals for all the products we have, have had, and use
  • Nearly all of our bookkeeping records, including all our corporate records going back to before the turn of the century
  • My very large collection of licensed loops and music clips, which I used in the composition of as-yet-unpublished music tracks
  • Our picture library, containing all our photos, including all of my wife’s snapshots, which she scanned in over the course of a few months, numbering in the tens of thousands
  • Our software library, containing ISO images and installers for almost all of our software, replacing the DVDs and CDs they were distributed on
  • My client work project center, containing organized, structured, encrypted, and secured client files for the clients I’ve done work for over the years (this is not complete – I have certain “special” client files that have highly secured, regulation-specified homes)
  • All my studio video production content and assets, including all the templates and media assets necessary to create broadcast quality video
  • Five separate video shares, containing our extremely large video collection
  • A special virtual machine share that hosts and runs all the VMs I use across all my projects
  • A backup share that backs up the servers that serve our various Web sites, including the more than 100,000 articles published by ZATZ since 1998
  • A private “My Documents” share each for my wife and me, containing our own, individual documents that we use and organize for our own projects
  • And a central, highly useful EasyShare storage unit that contains most of our operational documents for both running the house and our business interests.

There’s actually more, but that gives you a reasonably good picture of how we use the tank.

Over time, we became rather expert in organizing and digitizing our information.

I’ll tell you about some of it in future articles. We’ve done a lot of it ourselves, and we’ve jobbed out some of it to local service providers. There’s one local service provider we used that scanned in more than a decade of business documents, shredded them, and returned to us a perfectly formatted set of PDFs.

Having the Media Tank made it possible to very comfortably downsize our home.

We no longer had to rent thousands of square feet merely to warehouse physical media. When it came time for us to buy a place, we were able to buy an awesome place just about half the size of the monster rental house. Because it was a fixer-upper, we had the opportunity to tune it for our lifestyle. That meant gigabit Ethernet in every wall, a super energy-efficient cooling system, a gym/dance space, a combined media room and office in our great room, and a space for the broadcast studio in a small room.

Part of the reason we’ve been able to add these interesting function rooms is that we’re no longer warehousing all our media. Even though we’ve yet to go fully paperless, most of it is stored on our Media Tank. Instead of thousands of square feet going to waste holding up paper and plastic discs, we now have a 20-inch tall tower that takes up 1.17 square feet (and a second one in a closet that we use to back it up).

We now do most of our work directly on the tank.

We open files that live on the tank, we edit files remotely and store them back to the tank. While that’s no big feat for something like an article like this, I host all the VMs I use right on the tank, and even edit video straight on the tank. That’s why the gigabit Ethernet became so important.

So, now you know the back story and the amazing benefits we’ve derived from our Media Tank. In subsequent articles I’ll describe how we built the tanks, how we managed to stuff 10 drives inside one mid-sized tower, how we were able to build our massive Media Tank for around the price of a nicely-equipped MacBook Air, some of the software we’ve used to store our information, and more.

It’s been a fun and rewarding journey, and you’re invited along for the ride. Stay tuned for Part II.

UPDATE: Here's Part II.

Topic: Storage

About

David Gewirtz, Distinguished Lecturer at CBS Interactive, is an author, U.S. policy advisor, and computer scientist. He is featured in the History Channel special The President's Book of Secrets and is a member of the National Press Club.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

25 comments
Log in or register to join the discussion
  • wow david, your storage device looks very impressive!

    Can't wait to hear your experiences on it!

    I have something similar. I have a 12tb porn machine (yep, I admit it, its pretty much mainly porn.

    though its not nearly as impressive as yours! I just built a cheap computer and started stuffing drives in.....
    the_tyrant
  • I can't wait for the details!

    That is quite an awesome undertaking and I personally can't wait to see how you did it. That is exactly the sort of setup I have been dreaming about doing, but I'm not even sure where to start. All I can say is wow, that's cool!
    JustWow2000
    • Me, too.

      I'm planning a build out within the next year, so this article is one of the few I actually look forward to reading from ZDNet. I wish they'd do more like this.
      BillDem
    • Me Three!!

      Looking forward to hearing more of the story. Hope you'll include some info for us folks who just don't know where to start on a project like this. In other words, after you build the tank, how start digitizing your life!

      thanks!
      zzzmoosetech
    • Me four!

      So many drives around (13 TB) but having everything in one place it´s much better.
      I´ll look forward your article, thanks!
      cuitla68
  • Similar Experience

    Have a similar experience.

    Although we didn't digitize books, we did go digital with all our DVDs. We now barely have any, and everything's stored on a nice 16TB central server in a little closet below our stairs. Wires run throughout the house, everything's reachable from everywhere else. 4 fixed XBMC installations for viewing all the material. All our files are stored there. The most expensive components are the HDs, everything else was "repurposed". Periodic backups occur of our most important stuff, and they're stored offsite.

    Having everything centralized is wonderful.

    Look forward to more. In specific, I'm curious what sort of hardware failure you can survive, and associated costs.
    hades_2100@...
  • about the same experience

    We stopped buying physical media and paper books when we moved in our current house about nine years ago. All existing DVDs, CDs and tape cassettes were digitized but we keep the paper books (trick: part of this is actually stored in our parents houses). Our difference is that we are completely wireless.

    As we are about to move to a new much larger house, we intend to build few more storage servers, mainly for redundancy and of course put wires everywhere.

    But I wonder about the service provider you used.. How are you able to trust them they actually shredded your documents and that nothing was lost in the conversion process?
    danbi
  • Backup?

    A backup unit in the closet of the same house won't help if you have a fire!
    I bet "Carbonite" hates you!! LOL

    WHY does anybody NEED thousands of movies or songs......there might be a dozen movies I would watch more then one time.
    jimbritttn
    • WHY does everyone ASSUME

      that everyone else's needs must be the same as their own?
      frylock
  • Give us some of your music tracks to listen

    Put some samples on sound cloud :)
    Tomas M.
  • Bring it on!

    Sounds like you've gone a different way to me: I've plumped for a fast workstation with SSD's doing ... you know, work (in progress) ... and slow data servers doing ... storage, backup and cloud interface for all the stuff I never intend to look at again ;-).

    Areas I'm always looking for ideas on:

    1. GBE is fine but INFINIBAND would be better. The cost of an INFINIBAND switch is still too high though. (Robin Harris wrote about this a few years ago.)

    2. Despite all the ZDNET bloggers telling me how wonderful the cloud is and how generous MSFT, APPL and AMZN are going to be forever ... I don't trust them as far as I can throw a datacentre. So I'm buying ONE copy of Office 2103 and making it available via remote desktop. Not so good if one is off-site: any ideas about a cheap REMOTEFX or TERADICI product to speed up WAN access?

    3. Availability at home.

    a. UPS to protect from power outages. 6-8 hours. Which, how?
    b. Dual ISP connections. I'm thinking about CONNECTIFY http://www.connectify.me/dispatch/. Fibre + 3G backup?

    As long as I don't have to listen to Gewirtz-musik I'll be tuning in ;-)
    jacksonjohn
  • Possible copyright infringement!

    Technically, once you dispose of the hard copy you have no proof you ever bought it. This makes the digital copy illegal. So, all your digitized media could be considered copyright infringement.
    cuulblu
    • Very true...

      EULAs and copyright laws specifically mention that copying or "backing up" stuff is for archival purposes only. You are basically multiplying your copies of items, i.e. copying a movie instead of buying the hard copy and a digital copy. Of course newer movies like Blu-Rays that include hard & digital copies wouldn't apply.
      thomascwhitfield@...
    • Very true.

      When I build out my media server, I plan to pack all the discs away in boxes to store in the garage or attic. I never digitize something without keeping the original as proof that I own it. Storing my media in boxes is fine for that function and removes MANY shelf units full of media taking up floor space. I only have around 700 CDs, but I have over 1200 Blu-rays and DVDs (and growing.) My servers are going to be 24 drives each initially because of all the Blu-rays. My hundreds of books will be replaced over time by Kindle versions, rather than local storage. Digitizing books is just too much of a pain.

      I look forward to reading your entire series on this project. I want to see how your parts list compares to what I've come up with. I also want to hear what media server software you decide to use.
      BillDem
      • Re: I never digitize something without keeping the original as proof that I

        Makes no difference. It is the act of digitization that is illegal.
        ldo17
  • Good point!

    Never thought about this way, but it is true... unless you keep those scanned receipts.

    No one keeps them, but Gewirtz family do :)
    Tomas M.
    • Back to the 1990s

      We have all our paperwork, for everything, going back to the early 1990s (and some going back even further). Now, most of it is in PDF, but you begin to see what that took so much space.
      David Gewirtz
  • looking forward to hearing more

    My young man and I are looking for a house together and we've got something like 70 feet of bookcases. Each. Just finding a house with that much blank wall space is a challenge. We are starting to digitize the music and movies, but I have to admit I'm a purist and like opening the cases and playing Albums and so on. I'm also looking forward to hearing about installing the broadband.
    slfisher@...
  • This blows me away!

    And, has given me some ideas! I am looking forward to future installments. I follow you on Twitter, so I hope you will post a notification there when the next installment is ready!
    Rodo1
  • Just go all in - once!

    http://blog.backblaze.com/2013/02/20/180tb-of-good-vibrations-storage-pod-3-0/

    will likely take care of you for quite some time--and in a cost effective way.
    Bruce_B2