Getting started with our massive Media Tank home storage project

It’s been a while since I’ve introduced you to a completely new project, so I figured it was time to open the sliding trays of one of my favorite (and most useful projects): the Media Tank, Mark I. Here’s the most important spec: 20 terabytes of storage across 10 data drives, in one 20-inch mid-tower case.

The whole Media Tank project is actually more than just storage. We ran gigabit Ethernet to every wall of the house (include inside some closets), there’s a second tank (a clone of the first) that’s used for local video backup, and we have an additional three tiers of backup for other types of media.

Over the course of the next few articles, I’m going to take you through the full Media Tank project, including all the tricks that were necessary to cram so many inexpensive drives inside one box and make it all work.

In this article, though, I’m going to spec out the project and explain why we built it.

It all goes back to 2005. My wife and I had just gotten married. We’d decided to move from chilly New Jersey to warm Florida. When we started looking for a house, we realized we’d somehow have to fit the entire contents of my wife’s pre-marriage home, my pre-marriage collection of stuff, and the office for what was then my online publishing company.

We were gonna need a lot of space. A lot of space.

So we rented a big house in Central Florida. We filled a moving truck with 19,280 pounds of stuff. Her clothes, my T-shirts and jeans, her girl stuff, all my gear, her huge music collection, my very large video collection, and both of our book collections. Together we had thousands of CDs, close to a thousand DVDs, and wall upon wall upon wall of books.

Even though the house was a monster, it was still less expensive to rent than either of us spent on rent in New Jersey before we moved in together. Florida isn’t as inexpensive as it once was, but it’s still a lot less expensive than living in the metropolitan New York area.

We managed to combine it all, threw some stuff out, and got some new stuff. After about six years, our way-too-big-to-clean house was even more full of stuff than when we came down here. Our lease renewal was about to come due, and we realized it was time to buy a house of our own.

But we wanted something more manageable. We didn’t want to turn into hoarders. We’d each known a few hoarders in our lives, and we were not about to let ourselves become another hoarder couple. We were committed to cutting back on all that stuff.

Looking around the big rental house, it became apparent that a huge percentage of the space was given to warehousing our books, music, files, videos – our media. What if we could reduce that?

What if we went all digital?

Having the Media Tank made it possible to very comfortably downsize our home.

At first, my wife tried to scan in all our books. We bought an industrial paper cutter and a great scanner called a ScanSnap. Over the course of a few months, she made a valiant effort to scan in all our books.

As it turns out, that project – at the time – proved somewhat impractical, merely because of the physical effort of scanning in the books and keeping the scanner running. She did manage to scan in a hundred or so of our favorite books, plus most of our important manuals (if they weren't available for download).

In the middle of this, the first iPad came out, and we confirmed that reading scanned PDFs on a tablet was just as practical as reading a physical book. The concept worked, at least in theory.

In practice, we wound up getting rid of a lot of our books. This was the third or fourth major book purge I'd done in my life, and each time is as difficult. Each book is a friend. But we donated as many as we could (fewer people and organizations are willing to accept book donations these days). We realized we'd read many of them and were unlikely to read them again. Many of the reference books could virtually be replaced by the Internet. Many others are available on Kindle format if we feel we must reread them at some future time, or available for pennies used from Amazon.

We’re still (almost three years later) trying to get the remainder of our books digitized, though we're nearing the end of our mass scanning days. That’s a story for another article series.

We came to realize if we were going to convert all that physical media (not just books, but files, movies, videos, manuals, even software disks) to digital form, we’d need a central server to store it all.

Next up: what we store on the tank now...

That server would become the Media Tank.

Here’s some of what we store on the tank now:

• Audio, particularly audio we’ve created, along with audio books. We have a complete share dedicated to just non-album audio
• All our music, nearly 1,500 albums. We keep both lossless (flac) and lossy copies (MP3)
• All the books we’ve managed to scan, along with all the manuals for all the products we have, have had, and use
• Nearly all of our bookkeeping records, including all our corporate records going back to before the turn of the century
• My very large collection of licensed loops and music clips, which I used in the composition of as-yet-unpublished music tracks
• Our picture library, containing all our photos, including all of my wife’s snapshots, which she scanned in over the course of a few months, numbering in the tens of thousands
• Our software library, containing ISO images and installers for almost all of our software, replacing the DVDs and CDs they were distributed on
• My client work project center, containing organized, structured, encrypted, and secured client files for the clients I’ve done work for over the years (this is not complete – I have certain “special” client files that have highly secured, regulation-specified homes)
• All my studio video production content and assets, including all the templates and media assets necessary to create broadcast quality video
• Five separate video shares, containing our extremely large video collection
• A special virtual machine share that hosts and runs all the VMs I use across all my projects
• A backup share that backs up the servers that serve our various Web sites, including the more than 100,000 articles published by ZATZ since 1998
• A private “My Documents” share each for my wife and me, containing our own, individual documents that we use and organize for our own projects
• And a central, highly useful EasyShare storage unit that contains most of our operational documents for both running the house and our business interests.

There’s actually more, but that gives you a reasonably good picture of how we use the tank.

Over time, we became rather expert in organizing and digitizing our information.

I’ll tell you about some of it in future articles. We’ve done a lot of it ourselves, and we’ve jobbed out some of it to local service providers. There’s one local service provider we used that scanned in more than a decade of business documents, shredded them, and returned to us a perfectly formatted set of PDFs.

Having the Media Tank made it possible to very comfortably downsize our home.

We no longer had to rent thousands of square feet merely to warehouse physical media. When it came time for us to buy a place, we were able to buy an awesome place just about half the size of the monster rental house. Because it was a fixer-upper, we had the opportunity to tune it for our lifestyle. That meant gigabit Ethernet in every wall, a super energy-efficient cooling system, a gym/dance space, a combined media room and office in our great room, and a space for the broadcast studio in a small room.

Part of the reason we’ve been able to add these interesting function rooms is that we’re no longer warehousing all our media. Even though we’ve yet to go fully paperless, most of it is stored on our Media Tank. Instead of thousands of square feet going to waste holding up paper and plastic discs, we now have a 20-inch tall tower that takes up 1.17 square feet (and a second one in a closet that we use to back it up).

We now do most of our work directly on the tank.

We open files that live on the tank, we edit files remotely and store them back to the tank. While that’s no big feat for something like an article like this, I host all the VMs I use right on the tank, and even edit video straight on the tank. That’s why the gigabit Ethernet became so important.

So, now you know the back story and the amazing benefits we’ve derived from our Media Tank. In subsequent articles I’ll describe how we built the tanks, how we managed to stuff 10 drives inside one mid-sized tower, how we were able to build our massive Media Tank for around the price of a nicely-equipped MacBook Air, some of the software we’ve used to store our information, and more.

It’s been a fun and rewarding journey, and you’re invited along for the ride. Stay tuned for Part II.

UPDATE: Here's Part II.