Building a 500PB data center

Summary:Yes, as in 500,000TB, or half an exabyte. And it is a small company - not the NSA or Amazon - building it. Big data is closer, smaller and cheaper than you imagine.

30 years ago a terabyte of disk storage would have filled a data center, if you could afford the $20,000/GB price and the 25,000 hour MTBF. Today you can buy a 1TB USB 3.0 thumb drive, and a small company can start building out a 500PB data center.

That small company is Backblaze, the online backup company, who is currently storing about 40PB of customer data. But they ran out of space at their current data center and moved into a much larger - and less earthquake prone - site near Sacramento, California.

screen_shot_2014-02-02_at_10.27.06_pm

Using their open source storage pods they expect the new site to support 500PB when built out. In this picture each rack supports 450 hard drives or - with 4TB drives - 1.8PB. It's 8 racks wide, for a total raw capacity of 14.4PB.

Backblaze chose their new site with an eye on earthquakes, common in California, but also for flooding. It isn't well known, but in 1862, during the Civil War, a mega-flood made California's central valley a lake 300 miles long and 20 miles wide.

The Storage Bits take<br>There was a rumor last year that the NSA's new Bluffdale, Utah facility might support 1 yottabyte - 1,000,000 exabytes - of storage. Given that the Backblaze racks are fairly dense - I estimate almost 300TB/sq.ft - they'd need over 69,000 of the 8 rack row in the photo to get to 1YB. Plus a lot of power.

But that would be about 1.4 million square feet - or 30 acres - of data center space for storage alone. Given that the data center space in Bluffdale is 100,000 sq. ft. it seems impossible for them to store 1YB. But they probably could manage 50 zettabytes, which is respectable.

It was inconceivable 10 years ago that a small, low-cost storage company could ever build a 5PB facility. That shows how far we've come in the last decade, both in price and density.

Disk drives are one of the great success stories of modern technology. Learning to manage all the data that is stored on them will be the great challenge of the next 20 years.

Comments welcome, as always. What year did you learn what a petabyte was? For me I think it was the late 90s.

Topics: Storage, Hardware

About

Harris has been working with computers for over 35 years and selling and marketing data storage for over 30 in companies large and small. He introduced a couple of multi-billion dollar storage products (DLT, the first Fibre Channel array) to market, as well as a many smaller ones. Earlier he spent 10 years marketing servers and networks.... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.