Is it possible to build protected storage for less than $55/TB? Yes, it is. Here’s how.
Storage Pod II
Almost 2 years ago in Build a RAID 6 array for $100/TB I showed the 1st storage pod. This version includes what the developer and unlimited backup provider Backblaze has learned from running over 16 petabytes of storage on over 9,000 drives.
The new pod cuts the TB cost in half while increasing performance - the pods can saturate a gigabit Ethernet - mostly due to the hard and under-appreciated work of drive manufacturers. The new pod uses Hitachi 3TB drives to boost density and reduce cost.
Build?
Backblaze doesn’t sell the pod, so you’ll have to build your own. Their blog post includes a detailed parts list. You’ll need software expertise as well: they use Debian 5, the ext4 file system and a logical volume manager above the RAID 6 storage but below the file system.
This is a tested-in-production design:
Courtesy Backblaze
A major university stores medical images on them, and there are other production users as well.
Lessons learned
Backblaze is employee-owned, not venture-backed, and profitable - not least because they’ve cracked the cost nut on protected storage. Like major cloud providers, they’ve built their infrastructure on commodity hardware, building resiliency into the software. Fail-soft as it were.
Running such a large number of drives, they’ve learned a few things.
- High infant mortality. They’ve taken to burning in new pods for several days before putting them into service due to infant mortality and the relative difficulty of replacing drives in the middle of the box.
- 5% annual failure rate. That’s across all drives, young and old, of their population of over 9,000 drives. And in 4 years they haven’t seen any drives die of old age.
- Hitachi’s HDS5C3030ALA630 drives have a sub-1% failure rate. The latest and greatest from Hitachi has worked for Backblaze.
- All parts come with a 3 year warranty. That means that other than labor, your maintenance is free for 3 years. Compare that to your BigCo service contract.
- Heat doesn’t correlate with drive failure. Some drives run hotter than others, but Backblaze hasn’t seen any increased failure rates because of it. That finding agrees with earlier research.
Costs
They offer an interesting figure which shows just how profitable Amazon’s S3 storage service is:
Of course, Amazon offers many other services and features beyond a barebones box, but whoa!
The Storage Bits take
Backblaze is giving storage-loving peoples of the world a gift by open-sourcing their storage pod design. Most business files are opened only a few times, so why put them on the most costly storage you can buy?
I think Backblaze could have a nice little business selling these online for $25k - still a steal - but they’re focussed on offering unlimited backup for less than $4/mo. And I thank them for that too.
Comments welcome, of course. Backblaze is not, sadly, a client.
