Scale-out object storage is the highest growth segment of the industry - and has been for 10 years - and now there's another option. Optimized for low-cost Backblaze open sourced Storage Pods (see Build an 180TB storage array for $1,943*) the Backblaze Vault architecture promises 99.99999 percent annual durability. Backblaze is planning to open source the core of the Vault architecture in the near future.
How's it work?
The software creates Vaults across 20 Storage Pods - on 20 different racks. Incoming files are split into 20 shards - 17 data, 3 parity - and written to 20 hard disk drives.
The 3 parity shards means any 3 drives, pods or racks can go offline and all data will remain fully available. That's better than any RAID 6 and more efficient.
Linux and the ext4 file system, well tried and reliable, get the bits on and off disk. But the Linux RAID software is not used.
Instead, Backblaze used a simplified fountain code - an advanced form of the Reed-Solomon coding used in RAID arrays - to provide efficient and resiliant storage. The 17/3 data/parity ratio could be changed in a future release of the software.
There is no single choke point on the data path, so as each Storage Pod is added, another 1Gb/sec bandwidth is added as well. Each Vault can handle 20Gb/sec of I/O, more than fast enough for most active archive applications.
The Storage Bits take
Our data archives are growing faster every year, but prices aren't coming down as fast as they once did. Something has to give.
The combination of the ultra-low-cost Backblaze Storage Pod and the new Vault software is certainly competitive with Google and Amazon. If your business needs a lot of resilient storage at a low cost, take a look at both.
Courteous comments welcome, of course.