The bigger question is how much internet data will this thing drain? Using the Cloud is hard enough with the data caps, and you want to add downloading and uploading tons of data in the background. Not to mention killing speeds for those with low-speed connections. If you want to do anything else with your data (i.e. HD Streaming) then you might as well shoot yourself now.
It's the same reason you don't see many people actively seeding torrents.
I spoke to the ex-Microsofties, Parerit Garg and Bassam Tabbara, who founded Symform on a simple premise: the world’s cheapest data storage is sitting unused inside your computer or on external USB disks. To get that free 100 GB, all you have to do is donate 150 GB of your unused disk space.
At five cents per gigabyte - less if you buy a new 2 or 3 TB USB drives - that free 100 GB will cost you $7.50 plus negligible power costs, forever. Not completely free, but you’ll never see a better deal for online storage.
Just put a folder alias on your desktop and you have a way cheaper Dropbox replacement.
How does it work?
The technology uses an advanced form of erasure coding, used in a simpler and less robust form in today’s RAID 5 systems. In Symform’s much more resilient version your 100GB could be spread across as many as 150,000 disk drives around the world, instead of five or six.
To recover a file the system only needs to see the data from 2/3rds of the drives to reconstruct your file. Let’s see how this works.
Let’s start with a 1 GB video file. Using Symform’s client software and user account you put that file into your local Symform backup folder.
The software takes your data, breaks it into 64 MB blocks, encrypts each block with AES-256 encryption, using a hash of the block as the encryption key, and then shards the encrypted block into as many as 96 pieces, where each piece includes an additional 50% of erasure coding replication data.
That’s why to get 100 GB of free online storage, you have to offer 150 GB - 50% more - of your own local storage.
This level of erasure coding goes way beyond what RAID 5 offers. With RAID 5 one disk failure and then a single unrecoverable read error during data reconstruction wipes out your data. With this system fully one third of the drives can be down and your data will still be recoverable.
That is serious disaster tolerance.
More benefits
- Fast restores. Since the data lives on thousands of computers restores happen in parallel across thousands of Internet connections, making reconstruction much faster than relying on a single shared datacenter link.
- Deduplication. Data deduplication - all the rage in the enterprise world - is standard as well. By using a hash of the data block’s content, the system can see if a block is already stored online and avoid uploading it again.
- Parallel uploads. To speed uploads you can make copies of your data, move it to different upload sites, and run them in parallel, letting Symform deduplicate already stored blocks. Cool.
Control
If Symform doesn’t provide the storage, why do they charge at all? Because the control system that keeps tabs on your data requires some hefty computes and storage as well as serious redundancy. But it also means that you are buying a service, at a flat fee, rather than storage whose costs rise with capacity.
The only requirement is that you donate 50% more of your unused local storage than you use online. Fair enough?
The Storage Bits take
Average enterprise storage utilization rates are 30-40% of total capacity. There’s lots of already spinning, powered, cooled and unused storage capacity in the world today.
Unlike Airbnb, the online home-sharing network, the Symform system protects against data vandalism. Each shard is checked to ensure it is the right data and unmodified by another host. They don’t allow applications to access and modify the data, nor can the sharded, encrypted and parity-laden data act as an app on your system.
It is a great concept, but the proof is in the pudding. I’ll be testing out the system. If you do too, please comment or contact me directly on your experience.
Comments welcome, of course.
