Using the Internet 100x more efficiently

Storage and network bandwidth often substitute for each other, depending on which is cheaper. Now researchers at MIT, Stanford and Adobe have demonstrated how computation can preserve network bandwidth.

My rule of thumb is that storage and network bandwidth are interchangeable at the 80 percent level. If smartphones had copious storage, they could make do with only 20% of their bandwidth. And if bandwidth was high and nearly free, mobile devices would only need about 20 percent of their storage.

But to what extent can computation subsititute for network bandwidth and/or storage? That's a question that a recent paper from researchers at MIT, Stanford and Adobe offers a partial answer for.

The Siggraph Asia paper, Transform Recipes for Efficient Cloud Photo Enhancement, by Michaël Gharbi, YiChang Shih, Gaurav Chaurasia, Jonathan Ragan-Kelley, Sylvain Paris and Frédo Durand, looks at the problem of providing high-quality image processing on network and power limited mobile devices. Their solution maximizes efficient use of scarce resources through a creative mathematical architecture.

Network overhead

Much is made of cloud efficiency, but often without taking into account the network costs. A 6 second computation on a 16 megapixel image locally could cost 20 Joules, while performing the work on a cloud server would take 14 seconds and use 54J due to network overhead.

Here's the paper's diagram of their new process:

Image processing data flow.png
Network-efficient image processing data flow. Courtesy the authors

By reducing the network data load through using low-res proxy photos and innovative transform recipes, the researchers show that using only 1% of the data they achieved high fidelity results. Drastic bandwidth reductions such as this are key to bringing another several billion people on the Internet.

The Storage Bits take

Bandwidth is often the most expensive part of any system, be it a storage array or a cloud-based supercomputer. Network bandwidth doesn't follow Moore's Law, so its costs grow quickly relative to storage and computes.

I hope this research spurs others to look at how computes can replace network bandwidth. That will bring Internet costs down for all of us.

Courteous comments welcome, of course.