Long live the Sneakernet: Computing's most resilient network

Long live the Sneakernet: Computing's most resilient network

Summary: When Amazon Web Services' latest---and arguably most valuable---service is a system that allows you to ship terabytes of data to the cloud via snail mail you just have to chuckle. Yes folks, for all the fancy talk of cloud computing, terabytes---not to mention petabytes---of data and technological advancement the Sneakernet is alive and kicking.

SHARE:

When Amazon Web Services' latest---and arguably most valuable---service is a system that allows you to ship terabytes of data to the cloud via snail mail you just have to chuckle. Yes folks, for all the fancy talk of cloud computing, terabytes---not to mention petabytes---of data and technological advancement the Sneakernet is alive and kicking.

The Sneakernet, where someone puts data on a disk, flash drive etc.  and runs it to another computer, is arguably one of our most enduring networks. I still use it all the time. I'm sure I could network my home devices together, but the Sneakernet works just fine.

Multiply the Sneakernet on a grand scale and you understand why Amazon is launching a service called Import/Export. There's too much data to move to the cloud and not enough bandwidth to get it there quickly. Why take five days to move data---and hog up all your bandwidth---when you can toss it on a storage brick of some sort and just overnight it?

Amazon CTO Werner Vogels explains:

In some ways the computing world has changed dramatically; networks have become ubiquitous and the latency and bandwidth capabilities have improved immensely. Next to this growth in network capabilities we have been able to grow something else to even bigger proportions, namely our datasets. Gigabyte data sets are considered small, terabyte sets are common place, and we see several customers working with petabyte size datasets.

No matter how much we have improved our network throughput in the past 10 years, our datasets have grown faster, and this is likely to be a pattern that will only accelerate in the coming years. While network may improve another other of magnitude in throughput, it is certain that datasets will grow two or more orders of magnitude in the same period of time.

Simply put, if you wanted to move a terabyte data set to EC2 it will take you a while. On an enterprise scale, this data-moving problem is yet another hindrance to cloud computing adoption. Amazon gives the following time frame to shipping a terabyte dataset over the network:

But that doesn't capture the true costs. Microsoft Research notes that you still have to maintain that network. And there's labor and support.

Here's a look at the slightly dated statistics from a 2002 Microsoft Research paper:

Microsoft Research's Jim Gray concluded that Sneakernets are the answer to the above conundrum:

What is the best way to move a terabyte from place to place? The Next Generation Internet (NGI) promised gigabit per second bandwidth desktop-to-desktop by the year 2000. So, if you have the Next Generation Internet,  then this transfer is just 8 trillion bits, or about 8,000 seconds – a few hours wait. Unfortunately, most of us are still waiting for the Next Generation Internet -- we measure bandwidth among our colleagues at between 1 megabits per second (mbps) and 100 mbps. So, it is takes us days or months to move a terabyte from place to place using the Last Generation Internet.

That passage was written in 2002. And guess what? We're still waiting. Simply put, the Sneakernet is the most efficient means of moving a terabyte of data around.

Given that fact, Amazon's Sneakernet, the Import/Export service, may become its most appreciated if not technologically advanced feature. Go figure. In a nutshell, Import/Export allows you to ship data on storage devices with a manifest that explains how and where to load the data and map it to Amazon's storage system.

Here's when a move to Import/Export makes sense:

Now there are costs. Amazon will charge you $80 per storage device handled and $2.49 per data loading hour. And then there's the usual storage pricing. But add it up and it's cheaper per terabyte than waiting a week for a dataset to move.

Will the Sneakernet ever go away? Nope. Gray sums it up:

Until we all have inexpensive end-to-end gigabit speed networks, terascale datasets will have to move over some form of sneaker net. We suspect that by the time the promised end-to-end gigabit (next generation Internet) arrives, we will be moving petabyte scale datasets and so will still need a Sneakernet solution.

Long live the Sneakernet.

Topics: Amazon, Data Centers, Hardware, Networking, Storage

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

14 comments
Log in or register to join the discussion
  • Overall sneakernet has a positive upside

    "Never underestimate the bandwidth of a station wagon full of tapes driving down the Interstate" Goes back to the the earliest days of networking.

    Probably the biggest plus for Sneakernet is it forces you to create a backup via the exchange media.
    wkulecz
    • You beat me to it

      That's one of my favorite expressions in answer to the question of getting rid of tape.
      914four
  • RE: Long live the Sneakernet: Computing's most resilient network

    Just goes to show another reason why cloud computing will not be the business standard, at least for a LONG time.

    So now, my important, possibly confidential, has to be physically carried to an external location to be loaded up? Do I need to even go into how insecure that is and how many ways (lost shipping, damaged drive) that could cause a major problem?

    And pay an extra fee to do it?

    vermonter
    • Security?

      Why don't you encrypt the drives and send the key via a different courier.

      And lets face it if its that critical this information would you really want to put it in the cloud and in the hands of a third party anyway?
      Bozzer
    • Actually it's not as far off as you think.

      Take a look at this video: http://www.ted.com/index.php/talks/kevin_kelly_on_the_next_5_000_days_of_the_web.html
      914four
  • By the time we have ubiquitous gigabit broadband connections

    By the time we have ubiquitous gigabit broadband
    connections, hard drive capacity will be at one
    petabyte and it will still be easier to sneakernet
    that data over FEDEX.
    georgeou
    • Not just speed

      Not just speed but also reliability; if you've got a multigigabyte
      connection but the reliability is horrible it won't matter how fast it is.
      Coupled the poor reliability along with the metre's that are now
      running on internet connections in most countries (download/upload
      allowances) and its almost a certainty that sneakernet is here to stay permanently.

      Ironically the supply of high bandwidth could actually help
      environmentally given that big business buildings could become a
      thing of the past as more people work from home and have a flexible
      arrangement. Programmers - why is there even a huge office when
      development could be done at home with informal meetings occurring
      once a week?
      Kaiwai
      • Not Just Speed

        The problem with working at home is that management feels that they have to 'see' you otherwise you're goofing off. I know, I've experienced it firsthand.
        herb643
        • Then have a virtual office wall

          Then have a virtual office wall that transmits
          images both ways. When you walk past that wall
          as if it were some cubical, you see the
          employee. You can also have developer teams
          sit next to these virtual video walls to
          emulate sitting together.
          georgeou
  • vegov fashion is more than easy

    Greetings everyone here.Find out why more than fashion is that easy?
    Welcome To Visit The Fashion Shoes Warehouse:www.vegov.com,any inquiries will be appreciated.
    vegov33
    • Don't think this is what they had in mind.

      ***SPAM ATTACK*** (on ZDNet? You'd think they'd have this problem solved...)
      pocketchalker
  • RE: Long live the Sneakernet: Computing's most resilient network

    old joke... what's the fastest way to get 1 GB (or name your own amount of data) from New York to Los Angeles?
    Answer: FedEx.
    redking44
  • Never underestimate

    Never underestimate the bandwidth of a station wagon full of storage media driving down the new jersey turnpike.
    Rion.Motley
  • RE: Long live the Sneakernet: Computing's most resilient network

    Networked data transfer may become more prominent even before it becomes as fast as Sneakernet. It will become more prominent when it becomes more economical (i.e. when fuel prices to ship mass storage media pass a threshold level. Where the crossover is between the two efficiencies I have no idea).
    JW11