The optical storage surprise

The optical storage surprise

Summary: Have you bought a $40 BDXL Blu-ray disc? Me neither. But someone must be - the same people who are driving research into even higher density optical drives and media. Who is it?

TOPICS: Storage, Cloud

Remember when Amazon Web Services (AWS) announced Glacier, a data archiving service, almost 2 years ago? It offered long-term, slow-retrieval (3-5 hours) storage for the then unheard of price of 1¢/GB while promising to maintain several copies across geographies.

Pretty amazing. A little less amazing today now that disk prices are reaching 3¢/GB, but there's still power, cooling, mounting and drive replacement costs to consider in addition to those multiple copies.

At first many assumed that Amazon was using tape. But the couple of remaining tape robot vendors haven't shown any unusual signs of life and there were some denials. Plus the long-term storage requirements for tape require a level of climate control that large data centers may not support.

So it wasn't tape.

Then the general estimate was that Amazon must be using disk. Perhaps the new Shingled Magnetic Recording (SMR) drives that, in theory, could double existing drive density at the cost of expensive rewrites.

Seagate admitted they'd sold a million SMR drives - and it wasn't through NewEgg. WD is getting on board with SMR as well.

The plot thickens
But if Glacier's data was stored on disks - even spun down disks - why the tape-like 3-5 hour retrieval delay? Disk drive robots?

No. Disks are sensitive to physical handling. I've never seen an HDD handling robot or the Zero-Insertion Force drive connector that would be required to minimize physical shock.

One more thing: all current tape libraries are designed to handle ≈200 gram tapes, not 600+ gram 3.5" HDDs. 2.5" drives are ≈100 grams, and have much better shock and vibe specs, but there hasn't been even a whisper of SMR notebook drives, plus the economics are much worse.

The Storage Bits take
Therefore, by a process of elimination, Glacier must be using optical disks. And not just any optical discs, but triple or quad-layer Blu-ray discs.

Not single discs either, but the otherwise inexplicable Panasonic 12 disc cartridge shown at this year's Storage Visions conference. That's 1.2TB in a small, stable cartridge with RAID so a disc can fail and the data can still be read.

For several years I didn't see how optical disk technology could survive without consumer support. But its use by major cloud services explains its continued existence.

Comments welcome, of course. There's more evidence for this theory, but I'll spare you.

Topics: Storage, Cloud

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • How about Holographic Storage or DNA Storage?

    Did you try eliminating the next-gen storage using Holographic Storage or DNA Storage?
    • Amazon is good. . .

      But not that good. Nobody has been able to make holographic storage work yet, despite investing hundreds of millions. DNA? Well, Seattle has a strong biotech industry, but until we start seeing AWS recruiting lab techs I think we can discount that as well.

      Plus, Bezos likes robots.

      R Harris
  • spectralogic tfinity

    It is tape as I understand it. A t-finity unit
    • AWS has denied that Glacier uses tape

      And SpectraLogic's sales don't support that idea either.

      R Harris
  • @Robin I don't think a 600 gram weight or shock would be much of an issue.

    I've worked with some of the older machines (8mm) and they were easily strong enough to handle the weight.

    As far as shock goes, you know that HDDs aren't very sensitive to shock in a spindown state and these library handlers are pretty gentle anyway.

    Your best argument is the time. But even if it were optical disk - hours? Doesn't sound likely. I couldn't even explain this with tape because with modern tapes random access is possible though slow - but not that slow. I recall it taking just a few minutes to access a given file on some 4mm tapes.
    • Robots handlers are expensive.

      If it is disks, it will be done electronically, with few to no moving parts other than the disk itself.

      Wires are cheap. Robotic disk handling is not. And disks are still sensitive to being dropped by anyone - even when spun down.
      • Disk slots are like parking spots

        One in downtown Manhattan will cost more than one on a farm in Vermont. A big data center is Manhattan - even if it is located in Vermont - while a warehouse is Vermont.

        I suspect that AWS takes a group of Blu-ray 12 packs, slaps a barcode on them and trundles them off to nearby storage. They know about barcodes and trundling, so that would work.

        R Harris
    • Shock & vibe: agree

      Disks are much more robust than they were in the 90s, but they still don't like to be handled.

      And what about the ZIF connector? Sure, they could go with something like the Seagate Kinetic Object Store - Ethernet connectors - but that hasn't been around long enough to have supported Glacier.
      R Harris
  • Flaw in the math....

    Glacier is not $.01/GB it is $.01/GB/Month with a three month minimum so the lowest cost to place data in Glacier is $.03/GB for 90 days. If the typical customer retains data for 12 months, they are then paying $.12/GB and they have to remember to "de-archive" the data or the bills keep coming.

    So if it is on disk media, the disk @ $.03/GB for the hardware is paid for in the minimum required retention period and the subsequent months pay for power/cooling/network/profit/etc.... They can keep a lot of that cost down by using other disk architectures that you have written about that use a JBOD methodology. Once disks are full they can spin them down to save power and cooling (but not space) and only spin them up when retrieval is necessary. That delayed spinup and potentially low bandwidth access to storage to mitigate costs can all contribute to an extended (3-5 hour) recovery time frame and keep overall costs down.
    Freddy McGriff