Sony's 181 petabyte optical library

One of the mysteries of the storage industry is why Sony and Panasonic are still investing in optical discs and drives, especially in the 100GB disc and above market. Look on Amazon and 100GB disks are now about $10 each, or ≈10¢ per gigabyte - well above the 3¢/GB that large hard drives offer.

My hypothesis is that some large, non-publicized, players are buying 100GB discs by the boxcar load - and planning to upgrade ASAP. Facebook we know is using Blu-ray. Amazon's Glacier and the intelligence community are also likely mass consumers.

Now that hyperscale technology is filtering down to the enterprise from Sony, who bought an optical storage company started by a couple of AWS alums last year, in the form of a modular optical storage system - that Sony calls Everspan - that expands to store 181PB on 604,928 discs.

Alert readers will note that these must be 300GB optical discs - but no one is shipping 300GB optical discs. Instead, these appear to be writable 300GB cartridges - 12 discs per cartridge - in a robot-friendly cartridge. They are just becoming available online for about $45 each.

Sharpen the pencil

An LTO-7 tape cartridge with a 6TB raw and 15TB compressed capacity is now about $140 online, or about $10/TB compressed. How can a $150/TB raw optical disc compete?

Unlike organic dye-based recordable DVDs that you may have had a bad experience (I have!) with since they often self-destruct in a few years, these high density Blu-ray discs are much more stable. 50 year warranties are common, and the environmental requirements are less stringent.

Tape, on the other hand, is subject to a variety of maladies, including print-through, mold, binder failure and others that limit its useful life. While you may be able to watch a movie on a 25 year old VHS tape, that's analog, not digital storage. I've known tape users who replace their tapes every 3-5 years.

These issues can be minimized with periodic re-tensioning and air-conditioned and humidity controlled storage. But the hyperscale data centers are going with free cooling, which leads to larger temperature and humidity swings than tape likes.

Thus if the lower AC costs and long replacement cycles, plus the random access speed of discs, are factored in, optical has a case to make against tape.

The Storage Bits take

It is cheaper to store data than it is to sort it out and delete some - at least until AI systems makes it cheap - so I expect the archive market to grow at a healthy pace over the next decade.

While tape's doom is often announced, it has outlasted all the naysayers for several decades. I think it will continue to find a place in deep archives for years to come.

But the Sony archive system fills a gap by offering a very scalable system with good economics and - I would guess - significantly faster access times than tape. Also, the 12-disc cartridges should include some RAID-style data availability features to protect against disc failure.

But even the fastest robot can't keep up with spinning disks for archives. The "active archive" market has nothing to fear from optical disc-based archives.

Courteous comments welcome, of course.