DOTS: human-readable digital storage

DOTS: human-readable digital storage

Summary: Reading digital files require a computer, OS and app, which will all be different in 100 years. Why not store human-readable files instead? Now we can.

SHARE:
5

Digital files are dependent on apps to let us to read them. So even if the raw data is perfect - which it won't be -  we may not be able to read many digital files 100 years from now.

But what if we stored human-readable document/photo/video copies? That is the premise of DOTS – the Digital Optical Technology System – that I learned about at the 2013 Creative Storage conference in LA Tuesday.

DOTS technology was developed by Kodak and ignored. In 2011, Group 47 acquired all DOTS patents and tech documentation from Kodak.

Group 47 plans to license the DOTS technology, and is also working to extend the DOTS patent portfolio with improvements and new techniques of leveraging the technology.

What is DOTS?
G47 says that DOTS is:

  • Archival for no less than 100 years
  • Non-magnetic
  • Chemically inert
  • Immune from electromagnetic fields including electromagnetic pulse (EMP)
  • Stable in temperatures ranging from 15º - 150º F and humidity from 5% - 85%.

DOTS can survive almost anywhere humans would choose to live.

How does DOTS store an image?
DOTS stores data optically on a specially formulated, write-once tape in 15 micron-wide tracks:

dots_track

 

These dots can store binary - machine-readable - data directly as the absence or presence of a dot represents one bit. But it can also store tiny human readable high-res photos or movie frames.

dots_tape_sample

 

By breaking each frame into RGB - Red, Blue, Green - components, and each component into 8, 10 or 12 bit planes, a 4k image can be stored on a DOTS tape.

A camera can read each plane, combine them optically, and recreate the original movie frame. Since the data is read with a camera and not a computer, there is no dependence on exotic file formats or specialized editing apps.

Robustness
Computer image file formats are highly compressed - losslessly or not - to save space. But the compression that makes them efficient also makes them sensitive to bit flips in a digital file.

G47 took 24 frames of 4K JPEG 2000 Digital Cinema Initiative compliant images, and randomly flipped 13 bits of each of the JPEG images, a 0.001% error rate. Six of the 24 frames couldn't be opened and five of the frames had visual damage.

Since bit flips are endemic in magnetic storage, it clearly isn't suitable for 100 or even 50 year archives. DOTS solves that problem.

Infrastructure
G47 plans to deliver DOTS media and devices in LTO tape compliant form factors so current tape robots can use it. The optical images read by the tape drives will be converted to digital formats for computer use, but they can also be "read" by a camera and used without a computer.

Each LTO-sized cartridge can store 1.2TB and can be read at a faster than HDD 250MB/sec. And since it is optical, faster 2nd gen readers will be able to read 1st gen DOTS at 2nd gen speeds. Try that with magnetic tape!

The Storage Bits take
Optical storage is attractive because we understand how to manipulate light better than magnetism. DOTS is especially attractive because the data is human readable, dispensing with the entire computer data stack.

Since DOTS doesn't compress images its capacity requirements will be quite large compared to modern compressed images. But if media cost is low enough the ease of long-term storage will make it a bargain compared to magnetic tape.

But less efficient capacity will be a small price to pay for long-term human-readable data storage. Our digital civilization needs something more robust than simply copying digital data to new media every 3-5 years.

DOTS is the best answer yet to this critical problem.

Comments welcome, as always. I'm readying a review of another promising archive medium, the 1,000 year DVD. Images courtesy of Group 47.

Topics: Storage, Big Data, Security

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

5 comments
Log in or register to join the discussion
  • These folks should talk to the Mormon church.

    For theological reasons, the Mormon church has for over a century collected massive amounts of genealogical data worldwide, much of it currently stored on microfilm or microfiche. Their archives are MUCH larger than Ancestry's (which has about 10 billion records). They estimate they add 15 TERABYTES PER DAY and they are interested in EXTREMELY LONG-TERM storage WORM (write once, read many) storage. They store so much that they design their own technology (but not the actual hard drives, etc.).

    Sounds like they could really use something like this, and the Group 47 folks could use the financial and technology resources a tie-in would bring.
    Rick_R
  • This has been done for as long as I can remember

    There's were programs in the late 80's that printed data that could be interpreted back with optical readers. Currently there is this:

    http://ollydbg.de/Paperbak/

    Note the capacity and you'll see why it's not done much.
    happyharry_z
    • Conceptually simular on some levels BUT technically very different

      Data is not "printed". Note from above
      1.2TB in an LTO form factor (current)
      250MB/s RW speed
      Material is not paper
      techbiz2
  • Makes a nice introduction!

    Part of the media volume could be a human-readable description of the algorithm needed to use the other files on the volume. Then the future finders of the archive could REINVENT the old application logic. But if civilization does not collapse, someone will "format-refresh" the archives, that is transcribe them into newly invented formats, as each archive file becomes obsolete. In the original Star Trek: The Motion Picture, when the Enterprise made contact with the original Voyager probe embedded in the monster Vejur, they found the specs to understand the old ASCII code, and were able to communicate with Voyager as its "creator." Though the other premises in the movie were pure fiction, it is certainly that, just as some people become scholars of ancient history and learn how to read ancient scripts, some people in the future will become scholars of our human languages and computer data formats. I can still recite most of the character codes in the old 80-column Hollerith cards from memory, and the rest of the details can be found on the internet today.
    jallan32
  • Could analog re-emerge as the new digital?

    Using an optical representative format could introduce a new age of variable level of detail, highly compressible, highly expressible representations of data whose playback fidelity would only be limited by the equipment in use. This is something to take note of as we could see an emergent new symbology emerge to express data as well as software, and the logic processes in play could both be digital and analog at the same time. Reaching this stage of data representation could also hasten the emergence of AI, since true intelligence is based on the processing of analog input in organic ways. The compressibility of storage may also only be limited by the precision of the optical media involved, down to the angstrom, and we can get beyond the boundaries of 0's and 1's in what is represented.
    rcasey101