Did software put the universe in a shoebox yet? The world catches up to Yale's David Gelernter

(This is an updated post from March 2009. This marks the 20th anniversary of the first publishing of David Gelernter's profound work, Mirror Worlds: or the Day Software Puts the Universe in a Shoebox...How It Will Happen and What It Will Mean.)

Computing transcends computers. Information travels through a sea of anonymous, interchangeable computers like a breeze through tall grass. A desktop computer is a scooped-out hole in the beach where information from the Cybersphere wells up like seawater..."

-David Gelernter, The Second Coming: A Manifesto, 2000

In a powerful interview from Edge Foundation, Yale scientist David Gelernter was joined by John Markoff and Clay Shirky in an inspiring discussion of Gelernter's prescient visions of cloud, Web 2.0, and Semantic Web -- years before anyone else was talking about them -- and where we're going with technology. (With a trio like this, you know it's going to be good....)

Gelernter himself has always been a step or two ahead of the industry, publishing the book Mirror Worlds: or the Day Software Puts the Universe in a Shoebox...How It Will Happen and What It Will Mean in 1991, which predicted the rise of the World Wide Web among other things. In his 2000 work, pulished at Edge, The Second Coming: A Manifesto, he predicted the rise of cloud computing. As he explained in the recent interview:

"The central idea we were working on was this idea of de-localized information — information for which I didn't care what computer it was stored on. It didn't depend on any particular computer. I didn't know the identities of other computers in the ensemble that I was working on. I just knew myself and the cybersphere, or sometimes we called it the tuplesphere, or just a bunch of information floating around. We used the analogy — we talked about helium balloons. We used a million ways to try and explain this idea."

Gelernter also discussed his original vision (circa 2000) of the rise of the Semantic Web and advanced search, which he called "lifestreaming," in which all the documents and data that are part of your existence you need are retrievable at any time, on demand.

In preparing his initial work for Mirror Worlds, Gelernter said he had reservations about the idea of information being stored and managed across interconnected computers, wondering if it was a "stupid" idea that "would never work." (This was 1991, after all.) "It is grossly inefficient. There is no way that you can take information, just float it out there, and expect people to search this whole vast collection, or somehow or other find what they want. And, you know, how are you going to find out what computer to put it on? How am I going to know what computer to look for it on?"

He says as he gained experienced and continued to research the problem, he determined that "In the final analysis, "the question is not, what can software engineers build? It's the question, What do users need? If we identify our user need, the software technology will come along — in combination with hardware, obviously, and interconnect technology."

In the interview, Markoff mentioned that many people appear to be abandoning their "fetish" for computers, as the emphasis has shifted to the world of information and content that is accessible via the Web.

"Easily half the world doesn't like playing with machines," Gelernter said. "It's not something they enjoy doing. It's not something they take to." He added that the mission of the computer industry itself is shifting, albiet begrudgingly: "What are we looking for and should we have computers in mind, or should we have software in mind, or should we have actual users in mind? I think the average programmer still doesn't understand that he is not a typical user. The average programmer still thinks that insofar as people don't find his software easy to use, it's because they are childish, or ignorant, or just obtuse."

The original work of Mirror Worlds may be 20 years old, but, as Markoff put it, "the world is just now catching up with that.... It's clear now with cloud computing that that's the direction the world's moving in. But only now."

In his 2000 work, Gelernter also predicted the rise of cloud computing, as well as the rise of the Semantic Web and advanced search, which he called “lifestreaming:”

“A lifestream is a sequence of all kinds of documents — all the electronic documents, digital photos, applications, Web bookmarks, rolodex cards, email messages and every other digital information chunk in your life — arranged from oldest to youngest, constantly growing as new documents arrive, easy to browse and search, with a past, present and future, appearing on your screen as a receding parade of index cards. Documents have no names and there are no directories; you retrieve elements by content: “Fifth Avenue” yields a sub-stream of every document that mentions Fifth Avenue.”

Gerlernter’s career hasn’t always followed a smooth, predictable path. In 1993, he was a victim of a mail bomb delivered by Theodore Kaczynski, known as the “Unabomber,” who sought in a twisted, violent way to protest the rise of technology. The incident, which severely injured Gerlernter, brought him to the attention of the mainstream media.

Gelernter said that early on, he had reservations about the idea of searchable information across interconnected computers:

“This is a very pretty idea. This is a beautiful elegant idea. It’s stupid because it’s impossible. It will never work. It is grossly inefficient. There is no way that you can take information, just float it out there, and expect people to search this whole vast collection, or somehow or other find what they want. And, you know, how are you going to find out what computer to put it on? How am I going to know what computer to look for it on?”

He says as he gained experienced and continued to research the problem, he determined that “In the final analysis, “the question is not, what can software engineers build? It’s the question, What do users need? If we identify our user need, the software technology will come along — in combination with hardware, obviously, and interconnect technology.”

These ideas were further expressed in Mirror Worlds. Gerlernter said he was inspired by the ponds within the villages he saw around Cape Cod, reflecting the surrounding churches and buildings:

“The idea began with the idea of delocalized information floating out there so that I could look into my computer and without knowing where to look, what file on which computer, I can sort of tune in the information I wanted the way I tuned in CNN on a TV. I don’t have to know where CNN is and I don’t have to know on what operating system my TV is running, or the software on my cable box — I just tune it in and I assume it’s there.

“So this is going to the cybersphere and the real world will be mirrored on the surface of software essentially. Instead of having to penetrate the real world, go places, deal with institutions in their real-world manifestations, which involves a lot of trouble and a lot of time and a lot of energy, in some cases necessary and desirable but not always, I’ll be able to tune in any part of reality I want.”

The original work of Mirror Worlds may be 18 years old, but, as Markoff put it, “the world is just now catching up with that…. It’s clear now with cloud computing that that’s the direction the world’s moving in. But only now.”

Gelernter says the only major fault with the book, looking back on it, was that it was “too conservative.”

Thanks to JP Rangaswami (”Confused of Calcutta”) for originally surfacing this interview.