Singularity Summit 2007: Dancing with Dr. Doom
Guest post: This weekend I am at the Singularity Summit 2007 in San Francisco at the Palace of Fine Arts. About 800 people showed up to hear about the issues related to a future in which humans won't be the driving force in delivering scientific and technological innovations, eclipsed cognitively by "posthumans" or machine intelligences. I talked to four of the speakers prior to the event--the podcasts are here.
As we stood outside the Palace of Fine Arts, clutching our cardboard beverages and staring at the assembled throngs, a little graying man came up to Dan Farber and introduced himself.
He said he was Wendell Wallach. He said he was from the East Coast.
Little did I know who he would turn out to be.
When Wendell Wallach, a lecturer at the Yale Interdisciplinary Center for Bioethics, took the stage, he didn’t waste any time with posturing and niceties. He immediately addressed the most fundamental issues that affect human development.
He has thought very deeply about the ethics of artificial intelligence. He has thought very deeply about the morality inherent in creating any system that begins to have a mind of its own.
He presented his case with deep thought, deep feelings and he was by far the most affecting and effective speaker we have seen so far.
His hopes are that computers are capable of weighing up more options before they decide on a response. He hopes and believes computers will not exhibit greed. And the fact that computers have no emotions should make them less susceptible to emotional blackmail.
Yet, Wendell Wallach has another persona.
He is Dr. Doom. In a voice that would surely have won him a presidential debate or at least a part in Law and Order (as a DA, of course), he said he would make only one prediction. That in the next few years there would be a major human disaster caused by a mistaken decision taken by a computer.
Come on, Wendell. It’s almost lunchtime. And now I’m supposed to go sip an oaky chardonnay and wonder what kind of disaster this might be. He wouldn’t even give us a clue what might go wrong.
I suppose the most obvious thought would be that it would be around Thanksgiving and a low-paid IT worker at a nuclear power facility, with turkey on his mind and a paper cup from a wine in a box in his hand, pushes a wrong button or doesn’t notice that the hardware is on the blink.
He steps out the door, humming the words to Abba’s "Thank You for the Music." And poof, there blows Kansas.
What if one night the world’s air traffic control system failed and sent hundreds of planes crashing into each other while the humans on earth mistook it all for a government-sponsored fireworks display?
But perhaps it could be something far more pernicious, far more creepy. What if a government computer system suddenly enjoyed a glitch that changed all of our dates of birth, made us all younger, simply so that we won’t be paid our pensions when we’re supposed to get them, saving billions of government money.
Wouldn’t that be a disaster?
Or what if Ticketmaster suddenly forgot to add the convenience charge every time you bought a ticket? That would surely bankrupt that company and destroy our ability to ever see Genesis again. Or Ted Leo and the Pharmacists. Or the American Idol Roadshow (which didn’t even sell out in Glendale, Arizona, home of the fabulously forgettable winner, Jordan Sparks).
I can only hope that wherever Mr. Farber takes me to lunch, the wine will not have been made by an autonomous mechanical entity.
And I am, of course, not referring to the fine array of products, especially the shiraz, from the Yellow Tail Company when I say this.