Combating 'artificial stupidity' (with a little common sense)

Wolfram Alpha, billed as a computational knowledge engine, has come a long way since its launch in 2009. But overcoming its latest hurdle -- "common sense" -- may be its biggest challenge yet.

Wolfram Alpha, billed as a "computational knowledge engine," has come a long way since it launched in 2009.

If you're unfamiliar with the service, its mission is to tell you about things you want to know: either through description or computation.

For example, if you type in the name of SmartPlanet's parent company, CBS, it will tell you when it was founded (1941), how many U.S. households it reaches (97 percent) and how many full-power affiliate stations it owns (a little over 200).

Or if you type in, "how many degrees Celsius is 78 degrees Fahrenheit" -- the temperature of the moment here at SmartPlanet HQ -- it will tell you 25.56 degrees Celsius, as well as offer conversions in Kelvin, the Rankine scale and the Réaumur scale. (Clearly, people who use computational knowledge engines tend to be a bit geekier than the average population. Though to be fair, even the Siri Digital Assistant on Apple's latest iPhone uses the service.)

As with any computer that's tasked with trying to understand what a human truly means, Wolfram Alpha has its share of inaccurate responses. Founder Stephen Wolfram notes on his blog this morning that the system has surpassed the 90 percent accuracy mark -- but then, in a gracious display of humility, shows where the system has gone wrong.

Such as when you type in, "Fall of Troy":

The result: an answer so absurd it's funny.

But fixing these misfires is much harder than simply finding them. Wolfram whittled down the problem into two causes: when the system offers a response to a query it doesn't understand, and when the system doesn't know an alternate meaning for a particular phrase or query.

It's kind of like a human, he writes:

For humans, we don’t yet know the internal story of how these things work. But in Wolfram|Alpha it’s very well defined. It’s millions of lines of Mathematica code, but ultimately what Wolfram|Alpha does is to take the fragment of natural language it’s given as input, and try to map it into some precise symbolic form (in the Mathematica language) that represents in a standard way the meaning of the input—and from which Wolfram|Alpha can compute results.

By now -- particularly with data from nearly 3 years of actual usage -- Wolfram|Alpha knows an immense amount about the detailed structure and foibles of natural language. And of necessity, it has to go far beyond what’s in any grammar book.

When people type input to Wolfram|Alpha, I think we’re seeing a kind of linguistic representation of undigested thoughts. It’s not a random soup of words (as people might feed a search engine). It has structure -- often quite complex -- but it has scant respect for the niceties of traditional word order or grammar.

It's this fuzzy bridge between artificial and organic -- we call it "common sense" -- that the system must navigate. And, since "common sense" is not a global resource -- what's obvious to someone who works a certain job or lives in a certain location may not be to someone else -- it poses a problem to a computer.

The solution, Wolfram argues, is that computer systems simply need to know more about the world. A database of all the world's knowledge isn't sufficient for Wolfram Alpha to serve its intended purpose; it also needs to parse and understand, too. Be a little more human, basically.

But not too much. "After all these years," Wolfram writes, "perhaps it’s time to upgrade the Turing Test, recognizing that computers should actually be able to do much more than humans."

This post was originally published on