Why supercomputers will always have limits

Computational and Big Data problems -- from protein discovery to terrorism prevention -- may be better handled by smart users with weak machines than the most powerful supercomputers.
Written by Joe McKendrick, Contributing Writer

A couple of decades back, in his seminal work MegaTrends, author John Naisbitt pointed out that no technology can ever succeed without a compelling human benefit, referring to it as "high tech-high touch."

These days, in a classic example of high tech-high touch, smartphones and tablets are selling like hotcakes because they offer users ways to communicate, check Facebook and play games -- not because they are seen as portable computational devices.

Likewise, some technologists are discovering that many computational challenges may be better solved by interactive, online games -- not brute-force supercomputing.

In a recently released video of his latest TED talk, Shyam Sankar, director at Palantir Technologies, explains how a long-time biological puzzle in protein analysis was solved in a matter of days by three non-technical, non-biological amateurs playing a computer game called Foldit:

"Non-technical, non-biologist amateurs play a video game in which they visually rearrange the structure of the protein, allowing the computer to manage the atomic forces and interactions and identify structural issues. This approach beat supercomputers 50% of the time and tied 30% of the time. Foldit recently made a notable and major scientific discovery by deciphering the structure of the Mason-Pfizer monkey virus. A protease that had eluded determination for over 10 years was solved was by three players in a matter of days, perhaps the first major scientific advance to come from playing a video game."

For this task, which has significant implications for the treatment of diseases -- "supercomputer field brute force simply isn't enough," Sankar says. Forces such as Big Data are all part of a burgeoning movement toward what he calls "human-computer symbiosis" or "intelligence augmentation" (IA).  The concept was first advanced by computer scientist J.C.R. Licklider in the 1950s.

IA, in fact, turns the vision of artificial intelligence (AI) on its head, Sankar continues. The more technocratic "sexy vision" of AI that has been promoted over the years now requires rethinking in the era of Big Data, he says.  AI may extend human computational power, but it's meaningless without human interaction. "The imperative is not to figure out how to compute, but what to compute," he points out.

Take a couple of examples of human problems that machines can't solve by themselves: crime and terrorism. "When PayPal was first starting as a business, their biggest challenge was not, 'How do I send money back and forth online?'" says Sankar. "It was, 'How do I do that without being defrauded by organized crime?'" He explains why this is so challenging to machines:

"Because while computers can learn to detect and identify fraud based on patterns, they can't learn to do that based on patterns they've never seen before, and organized crime has a lot in common with this audience: brilliant people, relentlessly resourceful, entrepreneurial spirit, and one huge and important difference: purpose. And so while computers alone can catch all but the cleverest fraudsters, catching the cleverest is the difference between success and failure."

Likewise, machine intelligence can help dig up a lot of information about terrorism, but "Osama bin Laden was not caught by artificial intelligence. He was caught by dedicated, resourceful, brilliant people in partnerships with various technologies."

Computers "don't detect novel patterns and new behaviors, but humans do. Humans, using technology, testing hypotheses, searching for insight by asking machines to do things for them," Sankar explains. "As appealing as it might sound, you cannot algorithmically data mine your way to the answer. There is no 'Find Terrorist' button, and the more data we integrate from a vast variety of sources across a wide variety of data formats from very disparate systems, the less effective data mining can be. Instead, people will have to look at data and search for insight. The key to great results here is the right type of cooperation."

(Photo: Palantir Technologies.)

This post was originally published on Smartplanet.com

Editorial standards