The Intelligent Web, book review: How today's technology became smart, and how it may get smarter
Artificial intelligence has not happened in the way we imagined decades ago — neither 2001's HAL nor Asimov's robots are anywhere near reality. But could the combination of web search, big data, voice recognition and machine learning systems, and algorithms originally developed to do things like place adverts, bring us a useful kind of machine 'intelligence' — even if it's not an artificial consciousness?
Gautam Shroff, Chief Scientist at Tata Consultancy, calls this combination (the common technology threads behind driverless cars and voice recognition, and the way Google can translate a web page and Facebook can recognize your friends in pictures) 'web intelligence' and suggests it has a big future.
Shroff has a rather looser definition of 'big data' than usual — the sheer amount of web pages, tweets, YouTube videos and credit card transactions — and he assumes that both Moore's Law and the lesser-known Kryder's Law (storage capacity per dollar increases exponentially) will "remain relentless" when even Intel is rather more cautious. But the idea that knowing the right ad to show you means knowing so much about both people and the world around us that it looks like intelligence is a great way to think about the techniques behind a lot of the most interesting advances in technology, from IBM's Watson to driverless cars and Siri.
To explore this notion, Shroff starts by explaining how search engines and ad systems work, tracing relevant developments back in time. He covers a wide range of fields, including Shannon's theory about how much information you can send through a connection (it's salutary to see how many of the key techniques for search and ad placement are mathematical ideas from the 60s and 70s), natural language processing (starting with Chomsky) and machine learning techniques. He also explores how classic AI techniques like predicate logic, Turing machines and expert systems led to ideas like the Semantic Web and features like Siri.
An excellent primer
In fact, the first two thirds of the book provide an excellent, clearly explained, grounding in the key fields that underpin the way we do computing today. This includes not just the big names, like Turing; there's also a useful explanation of Oliver Selfridge's pandemonium theory, which suggests multiple demons shouting as they recognise patterns, which turns out to match at least some of the way the brain works. However, Shroff makes the leap from hierarchical to the networked systems he thinks will scale better to real-world problems without even mentioning the very different problems involved — this book is generally weakest when the author turns to his own theories about how things will develop.
The first two thirds of the book provide an excellent, clearly explained, grounding in the key fields that underpin the way we do computing today.
When Shroff turns to how technology can analyse and predict, it's not just the familiar stories like Google flu trends and Target spotting when customers are pregnant; how wine critic Orley Ashenfelter predicts wine prices based on weather patterns is a story you won't know unless you've read Super Crunchers by Ian Ayres (Shroff also handily summarises key themes from books by Douglas Hofstadter, Malcolm Gladwell and others).
And you'd have to be paying close attention to cognitive research to know about an idea called Sparse Distributed Memory that stores the concepts of images and experiences rather than the full details, which might soon graduate from research to being used for web tracking. Or Palm founder Jeff Hawkins' move into neuroscience and his similar theory of hierarchical temporal memory, which suggests that recognising objects involves some level of predicting what you're about to see. Although Shroff has no details on Google's self-driving cars, there's plenty on Junior, the Stanford-built self-driving predecessor — and a fascinating look at recent research into how flocks of birds fly. That in turn leads to chess-playing algorithms, classical problems such as the travelling salesman (which a self-driving car would need to solve to run your errands efficiently) and whether the behaviour of ants and other genetic algorithms could help.
Jarring notes
Shroff does a good job of leading the reader from field to field and making connections. But the recurring use of terrorist suspect investigation as an example of everything from searching across multiple information sources to deductive reasoning strikes a jarring note. Similarly, using Google search and tracking for ads to explain so many of these intriguing concepts can make you feel that great minds could be working on something more important than showing you just the right adverts.
And despite promising not to cover hard AI and the question of whether machines can think in the human sense, Shroff spends a lot of time exploring the similarities between the brain and the range of AI and 'web intelligence' techniques in the book to propound his theories about intelligence (with the answer usually being 'we don't know yet'). If this doesn't appeal, you may want to skim the final chapter. Plus any writer who credits Vint Cerf with the creation of the internet all on his own needs to do more research.
You may well disagree with some of Shroff's conclusions about cognition or his idea that the ability to "connect the dots" into a wider story is the key feature that distinguishes human intelligence. But thanks to his excellent survey of the mathematics, logic, algorithms and techniques that underlie the various systems and theories involved, you'll have a great basis for doing so.