Search data analysis opens up intelligence gathering

By tracking search terms, US officials may have foreseen the 'Arab Spring' uprisings before they happened. Imagine what it can do for business intelligence gathering.

A report by NPR’s Dean Temple-Raston posits that many US intelligence officials were caught by surprise by the “Arab Spring” uprisings, as traditional sources and analysis didn’t do a good job of foreshadowing the emerging popular eruptions. However, had they been tracking a commonly available search engine service — Google Trends — they may have detected the popular mood before it happened. There are US government agencies that are employing Google Trends to great advantage, however:

“Back in 2009, during the swine flu epidemic in the U.S., the National Institutes of Health used Google Flu Trends to track outbreaks of the disease. It turns out that when people started to feel feverish and nauseous, they would go to Google to check out their symptoms. While it wasn’t a perfect indicator, Google Flu Trends often beat government predictions about flu outbreaks by a week or more.”

The report continues: “Imagine using the Internet to do the same thing in predicting political unrest.” West Point’s Gabriel Koehler-Derrick and Princeton’s Joshua Goldstein have been studying this phenomenon, and say Google Trends may be a way to effectively tap into the public mood and spot trends before they manifest themselves. Such was the case with Egyptian citizens doing searches for Tunsia, the first country to erupt in popular protest.

Researchers and intelligence officials have long understood the power of open source content analysis to figure out what may be happening in a particular country. In World War II, intelligence officials monitored radio programs from Germany and occupied countries, and were able to accurately deduce changes in troop concentrations.

Savvy marketing firms are also leveraging search data in a similar way to spot business and consumer trends. Open source analysis provided through services such as Google Trends represent a potentially informative form of prediction.

In addition, many organizations are adopting enterprise search for internal data and content, and such analysis of search queries may help spot trends inside the corporate walls as well — such as an increase in sales in a certain region, a problem with a product line, a supply chain bottleneck, or even an employee uprising.