Drinking from the fire hydrant

COMMENTARY--Every day companies make bet-the-business decisions about their customers, competitors, new products and even their own reputation--and most do it through clenched teeth. That's because they know that decision making today is an art based on incomplete and conflicting information, and that hunches play a big role in determining which way to go.

In fact, what matters most in business is what most executives don't know.

What if organizations could know? What if they could make decisions based on a scan of the digital universe in real time, and glean from this immense volume of data key trends and relationships affecting their business--along a wide range of dimensions? In other words, what if they had a decision-making dashboard that allowed them to see virtually everything affecting their business and that provided them with on demand insights into decisions--before it was too late?

Banks could monitor risk factors that allow them to comply with new Sarbanes/Oxley regulations. Pharmaceutical companies could find out what physicians, pharmacists and patients really think of their products, and devise marketing strategies to meet these requirements. Record companies could monitor the buzz surrounding their artists and titles, and act on it; and organizations, in general, could take the pulse of key constituents to come up with the most effective strategies. The what-ifs are endless.

But there’s a rub. First, the digital universe known as the Internet is a very big place--literally billions of Web pages, and growing by 50 million new pages daily--that’s 50 million per day!

Second, 80 percent of this data is in unstructured formats like e-mail, spreadsheets, voice and graphics that defy routine searching. Then, think about the difficulty of extracting meaningful insights and relationships from the sheer volume of this data.

That’s why using the Internet as a database to support decision making has been compared to trying to drink from a fire hydrant. Up to now, the Internet has been uncharted territory for anything but routine searching. Of course, current search engines can turn up virtually unlimited results based on criteria like Web page “hits.” But getting beyond these simple search results to meaningful business insights seems like a laudable goal, but is one that requires "magic" that isn't going to be available anytime soon.

This perception is now being challenged. The growth in supercomputing power coupled with the integrative power of advanced text analytics and information management software is making it possible to scan the entire Internet and extract meaningful business insights on demand--as conditions change, and in time to make decisions that will affect business outcomes.

In the same way that the Hubble telescope changed 400 years of thinking about the physical universe, the ability to extract knowledge from the Internet about customers, competitors, products and overall business will become a transformational capability in the hands of innovative business leaders. What wasn’t visible before will become visible for the first time, and this will change the conventional wisdom of business forever.

Here’s how it works. In the case of the launch of a new drug, a pharmaceutical company can collect information on demand from patient chat rooms, advertising sites and competitor sites, which can be used to position and reposition the drug to meet changing patient perceptions. In the case of the release of a new movie, it brings together information from online chat rooms, movie reviewers, box-office sales and gossip columnists, and extracts insights that can to used to market the movie in the most effective way wherever it’s playing.

The underlying technology works in three stages. First, advanced text analytics algorithms use search and index technologies to perform the basic crawling of the Internet.

Second, industry-specific expertise provides the kind of algorithms needed to extract higher values from the information collected. This raw data is then fed into a central supercomputer as nouns, locations and entities, and the supercomputer and software extract insights that can be used to fine tune the organization’s decision making and business processes.

Finally, an application is developed for each subscriber that combines Web data with third-party data sources and, if necessary, information from the organization’s own information databases.

A major global energy company is currently beta-testing an application designed to help the company manage its corporate reputation. The company is able to monitor the Web on a monthly basis to identify and track emerging global and even local issues that could impact its business, which reduces the time needed to respond to potential business threats.

The data mining process involves millions of Web sites, billions of pages of text, third-party information sources and precise analytics defined by geographies, themes and subjects. From this information, the company obtains a 360 degree view of how it is perceived by its most important constituency groups. The results are turned into a monthly report analyzing items across themes, worldwide geographies, dozens of source communities and five competitors.

The best part of this emerging technology is its limitless horizons--breakthroughs are occurring virtually ever day in our understanding of how to harness the Internet as the world’s richest on-demand knowledge base. One thing is for certain, however. Having tasted what this fountain has to offer, no business executive will go back to the old ways of making decisions and suffering the unknown consequences. This kind of advance happens once in a lifetime and has the potential to change everything.

biography
Bob Carlson is vice president for IBM's Web Fountain emerging business opportunity, in Almaden, California.