Steve Lohr of the New York Times has an well reported story on the how various corporations, cities, crime fighters, financial institutions and even cement companies are using data mining to improve performance. Data mining isn't new to the world, but it is becoming more mainstream, Lohr reports.
He quotes Erik Brynjolfsson of the Massachusetts Institute of Technology, who place the current state of data mining and business intelligence in perspective:
“We’re really on the cusp of being able to understand what goes on inside corporations in a much more scientific way than ever before,” he said. “It’s similar to the way that the microscope opened up biology in the 17th century, so that you could see blood cells. Now, we can start to see bits of information as they flow through the organism of the corporation.”
Mainstream doesn't mean that you can simply dump your streams of information into a data warehouse and start getting brilliant insights that can save thousands or millions of dollars by inference. Data mining requires expensive "microscopes" from companies such as SAS, Business Objects and Microstrategy.
What's needed to make data mining truly mainstream is an on demand data mining mashup service that allows mere mortals to pour data, possibly in memory, use a simple configuration wizard to set up the analytics and click OK. Something like Google Analytics but generalized for almost any kind of query. Does such a service exist?