IBM discusses bringing big data applications to businesses

SAN FRANCISCO — Enterprises are recognizing the potential for leveraging the web sources for broader business intelligence, as well as their internal content, according to David Barnes, program director of emerging Internet technologies for the IBM Software Group.

The traditional type of analytics platforms that we’ve run in the past might have created what we wanted or wasted our time, Barnes argued. He posited that the next wave of analytics and big data is concentrated on content-centric web apps with longer running data collection and analytics.

“We hear business users asking for the ability to directly manipulate, analyze, and remix massive data sources and services,” Barnes said.

Barnes asserted that the highest value content is usually a combination of different sources, and what businesses need is the ablity to repurpose this data in the most budget-conscious way.

Big data collection presents all sorts of opportunities, and while Barnes acknolwedged that he hears so much about analyzing social media, he affirmed that there are other things that customers at the C-level want more analysis about. Examples include aiding chief legal officers, retail business planning, IT systems management, big pharma clinical trials, business fraud detection, evidence based medicine, web archiving, and computational journalism.

As far as the last one goes, Barnes explained that we’ll see much more big data colleciton and analysis during the 2012 presidential election season.

Thus, IBM has created Project BigSheets, an insight engine based on Hadoop for enabling ad-hoc business insights for business users at web scale. The idea behind BigSheets is to serve as a community source for use as a horizontal component, unlocking insights embedded in unstructured data and analyzing data previously unavailable to analyze.

Basically, the user points BigSheets to data sources of interest (i.e. unstructured web data, feeds, XML, etc.), and transform that data into visualizations that better define the activity in question.

Barnes offered an example of how BigSheets was implemented as an end user tool at USC’s Annenberg School for journalism and communications. Students — not IT managers or developers — created a “film forecaster,” which collected information from all over the Internet that revealed positive and negative comments about different movies to predict which ones would be blockbusters, and therefore identify those findings easily through an infographic.

The results, which were found to be mostly true in hindsight, were posted to the students’ blog within a day, and that example was also published by The Los Angeles Times.

“If you’re not in the forefront of analyzing the big data at your disposal, you’re not going to make any business,” Barnes remarked.