I attended a very worthwhile roundtable this week, organized by Actian, a Big Data business analytics company. It was a very good mix of analysts, investors, media and users.
I learned a lot about Hadoop, and Actian, and enterprise use and misuse of analytics primarily because the discussion was not all about how wonderful everything is, but a warts and all, frank discussion of the challenges in the enterprise space for Hadoop based analytics.
Actian says that SQL is still very important in business analytics and for its customers because that's what the enterprise workforce knows. And that Hadoop makes it cost effective to process analysis faster, or add additional data that wasn't practical before, potentially netting companies with a lift in revenues. As much as 2% is typical for large companies.
There's gold in the data but... which data? What are the questions? Do you need to know the right questions to ask? Or can you find them automatically, which is the claim of some? Big Data amplifies the challenges in enterprise business analytics.
It was good to see M.R. Rangaswami, from Sandhill Group, who helped lead the discussion. And Bob Rogers, Chief Data Scientist for Big Data Solutions at Intel, added some interesting perspectives.
Ray Wang, founder of Constellation Research, had a chance to promote his new book with a superb presentation where he didn't seem to take a breath yet it wasn't a breathless speech. I'll post a recording of it, it is well worth hearing.
Disrupting Digital Business: Create an Authentic Experience in the Peer-to-Peer Economy - By R "Ray" Wang.
Ray's take the on the digital economy is that the top company usually gets most of the market, (Please see: The Problem Of Scale: Building The Next Big Silicon Valley Company -SVW) and that the only way to compete against the giants is by using your intuition, your specialist understanding of a market. It's not a cheerful analysis because it implies small businesses can't grow into giant-killers.
Here are some of the findings from a recent survey Actian commissioned about Hadoop in the enterprise:
- Majority can't get no satisfaction (yet): Eleven percent of respondents are "not at all satisfied" their technology investments in reporting, analytics and big data are giving them what they need to help achieve their top business priorities. Twenty-five percent are only "slightly satisfied" and 41 percent are "moderately satisfied." On the upside, 21 percent are "very satisfied" and a rare two percent are "extremely satisfied," showing there is light at the end of the big data tunnel.
- Hadoop is the most valuable thing IT isn't asking for: More than half (51%) of respondents recognize Hadoop could make existing data analytics operations more efficient, a third (32%) believe it will help them find value from existing data, and a third (30%) believe it could make an organization more profitable overall. Despite all this promise, a minority (5%) of IT departments is asking for Hadoop.
- Botched big data projects get CIOs/CTOs fired (among other things): A quarter of CEOs (25%) would fire their CIO/CTO over the failure of a big data project or initiative; nearly half (43%) would fire over a tech investment that led to a security breach for the organization, and almost as many (36%) would fire for a tech investment that won't scale to meet tomorrow's demands.
- SQL is king of big data hill: Forty percent of data scientist and business analysts use SQL-based tools as their primary tool for analytics/data science, 18 percent primarily use SAS/SPSS and a surprising 10 percent primarily use Excel to round out the top three most popular tools.
- Data scientists see promise in Hadoop but also problems: A third (30%) of data scientists and business analysts recognize Hadoop as a cost-effective and scalable way to store data. A fifth (20%) seeks tools to make Hadoop fully enterprise grade, secure and fast, and another fifth (20%) find Hadoop too hard to use and say they lack the talent in-house to make it work.