Tom Davenport: big data is too important to be left to the 'quants'

Tom Davenport: big data is too important to be left to the 'quants'

Summary: The author and visiting Harvard professor says everyone should have a role in analytics. 'Narratives' might tell a clearer story.

SHARE:

One of the challenges with big data — or with most data analytics efforts over the years — is that it has typically been relegated to analysts and holders of Ph.Ds in mathematics or statistics. Thus, it's been highly intimidating territory for most business decision-makers. Organizations are striving to compete on analytics, and hiding the processes and management of data analysis in a back room isn't going to cut it anymore.

Thomas Davenport-from tomdavenport.com website
(Image: TomDavenport.com)

That's the view of Tom Davenport, visiting professor at Harvard University and co-author of the seminal work Competing on Analytics: The New Science of Winning. I recently had the opportunity to chat with Davenport, and the Q&A is posted at CBS interactive's SmartPlanet site.

Davenport, who is also a senior advisor to Deloitte Analytics, spoke about the difficulties of converting to an analytics-driven culture. He is finalizing a new book, Keeping Up with the Quants: Your Guide to Understanding and Using Analytics, which makes the case for better communicating analytics to business decision-makers.  

We touched upon a number of topics, and I was curious why he felt that the existing BI and analytics tools that have been on the market — graphic tools, dashboards, balanced scorecards, and the like — weren't up to the task. Davenport sais that it's time for a new type of platform for communicating data analytics results — a more "narrative" approach.

"We've all grown up on pie charts and bar charts, but there are probably at least tens, if not hundreds of alternative approaches to visual analytics," he explained. "Narratives are a pretty good way to convey information in the past, so maybe we should be converting our data and analysis into stories. People are starting to do that more. Most analysts were unfortunately not trained in how you communicate effectively about analytics, so we've got a long way to go in terms of doing a better job of that."

I also asked Davenport about the role of cloud platforms in promoting greater availability of analytics, and he said cloud isn't quite there yet.  Here's what he had to say: "It's certainly making a difference in terms of analytics on big data because its almost infinitely expandable, much cheaper, people don't have to build up the capability to analyze huge amounts of data all the time. They can expand to suit their needs. There are cloud-based versions of most of the visual analytics offerings, but they don't tend to be [as] sophisticated yet as the on-premises ones, so I think its not really driving things yet. For basic big data processing, and computation and analysis, visual stuff, narrative stuff — not as much at this point."

There is also a tendency to move or embed analytics into applications, with end-users fairly oblivious to what types of algorithms are spitting out the answers being acted on by applications. Isn't automation making it less necessary to understand the machinations of analytics? Too much reliance on automation may be a dangerous thing, Davenport responded. "We saw the challenge in financial services; you have situations like the flash crash, where there were all these automated trading things happening, and we had no idea why. The real challenge is going to be being able to trace the logic and how the algorithms work when things go wrong, so we can intervene and override."

Topics: Big Data, Cloud, Data Centers

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

2 comments
Log in or register to join the discussion
  • The future is backroom analytics

    Machine talking to machine to set prices, detect fraud, optimize campaigns. Just what quants are doing today with their HFT (high frequency trading) algorithms.

    The'll always be space for un-automated processes though, since automated big data processing must be tested by data scientist, before deployment and control by machines.
    vincentg64
  • Who is driving?

    Many decision makers feel constrained by Big Data and Business Analytics. Often they prefer to rely on instinct or intuition rather than the numbers whose derivation are likely a mystery to them. It is not an 'either or' situation. By embracing them, analytics can inform and enrich intuition and instinct.

    Many quantitative types say Big Data 'speak for themselves'. They don't. Human judgment informs the composition and interpretation of analytics.

    The morale of the story is that no one is driving. Big Data and Business Analytics are team sports that require the best judgment of the both the right and left brained.
    williamsholland