When I first heard the expression Big Data my immediate reaction was similar to when I heard about Enterprise 2.0. Here we go, another buzz phrase to get the marketers, anal-ysts and other hangers on to go nuts over. Only this time, rather than taking years to die, I believe the 'big data' crapolathon will be squished in the next year. Here's why.
In a post teasingly titled The Big Data Laggards, Vinnie Mirchandani catalogs a number of 'big data' use cases he has reviewed. Poignant to this discussion and putting to one side the snark aimed at the usual suspects (IBM, SAP, Deloitte, Infosys etc) is the observation that:
The long and short is these projects have been on-going for years, if not decades, and are spread across industries and processes. They may not call their projects “Big Data” but their results are surely impressive.
In other words this is nothing new. At least conceptually.
As I think about this topic I cannot help but go back to the Harrahs case study from ten plus years ago where CEO Gary Loveman applied his understanding of customer loyalty, developed during his tenure at MIT Sloane to the rejuvenation of a tired gaming hall and turned it into a gaming powerhouse. He understood the value of blended real time data coming from multiple sources. More important, he understood that data has to be actionable, business models have to be refined and processes need recasting to reflect the new information and what it is revealing. In Vinnie's example of Union Pacific we see a company that is no longer just a transportation business but a software development house.
If there is anything new here it is merely in the exploding growth of data from many new sources. The industry vendors like to position this as a scary monster that needs harnessing. Our own Zack Whittaker sucked up the Oracle warning that:
...that the world was "drowning" in vast amounts of data
No its not. That's a less than subtle pitch for Oracle X-boxes. Vinnie and others have no problems coming up with stories about businesses and organisations turning data into breakthrough value. I see a very different problem.
Another fascinating use case I came across is using social media as an early predictor of faults in automobiles. A social media monitoring tool developed by Virginia Tech’s Pamplin College of Business can provide car makers with an efficient way to discover and classify vehicle defects. Again, although at early stages of development yet, it shows promising results, and anything which can improve the safety of automobiles can have a very large impact (no pun!).
Now cross reference to Vinnie's stories. What do you see?
Each use case is describing a unique problem that is industry specific
Each case study talks about in-house development or partnering with relatively unknown developer or academic organisations. (There are exceptions, but they are just that, exceptions.)
Most cases involve a level of predictive analysis, not looking through the rear view mirror.
In other words, what we are most likely to see from 'big data' examples in the short term are not solutions that are readily productised except in limited industry specific terms, but unique differentiators. These can only be developed by those with detailed and industry specific knowledge.
I have a couple examples of my own. One I really like is the energy utility example (video) coming from Basis Technology that predicts when customers have the propensity to switch. It has been tweaked to predict when customers will likely need service. What does this mean for the big SIs. Vinnie pre-empts that question:
Against this backdrop, I see so much marketing of the concept from IBM, SAP, Deloitte, Infosys and others. You would think the pioneers above have been using their tools and their talent for years now. Have they? I hope prospects look at a broader list of success stories, not just the references the vendors show them.
Even more, I hope customers do not accept the excuse “the concept is so new, nobody has references” or the proposition that customers should sign up for their MDM, change management and a variety of other products/services they push as “essential” for Big Data success.
Vinnie and I have had a more robust discussion on this topic in back channels. I think there is a genuine business case for bringing change management experts (and I don't mean form fillers of the BPR variety) into the equation. It is where I think their puck is going.
From what I have seen, the real problem is not drowning in data as Oracle (and others) would like you to believe but drowning in possibilities. During a recent conversation with Vishal Sikka, exec board member SAP on what might happen as a result of being able to cost effectively mashup data from many sources and then expose it in real time, he correctly pointed out that in finding solutions, 'We will only be limited by our imaginations.'
I go further. I believe we will need people who can work out how to take the many potential answers to innumerable questions and turn that into strategies that drive exponential growth. The software companies cannot do it and I doubt the current crop of tenured SIs would know how. It will absolutely demand process changes on an unprecedented scale. Curiously though, I think this will be a lot easier than it might seem. Why? When people can see the answers to long held or vexing questions, the process changes they imply are often self evident.
Couple this with the need to build hundreds if not thousands of applications for every industry and you start to see the emergence of a completely different landscape. It is one that initially looks horribly confusing to the well ordered ERP mind-set but which is essential if 'big data' is to deliver on its promise.
That is why I am more than happy to see the power house vendors push the 'big data' message. But only as long as they recognise they are delivering weapons of mass creativity and not the specific solutions themselves. They don't have the depth or breadth of industry understanding. That's why I am equally happy to be associated (and compensated) by the surfacing of startups in the SAP HANA space. these are companies you've never heard of - in one case a youngster in grade school - doing things that were unimaginable in the recent past.