Big data initiatives not quite delivering yet, survey shows

Capgemini survey of 225 companies shows disappointing results for big data efforts so far; but there is movement to the cloud.

Big data is seen as the ticket to competitiveness for today and the years ahead in an intensely competitive global economy. The ability to capture data on transactions, customer preferences, and product adoption, and then be able to predict where things are going, will help organizations lead the way within their industry groups.

Photo: Joe McKendrick

Everyone's moving forward with big data; everyone is competing on analytics; and everyone is making smart, data-driven decisions.

Or so it would seem, reading all the reports.

But the reality is big data nirvana is still a long way off. Big data initiatives require a healthy mix of organizational impetus, new technologies, skills, and innovative thinking. The same gremlins that vexed many organizations' business technology advancements over the years -- be it client/server computing, ERP, or service-oriented architecture -- also are vexing big data projects.That is, enterprises are pouring a lot of money into technologies they hope will deliver transformative results. But technology is only a tool.

A new survey of 226 executives surfaced some interesting results pertaining to big data adoption.

For starters, there's some notable progress with cloud. The survey report states that "only" 36 percent of global organizations use cloud-based big data and analytics platforms. Actually, 36 percent seems like a remarkable uptake of cloud-based data analytics, considering the fact that most analytics vendors have only just begin to start moving their clients to the cloud-based configurations.

Other than what appears to be significant progress into the cloud, the authors of the Capgemini survey report a mixed picture of adoption and results. For example, there's general agreement that big data is a big deal to enterprises. For example, nearly 60 percent of executives in our survey believe that big data will disrupt their industry within the next three years.

However, there has been slow progress in bringing big data into the core of the enterprise -- the survey shows that only 13 percent have achieved full-scale production for their big data implementations, with predictive insights extensively integrated into their business operations.

The benefits have not materialized, either. In the survey, only 27 percent of respondents described their big data initiatives as "successful," and only eight percent described them as "very successful" In fact, organizations were found to be struggling even with their proof-of-concept projects, with an average success rate of only 38 percent.

The survey finds the main obstacle to big data success is scattered silos of data -- 79 percent of organizations have not fully integrated their data sources across the organization. This means decision-makers lack a unified view of data, which prevents them from taking accurate and timely decisions.

Ineffective coordination of analytics initiatives is another challenge that holds back big data efforts. "A significant number of organizations operate with scattered pockets of analytics resources or with decentralized teams that function without any central planning and oversight," the report's authors state. "As a result, best practices from successful implementations are not shared across the organization,initiatives are not prioritized, and resources are not deployed in the most effective ways."

Additional issues faced by enterprises include a lack of strong data management and governance mechanisms, and the dependence on legacy systems.

There isn't a lot of clarity as to what impact, if any, big data efforts may be having on organizations. In fact, 67 percent of executives admit they do not have well-defined criteria to measure the success of their big data initiatives. As with any initiative, there is a lot of noise and hustle-bustle going on in enterprises, and success may be the result of any number of these initiatives. The challenge is to find out how much of a big data initiatives are playing in success.