Big data is reaching a tipping point at which it stops being something reserved for special projects and ring-fenced expenditure - and instead becomes part of the IT department's regular expenditure.
According to the latest survey by analyst Gartner of 437 large organisations conducted in June, big data investment continues to increase through 2015, if only slightly. More than three-quarters of the member companies said they are planning to invest in big data in the next two years, a three per cent increase over 2014.
Organisations typically have multiple goals for their big data initiatives, for example "enhancing the customer experience, streamlining existing processes, achieving more targeted marketing, and reducing costs", the report said.
As in previous years, organisations are overwhelmingly targeting enhanced customer experience as the primary goal of big data projects (64 per cent). Process efficiency and more-targeted marketing are now tied at 47 per cent.
Perhaps unsurprisingly, as data breaches "continue to make headlines", enhanced security capabilities saw the largest increase, from 15 per cent to 23 per cent.
"As big data becomes the new normal, information and analytics leaders are shifting focus from hype to finding value," said Gartner research director Lisa Kart. "While the perennial challenge of understanding value remains, the practical challenges of skills, governance, funding and return on investment (ROI) come to the fore."
When asked about the ROI expected from big data, the majority of those companies with plans to invest in big data and those that have invested expect an, unspecified, positive ROI. However, a large proportion of companies (43 per cent of those planning to invest and 38 per cent of those that have already invested) don't know if their ROI will be positive or negative.
"This uncertainty highlights the challenges in determining the value of big data projects," the report said. Gartner has also warned that the majority of big data projects in the short term are likely to fail.
But another significant factor was the rise in importance of other managers apart from the CIO in big data projects. Last year, 37 per cent of such projects were initiated by the CIO, while 25 per cent were initiated by business unit heads. In 2015, the roles are nearly tied, at 32 per cent and 31 per cent, respectively.
The report, Survey Analysis: Practical Challenges Mount as Big Data Moves to Mainstream, is available from Gartner.