Your big data projects will probably fail, and here's why

Getting big data analytics to work is about more than buying some software.
Written by Steve Ranger, Global News Director
"Wherever there is a lot of data, uncertainty and complexity, there is opportunity," says Gartner.
Image: Shutterstock
Nearly two-thirds of big data projects will fail to get beyond the pilot and experimentation phase in the next two years, and will end up being abandoned.

Many organisations are unsure how to get started with big data and assume they need to make big investments in tools and trained staff but, according to Lisa Kart, research director at analyst group Gartner, that could be the wrong approach.

"A successful advanced analytics strategy is about more than simply acquiring the right tools. It's also important to change mindsets and culture, and to be creative in search of success," she said.

The analyst group has identified four best practices to steer BI and analytics projects away from the rocks.

1. Choose a business problem that offers a genuine benefit

Project managers should work with business leaders to spot real problems to tackle, by identifying the data-driven business decisions that could provide the biggest impact or the quickest payback. That could include focusing on a day-to-day operational matter, or a big strategic decision such as whether to enter a new geography. "Wherever there is a lot of data, uncertainty and complexity, there is opportunity," says Gartner.

2. Buy (or outsource) if you don't want to build it yourself

The analyst house said many organisations assume they must build advanced analytics capabilities themselves. However, there are other options that are better suited to quick wins, such as using external service providers or buying in analytics packages rather than building from scratch.

3. Identify the execs that need to be won over

Seek out the naysayers, the skeptics, and the decision-makers and get them onside or risk them derailing the project, Gartner advises. However, having a business case that demonstrates the value of the project is necessary but may not be enough to ensure the health of the project -- instead, organisations need to make sure they're embedding the importance of data across the business.

"Frequently, the success of advanced analytics initiatives comes down to the ability not only to deliver the analytics or communicate their value, but also to create a data-driven culture," said Kart.

4. Decide which skills you need in-house

Gartner said it does makes sense for an organisation to build advanced analytics internally if it is an area of strategic importance, and there are many opportunities across the organization to apply them.

In this case, a business will need to build a team that includes the business skills to choose the right priorities, the IT skills to access the data and identify the required infrastructure, and also the data science and quantitative skills to take the right analytic approach to the data. As Gartner notes: "Unfortunately, this skill set is rarely found in one person, but it's essential to have all three skills within a team."

More on big data

  • Four ways your business can start using AI for automation
  • New or old, number-crunching apps draw on big iron's power
  • Five open source Big Data projects to watch
  • Editorial standards