Some may be disillusioned with big data, but venture capital firm Accel Partners believes a new class of software could help deliver on its promise.
Accel on Tuesday announced it's allocating $100m to its Big Data Fund 2, a sequel to its 2011 fund aimed at startups building "data driven software" (DDS).
Accel said it will invest in companies that make it easier for line of business users like sales, HR and IT managers to exploit big data technologies rather than just data scientists, which are in short supply.
A company Accel thinks fits the profile is RelateIQ, which recently raised $29m from the company, Allen & Co, Battery Ventures and Facebook co-founder Dustin Moskovitz. The CRM product doesn't rely on manual data entry and instead automatically captures data from email, calls, calendars and social networks to generate a real time account of relationships.
Other similar investments it points to include Cloudera, Couchbase, Lookout, Nimble Storage, Opower, Prismatic, QlikTech, Sumo Logic, and Trifacta.
The new class of DDS software includes the "last mile" of big data that allows smaller companies to follow the big data lead of the likes of Google, Amazon and Facebook, which actually have the resources to exploit big data platforms like Hadoop.
"While the rest of the world can now leverage the platform infrastructure software, few companies have the internal resources required to build these last mile applications in house. There are not nearly enough analysts and data scientists to meet this demand and only so many can be trained each year. Software is a far more scalable solution," Accel notes in a recent whitepaper.
Earlier this year analyst firm Gartner research director Svetlana Sicular noted that clients using Hadoop were becoming disillusioned with outcomes after having linked together multiple sets of unstructured data. Difficulties included framing the right question, selecting which of many questions to ask, and how to reliably answer the question.
According to Jay Parikh, vice president of infrastructure engineering at Facebook and a member of Accel's Big Data Advisory Council, last mile big data technologies should aim to address these issues.
"The last mile of big data will be built by a new class of software applications that enable everyday users to get real value out of all the data being created," said Parikh.
"Today's entrepreneurs are now able to innovate on top of a technology stack that has grown increasingly powerful in the last few years – enabling product and analytical experiences that are more personalized and more valuable than ever."