Big data is one of the many buzzwords to have swarmed the tech world in recent memory, but those on the front lines say the concept is far from fledgling.
The idea of collecting data to gain insights spans industries and has been happening in various — sometimes primitive — ways for several decades. However, it's only in the last few years that the concept proved its staying power, with advancements in technology making it possible to produce unfathomable amounts of data about how people behave.
Two years ago it was estimated the world economy produced 2.5 exabytes of data per day, and research firm IDC forecasts that we will generate 40 zettabytes of data by 2020. That surfeit of data unlocked a trove of potential for the entities standing to benefit from it — including government bodies, education systems, hospitals and retailers — as well as for enterprise business intelligence providers ready to pounce on the prospect of boosting their product portfolios.
Big data: an old story
Through the implementation of enterprise applications, old-guard vendors such as IBM, Oracle and SAP have long been capturing enterprise data, and in recent years have joined the race to make that data fit for insightful analysis.
"Big data has been around a long time," said Byron Banks, VP of big data for SAP. "Now with social media, the Internet of Things, sensor data, mobile devices — new big data has transformed the business."
As traditional enterprise data took its modern shape, vendors started using BI tools like in-memory computing, batch-based architectures and the integration of products like Hadoop.
In addition to traditional enterprise data, companies are accruing unstructured information from places far beyond the confines of corporate walls, creating a stream of complexities for app vendors trying to keep their seats on the big data bandwagon.
Banks described how SAP has taken its HANA platform and extended it so it can talk to Hadoop, and how the company changed its business intelligence portal by adding Lumira so end users can see the data in a visual pattern.
"At SAP we are evolving our landscape, since we don't think it's a rip-and-replace strategy," Banks said. "All of the systems that have been built over the years are being improved upon rather than totally taken away. The whole idea is to extend existing apps into the big data realm rather than throw away apps that have been built up for 20 years."
Legacy vendors versus agile upstarts
Still, some skeptics question legacy vendors and their ability to offer big data solutions without simply rebranding or pursuing acquisitions, as companies are incrementally partnering more with third-party providers.
"A lot of the legacy companies are buying into big data," said David Steinberg, CEO of Zeta Interactive. "Oracle with Responsys, Salesforce.com with ExactTarget, IBM with Silverpop — a lot of these guys are buying assets that are very focused on big data specifically, which is smart because they can triple them quickly by rolling them into their customer base."
Naysayers also accuse legacy vendors of having a price-fueled identity crisis when faced with innovative competitors driving down market costs. Companies like Splunk and Cloudera can appear far more agile than their legacy predecessors by offering products at a cheaper rate than traditional enterprise applications.
"Customers expect big data, it's assumed in today's world when you are working with an enterprise."
— David Steinberg, Zeta Interactive
As for SAP, Banks recognized the challenges of the acquisition approach to big data, but said it's the forward-thinking vendors who can keep up with change that will end up faring the best.
"Big data is a useful technology, just like relational databases were useful technology, and there are going to be more that we can't predict," Banks said. "There will be new providers coming all the time and we will work with them. There's probably some guy or gal out there right now who going to invent something tremendous, and we will adopt that too if and when it makes sense for our business."
For the sake of SAP and other vendors' profit margins, adapting to new technologies will be requisite if they intend to maintain and secure customers. Big data technology moves at a frenetic pace, but companies using it aren't concerned with how long something takes to get to market — they want the best product, and fast.
"Customers expect big data, it's assumed in today's world when you are working with an enterprise," noted Zeta Interactive's Steinberg. "Companies on the customer side are not talking to vendors that can't bring them better data and better analytics than what they were doing before."
It's that mainstreaming of big data that's sure to cull the pool of vendors down to those who manage to execute it best.