How to use big data to transform your company

What's the best way to put big data to use to make marketing seem more like a service than an intrusion? IBM's Anjul Bhambhri explains how.
Written by Anjul Bhambhri, Contributor

Anjul Bhambhri is IBM's Vice President of Big Data Products.

A tweet, a text, an image pinned to a digital board. Each of these acts on its own is inconsequential. But combine them together with everything else everyone else is doing around the world, and they’re unleashing a data avalanche.

To put it in numbers, each day we create 2.5 quintillion (or the number 2.5 followed by 17 zeroes) bytes of data. Which essentially means that 90 percent of the data in the world today has been created in just the last two years alone, thanks in large part to consumer-created content.

We’re being propelled into the era of "big data." Organizations are getting the kind of insight into their businesses, customers, and employees at levels that were unimaginable in the past. But they’re also running head first into tough new challenges.

How should organizations collect these different kinds of data when they’re being churned out around the clock from every source imaginable: Twitter, sensors, texts, digital pin boards and videos. How should they make sense of this data in the smartest, fastest, most effective way? What is the most effective way to share it inside and outside the organization in ways that spur innovation, productivity and loyalty? How can companies use data to make marketing feel more like a welcomed service to consumers instead of an intrusion?

This new era is redefining how we think about and apply computing to solve the world’s biggest challenges. The role of big data and advanced analytics isn’t at the center of this evolution. It is being woven into every computing system, process, connection, and data source.

The result is a fundamental change in how organizations are structured, how operations are managed, and where investments are made. Other leaders within the organization, rather than just the CIO, are becoming major tech purchasers. Not that they care about the underlying technology. Instead they’re interested in what it can help them accomplish, how it can make their digital marketing campaigns more efficient, get a product to market faster, or pinpoint fraud.

As more managers embrace big data, though, it’s becoming clear that there’s a gap between the insights that organizations can gain and their ability to wring those insights our of traditional computing approaches, most of which rely on databases, warehouses, and the aggregation of structured files and documents. Because the traditional computing models simply cannot keep up with the explosion of all this new unstructured data.

At the same time, this new kind of rapid-fire, ever-changing data creates new challenges. To pinpoint meaningful insights in a short period of time, organizations need to consider both data at rest and data in motion. What do we know from the past versus what are we seeing in real-time? By learning to cross reference all this incoming, streaming data with information that we can refer to from the past, we can unlock even greater insights.

One of the biggest impacts of the rise of big data is a powerful new consumer dynamic, one that levels the playing field between buyers and sellers.

Organizations that can create a system that embraces high-volume, fast-moving data from all kinds of sources -- social media, emails, live chats, data warehouses, videos, sensors -- can put consumer insight and feedback into every step in the process, from procurement and product design to marketing, sales and distribution.

This is a critical step in understanding and anticipating consumer behavior. The more data we have to analyze, the more patterns we can identify. The more insight we can gain from those patterns, the better the business results.

Yet, embracing big data means nothing if you can’t protect  and secure it. As the world becomes more digitized and interconnected, new threats and vulnerabilities emerge. The line between personal and professional hours, devices and data disappears. The speed of attacks, and the number of attackers, increases.

Border protection and log files are no longer enough to fully secure a computing environment, particularly those that are relatively complex and distributed. Analytics must be embedded into the infrastructure, constantly seeking out anomalies and patterns that signal security breaches. By applying the same smarter analytics approach to security, companies can create a smart infrastructure that can predict problems before they occur.

On a smarter planet, analytics becomes the central nervous system through which information is received, analyzed and acted on.

New possibilities are unlocked, and through those deeper insights come the opportunity to not just increase revenue and decrease costs, but to solve some of society’s biggest challenges and improve our overall quality of life.

Imagine an energy company that can predict power consumption using 350 billion meter readings from 20 million households. Or an insurance company that settles legitimate claims 70 times faster. Or a city that can reduce emergency response times in half for citizens, while at the same time, raising advance warning times.

Vestas Wind Systems, a Danish energy company, uses big data and analytics to pick the best places to locate wind turbines around the world.

This isn’t the distant future. This is happening today in cities, organizations and businesses across the world. This is the power of smarter analytics. This is what big data is making possible. This is what any organization can claim as its potential. Now.

Photo: Finding the best places to locate wind turbines requires analytics of big data, such as this one in Crete, Greece, built by Vestas Wind System of Denmark.

This post was originally published on Smartplanet.com

Editorial standards