10 Reasons 2014 will be the Year of Small Data

Big data has dominated the hype and much of the application production cycle for the last 12-15 months. Now we come to Allen Bonde, noted thought leader, calling for small data!! Want to know what he means? Read on....
Written by Paul Greenberg, Contributor

Hey everyone, December is underway and I'm in the midst of reading all the CRM Watchlist entries which, I have to say, is a lot of work. Looks like its more than 2500 pages of reading, but who's counting? Me. That's who.

In any case, I'm going to pepper the month with guest posts from thought leaders that I think have something significant to say and I'm starting with Allen Bonde and I'll tell you why.

Allen (tweets at @abonde) is the partner and principal analyst for the very aptly named Digital Clarity Group. He brings clarity to difficult topics - those really important technology industry trends and initiatives that have direct and significant impact on businesses, even when the businesses are unable to wrap their arms around it. 

Allen has been a friend of mine for several years and my admiration for him pre-dated my relationship to him. He has been both a CMO (three times) and an analyst for Yankees Group, McKinsey and Extraprise, giving him all kinds of experience and authority that are meaningful. The man is smart and accomplished and as an aside, a helluva video producer too.  You want to know him and if you can't know him today, you want to read him - and then get to know him. 

He has a lot of other business experience too that I'm not going into. For a fuller bio you can go here. I really am anxious to get to what he wants to tell you. 

BTW, If you like what you read, you have the chance to read a lot more about this on his Small Data Group blog which is well worth reading. Seriously. Right after you're done.

Okay, enough said by me. Take it away Allen! 


The idea of big data is compelling. Want to uncover hidden patterns about customer behavior, predict the next election, or see where to focus ad spend? There’s an app for that. And to listen to the pundits, we should all be telling our kids to become data scientists, since every company will need to hire an army of them to survive the next wave of digital disruption.

Yet all the steam coming out of the big data hype machine seems to be obscuring our view of the big picture: in many cases big data is overkill. And most cases big data is useful only if we (those of us who aren’t data scientists) can do something with it in our everyday jobs. 

Which is where small data enters the picture. What’s small data? On one hand it’s a design philosophy I’ve been exploring for the past year or so, inspired by consumer apps and services that deliver useful data, content and insights to users on the go. On the other hand it’s the technology, processes, and use cases for turning big data into alerts, apps, and dashboards (the “last mile”) for business users within corporate environments. Finally, there’s the literal definition of small data referring to the size of our data sets as well.

In a recent study I conducted at Digital Clarity Group (Sponsored by my friends at Adobe, Actuate, HubSpot and Visible and available for download here) we created the following definition that ties together a number of the themes above:

Small data connects people with timely, meaningful insights (derived from big data and/or “local” sources), organized and packaged – often visually – to be accessible, understandable, and actionable for everyday tasks. 

Note, as we describe in our report, this definition applies to the data we have, as well as the end-user apps and analyst workbenches for turning big data sets into actionable small data. The key “action” words here are connect, organize, and package, and the “value” is rooted in making insights available to all (accessible), easy to apply (understandable), and focused on the task at hand (actionable).

As a philosophy and a set of tools, small data is gaining momentum and increased attention as I discussed here during the summer. In fact from the conversations I’ve had over the past year and what I learned from our research, small data looks to emerge as a mainstream movement in 2014.

Here are 10 reasons why:

1. Big data is hard (and the domain of the few). Doing it at scale and waiting for trickle down benefits can take time. Not to mention the fact that most marketers and online strategists don’t need full-on big data to target their campaigns or deliver personalized experiences. Even more so, you can argue that the tech guys want big data because it continues to perpetuate a centralized model, with big servers, big processes and big budgets – owned by a small number of gatekeepers. Small data thinking can help to break this cycle and do much to democratize apps and data.

2. Small data is all around us (and the domain of the many). Think about the social web and the vast amount of social and mobile signals that can help us understand changing customer needs and wants. Social channels are rich with small data that is ready to be collected to inform marketing and buyer decisions. At a personal level, we are constantly creating this small data each time we check in, search, browse, post etc., creating a unique signature that provides a glimpse into our digital and physical health – a topic being researched by Professor Deborah Estrin from Cornell.

3. Small data is at the center of the new CRM. As I explored in my post earlier this year on the state of Social CRM, to create a complete picture of customers, their segments, influencers and even competitors, we need to combine insights from social channels and campaigns with Web analytics and transactional data. Small data is the key to building these rich profiles that will be the center of the new CRM (and CX/CEM) solutions, and generating targeted offers and tailored experiences that delight our customers.

4. ROI is the thing. A focus on the last mile of big data offers to leverage investments to-date ($10 billion and counting according to IDC) on upstream systems, tools, and services. Furthermore, to realize the full value of data-driven apps (whether they use big or small data), they must be actionable and accessible for everyday work. Plus, we need to do a better job collecting the insights/content we already have, discovering their meaning in the context of the task at hand, and delivering the right data in the right format to the broadest set of users. This is the essence of the small data philosophy.

5. Data-driven marketing is the next wave. As we note in my DCG study, big (and small) data-driven marketing has the potential to revolutionize the way businesses interact with customers, transform how customers access and consume (and even wear) useful data, and ultimately redefine the relationship between buyers and sellers. Which is why we believe there’s an unprecedented opportunity for marketers to lead the charge for the next wave of data-driven apps and tools, all-in-one platforms providers such as Adobe (Marketing Cloud), HubSpot and Oracle (Eloqua), and data specialist up-starts like Anametrix, Attivio, Localytics and TrueLens.

6. Consumer examples abound. From Amazon recommendations, to Nike Fuelband, to Kayak’s price-driven “When to Book” tool - which uses trending data and some smart rules to help customers decide between booking their travel now or waiting for prices to fall, consumers have seen the potential of small data to streamline their shopping, power their fitness routine, or deliver recommendations about the best price for their next flight. With more smart, wearable data-driven devices on the way, and Google Glass ready for general release, there promises to be even more market demand for packaged data and data-delivery devices that “fit” the needs of everyday consumers.

7. Platform vendors are starting to pay attention. The promise of operationalizing big data and “turning insight into action” was a major theme at the last couple of SAP events I attended (see my write up on TechEd here) and it’s not a stretch to see HANA as a “small data” layer for SAP’s customers and partners. I’m hearing similar sentiments about the power of small data and actionable business intelligence from some of my contacts at Oracle as they look to leverage investments in Collective Intelliect, Endeca, and Vitrue. Even storage and big data behemoth EMC participated in my small data panel as the recent Digital Pulse event in Boston.

8. Tool vendors are getting on board. I’ve tracked a number of vendors in the enterprise space who have started to align with our small data theme and playbook (make it simple, make it smart, be responsive, be social), including Actuate (especially with its new positioning), GoodData, QlikTech, Tableau, and Visible. And recently we’ve seen TIBCO CTO Matt Quinn jump on the small data bandwagon in his latest article in AllThingsD, and a panel featuring executives from EasilyDo, Yelp, Tagged, and Microsoft talk up the power of small data for better recommendations and personal productivity apps.

9. It’s about the end-user. As I discussed in this post, the last mile of big data is where the value is created, opinions are formed, insights are shared, and action are made. By non data scientists. Everyday. For big data fans, “value” needs to be the last V, joining Volume, Velocity and Variety. Small data is about the end-user, what they need, and how they can take action (see my definition above). Focus on the user first, and a lot of our technology decisions become clearer. This has especially been the case for customer-facing systems and applications.

10. Simple sells. Just because you say less, doesn’t mean you have less to say. Small data is the right data. Sure, some small data will start life as big data, but you shouldn't need to be a data scientist to understand or apply it for everyday tasks. And if you are designing or using data-driven apps for sales and marketing, “simple” means making it easy to access all your tools and data from one place, create/reuse campaigns, build rich profiles, and share reports, templates and insights with team members. 

+ + + + + + +

Thank you Allen.  So, folks, what do you think? Intriguing? The way it is? Not exactly? I lean to the way it is - or at least should be looked at. Let us know. Dare ya. 






Editorial standards