What does big data mean for cloud providers? This is the topic under discussion at a EuroCloud UK meeting next week. This blog post is both a reflection on the topic as well as a taster for the event itself, which has an interesting and knowledgeable line-up of speakers — more details on that below [disclosure: I'm chair of EuroCloud UK, so I do have a vested interest in the success of the event].
I've seen a lot of definitions of big data over the past couple of years. I prefer a fairly loose interpretation: when it comes to industry buzzwords, my policy is to 'go with the flow' of popular usage, irrespective of what purists may argue. The underlying trends are what really matter, and the reason this particular buzzword has become commonplace is the confluence of cloud computing with smart devices:
- As well as providing on-demand access to a wealth of private and public data resources, the cloud delivers inexpensive storage capacity and elastically scalable, pooled computing resources that can be harnessed on demand and at very low cost. This has made it viable to perform sophisticated analytics over huge volumes of data that were never even thinkable before.
- The proliferation of intelligent devices is producing an explosion of data, whether it's location-based data from the cellphones people carry around with them, metrics from smart devices installed in homes and workplaces, or the activity streams of social media and other interactions in the cloud.
Put both of those factors together with the spread of automation out of core enterprise application silos into every aspect of business activity, and you have a perfect storm of data being created that is ripe for productive analysis — if only you can work out where the value lies.
That's the catalyst for interest in big data, and therefore the term is legitimately used for anything that answers those trends.
You don't have to be dealing with petabytes of data (though often you will be). You don't have to be running Hadoop or some other NoSQL platform (though it's often part of the picture). You don't have to be handling live streams of real-time data.
What you probably do have is a nagging feeling that you're missing out on valuable insights that are hidden in the data available to you today.
For cloud application providers, the opportunities of big data fall into three categories that are conveniently represented by the three speakers lined up for next week's meeting:
Enabling big data. Workday has added big data capabilities to its HRM and financials applications so that customers can bring in external data sources and analyze them alongside transactional data from their operational systems. Its CTO Europe, Annrai O'Toole, will discuss some of the use cases at customers, as well as some of the lessons learned in building the infrastructure.
Analyzing big data. Probably the biggest area of opportunity for cloud providers is analyzing the aggregated data they look after for their customers to find new sources of value-add. Collaboration vendor Huddle was able to use its archive of user behavior to develop a predictive algorithm that helps it decide which files users are most likely to need downloaded to local storage. CMO Chris Boorman will be talking about his experiences as well as bringing additional insight from his previous role at data integration vendor Informatica.
Productizing big data. A new wave of specialist providers are building products that harness big data sources for business advantage. Tim Barker, chief product officer of DataSift, will discuss how the company has harnessed the entire Twitter archive, along with other sources, to provide social data and analytics to enterprises. He will also speak about DataSift's experience of running one of the largest Hadoop clusters in Europe.
The event will be held in central London next Thursday April 11th from 4pm. It takes a business angle on big data: Economics of Big Data in Your Cloud Platform, and is aimed at product and business leaders of cloud providers. Admission is free to EuroCloud members and for a modest small registration fee to non-members — follow the link above to sign up. Or if you can't be there, join the Talkback discussion below.