Cloud providers working with big data

Cloud providers working with big data

Summary: The confluence of cloud computing with smart devices has created new opportunities for cloud providers to add value by harnessing and analyzing big data.


What does big data mean for cloud providers? This is the topic under discussion at a EuroCloud UK meeting next week. This blog post is both a reflection on the topic as well as a taster for the event itself, which has an interesting and knowledgeable line-up of speakers — more details on that below [disclosure: I'm chair of EuroCloud UK, so I do have a vested interest in the success of the event].

I've seen a lot of definitions of big data over the past couple of years. I prefer a fairly loose interpretation: when it comes to industry buzzwords, my policy is to 'go with the flow' of popular usage, irrespective of what purists may argue. The underlying trends are what really matter, and the reason this particular buzzword has become commonplace is the confluence of cloud computing with smart devices:

  • As well as providing on-demand access to a wealth of private and public data resources, the cloud delivers inexpensive storage capacity and elastically scalable, pooled computing resources that can be harnessed on demand and at very low cost. This has made it viable to perform sophisticated analytics over huge volumes of data that were never even thinkable before.
  • The proliferation of intelligent devices is producing an explosion of data, whether it's location-based data from the cellphones people carry around with them, metrics from smart devices installed in homes and workplaces, or the activity streams of social media and other interactions in the cloud.

Put both of those factors together with the spread of automation out of core enterprise application silos into every aspect of business activity, and you have a perfect storm of data being created that is ripe for productive analysis — if only you can work out where the value lies.

That's the catalyst for interest in big data, and therefore the term is legitimately used for anything that answers those trends.

You don't have to be dealing with petabytes of data (though often you will be). You don't have to be running Hadoop or some other NoSQL platform (though it's often part of the picture). You don't have to be handling live streams of real-time data.

What you probably do have is a nagging feeling that you're missing out on valuable insights that are hidden in the data available to you today.

For cloud application providers, the opportunities of big data fall into three categories that are conveniently represented by the three speakers lined up for next week's meeting:

Enabling big data. Workday has added big data capabilities to its HRM and financials applications so that customers can bring in external data sources and analyze them alongside transactional data from their operational systems. Its CTO Europe, Annrai O'Toole, will discuss some of the use cases at customers, as well as some of the lessons learned in building the infrastructure.

Analyzing big data. Probably the biggest area of opportunity for cloud providers is analyzing the aggregated data they look after for their customers to find new sources of value-add. Collaboration vendor Huddle was able to use its archive of user behavior to develop a predictive algorithm that helps it decide which files users are most likely to need downloaded to local storage. CMO Chris Boorman will be talking about his experiences as well as bringing additional insight from his previous role at data integration vendor Informatica.

Productizing big data. A new wave of specialist providers are building products that harness big data sources for business advantage. Tim Barker, chief product officer of DataSift, will discuss how the company has harnessed the entire Twitter archive, along with other sources, to provide social data and analytics to enterprises. He will also speak about DataSift's experience of running one of the largest Hadoop clusters in Europe.

The event will be held in central London next Thursday April 11th from 4pm. It takes a business angle on big data: Economics of Big Data in Your Cloud Platform, and is aimed at product and business leaders of cloud providers. Admission is free to EuroCloud members and for a modest small registration fee to non-members — follow the link above to sign up. Or if you can't be there, join the Talkback discussion below.

Topics: Cloud, Big Data, Enterprise Software

Phil Wainewright

About Phil Wainewright

Since 1998, Phil Wainewright has been a thought leader in cloud computing as a blogger, analyst and consultant.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Cloud providers working with big data

    Data Centers (Server Farms = Cloud) are going to have more than disc space to worry about in the time ahead. They are going to become targets of some of the most innovative and egregious technologies ever developed. Security in the Cloud right now is not at an acceptable level of competence.

    Anyone in business who is placing their data in the CLOUD is at risk. At risk from Hackers who hack for fun, Hackers who hack for Financial Information, Hackers who hack for Corporate Secrets, and Hackers whose only purpose is to cause mayhem. And one sad fact is always left out of the equation. That is that all of your data, business or personal, can be or is made available without your knowledge to the FBI, CIA, NSA and other agencies. Numerous recent articles pin point to the fact that they don't need your passwords nor your permission to get into your data stores.

    One of the biggest questions standing about Data Centers (CLOUDS) is whether their business models will allow them to remain viably in business in the time ahead. What happens if say "Carbonite" or one of the like services were to financially fail and close their doors tomorrow ? What happens to all of your data ? If you don't have a local server in place, you won't have a place to basically store your data.

    The absolute best place for your company, corporate or personal data is on your own local hardware. There you have it at your command and control. Not dependent upon a CLOUD SERVICE CENTER to get your data back. CLOUD computing does not save you money or time. Everything that gets put in the CLOUD takes a long time to get there and it trickles back slowly or at a huge cost to you via removable hard drive media shipped to you. For the price of one of those removable drives charged to you, you can own 2 of them yourself and keep your own data safe.

    We gotten away from SMART COMPUTING and started down the pathway of DUMB SERVICES and catchy phrases and names. It's time to go back to getting REAL and giving ownership of your data back to YOU ... not some corporate service hack scaring you with made up terms and phrases trying to convince you to put your data in the CLOUD that he can't protect.
  • Big Data Solution

    Phil, great article! Cloud computing is driving a new wave of innovation in the area of big data. The open source solution from HPCC Systems provides a single platform that is easy to install, manage and code. Designed by data scientists, HPCC Systems is a data intensive supercomputer that has evolved for more than a decade, with enterprise customers who need to process large volumes of data in a 24/7 environment. Its Thor Data Refinery Cluster, which is responsible for ingesting vast amounts of data, transforming, linking and indexing that data, and its Roxie Data Delivery Cluster are now offered on the Amazon Web Services (AWS) platform. Taking advantage of HPCC Systems in the cloud provides a powerful combination designed to make Big Data Analytics computing easier for developers and can be launched and configured with the click of a button through their Instant Cloud solution. More at
  • cloud and big data

    Big Data and cloud computing are meant for each other. Cloud computing with amazing access to high computing power is well suited for big data analytics. Big data analytics are going to be highly crucial for market trends prediction and business innovation, organisations cannot ignore it, hence the coming years would witness great developments in big data analytics and definitely it would be possible via cloud solutions.