Google launches Cloud Dataflow, says MapReduce tired

Summary:Google launches a service called Cloud Dataflow that aims to analyze pipelines with "arbitrarily large datasets."

Google on Wednesday launched Cloud Dataflow, a big data analytics service to crunch information in either streaming or batch mode.

The announcement, made at Google's I/O keynote in San Francisco, helps round out the search giant's cloud stack, which is adding features as it aims to take on Amazon Web Services.

Urs Hölzle, senior vice president at Google, outlined Dataflow and a demo revolved around crunching Twitter data and sentiment around World Cup games. Dataflow was the headliner in a series of cloud services outlined. 

Hölzle said that Dataflow has replaced MapReduce inside Google. Cloud Dataflow is designed to analyze pipelines with "arbitrarily large datasets."

"Cloudflow does for entire pipelines what MapReduce did for single flows," he said.

Roughly speaking, Google's Cloudflow would line up against Amazon Web Services Redshift, a datawarehouse service, and AWS' Elastic MapReduce, which uses Hadoop to crunch large datasets.

data flow
cloud flow
cloud data flow


Topics: Cloud, Big Data, Data Centers, Google


Larry Dignan is Editor in Chief of ZDNet and SmartPlanet as well as Editorial Director of ZDNet's sister site TechRepublic. He was most recently Executive Editor of News and Blogs at ZDNet. Prior to that he was executive news editor at eWeek and news editor at Baseline. He also served as the East Coast news editor and finance editor at CN... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.