MS-Google face-off: This time in distributed computing
At this week’s open house for press and analysts, Microsoft Research (MSR) will be showing off a multitude of projects, including a (so far) little-publicized distributed-computing platform under development that is code-named “Dryad.”
Dryad is one of a number of large-scale-computing efforts in which Microsoft researchers are engaged. From early accounts, it sounds as if Dryad is Microsoft’s competitor to Google Lab’s MapReduce technology.
According to Google’s researchers, "MapReduce is a programming model and an associated implementation for processing and generating large data sets”
In a New York Times piece on Google’s work to build out its hardware infrastructure, published in early July, there was a rather cryptic mention of Dryad. When asked for a response to Google’s plans, Microsoft Chairman Bill Gates told The Times that Microsoft was building its own parallel-processing software designed to support increasingly sprawling server farms.
“They (Google) did MapReduce; we have this thing called Dryad that's better,” Gates said.
Microsoft research labs all over the world are engaged in projects in the distributed- computing space, said MSR Senior Vice President Rick Rashid (during a phone interview yesterday).
In the Bay Area, MSR is supporting a number of large-scale computing teams. Researchers in the Cambridge UK lab are working on large-scale networking projects. (In fact, Rashid said, some of the P2P technology that will be integrated in Windows Vista came out of that particular effort.) In Beijing, MSR researchers are engaged in work around fault tolerance, and in Bangalore, large-scale-systems analysis is a focal point, he said.
The Dryad team, based in Mountain View, Calif., is developing software that is designed to provide operating-system-level abstractions for large clusters (thousands) of PCs in a data center.
"Converting a sequential and/or single-machine program into a form in which it can be executed in a concurrent, potentially distributed environment is known to be hard…. The Dryad project is an attempt to generalize this approach to provide a programming model which scales from future single-machine many-core PCs up to large-scale data-centers," explains Microsoft on the Dryad Web site.
The Dryad team is focusing initially on a few key areas: Composability (decomposing a program skeleton into a set of simple operating classes); fault tolerance; and applicability (discovering paradigms best suited for distributed programming, especially in computer vision, speech and machine learning). T
he Dryad researchers are working hand-in-hand with the MSN/Windows Live product groups, Rashid said, as a result of those groups’ need for ever-increasing scalability and bandwidth.
There’s not a lot more public info on Dryad, at this point. “We are still in the early stages of evaluating our design and its implementation,” note the Dryad researchers on their Web site.