Microsoft releases Dryad beta for data crunching
Microsoft has released a beta of Dryad, a toolkit for managing and analysing large quantities of data across clusters of Windows servers.
The Dryad toolkit has two main elements: a computation engine called Dryad, and programming environment called DryadLINQ.
The system should make it possible to run distributed data-crunching programs across a cluster of servers, as long as the servers are running Windows HPC Server 2008 R2 Service Pack 1.
"This initial preview is intended for developers who are exploring data-intensive computing," Microsoft wrote in a blog post that announced that beta. "The DryadLINQ, Dryad and DSC programming interfaces are all in the early phases of development and might change significantly before the final release based on your feedback."
Dryad is designed to run tasks across clusters of between 100 and 10,000 computers. Dryad underpins the data-mining elements of the Bing search engine, according to a separate Microsoft blog post.
DryadLINQ uses the Microsoft-developed Language Integrated Query (LINQ) extension to the .NET programming framework so developers can write Dryad applications.
The Dryad toolkit is directly inspired by Google's MapReduce system, according to a Microsoft academic paper (PDF).
The toolkit also bears similarities to Hadoop, which is an open source implementation of MapReduce and is currently used by a large number of major web companies such as Facebook, Twitter and Yahoo.
The beta opened for registrations on Thursday.