Microsoft delivers to testers beta of coming supercomputing service pack

On May 2, Microsoft made available to testers a beta build of the second service pack (SP2) for its coming Windows Server High Performance Computing (HPC) 2008 R2 platform.

What makes this interesting is HPC 2008 R2 SP2 is the slated delivery vehicle for Dryad, Microsoft's closest competitor to Google MapReduce and Apache Hadoop. In the early phase of its existence, Dryad was a Microsoft Research project dedicated to developing ways to write parallel and distributed programs that can scale from small clusters to large datacenters.

Earlier this year, Bill Hilf, General Manager of the Technical Computing Group at Microsoft, told me to expect Dryad to be part of HPC 2008 R2 SP2 and to look for the final version of it toward the end of calendar 2011.

The beta of SP2 is available on the Connect download site. The site mentions something called "HPC Pack 2008 R2," which seemingly is a cluster of HPC servers. Here's a description of the coming SP2/HPC Pack deliverables from that site:

"The Microsoft HPC Pack 2008 R2 software, and the Windows HPC Server 2008 R2 Suite, enables cluster based supercomputing based on x64 versions of Windows Server 2008 and Windows Server 2008 R2 as well as Windows 7 clients. Windows HPC Server can be used for massively parallel programs (computational fluid dynamics, reservoir simulation) as well as embarrassingly parallel programs (BLAST, Monte Carlo simulations).

"An HPC Pack 2008 R2-based system is a cluster of servers that includes a single head node, and one or more compute nodes. The head node controls and mediates all access to the cluster resources and is the single point of management, deployment and job scheduling for the compute cluster. It utilizes the existing corporate Active Directory infrastructure for security and account management."

Here is the list of features in SP2:

Ability to add Azure VM roles to the cluster Ability to run MPI-based jobs on Azure nodes Ability to write and run data-parallel applications using Dryad Support for 'workstation nodes' to be in a different domain from the dedicated cluster hardware Support for Azure Connect (codenamed "Sydney") on Azure Nodes

The full and detailed feature list for HPC 2008 R2 SP2 is in this TechNet article.

The supercomputing/high-performance-computing space remains one where Microsoft is duking it out with Linux.

Update: As we learned via Microsoft's reorg memo from May 2, the HPC team is now part of the Windows Azure organization, which makes sense given Microsoft's longer-term plan of turning the cloud into a supercomputer.