Microsoft makes available its Azure-based Hadoop service

Microsoft's HDInsight service -- a cloud-based distribution of Hadoop -- is generally available. Microsoft's planned complement for Windows Server has been replaced.
Written by Mary Jo Foley, Senior Contributing Editor

Microsoft's cloud-based distribution of Hadoop -- which it has been developing for the past year-plus with Hortonworks -- is generally available as of October 28.


Microsoft officials also are acknowledging publicly that Microsoft has dropped plans to deliver a Microsoft-Hortonworks developed implementation of Windows Server, which was known as HDInsight Server for Windows. Instead, Microsoft will be advising customers who want Hadoop on Windows Server to go with Hortonworks Data Platform (HDP) for Windows.

Windows Azure HDInsight is "100 percent Apache Hadoop" and builds on top of HDP. HDInsight includes full compatibility with Apache Hadoop, as well as integration with Microsoft's own business-intelligence tools, such as Excel, SQL Server and PowerBI.

"Our vision is how do we bring big data to a billion people," said Eron Kelly, Microsoft's SQL Server General Manager. "We want to make the data and insights accessible to everyone." 

Making the Hadoop big-data framework available in the cloud, so that users can spin up and spin down Hadoop clusters when needed is one way Microsoft intends to meet this goal, Kelly said.

Microsoft and Hortonworks originally announced plans to bring the Hadoop big-data framework to Windows Server and Windows Azure in the fall of 2011. Microsoft made a first public preview of its Hadoop on Windows Server product (known officially as HDInsight Server for Windows) available in October 2012.

Microsoft made available its first public preview of its Hadoop on Windows Azure service, known as HDInsight Service, on March 18. Before that, it had made a number of private previews available to select testers.

Hortonworks announced in February this year that its HDP platform was an extension of its two-year-old Hadoop partnership with Microsoft. (HDP didn't exist yet when Microsoft and Hortonworks initially announced their partnership.)

HDP allows users to deploy Hadoop on Windows Server in their own datacenters -- the same way they can already deploy HDP on several Linux distributions. Microsoft and Hortonworks are touting HDP for Windows as offering an easy migration path to HDInsight. 

Hortonworks announced general availability of HDP 2.0 last week. Hortonworks will update HDP for Windows Server to take advantage of HDP 2.0 within a month, Kelly said. The pair haven't committed to a timetable as to when they'll update HDInsight to take advantage of HDP 2.0. (It's currently on version 1.3.)

As part of its Hortonworks partnership, Microsoft has contributed back to Apache 25,000 lines of code for inclusion in the Apache Hadoop code base, Kelly said. 

Editorial standards