Hadoop 2.0 went GA back in October, 2013, but only for the Linux operating system.
That release of Hadoop, along with its "YARN" component, allows the Big Data technology to be used on petabyte-scale datasets without having to use the batch-oriented and laborious MapReduce algorithm. Moving beyond MapReduce is a big breakthrough, so the question of when the Hadoop 2.0 would be available for Windows has been lingering. Specifically, a release date for a Hadoop 2.0-based release of the Hortonworks Data Platform (HDP) for Windows distribution is what we've needed, since Hortonworks is the only vendor with a full Windows-based Hadoop distro.
Three months after the Hadoop 2.0-based release of HDP for Linux, the Windows release is now ready as well. In a post to the Hortonworks blog today, Hortonworks' VP of Strategic Marketing, John Kreisa, announced the immediate availability-for-download of HDP 2.0 for Windows.
In the post, Kreisa explains that HDP for Windows is the only Hadoop 2.0-based distro certified to run on Windows Server 2008 R2, 2012 and 2012 R2. Kreisa also highlights other hallmarks of Hadoop 2.0: high availability of the Hadoop Distributed File System (HDFS) NameNode;" Phase II of the Stinger initiative, which aims to increase performance of Apache Hive by a factor of 100; and the inclusion of release 0.96 of the Apache HBase NoSQL database, which includes new features like Snapshots and improved mean time to recovery (MTTR).
The post also points out that there is single-node version of HDP 2.0 for Windows available that includes an MSI (Microsoft Installer)-based setup program. This makes it very easy for developers to start working with HDP 2.0 for Windows, either for evaluation or debug and testing of code written against HDP.
For a more built-out single-node installation, Hortonworks makes available its HDP "Sandbox," a virtual machine image that includes HDP itself, along with a number of tutorials and other learning resources. Although only available as a Linux guest image, there is a Hyper-V version of the image available, making it accessible to Windows shops.
What about HDInsight?
Microsoft works closely with Hortonworks in the latter's development of HDP for Windows. In fact, the Redmond software giant's Windows Azure cloud-based implementation of Hadoop, HDInsight, is built on the HDP codebase.
When will HDInsight be updated to Hadoop 2.0? Neither Microsoft nor Hortonworks has yet announced a release date for this, but with Hortonworks putting HDP for Windows 2.0 in General Availability, HDInsight will likely catch up before too long.