Microsoft readies new Azure Data Factory service

Microsoft is working on another new Azure service focused on helping users deal with big data.

That service, known as Azure Data Factory, is meant to allow users to consolidate a number of Microsoft Azure services, along with select third-party services, into a more easily-managed analytics and data pipeline, according to sources who requested anonymity.

Currently, Microsoft allows users to move data across various accounts in Azure and/or on-premises SQL Server. The soon-to-be-announced Azure Data Factory service is meant to expand the list of data sources and types beyond blobs, tables and databases, sources said.

I'm hearing the new Data Factory service is currently in private preview, but should be released in public preview form within a month or so.

In searching for more information about Azure Data Factory, I found a LinkedIn profile of Microsoft Software Architect Chu Chen, who identifies himself as "co-owner of Azure Data Factory with Big Data (Solution, Hadoop, OneSI, SQL Elementary Pipelines and Power BI.)"

I asked Microsoft officials for more information on Azure Data Factory. A spokesperson said Microsoft had "nothing to share" about the service.

Microsoft already offers a number of Azure data and analytics services, including Azure SQL Database (managed relational database as a service); HDInsight (managed Hadoop clusters); cache, machine learning, DocumentDB (its recently introduced NoSQL document database as a service) and Azure Search (its new full-text search as a service).

Microsoft recently added support for Apache HBase cluster support to Azure HDInsight. (HBase is a NoSQL database component of Hadoop.) Microsoft currently doesn't support Apache Storm, a system for processing streaming data in real time in Hadoop 2.x, but I've heard that support may be coming soon to HDInsight. Amazon's AWS already supports Storm.

Microsoft officials also declined to comment on if and when HDInsight will get Storm support. But in the interim, some are finding ways to use Storm on Azure.