Microsoft Ignite 2017: MS Stresses Choice, From SQL Server 2017 to Azure Machine Learning

Video: These are the top business tech stories from Microsoft's Ignite

Microsoft is getting really serious about giving customers choices. That much was clear at this week's combined Microsoft Ignite and Envision events in Orlando and, in particular, in announcements around databases, data-integration, machine learning (ML), and artificial intelligence (AI).

Several announcements at Ignite were entirely about choice. On the hybrid front, for example, there was the general availability of Azure Stack, which lets customers put a slice of the Azure Cloud on premises -- on a choice of hardware-partner racks. But that's about infrastructure. My focus was on what Microsoft described as creating "systems of intelligence." I'll focus here on database, database migration, data integration, ML, and AI.

SQL Server 2017 Meets Linux, Docker

Microsoft announced that SQL Server 2017, the latest release of its flagship database, will be generally available on October 2. The big breakthrough is that this release runs on Linux as well as Windows (and the company is offering new-customer incentives including discounts for subscriptions and bundles with RedHat).

Another new deployment option is within Docker Enterprise Edition containers for portability across clouds and on-premises. Beyond portability, SQL Server 2017 introduces advances in adaptive query processing, the ability to add clustered column stores for faster analytical performance and support for running models entirely in-database by way of R and Python.

Analysis: Linux is the favored operating system of the cloud, and the Windows-only constraints on Microsoft SQL Server where getting in the way of growth. Together with the docker option, these multi-platform and hybrid options should accelerate adoption.

Azure DB Migration Service Courts Oracle, MySQL

Now in private preview, Azure Database Migration Service is designed to help you migrate on-premises Microsoft SQL Server, Oracle and MySQL instances to Azure. Also in limited preview is a coming Azure SQL Database - Managed Instance, a platform-as-a-service option with VNET and private IPs support.

Analysis: There's still only "close to full compatibility" for migration of on-premises Microsoft SQL Server to Azure SQL Database. The differences may be small, but Oracle touts its "same-DB-no-matter-where-you-deploy" advantage.

Azure Data Factory

This data-integration service for Azure, now in public preview, supports the creation, scheduling and orchestration of data-integration pipelines with the option to lift and shift SQL Server Integration Services (SSIS) packages into the cloud. Microsoft says this soon-to-be-GA service will include discounted rates for active SQL Server licensees.

Analysis: I'd like to hear more about nuances of practical differences between Azure Data Factory and SSIS, if any, in capabilities, management, administration and the overall user experience.

Next-Generation Azure Machine Learning

ML and AI are the underpinning of "smart" systems that predict, spot patterns and exceptions, develop inferences about intent, and offer recommendations. Microsoft put ML/AI modeling capabilities in the cloud several years ago with Azure ML/Azure ML Studio. But that first-generation offering was strictly a cloud service run by Microsoft on Azure. The next generation of Azure ML gives organizations options through three components announced at Ignite and now in public preview.

• Azure ML Workbench is a cross-platform client for data wrangling and managing experiments. It runs on Windows and iOS machines and is geared to developers and data scientists who need to take the first step to creating models, which is preparing the data. Users can tap into a broad range of data sources, including high-scale sources, and see samples, stats and distribution information about that data. The tool can learn the clean-up and normalization steps you want to take by example and then repeat them at scale. These steps are recorded for data transparency and lineage. From there you can use the data for your modeling experiments.
• Azure ML Experimentation service is built supporting collaborative model development at scale. It uses Git repositories and a command-line tool to manage model experimentation and training. It tracks the code, configurations and data used in experiments as well as the models, log outputs, key metrics and the history of how those models evolve. This ensures transparency around models over time, which is often a requirement in regulated environments. Providing choice, the Experimentation service supports Python and an array of frameworks, including Tensorflow, Caffe, PyTorch, MXNet and DIGITS as well as Microsoft's own CNTK and Microsoft Cognitive Toolkit. There are also plenty of deployment choices. Docker containers are used for portability to many environments while maintaining model and data governance, auditability and visibility. Experiments can run locally or remotely, on general-purpose VMs, scale up on Data Science VMs, scale out on Spark (in Azure HDInsight), and can even run on GPU-accelerated VMs.
• Azure ML Model Manager service is for deployment and operationalization, supporting hosting, versioning, management and monitoring. Here, too, there are many more choices, including in-database in SQL Server 2017, in VMs, on Spark, in the Azure cloud and anywhere you can run Docker containers.

Analysis: Together all these options give data scientists and developers yet more flexibility around where they do their experimentation, training of models and operational scoring. Significantly, there's more choice on frameworks, with Microsoft executives saying that algorithms shouldn't matter -- use whatever is best for the task at hand.

Docker is the primary means of model portability, but Microsoft says deployment can be as simple as a single line of code while also giving Docker power users options to tune and tweak the deployment.

You can also bring assets directly onto local machines, but you lose trace-ability. The whole idea here is supporting and bringing visibility to the entire, end-to-end lifecycle at scale. That's a must-have for banks, insurance companies and a growing list of organizations that are doing predictive, machine learning and AI modeling at scale.

My Overall Take on Ignite 2017

There were so many more announcements at Ignite that will make a big impact in the near term (like global-scale CosmosDB) and over the long term (like Microsoft's work on quantum computing). The overall theme was choice, with Microsoft offering an impressive, broad spectrum of cloud, on-premises, and hybrid options for data scientists, developers, data-management and governance professionals, and on up to business users and the customers of Microsoft's customers.

Many of this week's announcements are still in preview -- and there are gaps, here and there, yet to be filled. But I came away impressed.

Read more:

• Oracle Differentiates its MySQL Cloud Service
• SAP Machine Learning Plans: A Deeper Dive From Sapphire Now
• SAS Takes Next Steps to Cloud Analytics