Microsoft turbo-charges Azure SQL Data Warehouse
At Microsoft's Inspire partner conference event in Washington, DC today, the Redmond, WA-based software giant announced some serious enhancements to its Azure SQL Data Warehouse. Just one year after the cloud data warehouse (DW) service was released to general availability, this upcoming preview release will enhance both the compute and storage facets of the product.
Also read: Microsoft BUILDs its cloud Big Data story
Also read: Microsoft to partners: we're data-driven
What's new
On the compute side, new, larger virtual machine types will allow each node to be powered at up to 18,000 DWUs (data warehouse units -- Microsoft's measurement of DW compute capacity), a tripling of the 6,000 DWUs that Azure SQL DW nodes max out at today. A 9,000 DWU node type will be available as well.
On the storage side, the enhancement is even more fundamental: the new SQL DW nodes will feature large solid state drives (SSDs) and SQL DW will cache as much data as possible on them. Azure SQL DW had been reliant on remote cloud storage. While this separation of processing and storage allowed customers to scale those two facets of the service separately, and to pause and resume their clusters (since cloud storage is persistent even when virtual machines are shutdown and deallocated), it also made for slower disk access times.
Also read: Cloud data warehouse race heats up
Also read: Microsoft's data announcements, and what they mean
Have blobs, eat SSDs too
Contrast that with services like Amazon Redshift, which use local SSDs exclusively, and Microsoft had some explaining to do; to certain customers, anyhow. But now SQL DW will offer a hybrid architecture, where cloud storage is still the main persistence layer, which maintains the independent compute/storage scaling and the pause/resume capability, and local SSDs will be used as a second, cached layer, for optimized performance.
Microsoft is calling the SSD-based feature "bottomless columnar storage," a hat tip to the columnstore index support SQL DW (and its SQL Server and Azure SQL Database cousins) offer and the way local, SSD storage make that feature perform best.
Relational DW, redux
When I spoke with Microsoft's General Manager for Database Systems, Rohan Kumar, he told me SQL DW was a very fast-growing service for the company. With only that qualitative assessment, it's hard to know just how well it's doing, but I know from my own work in the channel that SQL DW has gotten real traction. Compare that with its on-premises cousin, Analytics Platform System, which has never really taken off, and you can understand why Microsoft is making investment in SQL DW. Microsoft has something good here, and wants to make it better still.
In general, the Big Data era, despite its poster children being Hadoop and Spark, has seemingly produced a bumper crop of data warehouse business, much of it in the cloud. That's another reason for Microsoft to be proactive in its SQL DW investment. Because despite its launch of the geo-distributed Azure Cosmos DB service in May, and the depth and rigor of its HDInsight Hadoop/Spark offerings, relational data warehousing is enjoying a renaissance now, and Microsoft's smart to cash in.
Disclosure: Microsoft covered the cost of my hotel room in Washington, D.C. during its Inspire conference, to facilitate my attendance there. This post was written before the conference kicked off and before I knew of this subsidy.