AWS re:Invent 2017: Amazon Aurora PostgreSQL headlines pre-show announcements

With Amazon Web Service's annual re:Invent spread across several miles of the Las Vegas strip this week, there's going to be a tidal wave of announcements. To get heard above the noise, Amazon has already issued a number of announcements in the run up to re:Invent for its data platforms, starting with the long-awaited GA release of Amazon Aurora PostgreSQL.

As we've noted, this is Amazon's shot across the bow to Oracle. Aurora is the RDS service where Amazon reengineers the database into cloud-native form. Essentially, it integrates the database engine with an SSL-backed virtualized storage layer designed to deliver high performance and 99.99 percent availability.

Also: AWS re:Invent Big Data spotlight shines on global footprint and storage access

But going cloud-native impacts more than just the storage engine. In Aurora, Amazon has changed the way it performs database writes, ACID support, fault tolerance, and disaster recovery. With PostgreSQL, Amazon has taken this a step further compared to its MySQL Aurora implementation by incorporating nondestructive updates (that is, updates don't overwrite old data). Doing away with overwrites actually simplifies database operations and improves performance, especially for larger databases.

With Amazon Aurora PostgreSQL already GA, we're waiting for the other shoe to drop: when AWS will get on the latest open source PostgreSQL 10 release, which introduces a major advancement: declarative table partitioning. A major step toward gaining functional parity with Oracle, this feature will simplify partitioning. That should prove especially useful to the large database deployments for which Aurora is targeted. Amazon's goal in the long run is to get current on the latest open source community stable releases within 30 days, something it has already achieved with EMR and Apache Hadoop. With Amazon Aurora PostgreSQL now generally available, we'll be watching for Amazon to meet its open source release targets in 2018.

Also: New AWS service will bring image recognition to real-time video

Other announcements leading up to re:Invent are filling gaps or adding feature extensions. Amazon has added a useful scaling option for DynamoDB that allows you to schedule changes in provisioning capacity. By comparison, Google Cloud Datastore handles scaling differently (it completely automates it, eliminating the need for scheduling), while Azure Cosmos DB has yet to automate the task can autoscale storage, although it is not terribly clear in their documentation.

The latest enhancements to AWS Database Migration Service (DMS) address one critical gap: providing the means to manage migration offline using Amazon's Snowball Edge appliance. This is a logical addition to the fleet of winter sport-themed appliances and vehicles that Amazon has (literally) rolled out; if you are moving data, DMS integration on Snowball Edge allows you to make the operation more self-contained by adding the ability, not simply to load data, but also to transform it for the target Amazon data platform. Other enhancements add pre- and post-migration data validations, so you can manage all the tasks necessary for moving and converting data, and validating that the data on the target is faithful to the source.

Also: Intuit to use AWS as its standard artificial intelligence platform

Rounding out the announcements, Amazon Aurora has added autoscaling of replicas to better control parameters like CPU utilization or average active connections. This way, the configurations for your replicas don't necessarily have to march in lockstep with the primary nodes. That's significant because primary nodes are often dedicated to writes while replicas might be configured for handling reads or serving the purpose of what we used to call hot standbys. As such, replicas may not always require the same resource footprints as primary nodes.

And finally, yes, Amazon's RDS Service adds SQL Server 2017 support, for now limited to Windows. Linux will have to wait another day.