AWS unveils Lake Formation for easy data lake building

Lake Formation takes the process down from several months to days, AWS CEO Andy Jassy announced at re:Invent.
Written by Stephanie Condon, Senior Writer and  Asha Barbaschow, Contributor

Amazon Web Services on Wednesday unveiled AWS Lake Formation to set up data lakes, creating a process that can take days rather than several months.

"This is a step level change in how easy it is going to be," AWS CEO Andy Jassy said at the AWS re:Invent conference on Las Vegas. "Everybody wants a data lake," he said, noting that there are more than 10,000 data lakes built on top of S3. "People realize there is value... in consolidating" data.

Also: Top cloud providers 2018: How AWS, Microsoft, Google Cloud Platform, IBM Cloud, Oracle, Alibaba stack up

However, the process is intensive: "First you have to ready your storage, configure the S3 buckets, move that data from all the disparate places, you've got metadata," Jassy said. "Then you have to go through the step of cleaning and preparing the data... to optimize the performance and cost of finding that data and running analytics." After all that, there are security and access policies to set up.

For most companies, it takes several months to set up a data lake.

AWS Lake Formation, by contrast, "lets you do it through a dashboard in a few clicks," Jassy said, cleanly moving data and setting up the right metadata tags. It offers a list of prescriptive security policies and access controls.

Jassy also announced the preview of AWS Control Tower, touted as giving customers an easy way to set up and govern a secure multi-account environment or Landing Zone.

"A lot of builders and a lot of developers are tinkerers," Jassy said. "But a second macro-type of builder has emerged."

Also: Cloud wars 2018: 6 things we learned

Jassy said this second type of builder is less interested in delving into the details of the services, instead they just want to implement and build faster.

"This second group of builders keep asking us for more," he said, noting they're also asking it to be easier and be accompanied by more prescriptive guidance. "It's the easiest way to set up and govern secure, compliant, multi-accounts."

"A much easier way, in just a few clicks, to manage on AWS."

Also: Amazon's Cloud Cam finds the right balance for home security CNET

Where the "second type" of builder is concerned, Jassy also announced the AWS Security Hub, which he said was a way to centrally manage security across an AWS environment.

"CISOs are pivoting between different consoles," Jassy said. "AWS Security Hub is going to give you a GUI that saves you a lot of time."

Using AWS Security Hub, users can also automate compliance checks to detect deviations against industry standards. It also provides a view summary of prioritised issues.

Cloud services: 24 lesser-known web services your business needs to try

Previous and related coverage:

What a hybrid cloud is in the 'multi-cloud era,' and why you may already have one

Now that the services used by an enterprise and provided to its customers may be hosted on servers in the public cloud or on-premises, maybe "hybrid cloud" isn't an architecture any more. While that may the case, that's not stopping some in the digital transformation business from proclaiming it a way of work unto itself.

Cloud computing: Here comes a major tipping point

Application spending has moved to the cloud fastest, but other areas of IT spending are catching up.

Related stories:

Editorial standards