Teradata Aster integrates Hadoop metadata natively

Summary:Data warehousing kingpin Teradata's Aster unit and Hadoop ecosystem all-star Hortonworks have teamed to integrate Hadoop's meta data store into Aster Database's. This will facilitate remote queries, locally persisted data, and relational-Hadoop data mashups.

As news continues to break out of this week's Hadoop Summit, Teradata Aster is announcing a partnership with Hortonworks resulting in a new technology feature called SQL-H.

SQL-H tightly binds the schema metadata of Hadoop and that of the Teradata Aster Database.  The enabling technology on the Hadoop side is the Apache HCatalog meta data store that creates a unified storage abstraction layer around Hadoop data stored in Pig, Hive and raw HDFS formats.  Hortonworks is a major source code contributor to the Apache HCatalog incubator project.

While Hive provides SQL-like querying capability natively, it only does so for its own tables.  HCatalog applies Hive's meta store to work with data in other formats and introduces a consistent schema and data type standard across them all.  SQL-H can then address any data in the HCatalog store and make it available to Aster Database.

With SQL-H, data in the HCatalog store can be queried as if it were local. Queries can run in a "one-off" mode, in which case they will execute remotely and return the appropriate result sets. Queries can also be configured to persist the resulting data locally in the Aster Database.

The raw power of Hadoop has made users forgiving of its sometimes low degree of fit and finish, especially between components like Hive and Pig.  But as Hadoop becomes more mainstream, its own idiosyncrasies, and the inconsistencies across its ecosystem components, need to be straightened out.

Businesses won't for much longer tolerate a double standard between their established database and BI tools on the one hand, and their Big Data technology on the other.  Most Big Data companies know this.  Teradata Aster is taking a sensible approach to this problem.  It's leveraging the open source HCatalog project to eliminate the double standard, and to have multiple technologies coalesce around a single standard .

Topics: Big Data

About

Andrew J. Brust has worked in the software industry for 25 years as a developer, consultant, entrepreneur and CTO, specializing in application development, databases and business intelligence technology. He has been a developer magazine columnist and conference speaker since the mid-90s, and a technology book writer and blogger since 200... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.