Teradata Aster integrates Hadoop metadata natively

Teradata Aster integrates Hadoop metadata natively

Summary: Data warehousing kingpin Teradata's Aster unit and Hadoop ecosystem all-star Hortonworks have teamed to integrate Hadoop's meta data store into Aster Database's. This will facilitate remote queries, locally persisted data, and relational-Hadoop data mashups.

TOPICS: Big Data

As news continues to break out of this week's Hadoop Summit, Teradata Aster is announcing a partnership with Hortonworks resulting in a new technology feature called SQL-H.

SQL-H tightly binds the schema metadata of Hadoop and that of the Teradata Aster Database.  The enabling technology on the Hadoop side is the Apache HCatalog meta data store that creates a unified storage abstraction layer around Hadoop data stored in Pig, Hive and raw HDFS formats.  Hortonworks is a major source code contributor to the Apache HCatalog incubator project.

While Hive provides SQL-like querying capability natively, it only does so for its own tables.  HCatalog applies Hive's meta store to work with data in other formats and introduces a consistent schema and data type standard across them all.  SQL-H can then address any data in the HCatalog store and make it available to Aster Database.

With SQL-H, data in the HCatalog store can be queried as if it were local. Queries can run in a "one-off" mode, in which case they will execute remotely and return the appropriate result sets. Queries can also be configured to persist the resulting data locally in the Aster Database.

The raw power of Hadoop has made users forgiving of its sometimes low degree of fit and finish, especially between components like Hive and Pig.  But as Hadoop becomes more mainstream, its own idiosyncrasies, and the inconsistencies across its ecosystem components, need to be straightened out.

Businesses won't for much longer tolerate a double standard between their established database and BI tools on the one hand, and their Big Data technology on the other.  Most Big Data companies know this.  Teradata Aster is taking a sensible approach to this problem.  It's leveraging the open source HCatalog project to eliminate the double standard, and to have multiple technologies coalesce around a single standard .

Topic: Big Data

Andrew Brust

About Andrew Brust

Andrew J. Brust has worked in the software industry for 25 years as a developer, consultant, entrepreneur and CTO, specializing in application development, databases and business intelligence technology.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


1 comment
Log in or register to join the discussion
  • OK ... really?

    It is entirely appropriate that Hadoop is invading the commercial BI space, but the Big Data movement is a lot bigger thank BI. Right now the BI/DW folks are incorporating things like MapReduce because they have to, their tools no longer comfortably handle the volume of data out there. Big Data is opening up whole new business concepts, things like real time behavioral profiling. This is where the action is.

    In the new world raw data is never thrown away.

    It is often collected without thought of what it might be useful for. The beauty of NOSQL is being able to decide later.

    Companies like LinkedIn turn data directly into product (People You May Know) skipping the whole BI process. This, unlike traditional BI, leads to monumental rather than iterative gains. Data is changing business, and in near real time.

    BI will continue on it's merry way, as Big Data has bigger fish to try.