Hortonworks says the deal struck this week to acquire XA Secure will help provide a comprehensive approach to Hadoop security for the first time.
The Hadoop software and services firm has acquired XA Secure, founded in January 2013, for an undisclosed sum and will open-source the Fremont CA-based company's Hadoop security layer, which offers role-based authorisation, auditing and governance.
Hortonworks said the acquisition provides it with key technology and engineering expertise to deliver a single way to administer security across all Hadoop workloads.
Ever since the introduction of the YARN resource-management tier last October, which allows multiple workloads to run on the Hadoop distributed big data platform, there has been an increased need for central security, according to Hortonworks VP product management Tim Hall.
"YARN unlocks the ability for you to implement a modern data architecture, a data lake, with multiple workloads, not just with batch. We have batch, interactive and real time," Hall said.
"Customers are saying, 'This is great. I now have this ability to take full advantage of Hadoop across all these different environments, a single data lake. That's great because now I have centralised operations on that — but what am I doing about centralised security?
"In particular, while authentication has been largely addressed, authorisation has really been a sort of fragmented approach thus far within a Hadoop ecosystem."
Authentication deals with establishing the identity of an individual while authorisation is about access control. Every individual Hadoop component has looked at implementing its own authorisation scheme.
"Of course as you realise a mixed workload of environment and distribution, this starts to bring in a lot of complexity and additional cost," Hall said.
Controls currently exist at the HDFS level, the file level, where access control lists have been set up.
"That's one approach and it's at the lowest possible level of the infrastructure. Most recently in Hortonworks Data Platform 2.1 there is a new feature for Hive that's called Next Generation Authorization," Hall said.
"That feature provides a DBA-style grant-revoke of privileges on tables and views. That's great if that's the only kind of workload that you're running on Hadoop. "
The acquisition of XA Secure will enable Hortonworks to offer a centralised security administration and co-ordinated enforcement across all the processing engines being used in Hadoop.
"For enterprises that want to adopt Hadoop, we're going to give them a single pane of security administration glass and they will be able to provide centralised authorisation across all those different engines, be it HDFS, Hive, HBase and so on," Hall said.
Hortonworks plans to open-source the XA technology in the second half of this year.
"We are committed to 100 percent open source, so we will be donating the underlying software components to the Apache Software Foundation in a Hadoop project that we will incubate through the process and open-source it for all," Hall said.
"We're been through this process before. We believe this will happen in the second half of 2014. The speed at which it will go through incubation and acceptance by the community members and so on is anyone's guess. But the introduction of the code into the Apache incubator we absolutely plan for the second half of the year."
While the transition to full open source takes place, Hortonworks will provide customers with access to the software in binary form.