Splunk's big data Hunk gives Hadoop muscle to non-techies

With the Hunk integrated analytics platform for Hadoop, Splunk is pursuing its goal of spreading the reach of big data beyond the specialists.
Written by Toby Wolpe, Contributor

Splunk says its Hunk product for Hadoop will open up big-data analytics to a wider business audience and also make life easier for developers.

Hunk, which is now downloadable for a free 60-day trial, makes Splunk's analytics available for use on machine-generated data stored in Hadoop.

Last year, the firm launched its Splunk Hadoop Connect add-on, which allows data to be moved between flagship analytics platform Splunk Enterprise and Hadoop.

Splunk technology evangelist Matt Davies said that while Hadoop Connect moves data in and out of Splunk Enterprise, Hunk explores and visualises data that's too big to transfer.

"There are a number of challenges with Hadoop implementations and the idea [with Hunk] is that having Splunk natively on top of Hadoop would help tackle a lot of them," Davies said.

"Hadoop is very good for storing lots of data but it's hard to get the value from it. It's hard to get the analytics on top of all that data.

"It's hard to get the skills, the people who can implement Hadoop, and the data scientists who have the skills to get the information back out of Hadoop are hard to find and expensive, and the technology itself — it's not a very easy thing just to take Hadoop and install it and off you go."

Davies said Hunk pursues the same philosophy as Splunk Enterprise 6, launched earlier this month, of making business analytics easier for non-experts.

"[Splunk Enterprise 6] made it easy to open up your web browser and start self-serving and creating your own analytics. All those things are part of Hunk," he said.

"You've got your petabytes of data inside Hadoop. You can very quickly drop Hunk on top of Hadoop and start searching straightaway and visualising that data. It's meant to be easy for those business people to get value from the data in Hadoop."

Splunk has also worked hard to make Hunk's development framework to make big-data applications easier to create for developers, Davies said.

"We have a number of APIs around the whole of Hunk but also a number of dedicated software development kits, so you can develop in the language you’re most comfortable with on top of all that data that's in Hadoop," he said.

Hunk, which has been in beta for about six months, includes a standards-based web framework, documented REST API and software development kits for C#, Java, JavaScript, Python, PHP, and Ruby.

Davies said a leading ecommerce retailer has been using Hunk in the beta, building a 360-degree view of what customers have done historically and in real time in terms of their purchases.

"They're taking all the data from the weblogs, what's going on on the ecommerce site, all the images that someone has looked at, all the proxy logs — there's a lot of data in there and they're building customer ecommerce apps based on top of that data," he said.

According to Splunk, Hunk runs on the major Apache Hadoop distributions, including Cloudera, Hortonworks, IBM, MapR, and Pivotal.

The product enables users to interact with data, change perspectives and preview results while MapReduce jobs are running. Splunk also said Hunk allows queries to be run on Hadoop data without needing to know upfront the questions to be asked later.

US Hunk pricing starts at $2,500 per Hadoop node for one year, with a minimum of 10 nodes.

More on Splunk and Hadoop

Editorial standards