Startup Jut today lifts the lid on its analytics software for operations teams, which it says can pull together historical and real-time cloud and on-premise data into a single view.
After six months in a closed scheme, San Francisco-based Jut's Operations Data Hub for DevOps is now available free in an open beta, with general availability planned in the next three to six months.
To be able to pool metrics, log and event data, Jut uses two purpose-built back-ends. One is based on the Apache Cassandra NoSQL distributed database and deals with structured data, while the other one uses the Apache Lucene-based Elasticsearch search server for unstructured and semi-structured data.
Because the back-ends are not real-time engines, Jut has built a layer on top of them to deal with real-time data and merge real-time with historical data.
"We just realised that with the emergence of open-source big-data projects there was an opportunity to build an analytics and visualisation layer on top of projects like Cassandra and Elasticsearch," Jut founder and CEO Steve McCanne said.
"The other thing we thought was important was there wasn't a good connection between real-time and live data with historical analytics and the merging of logs and events with time-series metrics."
According to McCanne, companies that want analytics for operational data either have to write the tools themselves or cobble together the information using a number of commercial products.
"You could go code up MapReduce jobs or some sophisticated Spark-based analytics as a development project to answer the kinds of operational questions you might have about your environment. Or you could go assemble a bunch of different tools that live into different silos. 'Hey, I've got my data in tool 1 but I want to see it in tool 2 and it's hard to do, so I've got to do an ETL or write a script'," he said.
"There's the log stuff, there's the metrics stuff and there's a bunch of open-source things. [Firms] are stitching together all these different tools that have different UIs and different data schemas and trying to make sense of it all. We want to make it easy, so a development team doesn't have to get together and write Java and obscure Scala code to solve these problems."
Jut employs a hybrid software-as-a-service approach to delivering the analytics platform, together with the Juttle dataflow programming language for creating queries across operational data.
"You create your account on Jut.io. It's in the cloud. When you do deployments you actually download software onto your server and your security domain and install that software on your hardware or your cloud instance but it's the customer's - so it's a sort of a hybrid," McCanne said.
"Our web-based applications interact directly with the infrastructure. The benefits of that is all the data and the security aspects are in the customer's domain, so they don't have this optic where they're having to fit their sensitive data into our cloud. It's in their domain."
Compute and storage operations take place in the customer environment where the data engine is installed, with a small amount of processing occurring in the browser to compute views.
He said customers are also spared the effort of upgrades and maintenance because the application is delivered from the cloud and continuously updated.
McCanne said the Juttle language, which unlike Java and Scala is declarative rather than imperative, resembles SQL and enables the creation of dataflow-style queries with answers returned in real time.
"For people familiar with the UNIX shell model where you do a query and then pipe it to something else and then pipe it to a view, they're comfortable with that," he said.
McCanne said some customers initially bridle at the prospect of mastering a new language.
"Depending on the level of sophistication, some of them will be, 'Hmm, this is kind of tricky. I'm not sure I want to learn a new language," he said.
"On the other hand, once people get over the hump, they really appreciate the power of what they can do with Juttle because they realise [to match what they can achieve] in three lines of Juttle they would have had to write a 100-line script or a 100-line Scala program on Spark."
Under the open beta, the Jut Operations Data Hub for DevOps is free and comes with engineering support. The company is considering using data points as the eventual basis for charging.
Jut was founded in 2013. After securing $3m in seed capital, it added a further $20m injection from Accel Partners, Lightspeed Venture Partners and Wing Venture Capital in November 2013.
McCanne said the company is exploring two avenues for future development. One consists of building applications on top of the platform, with the second approach involving embedded analytics, where customers use Jut software in their apps.