Hortonworks introduces DataFlow, acquires Apache NiFi-backer Onyara
Last month I wrote about Apache Ni-Fi, a project borne of (non-shady) work at the US National Security Agency, and now a top-level project at the Apache Software Foundation. NiFi is all about building data flow orchestrations, and features a browser-based "boxes and lines" graphical user interface for getting the work done.
NiFi certainly seemed an interesting project, but it turns out it's more significant than I thought. Hortonworks is betting big on it: it's acquiring Onyara, the commercial entity behind NiFi, and releasing its own distribution of NiFi as Hortonworks DataFlow, a separate subscription aside the Hadoop-based Hortonworks Data Platform.
You read that right; Hortonworks is essentially placing NiFi on equal footing with Hadoop, viewing it as its platform for data-in-motion (read: IoT) while now casting Hadoop as its platform for data-at-rest.
What this means for Hortonworks' future "gusto" behind Apache Storm and Spark Streaming is unclear, since those are constituent components of the company's Hadoop distro and they do deal with data-in-motion. For what it's worth, NiFi integrates with Spark, as well as Apache Kafka and Hadoop itself.
It will be interesting indeed to see how Amazon Web Services, Microsoft and Google, each of which has a commercial cloud platform for streaming data (and IoT) respond, or if they need to. Heck, Google's platform is even called Dataflow, and it went into general availability less than two weeks ago.
Just when you thought Apache Spark was Big Data's poster child for disruption, here comes NiFi.