SAP's HANA gets a lot of attention, but it's not the only in-memory game in town. GridGain today announced the release of its in-memory stack, coming only two weeks after the release of ScaleOut Software's hServer in-memory data grid v2.
By combining in-memory databases with grid infrastructure, both companies' products offer the ability to use memory to process data more quickly. GridGain and ScaleOut's technology can also both be used as accelerators for Hadoop, by providing custom readers and writers. This allows MapReduce code to execute over in-memory data, rather than data stored in Hadoop's Distributed File System (HDFS). And unlike HDFS, data in these in-memory stores are updatable.
Considering the latency introduced by MapReduce's batch mode operation, the acceleration of its actual processing is intriguing, to say the least. It's not just about speed though; it's about workload as well. In-memory MapReduce significantly raises the potential for Hadoop to be used on streaming data, in real time, without the latency introduced by disk access.
For his part, Nikita Ivanov, CEO and founder of GridGain says "Traditional computing has literally reached its limit in terms of processing." And Bill Bain, ScaleOut Software's CEO, explains that "By enabling real-time analytics for Hadoop, which has emerged as by far the most popular platform for analyzing big data, we aim to dramatically improve the effectiveness with which organizations can manage their live data."
While SAP HANA works side-by-side with Hadoop by querying data from it, and then processing that data, GridGain and ScaleOut offer Hadoop integration instead. Leave it to the smaller companies to push the envelope. And watch for in-memory products to become more mainstream, more accommodating of existing data infrastructure and, with products like SiSense's Prism -- more judicious in their use of the cache-RAM-SSD-HDD hierarchy.