SAP Introduces Spark-based HANA Vora
I've always found the segregation of Big Data and Business Intelligence (BI) to be contrived and counterproductive. The very reason I wanted to call this blog Big on Data is so that I'd have room to cover BI as well. Yes, the technology names allude to different eras in the industry and, yes, they are often based on different technologies. But they are more points along a spectrum than they are discrete positions.
Today, SAP is introducing a new technology, dubbed HANA Vora (a name derived from the word "voracious," reflecting its appetite for data), that almost epitomizes the idea that Big Data and BI are complementary. Vora melds Big Data technologies like Hadoop and Spark with the original SAP HANA, and downstream sources like SAP BW, Business Objects and ERP. In the process, it brings BI-style dimensional (drill-down) analysis into the Big Data world.
Old and new
HANA Vora is based on the combination of Apache Spark and Hadoop 2.0/YARN. It then provides connectivity to the original SAP HANA, premised on push-down query delegation. It also layers in Spark SQL enhancements to handle hierarchical queries and a pre-compiled query facility comparable to what relational databases and data warehouses have had for years.
Essentially, Vora federates data lakes with Enterprise systems of record and does so without incurring the costs of data movement (since "classic" HANA executes its own queries). Furthermore, it provides for the definition of dimensional hierarchies and the ability to use them in analytical queries against all the data that Vora can address.
Ready to wear
Vora requires no dedicated hardware infrastructure, as it co-locates on the cluster nodes on which Hadoop and Spark are themselves deployed. Clearly, if you're going to integrate Vora with classic HANA, the latter will need its own infrastructure. But Vora can also be used on a standalone basis with no additional hardware requirements.
Vora could end up being a very sensible way for SAP customers to move forward with Hadoop, Spark and Big Data in general. And since Vora is a commercial software offering from SAP, and not an open source offering, it fits with SAP's existing business model, rather than requiring the company to change gears in some contrived manner.
The art of compromise
HANA Vora hybridizes on many levels: Big Data with BI; startup technology with established Enterprise software; data lakes with vetted systems of record; and, finally, in-memory and disk-based storage and processing.