Five rules for success with 'Big Data as a Service'

Five rules for success with 'Big Data as a Service'

Summary: BDaaS is everyone's puppy — here are ways to make it work for the enterprise.


"Big Data as a Service..."  Sounds like the ultimate buzzword mashup, doesn't it? Yet, there needs to be some ability to bring data resources into the enterprise, package it, and make it presentable and easily accessible to decision makers, without requiring them or IT to spend weeks and months finding, securing and cleaning data sources. Big data should be in the cloud — at least a private cloud.

Varun Sharma, an enterprise solutions architect, proposes a tiered big data reference architecture for capitalizing on the power and potential of data while ensuring security and governance. He notes how new organizations with innovative approaches — such as Tesla Motors — are relying on gobs of data from a range of devices and processes in the next generation of products.

Writing in Service Technology Magazine, Sharma provides some common-sense guidelines for making BDaaS a well-functioning platform within enterprises:

Ensure there is well-designed data governance. "Data governance is a must-have, and no longer merely a good-to-have," says Sharma. In today's extremely hyper-competitive markets, insightful knowledge means the difference between success and being overwhelmed. But it has to be based on the right data, based on business requirements.

Ensure data is protected. "Ignoring data security, data quality and data access can cost organizations millions of dollars, hurting enterprise agility, efficiency and reputation."

Pursue a tiered data strategy. "Break the operational tiers for data flow into logical groups," Sharma advises. The way to do this is by establishing a "consumption tier, analysis tier, organization tier and acquisition tier, to allow agility via loose coupling and abstraction."

Don't try to rush all data out to everyone all at once. "Consider the whole cycle from the acquisition of data to the extraction of information, and consider the hygiene factors along this path." There is a time in which data should be immediately available to decision makers, and there is a time when it can be retired.

Remember that BDaaS is everyone's puppy. "Successful Big Data-as-a-Service implementation would require close collaboration between Enterprise Architects, Data Architects, Database admin, BI and DW SMEs, SOA experts, InfoSec representatives and business strategists," Sharma writes.

An additional observation: What Sharma talks about here may not even be called "Big Data as a Service" two or three years from now — it may simply be data services. But it will certainly go a long way in supporting analytics-driven organizations.

(Thumbnail illustration: Joe McKendrick.)

Topics: Big Data, Cloud, Data Management

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Big Data Solution

    Joe, great insight. With the explosion of big data, companies are faced with data challenges in three different areas. First, you know the type of results you want from your data but it’s computationally difficult to obtain. Second, you know the questions to ask but struggle with the answers and need to do data mining to help find those answers. And third is in the area of data exploration where you need to reveal the unknowns and look through the data for patterns and hidden relationships. The open source HPCC Systems big data processing platform can help companies with these challenges by deriving insights from massive data sets quick and simple. Designed by data scientists, it is a complete integrated solution from data ingestion and data processing to data delivery. Their built-in Machine Learning Library and Matrix processing algorithms can assist with business intelligence and predictive analytics.
  • Big Data as a Service

    BDaaS is certainly something we are seeing, indeed Google and Amazon are investing heavily in cloud based infrastructures for this. What we did on the Business Data Lake was start with a principle of ‘Store Everything’ in the lake and then shift towards distillation from the lake. It’s also very important to consider the different ‘heart-beats’ of information when looking at analytics. Making decisions in real-time within processes has a different flow than doing large scale predictive analytics. With the Business Data Lake this is why we split information into Batch, Micro-batch and Real-time on ingestion and a similar split on insights. This is also the basis for our Elastic Analytics approach.

    BDaaS is certainly something that requires governance, but it requires governance that enables collaboration, not the traditional IT approach of trying to govern everything, the concept of trust becomes more important as well as being able to provide lineage on data and analytics to understand and prove how a decision was made.

    The challenge today is that many BI people are approaching BDaaS from a BI perspective. The key thing is to think about how BDaaS would provide business insights within the context of a business service. This could lead to data services, or ‘insight services’, being the business view on a BDaaS solution.

    Steve Jones, Director of Strategy for Big Data and Analytics, Capgemini