Big Data: How close to enterprise-ready?

Moderated by Steve Ranger | October 1, 2012 -- 07:00 GMT (00:00 PDT)

Summary: Big Data technology can absolutely take Enterprise analytics to the next level....but when?

Lawrence Dignan

Lawrence Dignan

Will ramp quickly

or

Not even close

Andrew Brust

Andrew Brust

Best Argument: Not even close

44%
56%

Audience Favored: Not even close (56%)

Closing Statements

Big data projects will ramp faster-than-expected

Lawrence Dignan

And there are likely to be some home runs as well as spectacular failures. Ultimately, companies will have to spend time on what they want to know, why they want to know it and how this new insight will be used. It's a business imperative to get the big data efforts in gear. Based on the pilots and efforts already underway, I expect big data efforts to come out of the gate strong, ramp quickly and then there will be a re-evaluation period. Biggest roadblock to big data efforts may be a shortage of talent.

Far from imminent

Andrew Brust

The big data technology that is here today provides unprecedented functionality and processing power, but it's raw power. While highly-specialized professionals can use big data technology to produce crucial analyses, enterprise IT doesn't find it actionable yet. This is a question of standards, not just skills. Enterprise IT departments need products to be resilient and compatible with the management suites, procedures and best practices that they have in place.

Companies like Cloudera, MapR and others are working on this problem, and at some point the problem will get solved. But until the numerous big data pure-play companies consolidate and work more closely with the mega-vendors, big data will remain the province of Internet companies, skunk works teams and pilot projects, rather than the mainstream technology fabric of the enterprise.

The gap will close, but that's far from imminent.

This was a finely balanced debate

Steve Ranger

Companies big and small are desperate to get more use out of all the data they’ve been squirreling away for years, which is why big data is so attractive to so many. It’s undoubtedly cutting edge technology, but I’m not convinced that all the pieces are in place, just yet, to make it a mainstream option.

CIOs are looking at big data right now but it’s only the very brave -- or those with a really pressing need -- that are making the jump right now. So Andrew takes this one, by a whisker.

Talkback

16 comments
Log in or register to join the discussion

  • The low hanging fruit is driving fast adoption

    There are some fast wins (projects that are easy to impliment, low cost, small in scope, and will deliver business value greater than TCO) that will allow companies to say "We are also doing BigData" which seems to drive a lot of executive motivation (similar to cloud efforts in 2009). In the trenches I plan to see how we can virtualize the Hadoop products and minimize or replace parts of Mapreduce that are Single Points of Failure with either anternative solutions or with process automation to support an agile release of Hadoop functionality.

    I see it as more of us starting now so we have a running start as we learn about the pain points we discover as we race down to the valley of disalusionment and start up the next hill.
    Datalas
    Reply Vote I'm for Will ramp quickly

  • If it's not simple to understand, simple to implement, and simple to use,

    then, it will be an exercise in futility.

    If the tools aren't there, and companies and developers aren't adopting big data in large and small companies, then, it makes it even worse. SQL works are many different levels, and thus, we'll have SQL serving our basic and big needs for a long time to come, without having to adopt big data. SQL itself is capable of doing "big data", and, until somebody can make big data something that companies, big and small, want to adopt, and until a massive number of developers everywhere want to involved, then, big data will remain a curiosity that, those that absolutely need it, will adopt and use.

    Perhaps the marketing is all wrong, and "big data" sounds like something useful to only those that generate humongous amounts of data. Most companies don't generate "big" data, and many of those that do, can manage with what's already available, namely, SQL and flat data management.
    adornoe
    Reply Vote I'm Undecided

  • Slow uptake is an understatement

    Cool isn't always cost effective! the mid-sized businesses i have worked for recently are just starting to build their Data Warehouses using less complex solutions and in-house know how coupled with standard database technology. Big-data only makes sense for gigantic companies with nearly unmanageable amounts of data. It doesn't make sense for small-mid-sized business(the meat and potatoes of IT shops) to adopt big-data technologies/strategies.
    ammohunt
    Reply Vote I'm for Not even close

  • Not sure I really see the benefits . . .

    Not sure I really see the benefits - data mine huge amounts of data in hopes that you might find something special?

    If you find something special, great. But if you don't - haven't you just thrown out millions of dollars?

    Frankly, it sounds to me more like large scale gambling than a real business plan.
    CobraA1
    Reply Vote I'm for Not even close

  • Big data needs a strategic approach

    Big data, and big data management, have existed for some time in the form of massive numbers of transactions and huge amounts of data. For example, a typical digital media retailer now manages hundreds of suppliers and customers across tens, if not hundreds, of countries.

    Many firms over recent years have gained strategic advantage through customer data and CRM, processing millions of transactional data points with near real-time associated details linked to spending habits and behaviours. Big data of course will continue to become more complex with multi-channel, social media and mobile being added to the mix; but this does not need to be an overwhelming challenge if the board recognises this and implements a robust and transparent data management strategy.
    T Atkins - Microgen
    Reply Vote I'm for Will ramp quickly

  • just getting started

    While the technology stack is new, Big Data (and in particular machine learning algorithms) will automate some BI work out of existence.

    For the DBAs above that don't yet get it, your relational SQL database can't scale to terra/petabytes without a lot of pain and $$$. Hadoop, BigQuery, Drill, etc can and do. You can run a very capeable 8 node Hadoop cluster at AWS for < $1K a month (and easily process terrabytes per day).

    You might ask who has that data much data? I would counter anyone who has weblogs, text, or large tables across systems. The easy winds are in data we are already collecting but throwing away. We (as technologists) need to get away from the mindset that we don't have the space or processing capacity to interrogate what we are now throwing away. We do.

    Even better is the ability to deal with complexity, being able to throw hardware at complex joins that bring DWs to their knees is life changing. Right now I am doing million/million row full table scans in minutes and for spare change (EMR/S3).

    And it is not just SQL. Mahout and R give you the ability to create ML algorighms for predicting and driving real time system decisions. And not stuck in some DW somewhere!

    When I told our DW/BI team I had every action of our 50K daily users in our application, and I had the last few years of actions mapped to outcomes they just didn't get it. I have that data raw (not stripped down summary data), and can move it in and out of my cluster as needed. I can create models where I look at things happening in real time, and then based on what I have seen in the past try to steer things in the right direction.

    That, is where we are headed.
    mobile_manny
    Reply Vote I'm for Will ramp quickly

  • Uhm what?

    It kind of sounds like these guys are almost saying the same thing in all these questions. Is it just me?
    Chris Schrader
    Reply Vote I'm for Not even close

  • I didn't learn anything from Lawrence's arguments

    NoSQL + Hadoop + Datawarehouse = ? That's just the storage and the framework. With this your data will just sit there. You still need data processing and analytics tools.

    Andrew's ones make sense and are consistent with the questions asked. Thanks for the dabate.
    RelaxWalk
    Reply Vote I'm for Not even close

  • Larry is Always Right

    Voting on a concept that is best understood by the top analyst at ZDnet is like asking children to verify their father's calculus.

    You don't have to understand it for it to be true.
    tomogden
    Reply Vote I'm for Will ramp quickly

  • Big data might be ready, but enterprises are not

    The technology is there, but that must first be backed by strategy and what you want to achieve. mobile_manny gives an excellent example of using big data for systems analysis, but big corporates would ultimitely want to use big data to better understand their customers.

    They will have many different data silos, different formats and standards. A strategy will have to be put in place to effectively use this data first, and that will take time, LOTS of time.

    So although the technology is there and ready, big corporations are not ready yet.
    nicopretorius
    Reply Vote I'm for Not even close
CNET Join | Log In | Privacy | Cookies Close