Mic check
Are my debaters standing by? We'll be starting promptly at 11am ET / 8am PT
More Topics
Big Data: How close to enterprise-ready?
Big Data technology can absolutely take Enterprise analytics to the next level....but when?
Lawrence Dignan
Will ramp quickly
or
Not even close
Andrew Brust
Best Argument: Not even close
Audience Favored: Not even close (56%)
The moderator has delivered a final verdict.
Opening Statements
Will ramp quickly
Larry Dignan: Big data is being piloted in large enterprise settings with Hadoop clusters, connections to data warehousing and other plumbing being hooked up. In other words, big data is already happening in the enterprise, but it is admittedly early. I'm betting that adoption will ramp quickly because enterprises---already drowning in data---will need big data to make sense of internal information flows. Everything from fraud detection to network maintenance to sensor data to customer service will be touched. Once real ROI is achieved---and I've seen a handful of business cases up close---every corporation will want on the bandwagon. The only real limitation will be talent.In the end, enterprises will have a data fabric with Hadoop, analytics and data warehousing information. Many of those parts are already in place.
Not even close
Andrew Brust: Big data technology is exciting, innovative and genuinely powerful. It can absolutely take Enterprise analytics to the next level...but not yet.In Global 1000 organizations, and numerous smaller companies, skill sets and best-practices have been building for years around Business Intelligence (BI) technology and for decades around relational database management systems (RDBMSes). The products in these categories have superior tooling, manageability and fault tolerance. They offer user interfaces designed for non-developers. They are repositories for carefully crafted data models, refined over the years, representing unparalleled investment.
Meanwhile, Hadoop is typically used at the command line, controlled by MapReduce code that must be written in Java, using a file system (HDFS) controlled by a single, vulnerable name node. Some browser-based tooling is emerging and technologies like Hive provide a primitive connection layer for BI tools, but we’re still at a 1990s-era level of sophistication. This stuff is not enterprise-ready yet. It’s not even close.
The Rebuttal
Great Debate Moderator
Ready to go
Lawrence Dignan
I am for Will ramp quickly
Bring it on!
Andrew Brust
I am for Not even close
Great Debate Moderator
The big fix
Companies have always struggled to turn data into information. Do you think Big Data can really fix this now?
Posted by Steve Ranger
I think Big Data can help but it's not a cure all.
Enterprises will always be plagued by the following questions. What do I want to know? What do I do with the data? What are the returns and ultimate business impact? Big Data is another tool that enterprises will take for a spin sooner rather than later. Clueless companies and management will exist no matter what the tools are.
Lawrence Dignan
I am for Will ramp quickly
Turning data into information has indeed been a struggle for some time...
... in many cases because of the politics and logistics of acquiring, sharing and cleansing data in the enterprise. Big Data won’t fix this any more than Business Intelligence did. Making data “bigger” only increases the surface area of the data to be governed and arguably makes its analysis more complex.
One thing Big Data has on its side is a more flexible and agile approach to schema, allowing it to be defined at query/analysis time, thus removing some of the complexity and bureaucracy in curating the data. But the tooling for managing unstructured data is relatively immature, and data specialists in the enterprise are not conceptually accustomed to it.
The long term potential for Big Data here is good, as it should shorten innovation cycles. But in the near term things just aren’t that actionable yet.Andrew Brust
I am for Not even close
Great Debate Moderator
Big Data for all
Is Big Data only for big companies? Or will every firm need a Big Data strategy?
Posted by Steve Ranger
I think every company---and employee---will have to become data centric.
Use cases will be all over the place. A SMB that can monitor sentiment and tie to to growth will ultimately become a large enterprise. In some respects, smaller companies may be at an advantage. They will be more nimble and willing to experiment. Large companies just don't move that fast.
Lawrence Dignan
I am for Will ramp quickly
Data can help everyone...
If I’m a quick-serve entrepreneur owning five McDonald’s franchises in a mid-sized city, it’s not obvious how I could take advantage of Hadoop and MapReduce to get more customers and more visits. If I’m a large Web company, a major financial services firm, a manufacturing concern or a major retailer, with big, continuous streams of clickstream, market or sensor data, then the appeal of Big Data is much more straightforward. I do think that smaller businesses should begin developing their big data strategies now though. Even they will have substantial clickstream data if they’re online (and most are) and even brick-and-mortar operations can start to accumulate volumes of in-store video recordings (which can reveal shopping habits, store layout effectiveness and product affinities). Data can help everyone, and when you stop throwing it away it becomes Big Data. But mining it has to get easier or small and medium businesses won’t be able to make their move.
Andrew Brust
I am for Not even close
Great Debate Moderator
Is Big Data worth the effort?
Where’s the ROI? How do you build a business case for Big Data?
Posted by Steve Ranger
The business case revolves around revenue, risk and costs.
At this juncture, the Big Data pilots should probably start small with some underserved department who has a leader willing to make a few strategic bets. The business case today won't be the one tomorrow. I think Big Data as a service isn't too far away. When that happens the business case will be low risk.
Lawrence Dignan
I am for Will ramp quickly
In the Internet world, Big Data can pay off in increased eyeballs, session lengths and corresponding monetization
In the manufacturing world it can pay off in reduced or eliminated downtime of assembly lines (through predictive analytics on equipment breakdowns). In the Financial Services world, Big Data can lead to better, more effective, and therefore more lucrative, trading strategies. Media companies can sell more ad impressions. E-Commerce concerns can sell more product. But each of these companies has something in common that the average Enterprise business unit may not have: the ROI for them is demonstrable enough and big enough to get them over the barriers to entry. Will the line-of-business teams in the Enterprise have the smarts, the budget, or the appeal to bring in the Hadoop specialists, statisticians and data scientists necessary to attain compelling ROI? Probably not. Big Data value through off-the-shelf products and professional services has to get good enough, cheap enough and mature enough for these customers to buy in.
Andrew Brust
I am for Not even close
Great Debate Moderator
Putting Big Data to work
What are the essential Big Data technologies?
Posted by Steve Ranger
Essential technologies
Essential technologies to me include NoSQL, Hadoop and your friendly neighborhood data warehouse today. Unstructured databases are likely to complement your relational ones. It will be a fabric of things melding Big Data, data warehousing and analytics.
Lawrence Dignan
I am for Will ramp quickly
Top of the list
Hadoop, and some of its companion components, like Hive and Pig, are top of the list. For those that can afford them, Massively Parallel Processing (MPP) data warehouse appliances, which use in-memory and column-store technology to deal with Big Data workloads, are all but required as well. General purposes analytics tools like “R” (and its commercial competitors from SAS and IBM/SPSS) should be on the shopping list too. True predictive analytics technologies, including the Mahout machine learning tool that works on top of Hadoop, or more commercial Data Mining products, are excellent, provided you have the talent available to use them.
Andrew Brust
I am for Not even close
Great Debate Moderator
Developing the best strategy
How does Big Data tie into an organization’s broader IT strategy? Does it stand alone or will it force companies to rethink their architecture?
Posted by Steve Ranger
More of a business strategy
The model I'm seeing is that big data is more of a business strategy that happens to have IT. Shared services for analytics/big data are common at large enterprises. There will be a rethinking of architecture, but that's a few years away.
Lawrence Dignan
I am for Will ramp quickly
Big Data definitely has the potential to transform organizations’ overall IT strategy.
That’s because Big Data is about more than Big Data per se. For example, Hadoop’s use of direct-attached storage and commodity hardware is very disruptive to the common enterprise deployment of storage networks and expensive servers and appliances. Hadoop may also cause enterprises to emphasize Java skills more and SQL skills less, among other shifts in skillset priorities. The clustering approach used by Hadoop may accelerate adoption of hybrid on-premise and cloud strategies too: it’s easier to push data to on-premise servers, but the elastic nature of the cloud may be more effective in addressing intermittent demand for extremely large clusters.
Andrew Brust
I am for Not even close
Great Debate Moderator
Training camp
What new skills will enterprises need to make Big Data work?
Posted by Steve Ranger
Data literacy
The biggest skill will be data scientists (ask the questions, know the math, know how to use the answers). Database admins will be big data admins. The biggest skill will be data literacy on everyone's part.
Lawrence Dignan
I am for Will ramp quickly
May be very hard to recruit people
Math, statistics and data modeling skills are needed, and there’s a shortage of these. Universities are only now addressing this problem with degree programs in analytics and data science. Java programming skills, as I mentioned above, will be highly useful, even for jobs that are data-oriented and not developer positions. What may be most important though, and most difficult to find, are individuals who have these tech skills in combination with strong industrial domain expertise. That’s the winning formula, and it may be very hard to recruit people who fit with it.
Andrew Brust
I am for Not even close
Great Debate Moderator
Who's going to be first?
Which industries are mostly likely to adopt Big Data? And are there industries where Big Data is going to be a waste of time and effort?
Posted by Steve Ranger
Today, financial services and healthcare are leading the big data charge.
Retail will have to get some serious mojo out of big data. Can a retailer combine sentiment and social buzz with transactional data ultimately? Sure. I think all industries will be impacted because all use data. What will change is whether the data is used for supply chain, costs, revenue or customer service. That may vary by mature vs growing industry.
Lawrence Dignan
I am for Will ramp quickly
Every organization has Big Data...
Again, Internet-focused organizations, as well as media, financial services, online retail and manufacturing are the industries who may have the most to gain. Supply chain companies, be they purveyors of parts and components, or distributors, can certainly be added to the list. So too can healthcare, be it in the areas of research, hospital management or payer/insurance operations. Marketing organizations across industries can get great benefit from Big Data. I think every organization has Big Data…it’s just that some don’t monitor it, and many do not retain it. Those that can and do are in the best position to derive advantage from Big Data. Those that don’t have to evaluate the costs and benefits of changing their operations model in order to become data-driven.
Andrew Brust
I am for Not even close
Great Debate Moderator
Who's ready to take charge?
Which vendors are most likely to become synonymous with Big Data?
Posted by Steve Ranger
I think the usual suspects---IBM, Oracle, SAP---are already talking the big data game.
However, there are dozens of smaller vendors who have pushed the big data agenda. Cloudera and Splunk are notable, but there are companies like Opera Solutions. It's too early to tell really, but the field is open.
Lawrence Dignan
I am for Will ramp quickly
Cloudera and IBM come to mind right away...
... as they have the best public profiles in the Big Data arena. Eventually I think most other big software providers (like Oracle, Microsoft and SAP), storage and networking companies (like EMC, NetApp and Cisco) and certainly BI companies (like MicroStrategy and SAS) will be identified as credible in the space.
Andrew Brust
I am for Not even close
Great Debate Moderator
Happy together
What about the cloud and Big Data – do they play nicely together?
Posted by Steve Ranger
No choice
Cloud and big data have to play together because not every enterprise is going to have the horsepower to crunch the information. Luckily, there are cloud options for that like AWS.
Lawrence Dignan
I am for Will ramp quickly
This cuts both ways.
The commodity hardware and add-more-as-needed clustering approach of Hadoop has huge affinity to the cloud computing model. In general, elasticity is a feature of both. On the other hand, upstream bandwidth is still a limiting factor for Big Data in the cloud – it’s much easier to stream new data and maintain cloud databases (including Hadoop Distributed File System files) than it is to migrate that data en masse and establish the databases in the first place. This is yet another area where things will change, and barriers will melt away…eventually.
Andrew Brust
I am for Not even close
Great Debate Moderator
Troubleshooting
What are the potential pitfalls of Big Data projects?
Posted by Steve Ranger
Asking the wrong questions
The biggest pitfall to me will be asking the wrong questions and making the wrong moves based on the data. I think companies are also going to think they are know-it-alls because they have all the data. Just because you have the data doesn't mean you know what to do with it.
Lawrence Dignan
I am for Will ramp quickly
Data quality is probably the biggest.
So too is the broader question of data governance. In both cases, the prevalence of unstructured data can make data integration difficult and curb success rates. As well, the still modest level of maturity in many Big Data technologies is a potential pitfall as well. Lots of companies are in the R&D phase with Big Data because of this. The technology will have to become much more accessible, and defensive project management skills more widespread, before Big Data can truly go mainstream.
Andrew Brust
I am for Not even close
Great Debate Moderator
The need to know
Does the CEO understand or want big data? Does it matter?
Posted by Steve Ranger
The CEO gets it because it's ultimately his or her arse on the line.
CEOs are responsible for the bottom line, revenue and risk. Big data will look like a cure all in many respects. All CEOs will be talking the data game.
Lawrence Dignan
I am for Will ramp quickly
Has to get easier
I think lots of CEOs understand Big Data at a high level and therefore, yes, they do want it. But their management teams have to understand Big Data more granularly, and execute on Big Data initiatives. At the risk of being repetitive, I don’t think we’re there yet. Big Data has to get easier for non-specialists to partake in, and for managers to understand fully, before it becomes pervasive.
Andrew Brust
I am for Not even close
Great Debate Moderator
The best person for the job
Who is going to run Big Data projects – the CIO or CFO? Which is more likely to succeed?
Posted by Steve Ranger
Neither.
The big data projects will be overseen by the business line executive. To really work, all C-levels will have to be on board. The big data is really a way of thinking, experimenting and letting the data complement human judgement.
Lawrence Dignan
I am for Will ramp quickly
It’s an interesting question...
...because in many companies the Business Intelligence buying decision lies in the CFO suite. And if Big Data is the successor to BI, then it might stand to reason that CFOs will maintain this control. But Big Data project leaders will more likely come from IT and the line-of-business corners of the Enterprise. Being hands-on with the technology and being acquainted first-hand with the data are likely prerequisites for project success. And data for financials is relatively discrete too – maybe there are petabyte-scale general ledgers out there, but I haven’t come across them yet.
Andrew Brust
I am for Not even close
Great Debate Moderator
Fiive-year plan
How does Big Data develop over the next five years? What will stop it being just another fad?
Posted by Steve Ranger
Big data is likely to develop like Linux.
Right now the big data talk is everywhere and hype is at its peak. That'll die down and we'll move on to some other buzzword. However, in five to 10 years big data tools will be in every enterprise. We just won't call it out. Think about Linux. It made a big splash, then largely lost its buzz. The funny part is Linux is now in every data center.
Lawrence Dignan
I am for Will ramp quickly
Definitely not a fad
Big Data may be at the top of its hype cycle right now (or it might not be), but it’s definitely not a fad. In my experience, very little around data is. Whether it’s line-of-business app development and corresponding transactional database needs, or dimensional analysis, or the kind of predictive analytics and other insights to be gleaned from Big Data, we’re talking about useful, important technology.
Typically, a new data technology starts out as innovative and ground-breaking, then becomes mainstream and mission-critical, and eventually commoditizes, but it doesn’t often just fizzle out and go away. There’s no question in my mind that Big Data is going to be big and mainstream in the Enterprise, in the future. That future may or may not be within the five-year time horizon, depending on whether Big Data can get past its fragmented, cottage industry phase in that time interval.
Technology has to get very mature – even a bit boring – before it enjoys truly widespread Enterprise adoption and deployment. Big Data will get there, but it’s got to overcome several hurdles first.Andrew Brust
I am for Not even close
Great Debate Moderator
Great debate guys!
Look for closing arguments from Andrew and Larry which will be posted on Wednesday. Voting closes at 2pm ET Thursday and soon after you'll see my final verdict. Thanks for joining us!
Posted by Steve Ranger
Closing Statements
Big data projects will ramp faster-than-expected
Lawrence Dignan
Far from imminent
Andrew Brust
Companies like Cloudera, MapR and others are working on this problem, and at some point the problem will get solved. But until the numerous big data pure-play companies consolidate and work more closely with the mega-vendors, big data will remain the province of Internet companies, skunk works teams and pilot projects, rather than the mainstream technology fabric of the enterprise.
The gap will close, but that's far from imminent.
This was a finely balanced debate
Steve Ranger
Companies big and small are desperate to get more use out of all the data they’ve been squirreling away for years, which is why big data is so attractive to so many. It’s undoubtedly cutting edge technology, but I’m not convinced that all the pieces are in place, just yet, to make it a mainstream option.
CIOs are looking at big data right now but it’s only the very brave -- or those with a really pressing need -- that are making the jump right now. So Andrew takes this one, by a whisker.
Posted by Steve Ranger