X
More Topics

Business Analytics: Do we need data scientists?

In a world where data drives everything, do we need data scientists to make sense of this tidal wave of information?
Robin Harris

Robin Harris

Yes

or

No

Andrew Brust

Andrew Brust

Best Argument: Yes

56%
44%

Audience Favored: Yes (56%)

The moderator has delivered a final verdict.

Opening Statements

The required skill set is real and uncommon

Robin Harris: Big data is a new field where we're still figuring out needs, wants and tools. It is too early to commoditize data analytics. Beyond that, the kind of mind that can do deep analysis on massive data is not common. Just as Adobe hasn't made everyone a skilled designer, easy-to-use data analysis tools - if we had them - won't make analytics accessible to everyone or even most. The combination of curiosity, domain expertise, technical understanding, statistical skill and insight will never be a commodity. The term "data scientist" may be pretentious, but the required skill set is real and uncommon.

Even in the era of hundreds of millions of handheld computers, we still have - and need - computer scientists. It's the same with big data. 

We need tools that empower knowledge workers

Andrew Brust: “Data scientist” is a title designed to be exclusive, standoffish and protective of a lucrative guild.

Big data and analytics are powerful, and the technologies around them are exciting. But if they can only be harnessed by highly-paid specialists, then they haven’t fully evolved. We need data and analytics technologies, but we shouldn’t need expensive, scarce, Shaman practitioners to use them. We don’t need data scientists, we need tools that empower knowledge workers to do big data analytics on their own.

The Rebuttal

  • Great Debate Moderator

    Opening statement

    Are you ready?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    All set

    Did my homework.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Research complete

    Let's get going.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Let's begin: The skill-level issue

    Big data requires new skills and it's fairly clear there aren't enough people that possess them today. Why do or don't we need data scientists to make sense of this tidal wave of information?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Rapid changes

    Like any new field there is rapid change going on with big data. The tools are undergoing rapid development such as the various no-SQL databases, visualization tools, and understanding what we can do with the data. Much of the current effort concerns advertising, but as Google has demonstrated with flu-tracking, we can learn much about what is happening in real time by looking at our collective online behavior.
    The easiest way to think about this is to consider the transition from batch computing to real-time interactive computing. Our old form of big data – the data warehouse – is a static batch oriented data store. The new world of big streaming data is real time and interactive: we can look at the data as it is coming in and ask many kinds of questions.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Scientists or not

    Certainly we need many more people in the workforce who have the ability to collect, query and analyze big data.  But the notion that we need “scientists” to do this, and that we could somehow produce them in high volumes, seems misplaced at best.  What we need to do is make Big Data technology accessible by the people who have the drive to analyze it and would gain the most from doing so: people close to the business, looking for ways to optimize their sales, marketing or operations.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Career change

    Can people retool themselves into data scientists?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Not everybody

    Some people can. But it won't be for everyone. There is not only the technical proficiency required because of the relative infancy of the tools, but also the requirement that the scientists have a good grasp of statistical techniques as well as knowledge of the area that is being investigated.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Skills needed

    People can certainly obtain the literacy necessary to carry out analytics on big data.  Business people can be made capable of working with the data, and developers who are not currently analytics-focused can be made capable of collecting the data and performing analytics in their code.  Of course, certain people can be trained to become very highly-skilled specialists, but that would be the exception more than the rule, and that’s OK.  We don’t need people to retool en masse into scientists, we need them to obtain a competency.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Needs

    Is the term data scientist overhyped?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Searching for a better name

    Of course it is. It isn't clear that the term scientist – with its connotation of dispassionate experiment and analysis – is appropriate to what is an analytical discipline. Data analyst is more appropriate, but because it is streaming data and the technology and the possibilities are much greater than what we've had in the past it does make sense to use a new name. Perhaps data scientist isn't the best term but it will have to do.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Beyond the hype

    Yes, it is.  But in fairness, so are the terms “big data” and “analytics” and they are still quite valid areas of specialization.  The problem with the term “data scientist” goes beyond the hype; there’s an attitude and adversarial tone to the term. This tone discourages people from obtaining analytics competency, as it transmits an implicit message that the work must be outsourced to highly-trained individuals.  Aside from the hype, it’s pretension and snobbery that make “data scientist” an unhelpful term.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Flooding the market

    Is there a risk that everyone will be come a 'data scientist' in the name of finding a better gig?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    You can't fake it

    Resume inflation has always been with us. Anytime there's a new field that pays well there is a gold rush mentality. But the skills required for data science are sufficiently arcane that most people won't be able to fake it. If you don't have at least a masters degree in a technical or analytical field then you probably won't be able to get hired.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    It's title inflation

    Of course there is, in exactly the same way that has happened with other lofty titles in technology (“architect,” for example).  Title inflation happens in any field, but in the tech field, terms and titles are in any case viewed as metaphors, more than literal descriptions.  Tech folks tend to take poetic license with titles, and those who don’t do so find themselves at a disadvantage compared to those who do.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    The UI question

    Where does user interface fit in with the data scientist debate? In other words, is there a UI that makes the wonks irrelevant because even I can figure out the trends ahead?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Building on the past

    No doubt the tools for streaming data analytics will improve a great deal in the next 10 years. But the value of any analysis is not that it tells everyone the same thing. The most valuable analyses tell us more than what everyone else already knows. That is the value that top data scientists will add.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    It's evolving

    I don’t think it’s about a user interface per se, but, yes, it’s about tooling.  Analytics in general, and big data specifically, have terribly immature tooling compared to mainstream relational database and BI products.  That being the case, it’s no wonder that only “scientists” can get real work done…these tools were built for laboratory use, not business use.  Just as self-service BI is in vogue (and is legitimately quite powerful) today, so too should self-service big data and predictive analytics be a market phenomenon.  Once it is, people with the skills that we classify under data science today will still have a role, but it won’t be nearly so central as it is now.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Supply and demand

    Does the U.S. education system have enough heft to create a supply of data scientist?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Developing minds

    The question assumes that we know what we need to train data scientists. The great strength of the American educational system is not that the average education level is high - it isn't - but that the highest levels  produces exceptionally flexible and inventive minds. And, of course, our freedom and quality of American life draw the best and the brightest from much of the rest of the world.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Education system is failing

    The U.S. education system needs to do a much better job in instilling math and science skills in its graduates, especially among women – the gender imbalance is horrific.  We need better math, science and technology literacy overall, regardless of whether grooming data scientists is important.  And, yes, the U.S. education system can do it.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Talent show

    Where will the data scientist talent reside in the world?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Cultural understanding

    The very best analysts will bring a deeper cultural understanding to their analysis and interpretation. Interpreting the behavior of American consumers will not be outsourced to China or India anytime soon. But sensor-based data produced by, for example, utilities may not require a cultural component. In that case it may be feasible to outsource the work.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    The world market

    Countries that value and impart knowledge of mathematics and science, combined with business, will produce the best data professionals.  India seems to be in a very good position here.  The United States, which is home to a number of preeminent colleges and universities, will produce its share of data professionals as well, though they may hail from, and return to, other countries.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Language barrier

    Do we really need more data scientists or more data literacy and business and tech people who are bilingual?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Knowledge is power

    Let's be clear: we are inventing the future. There is no blueprint that tells us how big data and data science are going to change how we manage and improve our world.
    Knowledge of the practical uses of big data will inevitably diffuse into the managerial and professional ranks. But the sharp end of the stick is going to be the pioneers who are doing the work today to understand the world of big data.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Let's keep it simple

    It won’t come as a surprise that I believe the latter scenario to be the one that will solve the labor issues we face most successfully.  Data science is about having a command of both data technology and business domain expertise.  If the technology becomes simple, and business people become more adept with it, then business users can be bona fide analytics professionals.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Build the perfect analytics wonk

    If you were to build your perfect business analytics wonk what skills would they possess?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    The job description

    My ideal candidate would have a solid understanding of the underlying information technology required to gather and store the data; a strong understanding of statistical and Bayesian analysis; excellent communication skills; and the ability to develop or at least specify improvements to tools.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Sales skills included

    Well, if I had my druthers, it would be a sales, marketing or planning professional who was also a tech power user, had a command of statistics, knew Excel very well and could do some light programming.  But that’s an ideal…and in order for analytics technology to take off, we shouldn’t need people to fit this ideal in order to be productive Big Data analysts

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Becoming obsolete

    Can the role of the data scientist ultimately be automated with algorithms?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Not now

    Not until the field is much better understood. Even then, the role of expert judgment, intuition and insight will be very difficult to reduce to an algorithm.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Experience necessary

    No.  Implicit in the definition of data scientist is possession of business intuition and instinct that mere algorithms can’t replace.  If you accept that the term is legitimate, then you accept that a combination of human intelligence and technology expertise is what makes someone an authentic data scientist.  While I’m not a huge fan of the term data scientist, I do feel the experience of the business user and her non-algorithmic intimacy with the semantics of the data is very important.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    If data scientists take over...

    If data scientists take over business what happens to the decisions based on gut feel?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    It's all about the bottom line


    This isn't Skynet. Big data science is a new universe of tools and possibilities based on the aggregation and real-time analysis of enormous amounts of data. The insights will be deeper, but applying them to the real world problem of making money and improving collective outcomes will require more than analysis. It will require judgment, insight and deep domain expertise.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Gut feelings

    Again, the definitions of data scientist that I have collected say that such gut feel in combination with technical expertise are core requirements.  So, by definition, gut feel, validated (not replaced) by data, stays front-and-center.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    The future role of analytics

    Should data scientist roles be proven to be unnecessary, what analytics roles are most important?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Too early to tell

    We are at least two decades away from being able to answer that question. Why? Because today we are just barely scratching the surface of what big data and data science can tell us. Just look at how gene sequencing– as pure an example of big data as we have yet seen - has changed our understanding of biology. We don't know what we don't know. The next decade will be a great voyage of discovery.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    Here is what's important

    Expertise in data exploration and visualization tools, programming/developer skills, an understanding of statistics, and high-level database design skills will remain important, regardless of whether the data scientist role remains in vogue.  Equally important will be a deep understanding of the business, and the data sources that measure its activity and outcomes.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    Last question: The five-year plan

    In five years will the data scientist run be a boom or bust?

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

    Let's move forward

    We are currently running up the hype cycle with data scientist and, yes, there will be a deflation in the term and the field as early overblown expectations are not met. But over time, those with the determination and resources will discover the real advantages of data science and will move us all forward.
    I am reminded that back in the late 70s and early 80s people were struggling to figure out what to do with the home computer. Store recipes? Produce little bar charts?
    Few could imagine how the home computer would grow into a tool for individual production and consumption of information, knowledge and imagination. So shall it be with big data and big data science.

    robin-harris-60x45.jpg

    Robin Harris

    I am for Yes

    They will phased out

    The term will subside and may well sound dated five years from now.  The skills will become more commonplace and commoditized.  When that happens, the real boom will begin, because the technology will become widely adopted and thus more useful.  But for the relatively small club of people clinging to a data scientist identity and pay scale, it may seem like a bust.

    andrew-brust-60x45.jpg

    Andrew Brust

    I am for No

  • Great Debate Moderator

    It was a Great Debate

    Here's a special thanks to Robin, Matt and all of you who have joined us. Next up will be the closing arguments which will be posted on Wednesday. My choice for winner will be posted on Thursday.

    lawrence-dignan.jpg

    Posted by Lawrence Dignan

Closing Statements

Cutting edge requires cutting edge skills

robin-harris.jpg

Robin Harris

People are all in favor of change except for the change part. The change here is that big data requires new skills and new ideas to fully exploit it.

Whether we embrace this or fear it probably has more to do with our individual natures than the fact of big data and data science. The name we give these people is much less important than the societal value of their skills.

Perhaps, someday, the Watson Mk. XXXXII will be able to ingest huge data sets and make sense of them. But until then the cutting edge of big data will require people with a wide skill set to mine the economic value of big data.

 

Specialists don't scale

andrew-brust.jpg

Andrew Brust

Big data technology is powerful, and it keeps getting better. But the technology does, right now, require niche specialists to derive the greatest business value from it. These specialists have to be renaissance people – possessing a combination of technology, mathematics and business skills, and knowledge. It’s not clear that being so clever and versatile makes these specialists into “scientists,” but it does make them rarefied.

Nonetheless, for big data and analytics implementations to grow and become truly mainstream, having such diverse skill set requirements for them is not a sustainable situation. Market need is going to drive evolution in the technology such that the barrier to entry will not be nearly so high as it is now. If for some reason that didn’t happen, then adept use of big data would continue to be an option open only to a relatively small group of customers.
So the solution to our problem isn’t legions of new data scientists. Instead, we need self-service tools that empower smart and tenacious business people to perform big data analysis themselves. The specialists will still have an important role, but they won’t be the thing that scales big data across industries.

Big data for the masses

lawrence-dignan-150x105.jpg

Lawrence Dignan

This argument boiled down to timelines in many ways. Today, we need more data scientists, but in the long run big data has to be for the masses of workers.

Specialists only go so far, argued Andrew Brust. Robin Harris argued that we need more data scientists. In the end, Brust had a better argument. Data scientists are important, but you may not need an army of them to boost the analytics revolution.

Ultimately, we're going to need data scientists, innovative user interface and a lot more data literacy from average workers.

Editorial standards