Data science vs the hunch: What happens when the figures contradict your gut instinct?

Data science vs the hunch: What happens when the figures contradict your gut instinct?

Summary: Despite the widespread adoption of analytics and scientific testing by businesses around the world, the management sixth sense continues to flourish.

SHARE:
MenCorridor220x336
Many still believe company politics trumps evidence when it comes to management decisions. Image: Shutterstock

On the face of it, business people still hold wildly contradictory views about the relative value of data science as opposed to gut instinct when they make decisions.

Almost six senior executives out of 10 see themselves as making decisions based on data, according to new research. Yet more than seven out of 10 of the same group also say they trust their own intuition when choosing a course of action.

On top of that, despite most managers believing they are data-driven decision makers, 57 percent would reanalyse data that contradicted their intuition rather than trust it first time, the Economist Intelligence Unit study for predictive analytics firm APT has found.

However, the apparently conflicting positions taken by respondents to the worldwide survey are not necessarily incompatible, according to report editor Pete Swabey.

"It's not an analysis versus intuition debate. There is a role of intuition in the process of analysis, which you might think of as sense-checking or comparing with your experience, that can help you make sense of the data," he said.

He cited the example of Dan Humble, head of insights and research at pharmacy chain Alliance Boots, who contributed to the EIU-APT study.

According to Swabey, Humble has said he behaves exactly as the 57 percent does; if the data contradicts his intuition, he will reanalyse it.

"That's not to say that [Humble] does not ultimately trust data generally, but if it does contradict his intuition, then that is a possible sign that something has gone wrong with the collection, the analysis or the interpretation, or there needs to be more data to put that into context."

Having flagged up a counterintuitive result, if it ultimately turned out that the analysis had been conducted correctly, Humble would act on the data.

"Because if to your satisfaction it has been collected, analysed and prepared correctly, then what can you do but follow the recommendations of the data? But your intuition is an indicator that perhaps something is wrong," Swabey said.

"Neither should you blindly follow the data that comes out of the first analysis, because we all know the huge range of things that can go wrong in the analysis process.

"But then equally — as nobody is suggesting you should — the other side of the coin is that you should not just go on pure intuition and wander around ignoring all the evidence."

APT vice president UK Rupert Naylor said he had recently experienced the value of intuition in a would-be scientific trial into the effectiveness of sales flyers at a client company.

Having put in place a flyer campaign with a newspaper, the company examined the results coming back through APT's software and found the material had had a zero impact on retail and online sales in the trial areas.

"If there had been a 10 percent [rise in sales], they'd have said, 'Well, the flyers don't make any sense. They cost money to produce and distribute, so let's not do them'," Naylor said.

"We went back and checked the data, which was correct, the software was working correctly, so there was no problem there."

As a final measure, the company went back to the media agency involved in the research project.

"The media agency had forgotten to do the trial. So there had been no insertion and hence the test versus control performance was zero. You can see why they wanted to reanalyse," he said.

Regardless of the relative status of data science and intuition in an organisation, company politics continues to play a role, the researchers also found.

Respondents were asked the degree to which they agreed with the statement, 'Company politics trump evidence when it comes to management decisions'.

Some 44 percent agreed or strongly agreed with the statement, with 28 percent neutral and 28 percent disagreeing, report editor Pete Swabey said.

"Clearly the biggest answer is, 'Yes, company politics trumps evidence'. So that is not a great sign on that one," he said.

Nevertheless, APT's Rupert Naylor said analytics can help undermine support for internal political agendas by providing a scientific basis for decisions.

"Now whether people act on that scientific evidence — whether this initiative drove more sales or footfall and so on — is then a political thing because the loudest voice in the room could still dominate," he said.

"But by having one version of the truth what you're doing is removing one barrier to companies operating in a scientific way. Ultimately, given the pace of change and competition you have to move away from just politics."

Naylor said widening access to data within the business is another way of trumping company politics.

"In the old world, if a lot of data was being gathered and it wasn't very accessible, then people were able to cherry-pick data to support their arguments and then you could potentially, if something didn't meet your gut feel, get the right bit of data," he said.

"But in the current world you have one version of the truth. You can gather all the transaction data, say, for a company and all these external sources of data like weather and competitors and demographics.

"If you put that into a common place and make that accessible, then the debate moves from, 'Is the data real and am I cherry-picking the data?' to 'What is this telling us about this particular initiative?'. That's a trend that we're definitely seeing."

The EIU-APT study also suggests that more successful companies are better at running trials and analytics.

The researchers looked at companies that considered themselves to be growing faster than their business rivals and found that 45 percent of them said they can test outcomes based on trials.

That compares with only 10 percent for the groups that were either neutral or shrinking compared with competitors.

"The conclusion then, in the first cut analysis, is obviously that these [more successful] groups are doing tests and trials. Does correlation prove causation? I wouldn't comment."

Researchers surveyed 174 senior managers and executives from organisations around the world for the report Decisive action: How businesses make decisions and how they could do them better. More than half of the executives were C-level, and 49 percent represented organisations with over $500m in annual revenue.

More on big data and analytics

Topics: Big Data, CXO, Data Management, Enterprise Software, Business Intelligence

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

11 comments
Log in or register to join the discussion
  • The heart of the matter.

    This article gets to the heart of the "global warming debate".
    Those who are adamant that ONLY humans are the cause were running around screaming that the oceans are going to swamp the world because of human activity. Then we find out that the Antarctic ice is melting due to a completely natural long term cyclic pattern and that nothing humans do to reduce CO2 will make any statistically significant difference to the speed or amount of melting.
    So, IF, the "human caused global warming supporters" were so wrong (either deliberately purely for political gain or accidentally because of bad data) then why can't they also be wrong about other "predictions" they attribute to human activity?
    We KNOW they are doing everything possible to prevent ALL research that might prove them wrong and they have already attacked the group(s) that spilled the beans about the natural antarctic melting cycle. This makes it at least appear that the supporters of "human caused global warming" have some ulterior motive. Why do they instead attack anyone and everyone that even calls their attention to possible flaws in their data??
    For me the answer is found in their best ideas about how to stop "global warming.
    1. Tax carbon (either directly or indirectly through "cap and trade" schemes) and use the money to expand welfare programs. Those in the USA who support these want to use most of the tax money to "shield the poor from the higher prices" with programs that make more individuals more dependent upon government. (Not tax carbon and use ALL the money to fund a new "Manhattan Project" to find non-carbon forms of energy production and better forms of power distribution.)
    2. Have government make the direct decision about who can and who can't own and use private transportation. (It is one of the basic tenets of Marxism that if government controls the ability of the populace to move around, then government can prevent revolutions.)
    3. Force everyone (except the "poor") to pay for "smart" (remote controlled) utility meters so government can decide what the environmental comfort will be and if and how much of these services you can use. (Lest we forget, in Stalinist Russia, a Marxism based regime, the local party boss controlled the valves and fuse boxes in the basements of the Government apartment blocks and anyone that complained about the government found themselves shivering in the dark sometimes without even cold running water.)
    4. Ban or severely tax meat production, especially beef because livestock emit methane which is 20 times as bad a CO2. (Another page of Marxism is that if government can control what and how much food the people get, government can prevent insurrections. Look at North Korea today and Stalin's Russia in the 60”s.)
    All of these lead directly to the establishment of a one world Marxism based government.
    CutRightSharpening
    • It's all about the Money and the Power

      It's never been about what people are doing, other than the pathogenic desire to order them around.
      Dr_Zinj
    • Actually, the FACTS run the OTHER way.

      Antarctic ice would normally melt in some places and expand in others, but today it is melting much MORE than it is expanding (in fact, hardly expanding at all). The changes that would, in nature, take many thousands of years, allowing time for species to adapt, are happening in only two centuries, and at an accelerating rate. And the "money and power" are on the side of the DENIERS, unfortunately. There is no conspiracy to create a "one world Marxist" government, except in a few countries, but there is a danger that EVENTS will force such a government to be created as an EMERGENCY measure to save humanity. Progressives want to AVOID such a necessity, but the ideological blindness of those who would cater to the SHORT term profits of a wealthy few are making it harder and harder to save the world with democratic free market principles.

      Marxism is discredited in all the countries where it once ruled, except tiny ones such as Cuba, Vietnam and North Korea (even China only pays lip service to it, and Vietnam is also heading in that direction). However, SOME form of authoritarianism is developing in Western democracies: the authoritarianism of allowing those who already have the power of economics (i.e. wealth) to control the strings of what should be democratic government also, bending it to their will and denying a voice to the vast majority of middle class and poor (soon to include the former middle class) working people. One of their weapons is to DECEIVE the middle class into thinking that some outside bogey man (Marxists, Muslims, foreigners, minorities, some fictional group of "poor cheaters gaming the system"), rather than the "job creator" class, is causing their problems.
      jallan32
  • Data science and decision making

    "Data science vs the hunch: What happens when the figures contradict your gut instinct?"

    Well, this is something far, far older than data science: The human race has known about and has been able to use statistics for a long time. "Data science" is just the next logical step when we have the computing power to perform a bit more crunching on the data.

    There's always two things about data:

    1) What does the data actually say? It's very important to know what exactly has been examined. Numbers don't lie, but . . .

    2) How can the data be interpreted? This is something that humans do - interpret the data. Here is where the whole "lies, d*** lies, and statistics" comes in. Numbers don't lie, but humans are extremely good at making interpretations that just don't exist in the numbers.

    We often do things like mistake correlation with causation, or use anecdotes as a replacement for statistics. We like to make things that are theoretically possible into things that are statistically common, even when they're not.

    . . . and of course, we like to replace ethics and morality with statistics, even though that shouldn't be done. What is better for the bottom line isn't always better for society as a whole. I'm pretty sure that holding a gun to somebody's head would increase their chances of buying your products - but that doesn't mean you go around holding guns to people's heads.

    When all is said and done - data science is part of the decision making process, but it's only one part of the process. It's not a complete replacement for the entire decision making process. You should always be considering your business's goals and values, as well as the goals and values of the society you live in and the goals and values of your customer base.

    A well informed decision is far more than mere numbers and data.

    "Does correlation prove causation? I wouldn't comment."

    The answer is no. It's well-proven to be a logical fallacy. Sometimes a correlation is caused by a third factor.

    http://www.mathsisfun.com/data/correlation.html

    There is an "Ice cream vs Sunglasses" plot there. Ice cream and Sunglasses have a correlation.

    However, the causation is something not shown on the graph: Sunlight. Sunlight causes the correlation. Ice cream does not cause Sunglasses to be sold, and Sunglasses do not cause Ice Cream to be sold. Sunlight, which is not on the graph, causes both of them to be sold.

    So it is readily demonstrable that correlation does not prove causation.
    CobraA1
  • Trust but Verify

    Absolutely, it is the data that takes priority. However, sometimes the data can be interpreted incorrectly or may not be as definitive as hoped for. In which case, you should follow the data with caution and figure out what may be causing those doubts, perhaps taking steps to mitigate consequences if the instinctive reaction is correct.

    The numbers don't lie. However, most decisions are not made on the numbers but, as best, an interpretation of what the numbers mean. Caution and discretion are watchwords to be followed when a non-intuitive result comes from data interpretation. That doesn't mean that you shouldn't follow the data, though.
    slaskoske
  • The way you test out your gut...

    ...is to see if the numbers bear it out. People can formulate hypotheses in all sorts of ways, but the business of science is to formulate theories that are supported by the available evidence and that can be used to make reliable predictions.

    If your personal models really are that good, the numbers will support them. But test well as to avoid confirmation bias.

    And it's important to remember that statistical models are advisory only. It's still the responsibility of human beings to make decisions.
    John L. Ries
  • business decisions

    Aren't just about data. They are about company politics and money and competing demands for limited resources and winning over shareholders and employees and public opinion. What is best for the long term interests of a company might not be best for the Carl Icahns of the world. Every decision will have supporters and detractors and not everyone has the same agenda. Big data doesn't change human nature.
    krossbow
  • data

    73% of statistics are made up on the spot.
    apoteke
  • The real problem is that not

    everything is intuitive. I found things all the time in school in science and engineering classes that aren't necessarily intuitive. There's a fine line here. If you go over your data, methodology, and interpretation and you still don't agree, you just may be wrong.
    thewags05
  • #HPC is in the final end

    only depending on the accuracy of the deployed analysis software, supercomputers make calculations errors at very high speed . . . . especially if deployed by Economists, and such
    type GUI drones, without any thorough knowledge and practice in/of computing .
    Get ourselves involved, check, Jorge Salazar #HPC
    X15meshman
  • Gut vs Head

    Gary Jobson, a brilliant yacht racing tactician/analyst, and Dennis Conner, a brilliant seat-of-the-pants skipper worked together to win and retain the America's Cup back in the day. Jobson wrote in his book that whenever his analysis of data and Conner's instinct agreed (about half the time), they were right. When they disagreed, each was right approximately half the time. There is brilliance in both intuition and analytical logic.
    karps