Market research fights for relevancy in the era of big data

Steve Jobs set the market research industry back years because he was so disdainful of it.'

We may have entered the era of big data and analytics, but surprisingly few companies are basing their decisions on solid market research. There's an erroneous perception that big data by itself can provide all the answers.
Photo: HubSpot

As explained in the latest edition of Knowledge@Wharton, some corporate leaders think hard-core market research is on the wane, being usurped by the big data surging through enterprises. Certainly, market research has received its share of knocks. The most legendary story is around the development of 3M's Post-It Note in the 1970s, which market research initially showed would be a dismal failure. The most recent example of flawed market research -- or simply a lack of enough of it -- was 3D television. Despite all the fanfare and investments from companies in recent years, 3D TV has fizzled and has been put on hold.

Peter Fader, marketing professor at UPenn Wharton, says there's been a tendency to pooh-pooh market research techniques in recent times. Many saw the success of Steve Jobs' Apple, which propelled itself through markets by sheer force of personality, backed by top-notch engineering talent. "Steve Jobs set the market research industry back years because he was so disdainful of it," he says. "Jobs said, 'I am going to tell people what they want. And unfortunately, he was right, or maybe he was just lucky . . ."

Indeed, some experts and analysts say market research is a trap entrepreneurs and visionary business leaders need to avoid if they are to build innovative enterprises. At the same time, it is a vital tool for "holding up the light so we can see an otherwise dark world," says Fader.

What happens more and more now, Fader notes, is people say, "Let’s put out our concepts and see what gets clicked on the most." He adds that testing products this way may help pick a winner, but not necessarily the best or most well-designed product possible.

Brian Tarran, writing in Research-Live, says executives are increasingly becoming enamored with big data analytics, while questioning their expenditures on market research. However, big data and market research go together -- if anything, "big data is another tool in the market research repertoire."  As companies work with large volumes of transactional data, there will still be a need for market researchers to make sense of the information, and provide guidance to organizations.

"Nothing beats knowing why people make the choices they do," adds Shawndra Hill, management professor at Wharton. Big data and market research "can work together to find patterns, then market researchers test these hypotheses."

This post was originally published on