Traditional market research is dying!
This is a bold statement, but consider the fact that the art of market research, conceptualized in the 1930’s, has failed to keep up the rapid pace of change in technology. When created, we thought by asking people on a scale of 1 to 5, 7 or 10, to rate their preference would be sufficient in judging their thoughts and perceptions. Take the Likert Scale, created by Rensis Likert in 1932 – a bipolar agreement scale where high numbers indicate preference and low numbers indicate a lack of preference. This worked for many years.
Using traditional techniques such as a written or, in the last 20 years, an online survey, this approach worked. And hence, traditional market research has not significantly changed for the better part of the last 90 years.
But what Likert could not foresee, especially in the last 10 years, is there has been a sea-change in technology that has driven people to fundamentally evolve the way they think about preferences. In short, the advent of the emoji – a simple happy to sad face that expresses peoples’ true emotion, rather than a numeric evaluation, of how they feel.
In addition, technology now enables the presentation of video, images, statements, and other approaches that characterize or illustrate attributes that never existed before.
Bias In Response Scales
Recent research has now shown that responses to scales can vary by culture, region, or even time of day. For example, in late 2012, Gallup reported findings from an annual survey of nearly 150,000 people in 148 countries. In short there are 3 response styles:
Extreme Response: This is the tendency for a respondent to select the extreme ratings on a scale. For a 1-10 scale, this generally means 1-4 and 9-10;
Midpoint Response: This is the tendency for a respondent to select the middle ratings on a scale. For a 1-10 scale, this generally means 5-8; and,
Acquiescence: This is the inclination to agree with the interviewer/survey sponsor, or to respond in such a way that will “please” them.
Gallup found many differences between cultures. For example, Singaporeans tend to be the least emotional and least positive of all nationalities. Other countries in the bottom 10, with respect to positive emotions reported, included Iraq, Syria, Afghanistan and Haiti. In contrast, in Latin American markets (primarily Brazil and Mexico), respondents are likely to adopt an Extreme Response style. For the US, respondents have a fairly even distribution response style. Australian respondents provide some of the lower aggregate scores of any country, primarily due to very low acquiescence bias. Chinese respondents tend to have a slightly more even distribution of scores on the rating scale, i.e. using all parts of the scale. But for South East Asia, respondents in Japan and Korea adopt very different response styles from those in China. Though exhibiting a mid-to-high acquiescence bias, respondents in these countries focus heavily on the midpoint ratings (5-8), which when aggregated, leads to lower average scores than other countries.
How can you truly interpret research on a Likert scale with such a wide range of response bias? Bottom line, it’s very difficult to normalize such cultural biases. And while this research was conducted by Gallup on an international scale, imagine the cultural nuances on a local level. We are likely to see variances across regions, states, cities, and so on, simply based on where a person is from or how they were raised.
Everyone Does It The Same Way
Think about it, every online company has adopted the traditional approach to market research, from Amazon, OpenTable, Uber, Apple, Netflix, and the list goes on and on. Or how about online healthcare sites that try to evaluate physicians, procedures, and doctors, such as Healthgrades or Vitals. All use the old school style of a 5-point star rating. Let’s face it, it is very difficult to truly determine what the star rating means. You can read a 3-star rating which gives a positive rating, save for one small issue, or a 5-star rating full of caveats of reasons why they gave a 5-star but they qualify on their rating. So how can we as consumers of rating even begin to digest the scores and understand what is truth and what is fallacy.
I have dedicated the last 30 years of my career to market research, learning from the best research teachers at the University of Michigan and Northwestern’s Kellogg School of Business. Like every other market researcher I know, we took the fundamental principles created nearly 100 years ago as standard lore. There is a right way and a wrong way to do market research. And hence, we create surveys in the same way our ancestors did 5 score ago (yes, I had to use an Abraham Lincoln reference).
But then I started thinking, what if our founders of market research were wrong, or probably better said, dated. After my firm studied over 50,000 responses in varying studies we have conducted in the last 15 years we began to see a pattern that was not only curious, but alarming.
...so stay tuned for The Death Of Traditional Market Research - Part 2 of 5
Want to receive future articles and blog posts? Interested in learning more?