Explaining complicated subject matter simply since 1986
Two slick-talking tailors promised to make a suit of clothes that was so special it would be completely invisible to stupid and incompetent people. Word got around and soon the whole town knew about the strange power of this new suit. Everyone was impatient to find out who among them were so stupid as to say it was invisible. The emperor who ordered the special suit dispatched his most trusted ministers to see how it was coming along. The con men, working away at their empty looms, explained the fabric they were weaving in extravagant detail, declaring it Captivating! Breathtaking! Enchanting!
The ministers feared if they admitted they saw nothing, the emperor would think them stupid and incompetent. So they went back to their sire and told him the fabric, the colors, and the design were splendid and glorious. When his new suit was delivered, the emperor was stunned that he could not see anything. It would never do for the emperor to be stupid and incompetent, so he gave the suit his highest praise. None in his entourage could see the suit, but all joined the emperor in praising its wonderfulness. After having him undress, the swindlers elaborately dressed him in his new “clothes.” After much ooing and aahing by the con men and the courtiers, he took his place in a great procession through the town so everyone could admire his finery.None of the people watching the parade wanted to appear stupid and incompetent, either, so each praised the new clothes more loudly than the next – Magnificent! Majestic! Spectacular! – until a small child shouted, “But he doesn’t have anything on!” <Hans Christian Andersen wrote this satirical tale to expose hypocrisy and snobbery. It is popular with children because the child is the only one who has the courage to speak the truth.>
That was one of the first lessons I learned when I was a mechanic when I was a kid and it seems few in white collar jobs know this to be of critical importance. Businesses throw away billions of dollars every year because they use a clock to measure the size and weight of a chair. Tradespeople have known for centuries how important it is to use the right tool for the right job. Too bad the people who spend billions on focus groups don’t know that simple truth.
Nowhere is it more evident than when people are asked to come to a strange place so a stranger can ask them to reveal their innermost feelings to a roomful of strangers – for money. The entire research industry knows what people say and do are seldom the same thing. This is why userfocus.co.uk asks “So why does your company expect focus group participants to:
In groups, people tend to say what they think will be viewed favorably by others. This includes under-reporting “bad” things and over-reporting “good” things. Here’s one of my favorite illustrations of how focus group conclusions can be so out of whack with the scientific evidence.
Some tricksters like to invite only people with healthy eating habits. This naturally pre-ordains the outcome by stacking the deck. You’d be surprised how often this happens among bottom-feeders.
Some focus group participants need time to reflect, but moderators move on to the next question before some group members have anything to say. Groupthink is a part of every focus group. Less outgoing participants tend to agree with outspoken and assertive alpha participants. Recruitment is often hit or miss. Less-scrupulous vendors cut corners here, and too many of the people in your group are the wrong people to be asking about your products and services.
Terry Linhart of Arbor Research Group often politely asks clients if they really want to know the facts or if they just want the research to confirm the genius of what they’re already doing, saying it semi-jokingly, making the point nicely. When I met clients who clearly wanted confirmation and not exploration, I would tell them we could find out exactly what they wanted for a large fee, or for half that amount, we could find out what actually exists. About half opted for paying double to hear what they wanted to hear.
Robert Merton, the inventor of the focus group, despised how his methodology had been hijacked and bastardized, saying “Focus groups are supposed to be merely the source of ideas that need to be researched.” They were meant to be only a jumping off point and a general direction, with perhaps a hypothesis or two thrown in. Instead, focus groups became a favorite way to get quick answers by taking a fatal shortcut.
People have complex and conflicting motivations. These are impossible to get at with those ridiculously detailed discussion guides so many moderators rush through in a vain attempt to meet stakeholders’ conflicting demands. Focus groups also ask participants to make snap judgments about products they’ve never seen or used.
In his book How Customers Think, Gerald Zaltman says 80% of new products fail within six months of being vetted in focus groups. Why the lousy results? According to Zaltman, “The correlation between stated intent and actual intent is usually low and negative.”
No matter which, you should immediately seek new gatekeepers and providers because the ones you have are squandering your resources. Think of them as weavers of special clothes for you, the emperor.
Focus groups are best for:
Smarter businesses have moved on to better ways of conducting qualitative research, replacing focus groups with ethnographies, on-site observations, one-on-one and small group interviews, and a host of hybrid methodologies.
I can tell you where you are doing well, where you need improvement, and where you need to be doing something entirely different.
Decision-makers don’t need to be research experts to upgrade your organization’s knowledge-management IQ. In two-day workshops I show them: