Thousands of businesses threw away their money last year because they used focus groups for some very bad reasons.
- Most wanted to validate decisions they’d already made. Focus groups are great for this, as participants can be easily steered to provide the answers you’re looking for. All you have to do is have a cooperative moderator, stack the deck with leading questions, and make it absolutely clear that you are not only looking for agreement, but for group consensus.
- Too many treated focus groups as quantitative data. Nothing is more absurd than counting qualitative responses as if they were part of a statistically valid sample of randomly selected participants.
- Some did groups because they liked the voyeuristic feel of sitting in the dark behind a mirror eavesdropping on the conversations of others. Erika Hall of Mule Design calls focus groups “useless research theatre.” In her article, Focus Groups Are Worthless, she says “No one buys shoes or cooks dinner sitting at a table under fluorescent lights while engaged in a moderated group discussion.”
- They all believed focus groups provided answers, when the real purpose of focus groups has always been to generate more questions.
Most of us learned the hard way how what people say is not always what they do.
Nowhere is it more evident than when people are asked to come to a strange place so a stranger can ask them to reveal their innermost feelings to a roomful of strangers – for money.
The entire research industry knows what people say and do are seldom the same thing. This is why userfocus.co.uk asks “So why does your company expect focus group participants to inspect their cognitive machinery, understand what they find there, translate that into language, and then articulate their thoughts unambiguously?”
Social desirability is always an issue. In groups, people tend to say what they think will be viewed favorably by others. This includes under-reporting “bad” things and over-reporting “good” things.
Here’s a simple example. Fact One: People in focus groups claim to have healthy eating habits. Fact Two: Companies sell billions of dollars in unhealthy foods every year. Fact Three: Two-thirds of Americans are overweight.
Not every group member is a well-informed, insightful, and outspoken person. Some focus group participants need time to reflect, but moderators move on to the next question before responses that are well thought through have time to emerge.
Groupthink is a part of every focus group. Less outgoing participants tend to agree with outspoken and assertive alpha participants.
Recruitment is often hit or miss. Less-scrupulous vendors cut corners here, and too many of the people in your group are the wrong people to be asking about your products and services.
Many companies that pay for focus groups want to hear participants tell them how wonderful their products and services are.
Terry Linhart of Arbor Research Group often politely asks clients if they really want to know the facts or if they just want the research to confirm the genius of what they’re already doing, saying it semi-jokingly, making the point nicely.
When I met clients who clearly wanted confirmation and not exploration, I would tell them we could find out exactly what they wanted for a large fee, or for half that amount, we could find out what actually exists. Your choice.
Two slick-talking tailors promised to make a suit of clothes that was so special it would be completely invisible to stupid and incompetent people.
Word got around and soon the whole town knew about the strange power of this new suit. Everyone was impatient to find out who among them were so stupid as to say it was invisible.
The emperor who ordered the special suit dispatched his most trusted ministers to see how it was coming along. The con men, working away at their empty looms, explained the fabric they were weaving in extravagant detail, declaring it Captivating! Breathtaking! Enchanting!
The ministers feared if they admitted they saw nothing, the emperor would think them stupid and incompetent. So they went back to their sire and told him the fabric, the colors, and the design were splendid and glorious. When his new suit was delivered, the emperor was stunned that he could not see anything. It would never do for the emperor to be stupid and incompetent, so he gave the suit his highest praise.
None in his entourage could see the suit, but all joined the emperor in praising its wonderfulness.
After having him undress, the swindlers elaborately dressed him in his new “clothes.” After much ooing and aahing by the con men and the courtiers, he took his place in a great procession through the town so everyone could admire his finery.
None of the people watching the parade wanted to appear stupid and incompetent, either, so each praised the new clothes more loudly than the next – Magnificent! Majestic! Spectacular! – until a small child shouted, “But he doesn’t have anything on!”
Hans Christian Andersen wrote this satirical tale to expose hypocrisy and snobbery. It is popular with children because the child is the only one who has the courage to speak the truth.
I’m not saying don’t use focus groups. I am saying don’t use focus groups for the wrong reasons.
Robert Merton, the inventor of the focus group, despised how his methodology had been hijacked and bastardized, saying “Focus groups are supposed to be merely the source of ideas that need to be researched.”
They were meant to be only a jumping off point and a general direction, with perhaps a hypothesis or two thrown in. Instead, focus groups became a favorite way to get quick answers by taking a fatal shortcut.
Advertising genius David Ogilvy liked to say that “People don’t think what they feel, don’t say what they think, and don’t do what they say.”
Daniel Gross, author of Lies, Damn Lies, and Focus Groups, says “the widely documented mismatch between what people say and what they do costs companies millions.”
Analyst Kay Polit says focus groups are “a faulty process.” Mary Quinlan says they are “a dangerous way to get market intelligence.” Dev Patnaik calls focus groups “a customer terrarium,” with people taken out of their natural surroundings and put behind glass.
Joseph Pine and James Gilmore, authors of The Experience Economy, call focus groups “the great lie.” They say focus groups should be dumped and real customers observed in real settings instead.
Focus groups do not tap into true feelings, regardless of what someone may have told you.
People have complex and conflicting motivations. These are impossible to get at with those ridiculously detailed discussion guides so many moderators rush through in a vain attempt to meet stakeholders’ conflicting demands.
Focus groups ask participants to make snap judgments about products they’ve never seen or used.
In his book How Customers Think, Gerald Zaltman says 80% of new products fail within six months of being vetted in focus groups. Why the lousy results? According to Zaltman, “The correlation between stated intent and actual intent is usually low and negative.”
If your gatekeepers and research providers haven’t told you these essential truths about focus groups, they’re either withholding crucial information from you, which is not good, or they’re unfamiliar with the facts, which is also not good.
No matter which, you should immediately seek new gatekeepers and providers because the ones you have are squandering your resources. Think of them as weavers of special clothes for emperors.
Want to screw up your focus groups? Try these proven techniques.
- Overbuild your discussion guide. Instead of identifying some broad areas to explore thoroughly, go ahead and produce a rat’s nest of unconnected issues so long that a group can’t possibly give them any careful consideration.
- Use complicit moderators skilled at transmogrifying roundtable discussions into camouflaged sales sessions that flatter your business.
- Produce faux quant. Ask participants to vote on things. Tally responses so you can later claim that 75% love our idea.
- Avoid giving participants the opportunity to think carefully. Pressure them to provide answers that fit your questions. Keep on pounding at what you want to hear. Listen to only the praise and ignore all the criticisms.
- Send in notes. This not only interrupts the flow of the discussion, but also reminds participants that they are being watched.
- And by all means, use them to validate your decisions instead of asking questions and generating ideas.
To make absolutely clear what focus groups are good for, we developed this definition: “Focus groups are exploratory research we use to guide our survey research.”
Focus groups are best for:
- Learning what regular people really want to talk about instead of what industry experts want to talk about.
- Uncovering the real questions and issues.
- Identifying not only commonality, but variance, too.
- Writing better questionnaires.
In a long-ago focus group study of ours that became an MBA case study, we found bottled water consumers cared little about the things industry specialists held so dearly. To the experts, distilled, purified, and ionized were important distinctions that meant purity. To regular people, those words were nothing more than contrivances to avoid saying “treated tap water.” Regular people defined purity as the absence of visual imperfections.
Try using a ruler to measure room temperature, or a clock to measure the weight of a chair.
Smarter businesses have moved on to better ways of conducting qualitative research, replacing focus groups with ethnographies, on-site observations, one-on-one and small group interviews, and a host of hybrid methodologies.