Explaining complicated subject matter simply since 1986
Two slick-talking tailors promised to make a suit of clothes that was so special it would be completely invisible to stupid and incompetent people. The emperor ordered a special suit and later dispatched his most trusted ministers to see how it was coming along. The con men, working away at their empty looms, explained the fabric they were weaving in extravagant detail, declaring it Captivating! Breathtaking! Enchanting! The ministers, fearful of being exposed as idiots went back to their sire and told him the fabric, the colors and the design were splendid and glorious.
When his new suit was delivered, the emperor was stunned that he could not see anything. It would never do for the emperor to be stupid and incompetent, so he gave the suit his highest praise. None in his entourage could see a suit, either, but all joined the emperor in praising its wonderfulness. After having him undress, the swindlers elaborately dressed him in his new “clothes.” After much ooing and aahing by the con men and the courtiers, he took his place in a great procession through the town so everyone could admire his finery. None of the people watching the parade wanted to appear stupid and incompetent, either, so each praised the new clothes more loudly than the next – Magnificent! Majestic! Spectacular! – until a small child shouted, “But he doesn’t have anything on!”
Hans Christian Andersen wrote this satirical tale to expose hypocrisy and snobbery. It is popular with children because the child is the only one who has the courage to speak the truth.
That was one of the first lessons I learned when I was a mechanic and it seems few in white collar jobs know how important it is. Businesses throw away billions of dollars every year because they use clocks to measure the size and weight of chairs and screwdrivers to pound nails. Tradespeople have known for centuries how important it is to use the right tool for the right job. Too bad the people who spend billions on focus groups don’t know that simple truth.
Many vendors stack the deck in your favor. They want their client to be happy and know that will come only when the research validates what they already believe. If you’re bringing out a new line of healthy foods, they recruit people who eat only healthy foods. This naturally pre-ordains the outcome so clients can show their bosses how study subjects are gaga for all their new products.
In groups, people tend to say things they think will be viewed favorably by others. This includes under-reporting “bad” things and over-reporting “good” things. Here’s one of my favorite illustrations of how focus group conclusions can be so out of whack with the scientific evidence.
Nowhere is it more evident than when people are paid to come to a strange place so a stranger can ask them to reveal their innermost feelings to a roomful of strangers..
This is why userfocus.co.uk asks “Why does your company expect focus group participants to:
The more corners vendors can cut, the more goes to profit. The result is that too many of the people in your focus group are the wrong people to be asking about your products and services.
Less-scrupulous vendors employ ringers. These are people who research firms hire over and over again because they can be counted on to be positive, persuasive and follow instructions. If they were boxers, they’d be the pugs willing to throw the fight.
Less-outgoing participants tend to agree with outspoken and assertive alpha participants so paid ringers steer the group in predetermined directions with or without the help of the moderator.
Many companies that pay for focus groups don’t want to hear anything negative. They want to hear participants tell them how wonderful their products and services are. They are seeking validation and lots of research companies compete with each other to get their business.
Terry Linhart of Arbor Research Group often politely asks clients if they really want to know the facts or if they just want the research to confirm the genius of what they’re already doing. When I met clients who clearly wanted confirmation and not exploration, I would tell them we could find out exactly what they wanted for a large fee, or for half that amount, we could find out what actually exists.
Robert Merton, the inventor of the focus group, despised how his methodology had been hijacked and bastardized, saying “Focus groups are supposed to be merely the source of ideas that need to be researched.”
Focus groups were meant to be only a jumping-off point and a general direction, with perhaps a hypothesis or two thrown in. Instead, focus groups became a favorite way to get quick answers by taking fatal shortcuts.
People have complex and conflicting motivations. These are impossible to get at with the ridiculously-detailed discussion guides so many moderators rush through in a vain attempt to meet stakeholders’ conflicting demands. Focus groups also ask participants to make snap judgments about products they’ve never seen or used.
Why the lousy results? According to Zaltman, “The correlation between stated intent and actual intent is usually low and negative.”
They’re either withholding crucial information from you, which is bad or they’re unfamiliar with the facts, which is also bad.
No matter which, you should immediately seek new gatekeepers and vendors because they’re selling you one magnificent, majestic, spectacular suit of clothes after another.
To make absolutely clear to our clients what focus groups are good for, we developed this definition: “Focus groups are exploratory research we use to guide our survey research.”
Smarter businesses have moved on to better ways of conducting qualitative research, replacing focus groups with ethnographies, on-site observations, one-on-one and small group interviews and a host of hybrid methodologies.
Find someone to tell you where your research is top-caliber, where you need to fine-tune and where you need to be doing something entirely different.
Decision-makers don’t need to be research experts to upgrade their knowledge-management IQ. All they need to learn are these five things:
Click here for the home page where you can scroll through the articles I’ve written every week for LetsTakeACloserLook.com since 2016, always showing there is more to most things than meets the eye. Scroll down to to the bottom of this page, enter your email address and get a new article in your inbox every Monday. There is no charge, I don’t share your address with anyone, I send you no e-mails, there are no ads on the site and I reply personally to everyone who sends me a note.