The department chairman said to be fair to all students, he would start our postgraduate statistics course from the very beginning, as if none of us had any prior knowledge at all. With my heavy quant background, I figured this course would be a breeze. Forty-five minutes later, Elton Jackson had blown past everything I knew about stats and continued on with a full head of steam for the rest of the term. Welcome to the Indiana University doctoral program, where in less than an hour, I learned how little I really knew and how much I had yet to learn. As Nobel prize winner Daniel Kahneman tells us, “We’re blind to our blindness. We have very little idea of how little we know.”

Executives are shocked when I say most business research is riddled with errors as glaringly obvious to me as warts on a witch’s face are to you. There are two ways these too-common mistakes happen. 

• Accidental errors are made by people who don’t know what they’re doing. Imagine how often police must hear the boneheaded excuse, I didn’t know the gun was loaded. Few (if any) people in your employ fully understand the complex science that underpins quantitative research. This includes growing numbers of researchers and the gatekeepers who oversee them.
• Deliberate errors are made by people who have been ordered to cut corners. Most vendors gladly dilute the quality of the work to win the job. To them, making the sale is more important than providing full and honest disclosure of the many ways information quality suffers when research takes fast and cheap shortcuts.

Neither type of error is acceptable; both are your responsibility.

Your gatekeepers assume your researchers are professional-grade but they’re not, and the consequences are dreadful. Ever-increasing pressure for faster and cheaper means more and more of your research is corner-cutting stuff done by people who aren’t even well-meaning. Like so many executives, you end up paying Ethan Allen prices for IKEA furniture.

Credentialed researchers know better than anyone else the many imperfections inherent in all studies of human behavior. People are harder to study than lab sciences, especially getting them to lie still under the microscope. Recognizing all research is flawed leads us to realize we must be especially rigorous about the bits we are able to control.

Overspending, underspending, and throwing money away.

• Unless someone taught you how to conceptualize and scope research projects, you are probably overspending. A client asked me to look at the study that cost him $1 million every year. We showed him wart-by-wart why his prefab study was of so little use and so overpriced. The study we purpose-built for him provided vastly better information, and at half the cost.
• You could be underspending, like the new client who said here’s a million dollars, what can you do with it? Retire, I said, then suggested instead of starting with an arbitrary figure, let’s talk about what sorts of research you and your direct reports have planned or pipelined for strategy, brand, segmentation, new product development, advertising, customer satisfaction, et al. As it turned out, fulfilling the research wish lists of all her gatekeepers would cost her millions more. How common is underspending? Last year in the States alone, $900 billion was spent on marketing while only $7 billion was spent on research.
• You might be throwing your money away. A middle manager asked us to collect and analyze data for an annual study using their existing questionnaire. I asked what she wanted to know and how she want to be able to use the information. She was taken aback because she had never been asked these two most important research questions and her boss had never mentioned them. For years she had been dutifully completing a gofer task by holding a meeting, setting a calendar, and submitting a purchase order. 

The study they had been doing for years had more warts than a pondful of toads.

The manager was smart enough to grasp every shortcoming as I walked her through the whats and whys. She understood how if she wanted a study to deliver the information the company needed, it would have to be designed for that purpose. She saw how the now-obvious misalignment of goals and tools meant we would have to start all over again. Nervous now, she finally revealed the real goal (not the official one) was to provide the deliberately exaggerated numbers her boss wanted to show to his boss. You have to choose between two things, I said, you have to choose between goodness and badness. Either you put an end to this useless study and build the one your company needs or you knowingly collect crap for your boss’ slide.

She said she was deeply embarrassed to have to admit it, but replicating the useless one was her only choice because she was fearful of what would happen to her if she told her boss the truth. Unknown to her boss’ boss (the officer her boss wanted to impress), this senior gatekeeper had created a culture where his junior gatekeepers would rather throw away hundreds of thousands of dollars than speak up about serious issues arising on their watch.

Executives rarely believe anything this terrible could happen in their organization.

Few realize how in most well-regarded multi-billion-dollar corporations, research is buried so deeply in the bowels of the ship that it is never seen by the captain on the bridge. If you are a leader who knows bad things can happen – especially when they’re out of sight and mind – you can gain a competitive advantage by learning which of the gatekeepers now making your organization’s research decisions are overspending, which are underspending, and which are throwing your money away. 

Ask me about how I can help you with wart removal. Here’s my number: 800-652-7595. My email is david@davidallanvan.com.

Hey, managers and gatekeepers! Want to stand out at work? Show this to the boss. Want to lie low and go unnoticed? Don’t.

Take a Closer Look, Volume 2, is free to Kindle Unlimited customers. The best way to protect yourself against the manipulations, distortions and fabrications that are more and more prevalent these days is to learn how to see through them.