Those who provide the money for research want to hear good news, so it is not surprising that many researchers are willing to see they get it. How? One way is by interpreting findings in a positive light. We see this in particular when the goal of a study is validation.
Validation is defined as the action of proving, confirming, justifying, and endorsing.
Validation is starting with the answer and ensuring the research supports what is already assumed to be the best course of action. It is not objectively testing to determine if it really is.
Another way to guarantee positive results is to select samples that are positively inclined towards the company and/or the product. One company we worked with had previously done all its new product research with the same group of diehard loyalists who always produced rave reviews for the new product-to-be. This egregious failure was completely unknown to the senior executives.
A proven way to influence research outcomes is to suppress negative findings.
An independent study published in the New England Journal of Medicine showed 97% of positive clinical trials (those that found the drug was effective) were published. When studies found the drug was not effective, only 8% of were published.
In business, even it the research report contains bad news, it is rarely seen by executives. In most companies, marketing is the gatekeeper of all consumer research. Gatekeepers cherry-pick only the research that supports the story they want to tell, and anything that doesn’t support that position is left on the cutting room floor. Ethical researchers are obligated to report all the news – good, bad, and indifferent. Marketers are not.
Executives are unlikely to ever see their company’s research. Instead, they are shown a few doctored research slides included in a flashy marketing presentation.
It is our position that validation is for parking lot receipts.