Two criteria for establishing research quality are validity and reliability. Most of us know what these terms mean in daily life, but few of us know what they really mean when it comes to evaluating the credibility of study findings we read about in reports or in the news. For example, many of us learned in a statistics class that validity is the extent to which a study measures what it is supposed to measure. But they didn’t tell us enough about it. Most of us only learned about face validity, which is whether or not something appears to be true. You may know its cousin, the Sniff Test.

As it turns out, we were wading in the shallow end of the validity pool.

At the deep end, where the experts swim, are criterion-, content-, construct-, predictive-, concurrent-, convergent-, and discriminant- validity, each of which can make or break a study. Most businesspeople have never heard of any of these concepts. This is why we like to avoid insiders’ language and get right to measurements we do understand: trustworthiness and confidence.

When it comes to having trust and confidence in research, we need to look no farther than the houses built by each of the Three Little Pigs.

• A third of all studies conducted by and for businesses are made of Straw, the cheapest, quickest, and flimsiest of the Three Little Pigs’ houses. Straws research is worse than having none at all. How can this be? Because bad research provides decision-makers with a false sense of confidence while sending them off in the wrong direction.
• Another third are made of Sticks, which are better than Straws, but not much. Sticks research provide some good information some of the time, but facts and figures are not consistently trustworthy. Because much Sticks research is built on assumptions,  information is only of incremental value, and most of us can’t tell the good from the bad.
• Only one study in three is made of Bricks, the only one that withstood the rigors of serious testing. Not only are the materials stronger, but the construction processes are also more rigorous. Bricks research consistently does the right things in the right ways for the right reasons.

Most executives think their organization’s information can withstand the Huff and Puff Test but two out of three are being sold Straws and Sticks solutions, and don’t know it.