Eleven years after Charles Lindbergh was the first to fly solo across the Atlantic Ocean, Douglas Corrigan took off from New York in his plane and headed for California. He landed in Ireland, claiming he lost his direction in the clouds and his compass had malfunctioned. Aviation authorities didn’t buy the story and suspended his pilot’s license for making an unauthorized flight. By the time “Wrong Way” Corrigan returned to New York by steamship, he had become a national celebrity. Jim Marshall, a professional football player with the Minnesota Vikings, picked up an opponent’s fumble and ran 65 yards for a touchdown. Fine, but he ran the wrong way and scored a touchdown for the other team.

An unnamed consultancy that charges some of the highest fees in the industry went the wrong way, too.

As the story goes, a client company needed to decide which of two paths to pursue, and so commissioned a survey. One of the survey questions asked consumers to choose between two options. When all the responses had been collected, the lead analyst was pressured by his boss for instant results regarding this key question.

He told his boss the team needed to follow all the steps of their company’s strict written protocols before he could verify that all the data had been entered and coded accurately. This had to be done before drawing any conclusions or releasing any findings, of course.

You can do all that later, his boss said.

He insisted the analyst tell him if the customers preferred A or B. The analyst reluctantly told his boss that early, unconfirmed numbers showed 80% chose A and only 20% chose B, but cautioned that he could not verify those numbers until he had followed all the checklist procedures and completed a detailed examination of the data.

The boss immediately took the news to the client. 

He told the client that consumers preferred Option A by a four to one margin over Option B. The client was happy because he got the information sooner than expected and was able to immediately set the wheels in motion on some big changes to the company’s strategy.

He should have waited until the analyst confirmed the numbers.

Every survey is imperfect.

Data entry mistakes and coding errors are common. This is why – before any analysis – conscientious researchers conduct what is called a cleaning run. Cleaning runs check every question and every response, looking for incorrect data entry, faulty coding, and errors in mathematical calculations. Data cleaning, also called data cleansing or scrubbing, is the process of detecting, correcting, replacing, modifying, or removing incomplete, incorrect, irrelevant, corrupt, or inaccurate data. Qualtrics says “Before you analyze your survey results, data cleaning is a must-do. It helps you get the best quality data possible so you can make more accurate decisions.” The European Commission says not cleaning data can lead to a range of problems, including incorrect analysis that leads to the drawing of false conclusions (italics mine).

The analyst’s detailed examination of the data revealed that the labels for the survey’s key question had been transposed, thus reversing the findings. The cleaning run revealed that 80% of study subjects had actually picked Option B, with Option A chosen by only 20%. This was of course the exact opposite of what the boss had recklessly reported to the client. A candid discussion between the boss and the analyst ended with the boss deciding not to tell the client about the mistake and firing the analyst with a keep-your-mouth shut settlement.

Too many bosses value fast answers more than careful analysis. 

The more researchers are pressured to cut corners, the more mistakes pile up. On this particular occasion, the boss crossed an ethical line. Rather than acknowledge the error and take responsibility for it, he buried the mistake and let the client make a crucial decision with false information. It was the analyst’s opinion that while his boss justified the coverup as protecting the consultancy’s reputation, privately he was more concerned about protecting his own.

Things like this happen more often than you imagine. 

Even worse things happen, too. One of the things accounting firms sell is their skills with numbers. The other is their commitment to bulletproof truth-telling. Yet every other year or so we hear how a top-drawer global accounting firm was fined for falsifying records in violation of their licensed legal responsibilities. Did you know that research firms are not bound by such legal obligations?

Research firms are no worse than others in the business world. The problem is they’re no better, either.

To read more articles like this, click here.