What can your business learn from NASA’s Challenger disaster?

Volume of ramp is 1920cu in vs 3 cu in for test.

If you saw those exact words, would you have called off the Challenger launch because conditions were dangerous and the likelihood of failure was high? Me neither, and here’s why. 

Westerners read from top to bottom and from left to right. 

The farther things are down a list, the less important they are. We rarely think about hierarchy because it is one of the many things we do that operate on autopilot.

No one would ever think what was on line 18 in the smallest font on the page would be more important than everything above it in larger type.

The information used technical terms.

Understanding the terms and measurements used by presenters required a level of technical literacy not possessed by the executives in charge.

Important information was filtered out.

The Columbia Accident Investigation Board found people’s positions in the hierarchy governed what information they saw. Their report said “As information gets passed up in an organization’s hierarchy, from people who do analysis to mid-level managers to high-level leadership, key explanations and supporting information are filtered out.”

Humans read twice as fast as they speak.

This is why so many of us get annoyed by presenters droning on after we’ve already read the slide – twice.

Experts who study such things tell us when presenters read aloud slides we are looking at, what we see and what we hear are out of synch. Our eyes and ears are used to working together. When they are forced to compete, our comprehension and retention both suffer.

Experts say we are better off either reading slides without a presenter or listening to a presenter without slides.

Every day, thousands of people in conference rooms around the world are unaware their comprehension and retention are being diluted by legions of people who think reading slides is a good idea.

On a cold morning in January 1986, NASA executives launched the Challenger. 

Seventy-three seconds later it blew up.

A Presidential Commission was formed to investigate the disaster.

Their report identified two root causes of the catastrophic decision to go ahead with the launch in below-freezing temperatures.

One was the presentation of misleading information. 

The Commission said the critical information was buried in such a way to make it nearly impossible for executives to conclude that launching a shuttle in freezing temperatures would be courting disaster.

The other was the failure of executives to challenge assumptions.

Excused for being misled is not the same as absolved of any responsibility. 

After interviewing every high-ranking executive involved, investigator Richard Feynman said he was stunned by the low level of technical literacy he encountered. Executives so obviously didn’t understand the science behind the presentation that it would have been laughingly naive if not for the disaster.

NASA told executives the probability of catastrophic malfunction was only 1 in 100,000. 

Wise nods were exchanged by the executives around the table. This nice big fat round number was compelling and easy to remember. Executives did not ask presenters where it came from.

When Feynman asked if they realized 1 in 100,000 meant if NASA launched a shuttle a day every day for 274 years they would expect only one accident to ever happen, commissioners were dumbfounded. 

And when he told them how using what he had learned in his interviews with experts, he had been able to calculate the actual chance of catastrophic malfunction as only 1 in 100, they were stunned into silence. The actual risk of catastrophic malfunction was 10,000 times higher than the information presented to decision-makers.

Data visualization has been newly discovered as a hot topic for the past 40 years. 

Edward Tufte is the world’s leading authority on data visualization and information graphics. He says because statisticians don’t understand graphic arts, they are doomed to violate the principles of good design with every slide they build. Graphic artists are just as bad, because their concern is with how things look. The accuracy of the content is someone else’s responsibility.

The right person to build slides that include scientific data would be someone with a firm grasp of statistics AND graphic arts. 

But most presentations are churned out by people who are not only amateur statisticians but amateur graphic artists, too. These visual and statistical illiterates churn out endless piles of what Tufte calls chartjunk.

Slideshows are designed to persuade, not to inform. 

The purpose of a research presentation should be to convey information. But when people use slideshow software, they create the slides first and then cut and paste bits that fit the preformatted slides. As we used to tell project directors and managers, if your client absolutely insists on a slideshow, write the story first, then adapt it.

This is backwards.

It loses the entire point. Unless of course, the presenters’ real purpose is to guide your thinking to the conclusions they want you to draw. When that’s the case, they cram slide after slide into a deck that presents a manipulative and misleading façade of objectivity.

Tufte takes a different approach.

He believes the most efficient method of presenting information is by distributing a brief written report that can be read by attendees in the first five or ten minutes of the meeting. The rest of the meeting is devoted to asking and answering questions and discussing implications. 

Another point Tufte is clear on is the difference between compelling information display and over-designed slides. 

Be advised that the more eye-catching the graphics, images, and animation, the more you should smell a rat. Infographics that look like circus posters are tools for selling ideas. They are not ways to disseminate information in an organization.

Envisioning information.

Using a rubber ring and a glass of ice water, Feynman’s 46-second demonstration illustrated the fatal flaw better than any slideshow ever could. Click here and watch his famous c-clamp experiment. Listen as he ends by saying he believes this has some significance to our problem.

It is highly unlikely that research presentations made in your organization will have such tragic outcomes. 

But how your research is presented and who presents it will definitely have an impact on your company’s ability to provide products and services that keep customers coming back for more.

Presenters’ claims should always be viewed with skepticism.

Ask them to present their credentials and use this simple formula: the farther research presenters are from the actual gathering of the facts, the more distorted the pictures they are showing you. In the meantime,

Here are some things you can do, in bullet form:

  • Eschew obfuscation.
  • Ask for a written report to be distributed before the meeting.
  • Ask your presenters to explain their reasoning and evidence in simple terms. 
  • Ask where the numbers come from. 
  • Ask which are data, which are assumptions, and which are opinions.
  • Ask what has been filtered out – what things are they not telling you?
  • If your presenter can’t answer these questions, call the bullpen and bring in a reliever.

Give this week’s column to your boss. It will either get you in hot water or make you irreplaceable.