Goldilocks went for a walk in the forest. She came upon a house and knocked on the door. When no one answered, she walked right in and saw three bowls of porridge. Hungry, she tasted the porridge from the first bowl. “This porridge is too hot!” she said, and tried the porridge from the second bowl. “This porridge is too cold,” she said, and turned to the last bowl of porridge. “Ahhh, this porridge is just right,” she said happily and ate it all up. She saw a chair and sat in it. It was too big, so she tried another one. It was too small, but the third one was just right. Sleepy now, she tried three beds. One was too hard, one was too soft, and one was just right. That’s how I learned the story as a kid. Only later did I find that the original version involved a mean, foul-mouthed old woman and three bachelor bears (small, medium, and large) that lived together in a house in the forest. Further investigation revealed how other accounts modified the story by replacing the old woman with a little girl, dumping the bachelor bears for Papa, Mama, and Baby Bear, and adding foxes, caves, and Russian princes. The story line is always the same, though – a character comes upon an empty home, tries the food, chairs, and beds, and runs away when the owners return. Joseph Cundall wrote the version familiar to most of us. In a letter to his children, he explained why. “The story of the three bears is a very old nursery tale by Robert Southey. In my version, I made the intruder a little girl because there are so many stories of old women.”

The Goldilocks Principle

Christopher Booker was the founder of the satirical magazine Private Eye, a scriptwriter on the television show That Was The Week That Was, and a weekly columnist for The Sunday Telegraph. He defined the Goldilocks Principle as the dialectical three: “the first is wrong in one way, the second in another or opposite way, and only the third, in the middle, is just right. The idea that the way forward lies in finding an exact middle path between opposites is of extraordinary importance in storytelling.”

Being in the middle is central to survival

Animals in the middle of the herd are safer than those out on the edges where predators hunt. Catching a single animal is much harder when it blends into a large group. When animals move together as a herd, it is visually confusing to predators and makes it difficult for them to distinguish one from the next.

People naturally move toward the middle, too

Most of us would rather be warm or cool than hot or cold. The Goldilocks Principle is the notion that there is an ideal amount of some measurable substance, an amount in the middle of a continuum of highest to lowest, and that this amount is “just right.” Marketers provide a premium choice and a budget choice to make regularly-priced products more appealing. Software sellers create products with bare-bones and full-blown versions, knowing most will buy the “regular” version. Not too hot, not too cold. Not too big, not too small. 

Wharton Marketing Professor Jonah Berger says “As customers, our emotional reactions are similar to the protagonist from the children’s tale of Goldilocks and The Three Bears. Whether it has to do to the softness of the bears’ beds or temperature of their porridge, Goldilocks is turned off by the extremes. If something is too novel, it’s unfamiliar, weird and hard to understand. But if it’s exactly the same as what is happening already, it’s boring and there’s no reason to change behavior. In between, though, it’s just right.”

The same thing happens when people take surveys

Measures of central tendency are single values used to describe an entire data set: the mean, the median, and the mode. Our human tendency is to choose responses toward the center of the scale and avoid the extremes at either end. 

Another centering phenomenon is called regression toward the mean

When unusually large or small measurements occur, they tend to be followed by measurements that are closer to the average. This common statistical occurrence can make normal variation in data look like real change to the unwary. Statistician Francis Galton found that tall parents had (on average) children who were smaller than them, and that short parents had (on average) children who were taller than them. In both cases the children with parents at the extreme ends of the distribution had heights closer to the population’s mean height.

What goes up must come down

There is no shortage of stories about high-flying investment counselors whose stock picks beat the market by miles. In the short term, their success is seen as evidence of their shrewd financial insight. Investors jump on board only to find the value of their portfolio dropping over time because their advisor’s good fortune has regressed toward the mean. What went up came down.

Our brains do not deal well with statistics

And forget about using your intuition because it only makes things worse. Nobel prize-winning psychologist Daniel Kahneman wrote a book about the many ingrained biases that distort our perception of reality and muddy our ability to reason. He illustrated the concept of regression toward the mean for a group he was working with by drawing a circle on a blackboard.

One by one, he had everyone in the room turn their backs to the blackboard and throw a piece of chalk at the center of the circle. He recorded the results and then had everyone do it again. Those that did incredibly well on the first try tended to do worse on their second try and vice versa.

In Thinking Fast and Slow, Kahneman gave this excellent example:

If you treated a group of depressed children for several weeks by giving them energy drinks, they would show a clinically significant improvement. If you treated depressed children for that same time by having them stand on their heads three times a day, they would also show a clinically significant improvement. And if you treated depressed kids by having them hug a cat for twenty minutes a day, they would show a clinically significant improvement, too. How can that be?

Given the treatment “facts” above, it is very tempting to jump to the conclusion that energy drinks, standing on one’s head, and hugging cats are all perfectly viable cures for depression. These cases, however, embody the essence of regression toward the mean. Depressed children are an extreme group because they are more depressed than most other children – and extreme groups all regress to the mean over time. Depressed children will get somewhat better over time even if they don’t drink Red Bull, stand on their heads, or hug cats.

What we can do is learn to avoid seeing patterns that don’t really exist

Begin by understanding that whenever you come across extreme occurrences, they were caused by nothing more than chance circumstances. Add to this your knowledge that for all phenomena involving chance, extreme values tend to be followed by more moderate ones. If there is anything to be learned from regression toward the mean, it is the importance of examining things over extended periods of time rather than relying on the single-episode fantastic stories people like to believe in. The simplest way is to avoid drawing hasty conclusions, because doing so is assuming short-term trends will continue when they almost never do.

Want to look at old things in new ways, see the commonplace in more detail and hear complicated subject matter explained in simple terms?

Click here for the home page where you can scroll through the articles I’ve written every week for LetsTakeACloserLook.com since 2016, always showing there is more to most things than meets the eye. Scroll down to to the bottom of this page, enter your email address and get a new article in your inbox every Monday. There is no charge, I don’t share your address with anyone, I send you no e-mails, there are no ads on the site and I reply personally to everyone who sends me a note. How often do you find those things these days?