Daily Bulletin

The Conversation

  • Written by Adrian Barnett, Professor of Public Health, Queensland University of Technology

Statistics is a useful tool for understanding the patterns in the world around us. But our intuition often lets us down when it comes to interpreting those patterns. In this series we look at some of the common mistakes we make and how to avoid them when thinking about statistics, probability and risk.

Have you ever experienced the perfect evening out? The weather was great, you got the best table in the house at your favourite restaurant, the food was delicious and the wine superb, and the conversation was sparkling.

Then have you made the mistake of trying to repeat the experience and ended up disappointed? This is because your perfect evening was due to a series of chance events that all fell in your favour. A great experience is like tossing a coin and getting a long run of heads – unusual and difficult to repeat.

When you try to repeat a perfect experience, at least one thing is likely to be imperfect the second time around. The couple at the next table are loud and boorish, the waiter gets your order wrong, your jokes fall flat, and so on.

Happily, it works both ways. So if you’re forced to repeat a terrible experience, it’s likely that it won’t be so bad the second time around.

This phenomenon is called “regression to the mean” or “reversion to mediocrity”, which sums up how unusual events are likely to be followed by more typical ones.

The polymath Sir Francis Galton coined the term when he noticed that tall parents tended to have children shorter than them, whereas short parents often had children who were taller than themselves.

For a parent to be unusually tall, the genetic coin had to be “heads” many times in a row. Repeating that feat of chance for their children is not impossible, but it is unlikely.

image Outliers in a population, such as very short or tall parents, will tend to gravitate towards the mean, such as by having children that are closer to the average height in that population. CC BY

Is that a trend?

Regression to the mean is driven by chance, and so it occurs wherever chance occurs, which means it occurs almost everywhere. It is prevalent in sport and can explain the “manager of the month curse” in football. This award is usually won by managers who have had four or more wins in a row, often because of a combination of skill and luck. When the luck runs out, the “curse” strikes.

Regression to the mean will even occur in this article as unusually long sentences will tend to be followed by shorter ones. Check if you don’t believe me. It occurs in the published literature on regression to the mean, as years with many published papers on the phenomenon tend to be followed by years with fewer papers.

This article will itself cause some regression to the mean if it spikes interest in the Wikipedia page, but that interest will inevitably wane.

Regression to the mean is mostly harmless, but it becomes a problem when the change it creates is misinterpreted.

For example, imagine you ran a hospital and were told that hospital-acquired infections were five times higher than average last month. A colleague tells you they know the cause and it can solved by using more prophylactic antibiotics.

You agree and in the following month you’re told that prophylactic antibiotic use is through the roof and infections have come down. Your mind makes a causal connection and you’re now convinced of the need for widespread prophylactic antibiotics, a potentially dangerous connection given that the unusual infection rate could have been due to chance events.

Now your hospital budget will be tighter because of the costs of using more antibiotics, and you’re contributing to serious problem of antibiotic resistance.

image If you track the quality of your nights out, they might fluctuate up and down, but will still hover around the mean. CC BY

Making sham treatments look good

Regression to the mean is unwittingly exploited by quacks who often see patients when they are at their lowest. As many diseases have a natural ebb and flow, seeing patients when they are at their worst is the best time to exploit regression to the mean, because any treatment will appear to cause improvements in enough patients to make it look broadly effective.

Telling the difference between regression to the mean and a real change can be difficult. A chronically ill patient may have a very bad day, but is that the early warning of a downward trajectory or just a blip due to a random cluster of events, such as a bad meal, poor sleep, or an ill-judged sprint for the bus?

Gathering more data using watchful waiting can be useful, as once a clear pattern emerges in a patient’s well-being, it is less likely to be the random ups and downs of regression to the mean.

Regression to the mean is everywhere. Being aware of it might help you avoid overreacting to unusual events. However, if you didn’t find this article interesting or useful then why not read another one on The Conversation – the chances are you’ll enjoy it more.

Authors: Adrian Barnett, Professor of Public Health, Queensland University of Technology

Read more http://theconversation.com/regression-to-the-mean-or-why-perfection-rarely-lasts-74694

Writers Wanted

Birthdays, holidays, Christmas without mum or dad: how to support kids with a parent away fighting fires


With polls showing Labour could govern alone, is New Zealand returning to the days of 'elected dictatorship'?


How to Make Your Car Last Longer


The Conversation


Did BLM Really Change the US Police Work?

The Black Lives Matter (BLM) movement has proven that the power of the state rests in the hands of the people it governs. Following the death of 46-year-old black American George Floyd in a case of ...

a Guest Writer - avatar a Guest Writer

Scott Morrison: the right man at the right time

Australia is not at war with another nation or ideology in August 2020 but the nation is in conflict. There are serious threats from China and there are many challenges flowing from the pandemic tha...

Greg Rogers - avatar Greg Rogers

Prime Minister National Cabinet Statement

The National Cabinet met today to discuss Australia’s COVID-19 response, the Victoria outbreak, easing restrictions, helping Australians prepare to go back to work in a COVID-safe environment an...

Scott Morrison - avatar Scott Morrison

Business News

5 Ways a Printed Marquee Can Help You Sell More at Markets

If you’re a local business owner, you probably already know that outdoor markets are a great way to reach customers and sell your product. If you already have permanent premises, they allow you ...

News Company - avatar News Company

5 Essential Tools for Working Remotely in 2020

The average, modern office worker spends 8 hours a day, 5 days a week in a company building. Since the start of COVID, however, many of these companies have allowed workers to work from home due...

News Company - avatar News Company

What happens to all those pallets?

Pallets — they're not something everyday people often give much thought to. But they're an integral part of any business which receives or distributes large quantities of goods. But once the goo...

News Company - avatar News Company

News Company Media Core

Content & Technology Connecting Global Audiences

More Information - Less Opinion