The evidence that lockdowns work may not be gold standard, but it's good

As Australia begins gradually to relax some of its containment measures (so-called nonpharmaceutical interventions or NPIs) we are confronted with a number of questions

• when will children be back at school full time?

• when might restaurants and pubs be open and will the public continue to respect social-distancing rules when they do?

• do we have adequate testing and contact tracing to extinguish flare-ups in infections without reverting to harsher containment measures?

And more. They are hard and important questions.

But another one being asked is surprising: do lockdowns work?

The question has underpinned the position of lockdown sceptics such as Henry Ergas writing in the Australian and Christopher Joye,

The Australian’s Adam Creighton gave explicit voice to the question this week, writing that our excellent performance in keeping COVID-19 at bay had blinded some to the lack of evidence for lockdown restrictions

The human desire for ritual, and our laudable want to do the right thing, appear to underpin social distancing restrictions more than science.

It is odd that anyone should be seriously asking this question, and it is important to understand that the evidence most certainly does suggest that lockdowns work.

If we fail to accept it, our response to a second-wave outbreak, or to the next global pandemic, will be badly compromised.

It is also important to consider what constitutes “good evidence” when high-stakes decisions need to be made quickly.

The evidence is fairly clear

The gold standard is high quality evidence from randomised controlled trials subjected to peer review, but in emergencies there’s no time to wait for it.

It allows sceptics to point to uncertainly and cherry-pick low-quality evidence that supports their case.

As it happens, the cross-country evidence is fairly clear. Countries that locked down earlier have had better health outcomes, as the following chart from the Financial Times shows.

We don’t yet have the final word on this crisis, so it is also worth looking back to an earlier pandemic.

A 2007 study in the Journal of the American Medical Association examined what happened in 43 United States cities during the 1918-19 Spanish Flu pandemic.

It found measures such as school closures and bans on public gatherings were (statistically significantly) associated with fewer excess deaths.

Cities that implemented NPIs earlier had lower peak mortality rates and lower total mortality. Also the cities that kept the NPIs in place the longest had the lowest mortality.

As the authors put it,

these findings demonstrate a strong association between early, sustained, and layered application of nonpharmaceutical interventions and mitigating the consequences of the 1918-19 influenza pandemic in the United States

A more recent study by economists at the Massachusetts Institute of Technology and the Federal Reserve finds the cities that adopted the most aggressive social distancing measures had the highest economic growth after the restrictions were lifted.

Moreover, cities that implemented NPIs 10 days earlier than other cities increased their manufacturing employment by 5% more than those that imposed restrictions later. The difference lasted through to 1923.

Overall they found an additional 50 days of social-distancing was associated with a 6.5% percent increase in manufacturing employment.

Sure, it’s not gold standard

As his evidence that there was no evidence to support the measures Australia had taken, Creighton cited a World Health Organisation report that summarised a number of studies of previous pandemics.

In fact, the World Health Organisation report said two things.

One was that randomised controlled trials are the best, but that they are not always possible or available.

No surprises there.

The second was in the form of a series of recommendations, based on the evidence it assessed.