No, Education Minister, we don't have enough evidence to support banning mobile phones in schools

Last week, South Australia announced a mobile phone ban in primary schools. Federal Education Minister Dan Tehan endorsed the ban, saying:

Data shows a correlation between the uptake of mobile phones by young people and a downturn in student performance.

The federal government put “mobile phones on the agenda” in a September 2019 meeting with the Education Council. The minister then quoted research from Canadian Assistant Professor Louis-Philippe Beland, that he said:

[…] found the positive effect of banning mobile phones was equivalent to an additional hour a week in school, or increasing the school year by five days.

He went on to say the research:

[…] also found the positive effect was even greater for low-achieving students, with a mobile phone ban the equivalent of an additional two hours a week, or ten school days a year.

The study the minister is referring to was published in 2016 about schools in the United Kingdom. The authors — Beland and Murphy — actually showed only a very minor correlation between students’ exam scores and their schools’ mobile phone policies.

The claim students’ use of mobile phones at school is connected with lower academic performance has consistently featured in the popular debate around school mobile phone bans. It has been used to justify blanket bans, both in Australia — such as the one in NSW public primary schools and all Victorian state primary and secondary schools — and overseas.

Despite the claims, we actually don’t have sufficient data to back the policy.

What the science actually says

The best kind of policy is grounded in evidence. But not all evidence is created equal and, when cited out of context, can obscure rather than illuminate the path forward.

Unfortunately, the ways evidence has been mobilised in the school mobile phone ban debate in Australia has turned children’s digital practices — and potentially their digital futures — into a political football.

Read more: Should mobile phones be banned in schools? We asked five experts

Beland and Murphy’s study surveyed 91 high schools in four English cities about their mobile phone policies. It then matched those schools’ mobile phone policies with students’ standardised test scores to estimate the effect of mobile phone bans on student performance.

The study reported that, when schools instituted a ban, student test scores improved by 6.41% of a standard deviation — which is an effect size of 0.06.

Let’s unpack what this means.

A small ‘effect size’

First, the “data” did not show that students’ “uptake” of mobile phones related to a “downturn” in their performance. Rather, it showed restrictive school mobile phone policies related to an upturn in student performance. These are different things.

Second, the study did not find a strong correlation between schools’ restrictive mobile phone policies and student performance. An improvement in student performance of 6.41% of a standard deviation is actually an effect size of 0.06 — though it has been reported as a 6% improvement.

Effect sizes of 0.24 are considered small, 0.50 is moderate and 0.75 is large. So, an effect size of 0.06 is insignificant.

Students in the lowest quintile (fifth) of prior achievement gained slightly more in test scores when mobile phone bans were in place: 0.14% of a standard deviation, which is still very small. School bans neither positively nor negatively affected the test performance of students in the top quintile.

As for bans granting the equivalent of ten days of extra class time for low-achieving students quoted by the minister, this number doesn’t seem to be in the study.

In a Conversation article by Murphy and Beland about their study, they write:

We found the impact of banning phones for these [low-achieving] students equivalent to an additional hour a week in school, or to increasing the school year by five days.

But even in this case, the data was collected from 2001 to 2011 in the United Kingdom. So the specific number of days of extra class time don’t directly apply to the Australian context.

Read more: How smart is it to allow students to use mobile phones at school?

Correlation is not causation

Third, although media coverage correctly reported the study found a correlation, many people mistake correlation for causation. Cheese consumption strongly correlates to death by bedsheet entanglement, but we wouldn’t say cheese causes such deaths.

If we noted a strong correlation between increased ice-cream sales and sunglass purchasing, we might infer, not that wearing sunglasses causes one to eat ice-cream, but that a third variable influences both outcomes — namely hot, sunny weather.

In the same way, the correlation between mobile phone bans and student performance could be caused by other variables simultaneously impacting the school environment and student performance.

Beland and Murphy acknowledge this limitation when they write: “We may be concerned that student sorting by observable or unobservable characteristics may be driving this estimate”.

Read more: Banning mobile phones in schools: beneficial or risky? Here's what the evidence says

The results haven’t been replicated

Lastly, good science means repeating studies and finding similar results. Two other studies replicated Beland and Murphy’s research design.

A Norwegian study found no significant effect of mobile phone bans on academic results. However, when they divided the sample into public and private schools, they did find private schools experienced a somewhat positive effect of a mobile phone ban on academic performance.

This points to the fact school culture could make a difference to whether or not bans improve student performance. Or, put differently, blanket bans may not serve the best interests of all schools and their students.

And a study in Sweden found school mobile phone bans had zero impact on student performance. The authors said:

In Sweden, we find no impact of mobile phone bans on student performance and can reject even small-sized gains.

The main problem, though, is not the limitations of Beland and Murphy’s specific study. The real issue is the dearth of evidence to drive decision making about students’ use of mobile phones at school.

Until this vacuum is addressed, we cannot know for certain mobile phone bans in schools are a good idea. And it remains possible current policies are compromising students’ best interests, both now and into the future.

Read more: How creative use of technology may have helped save schooling during the pandemic