a few weeks late, but otherwise little change from previous years

This year’s NAPLAN results have finally landed. The results are a few weeks behind schedule, due to disagreement over how scores should be reported between the body that administers the test and state education officials.

Debate centres on whether data from the new online version of the test and the pen-and-paper version are statistically comparable. The online version is being phased in between now and 2020, and is designed to be more effective due to its adaptive testing design.

Read more: Why the NAPLAN results delay is a storm in a teacup

The Australian Curriculum, Assessment and Reporting Authority (ACARA), which is responsible for NAPLAN, maintains the online and paper tests are comparable. ACARA has sought assurance from assessment experts, who say the results are comparable. Others disagree, including two United States assessment experts who yesterday said the online and paper results are “inherently incompatible” and “should be discarded”.

Such comments add fuel to an already red-hot fire, driven by those who want NAPLAN scrapped, such as New South Wales education minister Rob Stokes, and those who want a broad scale national review, such as Queensland education minister Grace Grace.

But ultimately, we can only work with the data ACARA has released, which combines online and paper data. Overall, it shows 2018 results differ very little from last year’s results or longer-term trends.

How is NAPLAN run?

NAPLAN tests all young people in all schools (government and non-government) across Australia. It takes place every year, assessing Australian school students in years three, five, seven and nine across four domains: reading, writing, language conventions (spelling, and grammar and punctuation) and numeracy.

This year, 20% of students completed the new test online, with the remaining 80% doing the pen-and-paper version.

NAPLAN uses an assessment scale divided into ten bands to report student progress through years three, five, seven and nine. Band one is the lowest and ten is the highest.

ACARA has responsibility for the test (on behalf of federal, state and territory governments) and each year publishes NAPLAN data for every school in the nation on the publicly accessible My School website.

Read more: Five things we wouldn't know without NAPLAN

What did we learn this year?

Working from the assumption that the two test delivery methods are comparable, ACARA’s 2018 data indicate:

• Tasmania and the ACT had a statistically significant decline in year five writing performance from 2017
• WA had a statistically significant improvement in year nine grammar and punctuation performance from 2017
• NSW, Victoria and the ACT continue to be the highest-performing systems, scoring at or above the national average across all domains and year levels
• the Northern Territory continues to under-perform across all domains and year levels, relative to the other states and territories and in relation to national minimum standards
• year nine students who completed the writing test online performed better, on average, than those who completed the writing test with pen and paper (according to ACARA, these differences in results are at least partly attributable to the test mode used).

Similar to previous years, there are large discrepancies between year nine reading and writing across all states.

What about longer-term trends?

The current debate about comparability would be more concerning if 2018 results showed radically different trends compared to previous years. But they don’t.

For example, we see very little change to longer-term trends, which show:

• statistically significant improvements at the national level in spelling (years three and five), reading (years three and five), numeracy (years five and nine), and grammar and punctuation (years three and seven)

• statistically significant declines in writing achievement at the national level in years five, seven and nine (based on data from 2011 to 2018).

Read more: NAPLAN is ten years old – so how is the nation faring?

It’s also very likely the final results (to be released in December) will show a continuation of long-standing patterns of achievement between young people from different backgrounds, which reflect broader inequalities in Australia.

What are the implications moving forward?

Debate over NAPLAN is unlikely to subside any time soon and it may be the case that a national review of the program ultimately emerges. It will be interesting to see what comes from the current NAPLAN review in Queensland and how this contributes to broader national conversations.

Federal politics is also a moveable feast, with Dan Tehan assuming the role of federal education minister over the weekend, following last week’s leadership spill. Tehan replaces Simon Birmingham, who has defended the merits of NAPLAN and has been central to promoting a broader reform agenda in schools. This includes recommendations coming out of the Gonski 2.0 report released earlier this year.

The future of Gonski 2.0 may very well hold clues to the future of NAPLAN. The report recommends an online formative assessment tool be developed. This raises questions about whether such a tool, if created, might ultimately replace or serve as a supplement to NAPLAN.

In the short term, we will continue to see NAPLAN move online, unless any major new road blocks emerge.