Daily BulletinHoliday Centre

The Conversation

  • Written by The Conversation Contributor
imageThe iPhone is a good example of an entire industry built on the back of publicly funded research outcomes. The 'iPhone fish' is designed to teach people healthy eating through portion size control.Lucy Nicholson/Reuters

Prime Minister Malcolm Turnbull is rumoured to announce major changes to the way academic research is funded in his forthcoming innovation statement this month.

Media reports suggest the government wants academics to spend less time writing for journals and more time working with industry to ensure research has a commercial and community impact. It is suggested the aim is to strengthen and build ties between universities and industry and make sure all publicly funded research is contributing to society in some way.

Discussions about how to measure university research have been around for a long time.

Back in 2005, the then minister for education, Brendan Nelson, announced the Research Quality Framework (RQF) aimed at improving the assessment, quality and social impact of university research.

In 2007, the Labor government abandoned the RQF – which was based on using case studies – and focused on evaluating research quality under the banner of Excellence in Research for Australia (ERA).

While case studies have valid uses – such as showcasing real-world benefits of research – an independent body found that there are significant issues in using case studies to measure the social and economic benefits across the research system.

Case studies can demonstrate impact, but can’t be used to measure impact for these reasons:

  • Timing – research often has impacts long after the research is completed.
  • Attribution – innovation is a team sport involving multiple projects, actors and inputs, often in isolation from each other.
  • Appropriability – it can be difficult to know who benefits from research and account for the diverse impacts that can arise.
  • Inequality – it is difficult to compare across different types of impacts. For example, how do you compare the development of Gardasil with research on the effects of climate change in the Murray-Darling Basin?

Without a broad sample of case studies from all universities it’s also difficult to use these as evidence of who should get funding, especially if the research we are talking about can go back 10 or 20 years.

Last year in the UK, almost 7,000 case studies were used in the latest round of the Research Excellence Framework (REF). REF 2014 included an impact assessment worth 20% of the final rating.

A recent report has found that while case studies can show the benefits of research, based on the UK experience, it is unlikely that we will be able to develop specific metrics of impact. After all, like research itself, impact is a complex beast.

And so here we are, a decade on, still grappling with questions of why and how we should measure publicly funded research.

The government is now urging researchers to work more closely with the users of research – across the public, private and not-for-profit sectors. The idea is to help them innovate and so improve their competitiveness and enhance their products, processes and services.

At the same time, it is hoped that publicly funded research will stimulate innovative industries – and jobs – for the future.

The iPhone is one example of product built almost entirely on the back of publicly funded research.

A good idea for Australian higher education?

imageThe government wants academics to balance their effort writing for journals with time spent on research activities that deliver economic and societal impacts.www.shutterstock.com

At the moment, researchers are measured and recognised for the number of journal articles they have published and where they have published them. This has led to Australia being among the best research nations in the world.

But it does mean that while our universities are supposed to be doing a range of valuable tasks – like educating and training students, creating and advancing knowledge and applying knowledge to solve the most pressing issues of our times – academic practices and institutional incentives support them for only one activity: getting grants to undertake research that can be published in journals, to get more grants, and so on.

If we want our universities to work with the community, governments and businesses to deliver greater prosperity and wellbeing, then measuring their research by publications alone is unlikely to fit the bill.

Something like the plan by the Academy of Technological Science and Engineering (ATSE) is a good addition. What this does is measure where academics are working on research with groups outside the university sector based on their sources of research income.

The main challenges

But if such a plan is put in place, it’s important to be aware of a few things:

  1. We don’t want to focus all our university research on having an impact. We still need to do “basic research” – that is research that pushes knowledge forward for its own sake. This generates the insights that later get applied to specific problems.

  2. Research and innovation involve multiple actors, often working in isolation from each other and across long time frames. It involves a lot of chance and trial and error; it is almost never a linear processes. There are activities that improve the likelihood of impact occurring – such as engaging the public and making research widely available. Understanding the context that a researcher wants to deliver impact in (and how it is changing over time) is key to delivering profound and lasting impacts.

  3. There is no silver bullet when it comes to measuring research, only partial measures and proxies. Done well, proxy measures relate directly to the underlying processes they are measuring – for example, on average, researchers who get more of their income from industry sources are doing research that is most relevant to those industries, and so income can be a proxy for industry relevance. But a range of activities will be missed, such as research training or in-kind support. And so academic judgement is also needed to make sense of the numbers. As long as we understand the limits of proxy measures, we can anticipate and minimise perverse outcomes.

How should we measure research?

Given all of this, what we should be looking for is a multi-dimensional set of measures.

This means a robust measure of research excellence (such as ERA); measures of activities that increase the chances of research impact such as engagement with the public; and more than likely some measure of research training.

Missing any one of these means that we are missing out on the important benefits of university research:

  • Research excellence without engaging with the public, industry and government means that we will not apply our world-class research to improving the prosperity of the public that funds it;
  • Engagement in the absence of excellence means that we would not be delivering the best outcomes;
  • Research training builds a pool of qualified researchers who can work within the public and private sectors, and who can receive and apply the lessons of research to maximise its benefits.

It is our investment in research as a society that will give us the best chance of solving the pressing issues of our times – inequality, climate change, food security, global conflict – so there is a lot at stake.

A narrow focus on academic publishing has helped to create a world-leading university sector.

The pressing issue now is how to develop a system that can apply this research to improve the lives of the public.

This article is part of our series Why innovation matters. Look out for more articles on the topic in the coming days.

Tim Cahill consults for the Academy of Technological Sciences and Engineering on the Research Engagement for Australia project.

Mark Bazzacco is affiliated with CSIRO.

Authors: The Conversation Contributor

Read more http://theconversation.com/there-is-no-easy-way-to-measure-the-impact-of-university-research-on-society-50856


The Conversation


Closing the Gap Statement to Parliament

Mr Speaker, when we meet in this place, we are on Ngunnawal country. I give my thanks and pay my respects to our Ngunnawal elders, past, present and importantly emerging for our future. I honour...

Scott Morrison - avatar Scott Morrison

Prime Minister Interview with Alan Jones

ALAN JONES: Prime Minister, good morning.    PRIME MINISTER: Good morning, Alan.    JONES: I was just thinking last night when we're going to talk to you today, you must feel as though you've ...

News Company - avatar News Company

Prime Minister Bridget McKenzie press conference

PRIME MINISTER: Good afternoon everybody. The good news is that the Qantas flight is on its way to Wuhan and I want to thank everybody for their cooperation, particularly the Chinese Government as...

Scott Morrison - avatar Scott Morrison

Business News

Choosing the Right Coworking Space For Your Business

As the capital of Victoria in Australia, Melbourne is inhabited by millions of people and is known as one of the most liveable cities in the world. The latter is due to the city’s diverse community...

Sarah Williams - avatar Sarah Williams

What Should You Expect from A Carpentry Apprenticeship?

Those wanting to pursue a career in woodwork, whether it be to make furniture, construct buildings or repair existing wooden structures, will have to first commence a carpentry apprenticeship. This ...

News Company - avatar News Company

4 Tips To Choosing A Reliable SEO Company For Your Digital Marketing Agency

Working with a digital marketing agency Perth is the best bet in ensuring that your business is promoted well in the online space. If you are an app developer Perth, you may have to work closely wit...

News Company - avatar News Company


How to Be a Smart Frugal Traveller

You are looking through Instagram, watching story after story of your followers overseas at a beach in Santorini, walking through the piazza in Italy, and eating a baguette in front of the Eiffel ...

News Company - avatar News Company


Graduation is the stage of life when a student receives the rewards of hard work of years. It must have taken sleepless nights and tiring days to achieve the task. Now, as you have received your cov...

News Company - avatar News Company

A Travel Guide for Vacations Overseas

There are two types of tourists. Of course, that's a sweeping generalization, and we could be talking about any possible part of traveling.  In this case, we're discussing those who stick to the ma...

News Company - avatar News Company