Daily Bulletin

The Conversation

  • Written by The Conversation Contributor
imageThe iPhone is a good example of an entire industry built on the back of publicly funded research outcomes. The 'iPhone fish' is designed to teach people healthy eating through portion size control.Lucy Nicholson/Reuters

Prime Minister Malcolm Turnbull is rumoured to announce major changes to the way academic research is funded in his forthcoming innovation statement this month.

Media reports suggest the government wants academics to spend less time writing for journals and more time working with industry to ensure research has a commercial and community impact. It is suggested the aim is to strengthen and build ties between universities and industry and make sure all publicly funded research is contributing to society in some way.

Discussions about how to measure university research have been around for a long time.

Back in 2005, the then minister for education, Brendan Nelson, announced the Research Quality Framework (RQF) aimed at improving the assessment, quality and social impact of university research.

In 2007, the Labor government abandoned the RQF – which was based on using case studies – and focused on evaluating research quality under the banner of Excellence in Research for Australia (ERA).

While case studies have valid uses – such as showcasing real-world benefits of research – an independent body found that there are significant issues in using case studies to measure the social and economic benefits across the research system.

Case studies can demonstrate impact, but can’t be used to measure impact for these reasons:

  • Timing – research often has impacts long after the research is completed.
  • Attribution – innovation is a team sport involving multiple projects, actors and inputs, often in isolation from each other.
  • Appropriability – it can be difficult to know who benefits from research and account for the diverse impacts that can arise.
  • Inequality – it is difficult to compare across different types of impacts. For example, how do you compare the development of Gardasil with research on the effects of climate change in the Murray-Darling Basin?

Without a broad sample of case studies from all universities it’s also difficult to use these as evidence of who should get funding, especially if the research we are talking about can go back 10 or 20 years.

Last year in the UK, almost 7,000 case studies were used in the latest round of the Research Excellence Framework (REF). REF 2014 included an impact assessment worth 20% of the final rating.

A recent report has found that while case studies can show the benefits of research, based on the UK experience, it is unlikely that we will be able to develop specific metrics of impact. After all, like research itself, impact is a complex beast.

And so here we are, a decade on, still grappling with questions of why and how we should measure publicly funded research.

The government is now urging researchers to work more closely with the users of research – across the public, private and not-for-profit sectors. The idea is to help them innovate and so improve their competitiveness and enhance their products, processes and services.

At the same time, it is hoped that publicly funded research will stimulate innovative industries – and jobs – for the future.

The iPhone is one example of product built almost entirely on the back of publicly funded research.

A good idea for Australian higher education?

imageThe government wants academics to balance their effort writing for journals with time spent on research activities that deliver economic and societal impacts.www.shutterstock.com

At the moment, researchers are measured and recognised for the number of journal articles they have published and where they have published them. This has led to Australia being among the best research nations in the world.

But it does mean that while our universities are supposed to be doing a range of valuable tasks – like educating and training students, creating and advancing knowledge and applying knowledge to solve the most pressing issues of our times – academic practices and institutional incentives support them for only one activity: getting grants to undertake research that can be published in journals, to get more grants, and so on.

If we want our universities to work with the community, governments and businesses to deliver greater prosperity and wellbeing, then measuring their research by publications alone is unlikely to fit the bill.

Something like the plan by the Academy of Technological Science and Engineering (ATSE) is a good addition. What this does is measure where academics are working on research with groups outside the university sector based on their sources of research income.

The main challenges

But if such a plan is put in place, it’s important to be aware of a few things:

  1. We don’t want to focus all our university research on having an impact. We still need to do “basic research” – that is research that pushes knowledge forward for its own sake. This generates the insights that later get applied to specific problems.

  2. Research and innovation involve multiple actors, often working in isolation from each other and across long time frames. It involves a lot of chance and trial and error; it is almost never a linear processes. There are activities that improve the likelihood of impact occurring – such as engaging the public and making research widely available. Understanding the context that a researcher wants to deliver impact in (and how it is changing over time) is key to delivering profound and lasting impacts.

  3. There is no silver bullet when it comes to measuring research, only partial measures and proxies. Done well, proxy measures relate directly to the underlying processes they are measuring – for example, on average, researchers who get more of their income from industry sources are doing research that is most relevant to those industries, and so income can be a proxy for industry relevance. But a range of activities will be missed, such as research training or in-kind support. And so academic judgement is also needed to make sense of the numbers. As long as we understand the limits of proxy measures, we can anticipate and minimise perverse outcomes.

How should we measure research?

Given all of this, what we should be looking for is a multi-dimensional set of measures.

This means a robust measure of research excellence (such as ERA); measures of activities that increase the chances of research impact such as engagement with the public; and more than likely some measure of research training.

Missing any one of these means that we are missing out on the important benefits of university research:

  • Research excellence without engaging with the public, industry and government means that we will not apply our world-class research to improving the prosperity of the public that funds it;
  • Engagement in the absence of excellence means that we would not be delivering the best outcomes;
  • Research training builds a pool of qualified researchers who can work within the public and private sectors, and who can receive and apply the lessons of research to maximise its benefits.

It is our investment in research as a society that will give us the best chance of solving the pressing issues of our times – inequality, climate change, food security, global conflict – so there is a lot at stake.

A narrow focus on academic publishing has helped to create a world-leading university sector.

The pressing issue now is how to develop a system that can apply this research to improve the lives of the public.

This article is part of our series Why innovation matters. Look out for more articles on the topic in the coming days.

Tim Cahill consults for the Academy of Technological Sciences and Engineering on the Research Engagement for Australia project.

Mark Bazzacco is affiliated with CSIRO.

Authors: The Conversation Contributor

Read more http://theconversation.com/there-is-no-easy-way-to-measure-the-impact-of-university-research-on-society-50856

Writers Wanted

Physical Therapist Talks About This New Massage Gun On The Block - The HYDRAGUN


Too much information: the COVID work revolution has increased digital overload


Ammonite: the remarkable real science of Mary Anning and her fossils


The Conversation


Prime Minister's Remarks to Joint Party Room

PRIME MINISTER: Well, it is great to be back in the party room, the joint party room. It’s great to have everybody back here. It’s great to officially welcome Garth who joins us. Welcome, Garth...

Scott Morrison - avatar Scott Morrison

Prime Minister Interview with Ben Fordham, 2GB

BEN FORDHAM: Scott Morrison, good morning to you.    PRIME MINISTER: Good morning, Ben. How are you?    FORDHAM: Good. How many days have you got to go?   PRIME MINISTER: I've got another we...

Scott Morrison - avatar Scott Morrison

Prime Minister Interview with Kieran Gilbert, Sky News

KIERAN GILBERT: Kieran Gilbert here with you and the Prime Minister joins me. Prime Minister, thanks so much for your time.  PRIME MINISTER: G'day Kieran.  GILBERT: An assumption a vaccine is ...

Daily Bulletin - avatar Daily Bulletin

Business News

Getting Ready to Code? These Popular and Easy Programming Languages Can Get You Started

According to HOLP (History Encyclopedia of Programing Languages), there are more than 8,000 programming languages, some dating as far back as the 18th century. Although there might be as many pr...

News Co - avatar News Co

Avoid These Mistakes When Changing up Your Executive Career

Switching up industries is a valid move at any stage in your career, even if you’re an executive. Doing so at this stage can be a lot more intimidating, however, and it can be quite difficult know...

News Co - avatar News Co

4 Costly Mistake To Avoid When Subdividing Your Property

As a property developer or landowner, the first step in developing your land is subdividing it. You subdivide the property into several lots that you either rent, sell or award to shareholders. ...

News Co - avatar News Co

News Co Media Group

Content & Technology Connecting Global Audiences

More Information - Less Opinion