Daily Bulletin


The Conversation

  • Written by Alan Finkel, Australia’s Chief Scientist, Office of the Chief Scientist

In the main, Australia produces high-quality research that is rigorous and reproducible, and makes a significant contribution towards scientific progress. But that doesn’t mean we can’t do it better.

In the case of the research sector here and abroad, we need to acknowledge that as good as the research system is, there is a problem.

There are a significant number of papers that are of poor quality, and should never have made it through to publication. In considering why this might be the case, I have found myself reflecting on the role of incentives in the research system.

Because incentives matter, as we have seen through the findings of the Royal Commission into the banking sector led by Kenneth Hayne.

The commission shone a light on how the sector incentivises its employees. And there are some incentives in the research community that, in my view, need to be looked at.

We may be inadvertently encouraging poor behaviour. And to ensure research remains high-quality and trustworthy, we need to get the incentives right.

Lessons from the banking Royal Commission

Australia's top scientist Alan Finkel pushes to eradicate bad science Parallels can be drawn between inappropriate incentives in both the banking and research sectors, Dr Finkel says. AAP

The commission showed that over the past decade or two, the banking sector moved from salary-based to bonus-based remuneration. But those bonuses have been mapping to the wrong values: to sales and profit instead of compliance with the law and net benefit to customers.

Similarly in the research sector, we can’t ignore that there are many incentives pushing some to cut corners and lower their standards.

The competition for funding is fierce and is increasing every day. The temptation to judge a researcher’s performance through simple metrics, such as the number of published research papers, is strong. These metrics are incentives that drive behaviour - not all of it good.

Read more: Finkel: students, focus on your discipline then you’ll see your options expand

We all know of instances of poor research practice. Selective publication of results to support a hypothesis. HARKing: hypothesising after results are known. Manipulating data and research methods to achieve statistical significance.

If we can focus on improving the quality of research in general, we can achieve broad and long-lasting benefits. And I think the best way to do this is to look at the incentives.

Quality should trump quantity

Publication is a principal criterion for career advancement in the research sector. And I don’t want to change that. However, the institutionalisation of performance metrics has created incentives for researchers to publish as many papers as possible.

There shouldn’t be an incentive for a researcher to salami-slice their results into three or four separate publications, rather than one meaningful publication. If the purpose of publication is to share your results in a way that can be built on by other researchers, this kind of practice completely defeats that purpose.

Australia's top scientist Alan Finkel pushes to eradicate bad science Perverse incentives can encourage the manipulation of research results, Dr Finkel says. AAP

One model that places the focus on quality over quantity is the “Rule of Five”. With this rule, a researcher’s performance for grant funding or promotion is judged on their best five publications over a five year period, accompanied by a description of its impact and the researcher’s individual contribution.

The exact number of publications or years isn’t important, as long as it is less than ten.

Of course, there are disciplinary differences that may need to be taken into account. But what matters is the emphasis on the significance of the research.

Researchers must undergo integrity training

Unlike other professions, there are no national competencies and no national recognition of education and training in research integrity. While many institutions in Australia do provide training programs for their PhD students, these programs vary in quality, content and reach.

And, to the best of my knowledge, no Australian institutions have a training requirement for their existing research workforce.

Read more: The science is clear: we have to start creating our low-carbon future today

I strongly believe the overall quality of research in Australia would be strengthened by research integrity training for all researchers.

Training puts a spotlight on expectations for the whole community and encourages consistent behaviour. It also removes that old chestnut of plausible deniability: “Honest, officer, I didn’t know it was wrong!”

The training must be accredited, and must be high quality. It should not be a “tick the box” exercise. And if we circle back to incentives, the best way to encourage researchers to undertake the training is to make proof of training a requirement for obtaining a grant.

To those naysayers who say it will never happen, let me tell you that it already has. The Irish Health Research Board has recently implemented such a scheme.

Australia's top scientist Alan Finkel pushes to eradicate bad science Finkel: research integrity training should be mandatory for all researchers, and tied to grant funding. AAP

‘Predatory, evil’ scholarly journals

Finally, I am concerned that the incentives in the research system are not just driving bad behaviour for researchers, but are also creating a market for criminals to enter scholarly publishing.

What is motivating the crooks is the pay-per-page system that has come with the introduction of open access publishing.

Now, open access publishing has many benefits and I support the move towards it. But I remain concerned that it has opened the door for predatory, evil, crooked journals.

It is just too easy to set up a journal and a website with a highfalutin title, and appropriate the biographies of leading researchers for the editorial board – without their knowledge or permission. Before you know it, huge numbers of papers are being published without any rigour.

And there are researchers who are knowingly paying to publish in journals that have no peer review, even though they claim to. Journals that have no ethics. Not even an editorial team to consider the submitted paper.

These researchers might just be naïve, but we have to acknowledge that the current incentives reward this behaviour.

While this is not a major problem in Australia, emerging research nations are really struggling with this.

Australia's top scientist Alan Finkel pushes to eradicate bad science Finkel: a new publishing standard should be introduced to weed out unscrupulous research journals. AAP

In my conversations with senior research leaders around the world, they are looking for ways to improve performance metrics in a way that does not drive their researchers to these journals.

I propose a rigorous quality assurance system, designed to inform stakeholders that a particular journal’s processes for assessing a paper meets agreed publishing standards. I like to call it Publication Process Quality Assurance, or PPQA.

Compliance with PPQA would indicate to researchers, research institutions, libraries and granting agencies that the journal follows internationally accepted guidelines for the publication process.

Read more: Science isn't broken, but we can do better: here's how

Granting agencies are best placed to provide the incentive for researchers to only publish in PPQA-compliant journals by enforcing it through their grant application process.

Follow the money

You might have picked up by now a common thread; that in each of my three recommendations, I am looking to take the responsibility back to the granting agencies. It’s a concept referred to by others as “follow the money”.

If the granting agencies put in place these measures, they will ripple through into the research institutions and mitigate the ongoing risks of poor quality research.

It will change the culture and ensure that last century’s academic rigour continues for the 21st century research workforce and beyond.

This is an edited version of a speech Dr Finkel will deliver in Melbourne on September 12 at the Peter Doherty Institute for Infection and Immunity, to mark its fifth anniversary.

Authors: Alan Finkel, Australia’s Chief Scientist, Office of the Chief Scientist

Read more http://theconversation.com/there-is-a-problem-australias-top-scientist-alan-finkel-pushes-to-eradicate-bad-science-123374

Writers Wanted

Thank you to everyone who helped The Conversation survive (and thrive) in 2020

arrow_forward

Top Reasons Why Your Business Needs SEO

arrow_forward

The Conversation
INTERWEBS DIGITAL AGENCY

Politics

Prime Minister Interview with Ben Fordham, 2GB

BEN FORDHAM: Scott Morrison, good morning to you.    PRIME MINISTER: Good morning, Ben. How are you?    FORDHAM: Good. How many days have you got to go?   PRIME MINISTER: I've got another we...

Scott Morrison - avatar Scott Morrison

Prime Minister Interview with Kieran Gilbert, Sky News

KIERAN GILBERT: Kieran Gilbert here with you and the Prime Minister joins me. Prime Minister, thanks so much for your time.  PRIME MINISTER: G'day Kieran.  GILBERT: An assumption a vaccine is ...

Daily Bulletin - avatar Daily Bulletin

Did BLM Really Change the US Police Work?

The Black Lives Matter (BLM) movement has proven that the power of the state rests in the hands of the people it governs. Following the death of 46-year-old black American George Floyd in a case of ...

a Guest Writer - avatar a Guest Writer

Business News

How to Find the Best SEO Services Company That Offers Guaranteed Results

As a business owner, you have to be strategic about how you’ll be able to reach your target market. That is why entrepreneurs implement various marketing tactics to reach their goals. With today...

News Co - avatar News Co

Top Reasons Why Your Business Needs SEO

SEO is crucial for the ranking of a website. You may think that SEO offers greater searchability while it can do more than this. The most cost-effective tool for the survival of smalls businesse...

News Co - avatar News Co

Nisbets’ Collab with The Lobby is Showing the Sexy Side of Hospitality Supply

Hospitality supply services might not immediately make you think ‘sexy’. But when a barkeep in a moodily lit bar holds up the perfectly formed juniper gin balloon or catches the light in the edg...

The Atticism - avatar The Atticism



News Co Media Group

Content & Technology Connecting Global Audiences

More Information - Less Opinion