Researchers are divided over whether artificial intelligence (AI) is having a positive or negative impact on peer review. That is according to a new report from IOP Publishing, which looks at scientists’ perception and experiences of peer review. The study also finds that interest in a paper and the reputation of the journal remain the most important factors for researchers when considering whether to peer review an article.
Entitled State of Peer Review 2024, the report is based on a survey of more than 3000 researchers from over 100 countries. IOP Publishing carried out a similar survey in 2020 but researchers’ growing use of AI tools since then to write or augment peer-review reports has raised various ethical issues. In particular, there are questions over data protection, confidentiality and the accuracy of reviewer reports.
IOP Publishing, which publishes Physics World, currently does not allow the use of generative AI to “write or augment” peer-review reports or for AI tools to be named as authors on manuscripts. Instead, it encourages authors to be “open and transparent” about their use of such tools in their work. However, publishers do not yet have a way to accurately detect whether text has been generated by AI.
About 35% of respondents to the survey think that open-source generative AI tools such as ChatGPT will harm peer review, while 36% say it will have no impact. Just 29% believe AI can benefit scholarly communication. When asked to expand on their responses, researchers admit that such tools can provide some “useful outputs”. However, they warn that expert human verification and editing is vital before AI-generated text can be used in peer review.
Fighting algorithmic bias in artificial intelligence
The study also looks at how much peer review researchers carry out, finding that 30% of reviewers from high-income countries say they receive too many peer-review requests, compared with just 10% from low and middle-income countries. Moreover, 28% of senior researchers also say they get too many requests to peer review, compared to just 7% of PhD students and 9% of postdocs.
“Quality peer review is essential to the integrity and validity of science and relies on reviewers who are engaged, motivated and competent at providing constructive feedback,” says Laura Feetham-Walker, peer review engagement manager at IOP Publishing. “The insights we gain from this survey helps us to ensure we can continue to evolve the support we provide to the global reviewer community to help with their important work.”
