The growth of AI and papermills has caused scientific publishers countless headaches, but they are beginning to fight back, as Michael Banks discovers
On 17 April 2016, Elisabeth Bik, a microbiologist at Stanford University, published a paper that shocked the world of medicine – and also shook the foundations of scientific publishing. Working with Ferric Fang from the University of Washington in Seattle and Arturo Casadevall from the Johns Hopkins School of Medicine in Baltimore, she conducted a huge study into image duplication and manipulation covering more than 20,000 published papers in 40 journals published between 1995 and 2014.
Together, the team documented 782 instances of image duplication – where the same photo, graph or data is reused improperly to represent different experiments – as well as 196 published papers that contained “duplicated figures with alteration”. Before the paper, Bik had anonymously been submitting reports on plagiarism and image duplication to journal editors, which she was motivated to do after discovering that someone had plagiarized her own work.
But following the paper’s publication, Bik became a scientific “sleuth-in-chief”, kickstarting a whole industry of so-called amateur scientific sleuths, to which she provided advice and mentorship. In 2024 she was awarded the Einstein Foundation Award for “identifying misconduct and potential fraud in scientific publications, highlighting science’s problems policing itself”. She donated the €150,000 prize money to the non-profit Center for Scientific Integrity to create a fund that supports fellow sleuths.
Bik’s work was a wake-up call not only for academics, but also for scientific publishers, journal boards and journal editors to focus on, and hopefully halt, scientific misconduct. IOP Publishing, for example, which publishes Physics World, employs a team of six research-integrity officers handling misconduct cases, which has become a far bigger problem than 10 or 20 years ago. Back then, examples of scientific misconduct, which mostly involved data fabrication, plagiarism or self-plagiarism, were notorious – certainly in physics – for their rarity.
One high-profile outlier involved the Bell Labs physicist Hendrik Schön, whose work promised to revolutionize the fields of organic electronics, superconductivity and nanotechnology. He published several high-profile papers in journals such as Nature and Science, but some scientists gradually began to notice that data in some of his figures appeared to have been duplicated. An independent committee was set up to investigate and in 2002 it found that Schön had fabricated data and falsified reports in 16 of 24 papers published between 1998 and 2001.
More recently, condensed-matter physics was hit by another high-profile case in which Ranga Dias claimed to have discovered superconductors that could operate at high pressures and ambient temperatures. A subsequent investigation by the University of Rochester in New York, where Dias was employed, concluded that he had committed misconduct, including data fabrication.
While misconduct generally used to be carried out by an individual looking for academic notoriety, it has now evolved into a co-ordinated and commercial activity that is tapping into science’s dependence on metrics when it comes to prestige and career progression. Referred to as “papermills”, these businesses sell authorships, citations, data and even produce full papers. Such papermills have evolved incredibly quickly to evade publisher checks and the peer-review process. Papermills also often market their services to students, promising guaranteed acceptance and citations.
Few people at the time of Bik’s discovery, however, could have foreseen the growth of artificial intelligence (AI) and machine learning (ML). These developments have given bad actors a powerful new tool – letting them rapidly increase their output and also target journals that lack the necessary checks needed to stop scientific misconduct at scale.
Scientific publishing terms at a glance
Open access Where research and academic work is free for everyone to read, share and use online.
Article processing charge A fee paid by an author or their funder to make a research paper open access.
Peer review An evaluation process where academic or scientific research is scrutinized by experts in the same field before it is published. Its main goal is to check the validity, accuracy and quality of a study to ensure only credible work enters the scientific record.
Impact factor A number that shows how often articles in an academic journal are cited in other research papers. It helps measure the relative importance or rank of a journal within its field, with a higher number meaning the work published in the journal is used more often by other scientists.
Papermill A shady, profit-driven business that creates fake or poor-quality scientific manuscripts and sells co-authorship on them to researchers.
Predatory journals Deceptive, profit-driven publications that claim to be legitimate scholarly journals but bypass proper peer review, editorial oversight and quality control. Their main goal is to trick researchers into paying an article processing charge without offering real academic services.
Artificial intelligence Intelligent behaviour exhibited by machines. But the definition of intelligence is controversial so a more general description of AI that would satisfy most is: the behaviour of a system that adapts its actions in response to its environment and prior experience.
Machine learning As a group of approaches to endow a machine with artificial intelligence, machine learning is itself a broad category. In essence, it is the process by which a system learns from a training set so that it can deliver autonomously an appropriate response to new data.
Large language model An artificial intelligence program trained on massive amounts of text to predict the next word in a sequence, allowing it to write text, translate languages, answer questions and power conversational chatbots.
Dark side of science
The term “publish or perish” is a well-worn phrase in academia. It originates because scientific success is often measured by the number of high-impact articles a researcher has co-authored. Publishing in high-impact-factor journals, and the citations that often generates, can affect an academic’s career and influence job or grant applications. Some institutions have been known to even award cash bonuses for publishing in such journals; with research contracts even not being renewed when a researcher can’t keep up with the publication rate required.
This pressure to publish is partly why publishers and journals have grown so much in recent years. That is especially so for open-access journals, which remove the requirement for traditional subscriptions. Articles are instead made immediately and freely available for anyone to read, with publication costs covered by authors who pay an article-processing charge, which can typically be around £2500 per article and rise to as much as £10,000.
That cost has made scientific publishers less reliant on journal subscription fees paid by libraries and more on income earned from individual scientists. According to an analysis carried out in 2023 by researchers in Canada and Germany, scientists globally paid more than $1bn in open-access fees between 2015 and 2018 to the big five academic publishers: Elsevier, Sage, Springer Nature, Taylor & Francis, and Wiley.
But as the popularity of open-access journals has risen, so too has the growth of “predatory” journals. They exploit the open-access model – and the need for scientists to boost their publication records – by taking publication fees but not carrying out a proper peer-review process, if at all.
A study in 2025 by researchers at Northwestern University found that the publication of fraudulent science, aided by papermills, is now at a point where it is outpacing the growth rate of legitimate scientific publications. According to the Retraction Watch Database, which is now part of the not-for-profit organization Crossref, some 13,000 papers globally were retracted in 2023, compared to roughly 6000 in 2022 and 5000 in 2021 (see figure 1). There are now more than 60,000 retractions in the Retraction Watch Database, with estimates of the number of papermill papers that did manage to make it into the scientific record at 10 times that number.
1 Fighting back
Papermill activity in the early 2020s led to a significant rise in the number of retracted articles, according to data from Retraction Watch. Publishers subsequently built up research integrity teams and developed new tools to tackle the issue, but it still remains a significant challenge within scholarly publishing.
Part of that increase has been the huge growth in the use of AI tools and techniques. AI is particularly potent in microscopy, where generative AI has made it simple to generate fake images of any microscopy technique within minutes that are indistinguishable from real pictures. Indeed, it is estimated that between 1999 and 2024, about 10% of papers published in cancer research were from papermills. An analysis carried out in 2023, meanwhile, found that about 400,000 papers across all disciplines – representing about 2% of all published papers – were from papermills.
In 2022 Anna Abalkina, a research-integrity sleuth and social scientist at the Free University of Berlin, spotted papers with author e-mail addresses that had domains that did not match where the academic institution was based. In a subsequent analysis the papermill – which was dubbed Tanu.pro and is one of the largest in Europe – was found to have produced 1517 papers between 2017 and 2025. It also listed more than 4500 researchers affiliated with around 460 universities across 46 countries. The majority of the authors were in Ukraine, Kazakhstan and Russia.
The trouble that scientific publishers have is that Al allows a paper to be produced at the click of a button, which comes with some unintended consequences. In 2021 computer scientist Guillaume Cabanac from the University of Toulouse and colleagues discovered the use of “tortured phrases” in thousands of research paper thanks to algorithms taking the names of scientific terms rather too literally. Examples include “Sun oriented force” for solar energy, “motor vitality” for kinetic energy or “counterfeit consciousness” for artificial intelligence.
Cabanac began his efforts charting scientific misconduct in 2020 by working with the computer scientist Cyril Labbe to spot gibberish in computer-science papers that were automatically generated using SCIgen, a piece of software that can produce a scientific paper with just a few prompts. “My research began to move towards the quantitative study of science and so we began to collaborate,” notes Cabanac. The pair’s work led to thousands of papers being retracted with Cabanac adding that the scale of use of these tortured phrases in the literature was “a surprise”.
Cabanac says that as AI tools have improved, there are fewer papers with such phrases being published, and spotting misconduct now involves different “smoking guns”. In 2023, for example, Cabanac was one of the first to identify the so-called “regenerate response” fingerprint in dozens of publications. This specific phrase is the label of a button on ChatGPT and while many publishers allow authors to use large language model (LLM) tools to help them produce manuscripts, they must declare it. The problem was that many authors did not make such a declaration, but left the fingerprints of LLM use in their manuscript.
Cabanac says that the latest misconduct smoking gun can be found in bibliographies and the growing use of hallucinated references – citations to other papers that have been simply made up by an LLM. Along with data falsification, other forms of scientific misconduct include citation manipulation, in which irrelevant or unnecessary references are added to boost the citation figures of a colleague or other author. This can also be reciprocal, in which groups of scholars cite each other’s work.
Some fields suffer more than others. In mathematics, for example, the number of research papers and general citations are quite low, which makes citation numbers more prone to manipulation. In 2023 data firm Clarivate announced they would exclude the entire field of mathematics from their influential list of “highly cited researchers” due to the issue of citation manipulation – a decision they reversed two years later.
Run of the papermill
The boom in AI techniques and papermills has been a real headache for publishers, with papermill “attacks” resulting in mass retractions in recent years. Such attacks involve co-ordinated, industrial-scale assaults on academic journals, flooding editorial offices with large batches of fake, plagiarized or completely fabricated research manuscripts. While some papermills function in the so-called “shadow market”, others are officially registered businesses, or mimic legitimate businesses with a website offering services.
Although papermills have targeted all types of journals including open access and subscription and any publisher regardless of size, they have mostly focused on conference proceedings and special issues. They make for easy targets thanks to their softer-touch peer review and use of “guest editors” who are responsible for overseeing the peer-review process for the issue.
In 2022, for example, IOP Publishing retracted nearly 500 articles in one go, of which the vast majority – 463 articles – came from the Journal of Physics: Conference Series. In 2024, meanwhile, the Swiss National Science Foundation stopped paying article-processing charges for articles published in special issues over quality concerns.
Yet the problem is not solely focused on conference proceedings and special issues. In 2023 the Public Library of Science (PLOS) retracted more than 100 papers from its flagship journal PLOS One over manipulated peer review. That same year, Hindawi and its parent company Wiley identified some 1200 articles that had compromised peer review and were retracted – a few months after Hindawi had announced that it would retract 511 articles across 16 journals for manipulated peer review.
Wiley then announced it would end the Hindawi brand name, costing the company up to $40m in lost revenue, and compelling it to integrate its 200 journals into Wiley’s 2000 journal portfolio. Yet the issues didn’t go away for Wiley, and in March 2025 the publisher retracted a further 250 papers.
Fighting AI with AI
Since the issue of papermills has come to a head in the last few years, publishers are fighting back (see box “How IOP Publishing is tackling the rise of papermills”) and the industry as a whole is taking action. The Committee on Publication Ethics (COPE) – a non-profit organization founded in 1997 that aims to define best practices and promote integrity in scholarly publishing – has issued new guidance on the use of AI in publishing.
Publishers are also pooling their resources into the STM Integrity Hub, which is run by the International Association of Scientific, Technical & Medical Publishers (STM). This cloud-based platform offers services that publishers can use to examine a variety of patterns that are indicative of papermills or other research integrity concerns, serving as an “early warning system” for integrity issues.
Antonia Seymour, chief executive of IOP Publishing, which is a member of STM, thinks that the rise in fraudulent papers is one of the most concerning issues for the whole scholarly ecosystem, with initiatives like the STM Integrity Hub being critical. “All publishers are being attacked by these bad actors trying to infiltrate the system with fraudulent papers,” she says. “The STM Integrity Hub is about combining signs of fraud not just from our corpus of content, but from other publishers as well.”
Other programmes supporting research integrity are Silverchair and Morressier, which check submission criteria such as missing author information (for example, e-mail address and institutional affiliation), missing ethics statements or missing keywords; and also confirms author identities using ORCiD. Somewhat ironically, they both use AI to beat AI by running the text through AI tools to check for tortured phrases, fraud and plagiarism as well as analysing citations.
Last year, the Science family of journals, run by the American Association for the Advancement of Science, adopted the use of Proofig, an AI-powered image-analysis tool, to screen for manipulation. ImageTwin is another programme that can detect image duplication and manipulation as well as spot AI-generated content.
Cabarnac, meanwhile, has worked with the French National Centre for Scientific Research (CNRS) to develop an online bibliography checker called bibCheck. “I’ve personally used bibCheck to flag hallucinated references in manuscripts I have reviewed,” says Cabarnac. “Finding these informed my decision to immediately reject.”
How IOP Publishing is tackling the rise of papermills
Many publishers have had to respond to the rise of papermills and fraudulent papers (see main text). That includes building up internal integrity teams and investing in new technology to manage the huge numbers of allegations and corrections.
“Retraction reporting is often reliant on self-declaration by publishers, therefore these figures are likely underestimating the scale of retractions out there,” says Kim Eggleton, research integrity manager at IOP Publishing, which publishes Physics World. “We’re of the opinion that retracted articles should be very clearly marked and we report all our retractions to Retraction Watch and PubPeer.”
Since 2023 IOP Publishing has also donated revenues from author-processing charges (APC) that it has earned from retracted works to charity, supporting Research4Life, which helps researchers in low- to middle-income countries to access published work.
Thanks to technological developments, and the work of some “scientific sleuths”, it is becoming easier to spot problematic papers. At the same time, that brings challenges to work through them all, contacting the authors and giving them a chance to explain.
At the start of 2021 IOP Publishing had one part-time position handling cases of misconduct; now it has one research integrity manager as well as six research integrity officers, all of whom are full time and fully trained to guidelines outlined by the Committee on Publication Ethics. The organization has also strengthened its screening process for conference organizers and invested in new technology.
IOP Publishing rejects about 55% of roughly 7000 submissions each month before they have even been sent for peer review. About 700 of those rejected papers are due to research integrity concerns. IOP Publishing’s research integrity team, which works on cases raised both before and after publication, typically has hundreds of papers under investigation at any one time.
But it is not only the papers themselves that can be fraudulent; there are even fraudulent peer-review reports. IOP Publishing has developed a machine-learning tool to detect duplicate peer-review reports – where the same reviewer report is sent for multiple submissions. The most egregious examples of this are “review mills”, organizations churning out fake reviews, often to inflate citations for a paying customer.
Until now, such patterns in reviewer reports had been difficult to identify, but the new tool automatically flags duplicate reviews to editorial teams. Indeed, since its pilot in 2024, the tool has processed around half a million reviewer reports dating back to 2020, identifying nearly 2500 cases where more than 60% of the content closely matched other reviews. These included instances in which reports were reused across multiple manuscripts or submitted under different reviewer names.
Any duplicate report submitted is flagged for investigation to IOP Publishing’s research integrity team. “This tool is a powerful addition to our efforts to defend high peer-review standards and weed out bad actors who try to manipulate the peer-review process,” says Eggleton. “It reflects our commitment to tackling unethical reviewing practices head-on and reinforces our role as a trusted, transparent and responsible publisher.”
Researchers are aware that the problem will not resolve itself and that publishers also cannot do all the fraud prevention alone. There are calls for researchers to undergo compulsory training in research ethics, and for journals to require raw instrument file datasets as a criterion for publication, as well as having dedicated conference sessions on combating AI risks. In 2025 Sense about Science and Taylor & Francis released a research integrity toolkit, created with and for early-career researchers, that answers common questions and provides practical advice.
Some have suggested that replication could and should be incentivized, perhaps by journals inviting a research group to replicate studies that are getting considerable attention in their journal (for example, studies with suspected AI-generated images), or inviting the replicating group to submit their replication for publication in the same journal. The replication study could then be linked to the original publication, and given an appropriate level of prominence.
But all these approaches take time and money. After several years of large numbers of retractions, in 2024 the number of retractions globally dropped back down to 5000 (see figure 1) – although many will still see that as being too much and others still estimate that figure to be much larger. The reality is that a lot of fraudulent material still exists in the scientific record, and may never come to light or be removed. Indeed, of the 782 papers that Bik found problematic in her analysis, only 177 have so far been retracted, with 42 having an expression of concern and 256 having been corrected. The remaining 307 remain in the scientific record untouched.
That is an issue that Seymour at IOP Publishing says must be resolved. While organizations like Clarivate are trying to clean up the scholarly record by potentially delisting a journal if it has found to be publishing too much fraudulent content, that can actually be a disincentive for publishers to go and find cases of fraudulent research in their journals. “Transparency and correcting the record is really important,” adds Seymour. “But at the moment, we’ve got quite a lot of sticks for publishers and perhaps not enough carrots.”
