Systematic research fraud has outpaced corrective measures and will only keep accelerating, according to a study of problematic publishing practices and the networks that fuel them.
The study, published August 4 in Proceedings of the National Academy of Sciences, examined research fraud carried out by paper mills, brokers and predatory publishers. By producing low quality or fabricated research, selling authorship and publishing without adequate quality control and peer review, respectively, these three groups were well known to produce a large volume of fraudulent research.
“This is a great paper showing how much fraud there is in the scientific literature. The paper also looks at different methods on how to detect problematic papers, networks and editors,” Anna Abalkina, a researcher at Freie Universität Berlin and creator of the Retraction Watch Hijacked Journal Checker, said.
Researchers and journalists have been looking into paper mills for more than a decade, as they have affected multiple publishers over the years, even before the high-profile retractions of thousands of articles from Hindawi journals.
The latest study confirms the large volume of suspected paper mill products have been doubling more than twice as fast as corrective measures — both official measures like retractions and unofficial channels like the number of crowdsourced comments flagging problematic papers on PubPeer.
“It’s like emptying an overflowing bathtub with a spoon,” coauthor Luis Amaral, a professor at Northwestern University, told us.
To combat this systemic fraud, the researchers call for a global change in research incentives.
Case studies
To look at cases of paper mill activity to inform their larger network design, Amaral and a research team of metascientists examined metadata from the journal PLOS One, and DOIs from IEEE conference proceedings, along with the Retraction Watch Database and PubPeer comments to compile networks of editors who were significantly more likely to accept articles that eventually got retracted or flagged for concern. (Disclosure: Our Ivan Oransky is a volunteer member of the PubPeer Foundation’s board of directors.)
PLOS One has been a known target of paper mill activity in the past. Amaral and colleagues identified networks of editors who accepted an abnormally high rate of papers that were later retracted. The 45 editors implicated in this network edited 1.3 percent of all articles, but 30.2 percent of all retracted articles. More than half of the editors also authored articles that were later retracted.
Renee Hoch, head of publication ethics at PLOS, told us that since the increased paper mill activity was detected four to five years ago, the company has implemented measures to investigate existing articles and paper mill networks, and thoroughly screen new papers, authors and editors before publication. Even before then, the publisher had expanded its publication ethics team to investigate papers.
“PLOS was used as a data source in this study in part because PLOS publishes all of our articles Open Access, we list the names of handling editors on our articles, and we allow bulk access to our article content and metadata,” Hoch said. “However, the issues that surfaced in this article are not specific to PLOS One and instead are affecting journals and publishers across the industry.”
The researchers conducted the same analysis on papers from Institute of Electrical and Electronics Engineers (IEEE) conference proceedings, which have long been a known target for paper mill activity. They found hundreds of conferences produced abnormally high proportions of papers that were later retracted, flagged on PubPeer, or flagged for using tortured phrases. Seven of these conference series were flagged every year the conference was held.
“We believe our preventive measures and efforts identify almost all papers submitted to us that do not meet our standards,” an IEEE corporate spokesperson told us. “To adhere to our standards, IEEE continuously inspects our digital library and acts accordingly when we become aware of possible issues with content, takes the appropriate level of care and time in our review, and, if necessary, retracts nonconforming publications.“
In addition to looking at individual journals and publishers, Amaral and colleagues used large databases such as Scopus and Clarivate’s Web of Science, along with PubPeer comments about suspected image duplication to map a network of papers across many large publishers like Springer Nature, Spandidos, Wiley and Elsevier. They found multiple clusters of duplicated images that seemed to be published close in time. Of papers with suspected image duplication, 34.1 percent have been retracted.
The authors’ approach of looking at image duplication networks is innovative and had not been done before, Abalkina said. “It is fascinating to see how paper mills expand like [an] octopus engaging more and more scholars in the network.”
Amaral and colleagues also analyzed the impact of fraud on certain subfields, focusing on RNA biology, which study coauthor Jennifer Byrne of the University of Sydney and others have closely examined for years. The researchers found compromised subfields had a retraction rate around 4 percent, while uncompromised ones remained lower, at about 0.1 percent.
And the researchers quantified journal hopping using a known paper mill broker, the Academic Research and Development Association (ARDA), which some of the study coauthors described in detail last year for Retraction Watch. ARDA’s website listed journals in which the organization guaranteed publication. As the listed journals were de-indexed from Scopus (at a much higher rate than other journals – 33.3 percent versus 0.5 percent), new journals would be added to the list.
Despite publishers’ current efforts, the researchers calculated that only 28.7 percent of suspected paper mill products have been retracted. The figure came from the researchers’ dataset of paper mill products and the Retraction Watch Database, using data through 2020, although data show paper mill growth since then has been rapid. They extrapolated from current trends that “only around 25 percent of suspected paper mill products will ever be retracted and that only around 10 percent of suspected paper mill products will ever reside in a de-indexed journal.”
These numbers likely underestimate the full extent of the problem, because they “rely on the instances of scientific fraud that have been reported,” they wrote in the paper.
What can be done?
Amaral compared the scale of the issue to combating the ozone hole over Antarctica, and said that the scale of the solutions need to match. “We need the biggest, most important stakeholders of science to come together to talk about what needs to be done, what standards need to be implemented and not wait for the problem to solve itself,” he told us. He named national organizations like the U.S. National Academies, the Chinese Academies of Sciences, and the U.K. Royal Society as stakeholders big enough to influence large organizations like publishers to act.
“They need to implement decisions, and they need to advocate strongly for those decisions to be adopted by journals, by funding agencies, by employers, universities, the national labs,” Amaral told us. “It’s not going to be an individual choice.”
However, individuals can play a role in advocating for change. Anyone can “press on policymakers to end the culture of hyper-competition in science,” Reese Richardson, a coauthor of the study and a postdoctoral researcher at Northwestern University, told us. “Scientists are all competing against each other for an increasingly scarce pool of resources,” he said.
Scientists can help address the problem in additional ways, Richardson told us. “What scientists can do in their own personal capacity is do post-publication peer review, and take a critical eye to the literature in their fields because it’s clear that we’re only detecting a tiny, tiny fraction of the problem,” he said.
He also suggested researchers develop “high throughput approaches to identify problematic articles” and metascientists “can use rare signals, like the comments that people leave on PubPeer observations about image misidentification and image duplication and instrument misidentification, tortured phrases … to understand what’s happening behind the scenes. That’s the job of meta scientists – to use public visible indicators to get at what’s invisible.”
We’ve covered a number of these high throughput approaches in the past, including one Richardson and Amaral developed with others to detect papers that misidentify microscopes used in their studies.
“We are part of a community that has been fighting for recognition of something that has been concerning us,” Amaral told us. “We are grateful for everything that everyone else has done, and to be a part of that, because science is very important to us and we want to do things to maintain the ideals of what science should be.”
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].