Guest post: A look behind the scenes of bulk retractions from Sage

When I began my graduate work almost 15 years ago, retractions of papers in academic journals were rare, reserved mainly for clear misconduct or serious errors. Today, rarity has given way to routine, with retractions coming more often and increasingly in bulk. 

Sage is not immune to large-scale retractions, nor are we passive observers of their growth. As Retraction Watch wrote, we were “one of the first publishers to recognize large-scale peer review manipulation and begin retracting papers in bulk nine years ago.” Recently, we issued some major retractions; just in the last few months, we put out 37 from Proceedings of the Institution of Mechanical Engineers and 21 from Concurrent Engineering. And there are more to come. 

While we don’t celebrate this type of action, the news is not all bad. The high numbers of retractions at times reflect a problem of industrialized cheating, but also, as in our case, a belief that rigorous scholarship – robustly reviewed by researchers who are experts in their fields – can and should improve the world. Sage was founded on this principle, and it guides everything we do. 

We take our role of vigorously correcting the academic record very seriously because we believe in the scholarly process. We also know that every part of the process is managed by humans with biases (conscious or unconscious), agendas, heavy workloads, and – at times – dubious incentives.  

As research integrity manager at Sage, I work to safeguard the credibility of the research published in more than 60,000 articles every year across more than 1,100 journals. In my role, I see a lot of unethical practices: peer review rings, where researchers unfairly influence the review process; paper mills that produce mass-fabricated research papers, and the brazen trend of selling authorship or entire papers on private or public forums. When it comes to preventing and correcting this type of action,  much goes on behind the scenes.

What goes into a bulk retraction?

Behind every set of bulk retractions are teams of trained individuals using specialized tools, carefully going through several steps of a standardized process in line with industry best practice, often in concert with multiple parties representing various institutions.

My team at Sage comprises specialists who can recognize and resolve ethical breaches to help ensure that the articles published in our journals are based on sound, rigorously reviewed science. A large part of this lies in the unglamorous work of prevention – guidance and policies that aim to make publication impermeable to large-scale unethical activities. This means keeping authors and editors abreast of best practices and training editors and journal managers on research integrity matters such as paper mills or political pressure on research.

Of course, enforcing these policies and practices to perfection is impossible. But just as there are those who are using all the tools at their disposal to game the system, we’re using everything at our disposal to uncover questionable practices. We use a number of tools to spot common breaches of publishing ethics, and we’re constantly fine-tuning our methods to stay current with evolving trends in this field. For example, one of these tools scrutinizes submissions and reviews for signs of suspicious activity. Clues often come in the form of non-institutional email addresses or peculiar IP addresses used during manuscript submission. We’ve also recently joined a pilot program as part of the STM Integrity Hub, a cross-publisher initiative, which detects unusual activity and misconduct across publishers such as duplicate submissions across journals and publishers – a tactic  often employed by paper mills.  

When misconduct isn’t caught before publication, my team commits considerable time and resources to thoroughly – sometimes painstakingly – investigate so that we can make the best decisions following industry standards from organizations such as the Committee on Publication Ethics (COPE). This process is not always linear or straightforward but there are steps common to most incidents.

We begin by determining the the scope of the issue by listing affected submissions and published articles. This can be a challenge, as sometimes the list spans multiple journals. We then brief the editor(s) and if there are irregularities in peer review or acceptance processes, we work with them to understand what may have happened. We scrutinize the data from each submission, looking for any patterns that are suggestive of third-party interference, a common source or telltale paper mill tactics. We also examine the entire journal for explanations behind this unusual activity and look for indications of genuine errors before assuming wrongdoing.

Next, we contact the authors about our findings and give them a chance to respond. If authors can’t be verified or remain unreachable, we proceed without their involvement. (In many cases it becomes very clear when a third party has tampered with our process, but there are times when it’s less clear-cut.

We make the decision to retract with the editor based on COPE’s retraction guidance. For example, if there is clear evidence of an experimental error, data fabrication, or a compromised peer review process (e.g., a peer review ring), we will retract. Finally, when notices are ready we allow authors the opportunity to note whether they agree or disagree with our decision, for fairness and transparency.

Throughout every step of the process, we ask ourselves: Is a completely new review warranted? Is this a valid concern that impacts the publication? Is this an honest error, or is it a pattern of behavior? 

While it might appear that we’re dragging our feet before making a final decision, sifting through all the information to answer these questions can take a long time. But we strive to be as transparent as possible along the way. For example, when we do find sufficient indications of error or unethical practices, we are not afraid to issue Expressions of Concern to let the public know that an investigation is warranted. When our investigation is complete, we issue retractions or corrections as soon as possible when warranted or, in other cases, publicly state that we no longer see issues with the papers.

Still, with this level of attention given to each case, we can’t approach all cases as quickly as we would like, and we prioritize areas that affect people, policy, and the future of research. 

Why do we do it?

Sometimes the process of a bulk retraction is initiated when a specialist or an industry “sleuth”  notices a dubious pattern, such as  image manipulation, and comes to us with a list of problematic articles with evidence to indicate why we should take action. But this type of ‘package-wrapping-and-leaving on the doorstep’ is rare.

Instead, most of the time, a bulk retraction occurs when we notice something off with just one paper – a request to change the author list after acceptance, a peer reviewer contacts us to let us know that a paper was published despite their seeing a need for substantial revisions, etc. In those cases, we start with the dubious paper and keep pulling the string for as long as necessary. We could easily stop at the first article and move on and save hours of staff time without anyone noticing (at least in the short term). But doing so would mean that we are evading one of our central responsibilities as an academic publisher. 

As an independent company, we don’t have to bend to the views of shareholders. This gives us the freedom to prioritize academic integrity, make improvements, and shake things up if needed. Sometimes, this means making tough and even unpopular decisions to upload scholarly standards. For instance, my team and I have had to end publishing partnerships with highly regarded scholars when we discovered their unscrupulous behavior.  

This independence extends to our editors. We uphold editorial independence enthusiastically and unapologetically. While we offer help, training, and advice, it’s the editors who have the final say over article selection and peer review, as long as they are academically rigorous and work in accordance with best practice, including guidelines from the Committee on Publication Ethics (COPE). 

Post-publication, editors maintain authority over decisions to publish expressions of concern or other similar notes as well as retractions, again, as long as they are in accordance with COPE. We often help guide them through this process, and in the vast majority of ethics cases, are confident in the decisions they make. From time to time, when the editor is not willing to take action or is implicated in the unethical behavior, we believe it is our duty to step in. And we do. Finding the balance between accommodating editor wishes and gatekeeping is not easy, but we prioritize maintaining the integrity of the scholarly record as we navigate that balance. 

I often compare our role to the news media; when research is published in our journals, it goes “on the record.” Once on the record, it changes the direction of future research for years to come and impacts policy and patient protocols, so “bad” science can lead to bad research, bad policy, and bad protocol. It can also foster increased mistrust in science and so, by continuously correcting the scholarly record, we also help to maintain trust in honest research.  

In my experience, retractions also signal trust to our community of researchers and editors – they can feel confident that their rigorous research, which they have spent weeks, months, or more carefully conducting, writing, and/or reviewing, is in good company. As no publisher is immune to rising misconduct or paper mills, those that aren’t increasing their number of retractions to reflect that might not be vigilant enough. A clear message that we hear time and time again and was a major theme at this year’s STM conference is that we should be cleaning up the record and retracting more articles, more quickly.

Making Integrity Integral

But let’s keep in mind, the vast majority of scholarly research is conducted, reviewed, accepted, and published in good faith. And when retractions do occur, especially one-off retractions, they are often the result of error or oversight rather than something more nefarious – a teachable moment and a learning experience for researcher and editor alike.

I see the rise in bulk retractions as a hopeful sign for the future of scholarship. It means that we aren’t content to sweep bad published research under the rug. And, I can say with confidence, that with each new bulk retraction, we are refining, and sometimes redefining, our processes and practices to improve the system before tomorrow’s research is submitted to our journals.

While our processes and rigor are our own, we’re not unique; our colleagues at other reputable academic publishers are also acting to protect the integrity of the research enterprise. I’ve found that working with other publishers and bodies in the industry, such as the STM Integrity Hub, to regularly share new concerns and best practices is the best way to stop unethical research behavior before it gets accepted. 

My interest in research integrity comes from a deeply personal place of needing to be part of an ethical research ecosystem that does not infringe on the rights of others. With my work, I hope to instill in others a similar way of working so that students graduating today, 15 years from now, and beyond, publish the research that will improve the world.

Adya Misra is Research Integrity & Inclusion Manager at Sage Publications.

Update, 1800 UTC, 2/9/24: When originally published, this post contained a typo in this sentence: “Of course, enforcing these policies and practices to perfection is possible.” The correct sentence now reads: “Of course, enforcing these policies and practices to perfection is impossible.”

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in The Retraction Watch Database, you can let us know here. For comments or feedback, email us at [email protected].