H. Studd et al/medRxiv 2026
Neither retractions, expressions of concern, nor other editorial notices seem to keep authors from continuing to cite problematic papers, according to a look at what happened to more than 170 articles by one author.
“After the public notification of integrity concerns about an article, it would be expected that other authors would no longer cite the article because it is unreliable,” write the authors of a new preprint. But that’s not what they found in a limited comparative study. Whether the study is generalizable has yet to be seen, says one other expert.
Four sleuths – the University of Aberdeen’s Hugo Studd and Alison Avenell and the University of Auckland’s Andrew Grey and Mark J. Bolland – charted citation data for 172 papers on clinical trials from Zatollah Asemi, a nutrition researcher at Kashan University of Medical Sciences in Iran, whose work has come under scrutiny.
Of these, 23 had been retracted, 38 had expressions of concern, 41 had some other type of editorial notice, and 70 had not been subject to any flag. The preprint authors sought to test how the type of editorial response affected citations, given “differences in their visibility or perceived seriousness.”
Bibliometric researchers sometimes refer to retracted or marked papers that continue to circulate as “zombie papers.” Data suggest the vast majority of retracted papers continue to be cited as if they had not been retracted. Even when citations of retracted papers do dwindle, it’s been unclear whether that’s due to the paper’s age rather than its editorial status.
The authors of the new preprint have been involved with the Asemi case — and this specific dataset — for years. In 2019, they flagged the 172 papers to the journals that published them with allegations of unethical conduct, results that didn’t match the study design, data irregularities and myriad other integrity issues. For those papers on which editors have taken action, notices have been published an average of five years after publication. As Grey told us back in 2021 in regards to journals’ response times on Asemi’s papers: “Yep, pretty slow.”
The randomized controlled trials reported in the articles examined had collectively been cited more than 10,000 times in more than 6,000 articles. For both the flagged articles and those with no notices, “citations increased steadily, peaking 45-65 months post-publication,” with similar declines thereafter. The researchers found “little evidence that publication of an editorial response, whether it was retraction, publication of an EoC or issuing of an editorial notice, had any meaningful effect on the rate of citation.”
In a jointly signed response to our questions, the four authors told us they were unsurprised by their results, given the years of delay between publication time and notice. They were also unsurprised that the black-marked papers continued to be referenced in new work. “By then, these [trial] publications were probably imbedded within the literature and databases, resulting in continued citation regardless of editorial action,” they wrote.
The authors told us they believe the results of the paper are generalizable, even though it covered one author’s body of work. That’s because the trials spanned 65 journals and were published by 25 publishers.
Jodi Schneider, director of the Information Quality Lab at the University of Wisconsin-Madison, is not as convinced it’s generalizable. But, she added, “Their continued analysis of this case study is quite valuable for the research community — with sobering results.”
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
Evidently papers by Zatollah Asemi are cited not because an author thinks that their results are genuine and reliable, but for some other reason.
Playing the devil’s advocate here. How many of those citations are from other papers in the same area and how many are from papers about replication problems? I’m bringing this up because I found some evidence of this for a paper I used for one of my classes. The paper was still cited quite a bit but most of the new citations were about methodology and replication crisis.
Unfortunately, not at all surprising considering how obscure the retraction notices can be. I’ve been looking into the Guihherme Malafaia case with his 47 retractions in 2025 from a single Elsevier journal (presently at 13th place on the RW leaderboard). While the articles are clearly marked on their landing page, pdf, and in the individually downloaded RIS citation, when I downloaded RIS bulk citations from WoS for all 47, only deep in the metadata in the field “article type” did it say Retracted Article. The retracted article alert in Endnote was not triggered.
Ironically that might be just as well for this tranche of retracted articles since it may well be that few or even none of Malafaia’s co-authors were in on his shenanigans as corresponding author.
But the system of secondary data providers seems to be a problem, especially if one downloaded the article before it was retracted.
Thanks for the comments. Not reported in our preprint, but in earlier work we did examine whether removing citations by authors from the same country affected our results. We were unable to show an effect on our findings. We haven’t explicitly looked for citations related to methods, research integrity issues or replication issues. However, reviewing around 11,000 articles is beyond our capability. Based on simply scanning some of the titles, journals and authors, most citations are likely to referencing the articles as if there were no integrity concerns. See our earlier paper in another case, where we carefully examined this point, showing that the vast majority of continuing citations did not mention integrity concerns (https://doi.org/10.1080/08989621.2021.1886933)
Yes, the transfer of metadata to indexing services has been problematic, see Caitlin Bakker’s work (https://doi.org/10.1016/j.jclinepi.2024.111427). One of the reasons for our research was that Springer Nature and Thieme had posted many editor’s notes/notes of concern on article webpages for these trial reports, which were not indexed, and seldom updated even years later. Let’s hope NISO guidance makes a difference (https://www.niso.org/standards-committees/crec).
Jodi Schneider wonders whether our work is generalizable. We think that it should be regarded as such until evidence is presented that it is not. We would like to encourage others to address this question using different datasets.
Interesting finding. It really shows that once a paper is out in the academic ecosystem, citations tend to follow relevance and visibility more than reputation signals like “black marks.” Researchers often cite work based on usefulness to their own study rather than digging into the paper’s history or any attached concerns.
That said, it also raises a concern—if warning signs don’t impact citation behavior, it could mean problematic research continues to influence future studies. Maybe there’s a need for better visibility or clearer communication around these issues so researchers can make more informed decisions when citing sources.
One of the unfortunate aspects of modern academic publishing is that a lot of academics don’t read the papers that they cite. Things like when they state that there has been n previous studies in the area (lots of citations).
Even if someone does read all the papers they cite, they are not likely to keep tabs on the status of each one to see whether it has been corrected or retracted.