What should you do if a paper you’ve cited is later retracted?

We all know that researchers continue to cite papers long after they’ve been retracted, posing concerns for the integrity of the literature. But what should you do if one of the papers you’ve cited gets retracted after you’ve already cited it?

We posed this question to some members of the board of directors of our parent non-profit organization, who offered up some valuable advice based on many years of experience working at journals and organizations such as the Committee on Publication Ethics (COPE).

The first step: Determine whether the fact a reference has been retracted has any impact on the conclusions of your own paper. From Elizabeth Wager, publications consultant, Sideview; former chair, COPE:

For articles presenting original research it’s unlikely that any other cited work would alter the conclusions, since these will usually relate only to the new work, and that won’t be affected. However, it might slightly change the emphasis of the Discussion section since a retraction will remove a citation that presumably [either] supports or opposes the author’s own work.

However, for review articles, and particularly systematic reviews that combine study findings using meta-analysis, the removal of one study might well affect the conclusions, so authors of such articles should react promptly and, ideally, re-do their meta-analysis.

For instance, Wager noted that after the retraction of dozens of papers by Joachim Boldt, some evidence about the potential dangers of a commonly used synthetic fluid had to be revised.

But when journals issue undetailed retraction notices (such as “this article has been withdrawn”), what can authors do to determine a retraction’s impact on their work?

Steven Shafer,  editor in chief, Anesthesia & Analgesia; professor of anesthesiology, Stanford University:

I suggest that the author contact the editor for more information. The author can also contact the author of the retracted paper to ask what is going on.

Elizabeth Wager:

Generally, (and according to the [Committee on Publication Ethics] guidelines) retractions indicate that an entire article is unreliable rather than certain aspects of it.  If only a small part of a paper is affected this should normally be indicated by a correction rather than a retraction.  So, I would advise authors to assume the whole article is affected.

David Vaux, deputy director and joint division head, Walter & Eliza Hall Institute of Medical Research, Melbourne, Australia:

If the retraction notice isn’t detailed enough, or even if it is, I would err on the side of discounting all the results in the retracted paper.  (This answer might depend on whether the authors had retracted the paper themselves for an inadvertent mistake, versus a retraction for misconduct). Regardless, I firmly believe journals should include a full explanation in the retraction.

Are there any circumstances in which authors should consider correcting a paper when a cited reference is retracted?

Elizabeth Wager:

Ideally, ANY time they discover they have cited a retracted article, but especially for review articles where this may affect the conclusions of the review.

David Vaux:

Authors should [consider] correcting their own paper when a cited reference is retracted if not to do so would mislead the readers. I imagine the most common time this occurs is when there is an editorial, news & views, or review that is based on the retracted article.

Miguel Roig, professor of psychology, St. John’s University, New York:

My bet is that, data from most retracted studies are cited in the context of many other converging sources of evidence that an author uses in support of his/her hypothesis/conclusions. In such cases, just like in situations in which the data cited are later found to be irreproducible or a ‘dead end’, it is probably best to let the literature correct itself. An argument can be made to issue a correction alerting readers that research cited as evidence is no longer part of the scientific record. The benefit of doing so is to help insure that such retracted papers are not cited again. However, I am concerned that such a benefit might be outweighed by the increase in ‘literature noise’. In either case, upon discovering a retraction that has been cited in one of their papers, authors should always contact the editors of the journal in which the paper with the cited retracted study was published in and come to an agreement as to a proper course of action.

In situations when the retracted data represent a key element of the new research and compromise its conclusions, that is, if the retracted data were fabricated, falsified, plagiarized or even based on honest error, then some action must be vigorously sought. A correction or some other alerting mechanism, perhaps even a retraction- is especially important in the latter scenarios when the newer untrustworthy study outcomes might conceivably put the public at risk.

Steven Shafer:

I am not aware of a single case in which an author requested retraction of a paper simply because a previously published paper that was cited had been retracted. However, if the author believes that his or her own paper has been seriously compromised by the retraction of a fundamental paper on which the work was based, then a letter to the editor seems appropriate….In general this seems unlikely in primary research papers where authors are putting forth their own research. The data are the data, and should be unimpugned. The exception might be where the retraction was based on an assay that was found to be flawed, and the authors relied on the assay for their work. However, for a meta-analysis, where a significant portion of the data are no longer considered valid, a correction, or even a retraction, might be in order.

(As an aside, we know of one example where authors had to retract papers that included a figure and materials from a newly retracted paper.)

Another key question is: What was the paper retracted for? As Miguel Roig told us:

For example, some studies have been retracted because the data were obtained without proper consent or because some type of authorship dispute. Suppose that one such retracted study represents a key element in a newly published study. Is a correction necessary in this case?  Frankly, I think it depends on the nature of the ethical breach of the retracted study. On one end of the spectrum I am thinking of Mumford et al’s study in Science and Engineering Ethics, which was later retracted for [Institutional Review Board] violations.  If data from that study had been key in my own research, I would alert the editor of the journal that published my paper, but I might urge him/her not take any action on the matter as I believe that the retraction was completely inappropriate. In fact, in that particular case, I would not even mind citing data from the retracted study prospectively (though, obviously, I would do so with the caveat that it had been retracted and indicate the reasons for the retraction). On the other side of the spectrum, if a study is cited and later found to be retracted for egregious ethical human subjects violations, then a correction or similar alert ought to be issued if only to alert others that such data, even if otherwise valid, should not be used as it was obtained unethically.

What should you do if a cited reference is flagged with an Expression of Concern?

That’s a “trickier” situation, said Wager:

In that case, if authors have cited the article it hasn’t been proven to be unreliable so they must use their best judgement. A citation in a Discussion section should probably be left, but if a pivotal study included in a meta-analysis has an Expression of Concern, authors of the review article should consider doing a “sensitivity analysis” to see if removing findings from this study affects the conclusion of their review and, if so, might consider contacting the journal to ask for advice and perhaps adding a note to their review to alert readers to the fact that concern has been expressed about an included study.  However, the wording needs to be careful, as the study hasn’t actually been retracted, and an EoC usually signals uncertainty on the part of the journal, so people shouldn’t jump to conclusions that it is definitely unreliable.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.