Retraction Watch

Tracking retractions as a window into the scientific process

What should you do if a paper you’ve cited is later retracted?

with 12 comments

RW logoWe all know that researchers continue to cite papers long after they’ve been retracted, posing concerns for the integrity of the literature. But what should you do if one of the papers you’ve cited gets retracted after you’ve already cited it?

We posed this question to some members of the board of directors of our parent non-profit organization, who offered up some valuable advice based on many years of experience working at journals and organizations such as the Committee on Publication Ethics (COPE).

The first step: Determine whether the fact a reference has been retracted has any impact on the conclusions of your own paper. From Elizabeth Wager, publications consultant, Sideview; former chair, COPE:

For articles presenting original research it’s unlikely that any other cited work would alter the conclusions, since these will usually relate only to the new work, and that won’t be affected. However, it might slightly change the emphasis of the Discussion section since a retraction will remove a citation that presumably [either] supports or opposes the author’s own work.

However, for review articles, and particularly systematic reviews that combine study findings using meta-analysis, the removal of one study might well affect the conclusions, so authors of such articles should react promptly and, ideally, re-do their meta-analysis.

For instance, Wager noted that after the retraction of dozens of papers by Joachim Boldt, some evidence about the potential dangers of a commonly used synthetic fluid had to be revised.

But when journals issue undetailed retraction notices (such as “this article has been withdrawn”), what can authors do to determine a retraction’s impact on their work?

Steven Shafer,  editor in chief, Anesthesia & Analgesia; professor of anesthesiology, Stanford University:

I suggest that the author contact the editor for more information. The author can also contact the author of the retracted paper to ask what is going on.

Elizabeth Wager:

Generally, (and according to the [Committee on Publication Ethics] guidelines) retractions indicate that an entire article is unreliable rather than certain aspects of it.  If only a small part of a paper is affected this should normally be indicated by a correction rather than a retraction.  So, I would advise authors to assume the whole article is affected.

David Vaux, deputy director and joint division head, Walter & Eliza Hall Institute of Medical Research, Melbourne, Australia:

If the retraction notice isn’t detailed enough, or even if it is, I would err on the side of discounting all the results in the retracted paper.  (This answer might depend on whether the authors had retracted the paper themselves for an inadvertent mistake, versus a retraction for misconduct). Regardless, I firmly believe journals should include a full explanation in the retraction.

Are there any circumstances in which authors should consider correcting a paper when a cited reference is retracted?

Elizabeth Wager:

Ideally, ANY time they discover they have cited a retracted article, but especially for review articles where this may affect the conclusions of the review.

David Vaux:

Authors should [consider] correcting their own paper when a cited reference is retracted if not to do so would mislead the readers. I imagine the most common time this occurs is when there is an editorial, news & views, or review that is based on the retracted article.

Miguel Roig, professor of psychology, St. John’s University, New York:

My bet is that, data from most retracted studies are cited in the context of many other converging sources of evidence that an author uses in support of his/her hypothesis/conclusions. In such cases, just like in situations in which the data cited are later found to be irreproducible or a ‘dead end’, it is probably best to let the literature correct itself. An argument can be made to issue a correction alerting readers that research cited as evidence is no longer part of the scientific record. The benefit of doing so is to help insure that such retracted papers are not cited again. However, I am concerned that such a benefit might be outweighed by the increase in ‘literature noise’. In either case, upon discovering a retraction that has been cited in one of their papers, authors should always contact the editors of the journal in which the paper with the cited retracted study was published in and come to an agreement as to a proper course of action.

In situations when the retracted data represent a key element of the new research and compromise its conclusions, that is, if the retracted data were fabricated, falsified, plagiarized or even based on honest error, then some action must be vigorously sought. A correction or some other alerting mechanism, perhaps even a retraction- is especially important in the latter scenarios when the newer untrustworthy study outcomes might conceivably put the public at risk.

Steven Shafer:

I am not aware of a single case in which an author requested retraction of a paper simply because a previously published paper that was cited had been retracted. However, if the author believes that his or her own paper has been seriously compromised by the retraction of a fundamental paper on which the work was based, then a letter to the editor seems appropriate….In general this seems unlikely in primary research papers where authors are putting forth their own research. The data are the data, and should be unimpugned. The exception might be where the retraction was based on an assay that was found to be flawed, and the authors relied on the assay for their work. However, for a meta-analysis, where a significant portion of the data are no longer considered valid, a correction, or even a retraction, might be in order.

(As an aside, we know of one example where authors had to retract papers that included a figure and materials from a newly retracted paper.)

Another key question is: What was the paper retracted for? As Miguel Roig told us:

For example, some studies have been retracted because the data were obtained without proper consent or because some type of authorship dispute. Suppose that one such retracted study represents a key element in a newly published study. Is a correction necessary in this case?  Frankly, I think it depends on the nature of the ethical breach of the retracted study. On one end of the spectrum I am thinking of Mumford et al’s study in Science and Engineering Ethics, which was later retracted for [Institutional Review Board] violations.  If data from that study had been key in my own research, I would alert the editor of the journal that published my paper, but I might urge him/her not take any action on the matter as I believe that the retraction was completely inappropriate. In fact, in that particular case, I would not even mind citing data from the retracted study prospectively (though, obviously, I would do so with the caveat that it had been retracted and indicate the reasons for the retraction). On the other side of the spectrum, if a study is cited and later found to be retracted for egregious ethical human subjects violations, then a correction or similar alert ought to be issued if only to alert others that such data, even if otherwise valid, should not be used as it was obtained unethically.

What should you do if a cited reference is flagged with an Expression of Concern?

That’s a “trickier” situation, said Wager:

In that case, if authors have cited the article it hasn’t been proven to be unreliable so they must use their best judgement. A citation in a Discussion section should probably be left, but if a pivotal study included in a meta-analysis has an Expression of Concern, authors of the review article should consider doing a “sensitivity analysis” to see if removing findings from this study affects the conclusion of their review and, if so, might consider contacting the journal to ask for advice and perhaps adding a note to their review to alert readers to the fact that concern has been expressed about an included study.  However, the wording needs to be careful, as the study hasn’t actually been retracted, and an EoC usually signals uncertainty on the part of the journal, so people shouldn’t jump to conclusions that it is definitely unreliable.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.

Written by Alison McCook

November 1st, 2016 at 2:30 pm

Comments
  • Juan Reza November 1, 2016 at 3:11 pm

    This question gives us something to do proactively, rather than just wondering how to react in response to the (hopefully rare) event of a paper you cited is retracted. When preparing and checking all of the citations in your paper, consider each citation one by one. For a given citation, consider what should become of the claims or proofs of you paper if that citation were deleted. Is your work entirely dependent on the correctness (replicability, etc.) of that citation? Or does it help the reader understand a point you made? If the former (your paper is too dependent on a particular reference’s facts), consider finding a corroborative reference. If the later, your claims are reasonably safe. But in either extreme, consider keeping a copy of the document or pertinent passages so that you will have the ability to show future readers, investigators, etc. what you referenced, even when a reference is retracted or becomes hard to find (for other reasons). … my opinion.

  • Miguel Roig November 1, 2016 at 5:46 pm

    With respect to the following sentence in my first comment: “The benefit of doing so is to help insure that such retracted papers are cited again”. I should have written: “The benefit of doing so is to help insure that such retracted papers are NOT cited again”.

    • Ivan Oransky November 1, 2016 at 5:51 pm

      Fixed!

  • Miguel Roig November 2, 2016 at 8:22 am

    Just a thought: As retractions, corrections and the like appear to be more common, I can envision a service whereby each journal’s data base of published papers could be searched for citations in those papers for which a correction, retraction, expression of concern has been issued. A publisher could then flag that citation in the PDF and html versions of the paper with some universally agreed code, such as R for retractions, C for corrections, etc. In that way, future downloads or views of a paper with a problematic citation would be readily evident to the reader.

    I better get back to work …

  • Miguel Roig November 3, 2016 at 6:57 am

    Interesting! Thank you for the link and for correcting my earlier comment, Ivan. I suspect that the uneven adoption of the service has to do with its cost. C’est la vie.

  • psyoskeptic November 5, 2016 at 8:54 pm

    It seems odd to me how this question is being handled given that researchers will often selectively cite anyway. Giving advice on how to cite a retracted article has to be set in the context of why some articles are, or are not cited in the first place.

  • Frasier November 6, 2016 at 9:50 am

    The underlying premise is that research builds upon the results of previously published research.

    But to the extent that the previously published research is fraudulent or irreproducible, and the new research does not, in fact, need to be retracted when its citation basis is retracted, then either
    – the cited works were not, in fact, the basis for the new work, or
    – the new work is not actually supported

    In the first case, the reader may wonder why the citations were included at all.

    In the second case, the reader may wonder whether the new work is incorrect, or whether the new work is correct even though it is based on incorrect or fraudulent publications.

    • KeithT November 8, 2016 at 6:21 pm

      It should at least be noted that an

      Frasier
      The underlying premise is that research builds upon the results of previously published research.
      But to the extent that the previously published research is fraudulent or irreproducible, and the new research does not, in fact, need to be retracted when its citation basis is retracted, then either
      – the cited works were not, in fact, the basis for the new work, or
      – the new work is not actually supported
      In the first case, the reader may wonder why the citations were included at all.
      In the second case, the reader may wonder whether the new work is incorrect, or whether the new work is correct even though it is based on incorrect or fraudulent publications.

      Could be for padding and appearances. Or background info. Or definitions of terminology.

      Publishers and authors should at least be noted that a referenced article has been retracted, then the reader can decide if that retraction matters for the reader’s purpose in reading the article.

      There is no reason for the electronic forms of articles to fail to contain notes of retractions issued for referenced articles since the updating of references can be updated.

  • Frasier November 6, 2016 at 10:04 am

    On whether or not to cite Newton:

    A body by accelerated motion may trancend all distance in any finite tim assigned also it may becom infinitely long. …

    The world might have been otherwise then it is (because there may be worlds otherwise framed then this) Twas therefore noe necessary but a voluntary & free determination that it should bee thus.

    http://fedora.dlib.indiana.edu/fedora/get/iudl:1774499/SCREEN

  • KeithT November 8, 2016 at 6:09 pm

    As a recently retired IT professional, it occurs to me that there should be an academia-wide indexed database of all the articles retracted.

    For articles viewed as webpages, an independant browser add-on could be written that would, upon detecting an academic citation, automatically check it against that retraction database and produce a notation about the retraction (perhaps change the font of the citation to be or perhaps insert a “[retracted]” into the webpage displayed.

    For articles viewed as PDFs, journal publishers could periodically process the footnote files for their various articles against this database, using the results to insert a “[retracted]” after the citations that have been retracted. Then recreate the article PDFs using the original body with the newly updated footnotes.

    The process is fully automated once they’ve done the setup.

    —-
    You might want the database to be for all peer-reviewed articles published, with a flag to be set if the article is retracted or gains a special caution.

    You want the database “academia wide” because journal articles in one field might reference articles from another field or published in a different language.

    The US National Library of Medicine at the National Institutes of Health’s article database might be a good starting point.

    • KeithT November 8, 2016 at 6:14 pm

      Where it says, “change the font of the citation to be or perhaps insert a “[retracted]”” it should say “change the font of the citation to be ‘strikeout’ or perhaps insert a “[retracted]”.

  • Post a comment

    Threaded commenting powered by interconnect/it code.