Too many papers cite retracted research — even after it’s been retracted. It’s a problem. It can be especially a problem in clinical fields, where patient care is at stake. Recently, Richard Gray at La Trobe University in Australia and his colleagues examined the scope of the problem in the nursing field, noting how many systematic reviews included findings from retracted clinical trials. We spoke with Gray about their findings, published in the International Journal of Nursing Studies — and what they might mean for the safety of patients.
Retraction Watch: Retractions are a concern in any field, but as you note, when clinical practice is at stake, it can be particularly worrisome. Do you think your findings raise any potential concerns about patient safety?
Richard Gray: We identified 23 reviews that included a retracted trial. In a clinical discipline such as nursing, practitioners rely heavily on systematic reviews to inform their care and treatment decisions. If a review includes a study that is retracted, the integrity of the review must be challenged. There is no established mechanism to do this. As a consequence, clinicians may continue to base decisions on evidence from systematic reviews that is unsound. This is a concerning – largely unrecognised – threat to patient safety.
RW: You don’t list the reviews that cite retracted trials so you don’t “name and shame” them. But given that these papers do affect patient safety, do you see an argument for making that information public? How have readers reacted to your decision to not name the affected reviews?
RG: To date we have not had any feedback from readers about our decision not to include a list of included reviews. We did consider carefully if we should publish a list of the 23 systematic reviews we identified that included a retracted trial. And yes, we agree there is a strong case to publish the list, because ultimately these reviews impact on clinical nursing practice. In the spirit of transparent reporting, publishing would also enable readers to check the accuracy of our work. However, the aim of this study was to test a hunch; that retracted trials get included in systematic reviews (before and after they are retracted). Authors of the five reviews that cite already retracted trials had clearly made an error and we felt that the most appropriate course of action was for us was to write to journal editors informing them of our observation. We were less clear about how to deal with the 18 reviews we identified that included a trial where it appeared that the retraction happened after the review was published. Whilst authors may not have made an error, we felt that we still had a duty to write to these editors informing them of our observations. We did this after the publication of our study.
RW: Have you gotten any responses from the journals you’ve contacted regarding the reviews that cite retracted trials? If so, what have they been?
RG: On the 7th December 2017, we wrote to the editors of journals that published a review that included an already retracted trial; four of five responded. A follow up email was sent to the sole editor who did not respond. Three editors indicated that they would investigate the matter further. The fourth editor informed us that they intended to inform the authors of the review so they were aware. They went on to state that “of course, we can’t control retractions but the authors should definitely be aware.” This response did concern us as it seemed to indicate a failure to understand how to respond to the report of a significant error in a paper published in their journal. We thought we should check the status of the five manuscripts a month after writing to editors. Perhaps not surprisingly, no expressions of concern, erratums, or retraction notices have appeared against any of the five papers.
RW: You note an interesting question: Should a review that cites a retracted paper be retracted? Some journals have taken that step in the past. What is your opinion in this matter?
RG: Where review authors included an already retracted paper this is clearly an error and the quality and integrity of the review is compromised. It seems to us that retraction would, generally, be appropriate. Of course, this is a decision that individual journal editors have to make, in consultation with authors (and potentially author employers and study funders). Where retraction of a trial occurred after the review was published, authors have not made an error. That said, retraction of an included trial does still challenge the integrity of the review. Whilst it may seem harsh on review authors, there remains a compelling case that reviews should still be retracted. Ultimately, whether a researcher or practicing nurse, patient safety must be a primary concern and unsound reviews cannot continue to inform clinical practice, and consequently should be removed from the evidence base.
RW: Five reviews cited retracted clinical trials after they’d been retracted. This is a problem we’ve noted as well (and a major reason for our database). What do you think can be done to improve awareness of retractions among review authors (and the editors and reviewers who check their submissions)?
RG: We have given careful consideration to practices that could be adopted by authors, reviewers and editors to minimise the inclusion of retracted clinical trials in systematic reviews.
- A retraction filter (or whatever mechanism the database in question allows) must be applied to the end output of any search strategy.
- Journals/databases must make retractions more visible (step 1 above depends on it).
- Collaborations (e.g. Cochrane, Campbell, The JBI) need to incorporate into their handbooks directives around retraction. For example, a scan for retractions after data sourcing; a scan for retractions before data extraction; a scan for retractions before review submission.
- The reporting guidelines for systematic reviews (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, PRISMA) needs to include an item stating that authors have checked if any included studies have been retracted.
- Journal editors should require authors, when submitting manuscripts, to confirm that they have checked that none of the included studies have been retracted. Authors should also include a statement in the paper stating they have done this.
- Proofreaders may also have an important role to play. For example, authors of one review included in their reference list a citation that clearly indicated the reference was for a retracted paper. Proofreaders could be trained to spot and report these anomalies.
We hope our work in this space helps create awareness of this important issue.
Additional point: There is – as far as we are aware – no mechanism for alerting journal editors that a paper in a review they have published has been retracted. We argue that retracting editors have a duty to the scientific community (and in clinical disciplines to patient safety) to review the citations of retracted papers, identify any (systematic) reviews, and then write to editors who published the review informing them of the retraction.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here. If you have comments or feedback, you can reach us at firstname.lastname@example.org