Last February, Richard Pollock was reading a review in the Cochrane Database of Systematic Reviews — a prominent resource for evidence-based medicine — when he spotted an error.
In the first figure, which compared the effectiveness of two different treatments for the most common form of liver cancer, a label was switched. The error made it seem like the “worse” treatment was better than the more effective option.
Pollock, a health economist, was concerned enough to send an email to Omar Abdel-Rahman, the corresponding author, on February 20th. Abdel-Rahman, an oncologist at the University of Alberta, wrote back the next day, saying he would review the comments with experts at Cochrane, “and if there is any typos in the publication, it will be corrected immediately.” Emails seen by Retraction Watch show that, when replying to this email, Abdel-Rahman copied one of Cochrane’s editors, Dimitrinka Nikolova.
Months passed. Pollock sent another email to Abdel-Rahman and two Cochrane editors — Nikolova and Christian Gluud — on June 15th. Then, on November 16th, the journal pulled the review with a brief notice:
Due to a comment received and due to some other identified methodological problems, the present review is withdrawn.
The editorial group responsible for this previously published document have withdrawn it from publication.
The review, “Yttrium‐90 microsphere radioembolisation for unresectable hepatocellular carcinoma,” was published on Feb. 16, 2016, and has been cited by at least four studies after Pollock flagged the errors, according to Clarivate Analytics’ Web of Science.
Pollock — whose work is partly funded by a subsidiary of a Yttrium-90 microsphere manufacturer favored by the error — was frustrated by the delay:
So for the record that’s a total of 270 days between reporting the error and the retraction (sorry, “withdrawal”), with “some other identified methodological problems” added to the ultimate reasons for retraction. In those 270 days, the paper, reporting findings that were literally directionally incorrect, was freely available for download for all those with access to the Cochrane Library.
From my view at least, I would certainly have expected better from the Cochrane Collaboration.
Cochrane draws a distinction between “withdrawal” and “retraction,” as we’ve noted in a previous Retraction Watch post. Cochrane Library reviews are updated frequently to account for new evidence that weighs for, or against, a specific medical treatment, but can be withdrawn as they become outdated. (Here’s more on their policy.)
Besides the figure label switch, Pollock also identified “a rounding error in the hazard ratio,” which suggested that the inferior treatment — tiny, radioactive spheres delivered through the bloodstream to attack a tumor — were linked to a slightly lower probability of death than other studies had shown.
Pollock wrote a comment outlining these errors, which appeared on the review’s page on October 28th.
Abdel-Rahman did not respond to our requests for comment. Nikolova and Gluud confirmed that they were aware of the errors in February 2020. Nikololova blamed the delay in acting on the pandemic:
The comment could be easily addressed, but rereading the review we also wanted that the authors conducted other amendments before we republished the review. We eventually also wanted the authors to apply the latest recommendations in the updated Cochrane Handbook 2020, in terms of methods. We also asked the authors to try to find a third author as a guarantor of the review. As you can see, we now realise that we expected more from the authors than it was realistic in the COVID-19 infection year to expect. The authors did not have the time to do that (both working at hospitals). So we agreed that we would withdraw the review until they found the time for the requested improvement which would benefit the general reader, after which we planned to republish the review again. This is why the review is ‘withdrawn’ and not ‘retracted’.
Withdrawing Cochrane reviews (i.e. archiving them for a while), does not mean that they cannot be found and read through the Cochrane Library.
Nine months between spotting an error, and pulling a study, is also not a record, as Retraction Watch readers will know. For example, a few months ago, PLOS ONE retracted three papers six years after manipulated images were first spotted, and reported, by scientific sleuth Elisabeth Bik.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
It seems utterly reasonable to me that a doctor actively working in a hospital might not have had the time or energy this past year to give a paper revision the time it deserves. A temporary archiving until the authors can address the issue and either correct or retract seems like the proper move.
I don’t think it’s nefarious that it took 9 months when the author probably was constantly hoping, like the rest of us, that the pandemic calmed down enough for them to give this issue attention. Obviously that never happened.
I’m in favor of treating scientists like human beings.