Sometimes, a junior member of the team sees things an editor-in-chief misses.
Regular readers know that we’re always delighted when we get a chance to commend researchers and journals for doing the right thing. Here’s an example that sets the standard.
Anesthesia & Analgesia (A&A) is retracting a 2015 paper which purportedly found important differences in patient outcomes based on the quality of their anesthesiologists. The trouble with the article: The authors had made a subtle but significant statistical error that, once corrected, made the results meaningless.
The journal considered the findings of the paper, “The impact of anesthesiologists on coronary artery bypass graft surgery outcomes,” so important that they played it up on the cover of the issue, even running a picture of a King of Hearts. And the paper has been cited 18 times, earning a “highly cited paper” designation from Thomson Scientific’s Web of Science, meaning its citation rate is higher than would be expected. (Five of those citations were in the same issue as the original paper.) According to the abstract:
The rate of death or major complications among patients undergoing coronary artery bypass graft surgery varies markedly across anesthesiologists. These findings suggest that there may be opportunities to improve perioperative management to improve outcomes among high-risk surgical patients.
The handling editor in charge of the manuscript reportedly told the A&A’s (outgoing) editor, Steven Shafer, that the paper was among “the more important papers” he’d managed for the journal.
But even before the paper appeared in print, skepticism of the results arose. As Shafer — a member of the board of directors of our parent non-profit organization — details in an editorial accompanying the retraction notice, an anesthesia resident at Stanford named Lindsay Borg questioned the findings one day when Shafer was reviewing an in press copy with her and other residents. (The paper had been published online, but not yet in print.) As subsequent critics would claim, the results were “a tautology” rather than a real effect.
Willing to eat a foot-long crow sandwich, Shafer describes how he dismissed Borg’s analysis (with what turns out to have been ill-founded statistical authority) — only to find that many other readers were of the same opinion as the doctor-in-training.
Confronted with these concerns, the authors reanalyzed their data, and found to their dismay that their initial conclusions were incorrect. (Shafer writes that he was worried that the replication effort was itself mistaken, so he proceeded with the retraction only after the authors were certain that their new analysis was in fact accurate.)
According to the retraction notice (which, after all this transparency, sits behind a paywall!):
As explained by Glance and Dick, the findings of the 2015 article “The Impact of Anesthesiologists on Coronary Artery Bypass Graft Surgery Outcomes” are incorrect because of flawed statistical analysis. The article is therefore retracted. The data in the retracted article are a subset of a revised analysis appearing in this issue of Anesthesia & Analgesia. The authors’ diligence in identifying the error and correcting the analysis is acknowledged with appreciation.
The new paper, “Feasibility of Report Cards for Measuring Anesthesiologist Quality for Cardiac Surgery,” makes a much more modest claim than the first attempt:
The impact of anesthesiologists on the total variability in cardiac surgical outcomes was probably about one-fourth as large as the surgeons’ contribution. None of the anesthesiologists caring for cardiac surgical patients in New York State over a 3+ year period were identified as performance outliers. The use of a performance metric based on death or major complications for P4P may not be feasible for cardiac anesthesiologists.
Shafer, who has likely done more than any other journal editor to bring order to the chaos of retractions — think Reuben, Boldt and Fujii, for starters — closes his editorial with a few prescriptions.
What can we learn from this? First, the most critical peer review occurs after a paper is published, when it is reviewed by dozens of experts. If something is wrong, it will likely be
identified. Second, peer review is imperfect. Everyone who has received a rejection letter already knows that. Third, science is self-correcting when authors, reviewers, and editors
have the will to do it.The final lesson is that “retraction” is not a good word to describe the withdrawal of a manuscript when diligent authors discover and report a mistake in their work. There
is too much baggage associated with “retraction.” … “Retraction” seems a harsh reward for honesty. We need a better word to describe the result. We need a term that acknowledges honest error, a type of error familiar to every honest author. We also need to reward honesty with gratitude.On behalf of the Journal, the Editors, our readers, and the patients whom we serve, I express appreciation to Dr. Glance and his co-authors for their commitment to scientific
integrity.
To which we say: We couldn’t agree more. From the full explication of the events leading to the retraction, to the publication of several letters, as well as a lengthy response from the authors detailing their process and the error, and even the self-deprecating cover art of a playing card going up in flames, the May 2016 issue of A & A is a textbook example of transparency in scientific publishing. As a likely-not-irrelevant aside, it’s also one of Shafer’s last issues as editor in chief of the journal; his tenure expires later this year. We’re hopeful that the new editor, Jean-Francois Pittet, continues the example his predecessor has set over the years. He certainly has large shoes to fill.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
How about “withdraw,” as the term that “term that acknowledges honest error.”
Save “retraction” for the dishonest errors.
“Withdrawn” is already used by several Publishers for retractions. See for example http://www.ncbi.nlm.nih.gov/pubmed/26946097
It is not clear these papers are withdrawn for honest errors.
Maybe something for a ‘poll’ on RW?
One problem is that some papers are retracted for supposedly honest error but where some digging reveals those “accidental” errors were in reality “deliberate”.
Another option: Keep “retraction” for any publication found to be misleading or incorrect, but, for publications that are also known to be the result of misconduct, the term might be “retracted for dishonesty,” or “for misconduct,” or “retracted because fraudulent.”
I can’t see how we can take the successful recognition of error in one case, and generalise it into the prediction that all errors will be recognised.
Shafer is repeating his mistake of confusing a tautology with a real effect.
I don’t claim all errors will be identified. Such a claim would clearly be false. I say “If something is wrong, it will likely be identified.” That could be wrong, since it’s not possible to know the denominator. However, I have seen enough examples, many covered by Retraction Watch, to believe that serious errors are “likely” identified. I’ve retracted dozens of papers after identifying errors – not just this one case. Retraction watch has hundreds of other examples of post-publication manuscript review finding errors in papers.
Whether it is likely or less-than-likely misses my point. The point is about will. The process only works when editors, authors, reviewers, and readers are committed to scientific integrity. Without the will to act, and willingness to “eat a foot-long crow sandwich” (!), errors are less likely to be found, corrected, and explained transparently to readers.