On Friday, we reported on the retraction and republication of a paper in The Lancet. One of the paper’s authors, Yale’s Harlan Krumholz, took issue with how we characterized the reason for the retraction. We offered him a chance to write a guest post about the situation, which we are pleased to publish below. Please see our editor’s note at the end.
Retraction Watch has grown to play a very important role in promoting responsible conduct of scientific research. Its quest to ‘track retractions as a window into the scientific process’ performs a great service to society. They also have a great responsibility to be accurate in their characterizations of retractions, as all are not alike. I was disappointed that they, in my opinion, mischaracterized the reason for a retraction and republication of one of my papers and did not want to retract their own story (do they have a process to evaluate such concerns?). They said that the retraction occurred because of a major statistical error, when, in my opinion, it was the result of a minor statistical error that affected the results in a very minor way and had no effect on the conclusion. That seems like a more accurate characterization to me. And it makes a difference to the impression of what happened.
Here is the story:
We were part of a recent retraction and republication of an article. We published a paper in Lancet that describe trends in the care and outcomes of patients hospitalized with heart attacks in China. This paper was the first nationally representative study of these patients and highlighted a recent dramatic increase in hospitalizations, the lack of improvement of survival over time, and opportunities to improve the quality of care. The paper has implications for millions of people in China at risk for heart attacks.
After the paper was published online we discovered a very minor weighting error that affected almost all of estimates to a very minor extent. Our study was based on a sampling design that enabled us to evaluation 162 hospitals and make extrapolations to the entire country. This minor error changes most of our estimates very little and did not affect our conclusions to any extent.
We immediately contacted Lancet about the need to make the minor changes. After a thorough review, we proceeded. The paper had not yet been published in paper and so they decided to have the original online paper retracted and republished with the corrections. In my opinion they could have handled it as a correction, but that is their decision. The criteria for retraction in this case would be that the results were unreliable; there was certainly no misconduct. I would argue that the results were reliable and that the minor error required a correction of the estimates. I defer to their judgment – in the end I believe they felt that even though the conclusions did not change they wanted to retract and republish because there were many minor changes.
I took issue with Retraction Watch because they characterized the retraction as occurring because of a major statistical error. Using a weighting scheme that was slightly in error seems to me to be something less than major. I am sensitive about it because I am so proud that our team did the right thing. The statistician recognized the error and rather than ignore something that would not affect the conclusions, she immediately reported it to her supervisor. That person also immediately reported it to the team, who immediately reported it to Lancet and cooperated fully with their inquiry. The estimates were updated and the paper was corrected. The journal then retracted and republished the paper.
I consider this episode to be a good news story. We cannot be perfect, but we should always be honest. I wonder how many teams would have done the same. I just wish Retraction Watch had thought more carefully about their description of the problem.
Editors’ note: We thank Dr. Krumholz for his thorough comment. We agree that the error as he describes it may well be considered minor rather than major. However, as the journal’s actions indicate, the downstream effects of the error were significant enough to warrant retraction — suggesting the choice of words becomes a difference without a distinction.
In the end, everyone agrees — as is clear from the Lancet editorial from which we quoted in the post — Dr. Krumholz and his colleagues acted swiftly and appropriately to correct the mistake and the record. We try always to hold ourselves to that same standard. Therefore, we will remove the word “major” from our post.