A paper on dressing wounds with honey has been retracted after the journal realized that an outlier patient was throwing off the data analysis.
Honey has been used for millennia as an antimicrobial wound dressing. Doctors can even buy sterile preparations of the sweetener. But the evidence that honey is better than other wound dressings is still inconclusive.
According to the retracted paper, published in International Wound Journal in 2008, Manuka honey has an acidic pH which helps reduce the alkaline environment of chronic woulds. Indeed, the authors found that Manuka honey dressings lowered wound pH and reduced wound size.
Sadly, the paper was pulled in 2014, after someone realized one patient had a particularly large wound that was throwing off statistics. The injury was 61 cm^2 at the beginning of the study, while the others ranged from .9 to 22 cm^2. After removing that patient from the analysis, the results no longer held up.
Here’s the notice for “The impact of Manuka honey dressings on the surface pH of chronic wounds”:
The above article from International Wound Journal, published online on 20 May 2008 in Wiley Online Library (wileyonlinelibrary.com) and in Volume 5, pages 185–194, has been retracted by agreement between the authors, the journal Editor in Chief, Professor Keith Harding, and John Wiley & Sons Ltd. The retraction has been agreed due to problems with the analysis of the scientific data in Table 1. The regression results presented in this table are strongly influenced by a high outlying value (the 16th patient in the table). When the results for this patient are omitted, the association between initial wound pH and wound size at the end of the study is no longer statistically significant (P=0·132). As this relationship is pivotal to the conclusions of the paper, it is felt that the interests of patient care would be best served by a retraction.
The paper has 71 citations, according to Google Scholar. We’ve emailed the editor and corresponding author, and will update if we hear back.
Update 3:30 p.m. EST 3/20/15: Author Ronan Conroy of the Royal College of Surgeons in Ireland sent us the following statement, taking responsibility for the errors and saying they will be republishing a corrected manuscript:
As you can read on the journal website, there was an important outlier in the data (which were published in the paper itself) that adversely affected the regression results. I should have checked graphically and statistically for an outlier of this sort before running the regression, and my failing to do so led the paper to an incorrect conclusion.
The re-analysis of the data was undertaken in response to a query we received (which was actually about a different aspect of the analysis) and as part of the re-analysis I re-ran the regression, this time with the proper data checks in place, and the error came to light.
Having detected the problem, we retracted the paper, as we felt that the main conclusion was untenable in the light of the re-analysis.
This is not to say that the data are unimportant. The Manuka regime performed as well as the comparator, which is still an important finding, and we intend to publish a reworked manuscript.
Hat tip: Neil Saunders
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post.
Talk about a file drawer problem! This study went from quality research to nonexistent because the p-values isn’t what was wanted? Well, maybe the outlier biased results, or maybe deleting that case restricted range to the point that significance was lost. What happened to the actual statistics when the case was removed? Did the slope change, or is it the same? “Significance” isn’t everything.
Ed, whole-heartedly agree. At some point, almost the entire would have to be retracted simply because someone raised, or lowered, the “P” bar. I am fully supportive of retractions for misconduct, or duplications that have the intent of misleading the scientific community, and even of papers that are riddled with probelms, even if not misconduct. But this waivering conclusion as the P value waivers sounds like a dangerous precedent for retractions to me. In that sense, your query about the change in slope seems valid. I simply cannot understand why a pepr cannot remain, in this case, with an addendum that addresses the issue for the readers and other researchers to then challenge the findings. If we keep chipping away at the literature like this, then there will be nothing left to base any hypotheses on because all basal literature will be nothing but ghosts or skeletons.
Yikes! Did these journal editors ever consider the use of a power transformation to reduce the influence of the outlier? Deleting a case like this is at least as questionable a research practice as letting it bias the regression estimates without further precautions. The p level of the analysis without the outlier appears to suggest that perhaps the trend is already there with small wounds, and the outlier may just extrapolate this trend — and make it significant by virtue of pumping up the F ratio. If, on the other hand, the outlier runs counter to the trend in the rest of the sample and thus yanks the regression slope into the opposite direction, then, of course, the findings shouldn’t be trusted.
But the retraction notice appears to suggest the former, and not the latter scenario, and a retraction is hardly justified in that case. A brief note detailing what happens once the predictor (wound size) is whipped into a normal distribution would have been the better solution.