In our line of work, we see it all — mega-corrections that don’t quite rise to the level of retraction, letters to the editor that point out seemingly fatal flaws in papers that remain untouched, and studies retracted for what seem like minor reasons. It can make you wonder what makes a paper worthy of a retraction. A recent case in an obesity journal may not provide a definitive answer, but it gives us a lot to chew on.
Here’s the story: In September 2013, Rosely Sichieri and a colleague from the State University of Rio de Janeiro submitted an article to Obesity Facts, “Unbalanced Baseline in School-Based Interventions to Prevent Obesity: Adjustment Can Lead to Bias?” The article examined statistical issues in randomized controlled trials of school-based weight loss programs. Peer reviewers said the paper needed major revisions before it could be accepted; the authors revised the paper enough in a second draft, submitted in November 2013, that the original reviewers accepted it. The paper was published in June 2014.
Then, in September 2014, a group of authors including David Allison of the University of Alabama, Birmingham, and colleagues from Clemson, Thomas Jefferson, and the University of Minnesota, wrote a critical letter that was published in the journal in April. The letter, according to a just-published editorial:
expressed fundamental and severe criticism with regard to the above mentioned article that culminated in the conclusion that the article should be retracted.
More specifically, the letter argued that by criticizing some of the statistical tools used in these types of studies, the authors dissuade scientists from employing “legitimate power-enhancing analytic methods.” Here’s more from the letter itself:
The fundamental conclusion as stated in its title and elsewhere in the article is incorrect. For example, the statement ‘Although adjusting for the baseline values of parameters (sic, variables – Li et al.) that are highly influenced by baseline values is a standard procedure, this approach can bias the results …’ is simply untrue. Such erroneous conclusions could lead researchers to avoid legitimate power-enhancing analytic methods, and should be retracted.
Sichieri and her colleague responded to that letter, calling the criticisms “unfair” and saying that:
Our study cannot be retracted exactly because it indicates this possibility of bias. We are not arguing that randomization procedure with unbalanced baseline is due to chance, therefore there is no selection bias. However, in this situation, usually in school-based interventions to prevent obesity, adjustment may have biased the results.
In an editorial in the current issue, Hans Hauner, the editor of the journal, describes the critical letter and author’s response, including “their reasons for refusing to retract their article.” Because the letter’s criticism was “so severe,” an independent statistician was asked to review the material; that statistician agreed that the original authors have “not adequately addressed” the letter’s criticisms. Hauner concludes:
By publishing the Letter to the Editor and the Authors’ Response together with this Editorial in the current issue of Obesity Facts, I would like to open the discussion that arose between the authors of the original article, the authors of the Letter to the Editor and the Editor-in-Chief to our readership and recommend reading and forming your own opinion.
Stimulated by this debate, I would like to point out that it is each author’s responsibility to make sure that statistical procedures are correctly used and valid for the study submitted. Even though it would be desirable if an additional assessment of statistical methods could be established within the review process, it cannot be expected that all of our reviewers, in addition to their expertise in various aspects of obesity research, are designated experts for advanced statistical procedures.
Finally I would like to emphasize the importance of critically scrutinizing scientific papers even after their publication. Therefore, comments on articles are highly welcomed at all times to present and discuss different viewpoints concerning a controversial topic.
We asked Allison whether he was satisfied with the journal’s response:
I believe that the journal handled the matter respectfully, transparently, and in a manner that promoted dialogue – and that is all to the good. That said, I also believe that journal editors have the authority to retract papers in their journals. In this case, the editor had information from an independent third party expert that the original paper provided readers with erroneous conclusions and merited retraction. Therefore, I believe that the journal should have retracted the original paper.
We’d love to hear what Retraction Watch readers think.
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post.