“Strange. Very strange:” Retracted nutrition study reappears in new journal

In an unusual turn of events, a nutrition paper has come back to life a year after being pulled from its original publication.

After the paper was retracted from the journal Obesity, the authors revised it and republished it in another journal, Pediatric Obesity. Both journals are published by Wiley. The second version of the paper doesn’t mention the previous retraction. Indeed, the journal editor told us he didn’t know the paper had been retracted. Still, he stood by his decision to publish it.

The authors told us the paper was retracted after editors at Obesity raised concerns over the authors’ methodology. The authors revised the paper, adding some analysis and explanation of their methodological approach, and said the new version was accepted by peer reviewers before being published in Pediatric Obesity.

However, an outside expert who reviewed both papers for us said he thinks the authors didn’t change enough. According to Patrick McKnight, head of the Measurement, Research methodology, Evaluation, and Statistics group at George Mason University and a Statistical Advisory Board member of STATS.org:

The authors did not change the analyses sufficiently to correct for the nested design. Moreover, I was quite surprised to see the similarities between the two papers. After a retraction, I assumed the authors would heed the advice of others and fix the problem. Instead, they appear to have simply dismissed the problems and moved onto another journal. I found this behavior quite puzzling. Never heard of a retracted paper finding a new home. Usually, the authors admit the error(s) and move on. Strange. Very strange.

Here’s the backstory. The journal Obesity published the original study, “LA Sprouts Randomized Controlled Nutrition, Cooking and Gardening Program Reduces Obesity and Metabolic Risk in Latino Youth,” in June 2015. The paper describes the results of a randomized controlled trial for a 12-week gardening, nutrition, and cooking intervention called LA Sprouts, involving four Los Angeles elementary schools.

It was retracted five months later, in November 2015. Here is the retraction notice:

The above article, published online on 9 May 2015 in Wiley Online Library (wileyonlinelibrary.com), and in Volume 23, pp. 1244-1251, has been retracted by agreement between the authors, the journal Editors-in-Chief, Eric Ravussin and Donna Ryan, the Obesity Society, and Wiley Periodicals, Inc. The retraction has been agreed to because the statistical analysis was not correct given the cluster-randomized design, and the wrong degrees of freedom were used. The conclusion that the original paper drew about having demonstrated treatment efficacy was not supported in the corrected analysis.

Paper co-author Jaimie Davis, an assistant professor in the Department of Nutritional Sciences at the University of Texas at Austin, said that the paper was accepted by two different reviewers and one of the senior editors. However, another senior editor took issue with the authors use of a cluster-randomized control trial, which should include 12-16 schools — but the authors didn’t have the budget for that, said Davis.

Anyway, we controlled for school level in our analyses and emphasized that this was a pilot RCT in the revision, but this editor was still not happy with this response and insisted that the manuscript be retracted.

Co-author Nicole Gatto, an associate professor of public health at Claremont Graduate University, told us the journal should have addressed the concern prior to publication:

After the paper had been published in Obesity, one of the journal editors provided us with a forced choice: either we could agree to withdraw it, or they would retract it. We argued that the journal should have identified the issue during peer-review and allowed us an opportunity to address it then, of which they did neither.

According to Allison Templet, the managing editor of Obesity, the concerns about the analysis were raised after publication by two associate editors who were not involved in the review or acceptance of the paper:

After the manuscript was published online on May 9, 2015, concerns were raised on May 27 by two of our Associate Editors in the correspondence below:

We are writing to you in our roles as associate editors of Obesity. We noticed this new paper in Obesity titled “LA sprouts randomized controlled nutrition and gardening program reduces obesity and metabolic risk in Latino youth” (http://www.ncbi.nlm.nih.gov/pubmed/25960146) and believe the authors made a serious error in the analysis which needs to be corrected. Correcting the analysis will plausibly alter the conclusions of the paper.

Specifically, though they do not seem to be aware of it (or at least do not acknowledge it), the authors have conducted a cluster randomized controlled trial (cRCT). Such cRCTs require analyses which take the clustering into account as mentioned in numerous statistical texts and articles, including a tutorial commentary on this point which just appeared in JAMA this week (see: http://jama.jamanetwork.com/article.aspx?articleid=2297149). The current article published in obesity does not take the clustering into account, uses the wrong degrees of freedom, and plausibly has substantially overestimated the statistical significance of their results.

These two associate editors were not involved in the review or acceptance of the paper. Based on these concerns, our team conducted an investigation and asked the authors for the code used to analyze their data. This confirmed that the degrees of freedom in the analysis were incorrect; the SAS code dealt with the clustering of students in schools but not the nesting of schools within condition. Although the authors did correct and re-run their analysis, the statistical experts on our editorial board felt that the paper made strong claims about treatment effects that were not supported in the corrected analysis.

Now, here’s where it gets unusual, as was first reported by ConscienHealth.org: The article was re-published in February 2016 in Pediatric Obesity with almost the same title, “LA sprouts randomized controlled nutrition, cooking and gardening programme reduces obesity and metabolic risk in Hispanic/Latino youth.”

The Pediatric Obesity article does not mention the earlier retraction. Pediatric Obesity’s editor Michael Goran tells us he was unaware of the retraction, but stands by publishing the article:

I know from the authors that there were issues at Obesity but I was not aware of the retraction. Looking at it now, the authors do mention the limitation of the cluster design in the discussion, so it does not affect our evaluation since this was a small pilot study with stated limitations.

In the version submitted to Pediatric Obesity, Gatto and Davis added a corrected analysis and explanation of the methodological approach in the supplemental material, plus language emphasizing that it was pilot trial. For example, the conclusion in the summary now states (our emphasis in bold):

Conclusions: LA Sprouts was effective in reducing obesity and metabolic risk; however, additional larger and longer-term studies are warranted.

Gatto emphasized that the study should not be analyzed as a cluster-randomized design. She told us:

Essentially, the study was funded by a NIH mechanism intended for innovative (small) pilot studies. For this, our project was reviewed by three separate reviewers, who look at strengths and weakness of 5 different areas, including the “Approach” which includes our proposed methods. Our project was funded after the first cycle of review. We then completed the project and wrote up our results.

Because of the size of the study, we wouldn’t have had the requisite number of groups to analyze the data as randomized clusters, so this approach wouldn’t have been appropriate. We did provide justification to this in the paper (there was a low degree of clustering in our groups) and presented the data using the group variable as a covariate in analyses.

At its new home in Pediatric Obesity, the paper was again peer-reviewed, emphasized Gatto:

There, the paper was reviewed by three reviewers. We responded to the reviewer comments, they also re-reviewed the manuscript taking into consideration our responses, and the paper was accepted for publication in November 2015.

At this stage, the methods have been examined by 8 separate reviewers (5 of whom provided review on two occasions) at a minimum.

A question that has lingered from this experience is – what is the purpose of peer-review if a journal editor can selectively censor scientific studies at his/her discretion? This is a form of publication bias which prevents the entire story from being told, and certainly does not serve to advance science.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.