Over the objections of the authors, PLOS ONE has retracted a paper linking a diet designed to restore healthy gut bacteria to weight loss and other benefits.
The study, published in June 2017, claimed to show that a “Microbiome restoration diet improves digestion, cognition and physical and emotional wellbeing.” The diet was one championed by The Gut Makeover, whose author, Jeannette Hyde, is also a co-author on the paper (which the paper clearly disclosed). The diet is “designed to improve the health and diversity of the microbiome,” the microbiota that live within us.
The paper apparently prompted some criticisms, and even led one academic editor at PLOS ONE to resign. The journal now says “the conclusions of this study are not supported by the data presented,” and have retracted it. But the case may reveal more about the limitations of peer review at the journal than it does about any weaknesses of the study.
That’s because many of the reasons given for the retraction are limitations authors included in the paper. For example, retraction notice notes that there was no control group, and the participants weren’t blinded to the diet. But readers — and peer reviewers — would have known all that. From the Discussion section of the paper, which repeats some of what also appears in the Materials and Methods section:
There was no control group in the study and the participants were not blind to the treatment protocol.
If we may: It’s one thing to reject a paper because the study it’s based on is too flawed to be published — or “not even wrong,” as some might put it. It’s another to retract it because peer reviewers apparently missed the fact that it was too flawed to be published. That’s not among the Committee on Publication Ethics (COPE) criteria for retraction.
As Frank Harrell, a prominent biostatistician at Vanderbilt, tweeted:
Reader comment at https://t.co/ybNcKkp0se captures the issues. This @PLOSONE article is a travesty and casts doubt on the entire peer review process at @PLOSONE . https://t.co/5iFLnJauhI
— Frank Harrell (@f2harrell) January 12, 2018
Authors object
Not surprisingly, the two authors of the study objected to the retraction. Asked for comment, corresponding author Kate Lawrence referred us to a statement by Philip Booth, the director of research at her university, St. Mary’s, in Twickenham, UK. Lawrence told Retraction Watch that she agrees with the statement, in which Booth said he has “huge concerns regarding this decision,” calling it “bizarre and unusual,” and likely unprecedented. He adds:
A study was produced and went through the peer-review process. It was published and led to substantial discussion. The limitations of the study were clearly laid out in the article and were there for any reader and, of course, the editors and reviewers to see. Indeed, I believe that these facts are not in dispute.
There seems to be a serious problem here in that the journal’s editors do not understand how science is carried out and certainly have not followed normal guidelines for the retraction of papers. Empirical science is never definitive (or academic work would one day come to an end). There are greater doubts about some studies than others depending on the nature of the problem and the methodology used in a study. Science progresses through the production of work and then others use better methods, larger samples, different methodologies or different circumstances to produce different results. Debates then ensue and, as a result of this, the nature of problems and the answers to questions becomes clearer. This process of thesis, anti-thesis, synthesis is probably at least 800 years old and applies not just to sciences but to other disciplines. In this case, of course, a pioneering study was necessary in order to help start the debate – which it has done. The study had its limitations which were clear to the reader and, of course, any journal is free to reject such a study: PLOS ONE chose not to reject it.
Articles should only be retracted if there are problems that have been in some sense concealed or for other reasons of malpractice or where error is discovered. There is no suggestion of any of these issues arising here. Error is not the same as uncertainty due to the limitations of a study and the limitations were expressed in the article itself. For a journal to effectively retrospectively change its peer review decisions is bizarre and it reflects extremely badly on all the editorial processes connected to the journal. No academic could possibly be advised to submit an article to this journal. After all, they could never know if the article really had been accepted. How many years afterwards might it be “unpublished”?
The journal should be aware of how a retraction affects an individual academic. If the journal has come to the conclusion that its review processes were not handled competently, I am sure that I don’t have to point out the ethical flaws in an approach that leads to the consequences of the journal deciding that its processes were inadequate being borne by the author in the form of the kind of public humiliation that a retraction involves – this humiliation arising because retractions imply some kind of malpractice or author failure which has not happened in this case.
This article has, apparently, led to a great deal of discussion. If I were editor of a journal I would be pleased by that and would be welcoming further articles that contributed to the debate and led to the science being furthered.
PLOS told Retraction Watch:
PLOS ONE editors became aware of the concerns about this article when readers posted critical comments on social media and contacted the editorial office. Investigating this case, it became apparent that some of the issues were raised during peer review but were not adequately addressed. We are therefore taking steps to strengthen our editorial processes for manuscripts involving health-related interventions.
Duane Mellor, a nutrition lecturer at Coventry University in the UK, resigned from his position as an academic editor at PLOS ONE after reading the paper. We asked him if the retraction was likely to change his mind.
The reason I resigned was that it was not handled as an intervention study which it clearly is and would have prevented this happening potentially.
It would change my mind, about my resignation, although I have other journal roles now and as such may not have adequate time to do this role.
Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
I agree with PHILIP Booth. The authors are due an apology. Shame on Plus One.