A mess: PLOS mistakenly publishes rejected ADHD-herbicide paper, retracts it

logoPLOS One has retracted a paper that links the most commonly used herbicide to ADHD, after it was “published in error.”

According to the note, the paper was “editorially rejected following peer review and consultation with the Editorial Board,” but ended up going through the production process anyway.

When we contacted the authors, they filled us in with more details.

Paper authors Keith Fluegge, a research scientist at the non-profit Institute of Health and Environmental Research Incorporated, and his twin brother Kyle Fluegge, a post-doc at Case Western Reserve University, submitted the paper earlier this year. MIT researcher Stephanie Seneff, who has spoken out about potential links between health and glyphosate-based pesticides — such as Roundup — was also initially on the paper, but asked to be removed as an author. (The reason, she told us: “I thought that my contribution to the paper was insufficient to warrant authorship.”)

The remaining authors received an email (which they forwarded to us) from the manuscript’s academic editor on July 10th:

Congratulations! Your manuscript is now with our production department.

But on August 17th, the authors received an email (which they also forwarded) from three PLOS One editors, with some unexpected news. They had looked at the paper, again, and:

During our assessment, we identified several substantial concerns regarding the methodology and reporting of the study in your submission. Following careful consideration of the manuscript and the advice we have received from our further consultation, we have concluded that it does not meet PLOS ONE’s publication criteria (in particular #3 and #4 at http://journals.plos.org/plosone/s/criteria-for-publication) and must be rejected.

Here are criteria #3 and #4 from the aforementioned site:

3. Experiments, statistics, and other analyses are performed to a high technical standard and are described in sufficient detail.
Experiments must have been conducted rigorously, with appropriate controls and replication. Sample sizes must be large enough to produce robust results, where applicable. Methods and reagents must be described in sufficient detail for another researcher to reproduce the experiments described.

4. Conclusions are presented in an appropriate fashion and are supported by the data.
The data presented in the manuscript must support the conclusions drawn. Submissions will be rejected if the interpretation of results is unjustified or inappropriate, so authors should avoid overstating their conclusions. Authors may discuss possible implications for their results as long as these are clearly identified as hypotheses instead of conclusions.

In total, the email contained eight critiques of the paper — including that it was not clear that “people classified as cases of ADHD were ever exposed” to the herbicide, and that, “[e]ven if the authors did establish that herbicide use precedes ADHD increases, there are many other things that likely do, as well.”

Further, they said that Keith Fluegge should have disclosed more information about his current affiliation:

We note that you are affiliated at The Institute of Health and Environmental Research Inc (IHER) which describes its mission as being a “not-for-profit research institute with an interest in genetically modified (GM) organisms, particularly those destined for food”. We feel that this information should have been declared as a competing interest. For more details of our competing interest policy, see http://journals.plos.org/plosone/s/competing-interests.

The authors have forwarded us that email, including their rebuttal to each point. Here is the six-page PDF of that exchange.  

Then, apparently to everyone’s surprise, the paper was published anyway on August 19. “Glyphosate Use Predicts ADHD Hospital Discharges in the Healthcare Cost and Utilization Project Net (HCUPnet): A Two-Way Fixed-Effects Analysis” found that:

Glyphosate use is a significant predictor of state hospitalizations for all-listed ADHD hospital discharges, with the effect concentrated in urban U.S. counties.

The retraction note went up just three days later, on the 21st. Here it is in full:

The publisher wishes to retract this paper because it was published in error. This paper was editorially rejected following peer review and consultation with the Editorial Board. Unfortunately, the production process was erroneously completed on this paper resulting in publication, and the publisher apologizes for this mistake.

Senior Editor Eric Martens sent us this statement on behalf of the journal, and the paper’s academic editor:

We routinely check editorial decisions to ensure they are in accordance with our publication criteria. In this case, upon consultation with the Academic Editor and another member of the Editorial Board, we decided to rescind the accept decision and reject the manuscript. Unfortunately, a glitch in our system resulted in the paper publishing in error, as outlined in the retraction notice. PLOS apologizes for the error in publication and will work to improve processes to prevent this unfortunate situation from occurring in the future.

Martens also sent the authors an email (which they forwarded to us) that mentions us:

Due to the unusual sequence of events, we believe that the retraction notice will likely receive media attention and that you may receive requests for comments via tweets and/or blogs (such as Retraction Watch, https://retractionwatch.com/). We are available to provide our support in responding to these requests.

You are of course welcome to respond as you see fit, but please remember that the details of the review process remain confidential, as per the standard across scientific publishing. Feel free to contact me if you are uncertain what information is protected by confidentiality.Overall, our recommendation for responding to such queries is to let the retraction notice, which includes an apology on behalf of PLOS, speak for itself.

The authors stand behind their work, Keith Fluegge told us:

…the editors made a hasty, yet purposeful, last-minute decision born out of a complete disregard of the peer review process and their own prior judgment in the months that preceded peer review.  Furthermore, the editors did not engage with the authors regarding their erroneous and uninformed rejection comments.  Since retraction, the authors were seriously dismayed upon receiving notice from PLOS One editors that they preferred to let the retraction notice stand as is, which is a completely false account of the actual events.   Needless to say, the authors will not submit work to PLOS One again, as the internal editors are secretive, clearly not truthful, and do not honor the external peer review process and or even their own protocols.

In addition to the positive comments from peer reviewers, the authors have received praise from other esteemed researchers.  The authors, therefore, strongly disagree with the final PLOS One internal editorial decision.  The work deserves to be a part of the scientific record.

Keith Fluegge also published a press release about the study he wrote himself on the IHER website.

Seneff told us that she didn’t believe she had contributed enough to be considered a co-author:

I removed my name because I was not sufficiently involved in the details of the procedures – much less so than I usually am in papers where I am a coauthor.  In part, I thought that my contribution to the paper was insufficient to warrant authorship. Furthermore, I anticipated that if there were questions regarding the paper’s methods, I would not be confident to adequately address them, and so I thought it would be irresponsible to put my name on the paper.   However, I am confused as to why the journal would publish and then retract the paper without giving an adequate reason for retraction. This was certainly not what I expected would happen.

It’s been a rough period for publishing at PLOS — on Friday, the publisher pulled a controversial blog that defended the use of tools to scrutinize scientists’ behavior, including their emails.

Editor’s note: 8/31/15 5:23 p.m. eastern: The IHER mentioned in this story is not the same organization as the Institute of Health and Environmental Research Inc. based in South Australia.

Hat tip: Rolf Degen 

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post. Click here to review our Comments Policy.

10 thoughts on “A mess: PLOS mistakenly publishes rejected ADHD-herbicide paper, retracts it”

  1. I know this is a little petty, but I can’t think of anything that makes me less sympathetic to an argument than overuse of ALL CAPS for emphasis.

    On a more serious note, I am not an expert in this public health but the use of hospital discharges for a relatively minor and notoriously-overdiagnosed psychiatric condition as a quantitative assay seems really sketchy.

  2. When it is evident the internal editors did not read the paper, emphasis seems necessary. Using hospital discharge diagnoses better characterizes the burden of ADHD in society. ADHD prevalence rates widely vary, especially in community samples. Expressing data as a percent of all mental health disorder discharges controls for the co-occurring nature of these mental conditions. Why should we care that herbicide use predicts ADHD? Because the relationship is dependent upon healthcare utilization. That is a tangible, direct cost.

    Commenter is author.

  3. I suspect that a similar analysis would “connect” ADHD with anything whose use took off starting in the 2000’s. Such as iPhones and organic food. As well as any of a myriad of items that were invented, or whose popularity soared, in the same time frame.

    Have the authors ruled out a cause-and effect relationship between organic food consumption and ADHD?

    BTW, this is the sort of control that seems to be missing from the study, an omission that would justify rejection (or retraction).

  4. One more comment – I do not recall seeing any way to access the raw data the authors use in their data-crunching. Are these available? Were they made available to reviewers? If not, then why? If so, how can they be accessed?

  5. The relationship does not depend on time trends between the variables, indicating a more substantive relationship. All data is public and can be accessed at your convenience using the links provided in the paper.

    Commenter is an author.

  6. So, “% growth” in Fig. 2 is not a time trend? And you are not trying to assess the possible correlation of increased herbicide use with ADHD? This is what your comment suggests, and is not what I get from the paper.

    Also, I cannot find a link in the paper that takes me to the raw data. Maybe the retracted version is missing the link, or maybe my fast scanning is missing something (if my computer doesn’t highlight a link with blue letters, I find I often miss things). Could you post it (them) in a comment here?

    Finally, I’ll repeat my question – have you looked into possible links between organic food consumption (that has increased at least as much, and probably much more so, as glyphosate use in the time frame you look at in the paper) and ADHD?

  7. Yes, Fig 2 is % growth over time, but this fig is not part of regression analyses. Regression is different than correlation, and it seems as if you are conflating the two. If you had read the paper, I suspect you would find a correlation between organic food and ADHD. We do not find compelling evidence at all that direct glyphosate exposure predicts ADHD. Rather, the relationship may be much more indirect. We use nearly a dozen public databases, so it would be impossible for me to post all the raw data in a comment. I again encourage you to review the paper for all of the links. These are public data.

  8. Keith, when you submitted to PLoS ONE, you agreed to make all data (this would include the raw data you complied and used in your analyses) available to readers. That you did not (as far as I can tell) do so, and that you won’t here provide a link, or accession number, or database identifier, is grounds enough for a reviewer, and the journal, to reject and/or retract the paper. (Of course, as I said, if I missed any of these specific items, re-stating this information in a comment here would be most appreciated.)

    I have read the paper, and, honestly, your responses in this comment thread are pretty confusing to me. For example, you state that, if I had read the paper, I would find a correlation between organic food and ADHD. I don’t get this, because organic food is not mentioned anywhere in the paper.

  9. I will reiterate that all the raw data is available at the USGS and HCUPNET. These are public data and are available for anyone to download. All editors and reviewers were made aware of data availability at submission, and it was not brought up once, even amid the last-minute rejection/retraction. The methods clearly describe how the public data were treated.

    You are correct. We did not study organic food production, per se. But, we have associated glyphosate closely with nitrogen fertilizers. High-yielding agriculture – necessary for an expanding worldwide population – is dependent upon nitrogen fertilizers, irrespective of farming practice (i.e., organic vs conventional). The use of glyphosate may be exacerbating this reliance, as discussed. Anti-GMO activists have made the debate all about glyphosate, and this is unfairly biasing the debate.

    Leo Kanner coined early infantile autism in the early 1940s…at around the time when use of nitrogen fertilizers exploded, as a consequence of war preparations and postwar excess . In light of evidence, I don’t find this to be a coincidence.

  10. I must agree with Art about the data. To satisfy the PLOS submission guidelines, it’s not enough to point to a database and say you used public data from it. Databases can change. For the kinds of data I’m familiar with, I would be expected to show the exact data I used AND then also describe my processing steps and provide the processed data. Of course, by “expected” I mean by Art and me and people who wrote the PLOS ONE guidelines. As Keith has reminded us, PLOS ONE editors and reviewers often fail to apply the journal’s rules, despite what we would presume are the journal’s best efforts…

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.