Exclusive: Public health journal says it will retract vaping paper for questions authors say were addressed in peer review

The journal BMC Public Health plans to retract an article that found smoking rates fell faster than expected in the US as use of e-cigarettes increased, Retraction Watch has learned.

The authors contend that they addressed the issues cited in the retraction notice during the peer review process and say they addressed them even more extensively when the journal said they intended to retract.

The paper, “Population-level counterfactual trend modelling to examine the relationship between smoking prevalence and e-cigarette use among US adults,” was published last October. The authors are all employees of Pinney Associates, a consulting firm that they disclosed “provide[s] consulting services on tobacco harm reduction on an exclusive basis to Juul Labs Inc.” The article also disclosed that Juul Labs funded the research and reviewed and provided comments on a draft manuscript. 

Some journals, including several in the BMJ family and the American Journal of Public Health, will not publish research funded by the tobacco industry, which has led to at least one retraction. But the planned BMC Public Health retraction notice does not refer to that conflict of interest.

An unpublished letter to the editor prompted the investigation that led to the journal’s retraction decision, according to emails seen by Retraction Watch. The letter did not request retraction of the paper, but argued that its analyses “were flawed and therefore potentially produced misleading findings that would benefit tobacco industry profits and interests.” 

Lorena Verduci, an editor of BMC Public Health, shared the letter with the authors on May 24. She also included an assessment of the letter by an unnamed editorial board member who agreed with the critiques. 

Among other critiques, the letter called out Juul’s support for the article, and that the analysis assumed the prevalence of e-cigarette use in the US was zero in 2010 and rose in a linear fashion until a CDC survey began gathering data on usage in 2014. 

The authors’ filling in linear data points between 2010 and 2014 for their main analysis was “questionable,” the letter stated, and findings from an additional analysis that left out those filled-in values “are hidden in the manuscript.” Reading the paper’s supplementary materials, the letter said, revealed that “most estimated associations were no longer statistically significant” in that analysis. 

Floe Foxon, the original article’s corresponding author, sent the authors’ response on June 12. In their response, the authors highlight sensitivity tests they performed using 2009 and 2011 as alternative cutoff years, which they said “support the findings of the main analysis.” 

The paper’s peer review record, which is available online, indicates that the authors added those sensitivity tests in a revision responding to reviewer feedback. 

The authors also wrote that the analysis leaving out the filled-in data: 

showed that the results of our main analyses were robust for the younger adult and male cohorts in these sensitivity tests. These are the cohorts with the highest e-cigarette use prevalence, and thus the most relevant to the research question. Contrary to the Letter, which claims that our results were not mentioned in text, the findings from this analysis are reported in the Results section of our manuscript under the subheading ‘Sensitivity test results.’ Thus, the allegation in the Letter that we “directed attention elsewhere” is simply untrue. 

Verduci, who has not responded to multiple requests for comment over the past week, informed Foxon on July 18 that the journal had decided to retract the article in light of the editorial board member’s assessment of the authors’ response. She included the assessment, which did not explicitly recommend retraction. However, the editorial board member wrote: 

I am not convinced that the authors’ response addresses the main issues and serious flaws raised by the authors of the Letter. 

Verduci proposed the following text for the retraction notice, and asked the authors if they agreed with it, giving them a deadline of July 25 to respond: 

The Editors have retracted this article after readers have raised concerns around the methodology. In particular, the paper’s conclusions are based on assumption of ‘0’ prevalence of e-cigarette use in 2010 which contradicts available data. Without this assumption, the results become non-significant. 

Foxon replied on July 21 that the authors “object strenuously to the intention to retract our article.” 

The reply included additional analyses using 2006, 2007, and 2008 as cutoff years when e-cigarette use was zero in the US, and offered to add them to the paper. The authors wrote: 

The fundamental finding remains: smoking is declining at a faster-than-expected rate, in a manner that correlates with the rise in e-cigarette prevalence. In fact, the correlation coefficients change little with these variations in cut-off years, which suggests that the findings are robust.

Foxon told us the email informing the authors of the retraction was “baffling.” He maintained that the authors had “thoroughly addressed” the criticisms, and said: 

It was not previously indicated that a retraction was on the table; the previous correspondence we received from the journal did not contain any accusation of fabrication, falsification, or anything else in the COPE guidelines that would warrant a retraction. We were asked to respond to concerns raised by an anonymous reader and an anonymous Editorial Board Member, but at that time there was no mention of an investigation into a retraction; we were never notified that such an investigation was taking place. 

We asked what Foxon thought should happen instead of a retraction. He told us: 

I am a firm believer in open science, and that path would entail publishing both sides of the disagreement: the anonymous Letter to the Editor and the anonymous Editorial Board Member’s opinions, as well as our responses. That would allow interested readers to decide for themselves which conclusion best fits the available data. Retracting the paper does not afford readers this opportunity, and credits one side of a discussion while suppressing the other in a way that is incongruous with the pursuit of scientific knowledge. Readers may welcome seeing an analysis which the anonymous Letter-writer and Editorial Board Member believe is more correct. NHIS is a publicly-available dataset, and they have the same access to these data as we did.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].