Authors of a widely covered study that documented traces of plague and anthrax on surfaces across New York City have revised the paper after public health officials challenged their interpretations of the data.
It’s hard to overestimate the attention these findings received when first published.
“Bubonic plague found in NYC subway,” wrote The Daily Beast.
“Your subway seat mate: Bubonic plague, anthrax, & mysterious DNA,” said Yahoo!
“NY subway has bubonic plague,” declared Newser.
Not so fast. In an erratum published July 29, the authors write:
there is no strong evidence to suggest these organisms are in fact present, and no evidence of pathogenicity.
The article was published in Cell Systems earlier this year and detailed DNA samples collected from subway stations, parks and waterways. Researchers said that among the bacteria they found components of Yersinia pestis (plague) and Bacillus anthracis (anthrax) and theorized that the diseases:
likely represent normal co-habitants of a shared urban infrastructure [that] may even be essential to maintaining such an environment and likely represent a normal, “healthy” metagenome profile of a city.
Doctors at the Centers for Disease Control and Prevention and the New York City Department of Health and Mental Hygiene disputed the findings, writing that there wasn’t sufficient proof to show that the scientists had actually found these diseases and that “the authors chose the more sensational and less plausible interpretation of their data.”
In a response letter and erratum published earlier this week, the study authors acknowledged these points, corrected study figures and:
the Summary, Results, and Discussion sections have been revised to remove and clarify misleading and speculative text about pathogenic organisms.
For the study, “Geospatial Resolution of Human and Bacterial Diversity with City-Scale Metagenomics,” researchers gathered 1,457 DNA samples from across the city.
Some raised concerns about the research, including the authors of the letter, which highlights “a lack of evidence in the dataset of Afshinekoo et al. for the presence of plague and anthrax on the New York City subway”:
In their highly publicized report on the metagenomics of the NYC subway, Afshinnekoo and colleagues display an unfamiliarity with the genetics, microbiology, ecology, and epidemiology of some of the organisms they claim to have identified….If the co-authors lacked microbiology or public health background to ask themselves the right questions, then reviewers with expertise in biological threat agents were needed to point to these misunderstandings prior to publication. This was especially needed because, as the co-authors claimed, their work was meant to complement “monitoring to sustain and secure [NYC] against acts of bioterrorism” and envisioned “as a first step toward identifying potential bio-threats” in the NYC subway. Unfortunately, deeply flawed work that makes speculative, sensationalist, and headline-grabbing claims actually can detract from the quiet, ongoing, science-based efforts to secure critical infrastructure in NYC and elsewhere.
In response, the study authors penned a letter to address these concerns and announce changes to the paper. It outlines details of the changes they will make, and pulls back on some of its most dramatic wording:
The language in the Summary, Results, and Discussion has been revised, and speculative text about pathogenic organisms has been deleted. We now state that although all our metagenomic analysis tools identified reads with similarity to B. anthracis and Y. pestis sequences, there is minimal coverage to the backbone genome of these organisms, and there is no strong evidence to suggest these organisms are in fact present and no evidence of pathogenicity.
They also summarized the revisions in an erratum:
Figure 3B has been corrected to show the general coverage of the Yersinia pestis pMT1 plasmid, but not the murine toxin gene (yMT). The initial claim of “…consistent 20× coverage across the murine toxin gene…” was erroneously based on looking at gene annotation coordinates from different reference sequences. No reads mapped to the yMT gene when updated annotations were used. The Summary, Results, and Discussion sections have been revised to remove and clarify misleading and speculative text about pathogenic organisms. We now state that although all our metagenomic analysis tools identified reads with similarity to B. anthracis and Y. pestis sequences, there is minimal coverage to the backbone genome of these organisms, and there is no strong evidence to suggest these organisms are in fact present, and no evidence of pathogenicity. The figure and the text have been corrected online and in the print version.
When asked for a statement, Christopher E. Mason, an assistant professor at Weill Cornell Medical College and the study’s corresponding author, said:
Even though all of the most widely used tools for DNA mapping and metagenomics reported the presence of putative pathogens, (including BLAST, MetaPhlAn, SURPI, PhyloSift, and others), we felt it was important to unequivocally state the difference between matching fragments of DNA from a species and a pathogen.
We published an updated analysis of the low and variegated genome coverage of the putative pathogens, the coverage of defining virulence plasmids and marker SNPs (which were low or absent), and also reiterated the difference between DNA fragments and live organisms. We have no evidence pathogens are alive or evidence of pathogenicity.
All of our scripts, methods, and algorithms have been posted online with tutorials, as well as replicated and validated by other groups, and thus there is no question of the veracity of the processing of the data. The additional analysis we and others have published shows the importance of reference databases and the requisite, additional steps to build a more complete interpretation of data such as these. It also helped to further emphasize (as we did in the original manuscript) the overall safety of the NYC subway, and other subways around the world too.
Mason added that criticisms focused on how the data were interpreted, not the analysis.
Also I would note that there is not a retraction of our paper and there won’t be- our main results have been validated by several groups already and publicized. The question is not about the data analysis- it is about interpretation of the results.
Journal spokesperson Joe Caputo told us that the journal didn’t find any errors or statements that were made to intentionally mislead readers.
After this paper was published online in March, the authors of the letter (Ackelsberg et al) and readers on social media pointed out discrepancies involving Figure 3B and a lack of precision in how certain results were reported relating to the identification of pathogenic organisms.
The authors discovered that they had analyzed the wrong region of a plasmid by mistake in Figure 3B. This is acknowledged and corrected in the erratum, and at the same time the authors used this opportunity to revise the text of the manuscript to make it clear that they do not claim that pathogens had been found. In a response to Ackelsberg et al., the authors also provide a paragraph of additional experimental methodological details that were asked for.
Cell Systems found no evidence that any errors or potentially unclear statements in the paper were made with the intent to mislead readers, and we support formal corrections of the scientific literature.
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post. Click here to review our Comments Policy.