Failure to reproduce leads to retraction of Nature Chemical Biology herbicide paper

nat chem bioA group of researchers at Emory has retracted a highly regarded paper after being unable to reproduce its key results.

Here’s the notice from Nature Chemical Biology:

We wish to retract our article entitled “Reprogramming bacteria to seek and destroy an herbicide.”

While performing further studies on the riboswitch reported in the article, a postdoctoral researcher in our laboratory was unable to reproduce the results reported in Figure 4. Specifically, neither atrazine-dependent changes in β-galactosidase activity (Fig. 4a) nor changes in motility (Fig. 4b,c) were observed. Upon learning this, J.P.G. asked a second postdoctoral researcher with expertise in synthetic riboswitches to carry out the experiments. Again, no significant atrazine-dependent changes in β-galactosidase activity or motility were observed. Because atrazine-dependent changes in gene expression formed the central thesis of the paper, we feel that retraction of the work in its entirety is essential.

At the request of J.P.G., the Emory University Office of Research Compliance conducted an independent inquiry. The inquiry committee concurred with the authors’ decision to retract the article based on the irreproducibility of the results.

We sincerely apologize to the scientific community for any harm caused by this publication.

The paper has been cited 79 times, according to Thomson Scientific’s Web of Knowledge. As last author Justin Gallivan notes in his lab site, in 2010, the paper was “listed among the key discoveries in chemical biology of the past decade by Nature Chemical Biology.”

As per the paper, first author Joy Sinha, now at ANDalyze Corporation, performed the experiments described in Figure 4. Gallivan told Retraction Watch the story from his perspective, offering far more detail than many researchers we contact:

Our lab has developed a variety of synthetic riboswitches that control gene expression in a small-molecule dependent fashion. A lab member discovered that some riboswitches unrelated to those in the NCB paper operate via two mechanisms: 1) an equilibrium mechanism, which we had established; and 2) a kinetic mechanism, which was unexpected, but turns out to be the dominant contributor to the overall response. In light of these results, I suggested that another postdoctoral associate reexamine the mechanism of action of the riboswitch reported in the NCB paper. In the course of these experiments, the postdoctoral associate was unable to reproduce the results shown in Figure 4. After consulting with Dr. Sinha (who at that point had departed from the lab) to make sure we weren’t missing anything, we were still unable to reproduce the experiments (specifically, we noted no atrazine-dependent increase in beta-galactosidase activity, nor an atrazine-dependent increase in motility when the cheZ reporter gene was used). Soon after, the postdoc that attempted to reproduce the experiments moved on to a different position.

I was left with two sets of experiments with results that conflicted with one another. To get to the bottom of the situation, I used unrestricted funds to hire another experienced postdoctoral researcher on a temporary basis to adjudicate the issue. That researcher was also unable to replicate the published work, and confirmed the findings of the second postdoctoral associate.

At this point, I was satisfied that the published work was not reproducible, but was left with the questions of ‘Why?’ and ‘What changed since the time of submission and now?’. I did not have good answers to these questions, and even if I did, I did not feel it was appropriate to, for lack of a better term, ‘serve as judge, jury, and executioner.’

I requested that the Emory University Office of Research Compliance take an independent look at the evidence. The office appointed a committee of faculty members that investigated the situation (review of lab notebooks, interviews, etc.). The committee concluded that there was not clear evidence of research misconduct (as defined by the US ORI).

Kristin West, of Emory’s Office of Research Compliance, told Retraction Watch she was unable to provide any comment on the investigation, citing confidentiality requirements.

Gallivan, for whom this was clearly a difficult situation, said no other primary papers would be affected. He added:

On a personal note, I am devastated that this happened, and I still don’t have all of the answers as to why. That said, I believe that we did the right things: 1) we discovered a potential problem; 2) we investigated the science, including bringing in outside help; 3) we asked an independent committee to investigate; and 4) we retracted the questionable work in its entirety.

9 thoughts on “Failure to reproduce leads to retraction of Nature Chemical Biology herbicide paper”

  1. Good work.

    “I was left with two sets of experiments with results that conflicted with one another. To get to the bottom of the situation, I used unrestricted funds to hire another experienced postdoctoral researcher on a temporary basis to adjudicate the issue. That researcher was also unable to replicate the published work, and confirmed the findings of the second postdoctoral associate.”

    …indeed, unrestricted funds are nice. I worry about the labs that are scraping together on a shoestring which are likely not to have funds to go back and validate old data just-because.

    1. Good call! Perhaps ORI ought to consider some small grants for this kind of situation. This particular replication attempt sounds as if it might have been rather expensive. But, in many other cases, it should be possible to get quite a lot of bang for a relatively modest number of bucks.

  2. And that paper is probably why the first author got offered a job. Im not saying he cheated, but if he/she did–or just wasn’t careful—it paid off real well.

    While the rest of us with modest but honest publication records struggle with under or unemployment.

      1. While the incentive system is often to blame, in this case it could really be that a one off result was produced, or perhaps another factor that was not noted at the time. Don’t forget that the host institution did conduct an investigation and found nothing out of order; neither you nor I have access to those details.

        1. Errr, no they didnt, they found no clear evidence of misconduct.

          Unless someone writes in their labbook ‘dear diary, was so fed up with all the negative results trying to prove my PI’s idiotic proposal I have decided to start faking results’, the chances of finding clear evidence while not zero is small.

  3. I looked at Atrazine hormone mimetic effects in human B-Cells, grown in charcoal stripped serum media.
    I always got effects, in the low micro-molar range, on proliferation and mitochondria function. However, it was never serially reproducible; doing the same titration, with the same reagents and same cells, two weeks apart, would give me different line-shapes. It was very odd indeed. I had started with Atrazine, but in the end concentrated on other man made environmental xenoestrogens that behaved better.
    This could just be that slightly different starting conditions give you different results.
    I almost nailed it by getting rid of all the conditioned media when I did my growth assay, but still a lot of batch variation.
    The first author may have just been (un)lucky and could be completely blameless.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.