So when is a retraction warranted? The long and winding road to publishing a failure to replicate
Sometime in 2009, the University of Nottingham’s Uwe Vinkemeier thought something was wrong with two papers he read in Genes & Development, one from 2006 and one from 2009. The papers claimed to show how changes to a protein called STAT1 affect programmed cell death. So he did what scientists are supposed to do: He tried to repeat the experiments, to replicate the results.
So he submitted the results to G&D, which was initially willing to publish the data along with a rebuttal by the original authors. But everyone seemed to be dragging their feet.Meanwhile, Vinkemeier presented the data in a poster at the JAK-STAT meeting in Vienna in February 2010. The last author of the studies Vinkemeier’s team was questioning, Thorsten Heinzel, of the Friedrich-Schiller-University of Jena, was also there. Heinzel gave the poster a look, suggested a few reasons why the results might not be the same, and offered to send some samples to Vinkemeier’s team.
Vinkemeier requested some of those samples — in this case complementary, or cDNA, from which the authors had made the STAT1 proteins they tested — on the spot. But none came. Finally, in August, G&D, which was still mulling the manuscript, suggested Vinkemeier’s team test Heinzel’s team’s’ cDNA. We’d love to, Vinkemeier said, but they hadn’t sent it.
cDNA did arrive in Vinkemeier’s lab a few weeks later after G&D’s suggestion, but it “was not labelled as it should have been,” and it didn’t seem to be any of the material described in the papers, so the team declined to use it. Then, a few weeks later, they got what seemed like the right stuff.
They sequenced the cDNA, and found three mutations that the original paper hadn’t described. One of them changed the protein sequence in an important way.
By this time, Vinkemeier had tired of the foot-dragging at G&D, and had submitted the paper to Molecular and Cellular Biology, which published it on May 16. There, Vinkemeier and coauthors Filipa Antunes and Andreas Marg wrote:
…the results presented here conclusively exclude the possibility that STAT1 signaling is regulated by a phosphorylation-acetylation switch proposed by Krämer et al.
As reported here, in spite of our best attempts to exactly reproduce the previously reported experiments of references 22 and 23, we found no experimental confirmation of the results presented there. We are unable to provide a scientifically acceptable explanation for the complete dichotomy of our findings with respect to those reported by Krämer et al.
We would like to stress that several key findings of our laboratory that were criticized by Dr. Vinkemeier (e.g. STAT1 acetylation and the effect of HDAC inhibitors on STAT1 phosphorylation) have been independently published by other groups, some before and some after our works appeared. In our opinion this puts the failure to reproduce our data in Dr. Vinkemeier’s lab into perspective.
We also point out that Dr. Vinkemeier did not follow our published experimental protocols in all details. He likewise did not contact us directly to resolve technical issues. However, the Editorial Office of Genes&Development contacted us last year and early this year to inform us about Dr. Vinkemeier’s criticism. We submitted original scans of blots, additional information and new data to the Editorial Office to clarify this issue.
We are preparing a manuscript which extends our findings and also responds to the Antunes et al paper.
We asked Heinzel and Krämer for the references of papers that had replicated their results, but haven’t heard back.
We don’t know if the original papers should be retracted. That’s best left to experts in the field. The editors of G&D, for example, don’t seem to think that a retraction is warranted. (They haven’t responded to our requests for comment.) But we thought this was a good story to show what happens when someone tries to replicate findings, and start a discussion on Retraction Watch about what should happen next in such cases.
The comment thread, as always, is open.
Thanks to Jeff Perkel for help interpreting the papers.