Nature paper with massive correction can’t be reproduced, says independent group

In 2011, authors of a Nature letter caught some flak for issuing a lengthy correction to a neuroscience paper that had raised eyebrows within days of publication — including some suggestions it should be retracted.

The correction notice, published months after the original letter, cited errors in image choice and labeling, but asserted the conclusions remained valid.

Now, those conclusions appear up for debate. In a recent Nature Brief Communications Arising (BCA) article, a team that raised concerns about the paper five years ago says they are unable to reproduce the results. But the authors of the original paper aren’t convinced: They argue that the BCA fails to cite important evidence, has a “complete absence or low quality of analysis,” and the scientists disregard some of their data.

The original Nature paper, “Ephrin Bs are essential components of the Reelin pathway to regulate neuronal migration,” published in April 2011 by first author Aycan Sentürk and co-authors based at Goethe University Frankfurt, proposed that ephrin B proteins in the mammalian brain directly bind to and regulate a protein called Reelin, which then guides the migration of neurons. Disruptions in the Reelin signaling pathway have been associated with epilepsy, schizophrenia and Alzheimer’s disease, so components of that pathway may provide potential therapeutic targets for those disorders. The paper has been cited 65 times, according to Clarivate Analytics’ Web of Science, formerly part of Thomson Reuters.

Within a day of publication, Nature readers raised concerns about the figures. The paper also caught the eye of a team led by Joachim Herz and Mark Henkemeyer at the University of Texas Southwestern Medical Center, who had been studying ephrin B/Reelin interactions. Here is Herz’s account of what happened:

Our laboratories (Herz lab on Reelin, Henkemeyer lab on Ephs and Ephrins) had been working in this field since the 1990s and we began to collaborate on Eph/Ephrin/Reelin interactions around 2002. Immediately when the Sentürk paper came out, our laboratories as well as the laboratories of Jon Cooper (Seattle), Eric Olson (Syracuse) and Brian Howell (Syracuse) realized that there were major inconsistencies within the Sentürk paper itself and with their own and our data. We attempted to resolve these issues by directly contacting the Acker-Palmer lab and Nature. This resulted in the first extensive corrigendum.

But the correction notice didn’t answer all the questions, Herz told us:

However, not all inconsistencies with our own data were resolved by the corrigendum. Because of major mechanistic implications, we attempted to reproduce the key findings of the Sentürk study. Our recent report in Nature now highlights these inconsistencies. Please note, however, that several of the major inconsistencies were noted independently by numerous colleagues throughout the world, giving rise to a lively online discussion.

Six months after publication, the Sentürk et al. group, led by Amparo Acker-Palmer of Goethe University Frankfurt, issued an extensive correction. (See our original coverage of that correction here.) The correction cites, in the supplementary material,

We have also included results from a new, reproduced experiment recently performed with an additional cohort of animals that shows exactly the same results.

Acker-Palmer’s team said the errors did not alter the paper’s conclusions:

In all cases, choice of images was completely independent of the data analysis and so none of the conclusions in our original Letter are affected. We apologise for any confusion these errors may have caused.

Back in 2011, Nature would not comment on whether the new results in the correction were peer reviewed; corresponding author Acker-Palmer has not responded to our latest queries.

Again, the story didn’t end there. This past November, Herz and colleagues published new results in Nature, “Ephrin Bs and canonical Reelin signaling,” refuting the original findings. In a series of knockout mouse experiments, the team was unable to reproduce the original phenotypes.

Acker-Palmer’s team disagrees. In a reply in the same issue of the journal, they say the discrepancy in findings may stem from the different knockout targeting strategy used by Herz’s group, differences in the genetic backgrounds of the mice strains used, and

Most notably, the complete absence or low quality of analysis performed by Pohlkamp et al. (for example, they perform no quantification, show single examples of low-magnification images and use poorly stained samples with high background noise) should prevent any meaningful conclusions from being drawn and does not support the strong statements made by the authors.

In response to Sentürk et al.’s claim about the different type of knockout targeting used, Herz told us:

We did not have access to the exact strains used by the Acker-Palmer laboratory. However, the Henkemeyer laboratory has studied Eph/Ephrin signaling for the last two decades and has generated many of the widely used mutant mouse strains in this field. Given the magnitude of the effect reported by Sentürk et al., the phenotype should be consistent in different independently derived models. Our report shows that this is not the case, thus calling the universal validity of the original findings into question.

To the other criticisms, he added:

[T]he extent and the degree of the discrepancies not only over a single data point, but over multiple different genotypes, anatomy and biochemistry is too great to be dismissed with this kind of handwaving. But more importantly, the Acker-Palmer lab could have easily dismissed our challenge by providing an independent replication of their own prior results. They had as much time to revisit their own studies with their own animal strains to resolve the discrepancies as we did since we first alerted their lab shortly after the appearance of the initial Sentürk paper.

Herz said the team stands by their new results:

We had lots of opportunity to examine and reexamine our findings. The discrepancies were simply too fundamental and consistent, which strongly suggests that the biological interaction between the Eph/Ephrin and Reelin pathways is not as black and white as Sentürk and colleagues claimed. If the dependence of the Reelin pathway on Eph/Ephrin signaling during migration were as absolute as Sentürk et al. reported, we should have seen at least partial penetrance of their phenotype in our models.

… I consider the matter closed until new independent and pertinent data appear, preferably also by others, that help to further resolve the discrepancies. We should not spend our time squabbling and throwing dirt at each other, but better direct our efforts at doing experiments.

If we hear back from Acker-Palmer, we’ll be sure to update this post.

Update: 1/10/17, 2:55pm Eastern:

In response to RW’s query about whether the “results from a new, reproduced experiment” in the 2011 corrigendum were peer-reviewed, a Nature spokesperson tells us that all Nature corrigenda are peer-reviewed, but declined to comment on this specific case. Here is the journal’s response:

All corrigenda for Nature papers are peer-reviewed, and where possible they are sent to the same referees who reviewed the original paper. Where appropriate, we may also seek advice from referees who did not review the original paper. For confidentiality reasons, we cannot comment on specific cases.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.