The way science should work: A swift, clearly worded retraction in G&D, after legitimate questions by another group

A retraction appeared online last week in Genes & Development (G&D) that neatly brings together a few recent Retraction Watch threads: Whether retraction is appropriate for a failure to replicate, and whether retraction notices should give enough detail for readers to know what actually happened.

The retraction notice, for “Alternative splicing produces high levels of noncoding isoforms of bHLH transcription factors during development,” by Rahul N. Kanadia and Constance L. Cepko, reads:

We are writing to clarify the interpretation of the results from our above-mentioned paper. In this study, we used two methods to examine the structure of RNA from the mouse Math5 (Atoh7) locus. Our initial characterization was an analysis of the Math5 RNA structure using RT/PCR. We designed several sets of primers to interrogate the 5′ untranslated region (UTR) and 3′ UTR, as well as the coding region (CDS). The RT/PCR gave a surprising result. It appeared that the majority of the Math5 RNA molecules did not contain a CDS, but were short transcripts with the 5′ and 3′ UTRs joined together. We were surprised by this finding, and were aware that there are many artifacts created by RT and/or PCR. Rather than alter the conditions for these reactions, we chose to verify this finding using a different method, one that we thought would not be susceptible to the same artifacts as RT and PCR. We thus examined Northern blots using retinal RNAs and probes made for the CDS and each of the UTRs. The results from the Northern blot seemed to confirm that there were two Math5 RNA species: a larger one that had the CDS, and a smaller one that contained the UTRs but not the CDS. These data were consistent with the RT/PCR results, and we took these data to mean that the majority of the Math5 RNA species did not contain the CDS.

Prasov et al. (2010) have recently published an examination of the Math5 RNA structures using the same RT/PCR primers as used in our study. They can find the same RT/PCR products using the primers and conditions that we described. However, they were able to show that the short products that appear to be missing the CDS are due to a very tight secondary structure, which causes RT to switch strands or otherwise skip the CDS, likely due to an 85% GC domain in the CDS. After they published their findings, we understood how we incorrectly interpreted the RT/PCR products. We have since gone back and probed Northern blots using the same RNA probes used in our original study. We have now purified the probes using two different protocols. In addition, we washed the Northern blots using different levels of stringency. The probe preparation method and the washing conditions were found to change the hybridization results with the CDS probe.

The fact that two independent methods appeared to reinforce each other, to give the interpretation of two different RNA species, was quite unfortunate. It is likely that both the variability in the behavior of the probes on the Northern blot and the RT skipping of the CDS were due to the region of high GC content. In fact, the very high GC content might indicate that this mRNA is regulated by this structure, which may lead to poor, or at least regulated, translation of this protein. However, we no longer believe that the majority of the RNA is spliced such that the CDS is eliminated.

In other words, according to our molecular biology guru Jeff Perkel, the authors had found something surprising about the behavior of a particular bit of genetic material. They had done everything right, and had even confirmed that behavior using a second method.

Only one problem, as a paper published in PLoS ONE last year, and cited by the retraction, showed:

In a recent provocative study, Kanadia and Cepko [18] report that the vast majority of Math5 transcripts in embryonic mouse retinas are spliced, with donor and acceptor sites located in the 5′ and 3′ UTRs, such that the coding sequences are excised. This conclusion, which plainly differs from our previous studies [7], [13], was based largely on the size and abundance of particular RT-PCR products. Similar observations were reported for Ngn3 (neurogenin, Neurog3), a related bHLH factor. If correct, these findings raise important questions regarding the origin, extent and function of noncoding (nc) bHLH-gene RNAs, which may integrate into larger gene regulatory networks during neural development [19], and suggest that abortive splicing may be utilized as a novel post-transcriptional mechanism to regulate bHLH gene expression. Given the importance of Math5 for retinogenesis, the central role of bHLH factors in neuronal fate specification [20], and the possibility that functional coding and noncoding RNAs may be generated in the same orientation by alternative splicing of a single transcription unit [21], we have systematically evaluated Math5 mRNA splicing in the developing retina, using RNA hybridization and RT-PCR methods adapted for the extreme G+C content of the transcript.

Our data strongly suggest that the apparently frequent splicing of Math5 retinal mRNA is a technical artifact, resulting from: (1) profound secondary structure in the mRNA, promoting template switching during reverse transcription in vitro, (2) selective amplification of deleted products lacking the internal GC-rich segment; and (3) the existence of very rare mis-spliced molecules, representing less than one percent of Math5 transcripts. Our results refine the structure of the Math5 transcription unit, explore the concept of an intronless gene, and provide a cautionary lesson for PCR-based studies of RNA processing.

Translation: That finding that made everyone sit up and notice? Well, sorry, it was due to an artifact.

The G&D paper, which was written while Kanadia was a postdoc in Cepko’s Harvard lab, has been cited four times, according to Thomson Scientific’s Web of Knowledge. We tried to reach Cepko and Kanadia for comment, and will update with anything we hear back.

We did, however, get some more detail from Tom Glaser, the co-author of the PLoS ONE paper. First, we wanted to know: Had the authors done a good job explaining what happened?

The authors have explained part of what happened. This correction is important, but the Retraction notice does not address [1] the most likely technical basis for the artifacts (probe cross-hybridization with rRNA), [2] the magnitude of the discrepancy between their experimental findings and ours (>99% vs. <0.1% Math5 splicing), and [3] the logic that undermined the Math5 splicing hypothesis in the beginning (absence of spliced non-coding ESTs and absence of U1 or U11 splice sites for the majority of lacunar RT-PCR products). The last two points are explained in our paper (Prasov et al. 2010 PLoS ONE). The cross-hybridization of GC-rich probes to ribosomal RNA is a well-known technical problem in Northern blots (Dooley et al. 1992 Gene 110: 263-264) and may be exacerbated when cRNA probes are used. Finally, although the Retraction recounts a personal journey of discovery, no apology is offered to the scientific community.

Regardless of the elevated GC-content, or the misleading RT-PCR results, the original error and retraction could have been avoided if the authors had included a simple negative control in their Northern blots – total RNA from an unrelated tissue (e.g. liver) in an adjoining lane – or if polyA+ Northern analysis had been performed. Instead, the Northern data in the Kanadia and Cepko paper were obtained by probing nylon membranes containing isolated, single lanes of total embryonic retinal RNA. The exposure time was short and the size of hybridizing RNA molecules was not provided. We mention these details for the edification of junior scientists, journal editors and reviewers.

We were also curious whether the two groups had been in touch, and whether Glaser’s group agreed that a retraction was the best move.

Given the importance and novelty of the Kanadia and Cepko study (cover, G&D Feb 2010 issue), we attempted to carefully replicate and explain their unusual results. This required a thorough analysis and a substantial amount of work, involving six indepedent lines of evidence. We were able to explain everything except the discrepant Northern and Western data. We submitted our findings to G&D in March 2010, as a manuscript with 7 figures and 8 supplemental files. The editors declined to publish the paper, and it eventually appeared in PLoS ONE in August 2010.

We contacted Drs. Cepko and Kanadia prior to publication of our paper. We sent the manuscript and plasmid reagents, and exchanged primary data and protocols, in order to reconcile the Northern data. They were forthcoming and interested in understanding the technical and scientific issues. The laboratory notebook pages were informative. The editors at G&D considered publishing an exchange of Letters from both groups but, in the end, decided a Retraction was most appropriate, since the experimental basis for the abortive splicing hypothesis had evaporated. We agree with this decision.

If this scenario sounds familiar, it’s because we wrote about a failure to replicate a the findings of another G&D study a few weeks ago. Then, too, one group had failed to replicate the results of another. That case also led to publication of a paper in a different journal, Molecular and Cellular Biology. But there was no retraction, likely because the authors of that G&D paper have stood by their work, unlike in this case. We’ve asked G&D editor Terri Grodzicker for comment on that case, but have never heard back.

Of note, in its 24-year history, the journal has only published one exchange of letters about an experimental discrepancy, in 1991, and just one retraction.

We take Glaser’s concerns seriously; an apology appears in many retraction notices, and wouldn’t be out of place here. And it would always be better to know even more about how an error happened. Still, given some of the completely unhelpful retraction notices we’ve seen, we’d like to commend Cepko and Kanadia for their swift retraction, and highly detailed notice. It’s quite clear there isn’t even a whiff of misconduct or fraud. And the authors seem to have appropriately reserved about their results.

So to get back to those Retraction Watch threads to which we referred: Yes, failure to replicate seems a good reason to retract. And notices should explain what went wrong. Allowing authors to get away with failing to do so, in some well-intentioned but misguided attempt to lower the barrier to retractions — as the Journal of Neuroscience does, for example — is part of why some people seem to think that retractions mean fraud.

Hat tip: Uwe Vinkemeier