What do you do when new experiments show that you interpreted the data from your old experiments the wrong way?
Some scientists might just shrug and sweep those errors — and their previous papers — under the rug. But when it happened to Jeffery Kelly, of the Scripps Research Institute, and his colleagues, they decided to retract their earlier work.
Here’s the abstract of their new paper (we bolded a few sentences for emphasis):
The accumulation of cross-β-sheet amyloid fibrils is the hallmark of amyloid diseases. Recently, we reported the discovery of amyloid disaggregase activities in extracts from mammalian cells and C. elegans. However, we have discovered a problem with the interpretation of our previous results as Aβ disaggregation in vitro. Here, we show that Aβ fibrils adsorb to the plastic surface of multi-well plates and Eppendorf tubes. This adsorption is markedly increased in the presence of complex biological mixtures subjected to a denaturing air-water interface. The time-dependent loss of thioflavin T fluorescence that we interpreted previously as disaggregation is due to increased adsorption of Aβ amyloid to the surfaces of multi-well plates and Eppendorf tubes in the presence of biological extracts. As the proteins in biological extracts denature over time at the air-water interface due to agitation/shaking, their adsorption increases, in turn promoting adsorption of amyloid fibrils. We delineate important control experiments that quantify the extent of amyloid adsorption to the surface of plastic and quartz containers. Based on the results described in this paper, we conclude that our interpretation of the kinetic fibril disaggregation assay data previously reported in Protein Sci. 2009, 18, 2231-2241 and Protein Sci. 2010, 19, 836-846 is invalid when used as evidence for a disaggregase activity. Thus, we retract the two prior publications reporting that worm or mammalian cell extracts disaggregate Aβ amyloid fibrils in vitro at 37°C. We apologize for misinterpreting our previous data and for any confounding experimental efforts this may have caused.
The two papers to which the abstract refers are
- “A kinetic assessment of the C. elegans amyloid disaggregation activity enables uncoupling of disassembly and proteolysis,” published in 2009 and cited 12 times, according to Thomson Scientific’s Web of Knowledge, and
- “Discovery and characterization of a mammalian amyloid disaggregation activity,” published in 2010 and cited 13 times
Here’s how Kelly explained in an email to us what happened:
My laboratory, in the process of trying to isolate and identify a protein(s) exhibiting disaggregase activity, realized that too many candidates were active which led to us realize that the interpretation of our previous data was incorrect. It took us almost a year of effort to figure this out and convince my trainees who actually did the initial experiments that we misinterpreted our data.
The retractions haven’t yet run. Kelly:
My understanding is that the editor and experts at Protein Science have determined that this is a corrigendum instead of a retraction, we are agnostic about what the determination is, but clearly we misinterpreted our data which we have corrected and apologized for.
We’ve contacted the editor of the journal to learn his plans for the papers, and will update with anything we find out. Regardless of how the retractions are treated, however, Kelly and his colleagues deserve kudos for their efforts, at their own professional expense, to correct the scientific literature.
In short, we agree with the tip we received about the paper on Twitter:
I think this paper is a model retraction and should be highly praised.
Please see an update on this post.
Takes a lot of guts to do this. Given the pressures to ignore the problem and move on, the PI exhibited a strong moral compass. It will probably pay off in the long term, though.
Of all the interesting and useful information Retraction Watch provides, the one I’m most grateful for is the way you give authors a chance to explain what happened in a human way – not just what the technical error or problem was.
That said, in this post, you could have made it clearer whether the two quotations from Dr. Kelly came from the retraction (I’m guessing not) or from a conversation or e-mail exchange you had with him. In a publication more formal than a blog, would you put up with such sloppy citation practices?
Thanks for the feedback. We figured it was clear, but always appreciate the opportunity to make it even more so, so we added a phrase.
That does the trick. I hope I didn’t sound harsh; it’s a small issue in this case, but in others it could be important.
Ken
Like the editors, I have doubts that this should be retracted. All scientific interpretations will be overturned later when more complete or conflicting information is discovered.
There is no need to retract a paper just because you find further data that calls into question your previous conclusions. What if you even later find information that confirms you original paper? Can you retract a retraction?
Just publish more papers. Then publish a paper comparing your earlier paper to your later papers. Then publish a review summarizing the whole affair. Where’s the problem?
This is a bit of a different case than you’re thinking of. It’s not that the interpretation of the data turned out later to be wrong or incomplete, which as you say happens eventually to all or almost all findings. Instead, the data analysis itself is invalid; they reported the presence and properties of a disaggregase “activity” (which in biochemistry means an enzyme taking a specific molecular action) that was actually a physical interaction with their plasticware. (This is the sort of thing that keeps everybody who works on aggregation-prone proteins up at night, by the way; any experiments of this sort are extremely difficult, and effects like this probably underlie a lot of the conflicting results in the field. Much credit to the authors for putting in the effort to track down what happened, and for putting it out there publicly). Retraction is a strong response in a case like this, but it’s much more justified than in some other cases discussed on this site where later results simply called the conclusions into question but left the data itself intact. I wouldn’t really put this paper into the category of “superceded.”
So, there is no enzime? How many wrong interpretations were in vitro, now it’s in plasto! Research needs old good glassblowers. BTW I have right now problems with plastic ware and I am not sure the problems are with plastic ware.
Glass and proteins don’t always mix, either.
Someone should take a wet film of egg white, cross-link it with UV, wash and use as substrate. Or – another suitable protein.
I agree that retraction per se seems unwarranted, and that this issue could be solved by a mention in a future publication. I would say, however, that in this wonderful modern age of online publication, past papers that you (the original author) now know to have been in honest error should be presented with a digital link to the specific newer papers that overturn them. It is a simple enough thing to implement, and would greatly improve both the overall quality of the literature and the general understanding of this process. Everyone at some point runs into the problem of having to track a particular question backwards and forwards through time and across multiple journals; when journals were all in print there was no alternative, but now a simple digital “thread” would ensure that no vital newer information is missed.
I also extend my congratulations to Dr. Kelly who handled this exactly as I think it should be handled. The scientific literature is hard enough to follow- why should people be hindered by the existence of papers that are not reliable. In the present case, the retraction also serves as a warning for other experimenters, and is very valuable.
Perhaps there could also be a category called ‘superceded’ so that there is no question of experimenters doing anything malicious- this could be used for papers known to be incorrect but never retracted-
e.g. a paper reporting that E. coli has 2 genes for a particular enzyme, followed later by the sequence of
E. coli showing plainly that it has only one. This would save 2-3 hours of the time of a grad student who tried to understand the genes function.
It is from mistakes that one learns. So be it. The suggestion to mark these papers as ‘Succeeded’ fits neatly.
Anyone can make a mistake.
I would recommend retracting the retraction and putting a few clear corrections in the online version.
Applause to the honest PI and the researchers.
Reblogged this on lab ant and commented:
Thats the way all PI should deal with such issues. thats the way I always thought things should be handled!
Whether it’s retracted or corrected, the best result would be:
a) That the paper remains there.
b) But an unmissable message is attached that explains what happened.
c) The behavioral example is good.
d) But in addition, the combination of original paper plus description of how it went wrong might be valuable in helping others avoid mistakes. Honest mistakes are often not simply documented.
the paper which retracts the first two is essentially d) -it goes into what happened in great detail, with additional experiments to corroborate and justify the new intepretation.
I admit to have little to no understanding of the area of research in question. That said, I find myself in agreement with those who feel that these papers should NOT have been retracted. One has to wonder how many other published papers exist with reliable data that are similarly misinterpreted, should those be retracted as well?
Readers may be interested to see COPE’s guidelines on retractions: http://publicationethics.org/files/u661/Retractions_COPE_gline_final_3_Sept_09__2_.pdf