About a month ago, we reported on a retraction by Pamela Ronald, of the University of California, Davis, and colleagues. We noted then that this was a case of scientists doing the right thing. Ronald contacted us after that post ran, and let us know that there would be another retraction shortly. That retraction notice has now appeared, in Science:
AS A RESULT OF ADDITIONAL EXPERIMENTS IN P.C.R.S AND S.W.-L.S LABORATORIES, WE WISH TO retract our 2009 Report, A type Isecreted, sulfated peptide triggers XA21-mediated innate immunity (1). Specifically, we have not been able to consistently reproduce the results shown in Figure 3. We have also discovered critical errors in Figures 2 and S3. The strain PXO99∆ax21, used in Figure 2, was mixed up with another strain (PXO99∆raxSt). When we repeated the experiment with the validated PXO99∆ax21 insertion mutant, this strain is still avirulent on Xa21 lines. These results indicate that this insertion in Ax21 does not abolish the ability of PX099 to trigger XA21-mediated immunity. Regarding fi gure S3, by using more sensitive methods, we have discovered that Ax21 is also secreted in the mutant strains PXO99∆raxA and PXO99∆raxC. Although we recognize that some parts of this paper may remain valid, we note that key parts of the work depend on the results of Figures 2 and 3. For these reasons, we retract the main conclusion of the paper that a type Isecreted, sulfated peptide triggers XA21-mediated innate immunity.
The paper has been cited 131 times, according to Thomson Scientific’s Web of Knowledge.
In a blog post at Scientific American timed for the publication of the notice today, Ronald explains what happened. Excerpt:
For me, there was never any question that I would correct the scientific record if we had made a mistake. The bigger challenge was confirming that we really had made mistakes and generating sufficient evidence to warrant full retraction of the papers.
There were two important factors to consider. A delay in the retraction would waste the time of my colleagues who wanted to replicate our assays and build on our work. But publishing a hasty retraction with insufficient evidence would be equally counterproductive. I therefore needed to figure out how to strike the right balance between getting the answer right and getting the word out that our research may be flawed. I discussed these issues with the journal editors early in the process. They agreed that caution was important and graciously allowed us ample time to carry out further experimentation. Although at first I planned to repeat all 25-plus experiments in the papers, it soon became clear that this would be impossible to accomplish within a reasonable time frame. In April, at an international plant immunity meeting, I announced that we had made errors, that we were questioning some of our conclusions, and that I was working with the journal editors to publish a statement as soon as possible.
Sorting out the situation was personally and professionally painful for all involved. Instead of advancing the science in new directions as we’d planned, it was necessary to backtrack: re-isolate bacterial strains, optimize greenhouse conditions, and repeat experiments. Former lab members who had begun new positions as professors in Korea and Thailand were devastated to learn that the researchers in my lab could not repeat their work. Junior scientists in the laboratory worried their careers would be tarnished, and understandably did not want to spend too much time on the “clean-up” operation. I therefore assigned the bulk of the work of replication to experienced, highly qualified staff scientists, which took time away from their own projects but helped the junior scientists move to projects that were not affected by errors. It took persistence, courage and confidence to stick together as a team throughout this challenging year. I am proud that we were able to do this.
The process even touched our families. My 6th grade daughter sent me an email with this subject line: “Everyone makes misktakes” and linked to a relevant story she had found in the news: Helium, not so super after all!
Ronald writes that the team will soon report the latest results in the Ax21 story. The whole post is definitely worth a read.
This is model transparency. Bravo.
Incredibly painful, but an absolutely brilliant role model. If everyone was like her, you could happily dispense with RW.
Kudos! This is truly examplary behavior.
“For example, shortly after we reported our findings at a small meeting of Xanthomonas researchers, another laboratory rushed to publish a paper reproducing key aspects of our results in a human pathogen. Soon after, three more papers reproducing our results were published.”
I wonder if they have struck Prof. Ronald off their Xmas card list.
Kudos to Dr. Ronald, however… My query is rather simple, and perhaps Dr. Ronald would care to respond. Undoubtedly, Dr. Ronald and co-authors reaped benefits from the two papers that have now been retracted. After all, it’s not everyday or anyone that one publishes in Science (unless your name is Bohannon). Soon after those papers were published, what s important to understand is what actual benefits were made. For example, did Dr. Ronald (or any co-author) get a promotion? Could they maintain their position because of these papers? Were specific funds or financial rewards, either by UC Davis or by outside grants provided as a reward for these papers? After all, if she were in China, she would be raking in big bucks, but how about at UC Davis? Although I see this warm, heartfelt apology, even pulling her daughter into the picture, what I don’t see is absolutely any statements about how she should pay back a certain percentage of her profits and benefits because, well, frankly speaking, those papers just don’t exist any more. What would be fair, for example, were if she spent 5% of her time (and when I say her, I mean the group collectively) on those experiments, and XYZ US$ on chemicals, reagents, electricity, etc. and if she made a 10,000 US$ salary per month (for example), then surely, a percentage of all of these numbers should be paid back? Her and her co-authors? Independent of whether UC Davis is private, or public. Would my rationale be so wrong? Sorry, I don’t want to dampen the euphoria about the “scientist who did the right thing” feeling about this blog entry, but this whole aspect is strikingly missing from this story. Does anyone have some background information? Finally, if the paper was cited 131 times, although Dr. Ronald claims that “Although we recognize that some parts of this paper may remain valid”, this implies that most parts are in fact now invalid. Therefore, at minimum, all the other studies that used the methodology are basically invalid while the remainder that simply referenced her study in the Introduction or Discussion should contain an erratum by the journal or publisher? Any thoughts on this. If Dr. Ronald does not come onto this blog with a public response, I will approach her and UC Davis myself.
While I find your questions interesting I question the wisdom of attaching monetary punishment to honest error. Science isn’t easy. Even highly qualified scientists can make a mistake or have a cell line mixed up. I hate to use the phrase ‘humans make mistakes’ but it is true and wouldn’t severely penalizing a researcher for honest mistakes result in less honest researchers? Isn’t there enough or a sting from the blow to someone’s reputation or publication record?
Now, for manipulation & fabrication – I agree there should be consideration of what ‘ill-gotten gains’ have been received…..
I agree with MH’s response here. This retraction was not about deliberate misrepresenation or manipulation of figures, but of an honest error properly corrected. The holier-than-thou tone in some comments is a little disturbing, and seem to reflect a desire not only to hang, but draw and quarter also. Although it is a tad reductio ad absurdum, is it really sensible, reasonable or even feasible to go back and demand reimbursement from any scientist whose work, in the course of time, is found not to be correct or to at least be flawed? If that is the case, then I would suggest that is all of our fates, including those scientists expressing views like those expressed by JATdS. Perhaps we should exhume Newton and demand the return of his salary derived from Cambridge because he came up with what is now recognised to be an incorrect explanation of gravity.
Well, that would be too much…and enforcing such measures might actually prevent future researchers making similar retractions. Perhaps if false results are knowingly included, you might have a point, but not if it was an honest mistake.
I appreciate the criticism of the last three bloggers, and I would even partially agree with your perspective. Yet, how many “honest” researchers have come forward to date from 193 countries and expressed such “honest” mistakes until October 2013? I would say less than 5 (thus far – please correct me if I am wrong). And how many scientists are there in the world? That is not to say that more won’t come forward. But, Dr. Ronald admitted to a massive scientific gaffe “we have not been able to consistently reproduce the results”. The repeatability of a scientific study should be tested BEFORE it is rushed to press rather than tested thereafter. I am not saying that her honesty should be punished, but isn’t there a serious issue of lack of transparecy here? Look, at the end of the day, can we compare her with Prof. Das? Of course not. But did she (Dr. Ronald & Co.) corrupt the literature? Most definately yes. 131 times (for one paper). What you are suggesting is different strokes for diffferent folks (sorry for the cliche), but what I am advocating is that the system should be equal to all. Good for her for being honest, but finally, what she and her group did was a dishonest thing, i.e., release insufficiently tested data sets into the literature. So, does that mean she should be scott-free of penalties? Of course not? That is not justice! However, her a penalty should not be as severe as someone who tries to hide fraud or errors. Funds should be retunred, no doubt and then she and her colleagues can move on. This is not just about “human error” as MH suggests. So, the best way to clarify this impasse would be for Dr. Ronald to address these issues, isn’t it? What does UC Dabis say about this? I tried to find some notice on the UC Davis web-site, withot success…
Although I understand what you are saying, your proposed solution would accelerate the devolution of science into a completely transactional process (you get X reward for doing Y, and are penalized in turn for not producing something or doing something poorly). Academic institutions and grant funding bodies should really be taking note that the high risk, high reward mentality that pervades promotions, tenure, and funding is ultimately driving fraud, at the expense of the “team science” concept that everyone gives lip service to at these same institutions. That said, the individual scientist ultimately is responsible for his/her bad behaviors; however, linking a personal financial liability to the retraction process will ultimately be a disincentive to individuals taking it upon themselves to fix the scientific record.
JATdS,
Considering the rarity of such transparency, I think we need to applaud the reaction and retraction. In many fields of science, a PI would rather cut their arm off than retract a paper. What we have here is an honest mistake, not a deliberate calculated attempt of fraud. l have personally witnessed researchers fight and lie to protect a Nature paper that they knew was bad. Work they couldn’t, after 3 years of trying, reproduce themselves. Yet the work still stands, while everyone in the field knows it is bad, including Nature.
Do you give back part of your paycheck if you make an honest mistake and then correct said mistake?
How refreshing it is to see an example of such integrity. I hope it’s contagious. Role-model alert!
Kudos to Pamela Ronald to fix this problem. Many others would not have done the same in her situation. What my colleagues and me think is missing from the debate is what’s happening to early career scientists, who are at the receiving end of non-reproducible data. Here is our blog post about this topic wp.me/p3HcOl-22.
It looks to me that Dr. Ronald has written a poem about her temptations, her glorious victory over the devils and about the support of the family. The statement like “..some parts of this paper may remain valid..” was seen in other retractions. This item needed more space, I mean – if all this is about wrong data and not solely about another happy ending for crude experimentation.
Thanks to all commenters. I appreciate your kind words.
JATdS: For established, tenured professors at a non-profit institution, there is not much of an upside to publishing a paper that later needs to be retracted. UC Davis does not promote faculty members on a per paper basis. In the exceptional case, where a researcher had published very few or no papers, then a single paper may make a difference. However, most researchers publish many papers in a 3 year promotion cycle so one paper does not make much of a difference. Furthermore, the retraction needs to be included on the CV and in the next promotion cycle so a researcher does not come out “ahead”.
I am sure there are many cases where a grant is funded based on preliminary results that later turn out to be incorrect. In this case, the researcher would need to discuss the appropriate procedure with the program manager of the granting agencies. The agency would need to determine if the proposed work was still relevant or if the funds needed to be revoked. In my case, I did apply for a grant based on our earlier work, however the grant was not funded.
Dear Pamela, I must now fully agree with those who have been critical of my interpretation only because you came forward in defense. I think it was courageous to not only do the right thing and retract the paper, but to also then come onto this blog and provide a suitable explanation regardng the critics. In that sense, my previous blog entries now become invalidated because you have foward to address the critics publically. But that does not invalidade in any way the need to post criticism where it is due, because a scientists fails to see full declaration, an incomplete picture, or a failure of open transparency (which I was alluding to in my blog posts). I therefore agree that the title of this story is correct and that you remain now most likely the only role model in the entire publishing world accross the globe. Isn’t it sad to know that only one scientist has come forward to openly admit their errors, retract their paper and then to also openly defend critics? One in potentially several million scientists.
OK: so it looks like you messed up a label, which resulted in erroneous publications in journals the “public” sees as “high level”. (It’s adorable, all these “levels” humans give to “science”!) Where I come from (synthetic organic chemistry, amongst other things), this would be regraded as a “health and safety” hazard. On the Federal level, from what I can discern from your lab’s funds after an exceedingly short perusal.
Look: bottom line, professional chemists want to replicate work *safely*. Synthetic chemists used to have the shortest life span of all scientists. We have not forgotten this; and I would beg the public to recall this. We need the literature to be precise, for the health of our students! Even “mis-labeling” something can be hazardous simply in terms of time wasted- not to mention material that can go “BOOM” if not mixed correctly!
Part of the reason I am going on about safety is due to the lab accident at UCLA a while back where a student died, which seems like little to do with all of this but is actually a symptom of an endemic infection.
Yes, safety is paramount, but I do not necessarily think safety is the key factor here.
There has been controversy in this fascinating field where mixing up samples has been previously discussed.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3462622/pdf/3174.pdf
Figure 3 of another paper, this time in PNAS
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3479505/pdf/pnas.201202079.pdf
Interesting: Fig 3 A shows XAXI-YFP and GmMan1-CFP have IDENTICAL expression patterns. In my experience, this is simply impossible. they may be similar is some examples, but NEVER IDENTICAL as they show if you blow up the composite.
There are no isolated yellow or isolated blue in the composite – it is all green of varying intensities – showing IDENTICAL expression. This raises serious questions for me regarding the tightness of the data shown in fig 3B.
Stewart, I know it’s not the “key” point (if I hear that word one more time….gah, sorry, reading too many grants). The point is how this looks to scientists not in the field, and why some folks may consider mis-labling anything to be more than a bit odd in a professional lab. It’s annoying when it wastes tax money, and the audience’s time; but in other fields it can be deadly. OTOH, maybe this will make the public and the funding committees take the published literature a bit less seriously, which may be a good thing.
Perhaps in the interests of full disclosure Dr Ronald should have mentioned that a German laboratory had found her work to be unreproducible and had raised a suggestion of contamination as a possible cause?
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3462625/
“Two recent reports have suggested that FLS2 has further specificities for structurally unrelated peptides derived from CLV3 and from Ax21. We thus scrutinized these peptides for activity in Arabidopsis cells as well. While responding to <1 nM flg22, Arabidopsis cells proved blind even to 100 μM concentrations of CLV3p and axYs22 [an AX21 derivative]"
The response from August 2012, at this point still defending their findings, is here
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3462622/
One of the rebuttal points was the analogue of AX21 used was not the most active.
What does this case say about the volumes of unpublished, un-peer-reviewed and un-replicated corporate data routinely submitted to regulators with applications for the approval and sale of the products of science and technology? Australian food, GM and chemical regulators, for instance, claim to run science-based licensing systems but set no benchmarks or standards for the rigour – design, scope, scale, duration or replicability – of the experiments that produce the data accepted for assessment. Without a sound scientific basis for approving or rejecting an application, assessments forgive the deficiencies in the methodology and data, usually finding some rationale for licensing every genetically manipulated crop, irradiated food or agrochemical, despite informed objections and counter-evidence. An example: http://www.health.gov.au/internet/main/publishing.nsf/Content/6279C451E3D11E89CA257BF0001DAAE7/$File/ADI-June2013.pdf Most animal studies on which these ADIs are based were small and short term, done in the 1980s, never replicated and the ADI document is unreferenced. Interactional & inter-generational impacts of these chemicals were not tested. Note too that, for instance, a carrot can legally have 17 of these chemicals sprayed on it provided the residues are within the limits. Your advice please.
Hello Bob
It is an accepted practice but I suspect the politicians don’t know the system that forms the basis of ‘scientific’ decisions that may have major health implications of thier citizens for decades to come.
You rather eloquently show how good science is essential for us all the world over. We all eat this food and we are all affected.
To answer your carrot question: Grow your own. I do.
Breaking news on the Pamela Ronald retractions at: Can the Scientific Reputation of Pamela Ronald, Public Face of GMOs, Be Salvaged? http://www.independentsciencenews.org/news/can-the-scientific-reputation-of-pamela-ronald-public-face-of-gmos-be-salvaged/
Thanks to commenters at retraction watch and elsewhere for leads on this article.
I think both Pamela and her former students are not so free from suspicion of scientific fraud, based on two facts. Therefore ‘Kudos to Pamela’ from RetractionWatch is a bit premature.
1. Her disclosure was not full, excluding the fact that at least a single group (in Germany) raised an issue of possible problems previously. Therefore, her claim for a voluntary cleansing of an honest error does not really appear to be true.
2. Her own statement that some of the results were highly variable means that in their original publication they only selected data that fit with their model. This itself is a strong ground for suspicion for a foul play.
I think both Pamela and her former students are not so free from suspicion of scientific fraud, based on two facts. Therefore ‘Kudos to Pamela’ from RetractionWatch is a bit premature.
1. Her disclosure was not full, excluding the fact that at least a single group (in Germany) raised an issue of possible problems previously. Therefore, her claim for a voluntary cleansing of an honest error does not really appear to be true.
2. Her own statement that some of the results were highly variable means that in their original publication they only selected data that fit with their model. This itself is a strong ground for suspicion for a foul play.
Just to note, some of the authors of this retracted study (who went on to correct the record with a PeerJ publication) will be conducting an ‘Ask Me Anything’ session on their retraciton/correction experiences later today.
Details are at: http://blog.peerj.com/post/75497385626/ask-me-anything-correcting-the-scientific-record
Cleavage and nuclear localization of the rice XA21 immune receptor
Chang-Jin Park, Pamela C Ronald
Nature Communications 3 (2012)
Article number: 920
doi:10.1038/ncomms1932
http://www.nature.com/ncomms/journal/v3/n6/full/ncomms1932.html
https://pubpeer.com/publications/BDF59DD83A371BB0716A198805A506
Dr. Ronald states: “Following the publication of this Article, we discovered that Xanthomonas oryzae pv. oryzae (Xoo) lacking the peptide Ax21 is still able to trigger XA21-mediated immunity1. Furthermore, we were unable to consistently reproduce experiments demonstrating that synthetic AxYS22 triggers XA21-mediated immunity2,3,4.
In light of these results, we repeated the experiments reported in this Article. We found that neither AxYS22 nor Xoo enhance accumulation of the XA21-GFP protein beyond that observed following mock treatment. Shredding of leaf tissue (mock), with or without treatment, induces higher levels of XA21-GFP protein and correspondingly higher levels of the cleavage product as compared with the unshredded control. The presence of higher levels of cleavage product after wounding or Xoo treatment facilitates detection. Based on these results, we can no longer ascribe a role for Xoo or AxYs22 in XA21-GFP cleavage.
These findings do not alter the main conclusion of this Article that XA21-GFP is cleaved and translocates to the nucleus of protoplasts when transiently overexpressed and that the putative nuclear localization sequence (NLS) is required for localization. We are currently investigating the in vivo biological relevance of the putative NLS in the rice immune response. This notice of correction was first submitted to the editors April 7, 2014.”