2013 was a rough year for biologist Pamela Ronald. After discovering the protein that appears to trigger rice’s immune system to fend off a common bacterial disease – suggesting a new way to engineer disease-resistant crops – she and her team had to retract two papers in 2013 after they were unable to replicate their findings. The culprits: a mislabeled bacterial strain and a highly variable assay. However, the care and transparency she exhibited earned her a “doing the right thing” nod from us at the time.
After many months spent understanding what went wrong and redoing the experiments correctly, today Ronald and her team release another paper in Science Advances that reveals the protein they thought they had identified in 2013.
Ronald and co-first author Benjamin Schwessinger (who recently became an independent research fellow at the Australian National University in Canberra) spoke to us about the experience of recovering from the retractions and finally getting it right. The conversation has been edited for brevity and clarity.
-What did you do differently this time so you didn’t repeat the same mistakes?
BS: We implemented a lot of steps to make sure we were getting everything right. For instance, two to three people worked on new assays at a time, and always passed any result onto other members of the lab so we could reproduce it internally. We also performed many blinded experiments, for example where we received two tubes from the lab manager but weren’t told what was in them, in order to eliminate the bias of looking at something in a certain way. Bias wasn’t a concern in the previous paper, we just wanted to be extra careful.
We knew we had to be extra careful because we were trying to redo something people didn’t get right before – there was a lot at stake, and we expected some extra scrutiny this time around. And we are expecting jokes – ‘have you really figured it out this time?’ But jokes are fine, I understand it’s part of the process.
But our techniques are different, our controls are solid, and we are a different set of people this time around. I think the community is going to be able to differentiate between the two papers.
PR: This is really a happy ending to a horrible story.
-What else has changed in your lab as a result of your experience with the two retractions?
BS: On a practical level, the lab implemented electronic notebooks, and established a system where only the lab manager can access the storage where we keep essential tools, so they don’t get passed around. Everybody also genotypes any materials they receive – even from a colleague in the same lab.
PR: More generally, I have a newfound respect for my people. Truly. I would definitely say I never want to go through another retraction again, but we all benefited in some weird ways as a team, as scientists, and as human beings. The perseverance and loyalty of my new team, the people who dedicated so much time and effort just to correct the record, moved me to tears. I had a new postdoc join the lab just before the retraction in Science appeared, and I had to explain the whole story to her, expecting her to immediately want to jump ship for another lab where she could do new science and not have to work in the shadow of such an experience. She said: “Yeah, I know about the retraction. I want to work in your lab anyway.” And she ended up being instrumental to our newest paper.
I took it for granted that I had a happy lab for 20 years. Then, when the retractions came, I saw them in the pits of despair, feeling like the scientific process had failed them. But when it became clear what had gone wrong, we could scientifically move through the experiments again – that reaffirmed the scientific process. What an accomplishment by a team of people to put that behind them and move forward.
And it wasn’t just people in my lab who amazed me – I received overwhelming community support. Strangers called and wrote supporting letters. It really helped me crawl out from underneath the bed during the saddest days. It felt like there were a lot of hands reaching out saying “keep going.” I think I didn’t know that was part of science.
-I know it must have been a difficult experience, but are you glad you “did the right thing” and reacted swiftly/honestly when you realized something was wrong with your previous data?
PR: I get asked this a lot but I think it’s a misconception. For me, there was never a choice. I just wanted to get it right and make the information public as soon as possible before we wasted anyone’s time trying to reproduce experiments that we ourselves could not reproduce.
We care about this work more than anybody, and there was just no question about whether or not we were going to correct it. But there were many questions about the process of correcting it that we couldn’t answer – we had no resources to help us, no precedent to work off of about how to handle this. Retractions always happened to other people.
For instance, people think: “If you’re wrong, just correct it right away.” But it’s not so easy to know when you are really “wrong”—how many experiments do you need to do to know, for sure, that your original data were not correct?
I also now understand why people don’t retract papers they know are wrong – I hear many stories from people who want to “do the right thing,” but don’t have the time, funding, and person-hours needed to get it done. I used two staff scientists – Randy Ruan and Mawsheng Chern, major heroes – for 18 months to redo the work we’d already done. They were close to full-time – maybe 70% — on this one project. It was a huge time commitment but I didn’t even consider the alterative, because I was at such a critical point in my research.
BS: Yes, I agree that it should be easier to retract papers – I was lucky that the Human Frontier Science Program, which awarded me a postdoctoral fellowship, let me change my research direction to fix this problem, away from what I was supposed to work on originally.
I also think there should be less of a stigma attached to retractions. I think retractions should become a normal part of doing science, maybe even a positive thing. In my opinion, it’s always going to be better to retract a paper and publish something that is 100% correct, instead of posting corrections here and there, or partial retractions. It’s just much easier for people.
In our case, I’m absolutely convinced retraction was the only way forward. After joining the lab in 2011, I — together with others, such as Randy, Mawsheng, Rory Pruitt, and Ofir Bahar — spent 1.5 years trying to replicate the original finding and couldn’t. During that time, I would talk to other people in the field who were really excited to have this new ligand to study, and had spent a lot of time and money on their experiments, but couldn’t repeat our findings.
The only way forward was to clear the road and start again.
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post. Click here to review our Comments Policy.
Regarding the stigma attached to retractions, I believe that this is something that should be given serious attention by the scientific community. Perhaps a system can be developed that would readily differentiate between various types of retractions. For example, the term ‘withdrawal’ can be used in situations in which honest errors are made, whereas the term ‘retraction’ can then be left for those situations that are due to FFP misconduct or some other questionable practice (and let the stigma stay for those!). Alternatively, different types of descriptors could be associated with ‘good’ retractions vs. ‘bad’ retractions. For example, a number scheme could be used: Level 1 retractions to indicate known FFP misconduct; Level 4 retraction for honest error; Level 2 or 3 retraction (depending on the available evidence) can be used for possible minor misconduct or other questionable practice, etc. It’s just a thought
Let’s not complicate things further. Last thing we need is a psychological test to differentiate “levels” of errors. Stick to the science and stick to the facts. That should be the only thing that leads to a retraction or an erratum. No stigma should be attached, just a correction. However, the exception could be multiple retractions, in which a pattern of error exists, then one could begin to say that there may be a problem. In the Ronald case, there were errors, there were two retractions. The literature was corrected. Trust was reinstilled. No-one disbelieves Ronald et al. Simple.
A good idea, but which may not be applicable in the many cases in which obfuscation and collusion by authors, their institutions, editors and publishers make it impossible for outsiders to judge whether the root cause of the retraction was honest error or misconduct.
In any case, authors, editors and publishers who face the problem head on, with honesty and transparency, by publishing complete information on what went wrong and why, deserve to be singled out publicly for praise and congratulations. Miguel is absolutely right that there should be no stigma attached to honest error and honest efforts to correct the record — quite the opposite!