Last year, we wrote about two retractions by Pamela Ronald and colleagues, after the group found that a bacterial strain they’d been using was contaminated.
The group has now published a paper in PeerJ following their investigation into what went wrong. Ronald tells us the new paper, titled “The Xanthomonas Ax21 protein is processed by the general secretory system and is secreted in association with outer membrane vesicles,”
…provides experimental details on some of the experiments we carried out during the 18 month process of investigating the function of Ax21 and also describes additional experiments we conducted to decipher the function of this protein.
We also asked Ronald about criticisms — voiced in comments here at Retraction Watch and elsewhere — that in a narrative about the retractions, she had not cited two papers that had allegedly found her group’s work irreproducible. Those two papers — in PNAS (Danna et al) and the Plant Journal (Mueller et al) — were not cited in the PeerJ paper either. She explains:
The Mueller et al paper challenges the Danna et al paper – not the retracted papers from my lab. My two rice/Xanthomonas papers that were eventually retracted were not challenged by any group. We discovered the mistakes ourselves when reproducing our own experiments. The experiments described in the Danna et al paper were carried out in Arabidopsis in Fred Ausubel’s lab.
We did not cite the Danna et al and Mueller et al papers in our PeerJ publication because the experiments are unrelated. Yes, they are both in the field of plant pathology, and yes, the Danna et al experiments were initiated because of our Science publication, but that is where the similarity ends.
In the Peer J paper, we examined the function of the Ax21 protein in the bacterial pathogen Xanthomonas oryzae pv. oryzae, which infects rice. The Danna et al paper studied effects of “Ax21-derived” peptides on immune responses of Arabidopsis. The Danna et al paper uses a different plant species, different assays, and different pathogen species. Also it, addresses a different question. Danna et al investigated whether Ax21 or “Ax21-derived (also called A1 peptide)” peptides could trigger Arabidopsis FLs2-mediated resistance to Pseudomonas. Because rice Xa21 and Arabidopsis FLs2 are both stuctural similarities, Danna wanted to test if any of the 40+ receptor kinases in Arabidopsis also could recognize peptides related to Ax21.
The only overlap with our retracted Science paper is that Ausubel’s group examined a set of alanine scanning mutants that my lab had tested in rice (see Fig. S4A of the the retracted science paper). In Danna’s experiments, Ax21 did not trigger immunity Arabidopsis; only the A1 peptide was active in their experiments.
Mueller et al proposed that flg22 contamination in the A1 peptide preps could explain the results of Danna. This was a question that our team considered in the original publication. To address this question, Danna et al resynthesized the A1 peptide and carried out mass spectrometry analysis and flg22 dose–response experiments, which did not reveal any obvious contamination. However, contamination is still a possibility. To resolve the question, the Mueller et al and the Danna et al labs will need to carry out the same experiments with the same reagents.
For more details, please see: “FLS2-Mediated Responses to Ax21-Derived Peptides: Response to the Mueller et al. Commentary” in Plant Cell. [This commentary includes Danna, Ronald, and Ausubel as authors.]
In the PeerJ paper we discuss the more relevant publications- e.g. the paper from an Irish lab that reproduced the results of our retracted PLOS ONE paper.
I wish to use the Ronald story and case to expound on the frustrations being felt as a plant scientist in seeking justice in science publishing as part of an ongoing, and painful, effort to complete a post-publication peer review (PPPR) of the plant science literature. Bear with me. Explaining the intricacies is not easy.
The effort by Ronald and colleagues is admirable, no doubt. This is an admirable effort, no doubt. It is also a role model case, I believe, for others to follow. Previous critiques of the paper’s experimental results looked at things in a rather absolute light. Which is also correct, considering that these results were published in the world’s top science journals. The other exemplary action taken by Ronald and colleagues relates to accountability towards peers and society. Ronald herself responded publically to critics on this blog (and perhaps elsewhere), always frankly (at least so it seems), and diplomatically. If ever correcting a negative situation (or publication) could come close to perfection, then I think that this would be the model case. Simply because no situation is perfect, and since humans, by nature, make mistakes. So, in this case, malicious intent – which was somehow suggested in previous stories on Ronald – can be removed from the table completely. I personally don’t see how a corrective measure could get any better than this.
In some ways, this case also makes mockery of the RePair system in the US, which is another program that wastes public funding to try and rehabilitate scientists who have made errors, been found to be guilty of lack of ethics, or fraud. What Ronald shows is that if there is a personal professional conviction to set the record straight, that it can be done, although this would involve a painful amount of time, financial and human resources. No need to waste more tax-payers’ money. Those who want to auto-repair, or to repair what went wrong, will do so of their own initiative. Those who are forced to do so are risky players in the game and may continue to pose a future risk because they will play their cards more smartly in the future, even if to conceal further acts of malice.
However, what if we were to look at things in a relative light?
The efforts made by Ronald et al. would not be possible, I believe, in 95% of the world’s labs around the world involved with plant science or plant-related science. Firstly because lab funding is already pretty strained. Secondly, because a lot of laboratories have limited staff and researchers, so to find the time to repeat the results would be almost self-implosive. Without knowing Ronald herself, or her colleagues, I would guess that funding and human resources would not be limiting, so their ability to repeat the experiments, and then discuss its reproducibility would be possible. It would have two functions: 1) to save face in the light of critics and to shoo away unwarranted critics (including competing laboratories); 2) to regain trust that would save her position and possible future research grants. If this happens, then the system (and peer pool) is self-correcting and should be applauded.
However, imagine we were to sample scientific papers randomly from the plant science literature that had suspect data sets, or conclusions based on insufficient proof, which in some ways could be a parallel situation to the Ronald case. How would we be able to ensure that such experiments that are not of high profile get re-examined? How could we demand cross-national pressure on such laboratories to repeat the experiments to prove that the data sets and claims made are accurate? How could we, basically, ensure trust in published results if there is – at least in plant science – simply no appetite for reproducibility, for detailed scrutiny of the literature, or for applying pressure on groups to provide answers to critical questions about their work?
I wish to extrapolate from the Ronald case outwards into the broader plant science literature.
I will focus on chrysanthemum (sorry if this bores the bloggers) because this is a plant whose responses in vitro I truly understand and because of my career-long dedication to plant tissue culture. For those of you unfamiliar with plant tissue culture, it is a really fundamental technique that forms a basic backbone to a lot of lab-based studies in plant science, including molecular and applied techniques, possibly even some related to the labs that form part of the Danna, Ronald, and Ausubel laboratories. I will focus on chrysanthemum to expand on a previous listing I made on RW (http://retractionwatch.com/2014/01/07/journal-dumps-grain-paper-for-controversial-data/#comments) that already highlighted very specific cases of partial self-duplication of text (text recycling), data, tables and figures and questioned the ethics of the researchers, the lack of responsibility of the editors and the publishers for not doing anything about these cases. Ignoring the issue does not resolve it.
Let me take an extremely easy-to-understand case. Please observe the Verma (2012) paper:
http://pelagiaresearchlibrary.com/advances-in-applied-science/vol3-iss3/AASR-2012-3-3-1449-1453.pdf
To me, this must be one of the worst cases of chrysanthemum tissue culture research in the entire chrysanthemum literature in which almost every sentence carries an error. I can’t seem to find a single positive thing to say about this paper. But, I will list the main problems:
a) Poor, often incomprehensible English;
b) Incomplete literature revision in the Introduction and discussion, a serious case of snub publishing (http://www.globalsciencebooks.info/JournalsSup/images/2013/AAJPSB_7(SI1)/AAJPSB_7(SI1)35-37.pdf)
c) Incomplete and often irreproducible methodology. In this case, irreproducible does not mean that I have tried Verma’s protocol in the lab to reproduce the results obtained. Rather, it means that so much basic information is missing, that even if I wanted to repeat the Verma experiment, it would be impossible to do so.
d) There are no statistical analyses. Thus, data presented is inconclusive.
e) The photographic evidence is horrific.
So, to swing back to my comments related to the relative excellence of the Ronald lab in trying to fix the literature, how could we start to repair the Verma paper?
a) Would Verma be willing to respond to the dozens of criticisms about the paper?
b) Would Pelagia Research Library (www.pelagiaresearchlibrary.com), listed as a potential predatory open access publisher by Beall (entry 331; http://scholarlyoa.com/2014/01/02/list-of-predatory-publishers-2014/) be prepared to retract the paper based on BAD science?
The problem is that this paper remains in open access, the flawed protocol could be followed by young, ingenious researchers without any or little experience and, worse yet, the paper could be referenced, thus legitimizing this paper. Legitimization of bad, poorly-conducted science with no respect to basic research and publishing principles could be endangering science publishing more than acts of bad ethics, I believe (at least in plant science, this is now my gut feeling based on two decades of experience). So, in my view, the Verma paper, which should be retracted, not based on lack of ethics, but rather on extremely bad science and science publishing principles alone. There is no excuse or reason for publishing such bad science. What tangible benefits did Verma gain? Maybe one step towards a promotion? Maybe a way to secure a job, a salary, or a position? What does the Indian Ministry of Education think? Should India’s image, already quite tarnished with so many stories of corruption, be further damaged by such scientists? Why do other Indian scientists not act towards enforcing a retraction? The Verma paper also reflects a total failure of the review system. It enforces a key concept that make a publisher predatory: the fraudulent claim that peer review was conducted when it was clearly not (“Pelagia Research Library journals offering peer-reviewed, scientifically based articles and original research, this contains information that will assist you in understanding intricacies of sciences.”; “Pelagia Research Library Journals are multidisciplinary Peer-reviewed International Science Journals. The journals publish high quality reviews, full papers and communications in all branches of Sciences.”; “The articles are sent to the Editorial Board Members based on their research areas for reviewing. After receiving the comments from three reviewers and collectively drawing the conclusion, the comments are sent to the authors.”; http://www.pelagiaresearchlibrary.com/editorial-policies.html). How does one rid plant science of this scourge on one extreme end, while still being critical of cases like Ronald on the other extreme, when in fact, in a RELATIVE sense, what Ronald et al. did was exemplary?
Now, I briefly draw attention to five contentious cases regarding somatic embryogenesis. Those unfamiliar with the term, check some background information here (http://www.globalsciencebooks.info/JournalsSup/images/Sample/JJPS_1(1)1-6.pdf).
In chrysanthemum, the cases of reported somatic embryogenesis are extremely few. Thus, scientists involved with research in this plant will know that the chances of publishing a paper will increase (i.e., the chances of acceptance will be higher) if they can present a paper on somatic embryogenesis. To prove somatic embryogenesis, however, there should ideally be several key forms of proof:
a) Morphological proof, including of all four stages of the process;
b) Histological proof that shows the origin of tissues and the different developmental stages;
c) Molecular proof showing the expression of embryogenesis-related genes in somatic embryos.
Even the wider literature on somatic embryogenesis in plants, sensu lacto, fails to provide proof c). However, in my opinion, proof a) and b) are indispensable to make such a bold claim. If not, then basically any structure that forms on the surface of an explant can be classified as a somatic embryo. This, in my opinion, is a fake (i.e., unsubstantiated) or fraudulent (i.e., based on false assumptions and misleading proof) situation. Notice how this “gap” in the literature has been explored in 2010-2013 after knowledge that it is possible to publish papers on this rare developmental event, simply because a top level plant science journal (Biologia Plantarum) published by a top publisher (Springer) endorsed unsubstantiated research in the key Mandal and Datta (2005) paper.
I believe that this is the case in the following five papers (see final footnote #):
Barakat et al. (2010)
Kereša et al. (2012)
Mandal and Datta (2005)
Naing et al. (2013a, 2013b)
Xu et al. (2012)
Once the Mandal and Datta paper had been red-stamped as being OK to publish papers on somatic embryogenesis that were unsubstantiated by sufficient proof, the flood gates were opened to abuse. And, most likely, editors of journals, in a desperate need to secure papers that carried “original” information or data, fell for the trap. At least, this is my personal interpretation of at least the three subsequent Springer papers that all passed through apparent “peer review” and that approved the publication of papers that had insufficient proof of what they were claiming. Could the temptation of getting a paper published that contained something “novel”, “fashionable” or “original” (even if unsubstantiated) have driven the editor boards (and particularly the Editors-in-Chief) of Biologia Plantarum, Plant Cell, Tissue Organ Culture and Acta Physiologiae Plantarum, three very key (respected and reputable) journals in basic plant sciences to have approved the publication of these papers? How will they now deal with these facts and now claims made public? In one case, Biologia Plantarum, the case was already officially reported to the then (and still) editor-in-chief Jana Pospíšilová and to the editors and to Springer. Rather than taking the claim seriously, I was labelled as some sort of an irritant to the system, although, fortunately, one editor did state that the paper was published with unsubstantiated evidence of the claims of somatic embryogenesis. Why has Springer, specifically the Publishing Editor, Maryse Elliott, sat in silence, despite my formal complaint(s)?
What this short and simplified case reveals is that, as I have already claimed several times, is that there is little appetite by the status quo in plant science and plant science publishing, to deal with such claims. No (or extremely) errata, no corrigenda, no retractions. The pride of such individuals (and the image of the publisher) seems to be superior to the basic corruption of knowledge the science literature. This situation and this attitude towards the already published literature seriously needs to change.
In conclusion, I applaud, once again, Dr. Ronald, for doing the right thing. I apologize to non-plant science bloggers and readers at RW for this long story. And I encourage my colleagues in plant science, many of whom know my passionate desire to address such issues and inefficiencies in plant science, to start to critically re-evaluate the already published literature as there are tons of errors and bad science, ethical violations and fraud. If we do not work together as a community, then all of our efforts to conduct research, and publish its results, will have been in vain.
References used (including country of origin for consistency):
Barakat MN, Abdel Fattah RS, Badr M, El-Torky MG (2010) In vitro culture and plant regeneration derived from ray florets of Chrysanthemum morifolium. African Journal of Biotechnology 9(8), 1151-1158 (Egypt+Saudi Arabia; Academic Journals*)
http://www.academicjournals.org/article/article1380894398_Barakat%20et%20al.pdf
Kereša S, Mihovilović A, Barić M, Židovec V, Skelin M (2012) The micropropagation of chrysanthemums via axillary shoot proliferation and highly efficient plant regeneration by somatic embryogenesis. African Journal of Biotechnology 11(22), 6027-6033 (Croatia; Academic Journals*)
http://www.academicjournals.org/article/article1380984793_Keresa%20et%20al.pdf
Mandal AKA, Datta SK (2005) Direct somatic embryogenesis and plant regeneration from ray florets of chrysanthemum. Biologia Plantarum 49(1), 29-33 (India; Springer Science+Business Media)
http://link.springer.com/article/10.1007%2Fs10535-005-0033-6
Naing AH, Kim CK, Yun BJ, Jin JY, Lim KB (2013a) Primary and secondary somatic embryogenesis in Chrysanthemum cv. Euro. Plant Cell, Tissue Organ Cult. 112, 361–368 (South Korea; Springer)
http://link.springer.com/article/10.1007/s11240-012-0243-5
Naing AH, Min JS, Park KI, Chung MY, Lim SH, Lim KB, Kim CK (2013b) Primary and secondary somatic embryogenesis in Chrysanthemum (Chrysanthemum morifolium) cv. ‘Baeksun’ and assessment of ploidy stability of somatic embryogenesis process by flow cytometry. Acta Physiologiae Plantarum 35, 2965-2974 (South Korea; Springer)
http://link.springer.com/article/10.1007/s11738-013-1328-4
Verma OP (2012) Standardization of auxin concentration for root induction in Chrysanthemum morifolium. Advances in Applied Science Research 3(3), 1449-1453 (India; Pelagia Research Library)
http://pelagiaresearchlibrary.com/advances-in-applied-science/vol3-iss3/AASR-2012-3-3-1449-1453.pdf
Xu PS, Zhang ZL, Wang B, Xia XY, Jia J (2012) Somatic embryogenesis and plant regeneration in chrysanthemum (Yuukou). Plant Cell Tiss Org Cult 111:393–397 (China; Springer)
http://link.springer.com/article/10.1007/s11240-012-0201-2
* Also listed as an potential predatory open access publisher by Beall (entry 8, list 1), and that in fact lost its impact factor after having been found guilty by Thomson Reuters of manipulating the literature to enhance its IF.
# This claim of unsubstantiated evidence does not mean that other aspects of the papers are not valid, good, or correct. Although the papers do have other faults that will be released in a PPPR report later on, these are tiny compared to the lack of sufficient evidence regarding claims of somatic embryogenesis in chrysanthemum.
Re: Jaime’s comment: Longest retraction watch post ever?
Dr Ronald is also an author on the Dana et al paper and this paper is so directly related to the original rice publications it takes up about half the abstract. Rice, if not exactly an esoteric species, is not as widely studied as Arabidopsis. So it is hardly surprising that Mueller et al didn’t feel the need to duplicate their negative findings in rice. In anycase Mueller was just saying the work could not be reproduced and offering ways to explain why.
There isn’t really a distinction between Ax21 protein and Ax21 derived peptides for these purposes. It is quite common to used peptides which you believe contains the biologleical signalling activity of the original protein. It is far easier to ring up a company and order a peptide and as many mutations as you want to investigation and have them delivered in 48 hours than get a student to clone and purify the full length protein. Conceivably a full length protein might contain properties not found in a derived peptide, it would be unusual for derived peptides to contain properties that aren’t at least conditionally found in the full length protein.
I guess the question should be is the Dana et al paper – of which Dr Ronald is an author – going also to be retracted? Or is the position that Ax21 represents a plant immune function in Arabidopsis that was quite erroneously found originally in rice?