In 2008, a group of researchers published a paper in Current Biology reporting on what they said was a lungless water-loving frog in Borneo.
According to David Bickford, then of the National University of Singapore, and his colleagues, the Bornean flat-headed frog “breathed” the way most salamanders do: by absorbing oxygen through their skin or, during earlier phases of life for some species, through gills. (We’re not salamander experts, so if this characterization is a bit crude, don’t come for us.) Because the frog lived in fast-moving streams, the researchers reasoned, it could obtain adequate oxygen to meet its needs.
For the last 15-odd years, that understanding held. But in May, another team of herpetologists, using more sophisticated tools, said they’ve found evidence of lungs – tiny but functional – in the creatures. As the New Scientist magazine reported earlier this year:
David Blackburn at the Florida Museum of Natural History and his colleagues have taken another look at two of Bickford’s specimens. They ran them through a high-resolution micro-CT scanner and saw a respiratory system with small, thin lungs hiding in plain sight.
“Right away, both of us were like, why is there a hole on the bottom of the mouth? That’s not supposed to be there,” says Blackburn. He had spotted the glottis, where the mouth connects with the lungs.
Now, the paper has been retracted. Here’s the retraction notice for the article, “A lungless frog discovered on Borneo,” which has been cited 38 times, according to Clarivate’ Web of Science:
The authors previously claimed to have established lunglessness of the Bornean endemic aquatic frog, Barbourula kalimantanensis, by gross dissection and histological sections. However, these methods have been shown to not be sufficient to establish lunglessness (Blackburn et al., 2024, Curr. Biol. 34, R492–R493, https://doi.org/10.1016/j.cub.2024.03.017). Although the methods the authors used were necessary to demonstrate simple features of gross anatomy, newer techniques used by Blackburn et al. (i.e., high-resolution CT scanning) are vastly superior and were able to detect the greatly reduced lungs that clearly exist in Barbourula. Though substantial loss of function might accompany such a reduction in the size of lungs, the frogs’ habitat in fast-flowing streams and rivers still appears to be a key driver of lung reduction in this species. The authors regret that they prematurely came to a false conclusion that Barbourula kalimantanensis has no lungs based on methods that have now been shown to be unreliable in detecting greatly reduced organs.
Bickford did not respond to a request for comment.
Blackburn told us:
I led a large-scale collaborative project called oVert (https://www.floridamuseum.ufl.edu/overt/; https://doi.org/10.1093/biosci/biad120) in which we were creating high-resolution 3D anatomical datasets, mostly using CT-scanning. As an amphibian biologist, I was interested to generate datasets not only for exemplars across amphibian diversity but also of some taxa with interesting phenotypes. Thus, one species that we chose to CT-scan, as well as to create additional contrast-enhanced CT-scans to visualize soft tissues, was the “lungless frog”, Barbourula kalimantanensis. It was only after we began investigating these datasets that we realized something had been overlooked in the past. We often scroll through these x-ray cross-sections of these datasets beginning at the snout and scrolling backwards. We quite quickly noticed that there appeared to be a glottis and trachea, which, of course, wasn’t supposed to be there. That then led us to investigate this in more detail, including physically inspecting the two specimens on loan (two of the same specimens that had been investigated by Bickford et al. 2008).
To be clear, David Bickford has been a colleague for many years, and there was no animosity between us about this paper. Sometimes scientists take a second look and with different techniques and realize that not all is as it first seemed.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
Fascinating that they chose to retract the paper – it seems that it would be better to leave it. There was no ethical misconduct, data manipulation, statistical error, etc – only a finding overturned with better technology, which happens all the time. With the paper retracted, subsequent researchers won’t have the caution that gross dissection is inferior to imaging for detection.
Not sure this is a problem – that now-retracted paper has been cited, there’s a retraction notice that explains why it is retracted, and people can still read the now-retracted paper. I think with the paper retracted there will be more researchers that will encounter this caution. It is always possible they find the now-retracted paper, and not the newer one that shows its methodology was inferior to imaging.
This paper should not have been retracted. A follow up should have been published.
As a lawyer, we are faced precisely with this issue when we research caselaw.
That is, when we rely on or cite a particular case as authority for a legal principle, we need to ensure that the case has not been overturned (whether on appeal or by another subsequent case).
While overturned cases are certainly not removed from legal databases, most services will indicate that a case has been overturned or questioned (usually by the presence of a small red icon).
To be sure, careful lawyers know that the icons assigned by the legal database, though usually pretty accurate, should never be absolutely relied upon, and as a result should snoop around a little bit more just to ensure they haven’t missed anything.
Because litigators like to tell stories, I’ll tell one that was once told to me:
In the olden days, in his oral submissions in court, a lawyer cited a particular case contained in a volume of caselaw (obtained from the Courthouse), only to be met with an immediate objection by opposing counsel that the case had been overturned.
Somewhat embarrassed, the lawyer then proceeded to rip out the pages of the case from the volume, saying “Well we won’t be needing that anymore!”, as the courtroom erupted in laughter.
There is no reason to retract a paper because its findings are superseded many years later with better techniques. Science builds knowledge incrementally, and wrong conclusions are the typical by-product of this process. That’s why science is self-correcting. Retractions should be reserved for cases of scientific misconduct. Otherwise nothing will be left of older research in most fields.
@Frank: I am 100% with you on that. That’s why studying history of science is so much fun – it tells you so much about the trials and tribulations of the research process. Withdrawing from the published record all but those results which have stood “the test of time” (up to now …) would deprive us of an invaluable body of knowledge.
This being said, there is of course a ‘body of research’ (social priming, for example) which even at the time was so ridiculous that you one wondered how anyone could take it seriously (but that’s a matter for sociologists to consider – a discipline which, since the ’90s, has recovered from its follies). And then there are (RW’s specialism) cases of blatant fraud, where again we’d have to to study those who fell for it instead of asking themselves “How exactly does this work?”
This retraction itself is a scientific misconduct. Where do COPE and other guidelines instruct retraction of old and outdated evidence in light of new evidence?! I think this editor can and should be questioned and even sued by the authors and scientific community for scientific misconduct and defamation.
Retraction is a scientific PUNISHMENT. It shouldn’t be handed out like candies. As Frank and Rebecca said, retraction should be reserved only for fraudulent and/or unethical authors, NOT for honest researchers.
Yes!!!!!
This was not retraction-worthy or even a matter for retraction watch. It could be paralleled to cognitive science, where new findings and more advanced behavioral or brain-scanning techniques change our understanding of the mind and brain. Old, and now inaccurate, studies were left behind but not retracted, thus leaving a trail of scientific advancement.
Although I appreciate Marco’s reasoning, I’m in agreement with you, Frank, and others who believe the paper should not have been retracted. But, given the global reach of RW, it seems to me that this is precisely the right venue for having a discussion about the merits of this retraction.
Allow me to point out I did not say I agreed with the retraction. Rather, I disagreed with Rebecca Cohen’s claim this would make it harder for scientists to find out that “gross dissection is inferior to imaging for detection.” In fact, au contraire, as one now would have the paper that shows this and the retraction note pointing this out for the paper that used the dissection approach.
I would have guessed that a dissection would provide better evidence than a CT scan. Maybe a CT video can capture the lungs filling an emptying of air?
Exactly! The retracting editor committed a scientific misconduct here. Do COPE, ICMJE, or other authorities call for retraction of outdated results?
I take my word back. This retraction is NOT a scientific misconduct. I checked the guidelines and was surprised to see that “Honest Error” was among the reasons for retraction.
Of course smaller honest errors call for a correction and not a retraction. But bigger ones call for retraction.
Update: I guess my original comment holds, i.e., according to COPE, retraction should not be issued for old, outdated, and refuted results, if the reason for disapproval is simply new, better studies.
I think by the term “honest mistake”, COPE means something like the scenario you mentioned above (which is a form of innocent misconduct), not what I thought originally when I read COPE guidelines.
By “honest error”, COPE doesn’t mean those inevitable errors caused by weaker technologies, methodologies, and knowledge of the past which are inherent to science.
So what the editor of this journal did was against COPE guidelines. He should have simply issued and erratum, or ignored it altogether.
It seems to me that retraction was actually a good choice.
As Marco states, the article is still on line, marked with a retraction and explanation, so information is not lost and better information is added to the overall literature.
A retraction like this is not a blot on the reputation of the authors of the first paper; as noted above this is the way science works.
And given that this is the way science works, a clear path of the ideas results is the best outcome.
No one called for a retraction, there was no malfeasance, the world would have been fine without a retraction, but overall an improvement on the record.
There is no misconduct in any aspect.
In response to the comments framing retraction as “punishment”, maybe we as a community should reframe our understanding of retraction. Less about punitive measures and more as a way to signal future readers when new information fundamentally invalidates findings. No reason for it to be perceived as a stain on an academic record.
I think Schuyler nailed it with his answer and suggestion. Though I think there is no need to label old results in any way. Detailed to him.
One commenter stated “Retractions should be reserved for cases of scientific misconduct.” Another commenter stated “Retraction is a scientific PUNISHMENT… retraction should be reserved only for fraudulent and/or unethical authors, NOT for honest researchers.”
But retraction isn’t a “punishment,” and it isn’t only useful in cases of misconduct. On the contrary, if the results of a study are invalidated by any error—even an honest one—then retraction can help clean up the record. For instance, if it is discovered that a finding was based on a data entry error or a computational error or a miscalibrated instrument or a faulty assumption, then presumably retraction is appropriate—not to “punish” the researchers for their honest mistake, but rather to improve the literature and dissuade citation of the false finding. And it is obviously absurd to say the authors should sue over a retraction when it is the authors themselves who requested the retraction.
Some commenters also seem to be under the impression that retraction erases the study from history and removes information from the scientific record. That is not typically the case. More commonly—as is the case here—the article remains accessible but with a “RETRACTED” watermark and a note clearly stating when and why the article was retracted. That doesn’t obfuscate the record of scientific progress—it clarifies the record of scientific progress.
Andrew, thanks for your good comment. I take my word back. I checked it; “Honest Error” was a reason for retraction. About cleaning the literature as the main reason for retractions, I agree again. But correctly or incorrectly, many researchers, institutions and even journals do associate retraction with severely damaged reputation. And “RetractionWatch” is partly to thank for this image –though this image existed before RW too. Perhaps it was the other way around, ie, perhaps this image was the reason for RW to get started; after all if retraction is not bad, why should it be watched?
ps. It was very strange to see “Honest Error” among COPE’s reasons for retraction. With new evidence, refuted results that were obtained using outdated methodology and technology will be considered honest error. So journals should retract everything when new evidence refutes them! COPE should double-check this strange item.
It seems to me that it would be helpful to have two labels – “retraction” and, say, “outdated” or “disproven”. There is an important difference between scientific misconduct, which hinders the stepwise progression of scientific inquiry, and disproven findings, which are an inherent part of that progression. I think “retraction” should be reserved for the first, making it not applicable to the article in question.
I understand that retracted articles (usually) remain available for viewing with a notification as to why they were retracted. But that’s not really the point at the center of this debate. The word “retraction” contains a value judgement (linguist here to burst the bubble of objective definitions). As some of the more heated comments in this thread indicate, “retraction” suggests that something *should* be removed from the record. That value judgement is valid in cases of misconduct. It most certainly should not be valid when researchers do their due diligence and get it wrong (as others have pointed out, that is gonna be the case for pretty much all of our work one day).
Awesome comment! I agree that the term retraction carries a lot of negative connotation and should be reserved for punishment. COPE should exclude the item “honest errors” from its list of retraction reasons.
Honest errors should not get any label, to begin with. A cold label (even a neutral and more friendly one like “Refuted” or “Disapproved”) is unfair to someone who didn’t do anything wrong. Besides, what if only a part of findings are refuted?
I think old articles should at worst get an appreciative explanatory comment (preferably without any hard labels) that says that although the authors of this study are thanked for their valuable efforts, this or that part of their honest finding is now refuted by new evidence. This ensures that the authors are not tainted for life!
But why even a kind explanation? It too is unnecessary because a disapproval is already issued by the new, refuting study (and its following review articles and books and news). Every expert and search engine already know about it. So there will be no need to label each outdated article on its very webpage.
Kayfabe, under your proposed system, if it is discovered that the entire results of a study are based on a computational error, what should happen? Should the study not be retracted merely because it was an honest mistake?
Andrew, IMO such a huge calculation error is not an honest mistake. It is misconduct, not in its ethical/legal sense but in terms of lack of scrutiny and competence on the authors’ end. So I wouldn’t mind retraction.
By honest error, I meant cases where inevitable limitations in technology and methodology and prior knowledge are to blame, not an error caused by laziness, incompetence, or irresponsibility of researchers, which should indeed be marked boldly and even punished.
ps. If the error is small, an erratum is the way to go.
Update: I guess my original comment holds, i.e., retraction should not be issued for old, outdated, and refuted results, if the reason for disapproval is simply new, better studies.
I think by the term “honest mistake”, COPE means something like the scenario you mentioned above (which is a form of innocent misconduct), not what I thought originally when I read COPE guidelines.
By “honest error”, COPE doesn’t mean those inevitable errors caused by weaker technologies, methodologies, and knowledge of the past which are inherent to science.
So what the editor of this journal did was against COPE guidelines. He could simply issue and erratum, or ignore it altogether.
A question about “cleaning the literature by retraction”. COPE says that retraction is a way to prune incorrect studies. But many studies are OK methodologically; they are pruned (retracted) for legal/ethical problems outside the study itself like authorship cheating.
When the methodology and data/results are intact, should these studies be removed from the literature and be marked as unusable by a retraction? That would be lot of waste.
Perhaps it is better to keep the paper itself but somehow punish the authors at the same time using a new form of retraction that does allow for citation of the retracted paper. For example, something called “authors-only retraction”.
Retraction is appropriate for papers that would not have been published but for inadequacies in the review and editorial process. For purposes of research and scholarly citation in its field, the retracted paper never existed. Refuting a paper’s approach, findings and conclusions is an entirely different matter.
From my perspective, retractions are for correcting the record, full stop. They are not punishments. We have been trying for years (and years) to create a name for a “retraction lite” that is for honest error, and such a thing is still not standard or widely recognized. In the meantime, we continue to stigmatize error. This stigmatization is exactly the problem – people don’t correct the record because they are afraid.
Also, retracted articles are still posted, searchable and visible, and they are labeled as retracted, so transparency IS best served by retractions.
So I say – more retractions for honest error, please. Lots and lots of them. Kudos to the journal and the authors, for doing the right thing!
I think one can argue this was not so much an error, but rather a method that later turned out not to have sufficient ‘resolution’.
The main issue is that this insufficient resolution resulted in the surprising finding that some frogs may actually have evolved to have no lungs – a challenge to our understanding of the evolution of frogs. It now turns out it isn’t nearly as much of a challenge to frog evolution. If the old dissection method merely had the size of the lungs wrong due to its inadequacy, I doubt the paper would have been retracted.
Actually, this editor violated the COPE guidelines by retracting this paper (even at the request of its authors) because this paper was not a case of an honest error, which is a HUMAN error without any intention of wrongdoing or deception.
This frog paper was simply a methodological limitation of the study, that is a technology/methods error, not a Human error.
For example, without microscopes, we couldn’t see microbes. So should we call ancient medical theories of infection before the microscope “honest errors”? No. Those theories are wrong and silly by today’s knowledge, but are in no way honest errors.
1. what happened here was Not an honest error, to begin with. As I explained above.
2. As a side note: You are not correct saying that honest errors should be retracted to correct the record. Most honest errors are small and can and should be fixed with an erratum; no need for retraction at all.
3. Only big serious honest HUMAN mistakes that invalidate the whole article or most of it need to be retracted.
4. Finally, honest errors fall within the category of scientific MISCONDUCT, i.e., HUMAN errors without INTENT to cheat, but still not allowed in research. They are misconduct because they are caused by “misconduct” items e.g., negligence and laziness of the involved researchers (authors, reviewers, editors), lack of enough scrutiny by authors and reviewers/editors, proper supervision by thesis professors, lack of knowledge of reviewers/professors/editors, lack of responsibility of authors/editors/reviewers, etc., These are misconduct but without the INTENT to cheat.
————————————
Regarding the rest of your points:
1. The retraction lite concept is very nice. Thanks.
2. Besides, you are right that retraction is not designed for punishment. Though apart from honest errors, most forms of retractions (like those triggered by data fabrication or paper mills or unethical methods, etc) ** SHOULD ** be stigmatized as punishment, because some researchers should fear something, otherwise they will do literary everything they want.
3. Finally I agree that the greatest fear of many journal editors (let alone authors) is retraction!
It is not clear whether you could call this “not an error”. If they did not sufficiently validate their method, it IS an error.
Regarding your points 3+4, we have here a paper that makes a very large claim based on insufficiently validated methodology. This error (and you can argue a million times it isn’t, while others can argue it is) completely invalidates the article. The supposed first lungless frog, thereby challenging frog evolution, turns out to be a frog that does have lungs, thereby not challenging frog evolution.
Agreed. One might argue that the researchers should have been more cautious in their conclusions, since all known frogs had lungs. The researchers perhaps should have considered the possibility that they had failed to notice a tiny anatomical feature in a creature less than two inches long, rather than jumping to conclusions and assuming that they had discovered something totally unique that changed understanding of amphibian evolution. So I’m not convinced that the researchers did everything exactly as they should have. In any case, the researchers were objectively wrong, and they apparently support the retraction. So I don’t see a problem.
@Marco, some borderline cases may exist but at the end of the day, we can decide what is what. In general, I do agree with you that method errors can be considered a HUMAN error IF caused by negligence, i.e., not adequately checking their methodology before conducting the study. It will be an honest error and misconduct caused by negligence, and if too large, retractable.
————————————–
But in this frog case, their methodology was already the GOLD STANDARD methodology, i.e., the best method available that had already been validated a million times. The culprit was the frog’s lungs that were too tiny for their *gold standard* methodology. This isn’t a human (honest) error. This is not negligence. This is a methodological limitation.
No human made a mistake by using their gold standard of their time. It is not their fault (or honest error) if their gold standard was not enough.
Example: Before electron microscopes, they couldn’t see viruses under light microscopes. So their *gold standard* method was not adequate for detecting viruses. But was their microscopes ‘not validated’ too? At their time, their method was perfectly validated by thousands of earlier microscope studies. It was even the best available method. The culprit was their target finding (viruses) that was too small to be detected by their gold standard methodology, warranting newer technologies like electron microscopes.
No human ever made any honest error when they failed to see viruses under light microscopes. It wasn’t their “fault”.
This is obviously inappropriate use of retraction. Maybe the world needs a (different) way for journals to add a note “This article has been superseded by XXX”, to help those who are unable to carry out an effective literature search.
Retraction isn’t necessarily a punishment, but it’s a method to deal with a paper that should never have been published in the first place, because there was already something fundamentally wrong with it at the time. It’s not a way to correct a paper that subsequently turned out to be a mere stepping-stone at the start of a longer pathway of discovery, a paper flawed by the imperfect methodology of its era, or the preconceptions that then existed. The difficulty with retraction is that it tends to mark a paper as not worth reading, whereas often the early, honest papers are very much worth reading, by those who want to understand how modern understanding arose.
@Li, hypothetically speaking, if later it turns out that it was actually the newer study that was mistaken (false positive error), what should the journals do? Retract the newer paper and UnRetract the older one?!
No, unless the error is caused by misconduct (including negligence AKA honest error), journals mustn’t touch it.
@Li, you have a point, though I don’t agree with this: “[retraction]’s a method to deal with a paper that should never have been published in the first place, because there was already something fundamentally wrong with it at the time.”
I disagree with the part after “because”. According to COPE, being fundamentally wrong at the time *just because of being wrong* is not enough for retraction. For being retractable, the fundamental error needs to be caused either by negligence (honest error) or by other misconducts like fraud.
———————————————————
I for one agree with COPE here. And this frog paper was not a case of negligence. Please read my “virus” example said to Marco.
Ideas evolve through controversy and dispute. Many researchers were fundamentally wrong at their time. Sometimes, a groundbreaking paper (which stir controversy at its time) gets refuted later. Sometimes, it even gets re-confirmed again later (after the previous dispute). Many ideas remain in controversy with each other for decades. Many ideas Evolve and get polished through these controversies.
———————————————————
Should we keep the last version of each idea and retract all the previous ones?
In an unrelated search for information about retractions, I came across a 2012 RW post whose content and, especially IMO, many of the comments made in response to it, are relevant to the question of whether the lungless frog paper should have been retracted:
https://retractionwatch.com/2012/09/25/if-a-papers-major-conclusions-are-shown-to-be-wrong-we-will-retract-the-paper-plos/