Earlier this week, we covered the case of a retraction that happened against one of the author’s wishes. That’s not all that unusual. What was unusual in this story, however, is that the author who objected to the retraction had published a well-considered paper in which she identified an error in the original work, and corrected it. That led many scientists on Twitter and elsewhere to ask: Doesn’t a retraction send the wrong message? Don’t we want researchers to correct and update their work?
One of the people asking those questions was Ben Ashby, an evolutionary biologist at the University of Exeter in the UK — who, it turns out, reviewed the corrective paper. Here, we present his thoughts:
The decision by the journal Evolution to retract a 2007 paper after one of the authors corrected a mistake in a later paper has hit a nerve, with many academics (myself included) taking to Twitter to support the authors and criticize the journal’s handling of the case (the story was first covered on Retraction Watch here). As a reviewer of the later paper, and indeed, the person who requested that the original paper be corrected, I felt it was important to publicly express my opinion on the matter.
The story started in 2007, when Hanna Kokko and Bob Wong published a theoretical paper in Evolution about sex roles and mate searching. Several years later, Kokko (University of Zurich, Switzerland) decided to revisit the topic with Lutz Fromhage (University of Jyväskylä, Finland) and Michael Jennions (Australian National University). Fromhage spotted key mistakes in the original analysis of the model, which meant that Kokko and Wong had incorrectly concluded that the evolution of mate searching is not influenced by the unavailability of females (e.g. due to pregnancy).
The analytical error led Kokko and Wong to suggest that the reason males are usually the searching sex is instead due to polyandry. The corrected analysis, included in the later paper, contradicted these conclusions: time spent unavailable for mating does weaken selection for mate searching, and so it is not polyandry per se that causes male-biased searching but rather the presence of a mating window during which the timing of mating does not matter for females.
In fall 2015, I was asked to review the new paper, which highlighted these problems with the original paper. I concluded that the new study was solid, well-written and that I would be happy to see it published in Evolution. I did, however, raise the following concern:
The manuscript corrects significant technical problems with [an] earlier paper in this journal (Kokko and Wong, 2007). These problems need to be highlighted at source (i.e. as corrections/erratum next to the original paper), with readers directed to the new paper for a fuller exploration of the corrected model.
I believe this would have been a reasonable course of action, as it would have ensured that readers of the 2007 paper would be aware of the errors without purging the paper from the scientific record. At no stage did I feel that a retraction would be appropriate.
In the decision letter the Editor and Editor-in-Chief agreed that:
it would be most appropriate to publish a separate erratum to the previous paper. This could then be linked to both the previous paper and the current [manuscript]. In that case this current paper would not have to change the way it is handling the previous paper much at all.
At this stage, my role in the review process ended and I was unaware of the outcome until I saw the Retraction Watch article earlier this week. I’ve since discovered that the authors were unable to publish an Erratum on the grounds that the author list was not identical to the original paper. Fromhage et al. therefore submitted a Technical Comment to Evolution to highlight the problems with the original paper at the source. I was not invited to review the Technical Comment, which was rejected by another reviewer and the Associate Editor, both of whom recommended retraction of the original paper. The journal then retracted the original article and published the new paper. The decision as to who to invite to review is obviously up to the journal, but I think it was a little odd in this case seeing as I was the person who had requested the original article be corrected.
So why have I spoken out about this case? The problems of post-publication peer review have already been highlighted elsewhere, and it certainly isn’t rare for a paper to be retracted due to an honest mistake (although most retractions are due to misconduct). Moreover, one could argue that the mistakes in Kokko and Wong’s 2007 paper were sufficient to warrant a retraction as they significantly affected the conclusions. But by that logic, a large number of empirical studies should also be retracted due to incorrect statistical analyses or overreliance on fickle p-values, leading to irreproducible results.
My concern is that the forced retraction of the original paper sends a bad message to the scientific community. Kokko has effectively been penalized for critiquing her own work, when in fact she should be applauded for her honesty. We should be encouraging others to emulate Kokko’s approach, but I fear that Evolution’s decision will only serve to deter such behavior. In an era when funding decisions and faculty positions depend so heavily on publications, it is asking much of scientists to put aside self-interest and be openly critical of their own work if this risks damaging their future prospects.
The most important thing for science is that mistakes in the literature are identified and resolved as quickly as possible, which is why I suggested the problem be corrected at the source. Publication does not grant some special truth status; we should always critique the literature and not assume that results or conclusions are true simply because they have undergone peer-review. However, highlighting and correcting errors does not necessitate airbrushing the scientific record.
In the case of Kokko and Wong (2007), does a retraction achieve anything that an Erratum or Technical Comment could not? In my opinion, no. The editors were no doubt in a difficult position; they often need to err on the side of caution and the merits of a retraction in this particular case could be argued either way. But their decision has implications beyond this case and risks dissuading people from openly addressing their mistakes. Forced retraction is unavoidably associated with a certain degree of stigma, given its association with misconduct (although there was no suggestion of misconduct in this case, simply an honest mistake). Forcing a retraction on the authors therefore feels like a punishment, which is probably why this story has resonated with so many people.
Scientists are not perfect. Honest mistakes will happen along the way, but it’s important that we encourage people to admit when they have gone wrong, and as one Twitter user put it: “not make doing what is good for science bad for scientists.”
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
In theoretical modelling, much of the progress that innovative articles make is often in identifying the problem itself and then in setting up simplifying assumptions that isolate the essence of the problem whilst enabling it to be tackled analytically or computationally. These can be much more important steps for the field than correctly analysing the model. Errors in the analysis are likely to be recognised subsequently and can then be corrected in a new paper, which happens not infrequently in behavioural ecology. Nevertheless the original erroneous paper can continue to have a healthy influence on how the topic is addressed in subsquent work. In the several important examples in this field that I can think of off the top of my head, there has been no correction or erratum attached to the original paper; researchers are expected to make themselves aware of the follow-up literature and find the correction. This is nowadays easier than ever and seems to work well enough. Sure, a correction linking to subsequent work might be helpful to young researchers, but one cannot add links to all subsequent developments in the field, many of which may cast doubt on the the original conclusions.
At least Hanna Kokko has another paper to replace the retracted one. We shouldn’t forget that the greatest victim of this silly retraction is Bob Wong, the second author of the retracted paper.
“Fromhage spotted key mistakes in the original analysis of the model, which meant that Kokko and Wong had incorrectly concluded that the evolution of mate searching is not influenced by the unavailability of females (e.g. due to pregnancy).”
The retraction was correct. The conclusions of the older paper were incorrect and thus invalid and misleading. By leaving the older paper published and intact would risk having scientists referring to, and citing, that paper. The new, corrected paper simply replaces the older one. The literature has been correctly corrected in this case.
A proper erratum achieves the same purpose and the risk of misleading scientists is a small price to pay. In fact, an erratum points out what was right and what was wrong, and that is equally important for scientists who want to learn from each others errors.
PJTV, I disagree. When some important, but not core, issues of a paper have errors, an erratum is perfectly suited. In this case, the basic premise and core hypothesis was wrong (i.e., disproved due to error). The retraction was right, even though the errors made by the honest author were unfortunate. The editors took the correct line of action.
It would have been much more useful to science to add a comment to the original article than to retract it. A publication serves to account for the work done by the scientists and clearly the work has been done. Mistakes happen, science evolves and sometimes old results and models become obsolete for one reason or another. The article, however, does not consist only from the conclusions, there is the justification, the data, the analytic model, the numerical model, whatever. They all serve a purpose to subsequent readers. Ok, in the new article they improved on their analysis and conclusion, but retracting article means denying and even deleting the account for the author’s work! This is simply not fair.
As long as there is not misconduct, there should not be forced retraction.
In physics for example, models and data get constantly reconsidered in the light of new knowledge. If all of the older articles get retracted by the journals, just because of some error, that would be simply harmful! Making mistakes is part of science and we all learn from them.
The Guidance from the Committee on Publication Ethics (COPE) states that journal editors should consider retracting a publication if they have clear evidence that the findings are unreliable, either as a result of
misconduct (e.g. data fabrication) or honest error (e.g. miscalculation or
experimental error). Here it seems to have been an unreliable result due to honest error. But retraction is still appropriate.
I understand that some might view a retraction in a negative light – but anyone with a mature understanding of science will know that a retraction is a pretty normal event nowadays. It would be interesting to hear Kokko’s perspective but
in such cases authors are often somewhat relieved that the issue is now behind them for good. A good scientist who has published something they know to be potentially misleading would surely not wish it to be cited in error. Having it hanging around your neck for the next couple of decades doesn’t sound like to me like a situation I would want.
Incidentally, retracted papers are still very frequently cited, so I’m not sure this is quite ‘airbrushing’.
Kokko’s perspective is represented in our original post, to which we link above: http://retractionwatch.com/2016/04/27/biologist-critiques-own-paper-journal-retracts-it-against-her-wishes/
Mostly I’ve already expressed my concerns (in the original coverage of this case by Dalmeet Singh). Here’s some additional thoughts:
Wouldn’t we all like there to be a on/off badge “still valid!” vs. “now known to be unreliable!” that could be seen on each paper that we download. It would make life so easy. But science does not work like that; the zones are always grey, as pretty much any contribution contains ideas & methods that are of lasting value as well as those that do not stand the test of time. What makes science so wonderful is that we have to learn to live with that and even enjoy it.
The kind of scientific progress I like is perhaps best exemplified by a case where authors [Fowler et al. in Ecology 2006 (87:655-664)] were able to put a subheading “We were wrong!” into their paper, and explain why their own previous conclusions are now untrustworthy and how now we know better. We should celebrate that, rather than pretend that the arduous path towards better knowledge never happened (the simplistic idea behind “this new paper simply replaces the old one; case closed” fails in my mind for that reason).
As I believe it’s better for young researchers to learn that it’s OK to admit one was wrong, my own policy as an editor of a journal – I do wear that hat too (though not as editor-in-chief) – would be to handle an equivalent case in a different way than was done here. The main reason is to avoid changing the incentive structures towards hiding one’s own mistakes, which based on Twitter responses has worried a lot of people. As Ben Ashby said, there’s milder ways that can achieve the same corrective outcome without the drama of whether this was now a reward or a punishment.
Retractions these days are heavily linked with scientific fraud, so that having (involuntary) retractions on your publication record may be taken by some funding agencies and hiring panels as possessing a scientific criminal record, disqualifying the person from any consideration whatsoever. That is an unacceptable penalty for an honest scientist to suffer merely for the purposes of correcting the “scientific record”. Retractions for fraud and corrections due to honest errors should be two separate things and not used interchangeably in an opaque fashion.
This is a knotty problem. As I have mentioned previously it might be helpful to have a “retraction number” attached to an article that is retracted.
I.e. 1st class retraction (1) = evidence of misconduct./fraud
2 = methodology inaccuracies/honest error
3 = author disputes
etc.
This would allow funding bodies (or anyone) a quick idea of why a paper was retracted without wading through (sometimes oblique) retraction notices.
Publishing a limited model, or one that is state of the art at the time but later advances prove that there is a fault in methodology, is very different than publishing a model that has mistakes that should have been caught at the time if the reviewers or authors were more careful. It seems like this is a case of the latter, so the retraction is warranted.
Authors in this situation should self-retract, so a forced retraction is unnecessary. That is truly teaching a young researcher that it is OK to admit that you are wrong.