“Barriers to retraction may impede correction of the literature:” New study

One of the complaints we often hear about the self-correcting nature of science is that authors and editors seem very reluctant to retract papers with obvious fatal flaws. Indeed, it seems fairly clear that the number of papers retracted is smaller than the number of those that should be.

To try to get a sense of how errors are corrected in the literature, Arturo Casadevall, Grant Steen, and Ferric Fang, whose work on retractions will be familiar to our readers, in a new paper in the FASEB Journal, look at the sources of error in papers retracted for reasons other than misconduct.

Here’s the abstract (emphasis ours):

Retraction of flawed articles is an important mechanism for correction of the scientific literature. We recently reported that the majority of retractions are associated with scientific misconduct. In the current study, we focused on the subset of retractions for which no misconduct was identified, in order to identify the major causes of error. Analysis of the retraction notices for 423 articles indexed in PubMed revealed that the most common causes of error-related retraction are laboratory errors, analytical errors, and irreproducible results. The most common laboratory errors are contamination and problems relating to molecular biology procedures (e.g., sequencing, cloning). Retractions due to contamination were more common in the past, whereas analytical errors are now increasing in frequency. A number of publications that have not been retracted despite being shown to contain significant errors suggest that barriers to retraction may impede correction of the literature. In particular, few cases of retraction due to cell line contamination were found despite recognition that this problem has affected numerous publications. An understanding of the errors leading to retraction can guide practices to improve laboratory research and the integrity of the scientific literature. Perhaps most important, our analysis has identified major problems in the mechanisms used to rectify the scientific literature and suggests a need for action by the scientific community to adopt protocols that ensure the integrity of the publication process.

The authors — like us — are troubled by corrections that by all accounts should have been retractions. They cite several illustrative cases:

For example a report in Nature in 1993 that combination antiviral chemotherapy halted HIV replication (43) was later found to be erroneous (44), but a correction was issued instead of a retraction (ref. 45 and Table 2). Recently, a bacterium was reported to incorporate arsenic rather than phosphorus into its DNA (46). This sensational finding was subsequently shown to be erroneous (47, 48), and the journal issued an editor’s note (49), yet the original article has not been retracted. In other instances, we note that authors of erroneous articles have responded to criticism in a prompt and transparent manner (50), and we laud their actions as an example of the self-correcting nature of science.

The authors compared retractions before and after 2000, when “several new technologies in the biological sciences, including next-generation DNA sequencing, mass spectrometry, and RNA interference,” were introduced:

First, the number of retractions that admitted irreproducibility without providing an explanation was reduced from 23.3 to 11.5% (P=0.0017). Although we do not know the explanation for this trend, it is possible that the cost of retraction in terms of reputation and prestige has led investigators to provide more information to support their actions and/or conclusions. Second, the number of retractions attributed to contamination was significantly reduced from 27 to 11.5% (P=0.0002). Here, the explanation may be improved analytical techniques that provide more information with regard to sample purity. Furthermore, the widespread use of kits for carrying out molecular and biochemical techniques could be associated with a reduction in inadvertent contamination of reagents generated by individual investigators. However, just because a reagent originates from a commercial source does not guarantee its purity, and the increased reliance on commercial reagents raises the possibility that contamination at the source could simultaneously impact many laboratories. Third, there was a significant increase in retractions attributed to analytical error, rising from 12.2 to 23.1% (P=0.01). Although it may be too early to identify the causes for this trend, it is possible that studies that generate large amounts of numerical data make data manipulation errors more likely (40).

They conclude:

Finally, and perhaps most important, our analysis has revealed major problems in the mechanisms used to correct the scientific literature. These problems range from inadequate information in retraction notices to the continued presence of publications known to be erroneous in the literature and the use of errata to report major flaws in articles that should instead be retracted. Both the scientific community and society are dependent on the integrity and veracity of the scientific literature, which is now being questioned in the general media (11–13). This raises concern that future public support for the scientific enterprise could be eroded, and scientific findings of major societal importance might not be heeded. We are hopeful that our findings, together with our previous report that the majority of retractions are due to misconduct (9), will stimulate discussion to develop standards for dealing with error in the scientific literature and actions to improve its integrity.

We asked Daniele Fanelli, who has also published a number of papers on fraud and retractions, for his take:

A few studies before this one had looked at the nature of errors in specific fields, but this is the first to examine errors in biomedical research that lead to retractions, and is probably the largest analysis of errors to date. This study is a nice complement to the authors’ previous analysis showing that most retractions are currently due to misconduct.

Retractions due to misconduct can be very misleading if one assumes that such retractions reflect the actual incidence of misconduct, because in reality they come at the end of a long and complex process, and therefore their prevalence mostly reflects strength and weaknesses in the process itself.

Error-related retractions are different. They are initiated directly by the authors, and can therefore be considered a less biased proxy of problems affecting research. But, to be more precise, they are still a proxy of problems affecting those research areas where scientists have finally taken up the noble practice of retracting their own flawed studies. Such practice is still extremely rare, so it is still very likely that all the statistics reported by the authors do not tell us much about errors in general, and tells more about which fields and countries have started retracting their mistakes.

These limitations notwithstanding, the statistics shown in the paper are very informative, and the suggestions made could help improve laboratory practices in some areas.

When the practice of retracting papers will have become common across all the sciences, this type of study could help us improve research in all fields and countries.