About these ads

Retraction Watch

Tracking retractions as a window into the scientific process

“Barriers to retraction may impede correction of the literature:” New study

with 9 comments

faseb june 2014One of the complaints we often hear about the self-correcting nature of science is that authors and editors seem very reluctant to retract papers with obvious fatal flaws. Indeed, it seems fairly clear that the number of papers retracted is smaller than the number of those that should be.

To try to get a sense of how errors are corrected in the literature, Arturo Casadevall, Grant Steen, and Ferric Fang, whose work on retractions will be familiar to our readers, in a new paper in the FASEB Journal, look at the sources of error in papers retracted for reasons other than misconduct.

Here’s the abstract (emphasis ours):

Retraction of flawed articles is an important mechanism for correction of the scientific literature. We recently reported that the majority of retractions are associated with scientific misconduct. In the current study, we focused on the subset of retractions for which no misconduct was identified, in order to identify the major causes of error. Analysis of the retraction notices for 423 articles indexed in PubMed revealed that the most common causes of error-related retraction are laboratory errors, analytical errors, and irreproducible results. The most common laboratory errors are contamination and problems relating to molecular biology procedures (e.g., sequencing, cloning). Retractions due to contamination were more common in the past, whereas analytical errors are now increasing in frequency. A number of publications that have not been retracted despite being shown to contain significant errors suggest that barriers to retraction may impede correction of the literature. In particular, few cases of retraction due to cell line contamination were found despite recognition that this problem has affected numerous publications. An understanding of the errors leading to retraction can guide practices to improve laboratory research and the integrity of the scientific literature. Perhaps most important, our analysis has identified major problems in the mechanisms used to rectify the scientific literature and suggests a need for action by the scientific community to adopt protocols that ensure the integrity of the publication process.

The authors — like us — are troubled by corrections that by all accounts should have been retractions. They cite several illustrative cases:

For example a report in Nature in 1993 that combination antiviral chemotherapy halted HIV replication (43) was later found to be erroneous (44), but a correction was issued instead of a retraction (ref. 45 and Table 2). Recently, a bacterium was reported to incorporate arsenic rather than phosphorus into its DNA (46). This sensational finding was subsequently shown to be erroneous (47, 48), and the journal issued an editor’s note (49), yet the original article has not been retracted. In other instances, we note that authors of erroneous articles have responded to criticism in a prompt and transparent manner (50), and we laud their actions as an example of the self-correcting nature of science.

The authors compared retractions before and after 2000, when “several new technologies in the biological sciences, including next-generation DNA sequencing, mass spectrometry, and RNA interference,” were introduced:

First, the number of retractions that admitted irreproducibility without providing an explanation was reduced from 23.3 to 11.5% (P=0.0017). Although we do not know the explanation for this trend, it is possible that the cost of retraction in terms of reputation and prestige has led investigators to provide more information to support their actions and/or conclusions. Second, the number of retractions attributed to contamination was significantly reduced from 27 to 11.5% (P=0.0002). Here, the explanation may be improved analytical techniques that provide more information with regard to sample purity. Furthermore, the widespread use of kits for carrying out molecular and biochemical techniques could be associated with a reduction in inadvertent contamination of reagents generated by individual investigators. However, just because a reagent originates from a commercial source does not guarantee its purity, and the increased reliance on commercial reagents raises the possibility that contamination at the source could simultaneously impact many laboratories. Third, there was a significant increase in retractions attributed to analytical error, rising from 12.2 to 23.1% (P=0.01). Although it may be too early to identify the causes for this trend, it is possible that studies that generate large amounts of numerical data make data manipulation errors more likely (40).

They conclude:

Finally, and perhaps most important, our analysis has revealed major problems in the mechanisms used to correct the scientific literature. These problems range from inadequate information in retraction notices to the continued presence of publications known to be erroneous in the literature and the use of errata to report major flaws in articles that should instead be retracted. Both the scientific community and society are dependent on the integrity and veracity of the scientific literature, which is now being questioned in the general media (11–13). This raises concern that future public support for the scientific enterprise could be eroded, and scientific findings of major societal importance might not be heeded. We are hopeful that our findings, together with our previous report that the majority of retractions are due to misconduct (9), will stimulate discussion to develop standards for dealing with error in the scientific literature and actions to improve its integrity.

We asked Daniele Fanelli, who has also published a number of papers on fraud and retractions, for his take:

A few studies before this one had looked at the nature of errors in specific fields, but this is the first to examine errors in biomedical research that lead to retractions, and is probably the largest analysis of errors to date. This study is a nice complement to the authors’ previous analysis showing that most retractions are currently due to misconduct.

Retractions due to misconduct can be very misleading if one assumes that such retractions reflect the actual incidence of misconduct, because in reality they come at the end of a long and complex process, and therefore their prevalence mostly reflects strength and weaknesses in the process itself.

Error-related retractions are different. They are initiated directly by the authors, and can therefore be considered a less biased proxy of problems affecting research. But, to be more precise, they are still a proxy of problems affecting those research areas where scientists have finally taken up the noble practice of retracting their own flawed studies. Such practice is still extremely rare, so it is still very likely that all the statistics reported by the authors do not tell us much about errors in general, and tells more about which fields and countries have started retracting their mistakes.

These limitations notwithstanding, the statistics shown in the paper are very informative, and the suggestions made could help improve laboratory practices in some areas.

When the practice of retracting papers will have become common across all the sciences, this type of study could help us improve research in all fields and countries.

About these ads

Written by Ivan Oransky

June 16, 2014 at 10:00 am

9 Responses

Subscribe to comments with RSS.

  1. Just can’t admit they made a mistake.

    ed goodwin

    June 16, 2014 at 10:13 am

  2. Link to the abstract in your third paragraph seems to be a dud.

    Toby White

    June 16, 2014 at 10:53 am

  3. The authors — like us — are troubled by corrections that by all accounts should have been retractions. They cite several illustrative cases:
    Why is that troubling? Scientifically it sounds very similar.
    Maybe corrections are often even better for science because that allows one to still read the paper and to be able to judge whether papers that used the corrected paper are invalidated by it. Even if the main conclusion of a paper is found to be wrong another paper might cite it for a side result, a smart computation, a methodological innovation. In such cases, it would be a pity if that citation went into a black hole.
    Both the scientific community and society are dependent on the integrity and veracity of the scientific literature, which is now being questioned in the general media (11–13).
    Maybe the media should change? Stop with their naive believe that science is infallible. Only religion is infallible.
    And it would be nice if they would stop reporting on single new papers with spectacular results and would report on new developments in a scientific field during the last years. I know that that is a naive request. That would be more work and journalists would not consider that news any more. But it would have a much higher quality and would provide much more value to the readers.

    Victor Venema

    June 16, 2014 at 12:38 pm

    • Thanks for your comment. A general guideline, endorsed by COPE, is that corrections (also referred to as ‘corrigenda’) should be reserved for minor errors that do not alter the central conclusions of an article, and that articles containing major errors that invalidate its central conclusions should be retracted. In either event one can still read the article, but the retraction notice gives a strong message to the reader to interpret the contents with caution. It is troubling to find unretracted articles containing major errors, just as it is troubling to find articles retracted due to misconduct in which the retraction notices give the false impression of honest error.

      I do agree with you that the media often contributes to the sensationalism of science, thereby conveying a false sense of what science actually is and how it advances.

      Ferric Fang

      June 16, 2014 at 3:18 pm

      • Interesting. I must admit that I have never come by a retracted article. Thus I was not aware that you can still download and read them.
        Thus the only difference is how strong the warning is? Then I still think that the formulation “troubling” is a bit strong. If the article matters for your research, I would expect the researcher to read to the notice carefully. In that respect, it seems to me more important that the problems are explained honestly and in detail as how the notice is exactly labelled.

        Victor Venema

        June 17, 2014 at 8:58 am

  4. Casadevall et al state: “A number of publications that have not been retracted despite being shown to contain significant errors suggest that barriers to retraction may impede correction of the literature.” and “Our analysis has identified major problems in the mechanisms used to rectify the scientific literature and suggests a need for action by the scientific community to adopt protocols that ensure the integrity of the publication process.”

    See https://www.researchgate.net/post/Would_anyone_like_to_comment_on_my_Open_Review_of_a_flawed_paper_on_moulting_Shelducks_in_the_Wadden_Sea

    Klaas van Dijk

    June 16, 2014 at 1:07 pm

  5. I wonder if there might be a relationship between the fall in errors due to contamination and the increase in analytical errors. The use of black box kits should reduce contamination errors because reagent quality can be batch-tested rigorously and used consistently. On the other hand, black box kits probably have the same weakness as statistical software, resulting in more analytical errors. That is, black box tools can easily be used under the wrong conditions, or without a complete understanding of their limitations.

    After all, why should science be different? Everywhere else, we’ve seen tremendous efficiency gains from using standardized forms and procedures generated by outside specialists. But those gains are partly offset by errors stemming from the fact that the procedures aren’t really understood by the people who use them (ever read through your apartment lease, or the regulations governing your tax return?)

    Toby White

    June 17, 2014 at 10:35 am

  6. Reblogged this on Research Reviews and commented:
    I’d like to hope that one day my work will be comparable to the tireless and commendable work done by those at retractionwatch. I attempt to describe some of the dynamics of the research process, the ways in which the peer-review succeeds or fails, and other “behind the curtain” processes which result in the research published in journals and misrepresented in popular science articles. There is no better source to understand how seriously flawed the processes underlying essential components of virtually all scientific publication than this blog. I’m going to make a point of referring to it now and again just to ensure that however few readers check out my blog no where to go for vital information.

    legiononomamoi

    July 4, 2014 at 11:21 am


We welcome comments. Please read our comments policy at http://retractionwatch.wordpress.com/the-retraction-watch-faq/ and leave your comment below.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 35,047 other followers

%d bloggers like this: