We’ve known for a while that too many researchers cite retracted papers. But in what context do those citations occur? Are some authors citing a retracted paper as an example of problematic findings, or do most citing authors treat the findings as legitimate, failing to realize they are no longer valid? In a new paper in Scientometrics, Gali Halevi at Icahn School of Medicine at Mount Sinai in New York and Judit Bar-Ilan at Bar-Ilan University in Israel examined citations to 15 papers retracted in 2014. Halevi told us why she was surprised to see how many authors don’t realize retracted papers are problematic, and what the publishing community can do to get the word out.
Retraction Watch: We’ve noticed that many papers are cited long after being retracted, without notifying readers the paper is problematic. You looked at citations to retracted papers and tracked how the citing authors described the paper – noting that its findings were problematic given the retraction (negative), or treating the findings as legitimate research that affirms the newer paper’s results (positive). The vast majority of post-retraction citations – 83% — were positive. Did that surprise you?
Gali Halevi: Understanding the context of the citations was one of our main goals. We expected that although retracted articles were still cited these would be negative mentions. It did surprise us to discover that the vast majority of them treated retracted articles as legitimate citations despite of their faults. What’s worrying is that many of the retracted articles were due to faulty data, plagiarism and unethical behavior. Citing these articles as valid presents a danger to the progress and validity of science.
RW: You note that many articles become freely available once they’re retracted. Although the Committee on Publication Ethics has asked publishers to make retraction notices freely available, and clearly mark papers as retracted, you suggest that publishers remove retracted papers from their database entirely. Why?
GH: The problem with making retracted articles freely available is that they have a larger audience of readers. We also need to remember that there are many countries, especially developing countries, which have very limited resources and rely mostly on freely available content. Having retracted articles freely available means that thousands of scientists without access to properly peer reviewed content might use these as valid forms of content. In addition, research has shown that open access content receives more reads than paid-for content. Having retracted articles in open access form means that they would probably receive more reads and attention than they should. We believe it’s better to remove them from circulation all together, even retracted papers behind a paywall, especially considering the fact that they are still cited.
RW: You suggest that publishers check reference lists to see if any papers were retracted. Many papers list many references – do you think some publishers and editors may argue they don’t have time to do this?
GH: This can be accomplished via simple system guidelines and doesn’t require much time. It does require some technical development of retracted articles lists and comparing them to references lists. We believe that this can be accomplished via an automatic process. It will require some financial investment rather than time. Publishers use software to test for plagiarism. Checking for retractions is much simpler. There are only a few thousand retracted articles. So it is not difficult to create a database that lists all these articles, and to develop a tool that automatically checks whether there are retracted articles in the reference list.
RW: Some of your other recommendations include creating a retractions database (we agree, which is why we’re creating one) and urging all providers of scientific papers (PubMed, journal websites, etc) work together to universally mark a paper as retracted. What are some of the obstacles to achieving this integration between different sites, and how can they be overcome?
GH: We think that the biggest hurdle is aligning the interests of commercial publishers. Each company has their own priorities and financial investments. It might be difficult to find common interest and investment in such a venture. Maybe a joint effort between universities and publishers would be more successful. Universities wouldn’t probably want retracted articles from their faculty to show up in scientific literature. A joint committee of academia and commercial publishers might be a good way to begin such a conversation.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
If publishers went back to the original paper being retracted and updated the metadata and all versions (HTML and PDF) so that the title now starts with “Retracted:” AND if the publisher adequately linked the retraction notice with the paper AND if this new metadata were pushed out to indexing services AND if the indexers committed to actually processing metadata updates, it would be clear. We have been doing the first three steps and it’s pretty darn clear. The real problem now is the proliferation of versions. The authors put their paper in Research Gate or an institutional repository and NEVER make an update if the paper is retracted. I can’t tell you how many times I have asked librarians/repository managers about this. They have not had any interest in ensuring that retractions and corrections are noted on their platforms. It’s an even bigger problem with corporate sites like RG and Academia.edu.
I do not agree that the paper should disappear. The retraction notice would indicate why the paper was being retracted and users could still find some value in the data or methodologies presented. The same would be true for citations. If several experiments were conducted and synthesized into a paper and one was flawed, it may invalidate the conclusions for the entire paper but does not mean that the other other data was flawed. Some papers are retracted after authorship disputes that are unable to be reconciled. Does that mean that the entire paper is wrong? Certainly not.
There are several reasons for the continued citation of retracted papers.
1. Non-existent/lazy scholarship
People don’t read, they cite papers from the bibliography of a paper they have read. Therefore, they have no idea it has been retracted. This is compounded by the large number of ‘closed access’ journals, though with many routes now available to obtain an article, this is less of an excuse.
Another effect of weak scholarship is that citations in general are often wide of the mark. In other words, it often looks like authors are just citing the same paper as other authors, rather than finding one that actually has the evidence appropriate for the argument they are making.
2. A lack of critical thinking. If a paper supports my idea, I’ll cite it, regardless of whether the paper has been cited. This is a problem in all countries, because science education tends to involve little critical thinking. A problem compounded by politicians involved in driving the education and skills agenda, who often have no scientific background. They usually view science as a technical exercise, rather than one that requires a high level of critical thinking.
One solution is for “RETRACTED” to appear on searches in Pubmed and other bibliographic databases. A problem here is that many do not use these specialised resources and rely instead on Google. So ‘retracted’ would have to be placed into a field on a paper that allows it to come up in any search. This, as is noted in the interview, is not simple.
Another solution is for journals to scan submissions for citations to retracted articles. Such submissions would be screened by an editor and positive citations to retracted articles would mean return of the article to the authors. This may be easier to implement to some extent, in that some journals may be keen to do this to differentiate themselves from the pack.
While mistakes are made, the scale of the positive citation of retracted articles suggests that we need to address issues of critical thinking from the outset in science education.
I am not aware if someone has carried out an study on the reasons why papers get cited but my feeling is that, in some cases (an undefinited plural), papers get cited because are found in another paper, without reading them. A typical example happens when someone building an introduction to an article may use a review to cite original papers that s(he) has not checked. If the review has been published before the retraction, then it will pass undetected to the sloppy author.
Another question is if ‘citations’ are always done by relevance of a publication to a certain study. Passive engrossing of references may play a part.
Most of us use EndNote or Reference Manager (AFAIK). Dave Frenig’s suggestion for bibliographic databases to highlight retractions in search results is the best option, as it will flag even the laziest among us to realize that something is off with a particular publication. This has been suggested several weeks ago to Thompson-Reuters, owner of both above bibliography software, with links to the Retraction Watch as evidence of the problem. However, there seems to be no takers at Thompson-Reuters except to suggest workarounds.
http://community.thomsonreuters.com/t5/EndNote-Product-Suggestions/Flag-retracted-articles-in-search-window/td-p/133716