As many readers know, even after a paper’s retracted, it will continue to be cited — often by researchers who don’t realize the findings are problematic. But when, and in what context, do those citations occur? In a recent paper in Scientometrics, Judit Bar-Ilan of Bar-Ilan University in Israel and Gali Halevi at Icahn School of Medicine at Mount Sinai in New York examine what happened to nearly 1,000 retracted papers over time, including how long it took to pull them, and when and how often they continue to be cited. We spoke with Bar-Ilan and Halevi about what worries them about their findings — and why they believe Elsevier could help fix the problem.
Retraction Watch: As you note, there have been a number of studies of retractions. What do you hope this study contributes?
Judit Bar-Ilan and Gali Halevi: First and foremost, we believe that the more studies there are on this topic, the more awareness will be built around this problematic topic. Secondly, our paper continues to track the readership and citations of retracted articles over time and proves that despite the problems that they present to the integrity of science and despite all the awareness efforts around this topic, retracted articles continue to be read and cited positively. This should be concerning to everyone involved; both publishers and scientists.
RW: One of your sections is “Who gains from retracted articles?” What did you find?
JB and GH: The authors of retracted articles gain the most out of the continuous readership and citations of their articles. We should all remember that an author’s h-index score does not distinguish between positive and negative citations. Therefore, even if a retracted article is negatively cited (which not necessarily the case as our article shows), the citations counts are still calculated towards a higher h-index. In addition, the references appearing on retracted articles also gain since they are too counted towards total citations.
RW: You note that “One of the possible reasons for the continued citations is that Elsevier’s policy is to have free access to the retracted articles, even though the COPE guidelines require only the retraction notice to be freely available.” And you “recommend that Elsevier change its policy regarding free access to non-open access retracted articles, especially since it seems after conducting a small random check, SpringerNature and Wiley charge for non-open access retracted articles.” Can you explain?
JB and GH: Unlike other publishers, Elsevier makes all the retracted articles on ScienceDirect free in full text open access format. That means that anyone can download and read them. Since the vast majority of Elsevier articles are not open access, it is a bit strange that they choose to have retracted articles completely open access. In 2017 we recommended already that they change this practice and place retracted articles behind a paywall. We have to remember that there are institutions and countries who cannot afford to buy Elsevier content. These institutions rely on open access resources for their academic and scientific work. Therefore, Elsevier has to ensure that retracted articles are not easily accessible. These articles often represent erroneous research in forms of falsified data and findings. Elsevier should make them harder rather than easier to access.
RW: You write that you are “not completely against citing retracted articles.” What would you recommend as best practices for doing so?
JB and GH: Citing retracted articles in not necessarily a bad thing. Such citations can warn others about falsified and erroneous studies. However, citations of retracted articles must clearly state that they are citing a retracted article. This should be labeled in the references list and within the text so that the reader is fully aware of the context and nature of the citation. In order to achieve this, the metadata of retracted articles needs to be adjusted accordingly so it can be captured in reference managers.
RW: In looking at what retracted papers are most often cited, tweeted, and read, you note that the now-infamous 1998 paper in The Lancet by Andrew Wakefield et al about vaccines and autism — retracted in 2010 — is not indexed in Scopus, despite the fact that Scopus and The Lancet are both owned by Elsevier. Were there other such cases that you found?
JB and GH: We haven’t looked into this across all articles. It might be something to focus on in future studies.
RW: You note that we have developed a database that, at the time of your writing, contained 16,000 retractions. (We’re now past 17,000.) Can you say what impact such a database might have?
JB and GH: 1- Scientific purposes: the database could be of very immense importance for studies such as ours. We can potentially load all the articles in your database into our PlumX dashboard and track current citations and interactions with these articles over time. Findings of such a study could shed a light on the current impact these articles still have in the scientific world.
2- Much like predatory journals lists, this list can be used as a tool to check articles that might not show up with a retraction label. As we all know, Google Scholar indexed many versions of the same article. There might be some versions that are author pre-prints rather than official publisher’s papers. If suspected, an article can potentially be looked up in the database and identified as retracted.
3- For journal editors, this is an excellent tool to check the references of submitted papers and request an explanation why a retracted article was referenced. Even if the explanation is satisfactory, in the reference list retracted articles have to be clearly marked as RETRACTED. The authors can also do the same before submission. This would hopefully decrease the positive and neutral post-retraction citations.
Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
(1) I fully agree with Elsevier’s action of placing retracted papers outside the paywall. It would be wrong for publishers to collect further income from papers that have been identified as being faulty.
(2) In order to avoid unknowing cites of retracted papers their DOI’s should be flagged so such cites can be caught in the redaction process. (Probably all cited papers in the reference list of all papers should be required to include DOI’s anyway.)
I fully agree with your point 1. Besides that, the few times I checked a retracted paper on the Elsevier platform (ScienceDirect), that paper is marked “retracted” in big red all-caps on every single page. Making them easier accessible should thus not matter at all.