Retraction Watch readers may recall the work of Elizabeth Suelzer, a librarian at the Medical College of Wisconsin in Milwaukee. Two years ago, she and colleagues published a study on why the infamous — and fraudulent — 1998 paper by Andrew Wakefield alleging a link between vaccine and autism had been cited more than 1,000 times. As Suelzer notes in the Q&A below, that work led to more questions about how well bibliographic databases and journal publishers display retraction status, when appropriate. The answer, they report in JAMA Network Open this week: They were inconsistent.
Retraction Watch (RW): What prompted you to do this study?
Elizabeth Suelzer (ES): I published a study in 2019 that reviewed citations to the infamously retracted article by Andrew Wakefield that purported to show a link between MMR vaccines and autism. My coauthors and I discovered that not all the citing references indicated that the Wakefield article was retracted. We also noticed that sometimes the citation to the Wakefield article showed a prefix of “Retracted:” before the article title while others did not. Upon investigation, we discovered that Web of Science displays the article title as “RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children (Retracted article. See vol 375, pg 445, 2010)” while PubMed displays it as “Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children.” The way the title appears in the databases affects the way a citation will display if the citation information is taken directly from a database, hence the differences in citations of the Wakefield article. We were curious to see how the same retracted articles appeared in various databases.
While we were designing the study, we also thought to investigate if journals are consistent in labeling retracted articles on their websites. My coauthors and I are all librarians who access journal articles daily and we see the challenges researchers face in navigating different journal platforms. For instance, sometimes an article’s DOI appears at the top of a webpage, and other times it is towards the bottom of a webpage or somewhere else on the page. In our study, we wanted to see if there was consistency within and among journals on how they display retraction information.
RW: What did you find?
ES: When we looked at the results from the investigation, we were surprised at the variability in how well retractions were labeled in citation databases. We used PubMed as the standard for how retractions should ideally appear in a database. PubMed is freely accessible and widely used, and it does a good job of showing the retracted status of articles by using a noticeable retraction banner, changing the publication type to retracted, and linking to the notice of retraction. It did not go unnoticed that the only free database in our set had the best overall performance.
The review of journal websites was also surprising. In our analysis, we compared three articles from 50 journals, and only 39 journals consistently displayed retractions on their websites, and 31 consistently displayed retraction information on the PDFs that were posted on their sites. Differences in how the retraction information was displayed include retraction labels in different colors, retraction information showing up in different locations, and some of the articles did not even include retraction labels. Researchers may be used to looking in one place on a webpage for retraction or other important notifications, but these notices could show up in any number of places on the webpages.
RW: Can you explain why the best result was from a publicly accessible website, and the subscription websites were less accurate?
ES: I can’t fully explain why PubMed had better results in documenting retractions than the subscription databases. PubMed took steps in 2016 to make retraction information much more visible on their platform. I don’t know if PubMed’s actions in 2016 were prompted by input from their users or because they were aware of the problem and wanted to provide solutions. Whatever the case, the results from this article show that some users will be better informed about article retractions than others, depending on the resources they have access to.
I recommend that institutions who purchase access to subscription databases reach out to their vendors to let them know that we expect better output from their platforms, or output that is on-par with what PubMed is able to provide.
RW: Why does this matter?
ES: Inadvertent or unacknowledged citations to retracted literature has been a problem for a long time, and it can have negative effects on public health, among other things. Teixeira da Silva, Dobránszki and Bornemann-Cimenti give a good summation of the problem in their blog post “Citing retracted papers has a negative domino effect on science, education, and society:” “The continued citation of retracted papers can lead to the endless proliferation of erroneous literature, it can mislead young academics and confuse established researchers as to the veracity of a scientific claim.”
It should not be a challenge to discover if a paper has been retracted, however we discovered that researchers may need to hunt around on the platforms to find retraction information, and that is assuming they know to look for it. Database vendors and journal publishers hinder scientific progress when they inconsistently and inaccurately display information on their platforms.
While it is ultimately the responsibility of an author to ensure that they are not unwittingly citing retracted literature, journal publishers have a responsibility to ensure that if they retract an article, its retracted status is accurately and prominently labeled on their website, in all forms (HTML and PDF). Database vendors also need to ensure that they are providing accurate records for articles in their databases by incorporating publisher updates in a timely manner.
RW: As you know, Zotero, which you recommended in a tweet, and Redaktek, which you also mentioned, integrate our database to ensure that users are alerted about retractions of any of the papers in their libraries. What if the sites and databases where you identified problems did the same?
ES: We have shown that journal publishers and database vendors are not always displaying retraction information correctly, consistently, or prominently on their sites, and anything that can be done to change this should be considered. Retraction Watch has been successfully implemented in Zotero and has alerted me to retracted publications in my citation library that would have otherwise gone unnoticed. There is great value for researchers in getting these alerts as it can save them the awkwardness or embarrassment of inadvertently citing a retracted publication. The more opportunities to alert researchers about the retracted status of literature, the better. Implementing Retraction Watch into websites will let readers know in the moment if the article they are looking at has been retracted.
Like Retraction Watch? You can make a one-time tax-deductible contribution or a monthly tax-deductible donation to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].