Stuck in limbo: What happens to papers flagged by journals as potentially problematic?

Expressions of concern, as regular Retraction Watch readers will know, are rare but important signals in the scientific record. Neither retractions nor corrections, they alert readers that there may be an issue with a paper, but that the full story is not yet clear. But what ultimately happens to papers flagged by these editorial notices? How often are they eventually retracted or corrected, and how often do expressions of concern linger indefinitely? Hilda Bastian and two colleagues from the U.S. National Library of Medicine, which runs PubMed, recently set out to try to answer those questions. We talked to her about the project by email.

Retraction Watch (RW): The National Library of Medicine recently decided to index expressions of concern, which it hadn’t before. Why the change?

Hilda Bastian (HB): In PubMed, expressions of concern have been indexed when publishers have submitted them, but they are treated as comments – along with editorials and letters to the editor. That means they haven’t really functioned in PubMed as they are intended to – an alert to readers about the integrity of a publication. PMC (PubMed Central) introduced an article type for expressions of concern in 2013, but those could only be accommodated as comments in PubMed too. We want to do two things: speed up the time it takes for an expression of concern to be linked to affected publications, and make them more prominent. Introducing the tag as a data element is the first step.

RW: How many such Expressions of Concern have you typically indexed in a year?

HB: They aren’t common. We found 320 publications affected by expressions of concern up to the end of 2016, and we are continuously screening all records entering PubMed and PMC for them now. The earliest we found was from 1985, but they were very rare until 2005. There were just under 60 expressions of concern in 2016. More than half of all the expressions of concern we found were issued in the last three years. Some weren’t submitted to PubMed. We searched Google Scholar and several publisher websites, and found some expressions of concern for publications that aren’t in PubMed or PMC, too – only 40. It seems to be an editorial practice in biomedicine, but not more widely in the scholarly literature.

WR: In results posted today at bioRxiv, you surveyed expressions of concern and their associated papers, to see what happened to them over time. What did you find? 

HB: By early December last year, 25% of the publications had been retracted. However, many of the notices were issued recently. For those that had been retracted, 92% were retracted within two years of the expression of concern – the rest were retracted within 3 years.

We looked in detail at the expressions of concern between August 2014 and August 2016. Most expressions of concern were about validity of an aspect of the study or publication, or possible research misconduct, and 15% related to authorship or data disputes, overlapping text, or duplicate publication. No reason was given for 5%. Issues with images were involved in 30%.

The case was unresolved for 31% notices, and there were retracted publications for 29%. Some cases were closed without ever being resolved. The expression of concern was the end of the line for 28%.

Some expressions of concern were removed from the publisher’s site without explanation, or over-written with a retraction notice. One expression of concern was itself retracted. Some had a follow-up notice, reporting on progress with investigation. And 6% of publications had errata published.

RW: Are there any other changes in the works?

HB: We are working on getting the expressions of concern tagged, and making them more accessible and prominent. It’s part of a larger project looking at all post-publication events in PubMed.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.