‘It’s time to devise a more efficient solution’: Science editor in chief wants to change the retraction process

Holden Thorp

On the heels of a high-profile retraction that followed deep investigations by the Science news team, Holden Thorp, the editor in chief of the journal, says it’s time to improve the process of correcting the scientific record.

In an editorial published today, Thorp, a former university provost, describes the often time-consuming and frustrating process involving journals, universities, and government agencies that are often at odds, or at least have different priorities. Based on the experience of what can feel like gridlock, he calls for breaking the process into two stages:

The first stage should evaluate the validity of the paper without attributing blame. The university would then feel free to determine the validity of the paper before it plunges into a lengthy and more complicated investigation of the underlying wrongdoing.

If the paper is not valid, it can then be retracted much more quickly. The second stage, with journals out of the picture, would be for the university to determine whether there was fraud that rises to the level of research misconduct. This plan would accelerate correction of the scientific record.

At first glance, the idea makes sense. It would seem to be a way to streamline the process, and at least let readers know a paper shouldn’t be relied upon.

But we had some questions about how this would all work, how many problems it would really solve, and whether it would actually lead to less informative retraction notices. Thorp – who said he was “hoping to start a conversation about how to improve things” – was glad to answer.

RW: Who would do that “first stage” of review? Journals? Many say they don’t have the resources (and your editorial says journals “are not an investigatory body”), and that reviewing any allegations at all is the responsibility of institutions. 

HT: In the first stage, the hope is that the universities would be willing to do more (and more quickly) if the personnel issues were separated from the determination of the validity of the paper.  As I say in the piece, the alternative is for us to move more aggressively on our own, but the universities have a lot to lose if that’s where this goes.  According to COPE, we have the ability to move on our own because it’s our job to determine if the universities and authors are giving us a ‘satisfactory’ answer.  Most of the time, we can tell when they’re not, but ‘satisfactory’ is still a pretty subjective term.  The hope is to get them to see that we all have a lot to gain in terms of the credibility of science if they would engage more proactively.  It’s going to require the provosts and research officers to have some courage, because the general counsels at the universities can come up with a million reasons to stall.

RW: Provided journals have the resources, splitting the retraction process into two stages theoretically makes sense. But we have already seen countless cases in which journals have done nothing for years even after receiving definitive findings from universities. How would this new process fix that?

HT: The new process wouldn’t fix the problem of the journal not doing anything even when they have definitive statements from the university.  That’s on the journal.  We took responsibility for the delay in the Science Signaling case in your tweet.

RW: Would journals update retraction notices that simply say a paper is invalid but don’t assign any blame or say what happened? Or would journals allow universities, most of which don’t release investigation reports, to keep that information forever hidden? 

If we get information from the university that the paper is invalid but nothing about the reasons why, we still go ahead with a retraction that says the university has cast doubt on the paper.  One challenge of this, as you point out, is that the origins of the problem may never come out because once the journal posts the retraction, we won’t be hounding the university any more.  But the universities can easily hide all of that now, so at least we could get the record fixed.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at team@retractionwatch.com.

7 thoughts on “‘It’s time to devise a more efficient solution’: Science editor in chief wants to change the retraction process”

  1. “The hope is to get them [universities] to see that we all have a lot to gain in terms of the credibility of science if they would engage more proactively.”

    For a start the universities/scientific institutions could check their Pubpeer dashboards. Only they can do that.

    https://pubpeer.com/institutions

    “Discover PubPeer Institution Dashboards
    PubPeer Institution Dashboards are built to give Institutions specialized tools to keep track of and address comments on their articles. We provide a centralized dashboard with specialized search features and email alerts for everyone on your team.”

    Journals could also be proactive and request their own Pubpeer dashboards.

    https://pubpeer.com/journals

    “Discover PubPeer Journal Dashboards
    PubPeer Journal Dashboards are built to give Journals specialized tools to keep track of and address comments on their articles. We provide a centralized dashboard with specialized search features and email alerts for everyone on your team.”

    1. One might wonder what conflicts of interest might need to be disclosed in this endorsement. Is there a kickback? If so, how do I get a cut? 🙂

  2. First off, I do appreciate a prominent journal seriously considering a change to the system, there are absolutely flaws that can be rectified, kudos.

    I think part of the problem is actually the COPE guidelines themselves. Too many journals hide behind them and they are too permissive. For example the idea that if you don’t get a satisfactory answer from authors you are just supposed to wait and contact them again every 3-6 months is ridiculous. Bring in an expert, if they data/paper/figure/whatever looks fishy, tell them they have 3 months to provide original raw data, and post an expression of concern. If they miss that deadline, just retract the paper saying that the journal doesn’t have confidence in the work any longer.

    That might be an overly aggressive stance, but if there is ongoing work to extract a satisfactory answer from authors or institutes, at least post an expression of concern until you clear things up.

    PS. I believe there may be a typo in the Science article; it links to COPE guidelines, but the actual link is to a PDF specifically for COPE guidelines for _image_ manipulation, and doesn’t include other problems (such as fabricated data), which are in a separate location.

    1. I agree with you in all respects. I think perhaps the journals might use expressions of concern more frequently to inform all readers that a credible concern has been raised and is under investigation. Although individual readers can get similar information if a concern has been posted to PubPeer and the reader has the browser extension installed, that will not help everyone. An EoC also may prod authors and institutions to address the matter more expeditiously. (For example, it might be more difficult to use a paper with a published EoC to obtain new grants).

    2. I think that the COPE guidelines in general have become overly bureaucratic procedure, used mostly for journals and institutions to hide behind and avoid doing anything.

  3. I think this is an extremely bad idea unless the issue raised in the last question gets addressed (and I don’t think a way of addressing it exists; note that the “answer” essentially says nothing meaningful, it’s just corporate pr-speak). Allowing universities to decide a paper is to be retracted without telling anybody why is flagrantly opposite to the idea of transparency, and will be widely abused to protect fraudsters who bring in grant money.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.