Retraction Watch

Tracking retractions as a window into the scientific process

Meet PubPeer 2.0: New version of post-publication peer review site launches today

with 10 comments

Since it launched in 2012, PubPeer has grown to become a standard part of the scientific lexicon, and its numerous post-publication discussions have led to more editorial notices than we can count. But it’s also faced its share of critics, including a scientist who took the site to court to unmask commenters he alleged had cost him a job offer. The site won that case on appeal, but is today launching new features that will make it impossible for the site to reveal users’ identities, as well as easier to read and format comments. We spoke with PubPeer co-founder Brandon Stell about what to expect from the new site.

Retraction Watch: What changes have you introduced to the site?

Brandon Stell: There are many things that we have wanted to do to extend PubPeer but unfortunately all development ground to a halt a few years ago when the original developer, George Smith (who was working on PubPeer in his spare time) stopped working on the site after we decided to transition to a nonprofit.  Recently, thanks to a generous grant from The Laura and John Arnold Foundation, we were able to hire a very talented new programmer to work on the site full-time and completely rewrite it to to make it easier to add some of the ideas we have for new features in the future. [Editor’s note: Retraction Watch is also funded by the Arnold Foundation.]

There are many changes behind the scenes in this current rewrite but the two biggest changes for users are the improvement of readability/formatting of comments and the security of user anonymity.  Users will now have options to format their comments to be more academic and they will be able to easily insert and format images and equations.  We feel that strong anonymity is very important for effective post-publication peer review and to address the security of user anonymity on PubPeer we decided to do our best to not retain any identifying information for any users that wish to remain anonymous.

RW: You’ve added some identity management features. What are the most notable changes, and what problems are they designed to address?

BS: There are now two types of accounts: anonymous and signed.  Anonymous accounts with random user names from the tree of life (e.g. drosophila melanogaster) can now be generated when anonymous comments are submitted and users can sign into those accounts again later (using only a key that we provide them) in order to leave subsequent comments, receive feedback about why comments were moderated, check for replies to all their comments left with that account, etc.   (for the PubPeer aficionados out there: it is as if Peer X accounts and Unregs have been merged into one)

RW: Will the identity changes you’ve installed make it more difficult for scientists to unmask (and thereby seek recourse from) anonymous commenters?

BS: Yes, that is one of the main motivations for that change. Once the transition to the new site is complete our goal is to not be able to reveal any user information if we receive another subpoena or if the site is hacked.

RW: You’re adding links to retraction notices (as well as EoCs, errata, etc.) directly in the commenting threads. Often users post those on their own, so what’s the advantage of the site taking that over?

BS: It’s true that some users occasionally insert expressions of concern and retraction notices into PubPeer comments but it is far from systematic and not easily searchable.  Richard Smith and Steph Smith-Unna recently created a great API to easily access publication events and we are using it to more systematically pull information into PubPeer, make it searchable, and insert it into the appropriate timepoints of the PubPeer commenting timelines. Richard is using the data provided by Crossref and PubMed but if you know of anyone with a more complete API we would love to use that as well 🙂 [Editor’s note: We’re working on that for retractions, stay tuned!]

RW: What prompted these changes?

BS: As you know, we feel that most of the problems with scientific reproducibility stem from the fact that scientific evaluation committees around the world rely so heavily on journal names to identify promising science, and scientists.  To help alleviate that problem we want to continue to build a timeline for each publication that includes all comments from the community, references, and other publication events (i.e. retractions, expressions of concern, acceptance in an overlay journal, etc).  We believe that if all that information becomes more centralized and easily searchable, evaluation committees will use it to help them make more informed decisions.  That will hopefully loosen the grip that journals have on deciding the future of science, put evaluation back in the hands of community, and change some of the publishing incentives to ultimately strengthen science.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.

Written by Alison McCook

June 15th, 2017 at 8:00 am

Posted in pubpeer selections

Comments
  • Neuroskeptic June 15, 2017 at 8:44 am

    Great! Congratulations to PubPeer.

  • Dr. J. Smith June 15, 2017 at 9:44 am

    It’s notable that nowhere in his answer to “what prompted these changes” does he mention anything that justifies anonymous commenting. It’s disappointing that the most controversial feature of the site, which comprises the overwhelming majority of public activity on PubPeer, and which they insist on defending to the point of performing amazing mental gymnastics, isn’t at all something that addresses what “most of the problems with scientific reproducibility stem from.” You can do all those things signed, without affecting the de-centralization and openness of the forum.

    It’s also interesting that they freely admit making wholesale backend changes specifically for the purpose of preventing themselves from being held to the same standards of transparency they demand from others. I suspect that their actions might not legally protect them from liability completely, based on the precedents they’ve already set for themselves.

    • Cancer doc UK June 15, 2017 at 10:28 am

      Perhaps when reviewers of papers and grant are not anonymous, pubpeer may change.

    • Neuroskeptic June 15, 2017 at 12:22 pm

      “You can do those things signed”. Perhaps you can. Not everyone can.

      • Kenrod June 15, 2017 at 2:19 pm

        He’s not exactly sticking his neck out there by identifying himself as “Dr. J. Smith”

        • Cancer doc UK June 16, 2017 at 2:39 am

          But why is Dr Smith not willing to identify himself?

    • Paul Brookes June 15, 2017 at 12:57 pm

      Disagree with just about everything in this comment. For the umpteenth time, identity does not matter if content of allegations is sound. Focus on the message not the messenger. In addition, note that anonymity is allowed on the site but not mandated – anyone who wants to use their real ID is free to do so. If you feel comfortable doing that, go for it, but recognize that others may not share the same comfort level.

      Re: being held to a standard of transparency, it’s really about the legal excuse of plausible deniability. The ability to say “I’d love to tell you, but I can’t because I don’t know” is a very powerful legal defense. But, if questioned on this, it requires you to actually not know. All PubPeer is doing, is constructing the site so they can say this in good faith.

      It’s no different than LavaBit’s closure – they had operated on a defense of plausible deniability (“we can’t reveal your data to lawmakers because it’s encrypted and we don’t have the key”). When it became clear thanks to Snowden that the feds had a back-door, LavaBit could no longer confidently say this, so chose to shut down. Compare to the other major tech’ companies who for several months continued to claim your data was safe from the government, even though they knew otherwise.

    • Richard David Feinman June 15, 2017 at 7:24 pm

      I agree with this position but it should apply to reviewers in journals themselves. More generally, the whole journal process is anachronistic and very harmful. In the medical sciences there appears so much speculative, questionable and adversarial material that gets published but frequently remains inadequately reviewed, frequently by de facto parties. There is, for example, strong opposition to recommendations from the American Diabetes Association but they own the journals and appoint editors.

      PubPeer should be incorporated into the review process where there should be levels of publication. Publications should not be considered permanently accepted until they’ve gone through a certain number of months of open review. This idea has been proposed by others in different forms and might be considered important because of the importance of medicine for our health.

    • TL June 16, 2017 at 4:16 am

      Many PubPeer postings are images of western blots with boxes superimposed to identify duplications/splicings. Nobody needs to know the identity of the person posting the images to ascertain the veracity of such claims.

    • Fom June 18, 2017 at 12:14 pm

      1) There are many examples of people who had suffered a lot for criticizing the wrong scientists (even if they ‘won’ in the end). 2) People can (and in some fields most likely) will get back at you in grant and paper reviews. 3) In many cases supervisors or administrators discourage open criticisms and PPPR – as a distraction from direct work responsibilities and as a potential source of troubles. 4) In many institutions it is explicitly forbidden to openly/publicly criticize your colleagues (you should do it though an appropriate office).

  • Post a comment

    Threaded commenting powered by interconnect/it code.