PubPeer launched a new look on Nov. 9, including a dashboard showing the post-publication peer review platform has surpassed a half million comments in its 14 years of operation.
Also among the newly public statistics: 20,784 papers had comments on PubPeer before they were retracted, with a median of 10 months between the first comment and retraction. (According to the Retraction Watch Database, 59,843 papers have been retracted since PubPeer’s October 2012 launch.) More than 2,500 papers had comments a median of 11 months before an expression of concern, and 12,000 papers had discussion about them before a correction.
But what researchers think about the platform — and of scrutiny of their work — is harder to quantify. A group of researchers attempted to answer that question by contacting nearly 100,000 authors — over 10,000 of whom had comments about their papers on PubPeer and nearly 85,000 who cited a paper with comments. Less than 1% responded, the group reported in a preprint published earlier this year.
“Researchers do not seem very eager to talk about PubPeer comments they received or on a paper they cited,” said lead author Wytske Hepkema, a Ph.D. student at Radboud University in the Netherlands.
The authors are part of NanoBubbles, a group focused on post-publication peer review who are attempting to understand when and how science corrects itself. Among the authors is Raphaël Lévy, a professor of physics at the Université Paris Sorbonne Nord, who received legal threats for flagging conclusions made in an article by Kavli prize winner Chad Mirkin as misleading.
When deciding which authors to send their survey to, Hepkema and her colleagues focused on PubPeer comments that, if true, would affect the results or content of a given article, like image issues or tortured phrases, which can indicate plagiarism. They excluded retracted studies.
The majority of the respondents said they believed the comments didn’t affect the results of their own study. That was true among the 80 respondents with a paper flagged on PubPeer and the 720 respondents who had cited a study with PubPeer discussion.
Four out of five researchers with comments on their papers said they knew of the comments, and this group was more willing to concede that the comment pointed out an error in the paper. Over half of respondents in this group said they requested the journal publish an erratum.
Scientists who cited papers with PubPeer comments had a different outlook: Over half said the criticisms were not relevant to their work. Hepkema told us her team was the most surprised by this finding, specifically “the argument that even when they said the study they cited was fraudulent, that it had no consequences for their own article.” (Note: Retraction Watch cofounder Ivan Oransky is a volunteer member of the PubPeer Foundation’s board of directors.)
Boris Barbour and Brandon Stell, cofounders of the platform, told us the study’s findings “largely align with our qualitative experience from running the site.”
Of the 80% of authors who were aware of comments on their papers, only 10%-15% chose to reply directly on the platform. The difference is “possibly because they just hope the issue will be quietly forgotten,” Barbour said in an email signed by both himself and Stell. “Almost all replies assure readers that the conclusions of the article are not affected by whatever problem has been detected, but many readers remain unconvinced.”
On whether PubPeer’s new redesign will help quantify the impact the platform has, Barbour said it “does highlight a bit more expressly some running statistics,” such as the numbers of comments and pre-editorial-action discussions, so “it hopefully does make the impact a bit more visible.”
As for the low response rate to the survey, Hepkema told us the team wasn’t surprised. She said conclusions can still be drawn from the work, including a “first view” at researcher perspectives. However, she didn’t think the 80 responses were enough to make claims about how many of the researchers shared the same opinions about post-publication review.
José Luis Ortega, a researcher with the Spanish National Research Council who has looked at PubPeer trends, told us he felt the study may have suffered from response bias. He said it’s likely authors of commented studies who responded to the survey were involved in less serious cases.
As for the authors who cited a paper with PubPeer comments, he said, they “could perceive that citing erroneous studies is something more acceptable or that it does not require any action.”
Among the respondents, some had strong negative reactions to PubPeer, Hepkema told us, including comparing it to the U.S. Immigration and Customs Enforcement (ICE) or Truth Social. One respondent to their survey wrote: “These are all equally used for invalid spread of lies, defamation, harassment and racist purposes used in a whole other contest.”
“PubPeer is a vigilante policing of scientist [sic] by failed scientist [sic],” another respondent wrote. “It allows for all kinds of crazy people to develop conspiratory theories and discredit hard working scientists.” Similar sentiments have been shared before, with some scientists criticizing the anonymity of commenters. One researcher sued PubPeer (and lost) in an attempt to reveal the names of anonymous commenters on his papers.
Other studies have looked into the effect of the platform on retractions or corrections to the scientific literature. That work coupled with the latest study show PubPeer’s effect on the scientific record, the researchers wrote, is “operating through retractions rather than through behavioural change in researchers.”
Barbour and Stell acknowledged that “only a modest proportion of scientists include PubPeer information in their workflows.”
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].