As Retraction Watch readers probably know, we’re big fans of PubPeer, the post-publication peer review site that allows comments on papers. Discussions there have led to a number of corrections and retractions, and even more importantly, authors are starting to respond to clarify results, acknowledge errors, or otherwise advance knowledge. After all, as we often note, there’s a long way between raising questions about a study and retracting it.
With all of that in mind, we’re pleased to launch a new weekly post that will be called “PubPeer Selections” in which we’ll highlight featured discussions on PubPeer. Here’s the first installment:
- Can zapping the brain with electromagnetic pulses boost memory? Find out in the discussion of “Targeted enhancement of cortical-hippocampal brain networks and associative memory,” published in Science.
- “This is a neat paper, and potentially very important,” says Paul Brookes of a Nature paper, “The metabolite α-ketoglutarate extends lifespan by inhibiting ATP synthase and TOR.” A study co-author responds to Brookes’s reservations about the study’s central conclusions.
- “Electron spin changes during general anesthesia in Drosophila,” by Luca Turin and colleagues in PNAS, comes under scrutiny.
- Readers are deconstructing “Decomposition of spontaneous brain activity into distinct fMRI co-activation patterns,” a 2013 paper in Frontiers in Systems Neuroscience.
- The last author of a Nature paper on hematopoietic stem cells, “Maintenance of hematopoietic stem cells through regulation of Wnt and mTOR pathways,” gives detailed responses to questions from a commenter.
This is a fantastic move!
An aggregation of aggregated comments :). Most useful, including for the use of PupPeer in teaching and generally hectoring people to put their thoughts on their reading onto PubPeer.
great idea, thanks a lot!
May I suggest, to make the link and information more traceable to RW, that the following elements also be included with each “case study”:
1) the full list of authors;
2) the full title;
3) the full journal name, volume, issue and page numbers and DOI.
RW is read by Google very efficiently, probably because it uses Word Press. Articles here are tracked and get a much higher visibility score than the same cases on PubPeer. So, by having these key elements (authors’ names, key words of the title, journal name) would allow those searching for results on a topic to likely bump into a RW-listed PubPeer entry before the actual PubPeer entry itself! It clutters the text a bit more, but seeing that this initiative is at an early stage of development, it might be worthwhile sharing ideas about how to improve and/or optimize it.
PS: As RW notes, “there’s a long way between raising questions about a study and retracting it”. Not all studies that are examined or commented on need to be retracted. Some studies, even with errors, deserve to stay in the literature, but simply need to be followed up with open discussion and broader debate. Whether that follow-up is called an erratum, a PS, or some category not yet devised by publishers is a matter of debate. But, we should be careful about trying to make the end-game of criticisms retractions because it may reveal non-academic objectives.
Thanks for extending RW’s “watch” in this way to cover some of the debate about papers that precedes their retraction and not just the debate that happens after. It will be interesting to see how PubPeer comments evolve, and if they lead to faster or more transparent responses from authors and editors.
I actually think that Pubpeer out to be providing their own blog or editorial frontpage themselves (or a more active role, they do have a small blog most on site specific issues).
Or rather not “ought to” but it would be a very useful addition. Different voices and different perspectives from the publishing/science journalist angle of Retraction Watch can only be a good thing.
I have noticed great strides and improvements by PubPeer since its inception, which I have followed carefully over the months. There is no doubt that the RW-PubPeer alliance can only fortify accountability in science and aid as a powerful tool for whistle-blowers, anonymous, or not. So, hats off to all parties.
I took some time yesterday to analyze the PubPeer journals page [1], and noticed that only a handful of journals listed were related to pure plant science (Plant Ecology, Plant Physiology and New Phytologist), and a few other minor ones associated with applied plant sciences. This indicates that the PubPeer idea is only barely nascent in the plant sciences and I am making an extraordinary effort to bring RW and PubPeer to the attention of the global pool of plant scientists. Interestingly enough, some have expressed irritation simply because they prefer to stay in the dark about such issues and about public accountability, so I suspect that this war in (plant) science publishing is going to start heating up rather quickly in the next few months and we could get an exponential increase in comments on PubPeer about problems in the plant science literature.
I analyzed the comments that already exist, and found some to be really disappointing, including one sentence remarks, or compliments about a paper [2], which only serve to dilute the effect and impact fo PubPeer. I understand that it is a free world with free speech (well, almost), but should PubPeer not be focusing exclusively on the “negative” or “problematic” aspects of papers? That would increase the site’s impact and focus, which currently has too many “irrelevant” or unsubstantial comments diluting the impact of the “relevant” ones. I did find one interesting analysis by someone who had analyzed some issues with gels but found the evidence, with my untrained eye, difficult to believe or understand [3]. I also found an interesting cross-referenced critique of a paper in New Phytologist [4] and another that argues the contradictory nature of authors’ data [5]. Cases [3], [4] and [5] are good starting examples and precedents for others to follow. I need to experiment a bit with PubPeer first before trying to get the plant science community aboard. But once that ball starts rolling, there is no turning back… sort of like ten-pin bowling.
[1] https://pubpeer.com/journals
[2] https://pubpeer.com/publications/EDA562D88BBF6909EBE2246122A681 (see Peer 2’s comments)
[3] https://pubpeer.com/publications/895140D10F9809A546A041D7A4D238
[4] https://pubpeer.com/publications/82AFAE152ED95DED710F6E39DE8457
[5] https://pubpeer.com/publications/A01632233DC4B68C65A8B770E3908E