One journal’s trash is another’s treasure – until a former peer reviewer stumbles across it and sounds an alarm.
In April, communications professor Jacqueline Ewart got a Google Scholar notification about a paper published in the World of Media she had reviewed, and recommended rejecting, for another journal several months earlier.
At the time, she recommended against publishing the article, “Monitoring the development of community radio: A comprehensive bibliometric analysis,” in the Journal of Radio and Audio Media, or JRAM, because she had concerns the article was written by AI. She also noticed several references, including one she supposedly wrote, were fake.
Brown University physician-scientist Wafik El-Deiry has been a longtime critic of the post-publication forum PubPeer, where 75 of his papers have been flagged. For example, in an April post on X, formerly Twitter, he stated, “It is not good that PubPeer has been weaponized and has become tyrannical.” In July 2024, he referred to the emails authors receive when someone posts a comment about their papers as being “under attack by PubPeer and their anonymous mob.”
PubPeer, started in 2012, won the 2024 Einstein Award For Promoting Quality in Research. (Disclosure: Our co-founder Ivan Oransky is a volunteer member of the board of directors of the PubPeer Foundation.)
In April, a site called Science Guardians, an “online journal club” with remarkable similarities – but also key differences – to PubPeer, launched what it called an “investigation” into PubPeer. Since then the anonymous account has postednumerousposts on X critiquing PubPeer while promising to reveal the “perpetrators of the PubPeer Network Mob.”
A physics journal has informed an embattled rocket scientist that it will retract three of his papers, citing concerns raised by the retraction of another of his papers last year.
All three articles appear in Physics of Fluids, published by AIP Publishing, and describe a phenomenon called “Sanal flow choking.” As we reported last year, some scientists have denounced the concept as “absolute nonsense.” The researcher who coined the phrase is the lead author on all papers, V.R. Sanal Kumar, a professor of aerospace engineering at Amity University in New Delhi.
Driving those headlines was a December 2014 study in Science, by Michael J. LaCour, then a Ph.D. student at the University of California, Los Angeles, and Donald Green, a professor at Columbia University.
Researchers praised the “buzzy new study,” as Slate called it at the time, for its robust effects and impressive results. The key finding: A brief conversation with a gay door-to-door canvasser could change the mind of someone opposed to same-sex marriage.
By the time the study was published, David Broockman, then a graduate student at the University of California, Berkeley, had already seen LaCour’s results and was keen to pursue his own version of it. He and fellow graduate student Joshua Kalla had collaborated before and wanted to look more closely at the impact canvassing could have on elections. But as the pair deconstructed LaCour’s study to figure out how to replicate it, they hit several curious stumbling blocks. And when they got a hold of LaCour’s dataset, or replication package, they quickly realized the results weren’t adding up.
PLOS One has retracted a 2011 paper first flagged for image issues 11 years ago. The retraction marks the fourth for the paper’s lead author, Gabriella Marfè of the University of Campania “Luigi Vanvitelli,” in Caserta, Italy.
Elisabeth Bik flagged the article on PubPeer in 2014 for apparent image manipulation and duplication in six figures. In a 2019 email to PLOS staff, pseudonymous sleuth Claire Francis drew attention to Bik’s findings. The journal retracted the paper on May 6 of this year.
The Center for Scientific Integrity, the parent nonprofit of Retraction Watch, has launched a new initiative to investigate and rapidly disseminate problems in the medical literature that directly affect human health.
Thanks to a $900,000 grant from Open Philanthropy, the Medical Evidence Project will leverage the tools of forensic metascience — using visual and computational methods to determine a paper’s trustworthiness — to rapidly identify problems in scientific articles, combined with the experience and platform of Retraction Watch to disseminate those findings.
“We originally set up The Center for Scientific Integrity as a home for Retraction Watch, but we always hoped we would be able to do more in the research accountability space,” said Ivan Oransky, executive director of the Center and cofounder of Retraction Watch. “The Medical Evidence Project allows us to support critical analysis and disseminate the findings.”
For years, sleuths – whose names our readers are likely familiar with – have been diligently flagging issues with the scientific literature. More than a dozen of these specialists have teamed up to create a set of guides to teach others their trade.
The Collection of Open Science Integrity Guides (COSIG) aims to make “post-publication peer review” more accessible, according to the preprint made available online today. The 25 guides so far range from general – “PubPeer commenting best practices” – to field-specific – like spotting issues with X-ray diffraction patterns.
Although 15 sleuths are named as contributors on the project, those we talked to emphasized the project should be largely credited to Reese Richardson, the author of the preprint.
Barbara Smaller/The New Yorker Collection/The Cartoon Bank
President Trump recently issued an executive order calling for improvement in the reproducibility of scientific research and asking federal agencies to propose how they will make that happen. I imagine that the National Institutes of Health’s response will include replication studies, in which NIH would fund attempts to repeat published experiments from the ground up, to see if they generate consistent results.
Both Robert F. Kennedy Jr., the Secretary of Health and Human Services, and NIH director Jay Bhattacharya have already proposed such studies with the objective of determining which NIH-funded research findings are reliable. The goals are presumably to boost public trust in science, improve health-policy decision making, and prevent wasting additional funds on research that relies on unreliable findings.
As a former biomedical researcher, editor, and publisher, and a current consultant about image data integrity, I would argue that conducting systematic replication studies of pre-clinical research is neither an effective nor an efficient strategy to achieve the objective of identifying reliable research. Such studies would be an impractical use of NIH funds, especially in the face of extensive proposed budget cuts.
