The Center for Scientific Integrity, the parent nonprofit of Retraction Watch, has launched a new initiative to investigate and rapidly disseminate problems in the medical literature that directly affect human health.
Thanks to a $900,000 grant from Open Philanthropy, the Medical Evidence Project will leverage the tools of forensic metascience — using visual and computational methods to determine a paper’s trustworthiness — to rapidly identify problems in scientific articles, combined with the experience and platform of Retraction Watch to disseminate those findings.
“We originally set up The Center for Scientific Integrity as a home for Retraction Watch, but we always hoped we would be able to do more in the research accountability space,” said Ivan Oransky, executive director of the Center and cofounder of Retraction Watch. “The Medical Evidence Project allows us to support critical analysis and disseminate the findings.”
Several high-profile cases illustrate the need to scrutinize the medical literature. For example, European guidelines for giving beta blockers to patients before major surgery were based on results from a set of clinical trials, the DECREASE trials, later discredited due to misconduct. Dozens of systematic reviews and clinical guidelines on treatments for osteoporosis included clinical trials by Japanese bone-health researcher Yoshihiro Sato, who, it turned out, fabricated clinical trial results.
Beyond these high-profile cases are papers with what might appear to be small errors or mistakes that have an outsized impact on clinical results. The Medical Evidence Project aims to take a systematic look at these articles.
Scientific sleuth James Heathers will serve as the project’s director. “When we publish papers or change policies, we often get estimates of how many people would be positively affected by changes in treatment guidelines,” Heathers said. “I would like to produce that same estimate for the re-evaluation of the existing evidence. I strongly suspect that we’re about to do something seriously cost-effective, but I’ll only sleep well when I’ve proved it.”
Grantmaking organization Open Philanthropy has generously funded the Medical Evidence Project for $450,000 per year for two years. Staff for the project will also include part-time analysts and an editor to do investigations and write and publish results.
Investigations by universities, organizations and publishers often take months or years to complete, and findings are only communicated at the end of the process. The Medical Evidence Project hopes to accelerate that, swiftly disseminating the results of its investigations.
The project will take a broad look at all disciplines of medicine, Heathers said.
“There have been some unpleasant and relevant cases in anesthesiology, obstetrics and gynecology, surgery, trauma, and different elements of alternative and complementary medicine over the last few years,” he said. “I’d like to see if there are any more. But I’m quite agnostic. There isn’t much ‘patient-neutral’ medicine. It all matters.”
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
My sense is that the scientific community, even those who work in the area of research integrity, vastly underestimate the extent to which the scientific record is now tainted with bad data (fraudulent and otherwise). Hopefully, the success of the Medical Evidence Project will trigger additional sources of financial and other forms of much needed support for those dedicated professionals who voluntarily engage in these important sleuthing activities.
This is a huge project. I wonder what specific and transparent plans there are for the project in terms of time and scope?
“… forensic analysis of articles that affect human health”.
Scientific literature is full of fakery papers with fabricated data, fake/gifted authorship, and all other forms of misconduct that apparently do not affect human health, so, nobody cares about them. Nobody wants to spend time to look at plots, figures, images, numerical data, etc., and find issues.
The people behind those fakery papers have citation cartel to collects hundreds of false citations for their worthless trash they published for academic promotions and monetary gains. Many institutions and universities are aware of their fraudster staff but because they benefit from false credits (ranking, prestige, …), they protect and support cheating staff.