Category: not reproducible

Has reproducibility improved? Introducing the Transparency and Rigor Index

Anita Bandrowski

Some Retraction Watch readers may recall that back in 2012, we called, in The Scientist, for the creation of a Transparency Index. Over the years, we’ve had occasional interest from others in that concept, and some good critiques, but we noted at the time that we did not have the bandwidth to create it ourselves. We hoped that it would plant seeds for others, whether directly or indirectly. 

With that in mind, we present a guest post from Anita Bandrowski, who among other things leads an initiative designed to help researchers identify their reagents correctly and has written for Retraction Watch before. She and colleagues have just posted a preprint titled “Rigor and Transparency Index, a new metric of quality for assessing biological and medical science methods” in which they describe “an automated tool developed to review the methods sections of manuscripts for the presence of criteria associated with the NIH and other reporting guidelines.” 

Science seems to publish many things that may be true or interesting — but perhaps not both. Ideally all of science should be both true and interesting, and if we were to choose one, my hope would be to choose true over interesting. 

Continue reading Has reproducibility improved? Introducing the Transparency and Rigor Index

Researchers pull Nature paper over first author’s objections

Researchers have retracted a 2015 Nature paper about the molecular underpinnings of immune function after discovering they could not replicate key parts of the results.

The first author, Wendy Huang — who started working as an assistant professor at the University of California, San Diego, only months after the paper appeared — did not sign the retraction letter, published last week. The research was conducted while Huang was working as a postdoctoral fellow at New York University, home of last author Dan Littman (also an investigator at the Howard Hughes Medical Institute).

What happened appears to be a case of “he said, she said:” Littman asked to retract the paper after his lab couldn’t reproduce it, and Huang insists the data remain correct, saying the process had been “unfair and done without due process:”

Continue reading Researchers pull Nature paper over first author’s objections

Now-retracted chem paper’s problems “should have been noticed by the referees,” group says

Last year, chemist Marcus Tius at the University of Hawaii saw a paper describing the synthesis of some organic compounds, and was “struck by the implausibility” of the reported structures. So he joined up with some colleagues to try to replicate the data.

While Tius and his team were trying to repeat the experiment, however, in December 2017 the journal — Organic Letters — retracted the paper. The journal, published by the American Chemical Society, noted that the authors had not been able to produce crystal structures that confirm they had synthesized those compounds in particular. So Tius and his colleagues knew they couldn’t replicate the findings — but carried on their experiment anyway:

Continue reading Now-retracted chem paper’s problems “should have been noticed by the referees,” group says

Data in biofuel paper “had either been grossly misinterpreted or fabricated”

A biology journal has retracted a 2011 paper after the University of California, Los Angeles determined that the data in three figures “cannot be supported.”

In February, the journal Applied and Environmental Microbiology retracted the paper, which explores efforts to engineer bacteria to convert plant biomass into biofuel.

Claudia Modlin, assistant director of UCLA’s Office of Research Policy and Compliance, told Retraction Watch that the university informed the journal about the issues last October, after reviewing the work. Continue reading Data in biofuel paper “had either been grossly misinterpreted or fabricated”

“Major advance” in solar power retracted for reproducibility issues

The authors of a highly cited 2016 research letter on a way to improve the efficiency of solar panels have retracted their work following “concerns about the reproducibility.”

Given the potential importance of the data, it would be nice to know what exactly went wrong, and why. However, the retraction notice doesn’t provide many details, and doesn’t even specify if the authors did indeed fail to reproduce the data.

The letter, titled “Graded bandgap perovskite solar cells,” was published in Nature Materials by a group out of the University of California at Berkeley and the affiliated Lawrence Berkeley National Laboratory. The 2016 article has been cited 16 times, according to Clarivate Analytics’ Web of Science, earning it the ranking of “highly cited.”

Berkeley heralded the findings in a press release as a “major advance” in the field of solar energy:

Continue reading “Major advance” in solar power retracted for reproducibility issues

Study that said hate cuts 12 years off gay lives fails to replicate

A highly cited paper has received a major correction as a result of the ongoing battle over attitudes towards gay people, when a prominent — and polarizing — critic showed it could not be replicated.  

In December 2017, researchers led by Mark Hatzenbuehler of Columbia University corrected the paper, originally published in Social Science & Medicine in February 2014, which showed that gay people who live in areas where people were highly prejudiced against them had a significantly shorter life expectancy. The corrigendum came more than a year after a researcher who has testified against same-sex marriage was unable to replicate the original study.

“Structural stigma and all-cause mortality in sexual minority populations,” has been cited 102 times, according to Clarivate Analytics’ Web of Science, and attracted media coverage when it was published, from outlets such as Reuters and U.S. News & World Report.

Continue reading Study that said hate cuts 12 years off gay lives fails to replicate

Lesson one: How to design a good experiment

Len Freedman. Credit: Judy Licht/GBSI

Vivian Siegel. Credit: Maria Nemchuk

It’s clear science has a reproducibility problem. What’s less clear is how to address it. Recently, the U.S. National Institutes of Health awarded the Global Biological Standards Institute (GBSI) $2.34 million to train students in good experimental design (also covered by The Scientist). The first week begins June 2018 at Harvard Medical School. We spoke with Leonard P. Freedman, Founding President and Chief Scientific Officer of GBSI and Vivian Siegel, the education director of the program.

Retraction Watch: Why do students need extra training in experimental design? Continue reading Lesson one: How to design a good experiment

Caught Our Notice: Don’t count your chicken (genes) before they’re hatched

Via Wikimedia

Title: Molecular Characterization and Biological Activity of Interferon-α in Indian Peafowl (Pavo cristatus)

What Caught Our Attention: Soon after the paper appeared, the journal was alerted to the fact its findings were at odds with others in the field. When the editor approached the authors, everything fell apart: The authors couldn’t repeat the experiments, and “were also unsure of the molecular probes that were used in the study.” While it isn’t unusual to have doubts about data — since since research is a process of experimentation — it is odd not to know how your experiment was conducted. The paper was retracted less than two months after it was published. The manuscript was accepted two months after it was submitted in early May, theoretically giving reviewers enough time to catch these issues (along with the authors’ failure to cite relevant papers).  

Continue reading Caught Our Notice: Don’t count your chicken (genes) before they’re hatched

Science retracts paper after Nobel laureate’s lab can’t replicate results

Science is retracting a 2014 paper from the lab of a Nobel winner after replication attempts failed to conclusively support the original results.

In January, Bruce Beutler, an immunologist at University of Texas Southwestern Medical Center and winner of the 2011 Nobel Prize in Physiology or Medicine, emailed Science editor-in-chief Jeremy Berg to report that attempts to replicate the findings in “MAVS, cGAS, and endogenous retroviruses in T-independent B cell responses” had weakened his confidence in original results. The paper had found that virus-like elements in the human genome play an important role in the immune system’s response to pathogens.

Although Beutler and several co-authors requested retraction right off the bat, the journal discovered that two co-authors disagreed, which Berg told us drew out the retraction process. In an attempt to resolve the situation, the journal waited for Beutler’s lab to perform another replication attempt. Those findings were inconclusive and the dissenting authors continued to push back against retraction.

Berg told us:

Continue reading Science retracts paper after Nobel laureate’s lab can’t replicate results

Author blamed for misconduct in Cell paper declines to sign retraction notice  

Researchers have retracted a 2015 paper in Cell after an investigation revealed the first author committed misconduct.

According to the retraction notice, which first author Ozgur Tataroglu declined to sign, the researchers realized there was an issue with the 2015 paper when they were unable to replicate the findings. Corresponding author Patrick Emery and his team at the University of Massachusetts Medical School in Worcester reviewed the data and found “clear evidence” that Tataroglu  — who had been a postdoc in Emery’s lab — “had repeatedly misrepresented and altered primary data,” the notice states.

UMass subsequently conducted an investigation in which it “concluded that the first author committed scientific misconduct.”  

Here’s the retraction notice for “Calcium and SOL Protease Mediate Temperature Resetting of Circadian Clocks:” Continue reading Author blamed for misconduct in Cell paper declines to sign retraction notice