Retraction Watch

Tracking retractions as a window into the scientific process

Archive for the ‘not reproducible’ Category

What leads to bias in the scientific literature? New study tries to answer

with 4 comments

By now, most of our readers are aware that some fields of science have a reproducibility problem. Part of the problem, some argue, is the publishing community’s bias toward dramatic findings — namely, studies that show something has an effect on something else are more likely to be published than studies that don’t.

Many have argued that scientists publish such data because that’s what is rewarded — by journals and, indirectly, by funders and employers, who judge a scientist based on his or her publication record. But a new meta-analysis in PNAS is saying it’s a bit more complicated than that.

In a paper released today, researchers led by Daniele Fanelli and John Ioannidis — both at Stanford University — suggest that the so-called “pressure-to-publish” does not appear to bias studies toward larger so-called “effect sizes.” Instead, the researchers argue that other factors were a bigger source of bias than the pressure-to-publish, namely the use of small sample sizes (which could contain a skewed sample that shows stronger effects), and relegating studies with smaller effects to the “gray literature,” such as conference proceedings, PhD theses, and other less publicized formats.

However, Ferric Fang of the University of Washington — who did not participate in the study — approached the findings with some caution:

Read the rest of this entry »

Written by Alison McCook

March 20th, 2017 at 3:00 pm

Physics paper’s results off by factor of 100

without comments

Researchers from China have retracted a physics paper after realizing an error led them to report results that were nearly 100 times too large.

What’s more, the authors omitted key findings that would enable others to reproduce their experiments.

According to the notice, the authors used a value to calculate a feature of electrons—called mobility—that “was approximately 100 times too small,” which led to results that were “100 times too large.” The notice also details several gaps in the presentation of experimental results, which preclude others from duplicating the experiments.

Here’s the retraction notice for “Bulk- and layer-heterojunction phototransistors based on poly[2-methoxy-5-(2′-ethylhexyloxy-p-phenylenevinylene)] and PbS quantum dot hybrids:” Read the rest of this entry »

A new reproducibility fix? Get your work checked before it’s published

with 5 comments

Jeffrey Mogil

Malcolm Macleod

Most researchers by now recognize there’s a reproducibility crisis facing science. But what to do about it? Today in Nature, Jeffrey S. Mogil at McGill University and Malcolm R. Macleod at the University of Edinburgh propose a new approach: Restructure the reporting of preclinical research to include an extra “confirmatory study” performed by an independent lab, which verifies the findings before they are published. We spoke with them about how this could work.

Retraction Watch: You’re proposing to restructure animal studies of new therapies or ways to prevent disease. Can you explain what this new type of study should look like, and how researchers will execute it?

Read the rest of this entry »

Written by Alison McCook

February 22nd, 2017 at 9:41 am

Researchers disagree over how to explain doubts over physics findings

without comments

After an international group of physicists agreed that the findings of their 2015 paper were in doubt, they simply couldn’t agree on how to explain what went wrong. Apparently tired of waiting, the journal retracted the paper anyway.

The resulting notice doesn’t say much, for obvious reasons. Apparently, some additional information came to light which caused the researchers to question the results and model. Although the five authors thought a retraction was the right call, they could not agree on the language in the notice.

Here’s the retraction notice for “Atomistic simulation of damage accumulation and amorphization in Ge,” published online February 2015 in the Journal of Applied Physics (JAP) and retracted two years later in January 2017: Read the rest of this entry »

Authors pull virus replication paper after they cannot replicate results

without comments

Researchers in China have retracted a 2016 paper exploring the replication behaviors of a retrovirus, after discovering that the key results could not be reproduced — possibly because their cell cultures had been contaminated.

The authors also cite a disagreement with a colleague, who they say contributed to the work but does not want to be listed as an author.

Here’s the retraction notice for “Nuclear import of prototype foamy virus transactivator Bel1 is mediated by KPNA1, KPNA6 and KPNA7,” published in the International Journal of Molecular Medicine: Read the rest of this entry »

Why traditional statistics are often “counterproductive to research the human sciences”

with 10 comments

Andrew Gelman

Doing research is hard. Getting statistically significant results is hard. Making sure the results you obtain reflect reality is even harder. In this week’s Science, Eric Loken at the University of Connecticut and Andrew Gelman at Columbia University debunk some common myths about the use of statistics in research — and argue that, in many cases, the use of traditional statistics does more harm than good in human sciences research. 

Retraction Watch: Your article focuses on the “noise” that’s present in research studies. What is “noise” and how is it created during an experiment?

Read the rest of this entry »

Written by Alison McCook

February 9th, 2017 at 2:00 pm

At last, cancer reproducibility project releases some results — and they’re mixed

without comments

Nearly five years ago, researchers suggested that the vast majority of preclinical cancer research wouldn’t hold up to follow-up experiments, delaying much needed treatments for patients. In a series of articles publishing tomorrow morning, eLife has released the results of the first five attempts to replicate experiments in cancer biology — and the results are decidedly mixed.

As our co-founders Adam Marcus and Ivan Oransky write in STAT, the overall take-home message was that two studies generated findings similar to the original, one did not replicate the original, and two others were inconclusive.

They quote Brian Nosek, a psychologist at the University of Virginia, in Charlottesville, who runs the Center for Open Science, who has been leading the replication effort in his own field:

Read the rest of this entry »

Written by Alison McCook

January 18th, 2017 at 1:53 pm

Lack of reproducibility triggers retractions of Nature Materials articles

with one comment

The authors of a highly cited 2015 paper in Nature Materials have retracted it, after being unable to reproduce some of the key findings.

The move prompted the journal to also retract an associated News & Views article.

Here’s the retraction notice for “Fast and long-range triplet exciton diffusion in metal–organic frameworks for photon upconversion at ultralow excitation power:” Read the rest of this entry »

Error-laden database kills paper on extinction patterns

without comments

An ecologist in Australia realized a database he was using to spot trends in extinction patterns was problematic, affecting two papers. One journal issued an expression of concern, which has since turned into a retraction. So far, the other journal has left the paper untouched.

The now-retracted paper concluded that medium-sized species on islands tend to go extinct more often than large or small mammalian species. But a little over a year ago, Biology Letters flagged the paper with an expression of concern (EOC), noting “concerns regarding the validity of some of the data and methods used in the analysis.”

Now, last author Marcel Cardillo at Australian National University has come to a new conclusion about extinction patterns. A retraction notice that has replaced the EOC explains:

Read the rest of this entry »

Yes, “power pose” study is flawed, but shouldn’t be retracted, says one author

with 7 comments

psychological-scienceAfter the first author of a debated study about the benefits of positioning your body in an assertive ways — the so-called “power pose” — posted her concerns about the research, she has told us she does not believe the paper should be retracted.

As reported by New York magazine, late last night, the first author of a 2010 paper in Psychological Science posted a statement saying she no longer believes the effects of the “power pose” are real.

We contacted Dana Carney, now based at the University of California, Berkeley, to ask if she thought the next step would be to retract the paper. She told us: Read the rest of this entry »

Written by Alison McCook

September 26th, 2016 at 1:36 pm