Instead of retracting a flawed study, a journal let authors re-do it. It got retracted anyway.

When a journal discovers elementary design flaws in a paper, what should it do? Should it retract immediately, or are there times when it makes sense to give the researchers time to perform a “do-over?”

These are questions the editors at Scientific Reports recently faced with a somewhat controversial 2016 paper, which reported that microRNAs from broccoli could make their way into the nuclei of human cells — suggesting that the food we eat could affect our gene expression.

After the paper appeared, researcher Kenneth Witwer at Johns Hopkins — who was not a co-author — posted comments on PubMed Commons and the paper itself, noting that the authors hadn’t properly designed the experiment, making it impossible for them to detect broccoli microRNAs. 

But instead of retracting the paper, the journal decided to give the authors time to do the experiments again, this time with correctly designed molecular biology tools. When that failed, they retracted it — and as part of the notice, reported the exact opposite conclusion of the original.

Witwer said the authors did a “tremendous job” with the follow-up study, but he still thinks the journal should have retracted the paper immediately. Letting the authors redo it is “a dangerous precedent to set,” he told us:   

The idea that a fundamentally flawed study should be granted a “do-over” before retraction is deeply concerning to me, no matter what the outcome.

The journal’s decision to allow the authors to reperform the experiment might be reminiscent of other journals’ “retract and replace” strategy — except journals typically only allow authors to redo a study when the methodology is sound. 

A spokesperson for the journal declined to say why they gave the authors time to re-do the experiments, simply referring us to the retraction notice.

The notice explains that the new study yielded no data to support the original conclusion, so the original paper has been retracted, with the agreement of all authors. “Circulating plant miRNAs can regulate human gene expression in vitro” The paper has been cited twice, according to Clarivate Analytics’ Web of Science.

Kendal Hirschi, a biologist at the Baylor College of Medicine who wasn’t involved with the study, told us:

It is a retraction, but they’ve added value to the field. They did some systematic human feeding studies which have not been done outside of China and shows they don’t work.

In the original paper they were showing some bioavailability [of the broccoli microRNAs], and with this retraction, and the data they present in the retraction, they are showing no bioavailability of the plant microRNAs.

A CONTROVERSIAL THEORY

The idea that food contains RNA that can make its way through the bloodstream and inside cells first sprouted about five years ago, when researchers led by Chen-Yu Zhang, of China’s Nanjing University, published a paper in Cell Research suggesting that microRNA from rice could regulate gene expression in both humans and mice.

Witwer told us he’s tried in his own lab to reproduce the results of these experiments showing so-called “cross-kingdom” RNA interactions, but failed. That the UToronto team didn’t find anything worth publishing their second time around was “not surprising,” he said:

It’s such a fascinating idea, that food is this master engineer of our gene expression and is affecting us in ways we never imagined. But over the years, there’s been a  growing amount of negative evidence.

Last author Igor Jurisica, a computational biologist, told Retraction Watch the design mistake was due to human error. There were two options in the software used to design the PCR primers used to detect the broccoli microRNAs; whoever designed the primers chose the wrong option and the group didn’t catch the mistake.  

Hirschi told us that while he was skeptical of the paper when he first read it, he didn’t catch the primer mistake. Witwer noticed it right away. He emailed the journal to point it out, but to his frustration, the editors didn’t react until he contacted some senior members of the journal’s advisory board.

In the rest of the notice, the authors chronicled their second attempt in great detail. It’s unclear whether these data and conclusions went through peer review; Jurisica told us he wasn’t sure that it had and the editors did not respond to our follow-up question on this. Either way, Hirschi told us this was valuable information:

The people that follow this field want to know, when you do a human feeding study, do you see higher levels of microRNAs. We’ve tried small scale feeding studies and we can’t see the microRNAs. We done a lot in animals and we don’t see it there either.

Regarding the journal’s decision to allow the authors to re-perform the experiment, Hirschi said:

Maybe they should have retracted right away, but I still appreciate that they did the experiments.

I felt like emailing [Jurisica] saying, “That’s the way you should do a retraction.”

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.