When a researcher encountered two papers that suggested moonlight has biological effects — on both plants and humans — he took a second look at the data, and came to different conclusions. That was the easy part — getting the word out about his negative findings, however, was much more difficult.
When Jean-Luc Margot, a professor in the departments of Earth, Planetary & Space Sciences and Physics & Astronomy at the University of California, Los Angeles, tried to submit his reanalysis to the journals that published the original papers, both rejected it; after multiple attempts, his work ended up in different publications.
Disagreements are common but crucial in science; like they say, friction makes fire. Journals are inherently disinterested in negative findings — but should it take more than a year, in one instance, to publish an alternative interpretation to somewhat speculative findings that, at first glance, seem difficult to believe? Especially when they contain such obvious methodological issues such as presenting only a handful of data points linking biological activity to the full moon, or ignore significant confounders?
Margot did not expect to have such a difficult experience with the journals — including Biology Letters, which published the study suggesting that a plant relied on the full moon to survive:
What surprised me was when the journal, Biology Letters, had published such a weak result that has far-reaching implications in plant evolution, refused to publish the alternate hypothesis.
The Biology Letters paper, published in April 2015, suggested that the non-flowering plant Ephedra foeminea relied on light from the full moon to pollinate. The plant was said to secrete translucent globules of sugary liquid that attracts nocturnal pollinating insects, many of which navigate using the moon. Originally, the researchers thought these globules were present during a certain time of the year, but were astonished to find them appear exactly when the moon is full.
What made the result “weak,” in Margot’s opinion, was that the authors presented only three times when the globules were present within one or two days of a full moon. With so few data points, it’s entirely possible the results were due to chance, he said:
When I read their paper, I was very surprised to find that they only had three data points and they were claiming this association with the full moon on the basis of [these] three data points.
You would expect to find multiple studies in the literature with that type of coincidence with the full moon.
Margot appealed the journal’s decision to reject his letter, but was turned down again. Margot’s rebuttal was later published last October in the Journal of Biological Rhythms (JBR) after being rejected by two other biology journals (in addition to Biology Letters). (One month later, he published a corrigendum that presented a more accurate way of calculating the time to a full moon, but did not alter the argument or conclusions, he said.)
Catarina Rydin, a botanist from Stockholm University in Sweden and co-author of the original study, defended her paper:
The reasons we found it important to report our finding was not the strong statistical power of the results, but a number of small but relevant biological observations that all point in the same direction. And, more importantly, we wanted to report this so that also other scientists can contribute new information on the topic, in Ephedra and other plants.
Commenting on Margot’s criticism, Rydin added:
The paper by Margot is however not very interesting in our opinion, as it does not contribute any news to science; no new data, and no errors in our calculations. That our hypothesis is based on very few data points is clear from our paper, and the risk of Type I errors is known to every scientist. Sometimes, it is important to have the courage to present also quite bold hypotheses in order for science to progress.
We reached out to Rick Battarbee, the editor-in-chief of Biology Letters based at University College London, who told us:
We do indeed publish alternative viewpoints on the same data… However, on this occasion our peer reviewers advised rejection and this advice was upheld by our Editorial Board. Following your email we have scrutinised the reviewer reports and the manuscript again and are of the opinion that the course of action we took was correct.
The comment raised valid points and in particular it highlighted the need for more observations to test the hypothesis further, as in fact the original authors had called for. However, the reviewers of the comment were happy that the hypothesis itself remained valid, as were we, and this was the basis of our decision.
Most recently, the Journal of Systematics and Evolution has published a review paper citing the pollination claim as a fact. Margot has reached out to the paper’s first author, but has not received a response so far.
A few years ago, Margot came across a 2004 study published in the International Journal of Nursing Practice (IJNP), which claimed that admissions of patients in a hospital in Barcelona correlated with the lunar cycle. Here, too, Margot spotted some issues — the authors didn’t consider that lunar cycle times vary, nor factor in other potential variations, such as by days of the week. Taking these additional analyses into account undermined the article’s main conclusions, he said.
But once again, he found it difficult to get his analysis accepted, submitting to five different nursing journals before getting his paper published in Nursing Research last May – roughly a year and a half after he first submitted it to the original journal.
Lin Perry, editor-in-chief of the IJNP from the University of Technology Sydney in Australia, said her journal did not reject Margot’s manuscript, but the paper was “unsubmitted” three times because it did not comply with the journal’s formatting requirements. Perry added:
Times change, standards and expectations change; all readers should consider all papers in the light of current standards and expectations for academic and clinical practice… I welcome discussion and critique of papers published in the International Journal of Nursing Practice, but would prefer to focus on current rather than historical pieces.
But Margot was not convinced that formatting errors were the case in each instance:
On each one of my three submission attempts to IJNP, I received a message from the journal stating ‘We have forwarded your manuscript to reviewers for their comments.’ I do not understand how the manuscript allegedly failed on three occasions to meet the journal’s formatting requirements. It seems rather odd that the ‘unsubmission’ would take place after sending the manuscript to reviewers.
Marion Broome, editor-in-chief of Nursing Outlook, another journal that rejected Margot’s paper, based at Duke University in Durham, North Carolina, told us:
It was rejected because I do not have reviewers or readers who have expertise in the kinds of methods and analyses the authors chose to use in this manuscript.
Margot said that having an astronomer on the review team of the original papers would have helped their reviewing process. With interdisciplinary research on the rise, journals must look to contact experts outside of their immediate fields, he noted.
Journal editors are not alone in hesitating to publish negative results, notes Margot; media outlets also have a similar culture.
Out of concern, Margot contacted the reporters of every article by email, Twitter or both, informing them of the limitations of the study.
But only one website — Mother Nature Network — added a cautionary statement, said Margot, changing “undeniable correlation” to:
research shows that there appears to be a correlation. There is, however, disagreement among scientists that the shrub’s pollination is related to the lunar cycle.
And in case you were wondering — Margot has also debunked the long-standing theory that more women give birth during a full moon.
Update 2/26/16 3:06 p.m. eastern: Margot just informed us that a letter to the editor he submitted to the Journal of Systematics and Evolution about a recent review that cited the Biology Letters paper was rejected.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy.