Often, when confronted with allegations of errors in papers they have published, journal editors encourage researchers to submit letters to the editor. Based on what we hear from such letter writers, however, the journals don’t make publication an easy process. Here’s one such story from a group at Indiana University: Luis M. Mestre, Stephanie L. Dickinson, Lilian Golzarri-Arroyo, and David B. Allison.
In late 2018, in the course of reviewing papers on obesity, one of us (DA) noticed a November 2018 article in the BMC journal Biomedical Engineering Online titled “Randomized controlled trial testing weight loss and abdominal obesity outcomes of moxibustion.” The objective of the study was to determine the effect of moxibustion therapy on weight loss, waist circumference, and waist-to-hip ratio in young Asian females living in Taiwan.
Some of the tabulated data in the paper seemed odd, so DA sent it to members of our research team asking for their input. The research team agreed, finding some irregularities in the data that seemed inconsistent with a randomized experimental design. After that, the task of carefully and thoroughly checking the published summary statistics and text in the paper was delegated to another of us (LM) and all of his work rechecked by professional statisticians and the research team.
The apparent inconsistencies and anomalies identified in the paper (i.e., large baseline differences, variance heterogeneity, and lack of details in the explanation of the study design) led to concerns about the extent to which the study report represented an accurate description of a properly conducted randomized controlled trial (RCT) and, therefore, whether the conclusions were reliable. Given the importance of reliable conclusions in the scientific literature on obesity treatment, as well as simply the integrity of the scientific literature overall, we decided to write a letter to the editor of the journal seeking either clarification or correction.
Continue reading Tortuous and torturous: Why publishing a critical letter to the editor is so difficult