A group of pain management researchers have had three of their papers retracted since September, after another group published a critique of their work earlier this year.
The critique, published in the journal Pain in April, found that ten studies led by physiatrist Marco Monticone of the University of Cagliari in Italy may not be reliable. The studies had several inconsistencies, including data that diverged from almost all similar studies, impossible statistical significance values, and duplicated or very similar data from other studies by the group, though the studies were purportedly separate clinical trials.
The lead author of the critique, Neil O’Connell of Brunel University London, is the coordinating editor for the Cochrane Pain, Palliative and Supportive Care group. He told Retraction Watch that while conducting a Cochrane review on psychological interventions for chronic pain, some of his co-authors noticed that several papers with Monticone as the lead author had results that were different from every other study on the subject.
Though O’Connell’s group reached out to Monticone to try to understand the large effect sizes his studies found, he said they didn’t get satisfactory answers. O’Connell told us:
On that basis we designed a protocol to systematically identify trials in this field from this author and explore aspects of the trustworthiness of the trials. We use a tool developed by the Cochrane Pregnancy and Childbirth group for this. To summarise our findings, we found 10 trials and identified a number of concerns across those trials that included: a lack of trial pre-registration, baseline values that did not seem consistent with randomisation, cases of erroneous (impossible) p values in baseline tables, extremely large effect sizes at both short and long-term follow up, (which is something that is really not common in the field of chronic pain and certainly not seen in other trials of CBT), and notably low or zero attrition in some studies. In addition we identified identical, and highly similar data across 3 trials reported as independent studies. On that basis we recommended that these trials should probably not be used to inform the results and conclusions of systematic reviews, in clinical training, policy documents, or any relevant instruction regarding adult chronic pain management.
O’Connell and his co-authors reached out to the journals which had published the ten studies, but only three have so far been retracted.
Monticone, who is the corresponding and lead author on all three papers, did not respond to an email from Retraction Watch, though we did get an autoreply stating that he was unavailable.
The most recent retraction, from October, was of “Multidisciplinary program based on early management of psychological factors reduces disability of patients with subacute low back pain: one-year results of a randomized controlled study,” originally published in the European Journal of Physical and Rehabilitation Medicine (EJPRM) last year. It has been cited two times, according to Clarivate’s Web of Science.
According to the article’s retraction notice, Monticone, the corresponding author, decided to retract it:
The corresponding author in agreement with all co-authors decided to retract this article because a mistake has been noticed a posteriori in the management of the data. The Editors and Publisher strongly support this decision, according to the results of an independent International Commission nominated because of a recently published paper. (O’Connell NE, Moore RA, Stewart G, Fisher E, Hearn L, Eccleston C. de C Williams AC. Investigating the veracity of a sample of divergent published trial data in spinal pain. Pain 2022 Apr 25 [Epub ahead of print]). Eur J Phys Rehabil Med 2021 December;57(6):959-7. DOI: 10.23736/S1973-9087.21.06696-X.
When we contacted the chief editors of EJPRM, Stefano Negrini and Giorgio Ferriero, for comment, Negrini referred us to an editorial detailing the journal’s process for evaluating two studies they had published that were included in the Pain report.
Because both the journal editors and the authors of the study were Italian and in the field of physical and rehabilitation medicine, the editors decided to appoint an International Review Commission (IRC) with a non-Italian majority. The editorial stated:
After the IRC inquiry for further clarifications, the SGA [specific group of authors] decided to retract one of the two RCTs [randomized controlled trials] (3) they published in EJPRM because they found mistakes in the data collection (4). The IRC supported this decision. Concerning the other paper (2), the IRC concluded that it did not fulfil the COPE Guidelines indications for paper retractions (5) because “there is no clear evidence that the findings are unreliable, or that they are a result of plagiarism, or have been published elsewhere. The only concern is about the unavailability of prior trial registration and formal approval by the Local Ethics Committee. However, this is not enough, at present, to ask for the retraction of the paper.” The IRC added: “Looking at the risks that allegations of untrustworthiness may be raised in the future, the EJPRM Commission advises the Journal Editors to adopt a stricter publication policy, requesting the authors to provide evidence of the Ethics Committee approval as a mandatory step before starting the peer review process.”
We support the SGA decision and agree with the IRC judgment. The fate of the other SGA RCTs published in the other journals is not homogeneous. Two journals retracted their papers (6,7), and one reached the opposite decision on one paper (8) because the peer-review process was adequate, the results were exciting, and “discrepancy is a master stone in science” (personal communication). A fourth journal is still in the decisional phase.
In response to this incident, EJPRM editors decided to require all authors to publish their data in open-access repositories and publish in the text the reference of the ethical committee approval.
However, the editorial also questioned whether evaluating trustworthiness in the way the Pain report authors did is always the correct approach. They wrote:
We must reflect on the cure and prevention of these issues. PJP [the report in Pain] was a “post-hoc” suggestion of trustworthiness issues. We wonder if the approach was the correct one because a trustworthiness scale provides assumptions and not objective facts. Such a scale judges trust and trust is not necessarily the truth. Much progress is still needed, but we have also to recognise that we do not see real alternatives at the moment. In the end, we need a treatment for this disease caused by the “publish or perish” dogma.
The other two retracted papers were both pulled in September.
“Multimodal exercises integrated with cognitive-behavioural therapy improve disability of patients with failed back surgery syndrome: a randomized controlled trial with one-year follow-up” was originally published in December 2020 in the journal Disability and Rehabilitation. It has been cited two times.
Its retraction notice cited vague “concerns” rather than O’Connell’s article:
Since publication, concerns have been raised regarding the integrity of the data in the article. When approached for an explanation, the authors provided detailed responses, but were unable to provide an explanation that satisfactorily addressed our concerns, which includes the veracity of the data presented in the article. The authors do not agree with the decision to retract.
When contacted by Retraction Watch, Mark Robinson, a spokesperson for the publisher of the journal, Taylor and Francis, rehashed the retraction notice:
In terms of the chain of events, as per your query, when concerns were raised to us about the integrity of data in the article, that triggered an investigation which followed the Taylor & Francis ethics and integrity processes, informed by COPE guidelines. The authors were invited to respond and they provided detailed answers to our questions. However, unfortunately these responses did not satisfactorily address our concerns about the veracity of the data.
The editors of Disability and Rehabilitation did not respond to our email.
The other retracted study, “Group-based multimodal exercises integrated with cognitive-behavioural therapy improve disability, pain and quality of life of subjects with chronic neck pain: a randomized controlled trial with one-year follow-up,” was published in the journal Clinical Rehabilitation in 2016 and has been cited 24 times.
The retraction notice credited O’Connell and colleagues’ critique and called the concerns it raised “valid”:
A publication [1] raised concerns about the validity of the findings in several clinical trials published by the same senior author; including the above mentioned article. The concerns include retrospective registration of the trial, lack of clarity whether a trial protocol was developed and whether an appropriately qualified research ethics committee evaluated such a protocol.
The corresponding author has communicated with the authors of [1] but were unable to respond to any queries sent by the journal. One of the authors has provided some data, a brief protocol, along with an ethics approval document. The information provided did not sufficiently resolve the concerns raised. Due to unresolved concerns about the circumstances in which this clinical trial was designed and conducted, and the overall concerns about the integrity of this research, the Editor retracts this article.
The Editor believes that the concerns raised in [1] are valid, and has therefore retracted this article.
The authors MM, EA BR, DC, VL, AP, HV did not agree with the retraction.
When contacted for comment by Retraction Watch, Avril Drummond, the Editor-in-Chief of Clinical Rehabilitation, wrote in an email:
We were alerted about concerns about the veracity of this paper and undertook diligence checks, which included corresponding with the authors. We were satisfied that the concerns raised were valid and we therefore withdrew the paper.
O’Connell said he is pleased the journals have decided to retract these three papers, but he also said he hopes that more of the papers in the Pain report are eventually retracted.
He noted that eight of the studies had results that were highly divergent from similar studies, none were pre-registered, and he and his co-authors’ requests for individual patient data were never met. He told us:
We think this example raises important questions about how we explore the trustworthiness of primary studies when conducting systematic reviews.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].