An article estimating how many people might have died during the first wave of the COVID-19 pandemic due to the off-label use of hydroxychloroquine in hospitals was retracted in August after advocates for the drug launched a campaign criticizing the study.
French media have reported criticism of the retraction as inappropriate, and speculation the journal caved to pressure from hydroxychloroquine advocates.
In a statement to Retraction Watch, the journal stood by its decision to retract the article due to “some clear fatal flaws” identified in letters to the editor, which it said it declined to publish due to their tone it deemed “not suitable for publication in a scientific journal.”
The editor in chief’s decision “was based solely on scientific considerations, and the process was in line with Committee on Publication Ethics best practices,” a spokesperson for the publisher said.
Correspondence between the journal and authors of the paper obtained by Retraction Watch, including the critical letters to the editor and authors’ responses, shed some light on the specific arguments the editor considered.
The article, “Deaths induced by compassionate use of hydroxychloroquine during the first COVID-19 wave: An estimate,” appeared online January 2 in the journal Biomedicine & Pharmacotherapy, an Elsevier title. It has been cited nine times, according to Clarivate’s Web of Science.
The authors, led by Jean-Christophe Lega of Centre Hospitalier Lyon Sud in France, estimated 17,000 deaths related to hydroxychloroquine may have occurred in Belgium, Turkey, France, Italy, Spain, and the US, with a range from about 6,300 to 19,300.
“Although our estimates are limited by their imprecision, these findings illustrate the hazard of drug repurposing with low-level evidence,” the authors concluded.
As has been the case with hydroxychloroquine, the findings made international headlines. Then, the criticism began, in online articles, videos and blog posts, letters to the journal editor, and emails to Lega and the president of the Université Claude Bernard Lyon 1, where he is a professor. The university got so many emails, many of which accused Lega of fraud and questioned his integrity, the institution formally reported them to the public prosecutor.
On January 15, an Elsevier employee named Adam Fraser emailed Lega on behalf of the journal editor, Danyelle Townsend of the University of South Carolina in Charleston. The email acknowledged receiving a request from Lega to correct the article, and mentioned a “readers’ debate” about the paper. The letter continued:
The Journal has received a number of Letters to the Editor which are being assessed and, if they are accepted, you will be given a right to reply, prior to publication, in line with Committee on Publication Ethics guidelines and journal policies.
The journal on January 28 published an editor’s note, which has since been updated, about the plan.
On May 1, Fraser, whose title is executive publisher for pharmacology, sent another email on behalf of Townsend with eight letters to the editor for Lega and his colleagues to consider for a response. But by that time, Townsend had decided to handle the matter differently.
“After assessing all the information, and the extent and depth of the criticism, I have made the decision to change approach and handle this matter through the Committee on Publication Ethics (COPE) process for corrections to the scientific record,” she wrote. The substance of the complaints in the letters, “if true, would violate our publishing policies.”
An update to the editor’s note on the article echoed Townsend’s email.
We asked Townsend for comment on the change of tack and received the following statement from an Elsevier spokesperson:
The Journal assessed the letters to the editor received and had them independently peer reviewed by an expert reviewer, who submitted their opinion to the journal.
The initial plan, as published in an earlier version of the Editor’s Note, was to publish these Letters to the editor, after the Authors of the above article had been given the right to reply and submit a response (which would also be published, after review). After assessing all letters to the editor (which came from a wide range of geographies and persons) and the review of the letters to the editor, the editor decided to change approach. The criticisms were too extensive to be addressed by a corrigendum, and some clear fatal flaws/ key errors were identified by some of the letters and the review. The tone of some of the letters was not suitable for publication in a scientific journal.
On July 4, Lega sent the journal responses to each of the letters on behalf of all the authors. In a cover letter, Lega wrote:
The only letter that holds our attention is that of Viera et al. (BIOPHA-D-24-00992), which presents well-constructed and factual arguments (use of risk ratio, adjusted mortality rate) for a better estimation of the excess mortality related to the prescription of HCQ. The calculations lead to estimates of the same order of magnitude as those produced by our study.
However, this does not qualitatively challenge our results, as written by the authors of the letter. The authors mistakenly used a treatment effect combining hydroxychloroquine and chloroquine from a network meta-analysis for their estimates, which is not the purpose of our study.
One group of authors engaged a lawyer to pressure Biomedicine & Pharmacotherapy to publish their letter, according to a copy posted to the website of BonSens.org, a French organization that has since taken partial credit for the retraction. The authors published their letter July 30 in the Archives of Microbiology & Immunology, a title of Fortune Journals.
Fraser emailed Lega and his coauthors August 9 with Townsend’s decision to retract the article, and asked them to respond within five days. The email included a proposed retraction notice with the reasons for the decision:
Following publication, the Journal was made aware of readers’ debate about this article. The Journal received a number of Letters to the Editor and correspondence from readers. The plan, as published in an early version of an Editor’s Note, was initially to publish these Letters to the Editor, after the Authors of the above article had been given the right to reply and submit a response (which would also be published, after review).
After assessing all the information at this stage, and the extent and depth of the criticism, the Editor-in-Chief made the decision to change approach and handle this matter through the Committee on Publication Ethics (COPE) process for corrections to the scientific record: https://publicationethics.org/
Upon completion of this process, and following careful consideration of all available information, this article has been retracted at the request of the Editor-in-Chief.
The decision to retract was made due to two major issues:
Reliability of the data and choice of the data. The Belgian dataset in particular was found to be unreliable, based on estimates.
The assumption that all patients that entered the clinic were being treated the same pharmacologically was incorrect.
The above two issues meant that the Editor-in-Chief found the conclusions of the article to be unreliable, and needed to be retracted.
Lega responded on behalf of the authors on August 19, disputing the retraction and asserting the value of their work.
“We transparently emphasized throughout the manuscript that these were approximations with large confidence intervals given the significant uncertainties surrounding several parameters,” he wrote. The authors were aware of the limited data for Belgium, but “addressed and fully acknowledged” the limitation in the article. “We also specifically varied the parameters to generate uncertainty by taking the minimum and maximum values of exposure to hydroxychloroquine.”
He continued:
We understand that science can bring controversies, especially in the context of COVID-19, which has generated polarized debate. However, as provided in our point-by-point response, we truly believe that the validity of the arguments raised by the few readers who questioned the validity of the results were largely unfounded.
Lega proposed publishing a correction or “response from the authors” to address the issues in the retraction notice. He also sent an email to Sarah Jenkins, senior director for the Research Integrity & Publishing Ethics Centre of Expertise at Elsevier, asking for “your input on the ongoing retraction process.”
Jenkins’ reply, dated August 22, restated Townsend’s decision. “Within the Research Integrity & Publishing Ethics Centre of Expertise, we uphold the independence of our Editors-in-Chief in these matters,” she wrote.
The retraction notice, dated August 26, largely matched the proposed language.
After seeing the notice, Lega wrote to Jenkins:
I am saddened that our responses are not commented on more by the editorial board. The comments we received were from non-experts in the field of pharmacology and modelling, with activity on social networks aggressively questioning the scientific basis and the scientists. I would therefore have liked to understand better how they were assessed by the journal.
Furthermore, in order to prevent the resumption of the campaign of harassment, I would appreciate it if the journal would indicate that the accusations of fraud and falsification have not been sustained.
On August 29, after some back-and-forth, Fraser emailed the authors on behalf of Townsend, stating:
[She] has considered your response to the retraction notice, but does not feel this changes her decision to retract the article, and thus does not require any changes to the existing text.
She does not also believe it would be advantageous to publish letters to the editor and responses as the key information for the community is contained within the retraction notice, particularly once the drafted update is published. As noted this update will make it clear that you disagree with the retraction and dispute the grounds for it, and make it clear that there was no evidence of fraud.
The update, dated September 2024, stated:
The Journal wishes to clarify that at no point did the investigation find evidence of fraud in the article, and instead believe it was honest error on the part of the authors.
The authors were responsive to the Journal’s correspondence and engaged with the process throughout. The authors do not agree with the retraction and dispute the grounds for it.
Lega and his colleagues are working to revise the article and resubmit it to another journal.
“I deeply regret, as do the co-authors, the lack of transparency for the scientific community in the process that led to the retraction, despite the respect I have for the Editor,” Lega told Retraction Watch.
As he wrote in an email to Townsend regarding the update to the retraction notice:
We deeply regret the lack of communication from the Journal about the reasons and the context for the retraction. Scientists and the media are once again under pressure from French conspiracy groups, who support the use of hydroxychloroquine in COVID-19. In view of the public health issues at stake, we would have welcomed more appropriate communication from the Journal.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
For your information, we have in France a list of >3,000 scientific journals in the field of biomedicine that are considered as “recommendable”. This list has been established and is continuously revised by the College of Deans of French Medicine Faculties and the Conference of the Chairmen of the Medicine sections of the National Council of (French) Universities. Publishing in journals that are not present in this list is not recommended to biology and medicine researchers or investigators, with the risk that these publications would not be taken into account in their CV for recruitments, promotions or grant attributions. I had recommended 18 months ago to withdraw Biomedicine and Pharmacotherapy from this list of recommendable journals, and this has eventually be done recently.
Access to this list: https://sante.sorbonne-universite.fr/recherche/liste-des-revues-presumees-non-predatrices
Thank you for this very factual and well-documented post. I would like to clarify that the site mentioned BonSens.org is the expression of a group of French conspiracy theorists, with C Perronne, a close friend of D Raoult. The latter does not appear in this post, but he was very well informed about the various attacks… The journal used by these conspiracists is a predatory journal widely used by French researchers.
Editor’s note: Raoult says the paper will be retracted at the end of this video with a sympathetic interviewer.
If you cut out someone´s tongue, you don’t prove him wrong.
You are showing that you are afraid of what he has to say.
Dear Ellie Kincaid,
I’m astonished that you could write such an article, which is neither neutral nor objective, since it adopts the point of view of Léga, whose responses are absolutely no match for the editor’s arguments.
Indeed, the editor’s perfectly justified criticisms relate to aberrations in the method itself, which have nothing to do with mere ‘limitations’:
– on the one hand (argument 2 from the editor), the authors completely failed to take pharmacological realities into account: the dose of hydroxychloroquine used in the Recovery and Solidarity studies was scandalously TOO high, given the frailty of hospitalized patients.
Patients treated in other settings, notably as outpatients at the Marseilles IHU, in the French data, were often at the start of the infection and therefore in better condition, and in all cases received a lower dose;
– on the other hand (argument 1), the authors have done such a poor job of taking into account actual data from other countries – for example, Belgium, which was the subject of a very detailed article in Le Soir (which you did not quote) – that they have ended up inventing non-existent deaths!
Why didn’t you interview Professor Jean-Michel Dogne, for example, to balance your article?
The Belgian news website RTBF did:
“Among the scientists who had reacted to this publication, Professor and pharmacy expert Jean-Michel Dogné (UNamur) had informed the newspaper of the methodological problems including the analyses on the Belgian figures: “The principle of the method, which is open to criticism, is to apply a potential excess mortality associated with hydroxychloroquine obtained as part of a meta-analysis (an analysis based on the combination of existing studies, editor’s note) that did not take into account the way in which hydroxychloroquine was actually used, namely in particular use at doses that may have been lower than those that justified the excess mortality. And in doing so, they apply this excess mortality to all countries, whatever the way in which hydroxychloroquine may have been used, particularly in Belgium. And from there, they determine a number of deaths based on an excess mortality figure that does not correspond to the way in which the product is used”, explains the professor of pharmacy.
(Source : https://www.rtbf.be/article/l-article-sur-le-risque-de-deces-associe-a-l-hydroxychloroquine-au-debut-du-covid-19-a-ete-retracte-voici-pourquoi-11423912)
If the content of this article is not fraudulent, it certainly demonstrates flagrant incompetence. Moreover, the theoretical basis of this article is not derived from a scientific thesis, but from the first author’s simple “practice thesis”, which one wonders how it could have been validated, given the pharmacological inconsistency mentioned above.
Worse still, the authors’ incompetence is immediately apparent when you read the authors’ conclusion of the study to which they refer (and which, as a reminder, is the source of their basic data: 11% excess mortality calculated essentially on the basis of Recovery+Solidarity, applied uniformly to all the data collected, like a steamroller – or rather a “burier”).
In fact – and YOU should have quoted it if you had done your job properly – Axfors et al. write: “Findings have unclear generalizability to outpatients, children, pregnant women, and people with comorbidities.”
You don’t publish a study based solely on generalising results whose authors themselves say ARE NOT generalisable.
This is the basis of good science, and it should have been enough to spare you your absurd attempt to salvage this pitiful paper.
I’m sorry, but if you’re going to make a claim like “HCQ caused 17,000 deaths”, which is clearly designed to be picked up by news outlets and has netted this article a healthy number of citations in its short lifespan, simply acknowledging that your data is limited by pretty questionable choices (e.g. assumption of dose independence) isn’t enough to justify publishing it in spite of these limitations.