NEJM, Lancet place expressions of concern on controversial studies of drugs for COVID-19

[See update on this story.]

As controversy swirls around two papers that used data from Surgisphere, the New England Journal of Medicine and The Lancet have placed expression of concerns on the relevant papers.

Here’s the NEJM expression of concern:

On May 1, 2020, we published “Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19,” a study of the effect of preexisting treatment with angiotensin-converting enzyme (ACE) inhibitors and angiotensin-receptor blockers (ARBs) on Covid-19. This retrospective study used data drawn from an international database that included electronic health records from 169 hospitals on three continents. Recently, substantive concerns have been raised about the quality of the information in that database. We have asked the authors to provide evidence that the data are reliable. In the interim and for the benefit of our readers, we are publishing this Expression of Concern about the reliability of their conclusions.

Studies of ACE inhibitors and ARBs in Covid-19 can play an important role in patient care. We encourage readers to consult two other studies we published on May 1, 2020, that used independent data to reach their conclusions.

The move follows serious questions about a study of the side effects of hydroxychloroquine by many of the same authors in The Lancet. As Stephanie Lee of BuzzFeed News reported last week, 180 scientists sent a letter to the journal

…about some seemingly inconsistent data in the paper. Among the scientists’ 10 concerns are that the average daily doses of hydroxychloroquine were higher than the FDA-recommended amounts and that data reportedly from Australian patients did not seem to match data from the Australian government. This week, the Guardian reported that it could not confirm with several of that country’s health agencies that they provided data to the study.

The Lancet published a correction on Saturday, and an expression of concern today:

Important scientific questions have been raised about data reported in the paper by Mandeep Mehra et al—
Hydroxychloroquine or chloroquine with or without a
macrolide for treatment of COVID-19: a multinational
registry analysis
—published in The Lancet on May 22, 2020. Although an independent audit of the provenance and validity of the data has been commissioned by the authors not affiliated with Surgisphere and is ongoing, with results expected very shortly, we are issuing an Expression of Concern to alert readers to the fact that serious scientific questions have been brought to our attention. We will update this notice as soon as we have further information.

As The Scientist reported last week, the criticism has drawn scrutiny for Surgisphere, whose CEO, Sapan Desai, is a co-author of both studies.

Corresponding author Mandeep Mehra, of the Brigham & Women’s Hospital, did not immediately respond to a request for comment. [See update at end of post.]

The article joins our ever-growing list of studies of COVID-19 that have been retracted or subjected to an expression of concern.

A NEJM correction

Separately, the authors of a controversial paper that reported that the use of remdesivir could potentially help patients gain “clinical improvement” from Covid-19, have corrected their article in NEJM. But you would be forgiven for missing that fact. 

The article, “Compassionate use of remdesivir for patients with severe Covid-19,” appeared in in April and immediately triggered waves of both enthusiasm and skepticism. The researchers reported that:

In this cohort of patients hospitalized for severe Covid-19 who were treated with compassionate-use remdesivir, clinical improvement was observed in 36 of 53 patients (68%). Measurement of efficacy will require ongoing randomized, placebo-controlled trials of remdesivir therapy. (Funded by Gilead Sciences.)

Not surprisingly, the paper received an avalanche of page views, press coverage and attention on social media. It even prompted an open letter from Daniel O’Day, the CEO of Gilead, which makes the drug, sponsored the study and wrote “earlier versions” of the manuscript. 

The paper also triggered the same amount of attention in the other direction, such as here and here. 

Score one for the critics. In mid-May, NEJM posted several comments that pointed to a significant flaw in the calculations the authors of the paper used to determine the purported benefit of the drug, the mortality observed in the study and questionable statistical methodology. 

Those comments prompted a reply from the authors, led by Jonathan Grein, of Cedars–Sinai Medical Center in Los Angeles. Notable about the response, which we’ve included in full below, is that the researchers acknowledge their errors — particularly their overstatement of the “clinical improvement” outcome and provide a revised, somewhat lower figure. 

In our book, that’s a correction. But NEJM hasn’t officially called out the error, simply labeling the new text a “response”: 

The correspondence regarding our study of outcomes in 53 hospitalized patients who received remdesivir on a compassionate-use basis provides an opportunity to clarify several aspects of our article. During a median follow-up of 18 days, 68% of the patients had an improvement in oxygen-support class, 47% were discharged, and 13% died. As recommended by Bonovas and Piovani, and by Fätkenheuer and Lundgren, we have reanalyzed clinical improvement using a competing-risk approach, wherein death is rightly considered to be a treatment failure. On the basis of this analysis, the cumulative incidence of clinical improvement at 28 days after the initiation of remdesivir was 74% (95% CI, 55 to 86). We do not think this reanalysis substantially changes our assessment that remdesivir may provide clinical benefit in patients with severe Covid-19, although as we stated in our article, these findings require validation from ongoing clinical trials. As we reported, clinical improvement was less frequent among patients 70 years of age or older (hazard ratio as compared with patients younger than 50 years, 0.29; 95% CI, 0.11 to 0.74) and among those receiving invasive ventilation (hazard ratio as compared with patients receiving noninvasive oxygen support, 0.33; 95% CI, 0.16 to 0.68).

Fätkenheuer and Lundgren considered our characterization of these patients as having severe Covid-19 to be “somewhat misleading.” We respectfully disagree, since all the patients were hospitalized and received supplemental oxygen or had hypoxia. At the time of initiation of remdesivir, the majority of the patients (64%) were receiving invasive ventilation, including 8% who were receiving extracorporeal membrane oxygenation, and 13% were receiving noninvasive positive pressure ventilation or high-flow oxygen support.

Several of the correspondents highlighted limitations of our study, including its small size, lack of a control group, missing clinical data (e.g., levels of lymphocytes, lactate dehydrogenase, and d-dimer), and incomplete follow-up information. These limitations, which we acknowledged in our article, stem from the nature and urgency of this compassionate-use program. In fact, because of these limitations, we urged caution in the interpretation of the data, pending the results of ongoing randomized, controlled trials. The goal of this compassionate-use program was to enable access to a potentially effective therapy by patients who were severely affected by Covid-19, while researchers, clinicians, and patients await the results of a suite of trials designed to clarify the safety and efficacy of this therapy, the duration of treatment, and which groups of patients are most likely to have a response. The answers to these questions are now emerging.1,2

We asked NEJM why it did not opt to issue a formal correction in this case. They told us:

The authors did not make a correction in their reply letter, but provided an alternative analysis with results consistent with the original analysis.

Update, 1900 UTC, 6/2/20: Mehra forwarded our request for comment to the Brigham & Women’s media relations department, who sent this statement:

Yesterday, the authors proactively initiated an independent audit of the data used in the New England Journal of Medicine (NEJM) paper after discrepancies and concerns were raised about the reliability of the database. The third-party auditors will provide their findings directly to the editors of NEJM upon completion.

Updated 1950 UTC, 6/2/20, with news of The Lancet expression of concern.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].