Our computer ate the data: Expression of concern over blood thinner study raises concerns itself

Thrombosis and Haemostasis has issued an “expression of concern” over a 2004 paper by Tunisian researchers:

Concerns have been raised by readers about the accuracy and validity of the data reported in the September 2004 article by Abdelkefi et al., entitled “Prevention of central venous line-related thrombosis by continuous infusion of low-dose unfractionated heparin, in patients with haemato-oncological disease. A randomized controlled trial” (Abdelkefi A et al.Thromb Haemost 2004; 92: 654–661).

In the trial, 108 patients with blood cancers reportedly received infusions of either saline or heparin, a blood thinner. Those given the active drug were far less likely to develop clots related to their catheters, according to the researchers, and no more likely to experience severe bleeding. In the report, the researchers write:

This is the first prospective, randomized study, which shows that low-dose of unfractionated heparin is safeand effective to prevent catheter-related thrombosis in patients with haemato-oncological disease.

The article has had an impact, having been cited 32 times, according to Thomson Scientific’s Web of Knowledge. At some point after publication, however, the results evidently began to look fishy. Again from the journal:

Concerns have been raised over the practical aspects of the published protocol.

When asked by Thrombosis and Haemostasis to provide evidence that the study was conducted, the authors were unable to provide this information. Because Thrombosis and Haemostasis has no means of establishing the data or demonstrating that research results presented in this article are reproducible, Thrombosis and Haemostasis is publishing this Expression of Concern.

Reached by e-mail, Abdelkefi, of the Centre National de Greffe de Moelle Osseuse, told us that his group indeed conducted the study as described but that a computer malfunction in 2008 wiped out their data. Abdelfeki provided Retraction Watch a copy of an August 16 letter to Thrombosis and Haemostasis editor Gregory YH Lip in support of the paper:

Dear Dr. Lip

We are writing on behalf of  Centre National de Greffe de Moelle Osseuse.

Concerning Thrombosis and Haemostasis study:

1) We can confirm that the study protocol was approved by the local medical ethical committee, and written informed consent was systematically obtained from the patients or their legal representatives.

2) We can confirm that data was appropriately collected and analyzed.

3) Before submission, the accuracy of the manuscript has been checked by the medical committee.

4) List of included patients and medical records data (dossiers des patients, pancartes) are available and accurate. However, specific data concerning central venous lines characteristics (paper files) have not been preserved after publication. On October 2008, data files have been lost.

You will find enclosed the final protocol with the informed consent documentation.

In conclusion, this study was scientifically rigorous and there are no concerns about her integrity.

Sincerely,

Abdeladhim Ben Abdeladhim, MD

Former Medical Director (1997-2005)

Centre National de Greffe de Moelle Osseuse

Tarek Ben Othman, MD

Current Medical Director (2005-…)

Centre National de Greffe de Moelle Osseuse

But any fourth-grader with access to Microsoft Word could whip up a letter like that. It’s not even on letterhead (Abdelkefi said he and his colleagues also mailed Lip a copy on hospital letterhead, but why he couldn’t send us that version isn’t clear).

So neither we nor the journal found that particularly reassuring. And whatever materials the Tunisian researchers forwarded either never arrived or were insufficient to their cause — “not as much definitive proof as needed,” Lip told us.

Unfortunately, Lip, a cardiologist at the University of Birmingham, in England, declined to offer more details on the nature of the “practical” red flags that first drew scrutiny to the trial.

The feedback we received was that the clinical protocol seems impractical to perform, and it seemed unlikely that the study could not have been conducted the way as was claimed.

Lip added that he was not the editor when the paper was published, which limited his knowledge of the case.

We find this whole affair as troubling as it is bizarre. The “motherboard ate my homework” defense obviously deserves a raised eyebrow, but it has been known to happen. Presuming innocence, a flat-out retraction is unwarranted.

But assuming for the moment that the trial couldn’t have taken place as described, how is it that neither the reviewers nor the editors noticed before accepting the paper? To put it as bluntly as possible: That’s what reviewers are for. Any expression of concern in this case ought also point a finger at the journal.

As it happens, Abdelkefi does seem to have a strained relationship with data. In 2009, Blood retracted a widely-cited article, published the previous year, on which he was the lead author. The reason?

The authors wish to retract the February 15, 2008, article cited above. The initial data available at the time the paper waspublished were inappropriately collected. The first author,Abderrahman Abdelkefi, was solely responsible for the errors.The authors present their deepest apologies to the scientificcommunity and to the readers, reviewers, and editors of Bloodfor publishing these data. The undersigned agreed to the retractionby notifying the journal.

This time, however, it didn’t take six years to discover a problem. Two months of publishing the paper, titled “Single autologous stem-cell transplantation followed by maintenance therapy with thalidomide is superior to double autologous transplantation in multiple myeloma: results of a multicenter randomized clinical trial,” Blood published a letter to the editor that warned against putting too much stock in the Tunisian researchers’ findings.

In light of the findings of several large international groups, the results of the Tunisian Multiple Myeloma Study Group trial are puzzling

wrote Jayesh Mehta, a cancer specialist at Northwestern University.

The Tunisian study showed that 1Tx followed by6 months of thalidomide maintenance therapy was significantlysuperior to 2Tx without maintenance therapy in terms of OS andEFS. In addition to being somewhat counterintuitive, the findingsare unique because of the extraordinarily good outcome of the1Tx arm and the unexpectedly poor outcome of the 2Tx arm. [1Tx and 2Tx stand for single and double therapy, respectively; OS and EFS are overall and event-free survival].

Mehta concluded:

While complimenting Abdelkefi et al on designing a good study,I would like to sound a strong note of caution before its unusualfindings are used as a basis to alter clinical practice. Becauseof its unusual findings, not only does the study need longerfollow-up, but replication of the study (or a similar concept)is essential—especially in a setting where patients haveaccess to all appropriate salvage therapy medications.

In a subsequent letter to the Journal of the National Cancer Institute, Mehta took still more shots at Abdelkefi’s study, calling the findings “biologically implausible.”

A careful review of the multiple online versions of the article (4, unpublished observations) and correspondence with the authors of the article and the editorial staff of Blood (unpublished data) uncovered inconsistencies and inaccuracies that cast doubt on the integrity of the study and resulted in retraction of the two articles describing the study.

But this wasn’t piling on. Mehta was writing to alert JNCI readers that a recently published meta-analysis that included the Abdelkefi study was corrupted by the bogus data.

Although the authors make much of the statistically significant increase in treatment-related mortality with tandem transplantation, when the retracted study by Abdelkefi et al. is excluded and the denominator of patients in the single transplantation arm of the study by Sonneveld et al. is corrected to 148 from 158 (Table 1), the risk ratio for treatment-related mortality becomes 1.60 (95% confidence interval = 0.98 to 2.62) and its statistical significance becomes borderline (P = .059). …

… However, this meta-analysis is flawed and should not influence important treatment decisions, which can compromise patient welfare. Indeed, over a period of a month (July 2009), I have already seen two patients who have been counseled not to undergo tandem transplantation based on the results published by Kumar et al. I think it should be retracted.

Although the meta-analysis did trigger an exchange of comments, it has not been retracted.