Our computer ate the data: Expression of concern over blood thinner study raises concerns itself

Thrombosis and Haemostasis has issued an “expression of concern” over a 2004 paper by Tunisian researchers:

Concerns have been raised by readers about the accuracy and validity of the data reported in the September 2004 article by Abdelkefi et al., entitled “Prevention of central venous line-related thrombosis by continuous infusion of low-dose unfractionated heparin, in patients with haemato-oncological disease. A randomized controlled trial” (Abdelkefi A et al.Thromb Haemost 2004; 92: 654–661).

In the trial, 108 patients with blood cancers reportedly received infusions of either saline or heparin, a blood thinner. Those given the active drug were far less likely to develop clots related to their catheters, according to the researchers, and no more likely to experience severe bleeding. In the report, the researchers write:

This is the first prospective, randomized study, which shows that low-dose of unfractionated heparin is safeand effective to prevent catheter-related thrombosis in patients with haemato-oncological disease.

The article has had an impact, having been cited 32 times, according to Thomson Scientific’s Web of Knowledge. At some point after publication, however, the results evidently began to look fishy. Again from the journal:

Concerns have been raised over the practical aspects of the published protocol.

When asked by Thrombosis and Haemostasis to provide evidence that the study was conducted, the authors were unable to provide this information. Because Thrombosis and Haemostasis has no means of establishing the data or demonstrating that research results presented in this article are reproducible, Thrombosis and Haemostasis is publishing this Expression of Concern.

Reached by e-mail, Abdelkefi, of the Centre National de Greffe de Moelle Osseuse, told us that his group indeed conducted the study as described but that a computer malfunction in 2008 wiped out their data. Abdelfeki provided Retraction Watch a copy of an August 16 letter to Thrombosis and Haemostasis editor Gregory YH Lip in support of the paper:

Dear Dr. Lip

We are writing on behalf of  Centre National de Greffe de Moelle Osseuse.

Concerning Thrombosis and Haemostasis study:

1) We can confirm that the study protocol was approved by the local medical ethical committee, and written informed consent was systematically obtained from the patients or their legal representatives.

2) We can confirm that data was appropriately collected and analyzed.

3) Before submission, the accuracy of the manuscript has been checked by the medical committee.

4) List of included patients and medical records data (dossiers des patients, pancartes) are available and accurate. However, specific data concerning central venous lines characteristics (paper files) have not been preserved after publication. On October 2008, data files have been lost.

You will find enclosed the final protocol with the informed consent documentation.

In conclusion, this study was scientifically rigorous and there are no concerns about her integrity.

Sincerely,

Abdeladhim Ben Abdeladhim, MD

Former Medical Director (1997-2005)

Centre National de Greffe de Moelle Osseuse

Tarek Ben Othman, MD

Current Medical Director (2005-…)

Centre National de Greffe de Moelle Osseuse

But any fourth-grader with access to Microsoft Word could whip up a letter like that. It’s not even on letterhead (Abdelkefi said he and his colleagues also mailed Lip a copy on hospital letterhead, but why he couldn’t send us that version isn’t clear).

So neither we nor the journal found that particularly reassuring. And whatever materials the Tunisian researchers forwarded either never arrived or were insufficient to their cause — “not as much definitive proof as needed,” Lip told us.

Unfortunately, Lip, a cardiologist at the University of Birmingham, in England, declined to offer more details on the nature of the “practical” red flags that first drew scrutiny to the trial.

The feedback we received was that the clinical protocol seems impractical to perform, and it seemed unlikely that the study could not have been conducted the way as was claimed.

Lip added that he was not the editor when the paper was published, which limited his knowledge of the case.

We find this whole affair as troubling as it is bizarre. The “motherboard ate my homework” defense obviously deserves a raised eyebrow, but it has been known to happen. Presuming innocence, a flat-out retraction is unwarranted.

But assuming for the moment that the trial couldn’t have taken place as described, how is it that neither the reviewers nor the editors noticed before accepting the paper? To put it as bluntly as possible: That’s what reviewers are for. Any expression of concern in this case ought also point a finger at the journal.

As it happens, Abdelkefi does seem to have a strained relationship with data. In 2009, Blood retracted a widely-cited article, published the previous year, on which he was the lead author. The reason?

The authors wish to retract the February 15, 2008, article cited above. The initial data available at the time the paper waspublished were inappropriately collected. The first author,Abderrahman Abdelkefi, was solely responsible for the errors.The authors present their deepest apologies to the scientificcommunity and to the readers, reviewers, and editors of Bloodfor publishing these data. The undersigned agreed to the retractionby notifying the journal.

This time, however, it didn’t take six years to discover a problem. Two months of publishing the paper, titled “Single autologous stem-cell transplantation followed by maintenance therapy with thalidomide is superior to double autologous transplantation in multiple myeloma: results of a multicenter randomized clinical trial,” Blood published a letter to the editor that warned against putting too much stock in the Tunisian researchers’ findings.

In light of the findings of several large international groups, the results of the Tunisian Multiple Myeloma Study Group trial are puzzling

wrote Jayesh Mehta, a cancer specialist at Northwestern University.

The Tunisian study showed that 1Tx followed by6 months of thalidomide maintenance therapy was significantlysuperior to 2Tx without maintenance therapy in terms of OS andEFS. In addition to being somewhat counterintuitive, the findingsare unique because of the extraordinarily good outcome of the1Tx arm and the unexpectedly poor outcome of the 2Tx arm. [1Tx and 2Tx stand for single and double therapy, respectively; OS and EFS are overall and event-free survival].

Mehta concluded:

While complimenting Abdelkefi et al on designing a good study,I would like to sound a strong note of caution before its unusualfindings are used as a basis to alter clinical practice. Becauseof its unusual findings, not only does the study need longerfollow-up, but replication of the study (or a similar concept)is essential—especially in a setting where patients haveaccess to all appropriate salvage therapy medications.

In a subsequent letter to the Journal of the National Cancer Institute, Mehta took still more shots at Abdelkefi’s study, calling the findings “biologically implausible.”

A careful review of the multiple online versions of the article (4, unpublished observations) and correspondence with the authors of the article and the editorial staff of Blood (unpublished data) uncovered inconsistencies and inaccuracies that cast doubt on the integrity of the study and resulted in retraction of the two articles describing the study.

But this wasn’t piling on. Mehta was writing to alert JNCI readers that a recently published meta-analysis that included the Abdelkefi study was corrupted by the bogus data.

Although the authors make much of the statistically significant increase in treatment-related mortality with tandem transplantation, when the retracted study by Abdelkefi et al. is excluded and the denominator of patients in the single transplantation arm of the study by Sonneveld et al. is corrected to 148 from 158 (Table 1), the risk ratio for treatment-related mortality becomes 1.60 (95% confidence interval = 0.98 to 2.62) and its statistical significance becomes borderline (P = .059). …

… However, this meta-analysis is flawed and should not influence important treatment decisions, which can compromise patient welfare. Indeed, over a period of a month (July 2009), I have already seen two patients who have been counseled not to undergo tandem transplantation based on the results published by Kumar et al. I think it should be retracted.

Although the meta-analysis did trigger an exchange of comments, it has not been retracted.

0 thoughts on “Our computer ate the data: Expression of concern over blood thinner study raises concerns itself”

  1. The Abdelkefi paper in “Blood” seemed fishy the moment the first edition version appeared online. The story of my investigation of this work and its retraction will be published in “Bone Marrow Transplantation” (High-dose chemotherapy and autotransplantation in myeloma: Retraction of a study and lessons for the future). The investigation resulted in retraction of another paper related to the same study by the Tunisian group in “Bone Marrow Transplantation”.

    While investigating the “Blood” paper, I found 3 papers on central venous catheters by this group which were implausible clinically (i.e. not practically possible to perform). I brought my concerns to the attention of the 3 journals concerned in March 2009. “Journal of Clinical Oncology” published an Expression of Concern promptly (DOI: 10.1200/JCO.2009.25.7006). “Thrombosis and Haemostasis” did nothing until Dr Lip became the Editor. He dealt with the matter expeditiously when I enquired about the plan of action – because that was the first he came to know about the matter. I am not quite sure what the “Journal of Supportive Oncology” is doing!

    The letter vouching for the “Thrombosis and Haemostasis” study is worthless because both the signatories – Abdeladhim Ben Abdeladhim and Tarek Ben Othman – are coauthors on the paper in question (and on the other 2 central venous catheter studies as well as the retracted Blood paper).

    You write: “But assuming for the moment that the trial couldn’t have taken place as described, how is it that neither the reviewers nor the editors noticed before accepting the paper? To put it as bluntly as possible: That’s what reviewers are for. Any expression of concern in this case ought also point a finger at the journal.”

    The simple fact of the matter is that Editors are busy, and a lot of reviewers are very superficial in their analysis of manuscripts they are reviewing. I suspect the central line studies were reviewed by people who did limited clinical work themselves – and failed to realize the implausibility of the work.

    One of the problems I have faced is that editors become very defensive when problems with a paper published in their journal are pointed out to them (Dr Lip being an honorable exception). Their concern at that point seems to be more with not embarrassing the journal (and themselves) rather than investigating problems and uncovering the truth.

  2. I couldn’t agree more. When I first brought problems in the EMBO J paper 24, 4260-4270 (2005) to the attention of Bernd Pulverer, the editor-in-chief of the EMBO Journal he initially thanked me for my “vigilance”, but the next day he was recommending that I “desist from referring to the EMBO paper until a formal investigation is completed”. Surely they are “publications” not “privatations”?

    —– Forwarded Message —-
    From: Bernd Pulverer
    To: marcoberns@yahoo.co.uk
    Sent: Wed, 4 August, 2010 2:33:39
    Subject: EMBO J 24, 4260-4270: Reuse data

    Dear Dr. Berns,

    thank you for alerting us to the potential re-use of data in two different experiments in the above paper. I agree that this warrants further investigation, as the data indeed looks very similar. We will obtain the original data form the authors and update you in due course. of course, we will preserve you anonymity.
    As an aside, your notification reached me 8 hours before a talk on scientific misconduct I gave at the research integrity world congress in Singapore, and I used this as a timely example of the type of issues we receive with unfortunate regularity.

    You vigilance is much appreciated.

    Best wishes

    Bernd Pulverer

    The next day:

    —– Forwarded Message —-
    From: Bernd Pulverer
    To: Marco Berns
    Sent: Thu, 5 August, 2010 3:10:49
    Subject: Re: revenge in science: Query Regarding Allegations Toward EMBO J 24, 4260-4270, 2005

    Dear Mr. Berns,

    We are acting on the matter following our internal rules and processes. While we were not made aware initially by you that you had independently contacted the author’s institutional review board, we have noted this now. However, we will contact them only if we find sufficient grounds to do so.

    We are well aware of authorship recommendations and have instituted mandatory author contribution declarations. We are not in a position to comment on employment practices or allegations of data non-reproducibility.

    I am also being contacted by other journals now – may I recommend that you only contact journals with concrete information that affect publications in that journal and that you desist from referring to the EMBO paper until a formal investigation is completed. This is standard professional practice.

    Yours sincerely,

    Bernd Pulverer

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Bernd Pulverer, Ph.D.
    Head of Scientific Publications
    Chief Editor, The EMBO Journal

    European Molecular Biology Organization
    Postfach 1022.40, D-69012 Heidelberg
    Meyerhofstrasse 1, D-69117 Heidelberg

    Tel: +4962218891501
    bernd.pulverer@embo.org

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.