A group of researchers from Mexico has been forced to retract their July 2012 paper in the International Journal of Radiation Oncology *Biology*Physics after a reader noticed cracks in the data that proved to be signs of fatal instability.
Here’s the retraction notice for the article, titled, “Randomized Clinical Trial to Assess the Efficacy of Radiotherapy in Primary Mediastinal Large B-Lymphoma”:
This article has been retracted at the request of the Editor-in-Chief and Author.
We regret to announce that we must retract this article because errors have been identified in the publication involving several parts of the study including extraction of data, statistical analyses, and interpretation of results. Unfortunately, such extensive changes cannot be addressed in a corrigendum and would warrant a new manuscript. It is important to note that this is not considered to be the result of scientific misconduct, but rather honest errors by the authors. We regret any problems that this article may have caused and retract it from the literature.
Anthony Zietman, Editor-in-Chief on behalf of the Journal.
Augustin Aviles, Corresponding author on behalf of the authors.
The study has yet to be cited, according to Thomson Scientific’s Web of Knowledge.
Zietman provided us with more information about what went awry here:
This paper was very thoroughly reviewed by three senior figures prior to acceptance and multiple revisions made although few related to the statistics. Shortly after publication I was contacted by a reader, also a senior clinician, who felt that some of the numbers did not add up….inconsistencies between numbers in tables, text, and figures, and more. He felt these were so numerous and elementary that they undermined the credibility of the entire paper. I asked two local statisticians to take a look at the paper and both agreed. When I then challenged the authors they, without contest, recognized and accepted the errors and then a fuller story unfolded. This randomized trial was sponsored by the Mexican government. The funding dried up and the statistical analysis ultimately had to be performed by the investigators themselves with limited software packages (and probably limited statistical sophistication). They could not rerun and correct the analyses because the raw data was held elsewhere and there was no funding to pay for a statistician anyway. Thus a corrigendum was impossible and a retraction inevitable. The authors did not protest, they simply expressed their disappointment. …
I did check with the Mexican department of health that the trial had actually taken place and that interim analyses had been performed. It had!
Zietman, who took over the editor’s chair of the journal last January, said the case taught him a couple of valuable lessons.
1. even senior reviewers can miss elementary errors
2. statistical review of large and important randomized trials is probably always necessary especially when the work is not from a major, recognized, international trials group (e.g. one of the NCI co-operative groups) and especially when a statistician is not in the author line. The problem is the shortage of available clinical statisticians
Since taking over the journal Zietman has brought in
an additional editorial layer “of eyes” to try and catch such papers before they go into print and hope to act quickly in the future when they are identified.
How does that work in practice? Zietman explains:
Previously the Editor found the reviewers himself, and considered the reviews as they came in to reach a decision. This worked just fine for many years when the journal had just 500 submissions per year but the it has grown exponentially over the last decade and, with now 2000 manuscripts per year, it had become very difficult to give deep editorial thought to the quality of each review and decision.
I completely restructured the process and now have a working (and to some partial and token degree, paid) editorial board that is divided by expertise into 9 separate groups (cancer of the breast, cancer of the lung etc). Each editorial group has five experts ( a senior editor and 4 slightly more junior associates) and each group handles a manageable 200-300 manuscripts per year. They read the manuscript, carefully select reviewers, read and assess the quality of the reviews, and then further discuss the paper regarding its quality, its priority, and its credibility. By the time a paper is accepted it has been passed under many sets of eyes thus increasing the chance that errors will be detected before print. We really feel that this process is going to weed out the bad science and, in some (hopefully rare) cases, protect us against the bad actors.
Zietman said that after having no retractions in its first 38 years of existence, the journal is now working on its second:
The next one is also an unsophisticated analysis of complex data and, I think, represents another example of honest error over malfeasance.
There are several points about this study and its retraction which I find puzzling:
The authors stated in their original manuscript that the aim of the study to “observe a difference of 15% in OS [overall survival].” Then they go on to explain that the study was terminated early following an “[…] interim analysis
performed in July 2004 showed that patients who did not receive RT [radio therapy] had early relapse or progression”. There is of course nothing wrong with the early termination of the study, but what I find striking is that the overall effect of not receiving RT was so blindingly obvious (see also Figure 1 of the paper) (http://www.sciencedirect.com/science/article/pii/S036030161103210X# behind paywall) that there was essentially no need to perform any form of sophisticated statistical analysis to tease out differences between the arms of the study. At this point the main aim of the trial was already achieved. And as far as I can tell from the presented data this seems to be an important finding, which in the interest of future patients should probably be published so that oncologists and other clinicians can make treatment decisions based on these findings (I have no knowledge about this field, therefore I cannot judge what is and what isn’t standard of care, but he fact that the paper was initially accepted and that no concerns about the novelty of the research were raised seems to confirm that these results are new).
For me personally it is very hard to deduce where exactly the statistical errors occurred in the paper and/or where the inconsistencies between text, table and figures are to be found. In part that’s due to the fact that now “RETRACTED” is printed over portions of the results section. What immediately jumped out at me are the two survival curves, and their rather sparse labelling (e.g. “Fig.2 Overall survival is shown. [sic]”), but this is certainly something that could have been corrected (and that those three senior reviewers and the editor should have really addressed before accepting the paper for publication!).
As a reader after the fact (of the retraction) it is very hard to retrace the problems pointed out by the editor.
Another dimension is added by the fact that the data from this trial is for now buried and people involved in this process (authors, reviewers and editors) seem to forget that actual human lives were lost in this study (look at the survival curves!) – it may sound a bit melodramatic, but their deaths are (for now) in vain.
I agree with Sebastian here. If the benefits of radiotherapy in primary mediastinal large B-cell lymphoma are so “blindingly obvious” then this needs to be published–or else the controls will have been sacrificed in vain. Why did the funding dry up? Can’t the “Mexican government” pay for a decent statistical analysis? I know of at least one clinical statistician who was forced into early retirement (not through conflicts) and I was unaware that there was a shortage in this discipline. There is a back story here that I think would be very significant as well as interesting.