Panel says BMJ was right to not retract two disputed statin papers

A panel reviewing The BMJ‘s handling of two controversial statin papers said the journal didn’t err when it corrected, rather than retracted, the articles.

The articles — a research paper and a commentary — suggested that use of statins in people at low risk for cardiovascular disease could be doing far more harm than good. Both articles inaccurately cited a study that provided data important to their conclusions — an error pointed out vigorously by a British researcher, Rory Collins, who demanded that the journal pull the pieces.

In a letter to Godlee this spring, Collins wrote:

What the BMJ needs to do is withdraw these seriously damaging claims explicitly and unreservedly with a clear explanation of why they are so wrong and what is likely be correct, and to demonstrate that it is serious about rectifying the damage that it has caused by retracting both of these papers. …

He stressed:

the need to rectify the harm that has been caused –perhaps resulting in large numbers of unnecessary deaths, heart attacks and strokes among patients at elevated risk – by misleading doctors and the public with gross over-estimates of the rates of side-effects with statins.

The journal allowed the authors to excise the problematic citation, and BMJ editor Fiona Godlee called for a panel of experts to review the matter. According to the panel’s report, which was released today here:

The panel were unanimous in their decision that the two papers do not meet any of the criteria for retraction. The error did not compromise the principal arguments being made in either of the papers. These arguments involve interpretations of  available evidence and were deemed to be within the range of reasonable opinion among those who are debating the appropriate use of statins. In making this assessment, the panel is not expressing an opinion about the merits of these arguments, as that work was beyond the scope of the panel.

The panel did have one final comment. It became very clear to the panel that the fact that the trial data upon which this controversy is based are held by the investigators and not available for independent assessment by others may contribute to some of the uncertainty about risks and benefits. Different investigators may come to different conclusions with the same data. In fact, a particularly germane example occurred recently in which two experienced Cochrane groups were charged with evaluating a particular intervention and, despite being given the same instructions, data and resources, did not arrive at identical results or conclusions25 26. The panel strongly believes that the current debates on the appropriate use of statins would be elevated and usefully informed by making available the individual patient-level data that underpin the relevant studies.

It’s important to note that Collins — the researcher who called for the retraction of both papers — is the leader of the group that holds onto that data. As The Guardian noted earlier this year:

Collins and his team at Oxford University’s clinical trials service unit are the only investigators who have seen the full patient-level data for some – but not all – of the statins trials. They formed the Cholesterol Treatment Trialists Collaboration to monitor and analyse statins data over the years. His critics point out that drug companies have helped fund his work.

The panelists did ding the journal for moving slowly to correct the papers, pointing out that the error was first raised in October 2013 by readers and not fixed until May 2014.

The BMJ editorial staff should implement a significant event audit in relation to the need for the correction. The aim of the audit would be to try and identify what would need to have been in place to ensure that the correction was made in a more timely fashion.

Finally, the panel also noted “with concern” that

that despite the Editor’s repeated requests that Rory Collins should put his criticisms in writing as a rapid response, a letter to the editor or as a stand-alone article, all his submissions were clearly marked ‘Not for Publication’. The panel considered this unlikely to promote open scientific dialogue in the tradition of the BMJ.

Godlee, in a statement, said:

This has been a challenging time for The BMJ but I am very pleased the panel has taken the view that we acted appropriately. I echo the panel’s call for the individual patient data from the statins trials be made available for independent scrutiny. Patients and their doctors need access to all relevant information to make informed decisions about their health. Extending statins to healthy people is a topical issue of wide public interest and we will continue to cover the debate from all sides.

Of note for us here at Retraction Watch, the panel apparently searched in vain for

…any guidelines for the retraction of opinion pieces, and could not identify any precedent for retracting an opinion piece unless it was based on fraudulent data. The panel noted that retraction has almost always been reserved for papers where original data are presented.

Broadly speaking, this may be the impression one would get from looking at a lot of retractions, but we’ve certainly covered plenty of retractions of review papers that don’t present any original data. And last year, David Vaux told the story of retracting a News & Views piece from Nature. So perhaps there is an opportunity to create retraction guidelines for papers that don’t present original data.

One thing’s for sure: This is the most detailed justification for a journal’s decision not to retract a paper that we’ve seen in a long time, perhaps ever. And given the opacity of many journals’ processes for responding to questions about what they publish — not to mention the opacity of many retraction notices — that seems like a good thing.