Coding errors in a database maintained by Pfizer have led authors to retract two heart biomarker papers in the Journal of the American College of Cardiology.
The two notices, for “Prediction of cardiovascular events in statin-treated patients by lipid and non-lipid biomarkers” and “Plasma PCSK9 levels and clinical outcomes in the TNT (Treating to New Targets) Trial,” are highly detailed and say the same thing:
This article has been retracted: please see Elsevier Policy on Article Withdrawal (http://www.elsevier.com/locate/withdrawalpolicy).
This article has been retracted at the request of the author.
The main findings of the TNT Trial were published in 2005 (1). Since that time, members of the Steering Committee and other investigators have published 32 papers based upon additional analyses of TNT. The data for these papers were derived from analyses of the TNT clinical database, managed by Pfizer. The clinical database has been crosschecked many times and the data in it is valid. During the trial, blood samples were drawn from the patients at regular intervals for subsequent analysis. We performed a nested case-control study that included 507 patients who experienced a CV event and 1,020 control patients in the main biomarker analysis, and 496 patients who experienced a CV event and 1,117 control patients in the PCSK9 analysis. The biomarker database was separate from the clinical database. An anonymization code was run in 2007 to link patients from one database to the other.
In late 2012, the TNT frozen blood samples were integrated into a large automated biobank that includes samples from other Pfizer trials. At that time discrepancies were noted among the samples, indicating that an error had occurred when the samples were anonymized in 2007. Further investigation revealed that the code created to manually anonymize the data was accidentally run twice. During the first run, anonymized subject identifiers were successfully assigned to both biosamples and clinical data. However, after this first run had passed quality control checks, the anonymization code was re-run inadvertently, replacing the first correct set of identifiers with a random and incorrect set. We do not understand how or why the code was re-run. The study team, who were blinded as to patient identity, thus reported on mismatched clinical and biomarker data. The investigators of the biomarkers study were puzzled that none of the 18 biomarkers were predictive of cardiovascular events. However we were reassured because on-treatment LDL-cholesterol, HDL-cholesterol and triglyceride levels were all strongly predictive of events, and we reported this in the paper. These lipid levels were part of the clinical database, and thus were not subject to the error that occurred with the biomarkers. In the PCSK9 analysis, PCSK9 levels were predictive of events in the atorvastatin 10-mg group (p = 0.039) but not in the 80-mg group. This finding, which we now realize is totally spurious, was not unexpected and raised no red flags. Similarly, the failure of vitamin D levels to predict events, as reported in the AHJ paper, was not surprising.
Since the error was discovered, we have created a new anonymized clinical and biomarker database by restoring the original set of anonymized identifiers. We are currently reanalyzing the data according to our original study plans. However, the nested case-control feature of the original study design has been lost because the patient selection for biomarker sampling was random. Only approximately one tenth of patients now had an event, compared to one third in the original study design. Thus, the power to detect a difference in the level of a biomarker between patients with and without events has been attenuated.
All authors of these manuscripts are anguished to have made this mistake and publishing incorrect information.
There’s a lot going on here — a “totally spurious” finding that “raised no red flags,” and a big difference in the number of patients who had an event — and we have contacted some of those involved to help us figure out what it all means.
It may in fact allow the researchers, with more study, to turn what was essentially a negative finding into a positive one. The University of Amsterdam’s John Kastelein, an author of both papers and corresponding author of the “Prediction of cardiovascular events…” paper, tells Retraction Watch:
Since the retraction was the result of a sample mix up and the results of our analysis were negative with regards to the predictive ability of the biomarkers in question, I, in fact, do hope that with the corrected sample labels and a new analysis we will be able to make better sense of the data.
One question, of course, is whether any of the other 30 29 [see update at end of post] papers based on this database will be affected by the mixup. (Update, 12:30 p.m. Eastern, 4/16/13: Kastelein tells us none will be, because none of them used DNA.) It seems unlikely that the original TNT paper, published in NEJM in 2005, would be affected, given that the error happened in 2007.
These two papers have not been cited much — “Prediction of cardiovascular events in statin-treated patients by lipid and non-lipid biomarkers just once,” albeit by a Cochrane Library review, according to Thomson Scientific’s Web of Knowledge, and “Plasma PCSK9 levels and clinical outcomes in the TNT (Treating to New Targets) Trial” five times.
Retraction Watch readers may recall that JACC published an unrelated Notice of Duplication earlier this month. And two other studies by Pfizer, of a potential cancer drug, were retracted last year for errors.
Update, 1:30 p.m. Eastern, 4/16/13: Larry Husten, at Cardiobrief, reports that there’s a third retraction in the works, from the American Heart Journal. We missed the reference — probably because there isn’t actually a citation — in the JACC notice:
Similarly, the failure of vitamin D levels to predict events, as reported in the AHJ paper, was not surprising.
Kastelein must have thought we knew there were three papers already being retracted.
Hat tip: @bassman578
It looks like there was no reduction in the number of persons who experienced an event — just a loss of linkage to their “nested” (= matched) controls, i.e. to the 2000-strong subset of the entire population (of about 10,000) who were (a) most similar to those individuals who experienced events but (b) did not experience an event. Thus the roughly 1000 people who experienced an event in the two 2007 studies were 1/3rd of the population of cases-plus-matched-controls described in 2007, but only one tenth of the entire study population of about 10,000 described in 2005.