Science plays two — a retraction, and concern issued about genetics papers

It’s a busy week at Science. The journal is retracting a controversial paper about which it had previously expressed doubts, and has published an “Expression of Concern” about another article that looks like it might be headed for the same fate.

First, the retraction.

The move involves an October 2009 paper, on which we’ve previously posted, by European researchers who claimed to have made a major advance in the ability to watch how enzymes behave in cells — thereby giving scientists a new tool for monitoring the function of genes.

But back in December, Science editor Bruce Alberts issued this “Expression of Concern” about the research:

[S]erious questions have been raised about the methodsand data presented in this article. The questions focus in particularon the synthesis of the dye-labeled metabolites that are centralto the microarray technique. In addition, the spectroscopicdata the authors cite in support of their claim were not posted to the Bangor University School of Biological Sciences Web siteat the time of publication, despite the authors’ indicationin the Supporting Online Material that the data would be soposted. In response to inquiries from Science, the authors haveprovided new descriptions of the synthetic methods that differsubstantially from those in their published article. Based onour original concerns and the authors’ response, Science hasrequested evaluation of the original data and records by officialsat the authors’ institutions: These officials have agreed toundertake this task.

When we touched on this matter earlier this fall, Science had asked the researchers to retract their article. Now they have:

To our profound regret, peer inspection of the paper after publication revealed errors and omissions in the information provided on the chemistry underlying array compound synthesis, and the processing of array data obtained. After an investigation, the Ethics Committee of the CSIC in Madrid has recommended the withdrawal of the paper. Given the errors in the paper, and the skepticism about the array that they have generated, we retract the paper. We apologize to Science, our institutions, and the scientific community for any inconvenience caused by our paper and its retraction.

Manuel Ferrer, the senior author of the paper, tells us by e-mail that his group stands by its method.

Well known international companies have successfully used the metabolic array technology and that we continue using it. At the moment we are working in two complementary lines. The first one is to make the array available for research academic laboratories and to provide to international well recognized companies. In both cases, the technology have been successfully employed. In the last case the CSIC approved the testing assays by companies. The second is that some of the authors started a[n] exhaustive analysis of the data and we are doing additional analyses to get published again the chemical part of the paper.

As I mentioned several times, I agree with the fact that some errors were in the paper, although does not means that the technology is invalid. This was the reason to retract the paper. It is unclear still for me the CSIC reasons to ask for retraction.

The study has been cited 15 times, according to Thomson Scientific’s Web of Knowledge.

Now for the “Expression of Concern.” (A a bit about that construction: a frown or cocked eyebrow from your teacher is an expression of concern; a letter from the editor of Science casting doubt on the validity of a manuscript is more like a suspension conference with the principal and your parents).

This story is hardly new. In July, researchers at Boston University reported in Science that they had found certain gene variants, called single-nucleotide polymorphisms, or SNPs, linked to Methuselah-like longevity.

As they explained in the abstract to their paper, “Genetic signatures of exceptional longevity in humans:”

we undertook a genome-wide association study of exceptional longevity (EL) in 1055 centenarians and 1267 controls. Using these data, we built a genetic classification model that is based on 150 single-nucleotide polymorphisms (SNPs).

The Boston group, led by Paola Sebastiani, claimed to have found that 90% of centenarians “can be grouped into 19 clusters characterized by different combinations of SNP genotypes — or genetic signatures — of varying predictive value.” In centenarians, they added, the different SNP signatures appeared to either prevent or to delay the onset of age-related illnesses  such as dementia, high blood pressure and cardiovascular disease.

Not surprisingly, the findings made a splash in the lay media. But a letter this week from Bruce Alberts, editor-in-chief of Science, tries to walk back the initial report. (The study is too new to have shown up as cited in Thomson Scientific.) Other experts in genome-wide associations “have expressed technical concerns” about the paper, Alberts writes.

In their study, Sebastiani et al. used a number of different genotyping platforms and neglected to perform data quality-control steps, which resulted in their reporting several false-positive single-nucleotide polymorphism (SNP) associations.

The major flaw, Alberts continues, lies with the researchers’ use of a device called the Illumina 610-Quad array. What’s wrong with it? According to Alberts, the instrument “has been shown in unpublished studies by other investigators to produce artifactual genotype data at a subset of SNPs.”

Artifacts sound bad — much worse than this story about the 610-Quad when Illumina unveiled the device back in 2008. But as soon as the Science article appeared online, experts began casting doubt on the findings, saying they looked too good to be true (how familiar does that sound?). And less than a week after the Boston group’s article appeared online, Newsweek reporter Mary Carmichael reported that Illumina’s the way the researchers used the DNA chips was suspect.

Carmichael quoted Duke geneticist David Goldstein — who evidently brought her the story of the potential flaw with whom she discussed the study —  as suggesting that false-positives led the Boston group astray. Those false-positives could have resulted from the researches having used two different Illumina DNA chips for their analysis, Goldstein told Carmichael.

In laying out the case against the BU findings, Carmichael’s article took a few swipes at Science, including this one:

The flaw in question could be easily addressed with a little follow-up research. In very simplified terms, all that’s needed is for someone to rerun the analysis using a single different DNA chip. But this should have been done already, before publication. The fact that it wasn’t raises the question of how a paper with a missing piece like this got approved and published by Science.

How, indeed? We’ll ask Alberts that very question. Meanwhile, he states:

Science and the authors are taking these concerns seriously. Since learning of these potential problems, Sebastiani et al. have been performing a thorough quality control analysis on the original raw data, as well as generating new data to compare the genotype calls from the 610-Quad array and the other platforms within the same individuals. These steps aim to eliminate biases between platforms. Furthermore, they are undertaking an additional validation measure on several SNPs via the TaqMan® assay, a non–microarray-based genotyping method.

After ensuring that all data are clean, they will redo the statistical and modeling analyses, which they expect to be completed in December. At that point, Science will re-evaluate the paper, determine the extent to which the strength of its original conclusions has been altered by the revised data, and take the appropriate action.

Better late than never, right?

We should note that both the retraction and the expression were included in the materials sent under embargo to reporters this week — which we applaud. Other journals — we’re looking at you, PNAS and Nature — please take note.

Note: Mary Carmichael emailed us to clarify that David Goldstein did not “bring her the story,” as we’d stated, but that he was the “first of several sources I interviewed who brought up the general problem of potential “batch effects,” or artifacts that can arise from using different chips in different frequencies of cases and controls.”

Also, Carmichael points out that all DNA chips, not just Illumina’s, are prone to false positives. The heart of the matter in the B.U. study, she says, is that the researchers used different Illumina chips when analyzing their cases and controls.