Today, without us having planned it, has become the day of retracted papers that found a new home.
This morning, we posted an item about a chimp “culture” paper that was retracted from Biology Letters after its authors found some errors, and then published, with corrections, in the Journal of Human Evolution. This afternoon, we bring you the news of a PLoS ONE paper on longevity genes that is the corrected version of a Science paper retracted last year:
After online publication of our report “Genetic Signatures of Exceptional Longevity in Humans” (1) we discovered that technical errors in the Illumina 610 array and an inadequate quality control protocol introduced false positive single nucleotide polymorphisms (SNPs) in our findings. An independent laboratory subsequently performed stringent quality control measures, ambiguous SNPs were then removed, and resultant genotype data were validated using an independent platform. We then reanalyzed the reduced data set using the same methodology as in the published paper. We feel the main scientific findings remain supported by the available data: (i) A model consisting of multiple specific SNPs accurately differentiates between centenarians and controls; (ii) genetic profiles cluster into specific signatures; and (iii) signatures are associated with ages of onset of specific age-related diseases and subjects with the oldest ages. However, the specific details of the new analysis change substantially from those originally published online to the point of becoming a new report. Therefore, we retract the original manuscript and will pursue alternative publication of the new findings.
Science, in an accompanying statement at the time, emphasized
that there was no misconduct by Sebastiani and colleagues. The researchers worked exhaustively to correct the errors in the original paper and we regret that the outcome of the extensive revision and re-review process was not more favorable.
The new paper, by Boston University’s Thomas Perls, Paola Sebastiani, and colleagues, “Genetic Signatures of Exceptional Longevity in Humans,” concludes that
90% of centenarians can be grouped into clusters characterized by different ‘‘genetic signatures’’ of varying predictive values for exceptional longevity.
The Sebastiani-Perls group has published two papers since the Science retraction; today’s is the third. One, Perls told Retraction Watch, “Health Span Approximates Life Span Among Many Supercentenarians: Compression of Morbidity at the Approximate Limit of Life Span,” published in the Journals of Gerontology Series A, indicated
…that people who live to 110+ years on average compress the time they experience illness into about the last 5% of their lives. This supports the compression of morbidity hypothesis.. that is, that as a member of a species approaches the limit of life span, it necessarily has to compress the time they are sick towards the very end of their life. Along these lines we found the supercentenarians to be quite homogeneous and this is consistent with our finding in the genetic signatures paper that there is a strong genetic component to achieving these ages.
In the other, “Whole genome sequences of a male and female supercentenarian, ages greater than 114 years,” published in Frontiers in Genetics of Aging,
…the whole genome sequencing for two people over the age of 114 years, we found functional variants for a number of the genes that are part of the model that predicts exceptional longevity in the paper coming out this afternoon.
Unlike the paper we covered this morning, today’s new study does not cite the now-retracted paper that preceded it. We asked whether the authors had considered doing so:
…it frankly did not occur to me to do so (nor did any reviewer or editor suggest it). Afterall, it seems to me that it was Science’s intention to erase the paper we submitted to them from the scientific record as if it never existed so that we could publish it elsewhere… thus, what would be the purpose of citing the retraction? I do know that the current editor’s goal was to obtain as much of an unbiased review as possible, having the current paper reviewed for its current merit, independent of any history.
Fair enough, and that logic makes sense. There’s no official obligation to cite a paper you’ve retracted. At the same time, we’d argue that it was precisely because the previous paper had issues that reviewers and readers would want to know about it. Showing a particular thread of research as it progresses is, after all, consistent with the self-correcting nature of science.
Update, 11:15 p.m. Eastern, 1/18/12: The PLoS ONE editors wrote a blog post about the paper. It concludes:
While we recognize that aspects of this study will attract attention owing to the history and the strong claims made in the paper, the handling editor, Greg Gibson, made the decision that publication is warranted, balancing the extensive peer review and the spirit of PLoS ONE to allow important new results and approaches to be available to the scientific community so long as scientific standards have been met. We trust that publication will facilitate full evaluation of the study.
So why did they not publish a correction to their paper in Science?
I agree — it’s an interesting question (and I’d be interested to hear Science’s response if the revised version was submitted there). Surely, if research asks an interesting question and the methods are sufficiently robust to answer it, then journals should, if they aren’t biased, be interested in publishing the findings whichever way they go.
If they DID submit this again to Science and were rejected it might be an example of journals (and their reviewers) being prejudiced against negative results — which is something that a lot of authors believe in, but no studies have shown conclusively (at least not the ones I’m aware of).
A bit like faculty who don’t get tenure at Harvard and then are hired by other Universities. Ater all, if you were at Harvard it must mean something!
Another interpretation of
“that people who live to 110+ years on average compress the time they experience illness into about the last 5% of their lives”
is that many of those who experienced illness spread throughout their lives died from that disease, which, from its temporal behaviour we know was different.
Also
“the whole genome sequencing for two people over the age of 114 years, we found functional variants for a number of the genes that are part of the model that predicts exceptional longevity”
is a sample of two, and interpreting the data according to the model.
How can there be valid matched controls? Who knows what the “genetic signatures” of these two people over the age of 114 years mean? Causes or consequences?
A study which has a sample of 230,000 might be telling us more about human activity, and where the resources might be more fruitfully directed.
http://www.bbc.co.uk/news/world-asia-pacific-11258071
http://www.bbc.co.uk/news/world-asia-pacific-11299646
I just think that PLoS is becoming a bit of a joke really. In my opinion, it seems to be turning in to a bit of a dumping ground for poor quality/unimportant science. I’m not sure this was the intention of open access.
Dave, do you mean that statement to refer to PLoS One specifically, or all of the PLoS journals? I could see someone arguing that PLoS one is a dumping ground for “unimportant” science, as the journal intends to judge submissions based entirely whether they are technically sound rather than their perceived impact. I do not think that this characterization applies to other PLoS journals. PLoS Biology, for example, publishes many high-impact papers in my opinion, and they do retract papers at nearly the same rate as Science. While this could be because Science is more vigorous about retracting papers with problems, I think a better explanation is that it publishes only papers perceived to be groundbreaking, which are probably more likely to be bogus and later retracted. So perhaps it is journals like Science and Nature that only publish papers perceived to be very important that have become a dumping ground for bad science.
I agree with you with regard to PLoS One. I do not mean to sound elitist or anything, and I would really love to get around the issue of too much politics in publishing in traditional journals but, PLoS one publishes a lot of crap in my field. The disappointing thing is also that the Comment sections are deserted and even when comments are published the authors do not respond. What the hell? The authors should be required to respond to all comments within a week or so. The make money by publishing papers, and so there is a downward pressure to publish whatever garbage is submitted, as long as the authors pay the fee.
Just to clarify, my comments were about PLoS One specifically. I don’t really read enough papers in PLoS Biology etc to make a comment to be honest. I also really don’t want to sound elitist; god knows I don’t publish in S/N/C!
But Jon Beckmann is spot on when he talks about the comments section. They are truly, truly deserted and that side of open access is clearly a major failure, especially if you consider how many hits their sites get. The idea was to get discussions going online and stimulate some kind of rigorous post-publication peer review. But as Jon mentions, either there are no comments (which is the norm), or there are comments but no further “discussion”. Most comments I have seen are also fairly straightforward and merely point out typos etc.
I personally think that part of the issue here is that the papers are just not that important/interesting. I think it is also because scientists are a little stuck in their ways and are somewhat reluctant to be drawn into online open-access discussions. It is a bit like airing your dirty laundry in public. Science has always been conducted behind closed doors, and by that I mean away from the general public. I am not saying that this is necessarily a good thing, but it will be hard to get scientists to open up a little more in the future. The fear of retribution/payback following negative comments online is also very real for many researchers and is probably stifling the whole open access experiment.
It is also interesting to me that the impact factor of PLoS One has stagnated since it’s initial opening and I really don’t think it will rise much if they continue to be a bin for papers which have trouble getting published anywhere else.
I agree that it is clear that there for the most part no discussion going on, even in the higher-profile PLoS journals. Jon’s idea of trying to force authors to monitor and respond to comments is intriguing and could help, especially if readers are aware of this policy and therefore feel more compelled to comment.
As for PLoS One, I think the journal is designed as an island for (primarily) low impact papers, and I don’t think PLoS is concerned about this. They have other journals for “more important” papers, and as mentioned earlier are probably more than happy to continue to receive publishing fees. However, I am not so convinced this is a bad thing. It means that a paper that would otherwise go unpublished will show up in PLoS One, and if some readers find it more interesting than editors at other journals did then that’s great. Otherwise, the paper will be ignored (as is the case for most of the papers in PLoS One). As long as the review process prevents bad science from getting in, I don’t see what the big problem is. Of course, evaluating this aspect of a journal like PLoS One may be difficult.
I would like to note that the possibility for discussions is just one of the potential benefits of the PLoS system. I myself have commented on some PLoS One papers, and sincerely hope that readers of those papers will note there ARE comments to the paper, and based on those comments decide to ignore it (I pointed out some important flaws). At the same time, I have also wondered to what extent one may force the authors to respond to comments (surely not to typos, though). It’s something to consider for someone who has been asked to be on the Editorial Board. 🙂
Having disclosed that, note that the official impact factor for PLoS One is 4.4, which is not “low impact” for most of the sections, but more “middle of the road”.
I’d agree with Marco; but the comment system may be more effective than it appears. In response to one PLoS paper, I wrote some rather severe comments and sent them to an author, stating that I thought it might be best if the authors reviewed them and responded before anyone posted anything. As it happened, their response convinced me that the problems I’d seen were mostly their minor sloppiness with data, over-confident writing, and — to be sure — my own stupidity. So, I posted nothing. However, the availability of the public comment mechanism in fact forced the authors to respond, and I think we all learned from it. … but I’m lying in wait for their next paper to see if we have …
The whole idea of post-publication comments is a nice concept, but I think it runs against the basic incentive system in science.
Publications and grants are the basic units of scientific “reward.” Post-publication reviewers have no impact on whether the paper is published, and it’s not clear that their criticisms (or praise) will have any impact on whether the article is cited or taken seriously. If the post-pub reviewers are not anonymous, their comments can create political backlash in their own publication/grant reviews. The same is true for the authors: there’s no obvious gain in responding to criticisms and a potential for publication/grant backlash.
On top of this, there’s the time element. Most research faculty that I know are already swamped, and have trouble finding time for their families, let alone hobbies. It’s hardly fair to add a post-publication review as an expected service unless it can somehow directly help with grants or publications.
Unless the incentive system changes to incorporate post-publication review, I really doubt that it will ever go anywhere.
The thing is the current impact factor of PLoS ONE is actually quite high — 4.4. Yes, that may seem low in comparison to Nature or Science, but it is much higher than the majority of journals that are seen as more “prestigious” by name alone. For example, the standard journal of microbial systematics, “The International Journal of Systematic and Evolutionary Microbiology”, has an impact of only 1.9. Yes, I know there’s plenty of arguments why impact factors aren’t the be-all and end-all of journals. but it would seem that if the typical PLoS ONE paper is the sort of boring marginal paper that nobody cares about (as is so often argued), where is this impact coming from? Remember that self-citations aren’t counted in the metric.
Jonathan
All citations, including self-citations are included in the impact factor. The Web of Science does, however, include the impact factor without self cites metric in a table but it is the main impact factor (including self cites) that everyone uses and cites.
Let us not forget that Nature did an experiment on “open peer review” in the form of post-publication commenting and it was a resounding failure:
http://www.nature.com/nature/peerreview/debate/nature05535.html
It is a great idea in theory, but I just don’t see the evidence that it is really catching on. I think a far more important practice is the publication of the “review process files”, as is now routine at EMBO J.
At the time of Sebastiani et published a retraction of the original publication in Science, the Science editors issued a statement about why the journal wouldn’t publish the revised research:
Although the authors remain confident about their findings,
Science has concluded on the basis of peer-review that a paper built on the corrected data would
not meet the journal’s standards for genome-wide association studies. One such standard, for
example, is the inclusion of a reliable replication sample that shows comparable results to those in
the initial experiments.
The authors have therefore agreed to retract their paper. Science emphasizes that there was no
misconduct by Sebastiani and colleagues. The researchers worked exhaustively to correct the
errors in the original paper and we regret process was not more favorable.