A new paper in the MDPI journal Publications reports that the only controlled study on the effect of giving COPD patients Omega-3 has been cited 52 times since being retracted. Of those, only two mentioned the retraction.
In 2005, Chest published an article that found that COPD patients who took omega-3 supplements for 2 years experienced improvements in their condition, such as better walking tests and a decrease in sputum cytokines. But when an institutional investigation found the lead author had falsified the data, the journal retracted the paper in 2008.
That’s news to many researchers in the field. Among the 50 papers that cited the research after 2008 without stating it had been retracted, 20 included “specific data” from the paper, while the other 30 “cited the reference in passing.” Articles citing the retracted study have themselves been cited 947 times total, pointing to the ripple effect this kind of unwitting mention can have throughout the literature.
These findings suggest that the retraction “was unsuccessfully communicated to the scientific community,” the authors note. On Chest’s site, the retracted article, “Effects of Omega-3 Polyunsaturated Fatty Acids on Inflammatory Markers in COPD” appears with a link to the retraction notice; PubMed also contains a note saying it is retracted.
The perpetuation of bad data is not just a problem for scientists, as the authors note. Googling “fish oil and COPD” (without quotes) brings up both news articles and pages on reference guides like WebMD and Everyday Health that reference the research without noting the retraction.
John Budd et al have been noting this problem since 1999, when they found that 92% of post-retraction citations failed to mention that the paper had been pulled.
The authors tap Retraction Watch as a useful resource, but until we get our database up and running, we aren’t a comprehensive stop for scientists checking their citations. They suggest both that journals make retractions very clear on the paper itself, and that scientists be more proactive in making sure they’re not citing retracted work.
Here’s the abstract for “Persistent Citation of the Only Published Randomised Controlled Trial of Omega-3 Supplementation in Chronic Obstructive Pulmonary Disease Six Years after Its Retraction”:
Scientific articles are retracted infrequently, yet have the potential to influence the scientific literature for years. The only randomised controlled trial to explore the effects of omega-3 polyunsaturated fatty acids in people with chronic obstructive pulmonary disease was retracted in 2008 due to falsified data. The objective of this research was to determine the frequency and nature of citations of this retracted paper. Web of Science and Google Scholar were used to determine the number of times the retracted article was cited. Citations were classified as either “retraction acknowledged” or “retraction not acknowledged”. The search was conducted on 6 August 2013 and updated on 25 March 2014. Results: The search resulted in 76 citations, of which 24 occurred prior to the retraction of the article. Of the 52 citations occurring after the retraction, only two acknowledged the retraction. Of the citations not acknowledging the retraction, 20 referred to specific data and 30 cited the reference in passing. This retracted article continues to be cited by authors, suggesting that information about the retraction was unsuccessfully communicated to the scientific community. Continual citation of retracted literature has the potential to bias a field of research and potentially misinform end-users.
Hat tip Rolf Degen.
This is really sad, and it’s a difficult problem to solve. However, the information that you are working on a database of retractions makes me feel a little better about it. That will help, although I think more work will be needed.
Now why would anyone cite retracted papers as supportive to one’s own research? One explanation might be that their own research is possibly not that trustworthy. If even partially true, for a single proven wrong (i.e. retracted) paper there are in this case up to 50 potentially phony ones? Scary possibility, but maybe not that far from reality.
It would be revealing (and not hard to do; if no one else gets to it before I clear up some time, I’ll try to do it myself) to see how many of the “50 papers that cited the research after 2008 without stating it had been retracted” also cited one or more papers published before the retraction that themselves site the to-be-retracted paper. My guess is that the authors of a fair proportion of that unknown number were guilty “only” of including in the references a paper that they had not actually read. (The papers that featured “specific data” are probably not of that kind.)
It might even be that, in some cases, the post-retraction citing authors were (horror of horrors!) referring to paper copies of the journal, possibly because their institution did not have an online subscription. In that case, it would be much easier to miss the retraction, which would presumably be bound in a later volume (librarians who do otherwise, please let me know!) rather than included with the retracted paper.
It isn’t just paper copies. A lot of folks these days will download the PDF of a paper and store it on their own PC, then refer to that copy in the future … if the paper gets retracted thereafter, the user will never know. I’ll often also just scan a paper online once, note the “prior art” I need to cite in my own work, download the bib info to citation management software with appropriate annotations, and drop it into my papers on the subject forevermore. Again, if it then gets retracted I’ll never know unless I find out by chance.
It’s a hard problem to solve. If there was a consistent electronic flag associated with the DOI or (in my field) PMCID of retracted papers, then citation management software or publisher-side servers could be programmed to scan for it in the same way many publishers now automatically include database links in the reference list of the HTML version of the paper, and alert the authors if that flag shows up. But to my knowledge nothing that consistent is out there. From my perspective I’d loathe finding out that I had cited retracted work, but I don’t know how to prevent it short of taking on the onerous task of re-looking up every single paper I cite before I upload a manuscript.
Or, of course, that I wrote a paper in 2010, which got published in 2014, citing a paper retracted in 2012. Once the citation is in the manuscript, there’s zero chance I’ll go back during refereeing to see if it’s been retracted.
I would have some examples why one would cite a retracted paper. Not sure if any of this happened in those cases. 1) A given paper may have been retracted because of considerable plagiarism, however this does not preclude the existence of original information therein that one would like to cite. 2) Another given paper may have been retracted because of authorship or other conflict issues, which doesn’t directly say anything about the validity of other information therein. It is not hard to imagine a scenario in which a displeased influential PI could persuade a journal to retract a — possibly otherwise valuable — paper by a younger ex-member of his/her lab. 3) Another paper may get retracted because of a major issue in the status of data presented and/or resulting conclusions, by the honest mistake and request of authors. This does not mean that the methods and introduction described in that paper is invalidated or should not be used by another scientist.
One facet of retractions that I find negative is a direct power given to editors and publishers in deciding over what should is good or bad for the scientific community. I understand public statements and retractions should be made over concerns and published mistakes as inherent part of the process of publication, but how this should be interpreted by fellow scientists should be up to their judgement. Likewise, no one is obliged to take seriously “unretracted” papers. Publishers and editors should serve the scientific community and not the other way around.
Another omega-3 fatty acid paper.
Cancer Res. 2001 Feb 15;61(4):1386-91.
Suppression of tumor cell growth both in nude mice and in culture by n-3 polyunsaturated fatty acids: mediation through cyclooxygenase-independent pathways.
Suppression of Tumor Cell Growth Both in Nude Mice and in Culture by n-3 Polyunsaturated Fatty Acids: Mediation through Cyclooxygenase-independent Pathways 1
Mary D. Boudreau, Kyung Hee Sohn, Sang Hoon Rhee, Sam W. Lee, Jay D. Hunt, and Daniel H. Hwang2
– Author Affiliations
Pennington Biomedical Research Center, Louisiana State University, Baton Rouge, Louisiana 70808 [M. D. B., K. H. S., S. H. R., D. H. H.]; Stanley S. Scott Cancer Center, Louisiana State University Health Sciences Center, New Orleans, Louisiana 70112 [J. D. H.]; and Department of Medicine, Beth Israel Hospital, Harvard Medical School and Harvard Institute of Medicine, Boston, Massachusetts 02215 [S. W. L.]
Please see Pubpeer comments.
https://pubpeer.com/publications/11245439
Of course, having copy editors actually check references is one way to approach this problem, as well as failures to note errata and the sloppy propagation of just plain wrong citations. Unfortunately, I’ve found this practice to be almost universally discouraged (and occasionally the subject of reprimand when the Peter Principle is in full flower).
Perhaps an automated tool could flag citations to retracted articles?
Morgan, a friend of mine developed a tool that does something similar called rbutr. when you visit web pages the tool notifies you if there has been a rebuttal to the information given on that page. It obviously requires people to link the rebuttal to the page but i think a similar system could be used to do what you are talking about.
I was taught to fully and critically read any paper that I cite in my articles. If one does so then a retraction becomes self evident. I believe nowadays many scientists do not fully read any cited papers, relying on the abtract, available from a number of data bases, alone. These abstracts are not labeled with a retraction statement. Incidently, another problem with relying on abstracts alone is that research has shown that data in abstracts can be inconsistant with what is reported in the text of the paper.
Unfortunately it is worse than just reading the abstract, many papers are cited without being seen, the authors simply use the citations from someone elses paper. It is easy to see this when looking at citation reports and seeing how an incorrect citation propagates through the literature.
Rob, could you please provide the references that support your last claim: “research has shown that data in abstracts can be inconsistant with what is reported in the text of the paper”
Pitkin RM, Branagan MA, Burmeister LF. Accuracy of data in abstracts of published research articles. JAMA 1999; 281: 1110-1111.
Siebers R. Data inconsistencies in abstracts of articles in Clinical Chemistry. Clin Chem 2001; 47(1): 149.
Siebers R. How accurate is data in abstracts of research articles? N Z J Med Lab Sci 2000; 54: 22-23.
Siebers R. Data inconsistencies in abstracts in the New Zealand Medical Journal. N Z Med J 2002; 115(1147): 57-58.
Siebers R. Data in abstracts of research articles. Are they consistent with those reported in the article? Br J Biomed Sci 2002; 59(2): 67-68.
This is where an app like rbutr could really help. If retractions were immediately added as rebuttals to the article pages, then people would be alerted to the rebuttal when landing on the page.
See example of a retracted paper already in rbutr: http://rbutr.com/http://www.sciencedirect.com/science/article/pii/S0278691512005637
Zombie articles? They’re supposed to be dead, but they aren’t.
“cited the reference in passing” might simply be a case where the authors found A (reference), and I found B (evidences).
This scenario might happen if you were not aware of the retraction (for many of the reasons explained above).
I disagree with the statement “citing perpetuates wrong data” If 130 papers cite a study saying that the conclusion is wrong, then I would not put my money in the paper that was highly cited…
Although the entire body of comments above is entertaining, and in fact valid (all sides of the argument), it leaves us with one serious problem: who is going to alert the journals that published papers referencing retracted papers? Independent of the reason for using a retracted paper in the reference list of a manuscript, the “retracted” nature of the paper MUST be absolutely clear to the editors and the readers. For exmaple, in a PPPR paper I am now writing, I clearly add one retracted paper to my reference list. However, I clearly specify that it is retracted (adding also the URL of the retraction notice) and in the text itself, I explain the reason for using a retracted paper’s information. It all boils down to common sense, really. As long as we are fully up-front and honest about the use of a retracted paper, there is no problem. The problem thus lies when the reader is not aware that a reference in a reference list is in fact a retracted paper. I have on ample occasions at RW indicated that authors are first reponsible for contacting those journals that published papers that referenced a retracted paper. In the case where authors fail to, or think that it is not incumbent upon them to do this, then the editors and publisher must assume this responsibilty. But this do-nothing attitude that continues to exist is not the right way forward. And may I be so blunt as to say that the greatest responsibility right now lies with the publishers, who continue to make profit or gain unfairly (i.e., impact factors) from papers that have “corrupted” reference lists (in essence false, inaccurate, outdated or incorrect information).
I find it very odd that papers published after the official retraction did not that this paper had been retracted.
For satisfying my own curiosity, I plugged in the PMID (18842931) in RefWorks.
It brings up: “Retraction. Effects of omega-3 polyunsaturated fatty acids on inflammatory markers in COPD”.
Either there is a disconnect between the PubMed index and citation databases or authors manually edit this particular citation in their bibliography.
sQuatch, excellent point. Is “manually edit” the euphemism for manipulate? If so, why would any author want to do such a thing? In other words, I am struggling to see the practical importance of actually going out of one’s way to actively reference a retracted paper. Please give more insight about your idas because this perspective is not being sufficiently explored but I have a gut feeling that you are onto something important, which could be a valuable starting hypothesis well worth testing (there are enough cases to put to the test now, even as small pockets of case studies).
If you go to Chest website and look up the paper, the retraction note is clearly there but if you download the .pdf to read and bypass looking at the html page, there seems to be nothing that points out this is retracted. Very easy therefore when running through a long list of references to download all pdfs for later reading and not recognise the retraction. Downloading the citation from pubmed by endnote also doesn’t seem to flag as retracted in the endnote / ris file. Very easy therefore for the busy clinician /researcher doing all his/ her own work to overlook the retraction inadvertently.
>Downloading the citation from pubmed by endnote also doesn’t seem to flag as retracted in the endnote / ris file.<
This is incorrect. When exporting the citation from PubMed, the article information reads as: TI – "Retraction. Effects of omega-3 polyunsaturated fatty acids on inflammatory markers in COPD."
I'm not familiar if citation tools such as EndNote, RefWorks or Menderley synchronize a user built database with PubMed on a regular basis to update literature information. However, that should be a fairly easy to be implemented.
Yes, if you import this citation before the retraction was indexed on PubMed, then you will not notice that this paper has been in fact been retracted. However, if you import this paper after the retraction, than the paper will have "Retraction" in the title.
There are two separate entries in PubMed.
The original article is here:
http://www.ncbi.nlm.nih.gov/pubmed/16354850
Clicking “Send to” then Citation causes the original citation, with no retraction information, to download.
The retraction notice is a separate entry on PubMed.
http://www.ncbi.nlm.nih.gov/pubmed/18842931
Clicking “Send to” then Citation will download the citation that you’ve mentioned.
What I was trying to indicate is that if you click currently on ‘download citation’ from the Chest webpage, the Endnote file that you get doesn’t mention the retraction as far as I can see. This means the only indication available are on the html webpage where the erratum/ retraction are mentioned but quite easy to overlook if one is in the habit of reading pdf files and not the rendered article on screen
Yep. It seems the journal, Chest, preserved the original publication and citation, and issued the retraction notice as a separate publication. This explains why there are two entries in PubMed. It also explains why it’s possible to download citations to the original article (and even the original PDF version of the article!) from the journal itself, Pubmed, Google Scholar and possibly others without knowing it’s been retracted.
As an aside, what responsibilities do – or should – peer reviewers have for identifying retracted citations?
Is this the article?: http://www.medscape.com/viewarticle/519351_2
Don’t see any retraction notice, it would have to be obvious wouldn’t it?
Might be an interesting project to check with citers where they read it.
What would researchers have to gain by risking their reputations by seemingly promoting a cheap product? It might be good to do detailed articles on the faults of retracted articles.
when I peer review papers, I always check the cited references. It’s depressing how often citations erroneously get propagated from one paper to the next. Finding where a number or statement came from turns into a Sherlock Holmes adventure and sometimes ends in a dead end. I bet a large number of the citations of retracted papers simply carried over the reference from another paper without even reading the abstract.
That was what I suspected (in my post of February 18, 2015 at 2:42 pm). Your suspicions seem more firmly grounded than mine; thank you for posting them.
I have a rule: never cite unless sighted. It’s happened a number of times that claimed comments in one paper were based on citation chains, which start with a misunderstanding of a primary finding.
That said, just because a paper cites a retracted paper that does not necessarily mean its conclusions are invalid. In the case of review papers, they generally review the literature, so the reference to the retracted paper, while unfortunate, is unlikely to be terminal. For research papers, to be published there would need to have been novel findings, independent of what came before.