We’ve all seen the papers. You know, the ones that report a finding without citing the group that presented the same result years before. Or, more egregiously, claim outright to be the first to report it, when a simple literature search would reveal that not to be the case.
It’s a problem that affects every area of research: Authors omitting key citations, making the results appear more novel than they actually are. Sometimes it’s the result of an innocent oversight, sometimes an outright intent to deceive. The question is:
What should be done about it?
We’re resurrecting a regular feature on the site — “ask Retraction Watch” — to get your thoughts on the topic. Below, let us know how often you notice papers that lack essential citations — and how you believe journals should handle such omissions.
(In keeping with the topic, we’d like to make it clear we are certainly not the first to raise concerns about these types of “citation amnesia” — not by a long shot. There are simply too many references to include, but here’s one from 1980.)
[polldaddy poll=9312447]
[polldaddy poll=9315240]
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy.
My boss writes the papers and adds references through Endnote. The reason why key refences are not included is because he is too busy (or lazy ?) to add new references to endnote. Also having fewer references cuts down on page costs for publishing.
The reviewers are equally lazy. When I review papers, I make a point to say it when their list of references doesn’t include anything from the last four years other than their own papers.
And what should be done about authors/scientists who systematically ignore publications of other research groups on their field?
We previously faced such a situation, and here is our reaction.
Best,
Marco
* https://www.academia.edu/10016938/Disregarded_Conflicting_Results_with_Prior_Research_A_Case_Report_in_a_Leading_Biomedical_Journal
I’m not voting because, for question 1) I don’t read enough science papers to have an opinion, and for question 2) because the answer is “it depends.”
Put a comment on PubPeer or PubMed Commons pointing out the original work and the missing references. It’s important for credit to be given correctly. It’s especially important if authors are claiming some kind of novelty – often the case for papers in good journals.
Indeed, a claim for something new which is not new at all is particularly annoying. I just posted an example on PP:
https://pubpeer.com/publications/B174893B718E99402BE4CAAF1397BA#fb46808
It is the responsibility of reviewers to catch this before papers are published – full stop. I have often insisted on citations being added, and on several occasions have pointed out that results are based on previous work that is – conveniently or otherwise – not mentioned. In one case the entire paper repeated a study that was done ten years earlier, and much better (needless to say the new effort was outright rejected). It is also the reviewer’s responsibility to make sure that citations are appropriate. I know it’s time consuming and boring, but the editors are not going to do it.
I’ve been on the opposite side where reviewers have insisted on adding citations to “key papers” that actually have zero relevance (perhaps mild tangential relevance, but not worthy of citation in the paper) to the paper in question. Given that in one case, all 3 requested citations were for papers by one author, my only assumption was that this person was indeed the reviewer in question and trying to bolster their own citations, in the end I included a citation to one of the 3 papers (the least irrelevant of the 3) to appease the reviewer…
And sometimes, after the paper is published, people send angry emails because their “seminal” work wasn’t cited. Never mind that the old papers have no, or at most very little, relevance to the article in question. Different people may have completely different opinions of what relevant citations are. If we’re going to start correcting or retracting papers because of missing citations, maybe we should also considering the same if there are citations that are not relevant.
How then are we going to promote reproducibility and repetition studies?
Sometimes the authors don’t have a choice. There are multiple journals that limit the number of references allowed. I realize this is to reduce page count, but this is part of the problem. Journals should not contribute to the problem by limiting references.
I am glad that Retraction Watch raised this important question again. Actually, over a decade ago, I identified (intentionally) “missing citation” as “Another common and detrimental misconduct in scientific research” (Logical Biology 5:70-72, 2005). Over the years I have insisted that “Citation should credit pioneering researchers and original works” (see list of my publications referred in my article at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1973959/ ).
Under some past news in Retraction Watch I posted some comments revealing some citation problems in high-profile papers published in top journals, see examples at https://www.researchgate.net/publication/264839997_PNAS_A_Permanent_Niche_for_Aggressive_Sin
http://retractionwatch.com/2014/07/16/second-nature-paper-by-researcher-found-to-have-violated-academic-integrity-retracted/
http://retractionwatch.com/2014/04/15/nihharvard-team-loses-aging-study-to-manipulated-data/
If people Google search “citation misconduct”, many more publications and comments that I have written on this subject can be found, such as:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.604.6275&rep=rep1&type=pdf
https://www.researchgate.net/publication/241675709_I_Cannot_Believe_This_You_Shameful_-_A_Revelation_of_a_Severe_Publishing_Misconduct (full PDF: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.550.9106&rep=rep1&type=pdf )
http://www.the-scientist.com/?articles.view/articleNo/33464/title/A-Decade-of-Misconduct/
I really think it is a high time to seriously address this issue. I hope Retraction Watch will pay more attention to this kind credit robbery misconduct while still watching for data-manipulation/falsification misconduct.
Let me share my own experience in this area. As a graduate student I worked on a phenomenon originally known as Phantom Visual Imagery, later renamed Kinetic Visual Imagery (KVI), which refer to a visual sensation of movement that occurs when one moves one’s own limb repeatedly from side to side across one’s visual field with eyes shielded from light or in total darkness. The first known report (to us) in the psychological literature appeared in 1970 (Hofstetter, 1970), but Len Brosgole, one of my former professors at St. John’s, carried out some research to further verify and extend the parameters of KVI (Brosgole & Neylon, 1973) and explore it experimentally (Brosgole & Roig, 1983). To our knowledge no other research on this subject had been subsequently published until about 3 years ago when Dieter, Hu, Knill, Blake, and Tadin (2013) claimed to have made the discovery that moving one’s hand from side to side in front of one’s covered eyes causes visual sensations of motion. After reading Dieter’s paper, Len and I prepared a short paper detailing his earlier work and noting Hofstetter’s original report, but this submission was rejected. The rationale offered was that “at this stage, we still think the new study adds some important new controls and that using sharply limited journal space to give (rightful) credit to past authors is questionable”. There was no mention of a correction or of any further action on the matter. Only after I reminded the editor that a false claim of first discovery of a phenomenon that, at the time, was receiving considerable media attention qualified for a correction as per COPE guidance (i.e., item 1.7: “always be willing to publish corrections, clarifications, retractions and apologies when needed”, http://publicationethics.org/files/u2/New_Code.pdf), did information about this further step was communicated to us. While a correction, http://pss.sagepub.com/content/25/3/842.full.pdf+html was issued, we felt that it had been somewhat dismissive of the earlier work and that it failed to outright reject the claim of first discovery.
Yes, these lapses should be caught during peer review, but reviewers are human also and will miss relevant references. Heck, I know that I have missed some in my time. However, when key references are missed there is an ethical obligation by all involved to correct the record.
References
Brosgole, L. & Neylon, A. (1973). Kinetic visual imagery. Perceptual and Motor Skills, 37, 423-425.
Brosgole, L., & Roig, M. (1983). On the mechanisms underlying kinetic visual imagery: I. The role of eye movements and reafferent stimulation. Journal of Mental Imagery, 7, 57-66.
Dieter, K.C., Hu, B., Knill, D.C., Blake, R. & Tadin, D. (in press) Kinesthesis can make an invisible hand visible. Psychological Science, 23, 1013-1017,
http://pss.sagepub.com/content/25/1/66.
Hoffstetter, H. W. (1970). Some observations on phantom visual imagery. American Journal of Optometry and Archives of American Academy of Optometry, 47, 361- 366.
This is one phenomenon of “snub publishing”, a term I coined, in which a scientist is robbed of a valid opportunity to have their work cited. See Table 1, first 4 entries:
http://www.globalsciencebooks.info/Online/GSBOnline/images/2013/AAJPSB_7%28SI1%29/AAJPSB_7%28SI1%2935-37.pdf
https://www.researchgate.net/publication/283719169
I exemplified using the Anthrium literature:
https://www.researchgate.net/publication/271631101
http://link.springer.com/article/10.1007/s12109-014-9355-6
Totally agree that we should do something about it, and reviewers too to catch that before publication.
But I have seen too many journals whose instructions specifically state not more than 30-40-50 references. Should we ask them to relax these rules? Or to have an Online Supplement just for references? (but that wouldn’t make sense on paper anymore)
Readers may value this blog post I did a few years back on this topic: Seven sins of science: starting with failure to cite.
http://www.ipscell.com/2012/03/seven-sins-of-scientists-starting-with-failure-to-cite/
Whether to retract or correct is issue dependent. An omitted citation can be a careless mistake and a simple correction would be all that is needed. Omitted citations could be more egregious and warrant retraction if those citations would call into question or contradict the authors interpretation and conclusions, with the caveat that basic search could have been easily done and the title did not obfuscate the critical evidence that lies within the paper.
There is the further issue of making a citation to papers that contain contradictory evidence, but the citation is done in such a way that it suggests support. For example a paper claiming butterfly populations going extinct in a logged due to extreme climate change linked to the paper in which the extinctions were first reported but the author neglected to mention that in the same paper just 10 feet away in natural habitat the species was thriving which contradicted their claim that the extinctions were a function of climate change. I thought it should have been retracted as it was like ENRON keeping half the info off the books
http://landscapesandcycles.net/American_Meterological_Society_half-truth.html
Last year Nature published three pieces of news reporting a “credit robbery” as a result of “rushing into publication”. Dr. Zhang’s group in Tsinghua University of China submitted a paper on magneto-genetics over one month later than Peikin University’s submission of magneto-sensing paper by Dr. Xie’s group but Zhang’s paper was published online first. This was treated by some high-rank Chinese scholars as a research misconduct because they thought Zhang’s paper using a magnetic protein provided by Dr. Xie should never be published earlier than Xie’s paper even though the two papers are on different subjects. The two top universities actually sent request to the journal publishing Zhang’s paper to retract the paper.
However, after an open investigation done by Scientific Ethics which is edited by me, it was found that Dr. Zhang did not commit those alleged misconduct (http://blog.sina.com.cn/s/blog_502041670102wvho.html ). His assignment of authorship and attribution of credit are both within the normal of internationally accepted standard and practice. In the end the online version of the paper was not retracted and the print version of the paper was published normally (http://www.scibull.com:8080/EN/abstract/abstract510109.shtml ).
Ironically, Dr. Xie’s paper published later after Dr. Zhang’s paper intentionally neglected Dr. Zhang’s paper. Thus, Dr. Zhang wrote to the journal asking for a remedy of this deficiency. However, the journal refused to do anything about this.
With permission of Dr. Zhang, I am posting (part of) his communication to the journal. I welcome all kinds of opinions on this so that we can better resolved this kind of problem in the future.
Dear Editors of Journal Name,
I am puzzled with your persistence on no need for ANY citation for a quite NEW scientific term “magnetogenetics” and even more upset with your insistence on Xie’s “freedom” for deliberate ignorance of my intellectual contribution by wording an important ending paragraph of his magneto-sensing paper in a dishonest way. Xie’s behavior actually mounts up to a plagiarism: the practice of taking someone else’s work or ideas and passing them off as one’s own. This is a typical misconduct as defined by the ORI (Office of Research Integrity). Stopping this misconduct should be an intrinsic responsibility of your journal and refusing my request for doing so may bring severe liability to your journal.
As to your assessment on Dr. Liu and Scientific Ethics I can only say that your knowledge on him and on Scientific Ethics is very limited. If you Google for him and that journal you will easily find that Dr. Liu has published many articles on various aspects of scientific ethics, mostly in Scientific Ethics, which is the only journal in this world that is dedicated to publications on scientific ethics.
Scientific Ethics was established in 2006. Its publications were cited in peer-reviewed papers and even in Editorials of well-established journals such as J. Cell Biology and J. Exp. Medicine.
I am providing you some links to papers that have cited Dr. Liu’s publications on scientific ethics in Scientific Ethics. Please note an acknowledgement of Dr. Liu’s publication was added into an Editorial after the authors who are editors of a respectful journal found Dr. Liu had published a relevant paper earlier.
Again, the demand for asking Dr. Xie to give citation to a relatively new and thus unfamiliar scientific term is a normal requirement by internationally accepted standard. The plea for your journal to inform Dr. Xie to include a Note-Added-in-Proof for referring our published work is well justified because it is our paper that directly related to the paragraph that Dr. Xie added, after he learnt the importance of magnetogenetics from me.
So please show your scientific integrity by performing an ethical duty of asking the author of your journal to PROPERLY citing RELEVANT publication(s).
xxxxxxxxx
Sincerely yours,
Sheng-Jia Zhang
P.S.
http://iai.asm.org/content/79/10/3855.full
http://iai.asm.org/content/79/10/3855.full.pdf+html
ADDENDUM IN PROOF
It has recently been brought to our attention that previous independent analyses have also concluded that articles in journals with higher impact factors are more likely to be retracted (Liu, S. V. Top journals’ top retraction rates. Sci. Ethics 1:91–93, 2006; Cokol, M., I. lossifov, R. Rodriguez-Esteban, and A. Rzhetsky. How many scientific papers should be retracted? EMBO Rep. 8:422–423, 2007).
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1973959/
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1973959/pdf/7401053.pdf
EMBO Rep. 2007 Sep; 8(9): 792–793.
doi: 10.1038/sj.embor.7401053
And in the end most of these magnetogenetics papers turned out to contradict the laws of physics:
https://elifesciences.org/articles/17210
Thanks so much for appending that article from 1980. It bears reading today and perhaps always. I am a retired librarian who worked in the humanities (specifically, music studies) and it galled me how over the course of my career topic searching and bibliographic checking have deteriorated so markedly. In the humanities, some of them at least (music being one), English is not the only language of importance for the conveyance of original research. In fact, it plays second place to German in that regard, then and now. I find it galling that younger researchers AND THEIR PROFESSORS, who increasingly lack the language proficiency common in the past, ignore the literature, very rich, that exists in non-English, non-French languages. So much is unnecessarily repeated from ignorance of what exists in other languages. The lack of the intricate and highly developed citation sources which one finds in the sciences only aggravates the situation.
Jerry – you could make the same comment about “foreign” disciplines. Some disciplines, particularly in the social sciences, have a habit of ignoring previous findings on a similar subject if they were published in a different disciplinary journal.
One of the risks of the predatory open access journal movement is that “lazy” scientists will just go to one or two data-bases, like Scopus, or Google Scholar, and input their study’s key words, and then pick out the most relevant studies that appear on the first few pages, so studies that are buried a few pages deep will get ignored, or lost. In some ways, it is also a function of the sheer volume of information that exists at our disposal now. I am not sure if I agree with Bert Own above who places the blame squarely on the shoulders of reviewers. Bert should remember that most reviewers work freely, so to expect them to be the exclusive gate-keepers of a journals quality, in terms of references, is unreasonable. As I see it, the responsibility lies squarely on the shoulders of the authors. Whenever I write a paper, especially in more recent years, I tend to use at least 7 or 8 data-bases. Even then, I sometimes feel that it is not adequate enough. In some cases, working with plants where the nomenclature can change as the taxonomy evolves, it is very often possible that some studies are missed, simply because the name of a plant has changed. And that is why, at least in plant science, plant scientists have to be aware of, and use, free tools and lists such as The Plant List, etc., and multiple data-bases, to ensure that what they report represents the widest possible balanced representation of the literature. To date, 100% of my complaints about manipulated, imbalanced, skewed, or biased reference lists have fallen on deaf ears: editors simply don’t care, in most plant science journals, about correcting this very basic problem.
Allowing reference lists to remain problematic is literally the legitimization of false and inaccurate information. It must be corrected, like any other error in data, plagiarism, duplication, etc.
The inherent problem I see with flagging these issues (on pubpeer or elsewhere) is that the only people who are likely to notice the omissions almost certainly have a conflict of interest. That doesn’t mean the issue isn’t real, but I worry it comes off as petty.
(full disclosure: I am amongst those slighted by a major research group and have had multiple debates about this exact issue).
I the summer of 1984 my paper on the cloning of the human beta-actin gene was in press for MCB publication in October. I was visited by a competing Japanese scientist from Osaka. I told him about how I cloned the gene using homologous recombination. I gave him a copy of the page proofs which obviously told him how to clone the gene by a more straight-forward method. A year later his paper showed up in PNAS on cloning of this gene fraught with errors and without mentioning my paper. My co-author at Stanford and I wrote the member of the Academy who sponsored the paper to PNAS explaining what happened.
The question that begins this section is about articles that fail to cite prior works on the same subject which reached similar results. That is an issue. But no one has addressed a more worrisome (to me at least) problem. And that is the case of citations to works that improperly describe the results of those earlier papers so as to make the results of the current work look more acceptable. I have seen this type of behavior several times in my field of economics. Because these cites are almost always in the literature review, they tend to be treated as secondary and I certainly do not believe that editors would run a correct for misleading citations. Has anyone encountered this type of behavior?
I have a simple approach to all my work with references – regardless of whether I am an author, reader or reviewer – if the references don’t support the actual statement made it’s b*ll*cks. If it’s an article I read it simple wont get referenced, if it’s a manuscript under review I point it out and if the editors and authors don’t do anything about it the journal and authors end up in my less trusted pile – some journals has managed to publish so many of these manuscripts that they get a not-so-respectful answer on requests for reviewing and never appear as referenced in my own papers. The accuracy of referencing is mainly the responsibility of the authors, but reviewers and editors letting people publish statements unsupported by the references (or the original work) share in the blame.
In very few instances lack of referencing is a problem – as pointed out above, it is almost impossible to have a grip on the amount of information out there today and even if I get a bit ruffled reviewing manuscript not citing my pivotal work (as small as it is) – if they cite relevant work or put forward original research supporting their statements who am I to not acknowledge that it is mainly a facet of egoism? I do admit that for more experimental and method development based fields there are papers that simply cannot be ignored – and when done there is a great liability resting on reviewers and editors.
Reaction to thesis plagiarism, discovered years after degrees have been granted, is not the same from universities in the Spanish speaking world as from universities in other countries such as Germany, The United States of America and Hungary.
It is well documented that since 2012 two ministers from Germany, one senator from Montana, and even one president from Hungary had their degrees revoked from their respective universities after it came to light that they presented as their own stolen academic material in their thesis.
In Spain and in the Spanish speaking countries of Latin America is different, most universities from this region turn a blind eye to thesis plagiarism.
One example is in Guatemala, the University of San Carlos learned from the political magazine Contrapoder that then presidential candidate Manuel Baldizón plagiarized his PhD thesis. Here’s a link to the report (all links are in Spanish):
http://contrapoder.com.gt/2014/09/07/la-tesis-tambien-doctor/
As a result the university assembled a council to decide on how to proceed. After deliberating for over 60 days, and without even looking at the evidence!, they decided that due to the lack of regulations regarding plagiarism discovered after degrees are granted, there was nothing that they could do, therefore the PhD was upheld and valid. Here’s the report on the university’s decision, also by Contrapoder:
http://contrapoder.newscoop.pro/es/edicion49/actualidad/1203/La-Usac-valida-el-doctorado-de-Baldiz%C3%B3n.htm
That’s a high-profile case. Another high profile one is Boris Berenzon’s from the National Autonomous University of Mexico. He was fired from his professor post at the university due to “inadequate academic practices”. Not only did he plagiarized his Master’s and Doctorate’s thesis but also books and dissertations he presented in international forums. He was fired but his degrees, from that same university, have not been revoked.
Here’s a blog that criticizes the whole academic environment that allowed Berenzon to cheat for so many years:
http://yoquierountrabajocomoeldeboris.blogspot.com/2014/08/un-ejercito-de-mimos-aniversario-de-la.html
My own short thesis was plagiarized for their respective thesis by two grad students from the Autonomous University of Nuevo Leon (UANL) in Monterrey. That university apologized to me in writing, recognizing that in fact their former students did stole from my short thesis. They also retracted one of the two thesis from their online library. But they did not revoke either one of the degrees. Here’s the apology letter:
http://www.mundo-digital.info/about_sergio2014/CartaSergioCastro.pdf
And here’s proof that Noé Torres-Garza, one of the UANL alumnus that plagiarized my work, still holds his degree: http://cedula.buholegal.com/3323990/
Back in 2013 the University of Valencia in Spain came to the same conclusion that the University of San Carlos from Guatemala did, except that this university did not even form a council:
http://www.levante-emv.com/comunitat-valenciana/2013/11/21/universitat-carece-mecanismos-
investigar-plagio/1053020.html
There are many other cases like this through the region, the latest high profile case in undergoing in Peru, one of the leading presidential candidates, Mr. César Acuña, is being investigated for plagiarizing two Master’s thesis, one from a university in his country and another from Colombia, plus the thesis that granted him a PhD from a University in Madrid, Spain. Here’s a link to the latter case:
https://puntodevistaypropuesta.wordpress.com/2016/01/31/por-escandaloso-plagio-rector-de-la-
universidad-complutense-anuncia-investigacion-diligente/
Honorable exceptions are the Central University of Ecuador and the Colegio de México. They both revoked Phd degrees after discovering years later the academic fraud of thesis plagiarism. This in 1997 for Ecuador and 2015 for Mexico. Here’s a link to the case from Colegio de México:
http://www.eluniversal.com.mx/articulo/cultura/letras/2015/07/10/colmex-retira-grado-doctor-por-plagio