We all make mistakes – but when it comes to the scientific literature, too many authors are making critical mistakes in their list of references, making it difficult for readers to retrieve a cited paper. We spoke with Marilyn Oermann, the Thelma M. Ingles Professor of Nursing at the Duke University School of Nursing, who has studied this problem extensively in the nursing literature.
Retraction Watch: You’ve published multiple papers looking at reference problems in nursing research. What are the main types of “reference problems” that usually occur?
Marilyn Oermann: Errors in references can be major or minor. A major error is one that prevents retrieving the cited article or other publication or makes it difficult to retrieve it. For example, including the wrong journal title or volume number or misspelling the first author’s name would be major errors. Some reference errors, though, are minor — such as misspelling a coauthor’s name or having a punctuation error. Minor errors do not affect retrieving the cited material.
In some of the clinical specialty nursing journals we analyzed, errors in author names were common. However, we later reanalyzed the references across all of our studies. There were 7,650 references, and we randomly selected 10% of the references from each journal to check their accuracy with the original publication. Of 765 references, the most common error was in the title of an article, such as missing or having incorrect words in the title (e.g., substituting tool for instrument). In a study of pediatric orthopedic literature, Davids et al. (2010) also found that errors in the article title were the most common.
RW: What are the overall rates of reference problems you’ve seen in the nursing papers you’ve examined?
MO: In our studies of nursing journals, 28.4% of the references had errors.
RW: I would expect that some researchers think of reference mistakes as simple copyediting errors. But what is the potential impact of reference problems?
MO: If an error in a reference limits the ability to find the cited article, or makes it more time consuming to locate it, many readers will not bother to continue to search for it. The articles that readers are not able to find might be particularly relevant to studies they are doing or to clinical questions they need to answer for their practice. As an author, if your name is misspelled in a reference, it is less likely other authors will find and then cite your paper, affecting not only the citations of your work but also the impact factor of the journal in which it is published. As an editor, in my opinion, errors in references cast doubt on the quality of the journal.
RW: To what extent are reference problems the result of deliberate intent or simple ignorance? Do you think scientists are getting adequate training in proper citations and referencing?
MO: I do not think errors in references are deliberate but instead are the result of copying and pasting reference information into manuscripts and not checking its accuracy. Most of our studies of the nursing literature were done prior to widespread use of reference management software such as EndNote. Studies on reference accuracy need to be replicated to determine if errors are less common now considering there is greater use of reference management software. However, even with this software, you still need to be sure full and accurate information is being inputted into it. You should double check that references generated by reference management software are properly formatted, e.g., page numbers and doi’s are not left off when they should be present.
RW: Since your research is largely confined to nursing research, to what extent do you suspect these results apply to other fields?
MO: A Cochrane review published in 2008, around the same time many of our studies were done, found a median reference error rate of 38%— more than 1/3 of the references cited in articles in biomedical journals had an error. In Davids et al. (2010) study, the citation error rate across the journals they reviewed was 26%, which is consistent with our nursing journal studies.
RW: Do you suspect that journals with relatively high impact factors – say 10 and up – may be more exacting in their reference requirements?
MO: In the 2010 study of the pediatric orthopedic literature, there was a significant inverse correlation between the impact factor of the journal and reference error rate. The authors suggested that in high impact journals, more attention may be given to the accuracy of references by authors, editors, and staff.
RW: How often do you suspect these references can reasonably be assigned to typesetting or publisher issues?
MO: I do not think errors in references are from typesetting or publisher issues. These errors are made prior to that phase in the process. In my own experience as an author and editor, many of the queries authors receive on their page proofs relate to errors to correct or [add?] missing information in the references.
RW: Does the expansion of internet resources make it easier for researchers to get references right?
MO: I think the expansion of internet resources has created more reference problems. Some authors do not check the quality of information at websites they include in their papers, and the information at the website can change rapidly. On the reference list, the URLs may no longer be available; the author may include the URL to the main page but not to the actual document referred to in the article; and there may be missing or incorrect information with the reference.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
Very interesting, many thanks for posting. As a researcher, something else I’ve noticed is that many references don’t actually contain the information the citing paper claim that they do — which is a continuous source of disbelief and frustration. Would have been nice to see some numbers on this, but I understand that it might be more labour-intensive to retrieve such statistics. But it’s a reminder that you should always confirm that a paper says what you think it says before citing it in your own paper.
I’ve found that a lot also. It usually is easiest to discover when one’s own papers are misrepresented (duh!), but I’ve also had a few cases where the title of the paper that was cited was almost a dead give-away that the citation did not support the claim.
Another problem is that the paper in question may contain the information, but actually may not support that information. An example from one of my own papers is an equation that is commonly used, but so problematic for many reasons that I and my co-author argue against using it. You can probably already guess what has happened: several people have cited our paper when they use this equation, without considering any of the pitfalls we note. Britta Stordahl relates a similar story in Colloids and surfaces B: Biointerfaces 72(2):312 (2009), noting that some cited one of her papers in favor of a statement that the paper actually set out to disprove.
I find mistakes in the title and author names relatively trivial to track down what is most likely the correct citation – Google Scholar is pretty good at suggesting close alternatives. Irritating, but not a major error. Far more problematic is citing secondary material as though it were the source, easily noted when the citation is to a review but less obvious when simply trotting out a section from an Introduction section. Like Marco, I often have a chuckle when looking at citations to my paper. “They got that from that paper?!” I suspect citation lists are frequently developed without reading the abstract, and in some cases even the title.
Marilyn, an important issue that I have been trying to highlight for quite some time now, especially in he plant science literature, where the situation is unbearable, and getting worse by the day. What is lacking, however, among editors and publishers, is a consensus of the scale of the error, and if and how those errors should be addressed, and corrected.
More on this here (sorry, pay-walled):
Teixeira da Silva, J.A. (2016) An error is an error… is an erratum. The ethics of not correcting errors in the science literature. Publishing Research Quarterly 32(3): 220-226.
http://link.springer.com/article/10.1007/s12109-016-9469-0
DOI: 10.1007/s12109-016-9469-0
Hi Marilyn, Very interesting read. Can’t help thinking that many problems would be solved by encouraging researchers to use Reference Management Software [RMS]. I don’t want to advertise products here but there are good free and paid for versions. If authors check they have a DOI, where its available, the software will pull the correct information from the publishers website with no added effort, not to mention the benefits that RMS offers. BW Matt
I used to often cite a reference from a journal that repaginated every issue, so the reference had to include both the volume and the issue, not just the volume. Copy editors were constantly taking out the issue, so I resorted to putting a ‘NOTE TO COPY EDITOR’ embedded in the reference so they would leave the issue alone. In my experience, copy editors tend to cause reference problems also by making assumptions that are wrong. For instance they may think that version 3 of a software package is the ‘third edition’ and change the reference accordingly. As a reviewer, I always check reference lists and find quite a few errors, so I suspect a lot of people are not using reference software and don’t even know that you can easily download the accurate text of the reference from PubMed yourself even if you are not using software.
In my long experience as a journals manuscript editor, the meaningful editing of reference lists has come to lie somewhere between strongly discouraged and forbidden, not that that will stop someone who takes signing off on a job personally. I’m aware of one large journals family that has switched to “pre-editing” of references, which is to say, outsourcing it to the same outfit that does the “typesetting” (I work with a lot of heavily mathematical content) overseas. This is little more than doing enough formatting so that a script can pattern-match to some DOI.
Anyway, Helmut Abt, former EIC of the Astrophysical Journal, published several papers on errant citations and their propagation in this field back in the ’80s and thereabouts. I could try to dig them out of ADS if anyone is interested.
The problem is also often compounded by citation mismatching in the scientific databases. The problem is worse with all these new open access journals that do not provide traditional citation information, but only one single article ID. These are often miscited as “page 1-X”, when the article ID should be provided instead (Biomedcentral and Hindawi are the worst in this respect).
I highly recommend the article:
Cite with a Sight
Prashant Kamat and George C. Schatz
The Journal of Physical Chemistry Letters 2014 5 (7), 1241-1242
DOI: 10.1021/jz500430j
Including guideline how to change WoS records: http://pubs.acs.org/doi/suppl/10.1021/jz500430j/suppl_file/jz500430j_si_001.pdf
In retrospect, it’s a very strange design choice (and didn’t begin with OA). The idea, when originally trotted out, was to facilitate “electronic-first” publishing; see, e.g., Phys. Rev. B. There’s a lot of bloviating in that announcement, but the core motivation was to reduce the time from acceptance to final publication.
Yet decorations such as volume and even issue numbers were retained. It’s not exactly a hard problem to simply add sequential pagination in the order the papers are going to be published, unless one’s production department is extraordinarily disorganized.
The net effect seems to have just been eliminating the need to construct tables of contents based on semantics. This is a nontrivial task, but it’s hardly insurmountable. After all, publishers are at the forefront of the relevant technomonology, right?
In some cases authors are also more to blame than reference software: once I was reading a paper about fibrin and followed up on a reference, only to find that the referenced paper was about fibroin, an entirely different material…. suggesting that the authors hadn’t even read the abstract of the paper they so confidently cited.
I have seen my own papers cited at wrong place! but I keep quiet, not knowing what to do about it, once the paper is published.
What I could not understand is why would journals not adopt one style and save authors the hassle and the waste of time in references reformatting?
Interesting read. Reference errors are indeed really common, and I can’t say that my own papers have been immune to them. Making the inclusion of DOI mandatory – as some journals do is a step in the right direction.
I found a nice one recently: https://www.mdpi.com/2071-1050/15/7/5863#B50-sustainability-15-05863 ; many of the references are just not in the right place. It is amazing how the editor missed this. I can understand a few mistakes, but this one, after 5 minutes I gave up. Almost all of the references seem to be misplaced.