Category: reference problems

Following our coverage this summer of a book with citations that did not exist, we asked you to send us examples of other books with similar issues. One reader took the request as an assignment to find problematic texts.

Michał Wójcik, a Ph.D. student at the Free University of Berlin, saw a link to our article about the book on LinkedIn. “I started thinking that it shouldn’t be that hard to check those references automatically,” he said. “I decided to just spend some time on it, and I had a prototype in a few hours.” 

The Python script he wrote searches through books to verify the existence of each citation by checking if the DOI existed in Crossref. He told us he manually checked citations the script couldn’t identify by looking in other databases and searching on Google Scholar, which takes him between an afternoon and a whole day. 

A dean at an Australian university sought to correct some of his papers. He received a retraction instead.

We wrote last year about Marcel Dinger, dean of science at the University of Sydney, who was a coauthor on five papers with multiple references that had been retracted. In May 2024, Alexander Magazinov, a scientific sleuth and software engineer based in Kazakhstan, had flagged the papers on PubPeer for “references of questionable reliability.” Magazinov credited the Problematic Paper Screener with helping him find them. 

Dinger told us at the time he intended to work with editors to determine whether the five papers should be corrected or retracted.  

The dean of science at the University of Sydney is reassessing a series of review papers after commenters on PubPeer pointed out each cited several retracted articles, Retraction Watch has learned.

Marcel Dinger and his coauthors will submit addendums to the journals noting the retracted references, he told Retraction Watch, and work with editors to determine whether the reviews should be retracted. 

Dinger, who also is a professor of genome biology, is a middle author of four review articles and last author on one more that sleuths using the Problematic Paper Screener flagged as referencing retracted articles. The articles have been cited nearly 100 times altogether, according to Clarivate’s Web of Science. 

Guillaume Cabanac, who developed the screener, commented on one of the papers in August 2022: 

An August 2023 report on research integrity by consulting firm KPMG, commissioned by an Australian government agency, contains a made-up reference, Retraction Watch has discovered.

Reference 139 of the report, “International Research Integrity Policy Scan Final Report: Compilation of information about research integrity arrangements outside Australia,” reads:

Gunsalus CK, Marcus AR, Oransky I, Stern JM. Institutional and individual factors that promote research integrity. In: Macrina FL, editor. Scientific Integrity: Text and Cases in Responsible Conduct of Research. 4th ed. Washington, DC: ASM Press; 2018. p. 53-82. 

A book with that title exists, but the four authors listed did not contribute a chapter, and the 2018 edition does not appear to contain a chapter with that title. We – Adam Marcus and Ivan Oransky – have indeed published with CK Gunsalus, but nothing resembling this reference.

A preprint about millipedes that was written using OpenAI’s chatbot ChatGPT is back online after being withdrawn for including made-up references, Retraction Watch has learned. 

The paper, fake references and all, is also under review by a journal specializing in tropical insects.

“This undermines trust in the scientific literature,” said Henrik Enghoff, a millipede researcher at the Natural History Museum of Denmark, in Copenhagen, who first spotted problems with the preprint, as we reported last month.