Category: LLM

A citation alert led researchers to a network of fake articles. But who is benefiting?

Demianastur/iStock

A few months ago, when Elle O’Brien, a data scientist at the University of Michigan, was checking who had recently cited her work on Google Scholar, she came across something that would take her and her colleagues down “a rabbit hole.” 

When O’Brien opened a publication that had recently cited her, it appeared to be a rewritten version of an arXiv preprint she had co-authored with two colleagues, Grischa Liebel and Sebastian Baltes. Yet this did not seem to be a simple case of theft by other academics. 

For starters, the six authors listed on the fake article didn’t exist, although three had been given the same institutional affiliations as O’Brien, Liebel, and Baltes: the University of Michigan, Reykjavik University and Heidelberg University, respectively. The similarities in the texts read as if someone had typed, “ChatGPT, please rephrase this paper without changing anything else,” Liebel wrote in a post on LinkedIn. But why would fake authors need publications?

Continue reading A citation alert led researchers to a network of fake articles. But who is benefiting?

Librarian finds ‘preposterous number’ of fake references in paper from Springer Nature journal

Gunnar Ridderström/Pexels

As a hospital librarian, Jessica Waite is typically successful at tracking down elusive articles for clinicians at Royal Hallamshire Hospital in England. So when a colleague couldn’t locate two references in a paper and asked for help, the librarian grew suspicious.

“These were recent references, which usually we have no problem finding,” Waite told us. “I looked at the issues of the journals where the article should have been, and there were completely different articles, so I immediately thought that the articles we had been asked to find were not real.”

The references were from an article exploring mental health integration after bowel diversion surgery published in Digestive Diseases and Sciences (DDS), a Springer Nature title. 

Continue reading Librarian finds ‘preposterous number’ of fake references in paper from Springer Nature journal

Correction to a retraction highlights tortured phrases have been around longer than LLMs

Corrections to retractions have also been around longer than AI tools like the one that created this image. DALL-E

While large language models are taking the blame for hallucinations, punctuation and all manner of language choices these days, turns of phrase were being tortured well before the arrival of LLMs.

Overlooking that fact seems to have led to a recent correction to a retraction – yes, you read that right – in Sage’s Journal of X-Ray Science and Technology. The original article, published in February 2022, was on detecting coronary artery plaques. It contained several known tortured phrases, synonyms and rephrasings — often awkward and nonsensical — substituted in text to evade plagiarism detectors.

For instance, the paper used the term “cardiovascular breakdown” for “heart failure”; “outward appearance acknowledgement” instead of “face recognition”; and “attractive resonance” for “magnetic resonance.” 

Continue reading Correction to a retraction highlights tortured phrases have been around longer than LLMs

Guest post: Forget pickles and ice cream. I published a fake paper on pregnancy cravings for prime numbers

Image generated by Google Gemini

I had grown weary of the constant stream and abuse of spam invitations to submit manuscripts to journals and to attend fake conferences on the other side of the world, a trend extensively studied in academia. The last straw: a solicitation from the Clinical Journal of Obstetrics and Gynecology, well outside my work in mathematics education.

Accepting the challenge, I decided to submit a deliberately nonsensical, AI-generated manuscript in response to observe how the individuals behind these supposed journals operate.

In October 2025, I wrote to someone named Henry Jackson, who had sent the article invitation in August (despite the fact that no such person is listed on the journal’s website). I sent a manuscript generated entirely by ChatGPT to test how far a publication created with zero genuine effort could go and whether there was any filtering mechanism in place to prevent a meaningless article from being published. 

Continue reading Guest post: Forget pickles and ice cream. I published a fake paper on pregnancy cravings for prime numbers

AI-Reddit study leader gets warning as ethics committee moves to ‘stricter review process’

University of Zurich

The university ethics committee that reviewed a controversial study that deployed AI-generated posts on a Reddit forum made recommendations the researchers did not heed, Retraction Watch has learned. 

The principal investigator on the study has received a formal warning, and the university’s ethics committees will implement a more rigorous review process for future studies, a university official said.

As we reported yesterday, researchers at the University of Zurich tested whether a large language model, or LLM, can persuade people to change their minds by posting messages on the Reddit subforum r/ChangeMyView (CMV). The moderators of the forum notified the subreddit about the study and their interactions with the researchers in a post published April 26. 

Continue reading AI-Reddit study leader gets warning as ethics committee moves to ‘stricter review process’

Experiment using AI-generated posts on Reddit draws fire for ethics concerns

Note: We’ve published a new story with the University of Zurich’s response, as well as comments from Reddit’s chief legal officer.

An experiment deploying AI-generated messages on a Reddit subforum has drawn criticism for, among other critiques, a lack of informed consent from unknowing participants in the community. 

The university overseeing the research is standing by its approval of the study, but has indicated the principal investigator has received a warning for the project. 

The subreddit, r/ChangeMyView (CMV), invites people to post a viewpoint or opinion to invite conversation from different perspectives. Its extensive rules are intended to keep discussions civil. 

Continue reading Experiment using AI-generated posts on Reddit draws fire for ethics concerns