A journal on adolescent issues has retracted a pair of papers linking exposure to violent media to aggressive behavior in youth after critics questioned the validity of the data.
The studies, which appeared in Youth & Society, were led by Qian Zhang, of Southwest University in Chongqing, China and were published in 2018.
According to the retraction notice, which covers both “Short-term exposure to movie violence and implicit aggression during adolescence,” and “The priming effect of violent game play on aggression among adolescents”:
Several independent researchers contacted the journal with concerns about the accuracy of the data reported in the above-listed manuscripts. We thank Dr. Joe Hilgard of Illinois State University for his time and effort for identifying this issue and bringing it to our attention. After review of the identified discrepancies and completing our own analysis, the editors were convinced that the data discrepancies rendered this manuscript too flawed to correct. We are therefore, retracting the manuscripts as reporting unreliable findings.
The priming paper also had a correction, now moot:
The authors regret that there are some errors in the above-mentioned article on page numbers 5, 6, 7, 8, 9 (including Table 1), and 10. We have incorporated the corrections and the updated version is online now.
Hilgard told us that:
I read one of the papers back in early 2018, saw that lots of the numbers didn’t make sense, and contacted the first author. The first author said he was going to correct it, and in May 2018 I followed up with the journal to make sure they did. The correction dated November 2018 was a mess: lots of numbers still didn’t make any sense, and it seems that F-statistics that had been incorrectly reported as statistically significant became statistically significant by the addition of a missing tens digit.
Earlier this fall, he contacted the journal again:
On Sept 26, 2019 I approached the journal with further concerns. I’d received raw data from one paper (Zhang, Espelage, & Rost, 2018), and certain properties of that raw data seemed most unusual. Additionally, I found that the means reported in that article were very similar to those reported in two experiments in Scientific Reports in 2013. This was particularly surprising given that these three experiments featured different populations and measures.
I feel like Youth and Society did a good job in good speed, and I appreciate their response. Of course, I feel like these papers were pretty obviously flawed, which probably helped. Because one never knows just how serious and how extensively cataloged the errors must be to motivate correction, I spent a fair amount of time organizing and documenting the problems.
‘A pattern consistent’
Hilgard wasn’t the only research to raise questions about the work. Shortly after publication, the papers drew the attention of Arlin James Benjamin Jr., of the University of Arkansas–Ft. Smith, who studies human aggression. Benjamin took to Twitter to point out various “inconsistencies” in the articles. These problems included reported data that didn’t match what he found when he ran the results through Statcheck and bizarre standard deviations:
Benjamin said:
These papers fit a pattern consistent with other published reports from Zhang and colleagues in which there are reporting errors, tables that make no sense, and reported methodology that at the least would raise eyebrows. …
One real concern with media violence research is whether or not the violent and nonviolent stimuli (whether film clips, video games, etc.) are really equivalent in terms of interest, difficulty, etc. If the authors cannot establish that is the case, any subsequent experiments using those stimuli would be ones I would interpret very cautiously.
For more of his thoughts about the studies, check out his blog.
Readers of this blog might recall that Hilgard and Benjamin have crossed paths before. In 2017, Benjamin corrected a paper on which he was a co-author — which claimed that seeing a weapon could increase aggression — after Hilgard raised questions about the data.
One of Benjamin’s co-authors on the weapons paper was Brad Bushman, of The Ohio State University. Bushman’s group has retracted two papers which suggested that violent media can harm children.
That effect is unproven. But what is clear is that criticism of research into violent media can be treacherous to scholars. As we reported last year, two scientists found themselves under investigation by Ohio State after they challenged Bushman’s findings — even though their criticisms were valid.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].