Archive for the ‘psychology’ Category
Making error detection easier – and more automated: A guest post from the co-developer of “statcheck”
We’re pleased to present a guest post from Michèle B. Nuijten, a PhD student at Tilburg University who helped develop a program called “statcheck,” which automatically spots statistical mistakes in psychology papers, making it significantly easier to find flaws. Nuijten writes about how such a program came about, and its implications for other fields.
Readers of Retraction Watch know that the literature contains way too many errors – to a great extent, as some research suggests, in my field of psychology. And there is evidence that problem is only likely to get worse.
To reliably investigate these claims, we wanted to study reporting inconsistencies at a large scale. However, extracting statistical results from papers and recalculating the p-values is not only very tedious, it also takes a LOT of time.
So we created a program known as “statcheck” to do the checking for us, by automatically extracting statistics from papers and recalculating p-values. Unfortunately, we recently found that our suspicions were correct: Half of the papers in psychology contain at least one statistical reporting inconsistency, and one in eight papers contain an inconsistency that might have affected the statistical conclusion.
The origins of statcheck began in 2011, Read the rest of this entry »
Following questions about the veracity of multiple papers by his former employer, high-profile social psychologist Jens Förster has agreed to retract two papers as part of a deal with the German Society for Psychology (DGPs).
Last year, Förster had a paper retracted at the request of his former employer, the University of Amsterdam (UvA). In May, an investigation commissioned by UvA found that many of his experiments looked “too good to be true,” and eight papers showed strong signs of “low veracity.”
Just two of those papers are acknowledged in the settlement of a case by the DGPs against Förster, who currently works at Ruhr University Bochum. Here’s a translation of a notice from the DGPs from One Hour Translation:
A study that found a 15-fold increase in the rate of sexual trauma among men in the U.S. military — and sparked suggestions of “an epidemic of male-on-male sex crimes” in the military among conservative media outlets — has been retracted because of a flaw in the analysis.
The study, published just last week, appeared in Psychological Services, an American Psychological Association (APA) journal. In an announcement Sunday titled “American Psychological Association Retracts Article Positing Excessively High Rates of Sexual Trauma Among Military Men,” the APA said that “Scholars raised valid concerns regarding the design and statistical analysis which compromise the findings.” Here’s the text: Read the rest of this entry »
A paper published in August that caught the media’s eye for concluding that feeling sad influences how you see colors has been retracted, after the authors identified problems that undermined their findings.
The authors explain the problems in a detailed retraction note released today by Psychological Science. They note that they found sadness influenced how people see blues and yellows but not reds and greens, but they needed to compare those findings to each other in order to prove the validity of the conclusion. And once they performed that additional test, the conclusion no longer held up.
In the retraction note for “Sadness impairs color perception,” the editor reinforces that there was no foul play:
A disability journal is “paying significant attention” to papers authored by Anna Stubblefield, a former Rutgers researcher recently convicted of sexually assaulting a disabled man who participated in her research.
Stubblefield was convicted of sexually assaulting “DJ,” a man in his thirties with cerebral palsy who was “declared by the state to have the mental capacity of a toddler,” according to a lengthy piece in the New York Times. Stubblefield and DJ published papers in Disability Studies Quarterly; in one, Stubblefield describes a controversial technique which she claimed helped DJ communicate. But when she eventually used the technique to say DJ was in love with her, his family took her to court, and she was convicted of aggravated sexual assault.
In the original paper, the authors claimed that three out of eight patients who underwent a procedure that used gamma rays to kill brain cells showed improvements 12 months later (versus zero in the group who underwent a “sham” procedure). But after a reader noticed an “inadvertent” error in the calculation of how many patients had improved, the authors realized that only two of the patients had responded meaningfully to the procedure.
The new results “did not reach statistical significance,” the authors write in a “Notice of Retraction and Replacement.” JAMA Psychiatry published it yesterday, along with a new version of the article, a letter from psychiatrist Christopher Baethge pointing out the error, and an editorial. The original article is available in the supplemental material of the new version, with the errors highlighted.
Here’s the note in full for “Gamma ventral capsulotomy for obsessive-compulsive disorder: a randomized clinical trial,” which explains the error:
A study that looked at how entrepreneurs’ confidence levels change depending on market conditions has been corrected to fix an error that flipped the results of one of the experiments.
The paper was published in 2013 by the Strategic Management Journal, and explored how entrepreneurs stay confident in difficult marketplaces by studying how people reacted to tasks of varying difficulty. In one experiment, participants were asked how well they thought they did on an easy quiz and how well they did on a hard quiz. Results showed that “participants underestimated their scores on the easy quiz” and “overestimated their performance on the difficult quiz.” However, authors wrote the opposite in the final paper.
Here’s the correction notice for “Making Sense of Overconfidence in Market Entry”:
The Cochrane Library has withdrawn a criticized 2014 meta-analysis about a technique to help young people avoid alcohol abuse, because of “major errors.”
The review found that motivational interviewing, a form of counseling to help people change behaviors, showed some effects but had “no substantive, meaningful benefits” in preventing alcohol abuse among people 25 and younger. However, other researchers in the field, including some whose studies were included in the analysis, soon raised concerns about the review’s methods and data calculation, and the authors withdrew it.
Here’s the brief notice for “Motivational interviewing for alcohol misuse in young adults:”
We’ve learned about two more retractions we missed for Diederick Stapel, the Dutch social psychology researcher who has now racked up a total of 57 retractions by our count.
Both retractions were issued after a committee released a report which established fraud in dozens of papers co-authored by Stapel.
Stapel is still #4 on our leaderboard.
A correction to a 2011 paper doesn’t change its main conclusion: Hearing song lyrics about violence — “let the bodies hit the floor,” for example — can prompt aggressive behavior, even more so than violent imagery in music videos.
The correction follows an investigation by Macquarie University that found errors in data analysis to be an “honest mistake.”
During the study — “The effect of auditory versus visual violent media exposure on aggressive behaviour: The role of song lyrics, video clips and musical tone,” published in the Journal of Experimental Social Psychology — the authors measured the effect of violent songs or imagery using the “hot sauce paradigm.” In this model, researchers estimate people’s level of aggression by how much hot sauce they give another person to eat. The study found that, indeed, people who are exposed to violence — particularly, lyrics — give more hot sauce to their neighbors. It has been cited 6 times, according to Thomson Scientific’s Web of Knowledge.
One aspect of the paper prompted a tweet from a Dutch journalist: Read the rest of this entry »