At the beginning of November, Science issued an “editorial expression of concern” over a 2011 paper by the disgraced Dutch social psychologist Diederik Stapel, in the wake of an announcement by his former employer Tilburg University, that it had found evidence of fraud in Stapel’s body of work.
A month later, Science has gone the extra step, publishing a retraction notice by Stapel and his co-author, Siegwart Lindenberg. The notice, dated Dec. 1, 2011, makes it clear that Stapel acted alone in the matter:
Our Report “Coping with chaos: How disordered contexts promote stereotyping and discrimination” (1) reported the effects of the physical environment on human stereotyping and discriminatory behavior. On 31 October 2011, the University of Tilburg held a press conference to announce findings of their investigation into possible data fraud on the part of author Stapel. These findings of the university’s interim report (2) included fabrication of data in this Science paper. Therefore, we are retracting the paper, with apologies from author Stapel. Coauthor Lindenberg was in no way involved in the generation of the data, and agrees to the retraction of the paper.
The nut of the now-retracted article?
Being the victim of discrimination can have serious negative health- and quality-of-life-related consequences. Yet, could being discriminated against depend on such seemingly trivial matters as garbage on the streets? In this study, we show, in two field experiments, that disordered contexts (such as litter or a broken-up sidewalk and an abandoned bicycle) indeed promote stereotyping and discrimination in real-world situations and, in three lab experiments, that it is a heightened need for structure that mediates these effects (number of subjects: between 40 and 70 per experiment). These findings considerably advance our knowledge of the impact of the physical environment on stereotyping and discrimination and have clear policy implications: Diagnose environmental disorder early and intervene immediately.
We’re struck by a couple of things. The first is that the paper purported to have data on no fewer than 150 subjects (40 + 40 + 70), which seems like a lot for such a study. The second is that its conclusion — pay attention to physical environment and correct problems before they can fester — seems intuitive, unobjectionable and, in a sense, progressive. In other words, lots of data (well, what seemed like lots of data) plus a socially conscious message. Just saying.
In an editorial published alongside the retraction notice, two social psychologists, Jennifer Crocker and M. Lynne Cooper, state that since 2003, Stapel had submitted 40 manuscripts to journals under the umbrella of the American Psychological Association. Of those, 16, or 40% were rejected. In other words, 24, or 60%, made it into the publications — an impressive batting average. As Crocker and Cooper write:
This creates a sufficient body of work that one might expect irregularities to be detected. However, the 40 initially submitted manuscripts were handled and processed through the peer-review system by 25 different editors. Under such circumstances, it would be almost impossible to detect a pattern of data fabrication.
Crocker and Cooper point out that Stapel was unmasked by “people close to the perpetrator.” That’s fair enough. But they go on to say that “other researchers” had been raising questions about Stapel’s work. Which prompts us to pose some possibilities: If other researchers were concerned about his results, and peer reviewers are “other researchers,” then either the peer reviewers for the APA journals weren’t doing a good job, or they weren’t the right reviewers.
Expect many more retractions of Stapel’s papers in the coming weeks and months.
I have no idea how the peer review system, which examines MS after they have been written and generally without the data, could be expected to detect fraudulent data. If the publications required data and if the reviewers looked at the data and if they used a “forensic perspective” (is this data possibly screwy), then MAYBE the fraud could have been detected. But also probably not. Who would take a “forensic perspective” to the data from a paper written by such a prominent person?
I agree with Paul Thompson’s above comments, with one exception. Reviewers didn’t pass on “taking a forensic” approach because Stapel is a prominent person; they passed because taking a forensic approach is not normative in the field. (Chances are, reviewers did not even know that Stapel was the author while reviewing). There is a debate to be had about the need to take a forensic approach, but otherwise, I think its just silly to blame the reviewers for not somehow knowing that Stapel fabricated his data.
Also a sample size of 40 – 70 participants in a social psychology study is not at all out of the norm. Its certainly not on the extreme high end. In fact, I would put it on the lower end of acceptable.
Jennifer Crocker also published a very odd, defensive editorial piece about Stapel in Nature. Several people (including myself) expressed their serious dissatisfaction with her approach, which seemed just short of sweeping the whole thing under the rug. Since then, Nature has published a much more thoughtful piece by Jelte M. Wicherts:
http://www.nature.com/news/psychology-must-learn-a-lesson-from-fraud-case-1.9513
This article seems, in turn, to have attracted commentary which I haven’t yet had time to read at length, but is obviously much better thought out than either Crocker’s original editorial or the reader comments to Crocker.
I think I disagree with the contention that most submitted papers get rejected. You wrote, “Of those, 16, or 40% were rejected. In other words, 24, or 60%, made it into publications — an impressive batting average.”
This seems to me a fairly ghastly batting average. I think almost everything gets published, if not at the first journal of submission, then certainly at the second. So, if a paper is submitted to two journals in sequence and published by the second journal, is that a 50% acceptance rate or a 100% acceptance rate?
Going back to Stapel, does your quote suggest that he has 16 papers that were never published? That’s a lot of tax dollars wasted… But how do we even know this?
Let us just hope that forensic experts in Science do not come from the same lot of the Brazilian Forensic Entomologist Leonardo Gomes, also a star in this very website and Abnormal Science Blog….
And now, seriously: it is natural that fraud will happen and get past under the reviewers eyes, especially nowadays. The self-correcting mechanism of science depends on deep scrutinization, and deep reflection. And also, on oblivion. Thus any paper that gets more popular usually gets questioned with time, and papers that fall into oblivion stop from being. Another issue is preventing fraudsters from getting their edge: many of them will be happy to produce forgotten papers as long as they get money and momentary prestige from that. In these cases, investigative vigilantes and blogs play a central role in preventing the system from coming down. It is, in the short-term, up to the readers, to bring Stapels and Gomeses down.
Because I’ve seen RW comment on this before, I guess it’s worth noting that the editorial is free (registration required) but the retraction itself is behind a pay wall.
What do you expect from the Dutch?
If they are not perpetrating the most complete genocide yet recorded – the eradication of the aboriginal Orang Kaya peoples of the Banda Islands in 1621 because they refused to be enslaved – they are engaging in the worst real-estate transaction of all time – trading the island of Manhattan (yes, THAT Manhattan!) for the island of Run (a small island in the Banda chain previously denuded of Nutmeg trees by the Dutch themselves when the English ‘garrisoned’ it – and hence worthless!) so they could have a ‘monopoly’ on Nutmeg and Mace.
My ancestors hail from the Netherlands, I know about which I speak. Despite my many furtive blood transfusions to get rid of this taint it might be prudent to consider me unrealiably self-xenophobic when it comes to all things Dutch.
Wow, this really does not belong within this discussion. I’m just going to assume you are a troll and politely ask you not to muck up this site with this stuff.
My expose is not at all malapropos. We’re referencing the ‘disgraced Dutch’ are we not? I merely proffer a possible explanation for the poor man’s reprobation.
I do not troll – I’ve been told my voice is very grating and I do not wish to be the cause of other’s discomfort.
Trawling? Mea culpa.
I bet the IP of that troll is in the Netherlands. How about posting it so we do a little hunt? Perhaps it’s somebody who is guilty of the crimes that are being discussed on this site!
I trawl!
I do not troll! My voice is atrocious.
Sheesh, get it right!
My ancestors landed in New Amsterdam in 1646. Later generations, having fought on the wrong side in the American War of Independence, legged it to the Canadas as U.E.L.s in the 1790s.
My IP would let y’all know I am currently festering in the anal sphincter muscle of the Upper Ottawa Valley – Pembroke, Ontario, Canada.
According to http://www.whatismyip.com my IP is:
76.75.67.218
Send me warm blankets it is mighty cold up here.
Apologies, I forgot to give yins my address,
Bilious C. Pudenda
Care of:
Sweet Williams Floral and Gift Shoppe
171 Alexander Street
Pembroke, Ontario, Canada
K8A 4L8
1 – 877 – 732 – 9300 (Toll free only in the Canadas)
Check out flower girl’s web site:
http://www.sweetwilliamsfloral.com She is the purveyor of the finest severed reproductive organs of specific plant species north of the 49th. With advance notice, she can also provide a bouquet of the ones from Equine or Bovine species as well.
“Bilious” — please consider yourself on a short leash. Your attempts at wordplay aren’t adding anything to the conversation, and if no more comments from you appear, our readers will know why.
This is a totally inappropriate post. And, yes, I do have a Dutch surname….
So do I!
What are the odds?
Stapel’s fraud aside, it is beyond me how this kind of scientific trivia can find its way into monster journals like Science and Nature. My working hypothesis is that the fancy title (“Coping with chaos: How disordered contexts promote stereotyping and discrimination”) is what made the manuscript accepted. Actually, Stapel’s students, if he still has any, could test this hypothesis with raw experimental data their PI would be able to produce in no time.
http://www.weeklystandard.com/keyword/Diederik-Stapel
The website of “De Volkskrant”, a Dutch newspaper, reports that a website detailing the findings of the committees looking in to Mr. Stapel’s publications has been launched today. It can be found at https://www.commissielevelt.nl/
and will report results as they come in.