The journal Memory has retracted a paper on repressed sexual abuse after a protracted dispute between the authors and an institutional investigation in The Netherlands that led to no findings of misconduct against the first author, Elke Geraerts — a rising star in the field of social psychology. (The title of hers TEDx talk, by the way, is “Resilience as a key to success.”)
The article, titled “Linking thought suppression and recovered memories of childhood sexual abuse,” was published in 2008 and has been cited 10 times, according to Thomson Scientific’s Web of Knowledge. Here’s the retraction notice:
The following article has been retracted from in Memory by the first author. The co-authors have requested that their names be withdrawn from the paper.
We reached Harald Merckelbach, a Dutch researcher and a co-author of the article, who gave us a lengthy run-down of events:
The chronology of the case is as follows (brief version): way back, in 2009, long before the Stapel affair we – the co-authors – began to have concerns about an unpublished manuscript. We then decided to look into the data of a published paper (i.e., the Memory article) that immediately preceded this unpublished ms and addressed a similar research question. We found the same type of irregularities (see below). We then filed a complaint at the Maastricht University Integrity committee. After months of investigations, this committee concluded that yes, there were errors in the Memory paper, but no these errors had not been intentionally created.
So then we communicated to the first author that either way the errors and omissions had to be addressed in an official addendum or corrigendum to the Memory article. The first author refused. According to her, there were no errors or omissions in the Memory paper. We had – for over a year – intense e-mail exchanges in which we – the co-authors – insisted on an addendum/corrigendum and in which the first author denied that there were problems.
Then we – the co-authors – learned about the Klessig case, which illustrated that it is possible to withdraw unilaterally your co-authorship when you have concerns about the data analysis. So, we wrote letters to the Memory editors that we wanted to retract our co-authorships of the article. In the same period, journalists from the Dutch paper NRC Handelsblad became interested in the case and published their article about it.
Rotterdam University authorities – where the first author is now employed – then decided to have their own investigative committee. This committee looked into the data analysis and concluded that there were errors and omissions in the paper. This report was released two weeks ago. The first author then decided – at last – to withdraw the Memory article.
The problem with the Memory paper is basically this: the dataset (N) consisted of two subsamples (n1 and n2). The co-authors believed that there had been no selection on n1 and n2, but they found out that there had been subjects tossed out and new subjects included. This whole selection process had not been mentioned in the Method paragraph (which is an ommission). and it had produced errors in a table displaying means and standard deviations (which should have been corrected).
The Dutch paper NRC Handelsblad has been covering the case extensively. In one article, from September 2012, the paper reported:
The co-authors tried to convince Geraerts to correct the errors in the Memory article. “We asked her on numerous occasions to show us all the raw data and the ‘cleaned’ tables. Then we could have prepared a correction or addendum together. However, she didn’t want to do so,” says Merckelbach, who never received the cleaned tables. Geraerts says that she believes that no correction is necessary. “Because the findings are correct with my adjusted dataset. The committee thought so too.”
Geraerts was consistently supported in this by the Executive Board [of Maastricht University, where Geraerts was working; she’s now at Erasmus University in Rotterdam]. In the summer of 2010, [then] President Mols sent an email to the co-authors saying that the case should be considered closed, and insisted that any further communications with Rotterdam go through him. Otherwise Geraerts might complain about stalking. When asked about this, Mols now says that he did not notify the EUR about the case, “because there were no questions”.
And why did Maastricht University never insist on a correction? Mols explains, “The case was permanently closed when no appeal was filed. As such, the Executive Board saw no reason to take further measures.”
Richard McNally, a Harvard psychologist who supervised Geraerts when she did a post-doc at Harvard in the mid-2000s, and whose name also appeared on the paper, said that although the retraction was long-awaited the issue is not fully resolved.
We never got our concerns [about the integrity of the data] addressed.
Geraerts has refused to share her data with the rest of the team, McNally said, citing principles of confidentiality that he found risible. And McNally told us that the institutional investigation didn’t examine the data either.
Update, 11/26/13, 2:30 p.m. Eastern: We have replaced the photo of Elka Geraerts, taken from her lab homepage, that originally ran with this post after a claim by Geraerts that it violated her copyright. We believe that our use fell within U.S. Fair Use provisions, given that Geraerts’ work was the subject of the post, but we have other priorities and a picture of the journal will do just fine.
thanks for this. nice to see a model of what path someone might be encouraged to take to retract unsound research.
The new investigative committee was based at the University of Amsterdam and found, among other things, “anomalies in the data and very large effect sizes”. It is unclear why these occurred and so their conclusion was that from a scientific point of view the results are untrustworthy. It is a good step for Geraerts to retract the paper, regardless of how these problems came about. Also, note that Geraerts is a clinical psychologist specialising in research on memory and cognition and NOT a social psychologist.
Struck through “social.” Thanks.
It would be more appropriate to have “clinical psychologist” as “psychologist” usually implies “experimental psychologist”.
Is it known what kind of anomalies occurred, besides large effect sizes? Very large effect sizes are in itself not a very good argument in my opinion (then, we can retract many articles), so these anomalies must be very serious. I think it is important to know why these anomalies occurred.
What worries me, is that there seems to be another unpublished manuscript that has the same problems (if I read the text well).
Went to her website through the link provided on RW. I was surprised by the size of the lab she runs!
You shouldn’t be surprised – this might make an interesting graph: funding level versus retractions. I would doubt it is linear…
A nice analysis of the unlikely statistics in some of the papers of Geraerts and Smeesters (who has been covered extensively on RW):
http://www.math.leidenuniv.nl/~gill/Integrity.pdf
This is serious to me. We knew about the Smeesters stuff. So, it seems that there are “too good to be true” findings in not only the Memory paper, but also an unpublished paper (in which the means are super close to each other, see slides 24-25), and an earlier published paper (last slide). Also, unwillingness to share data and data got lost. Will her university open any further investigation?
Take a look at the TEDx talk.http://tedxtalks.ted.com/video/TEDxAmsterdamWomen-Elke-Geraert Eight minutes to tell us that what doesn’t kill us may make us stronger. If this is the ‘smartest woman in the Netherlands’ the Dutch are in trouble!
It seems that the video is not there any more.
Try again. I can still see it.
Typo in Merkelbach quote: unliterally, presumably he meant unilaterally?
Good catch, fixed.
Well, this is disappointing. I have several of Geraerts’ papers in the reading pile beside my desk. As Merckelbach and McNally are co-authors on most of those papers and haven’t repudiated them, I hope the findings are sound. I would hate to see a paper retracted after I’ve cited it while debating a proponent of repressed memory.
I have been reading papers on repressed memory to inform my review of alien abduction claims. Abduction proponents have a fatal inability — or plain unwillingness — to cite science papers when asserting their folk psych notions of memory function. These people use any and all excuses to dismiss scientists as frauds or conspirators (i.e., scientists all conspire to suppress “true” knowledge). Of course, since abduction proponents don’t read science papers, they don’t know anything about retractions. But still…
Better a retraction than an abduction……or is it?
Some abductions are OK
Reblogged this on lab ant and commented:
Ok, this is one of the reasons why I’m a strict supporter of a radical open data policy. In my opinion the raw, unaltered data a paper is based must be made public upon publication (In order to ensure transparency). If one author refuses to communicate the raw data even with co authors that smell very very fishy………
Every time people start talking about a “rising star”, I’m very skeptical. Rising stars, typically, either have a major advantage (e.g., the PI they worked for as postdocs promotes them heavily) or they cut corners (which explains why they can push out more stuff than their colleagues). In my entire career, I have not met a single “rising star” that did not trigger my phony detector.
Groan. Not this repressed memory stuff again.This became a significant little industry among family lawyers and psychologists in the 1980’s. In Houston, we developed a whole revolting tribe of entrepreneurial psychs who achieved fame, and great fortunes, “recovering” memories of childhood sexual abuse and testifying at custody hearings. Some large clinic systems even got into the business on a wholesale level. Eventually, the science was exposed as largely crap (and sometimes fraud), but the damage done was considerable. My personal belief — no data, just strong suspicion — is that the eventual popular backlash against such legal-scientific industries of that era played a significant part in shaping both the best and worst parts of today’s Texas political culture.
Sorry, that’s all somewhat OT. The point is that this subject is strong social poison. When very carefully used, in a few cases, it can solve otherwise intractable individual problems. But if a professional gives any appearance of using it carelessly, or for self-promotion, they should be stopped. It is extremely corrosive and socially dangerous to children, to families, to the professions, and to science.
I live in Houston. I had the misfortune to hit puberty at just about the time the FDA or FTC or whoever it was decided it was legal for psych hospitals to advertise on TV. You remember the ads–“Do you have a sullen teenager? Well, just send them to Expensive Inpatient Psychiatric Hospital, and we’ll fix them right up!” This was not long before so-called “Oppositional Defiant Disorder” became classified as an alleged mental defect. When I was 14, my parents sat me down in front of my 8-year-old brother. The Paternal Unit looked at me with a fake-sad expression and announced that my parents would really like to send me to Expensive Inpatient Psych Hospital, but they just couldn’t afford it, so they (at the instigation of the school district) were sending me to a private shrink instead. They decided they’d rather pay a shrink to talk to me, than talk to me themselves for free. I’m 41 now, and the Parental Units still don’t understand why we (hubby and I) rarely speak to them, see them even less, and that’s just fine with us. It’s a helluva thing to force your daughter into two unnecessary years of “Adaptive Behavior”, purely because you’re so busy wallowing in your own careers and your own self-pity to actually pay attention to her, or her needs. So much more convenient to let the school handle your “problem child”!
Actually, McNally, along with Loftus, is among the most outspoken critics of the construct of “repressed memory”.
There is an important difference between the “Memory paper affair” and those of Smeesters, Stapel etc. And I’m not talking about the difference between “psychology” and “social psychology”.
Stapel admitted to faking data.
There is strong evidence that Smeesters faked data (the data-set which he sent to Simonsohn before his computer crashed shows the typical pattern of someone filling in “random” numbers to get a desired pattern.
However, according to the report of the Amsterdam investigators, the Memory paper data shows a different kind of anomaly: the sort of patterns you’ld get if clerical errors were made copying data from one spreadsheet to another. And that kind of error could easily generate “too good to be true” statistics. (Copy part of the data of one of the groups into the wrong group. Reorder part of some of the columns but not all. Who hasn’t made that sort of mistake before?). Since original paper forms, original computer files, etc. are all gone, and since there are no log-books or R scripts there is no way to check.
“Data preparation” is part of the study, part of the science … but in this case t’s not reproducible.
This reinforces my view that there is a huge chunk of scientific methodology that is never reported and no one seems to talk about or care to make transparent: what happens to data once they’ve been collected all the way to the publication of the final report? How is quality control implemented, how do researchers ensure that no errors of transcription, copying, transformation etc creep in and make or break effects? Whenever I talk to people working in other labs about this issue, my sense is that many are approaching this gray zone between data collection with a fly-by-the-seat-of-my-pants attitude.
This is troubling.
Moreover, there appear to be no books or articles in psychology (maybe there are in other disciplines) on proper quality control procedures from data collection to the final data set and syntax used in a publication (even the term “syntax” seems to baffle some colleagues — they do everything by point & click). I had the good luck to have a role model in my dissertation adviser, Joachim Brunstein, who showed me how to document every single step of data aggregation, transformation, and analysis in one syntax file. I later added to this procedures for ensuring that no substantial errors creep in in the process. But if you find them, it’s easy enough to spot where and when it happened when you have a syntax file. I can’t imagine working without such safeguards. They’s saved my ass more than once before I even considered publishing (or not publishing) something that turned out to be due to some data transformation flaw in the end. With data collection being as detailed and rich as it is in these days of computer-based assessment, you haul in so many littly-bitty pieces of data that you need to sift through, aggregate, and process that you really CAN’T afford to not think about and implement rigorous QC procedures.
PS according to announcements (in Dutch) on the Erasmus University website, if you are a bona fide scientist you can obtain a copy of Prof Han van der Maas’ report, and even of the Memory paper dataset, from the department of psychology at Erasmus University. It would be nice to have the report translated into English because it is well written and careful work, and the statistical methodology is of course interesting in its own right.
I too am concerned about the other Geraerts papers. While those who support the existence of recovered memory have been pouring out scientific support for their position for more than a decade, it has been a real trial to get false memory advocates to engage in something other than polemic. Geraerts was an exception, and I am now worried. Do the co au.thors stand behind their other papers with her? Does anyone know? I too cite these papers and want to believe in their integrity.
Dear Constance, I guess it has not been in international media but her coauthor Merckelbach said in a Dutch newspaper that he now believes Geraerts has not done anything intentionally wrong in the Memory paper. I’d assume this implies that their other papers together are supported by them as well.
There is only one other *published paper* by these authors, as far as I know, with the same “too good to be true” pattern. It’s E. Geraerts, H. Merckelbach, M. Jelicic, E. Smeets (2006), Long term consequences of suppression of intrusive anxious thoughts and repressive coping, Behaviour Research and Therapy 44, 1451–1460. I asked the authors for the original data and it’s lost. I don’t know if they are considering retracting it.
But if another published paper and another unpublished paper also show these too-good-to-be-true findings, can we still talk about a “benign” explanation for the patterns of data in these studies? Does that mean that there were copy errors from one spreadsheet to another in all these papers? That’s very hard to believe. Given that the original paper forms and computer files are “lost”, this trigger other thoughts in me. Does her university know this, because this does trigger concerns about either her integrity or otherwise serious sloppiness.
Geraerts’ thesis consists of 9 chapters of which the first is an introduction, the other 8 all correspond to published papers. I skimmed through these papers looking for similar statistical designs and findings as in the Memory paper – four groups of subjects, three expected to be similar, one strikingly different from the other three. Only two papers had a similar format to that of the Memory paper. One of them showed the same “too good to be true” aspect – the three groups which should be similar are statistically speaking much more similar than you would expect, if the data consisted of three independent random samples from three very similar populations. The other didn’t. But notice, the statistical significance is not overwhelming: p-values of around 1%. In each case it *could* just be chance…
But indeed because of the repetition of the same “too good to be true” pattern in two published papers and one unpublished I am inclined to reject the very benign explanation “chance”. However I am ever mindful of Hanlon’s razor: “Never attribute to malice that which is adequately explained by stupidity”, though the word “stupidity” is also about as extreme as “malice”. The British-English version of Hanlon’s razor is “cock-up before conspiracy”. I also strongly believe in “innocent till proven guilty”, and in the “good faith” assumption. I’m also mindful of the “unknown unknowns” problem. Truth is often stranger than fiction. Just because we can’t imagine a particular cock-up scenario which “explains” the data, doesn’t mean that there isn’t one. Our own lack of imagination of possible “cock-up” scenarios does not mean proof of the “malice” scenario.
But so what? Because original data is lost and there is no “audit-trail” of data manipulations, selections, corrections we simply cannot verify anything. What is the point of wondering about possible benign or less benign explanations? This fact, together with the “too good to be true” pattern in itself, is good enough reason to consider the conclusions of these papers as scientifically unreliable.
At the time the research was done, Geraerts was a young promissing PhD student and later postdoc. The actual data-processing was done by student-assistants, not by Geraerts personally. I think that her senior-coauthors, who co-authored these papers, must share blame for the failure of a young scientist to adhere to “good research practices”. There was a large group of scientists from very young to very senior working together on these projects and the group as a whole failed in some way or other.
Is it significant that Geraerts’ dissertation can no longer be found on Maastricht University’s website? (There are links to it from other websites but those no longer function.) Even the university’s dissertation database does not seem to include it, even though Merckelbach’s dissertation is there. For those wishing to read it, archive.org fortunately still has a copy: http://web.archive.org/web/20060808005622/http://www.personeel.unimaas.nl/e.geraerts/PROEFSCHRIFT_Geraerts%20V3.pdf
There is one thing that confuses me. In an interview, the author explains that the article was retracted rather than amended because the raw data is no longer available (because it was erased by the university as a matter of routine), and so it would not be possible to ascertain what went wrong, and thus uncertainty would remain. (Observant 34 (2013), issue 2, translation mine.)
However, it was widely reported that Maastricht University negotiated with Richard Gill (and, apparently, other outside statisticians) for access to the raw data, implying that it is still in existence. In addition, Richard Gill reports in the comments above that the data is available to bona fides researchers who request it!
What gives?
Regarding the Memory article, the situation is that Geraerts does possess the final data set corresponding to the published article: ie the data-set (probably an SPSS data file) on the basis of which the summary statistics and statistical tests reported in the article can be reproduced. What has got lost, are all original paper filled-in questionnaires and any intermediate phases of data-cleaning, reduction, merging, classification of the subjects into groups, etc.