It turns out, snorting the so-called “love hormone” may not help reduce psychiatric symptoms such as depression and anxiety.
At least, that’s the conclusion the authors of a 2015 meta-analysis, which initially found intranasal doses of oxytocin could reduce psychiatric symptoms, have now reached. After a pair of graduate students pointed out flaws in the paper, the authors realized they’d made some significant errors, and oxytocin shows no more benefit than placebo.
First author Stefan Hofmann from Boston University in Massachusetts explains further in a lengthy letter he sent to Psychiatry Research, which he passed on to us:
We have discovered several significant errors in our published meta-analysis comparing the effect of intranasal oxytocin versus placebo administration on psychiatric symptoms (Effect of Intranasal Oxytocin Administration on Psychiatric Symptoms: A Meta-Analysis of Placebo-Controlled Studies. Psychiatry Research. 2015;228:708-714.)
Correcting these errors changed the main result of this study. The corrected result is that oxytocin is not more efficacious for psychiatric symptoms compared to placebo. Our overall placebo-controlled effect size, which was previously reported as significant, is no longer significant. In addition, none of the symptom-specific effects of oxytocin versus placebo are significant.
The primary error with our original data was the misspecification of the direction of several outcomes included in the meta-analysis. The reason for the error is that the program we used, Comprehensive Meta-Analysis, requires users to manually enter the direction of the effects. We mistakenly assumed that individuals in the placebo group would never show greater reductions in psychiatric symptoms than individuals receiving oxytocin. Therefore, we assumed that the direction of the placebo controlled effect size of oxytocin would never be negative. However this assumption was incorrect for 7 effect size estimates. In addition, we discovered some more minor data extraction errors.
He concludes:
We greatly apologize to the journal, the reviewers, and readers for the errors in the original article, and we thank the readers who brought the errors to our attention. We would like to urge researchers to be mindful of the direction of the effect sizes, especially when using certain software programs, such as the Comprehensive Meta-Analysis program. Effect sizes and effect size directions should always be carefully examined manually, even when using a software program and it cannot be assumed that the placebo group is never superior to the treatment group.
The original paper, “Effect of intranasal oxytocin administration on psychiatric symptoms: A meta-analysis of placebo-controlled studies,” has been cited three times, according to Thomson Reuters Web of Science.
The readers who brought the error to the journal’s attention were graduate students Paul-Christian Bürkner at the University of Münster in Germany and Donald Williams from the University of California, Davis, who were reviewing the literature on oxytocin and psychological disorders. Out of three meta-analyses they examined, they found two to be flawed: The 2015 paper by Hofmann and his colleagues, and a 2014 paper in the British Journal of Clinical Psychology.
According to Williams, during the peer review of their own upcoming meta-analysis on the same topic, a referee recommended that the pair submit their concerns to the relevant journals.
Williams and Bürkner informed Psychiatry Research, and the editor contacted Hofmann, who re-analyzed the study with his student, and ultimately decided it should be retracted.
The pair also submitted a letter outlining their concerns, which the journal declined to publish since Hofmann was retracting the paper. So Williams and Bürkner posted the letter on the Open Science Framework.
Williams praised Hofmann for taking swift action, but criticized him for not mentioning their letter in the retraction request.
Hofmann told Retraction Watch that it is “highly likely” that other researchers have made similar mistakes, so he is considering taking a closer look at the direction of effect sizes in a sample of studies using software like the Comprehensive Meta-Analysis program.
We’d like to credit Hofmann and colleagues for taking action, and Williams and Bürkner for taking time to examine the studies; this episode adds to several more we’ve lauded for “doing the right thing.”
The other paper that Williams and Bürkner questioned is this study in the British Journal of Clinical Psychology: “A meta-analysis and theoretical critique of oxytocin and psychosis: Prospects for attachment and compassion in promoting recovery,” cited 13 times.
Williams and Bürkner’s letter on this study has gone through peer review at the journal, and is still being considered for publication, before the authors are contacted, Williams said. Meanwhile, the pair have posted it online. It reads:
While they reported oxytocin had significant effects on three of four outcomes, we show that all effects were non-significant.
The letter adds:
Based on these null results, we hope this report encourages a re-evaluation of oxytocin as a treatment for schizophrenia.
We’ve reached out to Andrew Gumley, the first and corresponding author of the second meta-analysis from the University of Glasgow in Scotland, UK, and will update the post if we hear back.
Researchers may be taking a closer look at meta-analyses in general: Last month, we reported on a study that showed that the number of meta-analyses had increased by more than 2500% since 1991, resulting in what author John Ioannidis calls “massive production of unnecessary, misleading, and conflicted” information.
Update: 10/05/16 4:11pm Eastern: We’ve now heard back from Gumley, who told us:
The direction of Williams and Buerkner’s work is helpful and the conclusions echo our own concerns regarding the literature. Certainly we found weak and inconsistent evidence supporting the role of OXT in psychosis. We also identified consistent and significant sources of bias in the literature.
We were in dialogue with Donny Williams up until 24th June this year but our last correspondence with him regarding our meta-analytic approach remained unanswered. As far as we were aware we were still in a dialogue regarding our analytic methods.
We are revisiting the analyses of these data and I will be happy to provide further comment once we have reanalysed the data and where appropriate updated our own publication. It is especially important that we report this if there are errors in our analysis that we report these transparently and in accordance with good ethical and publications practice.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
Why does Hofmann et al.’s notice of retraction begin “We have discovered,” when graduate students Donald Williams and Paul-Christian Burkner are the ones who discovered the errors?
Why does the notice of retraction never mention Williams and Burkner by name?
Why was their re-analysis rejected by the journal – which would have corrected the record by putting the null result into the journal, and which would have given them credit for the discovery?
Good point.
The Oxford dictionary gives the following definition for the pronoun “we”: “Used by a speaker to refer to himself or herself and one or more other people considered together.” With such a wide definition, both graduate students involved in the (negative) discovery are perhaps considered, although not named.
There is however a second definition, taken from the same dictionary: “Used in formal contexts for or by a royal person, or by a writer or editor, to refer to himself or herself.”
In the context of this retraction notice, I’m afraid that this second definition is used.
Anyway, it is unfortunate that this class of retraction notices does not end with something like: “we [the authors] acknowledge the correspondence with [X, Y, Z], whose interest initiated this notice”. This was a common practice 10-20 years ago.
*blink*
Understandably, the original authors are trying to save face by somewhat blaming the program they used.
So very human.
Well, sure, if you force all outcomes from the treatment side to go in one direction, you will conclude that treatment has a positive effect. Wow.