Catholic medical journal pulls paper on conversion therapy over statistical problems

The journal for a religious medical group is retracting a paper that supported the discredited practice of conversion therapy for homosexuals over concerns about the statistical analyses — or lack thereof — in the research.

The paper, “Effects of therapy on religious men who have unwanted same-sex attraction,” was published last year in The Linacre Quarterly, the official journal of the Catholic Medical Association. (According to its website: “LQR explores issues at the interface of medicine and religion, focusing on bioethics and also exploring medical topics which have an ethical dimension.)

So, what were those effects? Pretty darn good, according to the article. Per the abstract:

The American Psychological Association and other organizations have formally claimed that sexual orientation change therapies should not be used because they are probably ineffective and may cause harm. Asurvey asking for negative and positive experiences of 125 men with active lay religious belief who went through sexual orientation change efforts (SOCE) strongly conflicted with those claims. In our study, most of those who participated in group or professional help had heterosexual shifts in sexual attraction, sexual identity and behavior with large statistical effect sizes, similarly moderate-to-marked decreases in suicidality, depression, substance abuse, and increases in social functioning and self-esteem. Almost all harmful effects were none to slight. Prevalence of help or hindrance, and effect size, were comparable with those for conventional psychotherapy for unrelated mental health issues. Judged by this survey, these therapies are very beneficial for lay religious people, but no Catholic priests were in the sample, and this study makes no recommendations for them.

In other words, gay conversion therapy — which, we note, is strongly opposed by the American Psychiatric Association, as well as the American Psychological Association — works, with virtually no downsides. Not surprisingly, the article was trumpeted by the Liberty Council, a foundation that pushes conversion therapy, in a press release that the group has since taken down from its website after Retraction Watch asked whether they were aware of the retraction.

The corresponding author of the paper, Neil Whitehead, told Retraction Watch the situation was “quite peculiar” and that two other journals are interested in publishing the work.

‘Absolutely shoddy research’

Not long after the original article appeared, it drew the attention of the blog Friendly Atheist, which posted a lengthy take-down last August of the research. We won’t quote the entire post, but here are some highlights: Of the three authors, just one has a background in mental health — and that person is affiliated with a religiously conservative outfit called the Southern California Seminary. (Whitehead trained as a biochemist and has a history of writing controversial things about homosexuality, including that being gay is not rooted in genetics.)

Another point:

Wait a minute. So the participants for this research were all people who “wanted help with their unwanted same-sex attractions” That’s not a random sample. That’s a sampling of people who think you provide good service. It’s like a restaurant owner asking her regular customers to leave a review on Yelp — of course it’s going to be positive; that doesn’t mean everyone feels that way.

This is the equivalent of going to a fortune teller, chatting with her regular customers, then telling the world, “See? Psychics are effective because these people felt better after their session.”

It’s absolutely shoddy research that would never get published in a legitimate journal because it’s so flawed.

In other words, potentially biased researchers doing potentially biased research.

And those weren’t the only flaws. According to the retraction notice:

The journal is retracting the article based on unresolved statistical differences. Specifically, after receiving questions about the article, the editor determined that a statistical review of the paper, which was recommended during peer review, had not been conducted. The editor commissioned the statistical review after receipt of the questions. The statistical review identified the following concerns regarding the methodology followed in the article:

1. No common intervention was given to participants that would allow for a valid conclusion to be drawn.

2. The paper did not establish a demonstrated relationship between the intervention and the survey that measures the intervention in that the paper did not clearly address whether all respondents were treated according to the same (or similar) protocols and for the same periods of time, and/or by therapists of like or similar training and expertise.

3. The chi-square test requires groups that are similar and the reported variation in the treatment times and modalities in this study is significant, rendering the chi-square test absent a control group (which was not present in this study), an invalid measure.

According to the notice, the authors provided backup for their analysis, but that didn’t satisfy the editorial concerns:

The Editor considered this information and the authors’ response and concluded that in spite of this additional information, the original review still required retraction of the paper.

The Journal Editor and SAGE Publishing wish to emphasize that the retraction is not based on any action taken by the authors but only the statistical concerns outlined above, and they regret that the concerns identified by the statistical review were not addressed prior to publication of the article.

This is not the first time we’ve reported on papers about conversion therapy.

Corresponding author speaks

Whitehead responded to our request for comment with the following:

The situation about this paper is quite peculiar. The core statistical test (“probability of superiority” ) was that recommended by the American Psychological Association in its 2009 review of research on the subject.   The paper was submitted to Psychological Reports and the statistical side was examined by an assigned expert and was improved to his satisfaction. However then before completion of other review the journal was sold to Sage, and a number of articles in progress including ours were sent back to square one for resubmission. Rather than do this, our paper was submitted to Linacre Quarterly. A number of reviewers commented, and slight changes were made, following which the paper was accepted for publication. We told the editor the past history including the previous statistical vetting and the use of the APA criteria is mentioned in the text.

A new editor now looks after the journal, and observed that there had been no statistical review before publication. (the reviewer was asked but did not supply a review) The approval given by Psychological Reports was apparently not considered adequate by the editor, though mentioned to her, but we do not know whether the reviewer knew about this. The review then obtained is given in the retraction notice.   We regarded the reviewers comments as wrong for the reasons we state, but there has been no further comment as to why our reasons are wrong and we would have welcomed further engagement as we mentioned to the editor. Another researcher who from his published papers is well versed in statistical methodology, having seen the paper, without our knowledge wrote to the editor and recommended publication, but thought we may have somewhat underestimated the strength of the effects we found if more powerful tests had been used. This input did not alter the final opinion of the editor.

We therefore regard the statistical review by the journal as not really dealing with the methodology, but in their defense we would want in retrospect to say that this detail of statistical analysis is probably well outside their corporate expertise. Their board has mentioned to us they would welcome another article based on the same data less the statistics – but lack of them has been the core criticism of papers published  by others, and we think this would add nothing to the debate.

We have information that any influence of gay activism on the process was probably not significant and should be disregarded.

We see in the literature that another pattern is not to retract, but publish articles by others discussing any controversial points. However we can see that this could still lead to problems for the editor who would find it very difficult to evaluate any such papers. On another level, it seems possible that bioethics, the raison d’etre of Linacre, is moving more and more to the detailed statistical evaluation of ethical claims, and the journal will have to decide whether it wants to follow that route.

In some senses no-one is much at fault here, which is why the retraction takes the form it does. We have two other journals in the wings who are quite interested in the paper, so retraction in this case should not mean disappearance of this study from the literature.

We found the whole process highly unusual but quite explicable in retrospect. We wonder if you know of parallel cases, where there has been unilateral retraction of a study by an editor post publication, except in cases of obvious fraud or failure to replicate, which we suppose is mostly  revealed in the harder sciences.

Update, 1700 UTC, 6/18/19: Barbara Golder, the editor of the journal, tells us:

We are unsure how this was missed given peer review occurred under previous editorial leadership and with a different publisher.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].