A
publisher has issued an expression of concern (EoC) about a study that claimed children with same-sex parents were at greater risk of depression and abuse, after posters using statistics from the paper to support a homophobic message appeared in Australia and the US.
On Aug. 21, several news websites reported that these posters were appearing in Melbourne, Australia, citing claims from a 2016 paper published in Depression Research and Treatment, which said that children with same-sex parents are more at risk for depression, abuse, and obesity than children with opposite-sex parents. The poster had also appeared previously in Minneapolis and has been traced to a neo-Nazi group, as reported by HuffPost Australia. Australia is preparing for a national, non-binding, mail-in vote on whether to provide marriage equality for same-sex couples.
The EoC mechanism, which was chosen by the journal’s publisher, Hindawi, is an unusual choice here. The paper’s author, D. Paul Sullins, a sociology professor at The Catholic University of America and the paper’s author, told Retraction Watch that Hindawi contacted him Aug. 21 about the decision. Initially, he told us he didn’t have any “particular objection to it,” but later told us he changed his mind after he read more about COPE’s guidelines for EoCs:
an Expression of Concern is a step in addressing a possible issue of scientific misconduct. No misconduct of any sort was alleged to me in the review and acceptance procedure of this article, nor at any time since. Hindawi has stated no reason for the expression of concern that relates to the study’s evidential basis or credibility. Instead, the publisher has issued the Expression of Concern only a day or two after two of the study findings were cited on a poster using pejorative anti-gay language in Australia, citing use of study information by other parties that it “believes to be hateful and wrong”. As any scientist who publishes in an open forum, I cannot be responsible for the use or misuse of my research by other parties. Nor does misuse by others impugn the accuracy and credibility of my findings.
In the EoC notice, published Aug. 22, Hindawi said it investigated the peer review process for the paper, but found no evidence of misconduct. Hindawi also said it condemned the “hateful and wrong” arguments the paper had been used to support and noted it had investigated the article’s peer review process after multiple readers raised concerns about it soon after publication.
Immediate controversy
The original paper, “Invisible Victims: Delayed Onset Depression among Adults with Same-Sex Parent,” was published April 19, 2016 and has not yet been cited, according to Clarivate Analytics’ Web of Science.
Critics had attacked Sullins’ paper as soon as it came out. Nathaniel Frank, a historian of public policy and the director of What We Know, a project at Columbia Law School collecting public policy research related to LGBT equality, slammed the paper in a column at Slate, calling it “a dishonest, gratuitous assault on LGBTQ families.” Frank told Retraction Watch that Sullins hadn’t controlled for family disruption, which is known to affect children negatively and used an overbroad definition of abuse, which included instances where a parent or any other caregiver said something that made the child feel bad.
On July 18, 2016, Hindawi invited Frank to write an official “comment” to add his criticisms to the record. Sullins has posted a response to the comment. Neither of these articles have been cited.
Frank told Retraction Watch that he wasn’t surprised to see Hindawi’s response, but that he felt it was “too little, too late:”
It was always irresponsible of Sullins to make these claims and for the journal to publish it. That’s even more the case in the current political climate.
What’s good about what happened is that there is daylight and the publisher and author are beginning to be held accountable as part of this process. It should have happened earlier but it’s better that it’s happening now than not at all.
I think people need room to express unpopular beliefs without being accused of being responsible for violence maniacs may perpetrate in citing their beliefs, but that doesn’t mean you can distort reality.
An internal ethics investigation
In the notice, Hindawi described how its research integrity team led an “internal ethics investigation” of the paper in 2016, after the journal received criticisms of Sullins’ paper:
[W]e evaluated the article’s peer review process and brought several concerns to the handling editor’s attention. These included: the study’s small sample of same-sex parents, the lack of discussion of other influences such as family breakup on the wellbeing of the children included in the study, the implied causation in the title “Invisible Victims,” and the potential conflict of interest implied by the author’s position as a Catholic priest.
But the journal’s editor determined the study, despite its limitations, did not warrant correction or retraction and Hindawi did not overrule him.
Concern over how a paper is being cited is not typically accepted as a reason to issue an EoC, under the retraction guidelines set by the Committee on Publishing Ethics (COPE). According to COPE, an EoC should be reserved for when there is: inconclusive evidence of misconduct, evidence the findings are unreliable but the author’s home institution will not investigate, a belief a misconduct investigation isn’t — or couldn’t be — fair or impartial, or an ongoing investigation that won’t produce a judgment for a long time.
The Hindawi notice is unusual, but not unprecedented: In June, the New England Journal of Medicine issued a similar post-publication comment to a short letter from 1980 that suggested patients rarely developed opioid addiction after hospital prescriptions. Like the NEJM warning — which stopped short of calling it an official EoC — Hindawi’s EoC appears to be a comment on how the paper was used, rather than a signal that there are issues related to misconduct.
When asked why Hindawi chose to post an EoC, Andrew Smeall, head of strategic projects, told us:
an “Editor’s note” would not have been appropriate in this case as this Expression of Concern came from the publisher.
Going forward, we will discuss further with COPE, our editors, and other publishers the most appropriate way to use Expressions of Concern or related terms like the “Publisher’s note.” At the moment, there do not seem to be well established mechanisms for publishers to provide such context while preserving the editorial independence of the journal’s academic editors.
In a blog post, Paul Peters, CEO of Hindawi Limited, said that the publisher decided to issue the EoC after determining its “previous steps to highlight the concerns that had been raised with this article were insufficient:”
[R]eaders who were referred directly to the original study were likely to be unaware of the subsequent critique that was published in the journal. Therefore we felt that it was important for us to highlight the concerns that had been raised about this article in the form of an Expression of Concern, as this would be made clearly visible on both the HTML and PDF versions of the published article.
Retraction of the article was among the “range of options” Hindawi considered; publishing the review reports was also considered, however, Peters wrote:
Following lengthy discussion we agreed that pursuing either of those options could set a concerning precedent by interfering with the editorial independence of our journals’ Editorial Boards or violating the confidentiality of the peer review process.
We asked Hindawi to share the referee’s reports, but a spokesperson declined to release them to us.
Peters also said Hindawi will be reviewing its policies and procedures.
Not unusual: Posters don’t use confidence intervals
Sullins told Retraction Watch that he was “not happy” his paper had been cited in the inflammatory poster:
I unreservedly repudiate any use of my research to justify bigotry and stigmatization….
However, the statistics that [the poster] cites are essentially accurate.
The problem, Sullins told us, was that the figures weren’t given proper context:
They aren’t put in a scholarly context where you would consider the pros and cons of the certainty of the evidence.
“[The poster authors] don’t put the confidence intervals in,” he said, referring to one way that researchers indicate the uncertainty of results.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
Yikes! If there’s misconduct here, it may be on the part of the publisher. In the key comparison, the N of opposite-sex couples was 12,268, but the N of same-sex couples was only 20. You read that right–twenty! It’s absurd that controversial claims be based on such a tiny sample size and referees and the publisher should know that.
I’ve read through some of the papers Sullins published or co-published. It appears the 5 papers I perused tell stories similar to those in Huff’s, “How to Lie with Statistics” as a simple look at correlation and coherence, one could come to very different conclusion. In addition to sample size or statistical significance, I have reservations about the observational assessments and conclusions based on the data provided.
I find the political struggle and ‘shaming’ aspect of this episode quite deplorable. Comments on science should be limited to science: identify problems in the methodology (the N for instance), the interpretation, or other valid issues. Don’t attack the guy because you don’t like his conclusions, or because some morons are deploying the findings improperly.
Nathaniel Frank’s language smacks of political orthodoxy: “irresponsible,” “held accountable,” and the backhanded: “I think people need room to express unpopular beliefs without being accused of being responsible for violence maniacs may perpetrate in citing their beliefs, BUT that doesn’t mean you can distort reality.” [emphasis added]
Of course, it is Nathaniel Frank who gets to define reality — not the scientific method. Seeing this kind of discourse is deeply troubling. He is simply accusing Sullins of wrongthink.
Nor should the publisher have engaged in the COPE protocol for issues not related to ethical breaches.
The incursion of political struggle into the practice of science should be resisted wherever and whenever it appears.
Well said.
Right on! Political and identitarian orthodoxies, and raw anger now trump science and the scientific method! Very unfortunate.
> Don’t attack the guy because you don’t like his conclusions
But if his conclusions are derived from willfully deplorable methods, an “attack” is warranted, yes?
[R]eaders who were referred directly to the original study were likely to be unaware of the subsequent critique that was published in the journal
The Editor seems to accept that the study was bad from a scientific perspective, but it brought them so many follow-up publications that they’re not sorry for publishing it.
The author is refreshingly honest about the weak nature of this study:
“Limitations. Despite the signal strengths of Add Health as a large nationally representative longitudinal dataset and notwithstanding the strong significance for contrast effects reported above, the very small size of the sample of children raised by lesbians imposes important limits and prompts great caution regarding the conclusions of this study. As with all observational studies, causal inference is not possible. Moreover, many subtle distinctions and pathways of influence simply cannot be addressed with only 20 cases, and unobserved differences between the parent comparison groups may well confound some or all of the child differences observed. In particular, the lack of useful measures for parent mental distress, depression, family history of violence, alcohol consumption, and substance abuse precluded examination of important familial risk factors which may be associated with child distress. For these reasons, the findings of this study should be considered only provisional and exploratory until and unless they are confirmed by further research.”
So many papers published these days should be so honest. Too many papers are only provisional and exploratory yet hype themselves as great scientific discoveries.
Shame on anyone that takes findings from a provisional and exploratory unconfirmed study to make any argument save conjecturing as to further necessary studies to sort out the ambiguities.
Yes, though, they did title their article “Invisible victims…”
So all open access journals are clearly biased rubbish – and liable to thorough, scholarly rubbishing — on Slate!
“…come from a thoroughly rubbished study from D. Paul Sullins, which was published in an open-access journal which requires authors to pay for publication.” From the linked HuffPo piece.
Author-supported publication has traditionally been known as “vanity publishing”. Does this description fit the open access model (beyond ‘predatory journals’)?
Clearly not. But obviously in this case the author wants to score a point against the credibility of the research – so he reaches for this one. The same thing happened with the ‘conceptual penis’ article.
Sigh.
Catalonia, I’m not sure what’s “deplorable” or “smacks of political orthodoxy” about calling a researcher irresponsible for drawing wholly unfounded conclusions about a politically charged topic like this, especially if one has any awareness of the concerted effort undertaken since 2010 to use research to attack LGBT equality. What Sullins has done here is equivalent to surveying a cancer ward in, say, Albania, to derive national mortality rates and concluding that “Albanians” have a life expectancy of 45.
The point is that, across a lengthy exchange, we have, in fact, focused on the science, and it is from that science, including the studies amassed at http://whatweknow.law.columbia.edu/topics/lgbt-equality/what-does-the-scholarly-research-say-about-the-wellbeing-of-children-with-gay-or-lesbian-parents/ that I draw my conclusion that Sullins is distorting reality. It takes a healthy dose of naiveté not to recognize what’s going on here, and it tends to be the luxury of the well-positioned to politicize science and then cry foul at the “political incursions” that come in response.
The focus on science is fine and good. The tone policing that comes after is not. If the study has methodological mistakes, they should be pointed out, if not, not. Mentions of the current political climate and allusions about the authors’ hidden political motives have no place in scientific criticism.
Someone else can tell you about the choice of language in detail, the main point for me is that there are so few of these ssm parented families that samples are bound to be small in comparison.
It is hardly scientific to wait until there is some sample large enough, and forgive me if I am wrong but this criticism wouls be different if the study indicated that children of ssm families did wonderfully well
This is monstrous, wildly unethical, and politically partisan. An Expression of Concern must be about problems with the paper – misconduct or scientific problems, such as those in my former advisor Barbara Fredrickson’s “positivity ratio” paper, which used invalid statistical methods: http://retractionwatch.com/2014/07/29/positivity-ratio-research-now-subject-to-an-expression-of-concern/
Or an EOC can be about misconduct, where it’s often a cowardly compromise in cases where the paper should simply be retracted.
The modern left is making it increasingly difficult to safely conduct scientific research across a wide range of domains, such as the group differences that got James Damone fired at Google, the effects of gay parenting as we see in this example, and anything else that threatens to add nuance or complexity to the simplistic worldview of a crude political ideology. (Of possible interest: My and my colleagues’ work on how leftist control of social science has undermined the quality and validity of research: http://www.rci.rutgers.edu/~jussim/Duarte%20et%20al,%202015,%20BBS,%20target,%20commentaries,%20reply.pdf)