Study claiming dramatically higher rates of male military sexual trauma is retracted

A study that found a 15-fold increase in the rate of sexual trauma among men in the U.S. military — and sparked suggestions of “an epidemic of male-on-male sex crimes” in the military among conservative media outlets — has been retracted because of a flaw in the analysis.

The study, published just last week, appeared in Psychological Services, an American Psychological Association (APA) journal. In an announcement Sunday titled “American Psychological Association Retracts Article Positing Excessively High Rates of Sexual Trauma Among Military Men,” the APA said that “Scholars raised valid concerns regarding the design and statistical analysis which compromise the findings.” Here’s the text:

WASHINGTON – The American Psychological Association has retracted an article that reported excessively high rates of sexual trauma among military men because the data analysis was flawed.

The article, “Preliminary Data Suggest Rates of Male Military Sexual Trauma May Be Higher Than Previously Reported,” was published in a special issue on military sexual trauma of the journal Psychological Services. Other articles in the same issue, dated November, also indicated that the rates of sexual trauma in the military may be higher than previously reported.

“Although the article went through our standard peer-review process, other scholars have since examined the data and raised valid concerns regarding the design and statistical analysis, which compromise the findings,” said Gary R. VandenBos, PhD, APA’s publisher. “After further analysis, and discussion with the article authors, we have decided to retract this one article.”

VandenBos noted that the same journal issue includes 12 other articles on the extent of military sexual trauma and assault — with a common message that they occur more often than realized and are generally under-reported for many reasons—and they have long-term negative impact. “One article having some problems with its statistical analysis should not undo the power and facts of the other 12 articles as a collection,” he said.

“Military sexual trauma” was broadly defined in the journal articles and included sexual assault, sexual battery or repeated, threatening sexual harassment experienced during military service.

A revised press release on this special issue can be found online here.

Last author Kevin Kip, of the University of South Florida, tells Retraction Watch all of the authors agreed “a retraction was the most appropriate and scientifically responsible course of action.” Kip, who said he  “provided substantive comment to the paper, but was not involved with the original analysis,” explained why the paper was flawed:

As you know, this study used a prevalence estimation technique known as the Unmatched Count Technique (UCT). The UCT technique has been reported to be able to achieve more accurate estimates of the prevalence of “sensitive” questions that may be under-reported, such as illicit drug use and sexual assault.

Using the UCT method from the online survey that was administered, the data suggested potential substantial under-reporting of male military sexual trauma (MST). In addition to the estimate itself, as you know, statistical testing is based on the variance of the estimate. The original analysis used a published statistical formula and probability distribution that have since been shown to be less reliable than more current methods. Thus, whereas the calculation of the potential magnitude of under-reporting of male MST was done correctly, the statistical test applied to determine statistical significance was not based on the current, more reliable methods.

Thus, although the study data suggest that male MST may be under-reported, after applying the correct variance estimate, this study is unable to determine the magnitude of potential under-reporting, let alone, if it is under-reported at all with respect to the conventional estimate of about 1%. Thus, all of the authors were in agreement that a retraction was the most appropriate and scientifically responsible course of action.

Corresponding author Sean Sheppard was “unexpectedly called out of the country,” according to his email auto-response.

Update 11/9/15 10:13 a.m. eastern: Kip filled us in on some more details on how he discovered the issues with the paper:

In terms of history, we (the authors) were first contacted by researchers from the RAND Corporation. They reviewed our paper and noticed the error in the calculation of the variance. They were quite concerned because they are in the midst of a survey of 150,000 veterans initiated by Congress, and our estimate of male MST from this small sample did not comport at with theirs. They provided some calculations with the correct variance estimation procedure. This led the co-authors to consider these new calculations. Also, I personally asked for a copy of the data and ran some calculations. From these calculations, I too became concerned about the lack of reliability of the estimate of the apparent under-reporting of male MST. Whereas the RAND researchers originally suggested that a retraction would be in order, I believe from our email correspondence that I was first of the authors to formally recommend this after seeing the data myself – all of the remaining authors agreed.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post. Click here to review our Comments Policy.