A psychology journal is retracting a 2015 paper that attracted press coverage by suggesting women’s hormone levels drive their desire to be attractive, after a colleague alerted the last author to flaws in the statistical analysis.
The paper, published online in November, found women prefer to wear makeup when there is more testosterone present in their saliva. The findings were picked up by various media including Psychology Today (“Feeling hormonal? Slap on the makeup”), and even made it onto reddit.com.
However, upon discovering a problem in the analysis of the data, the authors realized that central finding didn’t hold up, according to Psychological Science‘s interim editor, Stephen Lindsay:
Last month, senior author Benedict Jones notified the Associate Editor who had served as Action Editor on this manuscript, explaining that he (Jones) had heard from another psychologist that there was a flaw in the central statistical analysis. Jones, having looked into the matter, agreed that there was indeed a flaw and that the central finding did not hold up when the flaw was corrected and therefore asked the AE to arrange for retraction. The AE relayed this information to me along with his judgment that the author was correct, and I judged that retraction was appropriate and worked with Jones and APS staff on a retraction statement.
Lindsay provided more details about how the problem was discovered:
The error had to do with a fine point in the specification of a linear mixed model analysis. The authors included in the supplemental online material all of the details of the model so in principle we should have caught it. I understand that the person who detected the error did so because he re-analyzed the data, which Fisher et al. had posted on [the Open Science Framework] OSF, as part of a statistics course.
Lindsay gave us a heads up about the retraction before it happened, along with an explanation of what went wrong, so he (and the authors of the paper) definitely belongs in our “doing the right thing” category. As he told us:
We are continuing to work on ways to detect these sorts of errors before they are published. But when that fails we do our best to take corrective action and get the word out quickly.
Last author Benedict Jones at the University of Glasgow confirmed this account to us:
Yes, we asked the journal to retract the article after a colleague had alerted us to a problem with the analysis. We had made the data and our analysis files available through OSF.
The colleague drew our attention to a 2013 paper (Barr et al., 2013 Journal of Memory and Language) that shows the type of analysis we reported can inflate the Type 1 error rate. Reanalyses that addressed this issue following recommendations in the 2013 paper did not show the key effect reported in our original article.
Here’s a link to the paper Jones mentions.
(Disclosure: Our parent organization, The Center For Scientific Integrity (CSI), is partnering with The Center For Open Science (COS) to create our retraction database on the Open Science Framework, or OSF.)
Here’s more from the abstract of the to-be-retracted paper, “Women’s Preference for Attractive Makeup Tracks Changes in Their Salivary Testosterone,” which has not yet been cited, according to Thomson Scientific’s Web of Knowledge:
We found that women’s preference for attractive makeup increases when their salivary testosterone levels are high. The relationship between testosterone level and preference for attractive makeup was independent of estradiol level, progesterone level, and estradiol-to-progesterone ratio. These results suggest that testosterone may contribute to changes in women’s motivation to wear attractive makeup and, potentially, their motivation to appear attractive in general.
Update 2/5/16 10:33 a.m. eastern: The retraction has been posted. It reads:
At the request of the authors, the following article has been retracted by the Editor and publishers of Psychological Science:
Fisher, C. I., Hahn, A. C., DeBruine, L. M., & Jones, B. C. (2015). Women’s preference for attractive makeup tracks changes in their salivary testosterone.Psychological Science, 26, 1958–1964. doi:10.1177/0956797615609900
The authors of this article have notified the Editor as follows:
Our article reported linear mixed models showing interactive effects of testosterone level and perceived makeup attractiveness on women’s makeup preferences. These models did not include random slopes for the term perceived makeup attractiveness, and we have now learned that the Type 1 error rate can be inflated when by-subject random slopes are not included (Barr, Levy, Scheepers, & Tily, 2013). Because the interactions were not significant in reanalyses that addressed this issue, we are retracting this article from the journal.
It also includes a note from the editor at the bottom:
Editor’s Note I would like to add an explicit statement that there is every indication that this retraction is entirely due to an honest mistake on the part of the authors.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy.