Retraction Watch

Tracking retractions as a window into the scientific process

JAMA authors retract (and replace) paper about moves and kids’ mental health

with 4 comments

JAMAJAMA authors have retracted — and replaced — a 2014 paper about the mental health effects of household moves on kids, after they found errors while completing an additional analysis.

The original paper concluded that in “families who moved out of high-poverty neighborhoods, boys experienced an increase and girls a decrease in rates of depression and conduct disorder,” according to a press release issued by the journal along with the paper (which also got some press attention from Reuters). But part of that conclusion is wrong.

The authors write in a notice for “Associations of Housing Mobility Interventions for Children in High-Poverty Neighborhoods With Subsequent Mental Disorders During Adolescence” that:

we inadvertently reported incorrect confidence intervals and a P value in 2 tables. This study explored the associations between 2 types of vouchers given to volunteer public housing families to encourage them to move out of high-poverty neighborhoods (when children were 0-8 years old) and no intervention and subsequent mental disorders in 2872 adolescents (at age 13-19 years).

The errors were due to failure to update results from an earlier set of models. These errors were discovered in the course of rechecking the code in conjunction with a secondary analysis. We have corrected these errors and confirmed that there are no other errors after reviewing our original analysis and findings. The corrections for these errors have changed 1 of the major findings of the study: the previously reported statistically significant reduction in major depressive disorder in girls was not statistically significant. Thus, we have requested that the original article be retracted and replaced.

The paper has been cited 17 times, according to Thomson Reuters Web of Science. The retracted article — with errors highlighted — is available in the supplemental content under the new version.

In the notice, the authors explain the correct data in detail — and how they alter the conclusion:

The corrections for these errors indicate that the previously reported statistically significant reduction in major depressive disorder in girls was not statistically significant, and this result has been removed from the conclusion of the article. The article now concludes: “Interventions to encourage moving out of high-poverty neighborhoods were associated with increased rates of depression, [post-traumatic stress disorder], and conduct disorder among boys and a reduced rate of conduct disorder among girls. Better understanding of interactions among individual, family, and neighborhood risk factors is needed to guide future public housing policy changes.”

 They note that their errors could have effects outside of academia:

We regret these errors as well as the confusion caused to JAMA, readers, and potentially to public housing policy planners.

The finding that boys in households receiving moving vouchers experience an increase in PTSD — which remains the same in the replacement paper — was the subject of a 2014 Comment and Response in JAMA. Another researcher calls the finding “paradoxical;” in his response, first author Ronald Kessler, who works at Harvard University, notes it’s long been known that:

community contexts can influence interpretations of traumatic experiences and that the effects of these experiences on PTSD vary with the extent to which the experiences shatter deeply held world views about such things as community safety and justice.

Although we don’t see many journals retract and replace papers, it’s become more common at JAMA journals. For instance, in April, JAMA Psychiatry did the same for a study about depression that failed to account for some patient recoveries, among other mistakes. In our post about that retraction, we quoted Annette Flanagin, the Executive Managing Editor for The JAMA Network, who pointed us to an editorial that explains the decision. She told us in April:

As we stated when we announced our policy on use of retraction and replacement in an Editorial 2015, “Retractions are typically reserved for articles that have resulted from scientific misconduct, such as fabrication, falsification, or plagiarism, or from pervasive error for which the results cannot be substantiated”…In this case, inadvertent errors resulted in changes to some of the findings, although the general conclusions of the study are unchanged. As we also noted in that Editorial, errors do occur, and if the errors are pervasive and result in a major change in the direction or significance of the findings, interpretations, and conclusions, and the science is considered reliable, we will consider retraction and replacement as an effective approach to ensuring transparency and an accurate scientific record.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.

Comments
  • Seth Latimer June 18, 2016 at 9:59 am

    Is there anywhere in the academic record where the authors respond to the serious methodological criticism from the recent Norris paper http://f1000research.com/articles/5-1014/v1 I don’t understand how JAMA could allow for a republication of the same methods without a response, at minimum a citation.

    • Marco June 18, 2016 at 11:34 am

      I don’t know of many journals who would ask people to respond to or cite a non-peer reviewed paper. Kessler may also believe he already responded to Norris, a response that the f1000-paper does not cite.

      • David C. Norris June 18, 2016 at 1:07 pm

        Marco, thank you for reading our reference list; please do read the paper as well. You’ll see (p. 2, left column, 3rd paragraph down) that the 2014 Comment & Response — which was about *construct validity* — is only incidentally related to the matter of *statistical conclusion validity* that we address in our reanalysis. Thus, no need to cite that reply in our F1000Research reanalysis.

  • David C. Norris June 18, 2016 at 5:13 pm

    The description above of my half of that 2014 Comment & Response seems to owe more to Kessler’s mischaracterization of it in his ‘reply’ than to an actual reading of my Letter. Each of my 2 uses of ‘paradoxical’ was conspicuously preceded by a qualifying ‘apparently’. The bulk of the letter was in fact devoted to suggesting *how* the “apparently paradoxical voucher effects” might have arisen from “predictable limitations of the instruments used” in Kessler’s study. That is, my letter did *precisely the opposite* of calling Kessler’s PTSD and conduct disorder findings ‘paradoxical’ — it argued that they are only *apparently* paradoxical, and explained why. (As I have detailed in a PubMed Commons comment, such elision of key qualifying words from my letter played a essential role in Kessler’s reply: http://www.ncbi.nlm.nih.gov/pubmed/25005663#cm25005663_16261.)

    As far as I can tell, the ‘construct validity’ concerns I raised in that letter are related to the present Retraction & Replacement ONLY through the following temporal sequence of events:

    1. While working to specify a protocol for investigating the (unanswered) construct validity questions of my Letter, I discovered that an undocumented PTSD imputation procedure had been used by Kessler et al — one that seemed likely to undermine their study on the level of ‘statistical conclusion validity’.

    2. Together with Andy Wilson, I undertook a formal reproduction and reanalysis of the PTSD result, which we formally announced to the Kessler team (the JAMA Editorial Office was cc’ed) on August 12, 2015. This work was recently published by the open-access journal F1000Research, and awaits peer review: http://f1000research.com/articles/5-1014/v1.

    3. In the process of assisting us with our reanalysis, the Kessler team assembled and re-ran their SAS code, and (as I understand) in this process discovered the errors that led to this Retraction & Replacement. I suspect (but am not certain) that the “secondary analysis” to which the Retraction Notice refers would be our reanalysis.

    Otherwise, the construct validity discussion in that 2014 Comment & Response is unlikely to be related to this present Retraction & Replacement, which I understand to relate to purely mechanistic difficulties in running a complex cascade of SAS programs.

  • Post a comment

    Threaded commenting powered by interconnect/it code.