Researchers have retracted and replaced a June 2016 paper in JAMA Internal Medicine after discovering errors in their data.
The paper explored whether Accountable Care Organizations (ACOs) — groups of health care providers who earn more when they deliver high-quality care without boosting costs — improve care and lower health care costs for Medicare patients. The paper’s corresponding author, Carrie H. Colla, and her colleagues examined Medicare data over five years and found the ACOs provided “ modest savings on average” and less hospital care.
But the data from the Centers for Medicare & Medicaid Services (CMS) contained errors. According to Colla, after the paper was published, CMS “let us know in the fall [2016] that there were errors in the files, but weren’t able to give us final replacement files until winter.”
When Colla, associate professor at The Dartmouth Institute for Health Policy & Clinical Practice in Lebanon, N.H., and her co-authors received the corrected files, they re-analyzed the data and subsequently contacted the journal about the issue. Colla told us the changes were “extremely subtle” and “do not warrant changes to our conclusions in the paper.” But, according to the retraction notice, the updated analysis did lead to significant changes to some specific results.
CMS responded to a request from Retraction Watch last week for details but said it would need more time to provide them.
Here’s the retraction notice for “Association between Medicare accountable care organization implementation and spending among clinically vulnerable beneficiaries:”
We write to report and explain errors that occurred in the Original Investigation, titled “Association Between Medicare Accountable Care Organization Implementation and Spending Among Clinically Vulnerable Beneficiaries,”1 that was published online on June 20, 2016, and in the August 2016 issue of JAMA Internal Medicine. The article reported the results of a cohort study designed to estimate the association between Medicare accountable care organization (ACO) contracts with spending and high-cost institutional use for the overall Medicare population and a clinically vulnerable subgroup of Medicare beneficiaries from January 2009 through December 2013. The main outcome measures of our study were total spending per beneficiary-quarter, spending categories, use of hospitals and emergency departments, ambulatory care–sensitive admissions, and 30-day readmissions. We determined that the Medicare ACO programs were associated with modest reductions in spending and use of hospitals and emergency departments and that savings were realized through reductions in use of institutional settings in clinically vulnerable patients.
After our article was published, the Centers for Medicare & Medicaid Services (CMS) Research Data Assistance Center informed us that there were errors in the Medicare Shared Savings Program provider files, which we had used to determine attribution of Medicare beneficiaries to physician practices. The CMS issued corrected versions of the files, and we used these corrected files to rerun attribution across the 5-year study period and reestimated the main models. This reanalysis results in corrections to the data reported in our article. The corrections have yielded similar overall findings that Medicare ACO programs were associated with modest reductions in spending and use of hospitals and emergency departments. However, there are some important changes to some of the specific results:
- There is no reduction in emergency department visits in the clinically vulnerable cohort.
- There is no reduction in skilled nursing facility spending in the overall Medicare cohort.
- In both cohorts, there is now a small but statistically significant increase in 30-day readmissions.
- Spending did not vary monotonically with length of ACO implementation.
- The difference between mean spending in the ACO group and the control group is larger.
Thus, we have requested that our article be retracted and replaced with the correct data and findings. The replacement article has corrected data and information in the abstract, text, Tables, Figures, and eFigures in the online supplement.
We regret any confusion the errors in the CMS data may have caused the readers and editors of JAMA Internal Medicine. The abstract, text, Tables, Figures, and eFigures in the original article have been corrected and replaced online.1 An additional online supplement has been added that includes a version of the original article with the errors highlighted and a version of the replacement article with the corrections highlighted.
The 2016 paper has been cited five times, according to Clarivate Analytics’ Web of Science.
We asked Rita Redberg, editor of JAMA Internal Medicine, whether the journal would have accepted the new version of the paper. Redberg told us:
Yes, we would have accepted this new version. That is why we have Retraction and Replacement – to allow for correction of serious honest error when the science is still valid.
And she said:
… the errors in the analysis were inadvertent and due to a CMS error in data supplied.
Last year, Executive Managing Editor for The JAMA Network Annette Flanagin told us JAMA journals may expand the use of retract and replace in papers affected by honest error, and this month they’ve made good on that promise. Along with this paper, we’ve reported two other retract and replaces in JAMA journals—one for a 2017 JAMA paper analyzing penalties U.S. hospitals face and one for a 2012 JAMA Pediatrics paper by high-profile food researcher Brian Wansink exploring children’s food preferences.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.