The Annals of Emergency Medicine has republished a controversial paper it withdrew earlier this year which compared the cost of emergency care at different types of facilities.
Because the paper drew heavy criticism when it was originally released, the journal has published a revised version, along with several editorials and discussions between the authors and critics. One point of contention: The analysis stems from data provided by an insurance company — Blue Cross Blue Shield — which it declined to share.
The paper — originally published in February — caught national attention (and raised concerns among some emergency care providers) when it reported the cost of treatment in emergency departments can be significantly higher than at urgent care centers, even for the same conditions. The journal withdrew the paper in spring, and re-published it Tuesday, with minor changes.
First author Vivian Ho at Rice University told us she made “slight changes” to some headings, phrases, and the appendix, but:
the main results stay the same…I’m very happy the article has been re-released.
However, Ho added that she was “disappointed” the journal felt it had to publish so much extra “wording, and verbiage, and various editorials.”
Journal editor Michael Callaham at the University of California, San Francisco, told us:
We decided not to retract, and instead address and share the issues directly…We could not conclusively validate or exclude all the findings.
The findings have not been independently validated because the paper is based on data provided by health insurer Blue Cross Blue Shield, which it declined to share. Callaham told us:
…the entire analysis is still based on that data. We received an attestation of sorts from the Texas branch that they “stood behind the data”, but our efforts to find out if [Blue Cross Blue Shield] would allow impartial third party review of the data, went unanswered. And it certainly concerns me, but as you can see our efforts to resolve it, although very labor intensive, were unsuccessful.
Ho told us she has no doubts about the data:
I’m not at all concerned about the validity of Blue Cross Blue Shield’s data. I’ve seen their code, I’ve discussed the results extensively.
Both she and Callaham noted that other studies have relied on commercial insurance data to arrive at conclusions about clinical care; Callaham told us:
As far as I know, that data was not reviewed by anyone else either. This issue is a concern in our research community and has been discussed a good deal in our last several editorial board retreats, without arriving at any practical and moderately efficient solution.
Ho said insurers often can’t share the raw data due to issues with patient consent. It’s also an economic problem, she added, since insurers don’t want individual treatment centers to be able to compare reimbursements, which can vary significantly.
Conflicting reports
“Comparing Utilization and Costs of Care in Freestanding Emergency Departments, Hospital Emergency Departments, and Urgent Care Centers” reviewed insurance data from places that offer emergency care in Texas: hospitals, freestanding emergency departments, and urgent care centers. It found emergency departments charge sometimes 10 times more to treat the same conditions as urgent care centers. (Texas has the most freestanding emergency departments of any state.)
The original article quickly raised concerns — for instance, some argued costs can vary depending on patient severity. Ho said some values could be influenced by patient care, but one table in the paper compares procedures, which are not influenced by patient severity; that table shows a routine urinalysis cost $51 in a freestanding emergency department, and $3 at an urgent care clinic.
Ho told us she believes some of the criticisms have been politically motivated — specifically, that some critics “worked really hard to make this article withdrawn” while the Texas legislature (which meets only a few months every two years) was considering legislation around emergency care. Ultimately, the governor signed a bill that makes it easier for patients to challenge their bills following emergency care, and expands requirements to disclose if facilities are out of a patient’s insurance network.
One major critic of the paper has been Paul Kivella, president-elect of the American College of Emergency Physicians, which owns the journal. In one of his letters asking to retract the paper, he writes:
…we have grave concerns about the study’s methodology, the integrity of the data, the potential conflicts of interest on the part of the authors, and the possible serious consequences that the report’s flawed conclusions and recommendations could have in terms of supporting policy changes that could jeopardize access to emergency care and undermine the “prudent layperson” standard.
Before republishing the paper (a decision already reported by Health Data Buzz), it underwent extensive review, according to Callaham’s editorial:
This unusually extensive review took the energies and expertise of 4 peer reviewers, 1 regular editor, and 4 expert editorialists. Additionally, the editor in chief and 4 deputy editors, who have a collective total of more than 86 years of experience in high-level editorial decisionmaking, were engaged in many hours of discussion…After digesting all the assessments and arguments, we concluded the article had some original and important data to report (on a topic with a paucity of such information) and believed it should be published. But because the concerns were also important and could not be conclusively resolved, they should be published too.
Callaham’s editorial acknowledges that some critics had a range of conflicts of interest (in addition, some of the co-authors on the paper itself are based at Blue Cross Blue Shield):
These potential conflicts of interest included those of some individuals who had financial interest in freestanding EDs, and of others who participated in insurance company marketing campaigns to the public to limit freestanding ED use. These campaigns have led to the announcement in some states that Blue Cross Blue Shield will deny claims after the fact for ED visits that in their judgment could have equally well been cared for in an urgent care center. Obviously, this insurer could benefit directly in instructing patients to use less expensive care, so the integrity of the raw data used in this analysis may be more crucial than the analysis itself.
Kivella, for instance, reports that he is an investor in a facility that operates four freestanding emergency centers, and lectured at the National Association of Freestanding Emergency Centers conference last year.
Here are links to the other documents published alongside the revised paper and editorial:
- “Comparing Freestanding Emergency Departments, Hospital-Based Emergency Departments, and Urgent Care in Texas: Apples, Oranges, or Lemons?”
- “Original allegations about data integrity by emergency physicians”
- “Additional information regarding request for retraction of Ho article”
- “Response by Dr. Ho to Dr. Kivela’s allegations”
- “Rebuttal by emergency physicians to Dr. Ho response”
- “The Prudent Layperson’s Complicated and Uncertain Road to Urgent Care – editorial by Dr. Callaham on patient decision making in choosing urgent care”
- “Don’t Hate the Player; Hate the Game”
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.