Lawsuits are usually dry and boring, so it’s always fun to read one with a little life.
Here’s one of those: CrossFit, the fitness program famous for its brief, strenuous exercises and passionate devotees, is suing the National Strength and Conditioning Association (NCSA), which it considers its staid competitor for the nation’s sweat and cash.
According to CrossFit, the NSCA published a study with a “falsified rate of injury,” “in an effort to portray CrossFit as ‘dangerous’ and therefore a fitness program that should be avoided.”
No matter that the study, published in NSCA’s official research journal, the Journal of Strength and Conditioning Research, concluded overall that CrossFit is a useful form of exercise. The suit says that the authors fudged a few statistics about participants’ injuries. Here’s the relevant section from the paper, titled “Crossfit-based high-intensity power training improves maximal aerobic fitness and body composition:”
Out of the original 54 participants, a total of 43 (23 males, 20 females) fully completed the training program and returned for follow up testing. Of the 11 subjects who dropped out of the training program, two cited time concerns with the remaining nine subjects (16% of total recruited subjects) citing overuse or injury for failing to complete the program and finish follow up testing.
And from Russell Berger, a CrossFit trainer who writes for the company’s blog:
What is “overuse or injury”? The study does not define what it means by the term “overuse.” The study also does not detail what specific cases of “overuse or injury” the subjects cited, what caused them, whether the cases were pre-existing conditions, or how long the subjects experienced “overuse or injury.”
Furthermore, the study was “blind,” meaning the researchers in the lab were only able to identify participants by a single number. If the 11 subjects who failed to show up for the test-out were de-identified in this way (and obviously not present at the Ohio State lab), how could Dr. Smith collect any data on the reason for their absence?
Chelsea Rankin, a member of CrossFit 614, volunteered to be the study coordinator for Dr. [Michael] Smith. During our conversation, I asked Rankin how Dr. Smith could have gathered data on why the 11 didn’t show up to the lab.
Rankin gave me her own opinion on Dr. Smith’s work: “I did all the data collection for the study, and I know every person who didn’t re-test. It was easy to figure out they weren’t injured. This data is inaccurate. Those individuals were not injured, and that wasn’t the reason they didn’t test out. To me this questions the validity of the research.”
Berger published a transcript of a phone conversation he had with the paper’s last author, Ohio State University’s Steven Devor, which the lawsuit cites extensively:
Dr. Devor was made aware of the discrepancy between the “overuse or injury” findings in the Devor Study and the responses of the participants to inquiries by the CrossFit affiliate owner and CrossFit’s own representative. In an April 23, 2013 telephone conversation with Russell Berger of CrossFit, Dr. Devor admitted that his team conducted a “blind study” and, therefore, did not know the identity of the nine participants who did not return. In fact, Dr. Devor stated that he did not collect any of the data at all. He said the data was collected by Dr. Smith.
The lawsuit goes on to cite a number of publications that have cited the Devor study as proof that CrossFit is dangerous, and to ask for both monetary compensation and for the court to:
order the recall of all copies of any version of the Devor Study and any excerpt or portion thereof, including disabling copies available via the Internet over which the NSCA has control.
That seems like a lot of words to use instead of just saying “retract the Devor Study.” The NSCA has responded to the suit, as Berger reports. In his words:
…the NSCA claims they don’t have enough information to have an opinion about the Devor study, its data, or the study’s validity to affirm or deny our claims about it.
We’ve reached out to CrossFit’s lawyers and Devor, and will update with any further information.
Hat tip: Ben Gallarda
On the face of it this is an absurd complaint. The paper does not claim that 9 participants were injured but rather that they “cited overuse or injury” as their excuse for not completing the program.
So all of the grand-standing under the heading of “what is overuse or injury”? should be addressed to the participants, not the researchers, who were only reporting what they’d been told.
It may well be that those individuals weren’t actually injured, but the paper never claimed that they were. Presumably they used injury as a cover for laziness, or they believed they were injured when they weren’t.
…or they *were* injured, just not due to the excercises.
One obvious issue in the Devor paper is that they state “In spite of a deliberate periodization and supervision of our Crossfit-based training program by certified fitness professionals, a notable percentage of our subjects (16%) did not complete the training program and return for follow-up testing. While peer-reviewed evidence of injury rates pertaining to high intensity training programs is sparse, there are emerging reports of increased rates of musculoskeletal and metabolic injury in these programs(1)”
In my reading, this section directly links the non-completion of the 9 “overuse/injury” to injury related to the excercises.
@NeuroSkeptic – injury and overuse in strength sports are a tricky subject. It might be that participants could not cope with the delayed onset muscle soreness, maybe they felt bone tired, maybe the way in which they used their bodies aggravated old injuries … we need to see the protocol for collecting reasons for drop out to make a final judgement.
@Marco – the authors had to say something about the high rate of “overuse and injury” comments. They did not state that they had established that Crossfit was the cause of injury.
Let the jury decide based on the evidence!!!! Its called evidence based!!!
What is? The credibility of the Columbus, OH, CrossFit affiliate who purports to have breached the blinding? Whether the “Consortium for Health and Military Performance and American College of Sports Medicine Consensus Paper on Extreme Conditioning Programs in Military Personnel” paper sufficiently examined CrossFit? Whether “the Hak Study” did “not adhere to scientifically valid principles”?
If they can prove the data was faked then they may have a case.
And they are not asking for the paper to be retracted, but disappeared. They want all physical copies destroyed and all electronic copies removed from the internet.
If the data is junk it should disappear.
No, if the data is junk then it should be retracted and an investigation started at the relevant institution. But retracted papers should not be disappeared because then no one will known there was misconduct. Right?
In case anyone loathes Scribd as much as I do, I’ve updated the RECAP docket (please use this plugin if you are downloading from PACER), including the complaint, answer, and settlement-conference order. I haven’t read any of it yet.
I’ve read the paper – there was no blinding because it’s a single-group repeated-measurements descriptive study. There’s nothing to be blinded to (maybe the tech doing the VO2 measurements wasn’t told the subjects were CrossFitters?). Central study staff would have had contact information for all participants, and could (should) have followed up with noncompleters without involving the volunteer study coordinator. CrossFit’s people don’t seem to understand that this paper basically says that CrossFit works a treat. Clearly the PIs needed to do a better job communicating with CrossFit about likely outcomes, but CrossFit is not coming off looking good, here.
This whole thing is absurd and should be dropped immediately. It looks like the injury claim is basically a foot note in the paper concerning the outcome of the study design, and has nothing to do with the overall positive message favoring Cross-fit. But I must say that I find it entirely believable that nine of the subjects dropped due to injury. I know several people who have sustained serious injuries doing cross-fit, including one colleague who spent a few weeks in a hospital followed by months of physical therapy. I wonder when Cross-fit will file a lawsuit against my colleague for telling people how he was injured.
By the way CrossFit, the appropriate response to the study would be “We’re excited that the NCSA has empirically found that our program works. We are concerned about the high percentage of reported injuries, and will emphasize the importance of proper form and appropriate rest in our program materials in the future.” Any PR person could tell you that a positive message is best.
Someone needs to pass this along to the Crossfit PR folks: http://en.wikipedia.org/wiki/Streisand_effect
Here is the full “Subjects” section:
“The participants of all levels of aerobic fitness and body composition were recruited from and trained at a Crossfit affiliate (Fit Club, Columbus, OH, USA). Out of the original 54 participants, a total of 43 (23 men, 20 women) fully completed the training program and returned for follow-up testing. Of the 11 subjects who dropped out of the training program, 2 cited time concerns with the remaining 9 subjects (16% of total recruited subjects) citing overuse or injury for failing to complete the program and finish follow-up testing. The subjects had already been following a “Paleolithic” type of diet before and after completion of the training protocol. All the subjects provided written informed consent, and all study methods and protocols were approved in advance by the Institutional Review Board at The Ohio State University.”
The authors must report the number of non-completers. In the study methods and protocols submitted to the Ohio State University IRB, there may well have been a section on following up non-completers, because the reasons for dropping out early are often key to meaningful analysis of the results. It would be interesting to see what the protocol for that follow up was, whether it was a questionnaire or a structured interview.
Given the high percentage of non-completers, the authors must also discuss this finding in the Discussion section of their article. Anecdata about CrossFit injury rates and injury severity have been swirling around the Internet for a long time; the authors substantiate their concerns by referring to a consensus report.
I’m with failuretoreplicant – we have a massive Streisand effect here.
What worries me far more is that the CrossFit affiliate actually followed up with participants who dropped out. I wonder about the tone of those “discussions”.
Russell Berger mentions assistance from “scientist” Jeff Glassman PhD, ostensibly Greg Glassmans’s brother. (Greg Glassman founded Crossfit). Years ago daily postings of the Crossfit workout were accompanied by gratuitous right wing messages from Greg Glassman. These included arguments by Jeff Glassman PhD, that climate change and global warming didn’t exist.
Now Crossfit has grown to over 1000 affiliates but they obsess with a minor point in a few academic fitness journals suggesting that Crossfit ‘may’ be dangerous. How insecure can someone be? By the way I have been doing Crossfit for years and think it’s a great program, but I also know one becomes ‘certified’ to teach Crossfit by attending a weekend seminar – not exactly a high standard.