An exercise scientist who ran a study of the CrossFit exercise program without an approved human subjects protocol has lost two more papers to retraction.
Both papers were retracted on June 26 by the editors of the International Journal of Exercise Science (IJES) with the agreement of last author Steven Devor, a former professor at The Ohio State University. Both have been retracted because the studies were carried out without proper IRB approval.
Earlier in June, another paper from the CrossFit study — which is still at the center of a legal battle between CrossFit and a competitor in the market for exercise instructor licensing — was retracted by the Journal of Strength and Conditioning Research (JSCR) for improper IRB approval. Devor resigned from OSU the day after the first retraction.
Only one of the newly retracted papers had anything to do with the CrossFit study: Published in April 2014, this paper suggested that the so-called “paleo” diet — a diet focused on meat and vegetables — is associated with “unfavorable” changes in cholesterols and other blood-based cardiovascular biomarkers. In addition to following the diet, subjects also participated in a CrossFit exercise program.
The other was a case study of the effects of running a marathon and had no connection to CrossFit. But, like the two CrossFit papers, its first author was Devor’s frequent collaborator Michael Smith, now at California State University-Chico.
Both IJES retraction notices replaced the original papers and simply read:
Manuscript has been retracted.
The Committee on Publication Ethics guidelines for retraction notices recommend leaving the originals and creating separate notices, though we were able to find a pdf copy of the Paleo diet paper through the Internet Archive. Co-Editor-in-Chief Scott Lyons, of Western Kentucky University told us this was the journal’s first time retracting a paper:
We thought that was appropriate. Retraction, by definition, means to withdraw, so we thought removing them was the right thing to do. Putting up a notice that these papers have been retracted, yet leaving the papers posted for everyone to see, seems like “piling on” to the authors when that is not necessary. I think they have been punished enough.
COPE also recommends that notices include information on who retracted the papers and the reasons for doing so. Lyons said “time was the primary factor” for making the decision to post the brief notices:
More detail might be added if we think it is necessary…and we will certainly review the best practices.
IJES has not yet been indexed by Clarivate Analytics’ Web of Science.
The JSCR CrossFit paper, which contained false injury statistics, has provoked several lawsuits. Readers may recall that Mitch Potterf, owner of the Columbus, Ohio gym where Devor conducted the study, settled a lawsuit with OSU in October 2016 and that CrossFit headquarters is involved in ongoing state- and federal-level suits with the National Strength and Conditioning Association, which publishes JSCR.
In response to our story on the JSCR retraction, commenter Yuri Feito (astutely) pointed out that “Unrestricted Paleolithic Diet is Associated with Unfavorable Changes to Blood Lipids in Healthy Subjects” might not have the proper IRB approval, too.
Retraction Watch contacted the editors of IJES on June 19, which prompted the journal to initiate the retraction process, Lyons told us:
That email is what triggered us to start to talk about this.
Lyons told Retraction Watch that he first discussed retracting the paper with the journal’s executive committee, but then contacted Devor directly:
We talked about the whole situation and that [retraction] was going to be best for all parties. He understood completely and didn’t have any issue at all with us retracting those papers. After discussion with him and the executive committee, it was a pretty easy decision.
Lyons said that Devor explained the problems with the papers in a letter. Lyons declined to share the full letter, but told Retraction Watch that Devor wrote:
The above referenced studies were not conducted under an OSU IRB protocol that was specifically approved for each individual study.
The second paper, originally published in July 2013, was titled “Endothelial Response of Running a Marathon: A Tale of Three Runners.”
Lyons added that he had served on Western Kentucky’s IRB, which he believed gave him insight as to how the study ran afoul:
I do see how it could have happened. I don’t think it was intended to be malicious, they merely collected data under a different IRB protocol that had already been approved, but not specifically approved for these two studies.
Technically, that is a violation.
Hat tip: Russell Greene
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
Thank you for the coverage.
The Paleo study also made false claims about the nutritional intervention followed, beyond the IRB / informed consent issue.
Namely, the study reports that the subjects strictly followed the Paleo diet, but they did not, and were not even instructed to. More info here: https://therussells.crossfit.com/2015/01/19/paleo-fraud-at-the-international-journal-of-exercise-science/
So any changes to blood lipids that occurred cannot be attributed to the Paleo diet. Like the retracted NSCA CrossFit study, the Paleo study was thoroughly discredited by gym owner Mitch Potterf and the study’s other subjects.
I informed the International Journal of Exercise Science of these issues two years ago and they declined to issue an erratum at the time. They instead took the word of Devor and Smith over the gym owner.
The Paleo retraction and NSCA CrossFit retraction both occurred 2+ years late and reflect poorly on the journals responsible for them, not just Devor and Smith.
While the paper may be a bogus paper, the fact that the paleo-diet is very bad for the body is a no-brainer. The paleo-diet is not what people in prehistoric times ate, it’s not what the human body has evolved to be adapted to. As mentioned here:
https://www.theguardian.com/society/2017/mar/17/tsimane-of-the-bolivian-amazon-have-worlds-healthiest-hearts-says-study
precisely the opposite of the paleo-diet is healthy:
“A high carbohydrate diet of rice, plantain, manioc and corn, with a small amount of wild game and fish – plus around six hours’ exercise every day – has given the Tsimané people of the Bolivian Amazon the healthiest hearts in the world.”
““Most of the Tsimané are able to live their entire life without developing any coronary atherosclerosis. This has never been seen in any prior research. While difficult to achieve in the industrialized world, we can adopt some aspects of their lifestyle to potentially forestall a condition we thought would eventually effect almost all of us.””
And what about the Eskimos who eat a high fat, high meat, low carb diet, they are free of heart disease too! No, that’s not true, see here:
http://www.onlinecjc.ca/article/S0828-282X(14)00237-2/fulltext
“During the 1970s, 2 Danish investigators, Bang and Dyerberg, on being informed that the Greenland Eskimos had a low prevalence of coronary artery disease (CAD) set out to study the diet of this population. Bang and Dyerberg described the “Eskimo diet” as consisting of large amounts of seal and whale blubber (ie, fats of animal origin) and suggested that this diet was a key factor in the alleged low incidence of CAD. This was the beginning of a proliferation of studies that focused on the cardioprotective effects of the “Eskimo diet.” In view of data, which accumulated on this topic during the past 40 years, we conducted a review of published literature to examine whether mortality and morbidity due to CAD are indeed lower in Eskimo/Inuit populations compared with their Caucasian counterparts. Most studies found that the Greenland Eskimos and the Canadian and Alaskan Inuit have CAD as often as the non-Eskimo populations. Notably, Bang and Dyerberg’s studies from the 1970s did not investigate the prevalence of CAD in this population; however, their reports are still routinely cited as evidence for the cardioprotective effect of the “Eskimo diet.” We discuss the possible motives leading to the misinterpretation of these seminal studies.”
It’s our love for fatty foods that’s distorting the science, we could have known 60 years ago that what the Tsimané eat is healthy and what the Eskimos eat is very unhealthy, e.g. we can read in this reprint of the landmark 1959 Lancet article that made the link between heart disease and cholesterol:
https://academic.oup.com/ije/article/41/5/1221/712631/Serum-cholesterol-diet-and-coronary-heart-disease
“In the African population of Uganda coronary heart disease is almost non-existent. This statement is confirmed by adequate necropsy evidence [1]. In the Asian community, on the other hand, coronary heart disease is a major problem.”
The article points out what the differences in the diet are and the differences in cholesterol levels.
So, while it’s bad to have bogus papers published, what’s far more damaging is the 60 years worth of bogus research that is conducted according to the rules, but which contain results that are outright misleading. Hundreds of millions of people have died from heart attacks and strokes just because we have been unable to do proper research.
The Tsimane people have the healthiest hearts in the world according to the article you cited. I would like to suggest to you that the “around six hours’ exercise every day” plays a significant role in that and so you can’t claim that it is the diet alone that is responsible for their healthy hearts.
Similarly with the Ugandan example, the African people – with the almost non existant CAD – are noted by that article as being employed in ” unskilled and menial work.” In other words, they are regularly, physically active while their Asian counterparts in this study “provide most of the skilled labour and business enterprise and are a major source of professional skills,” or in other words, they are much less physically active. So again I suggest to you that this difference in physical activity contributes significantly to the difference in CAD rates.
So I agree with you when you state that it is bad to “have bogus research published” and to conduct “bogus research” in the first place, but it’s also equally bad to draw incorrect conclusions from properly conducted research.
I would be keen to read a report on CAD rates for groups of people eating different diets while also being equally physically active.
The conclusions about the diets are drawn in the papers themselves. Note that we do have results on other indigenous populations like the Eskimo who eat a high fat diet and who are physically very active. They do get atherosclerosis. Physical activity is still very important for heart health but you need to eat a very healthy diet to prevent atherosclerosis.