Paper used to support ban on Caster Semenya competing earns massive correction

The authors of a controversial paper on what constitutes “normal” hormone levels in men and women  — and, by implication, “male” and “female” athletes — are set to issue a massive correction of the work, Retraction Watch has learned. But an outside, albeit not disinterested, researcher who prompted the correction says the correction itself is amiss. 

The article, “Large divergence in testosterone concentrations between men and women: frame of reference for elite athletes in sex-specific competition in sports, a narrative review,” appeared earlier this year in Clinical Endocrinology, a Wiley title. The authors of the meta-analysis, led by Richard Clark, of the US Anti‐Doping Agency, argued that testosterone levels between the sexes do not overlap in the absence of chromosomal or genetic anomalies that blur the distinction between male and female. 

That finding was cited recently by an architect of the International Association of Athletics Federations’ decision to bar the South African trackstar, Caster Semenya, and other “hyperandrogenic” women (Semenya’s hormonal status has not been made public) whose hormonal constitution is arguably more male than female. 

But Roger Pielke Jr., who directs the Sports Governance Center of the University of Colorado — and who has testified in support of Semenya — decided to reanalyze some of the data in the article. In so doing, he found several errors (which he describes on his blog), some modest and at least one that he believed was fatal to the central claims of Clark and colleagues: chiefly, that testosterone levels of women and men do not overlap. 

Pielke, who has questioned, with others, a different paper used in court proceedings in support of the ban  on June 19 submitted a letter to the editors of the Clinical Endocrinology, which rejected the submission. But they decided to issue a lengthy correction (which has yet to be published, but which Retraction Watch has seen) which begins:

In the published article by Clark et al included in the January 2019 issue of Clinical Endocrinology , ‘Large divergence in testosterone concentrations between men and women: Frame of reference for elite athletes in sex-specific competition in sports, a narrative review’, it has been brought to the authors’ attention there were some anoma-

lies that required further clarification within Table 2 , Figure 1 and the References.

The 2.5 page erratum — which we recommend you read for yourself when it is published — makes many minor mistakes and, according to Pielke, one fundamental, error. The most significant flaw, he said, was use of a point estimate instead of range in describing the testosterone levels of people with a hormone condition called 5-alpha reductase deficiency (5ARD2). According to the National Institutes of Health, people with this condition are chromosomally male but often have apparently female external genitalia. 

In the initial paper, as Pielke wrote on his blog, the authors claimed that: 

the range for all 5ARD2 individuals is ‘well beyond the range for normal females.’

Second, the claim that such individuals “whose phenotype at birth was ambiguous or female, have testosterone levels within or near the normal male range” is also incorrect. Finally, the claim that “existing studies strongly support a clear bimodal distribution of serum testosterone levels in females compared to males” depends upon the oversight of material details [in one of the studies Clark and colleagues reviewed].

The correction addresses Pielke’s argument: 

The authors appreciate the opportunity to correct these errors and clarify subclasses of patients in the studies reported. The revised forest log plot continues to show a clear gap between healthy women and men and a large gap between conditions of elevated

testosterone in women with PCOS compared with the 46 XY DSD genetic conditions. The size of this gap may be masked somewhat by the fact that the plot is logarithmic. We wish to stress that the changes to Table 2and Figure 1do not alter our conclusion that the majority of 5ARD2 XY individuals have testosterone concentrations within the normal male reference range. However, a few boys younger than 18 years old who were prepubertal had testosterone concentrations which were only 2-fold higher than the upper limit of the normal female reference range and only one 14-year-old patient (who was noted to have delayed puberty in the original report 6) had testosterone concentrations within the normal female range.

The authors would like to apologize for these necessary changes and convey the above additional information.

Although the authors stand by their main findings, Pielke says the consequences are disastrous for the paper:

The authors put on a brave face but their conclusions have reversed 100%. Instead of a clear demarcation between all, they find a demarcation for most. That is a huge difference which undercuts IAAF arguments.

But wait, there’s more…

What’s more, Pielke says the erratum is itself seriously flawed. In an email to the journal dated August 16, he wrote: 

In Table 2 it identifies the individuals listed as “46 XY Individuals with 5ARD2 Men.” This is incorrect, as not all individuals in these studies are in fact men. The erratum states clearly that it is presenting “testosterone ranges for women and men from reference articles.” The classification of individuals as women or men is clearly based on the classification of individuals in the reference articles.

To just take one obvious example, Shabir et al. Table 3 clearly indicates that two of the post-pubertal individuals are female who did not undergo gender reassignment. They are not in fact men, but women. Yet, the Clark et al erratum classifies them as men.

By mistakenly classifying individuals in the reviewed studies all to be “men” the Clark et al. summary gives a false and misleading impression as to what the data in these studies actually says.

The authors are not impressed. One of them, Alvin M. Matsumoto, of the University of Washington School of Medicine, wrote to Retraction Watch:

All patients in the references with 5ARD2 deficiency were genetic males, hence classified as a 47XY DSD.  It is true that some were raised as females because of ambiguous genitalia at birth and others raised as females.  However, this misses the main point that regardless of the sex of rearing, all serum testosterone levels in ADULTS (>18 years) with 5ARD2 deficiency are essentially within the male range.  I do not feel that any correction of the erratum is needed.  

Matsumoto added:

These comments are getting ridiculous and reflect a uninformed reviewer.

That “uninformed reviewer,” of course, was Pielke, who had led to a two and a half page correction of their paper — in which he is unacknowledged.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].