A closer look at the ‘chocolate with high cocoa content’ hoax

We are pleased to present an excerpt from The Predatory Paradox: Ethics, Politics, and Practices in Contemporary Scholarly Publishing by Amy Koerber, Jesse C. Starkey, Karin Ardon-Dryer, R. Glenn Cummins, Lyombe Eko, and Kerk F. Kee, published by Open Book Publishers, October 2023. 

In 2015, Johannes Bohannon, along with three coauthors, published an article titled ‘Chocolate with High Cocoa Content as a Weight Loss Accelerator’ in the International Archives of Medicine. The article reported results from a study that divided participants into three groups, with a different diet assigned to each group, and concluded that ‘Subjects of the chocolate intervention group experienced the easiest and most successful weight loss’ (p. 1).

In a personal account published later, journalist John Bohannon described the article as an intentional hoax that he and his coauthors had carried out in response to a request from a German film crew who was making a documentary on the ‘junk-science diet industry.’ To implement the hoax, Bohannon and his coauthors created an ‘Institute of Diet and Health’ that existed only as a website, and he assumed the name ‘Johannes Bohannon’ as lead author of the study. As he explained, the research reported in the article was actually conducted, but it was “terrible science,” including major flaws that would have been detected if the article had undergone a legitimate peer-review process.

Bohannon and his colleagues’ ‘Chocolate with High Cocoa Content’ article was retracted shortly after it was published, and the editors of International Archives of Medicine published a retraction notice dated 10 June 2015 (Editorial Office 2015). The editors’ decision to retract this article ostensibly served to correct the scientific record and prevent the erroneous data reported in the published study from being circulated in subsequent literature. This manner of correcting the scientific record is an important purpose of retractions, as defined by the Committee on Publication Ethics (COPE):

Retraction is a mechanism for correcting the literature and alerting readers to publications that contain such seriously flawed or erroneous data that their findings and conclusions cannot be relied upon. Unreliable data may result from honest error or from research misconduct. (Wager and others 2009: 202)

Although the editors’ retraction of Bohannon’s hoax article seems to correct the scientific record, it does not entirely align with these COPE guidelines. For example, the editors’ language in the retraction notice is confusing. The retraction notice published on the journal’s website states that the manuscript had been published ‘accidentally’ and that it ‘was finally rejected and never published as such.’ Furthermore, the journal editors did not fully follow the COPE guidelines, which state that ‘Notices of retraction should… be linked to the retracted article wherever possible (i.e., in all electronic versions)’ (Wager and others 2009: 201). It is not clear why, but in this case, the retracted article is no longer available at the journal’s website.

In fact, making the situation even more confusing, the Digital Object Identifier (DOI) for the original article now links to a study called ‘The Comparison of Resilience and Spirituality in Addicted and Non-Addicted Women’ (Ramezani and others 2015). This is extremely problematic, given that the sole purpose of a DOI is to provide a unique and persistent identifier that links to a published article, and it is the publisher’s responsibility to maintain the integrity of that link (‘How the ‘Digital Object Identifier’ Works’ 2001). In short, this retraction seems to be intended as a ‘mechanism for correcting the literature’, as the COPE guidelines advise, but the error that needed to be corrected was not an error made by researchers; rather, it was, according to this retraction notice, an error made by the publisher.

The series of events preceding the article’s retraction is also subject to dispute. According to a 28 May 2015 Retraction Watch post, the hoax article was available on the journal’s website on the morning of 28 May but appears to have been retracted later that same day (Schwitzer 2015). A 28 May 2015 Facebook post (Perez 2015) from someone who appears to be one of the journal’s editors also indicates the article was retracted on that same date. By contrast, the retraction notice that appears on the journal’s website (Editorial Office 2015) indicates 10 June 2015 as the publication date of the notice; it is not clear why the retraction notice would be published almost two weeks after the article was retracted on 28 May.

In addition to this confusion about the date of retraction, these different sources also offer varying accounts of the series of events that preceded retraction. Perez’s Facebook post suggests that the article was never accepted by the journal — that it was published ‘by mistake’, based on a misunderstanding of a managing editor who had been copied on Bohannon’s initial email submission. The Facebook post even claims that sometime after the article was accidentally published due to this miscommunication, ‘the manuscript was rejected by the editorial board.’ However, Bohannon offers a different narrative of events. He quotes an email acceptance notice that is also quoted by Retraction Watch. In this personal email correspondence that Bohannon had apparently shared with Retraction Watch, the journal editors praised Bohannon and his colleagues’ article for its high quality and potential contributions:

I’m contacting to let you know your manuscript ‘Chocolate with High Cocoa Content as a Weight-Loss Accelerator’ has been pointed by our editors as an outstanding manuscript and could be accepted directly in our premier journal *International Archives of Medicine.* (McCook 2015)

Adding even more confusion, Perez’s Facebook post was edited on 10 June 2015, almost two weeks after its initial 28 May publication, to add the second paragraph, which seems to be an attempt to account for the journal’s problematic set of actions. This 10 June edit also contributes to a discrepancy about the length of time that the fake article was available on the journal’s website. For instance, Perez’s (2015) Facebook post suggests it was online for only a few days (from Thursday one week to Monday of the next week, with no specific dates given). However, Retraction Watch indicates that the article was accepted soon after Bohannon submitted it in early March and was not retracted until 28 May 2015.

It may be easy to look at this situation in retrospect and conclude that so many problems are apparent in this article that no one could ever take it seriously, and because it was retracted and is no longer available, its damage was minimal. In fact, any science journalist or academic researcher who took time to assess the International Archives of Medicine’s online presence would have had good reason to question the credibility of this journal.

For instance, a search in PubMed Central’s journal list reveals that the journal was indexed from 2008 until 2014 but is ‘no longer participating’ (‘PMC Journal List’ [n.d.]). Articles published in the journal are full of grammatical errors, indicating a lack of attention to copy editing, at the very least, and probably indicating an absence of peer review as well. The journal publishes a lot of articles on a wide range of topics.

For example, Volume 14, published in 2014, includes fifty articles. These are indexed in PubMed Central as the entire volume, without any indication of issue numbers. Categories include Case Reports, Hypothesis, Original Research, Reviews, and Short Reports. The journal website linked from its current publisher, iMedicalPublisher.com, describes the journal as ‘The new megajournal on all areas of medicine’, and says the journal is ‘Really international’ (‘International Archives of Medicine’ [n.d.]). These obvious cues should be enough to raise questions about the journal’s legitimacy, even aside from the glaring scientific flaws already noted in the hoax article itself.

However, none of these factors, and even the fact that the article was retracted, were obvious enough to stop the article from being cited in subsequent literature. In fact, when this chapter was written, the article had been cited twenty-eight times, according to Google Scholar. Although some of these citations refer to the article as an example of a publication in a predatory journal, other citations are in legitimate journal articles that genuinely cite the scientific findings reported in the article. Specifically, as shown in Table 4.1, almost one-third of these citations (eight out of twenty-eight) cite the article to support a scientific claim.

None of the eight publications that cite the Bohannon hoax article to support scientific claims acknowledges that the publication was retracted, even though all these publications appeared well after the 10 June 2015 publication date of the International Archives of Medicine’s retraction notice. Instead, these eight publications cite the Bohannon hoax article as if it were any other legitimate peer-reviewed scientific text. For example, a 2017 review article (Ramos and others 2017) cites the Bohannon study, along with several other sources, to support the claim that appears in the following sentence: ‘In addition, despite the fact that cocoa products commercially available are frequently high-caloric foodstuffs, they have been reported to have a similar [anti-obesity] effect in humans’ (p. 5). The note that cites the Bohannon and others article includes a CrossRef link that links to the DOI, which now links to a different article, as noted above. This 2017 review article, interestingly, is published in the journal Antioxidants, which is published by MDPI.

The Bohannon hoax article is also cited in another article (Rodríguez-Lagunas and others 2019) published in the MDPI journal Molecules. This article cites the Bohannon hoax article to support the claim that ‘anti-obesity actions of cocoa have been reported’ (p. 7). Oddly, the Bohannon hoax article is also cited at the end of the following sentence, which does not seem to have anything to do with the findings reported by Bohannon and his coauthors: ‘Regarding the health questionnaire, the university students reported, logically, a good health status, far away from suffering chronic diseases involving neoplasm and cardiovascular diseases, the main causes of death in the Spanish population’ (p. 8). These citations of the Bohannon hoax article, well after it was retracted and without any acknowledgement that it was a hoax, raise some serious concerns about the quality of content published in these two MDPI journals. It appears, at least in these two cases, that authors are citing literature without paying much attention to its quality — or maybe without even reading the texts they are citing — and peer reviewers are not catching these sloppy citation practices.

Another citation of Bohannon and others appears in a 2019 article in the Journal of Agricultural and Food Chemistry (Camps-Bossacoma and others 2019). This article, ‘Role of Theobromine in Cocoa’s Metabolic Properties in Healthy Rats’, links to the Bohannon hoax article through Google Scholar instead of CrossRef, but like the other citations, this one cites the study without any acknowledgement that it is fraudulent or that it has been retracted. Specifically, the Bohannon hoax article is cited in the following sentence: ‘Cocoa effects on body weight increase have already been reported both in animal models and humans, […] and it has been postulated that cocoa is a weight loss accelerator’ (p. 3611). Unlike the other two journals, published by MDPI, this journal is published by the American Chemical Society, described on its website as follows: ‘As a non-profit scientific organization with more than 140 years’ experience, we are a champion for chemistry, its practitioners and our global community of members’ (‘About ACS’ [n.d.]).

This organizational affiliation may seem to grant legitimacy to the journal. Again, though, it is troubling that the Bohannon hoax study is cited uncritically, as a source within the authoritative scientific record, to support a statement about cocoa’s weight-loss properties. This must lead us to question the overall legitimacy of the article and the soundness of the science that it reports.

Persistent citation of retracted articles is not uncommon. In fact, Retraction Watch keeps a list of retracted articles that have received the highest number of citations (‘Top 10 Most Highly Cited Retracted Papers’ [n.d.]). As of the publication of this book, the top article on the list was published in 2013 in the New England Journal of Medicine and retracted in 2018 (Estruch and others 2013). The Retraction Watch site reports that the article has 2735 citations. Next on their list is a Lancet article published in 1998 and retracted in 2018; Retraction Watch indicates 1509 citations of this article.

A recent study exposes the serious nature of this problem by examining a high-profile retraction case that involved the work of Scott S. Reuben (Bornemann-Cimenti and others 2016). Reuben was a well-established medical researcher who studied pain medicine. In 2009, it was discovered that he had fabricated data in many of his published studies, which led journals to retract twenty-five of his published articles. Bornemann-Cimenti and others (2016) track the extent to which Reuben’s articles continued to accrue citations for many years after they were retracted. Although some of the citations note that the article has been retracted, many do not (Peng and others 2022).

According to Hyland (1999), ‘the attribution of propositional content to another source’ (p. 341) is a fundamental rhetorical feature of academic writing. Problematic citation practices include the citation of retracted articles, before or after the article is retracted (Bolland and others 2022); citation, knowingly or unknowingly, of articles published in predatory journals (Akça and Akbulut 2021); or citing an article without having read it (Akça and Akbulut 2021; Wetterer 2006). Offering an especially concerning example, Wetterer tells the story of an outright falsehood that was perpetuated for many years among ant scientists, about the extinction of a particular ant species that allegedly occurred on the Atlantic Islands of Madeira due to the invasion of an exotic species. According to Wetterer’s analysis, this myth was perpetuated and became accepted as scientific fact reported in peer-reviewed journal articles because of practices that he labels ‘quotation error and citation copying’ (p. 352).

In short, scientists were repeating the myth that this species had been made extinct, based on a misinterpretation of an 1898 article (Stoll 1898) that ended up being widely cited as the source of this information. Wetterer speculates that Stoll’s report was misinterpreted because his article was published in German, and the person who initially cited it mistranslated his findings; thus, the erroneous belief that Stoll’s report was based on first-hand evidence became widely cited in subsequent literature, even though, according to Wetterer, Stoll never claimed to have access to first-hand evidence. 

Thus, like many of the other phenomena explored in this book, the phenomenon of sloppy citation practices is nothing new; it is well documented in science, and the phenomenon of poor citation practices has been documented and studied well before the relatively recent development and proliferation of predatory journals. In the case of the Bohannon hoax, articles that cited it later are articles that were presumably subject to actual peer review, and they link directly to the Bohannon and others’ study, which was retracted shortly after it was published, without acknowledging that it was retracted. An especially puzzling aspect of this case is that it is not clear how the authors of these articles would have located a correct version of Bohannon and others’ study, given that its DOI — which is supposed to provide a unique and persistent link to a digital publication — does not even link correctly to the retracted article. 

In short, the Bohannon ‘Chocolate with High Cocoa Content’ hoax may have been designed to expose substandard publishing practices at predatory journals, but it ultimately exposed a much larger, more complex set of problems. Everything was broken in this situation, including the various checks and balances that are expected to detect bad science and prevent it from being shared with the public. The science reported in this article was conducted, but it was conducted poorly by anyone’s standards. A thorough peer review would have detected the obvious flaws in the research and would have prevented the paper from being published.

A proper retraction, meaning that the retraction notice would appear along with the original article at the original DOI, and a more careful vetting of the literature by scientists after the article was published, may have prevented the retracted article from subsequent citations. However, in addition to all these scientific problems, the science journalists who reported on the findings also should have done due diligence by asking questions to relevant experts about the science behind the article.

Retractions of published articles are not uncommon, even at journals that are not predatory. In fact, the frequency of article retractions is on the rise, and even the most prestigious journals in their respective disciplines are not immune (Bornemann-Cimenti and others 2016). The extent to which the citation of a retracted article will alter the course of science depends on the nature of the evidence that is cited. If the evidence is in line with most other evidence, then the effect may not be too great, but if it is the only study that is cited to support a claim that departs from other available evidence, then this is obviously problematic. For all these reasons, retractions of published articles, whether they are published in a ‘predatory’ journal or not, are something that researchers and other stakeholders in scholarly publishing need to be aware of.

Amy Koerber, is Professor in Communication Studies and Associate Dean for Administration & Finance in the College of Media & Communication at Texas Tech University. Jesse C. Starkey is an independent grant writer and strategic communication consultant, serving clients in the academic, private, and non-profit sectors who are seeking to make positive social change in their communities. Karin Ardon-Dryer is a faculty member in the Department of Geosciences at the Atmospheric Science Group at Texas Tech University. R. Glenn Cummins is a professor of Journalism and Creative Media Industries at Texas Tech University. Lyombe Eko is a journalism professor in the College of Media & Communication, Texas Tech University.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, subscribe to our free daily digest or paid weekly update, follow us on Twitter, like us on Facebook, or add us to your RSS reader. If you find a retraction that’s not in The Retraction Watch Database, you can let us know here. For comments or feedback, email us at [email protected].