A widely-touted 2019 study in Nature which argued that large societies gave rise to belief in fire-and-brimstone gods — and not the other way around — has been retracted by the authors after their reanalysis of the data in the wake of criticism diluted the strength of their conclusions.
The article, “Complex societies precede moralizing gods throughout world history,” came from a group of scholars in the United Kingdom, the United States and elsewhere, and was led by Harvey Whitehouse, an anthropologist and the the director of the Centre for the Study of Social Cohesion at the University of Oxford.
The study prompted a significant amount of interest on social media and in the global press, according to Altmetric, with articles in Scientific American, Yahoo! News, PBS, El Pais and many other publications worldwide. As Scientific American put it, Whitehouse’s group found that the advent of moralizing gods did not lead to the formation of complex societies. Rather:
the study suggests pro-social religions appeared after complex societies had already emerged. Although these religions may have helped sustain and grow large societies, the analysis makes the case that they were not necessary for societies to expand in the first place.
The paper, which has been cited 49 times, according to Clarivate Analytics’ Web of Science — giving it a Highly Cited Paper designation among papers of the same age — also met with skepticism. Immediately after publication, a group led by Bret Beheim, of the Max Planck Institute for Evolutionary Anthropology, submitted a response to the journal questioning the validity of the results. Beheim and his colleagues posted a preprint of their argument on PsyArXiv in early May 2019. Nature accepted the article on Feb. 18, 2021, roughly 21 months later.
This blog post on ArcheoThoughts describes the controversy, which, as the author notes, boils down to this:
The core challenge from Beheim et al. is rather that the original paper treats absence of evidence for moralizing gods as evidence of their absence. It changes all N/A (not available) values to absent. In other words, the original paper considers that if there is no information on the presence of moralizing gods in a certain region at a certain time, there is no moralizing god.
After reanalyzing their results in light of the challenge, Whitehouse’s group agreed that the findings weren’t quite as robust as they’d initially claimed.
According to the retraction notice:
Following the publication of this Letter, Beheim and colleagues submitted a Matters Arising in which they argued that our primary results were called into question by our treatment of missing data. In our research, we attempted to test the ‘big gods’ hypothesis even-handedly using the best available evidence, and we made our data and code available during the review process and after publication, in line with best practice in open science. Nevertheless, we accept that we should have labelled moralizing gods as ‘absent’ or ‘inferred absent’ rather than ‘unknown’ in portions of our dataset before the dates of the first appearance, rather than converting ‘NAs’ to zeros during the phase of analysis. Since this Letter was published, we have thoroughly refined our data and analyses, and have found that our original conclusions are still strongly supported. However, the differences between our revised analyses and the original Letter are substantial enough to warrant a Retraction of the original Letter. We have submitted the enhanced analyses for peer review and potential publication in another journal. We encourage the community to refer to these new papers in future instead of this now-retracted Letter. We apologize to the scientific community for the unintended confusion.
Beheim told us his group was surprised by the long delay between the submission of their article and publication:
[It] took nearly two years from submission to acceptance, with only clarifying revisions made to the analysis. Our three referees were excellent, but the journal was not forthcoming about anything for over a year. Thank god we posted a preprint at submission.
I look forward to reading the … revised argument. The objection I had to the original analysis wasn’t so much their conclusion, but that they weren’t transparent about the amount of missing data involved. If our critique can help them nail that problem down, I think the field will progress.
We asked Nature why the delay. A spokesperson said:
Matters Arising submissions that meet Nature‘s initial selection criteria are sent to the authors of the original paper for a formal response. The comments and formal response may then be sent to independent referees for peer review, as was the case in this instance. As with primary research papers, the period between submission and acceptance for Matters Arising can vary, as it incorporates peer review, careful consideration and evaluation by the editors of the issues raised, and author revisions. In addition, the consideration of the Matters Arising in this case was associated with the retraction of the original paper, which added complexity to the process.
Like Retraction Watch? You can make a one-time tax-deductible contribution or a monthly tax-deductible donation to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
What drivel! Why was it in a scientific journal? If the people did not write down their thoughts at the time we will never know. Nearly as bad as papers on the colour of dinosaur feathers. Where is the experiment? Make-work for the self-supporting professional middle classes.
I would make bold and claim the opposite to be true. With living people today I have both, what they say and claim and what they actually do, and I’ve leart to disregard the former and only believe the latter. In earlier times things are much worse, as the few bits of epigraphic writing we have is all state propaganda and nearly nothing else. Contrarily what people actually did or were forced to do and how society was structured can be deduced from their leftovers to a certain degree.
“what they say and claim and what they actually do, and I’ve lear[n]t to disregard the former and only believe the latter. ”
I think that is why communication in science should be true and accurate.
“In earlier times things are much worse, as the few bits of epigraphic writing we have is all state propaganda and nearly nothing else.”
You might say that is still true of present day authoritarian regimes. All the more reason why communication in science should be true and accurate.
I would agree with Axel Berger’s statement and point to the wild disparity between how piously the Puritans of the Massachusetts Bay Colony claimed they lived and how riotously they actually lived, with contemporary court records showing all the same rates of various crimes and transgressions as other colonies.
although the puritan provinces contained people who professed piety and who practiced licentiousness, they were not necessarily the same people. this points less to hypocrisy (saying one thing and doing another) and more to increasing social diversity and complexity. by the late 17th century, at any rate, the power of the puritan elite was being overthrown by a commercial and cosmopolitan elite.
Exactly – Anthro is NOT science.
The accuracy of such a statement depends upon your definition of “science,” of course, and in any case Anthropology has always called itself the most humanistic of the sciences and the most scientific of the humanities.
In this case, the authors of the original paper propose a hypothesis — the relationship of social complexity to a form of religion — and examine evidence to test that hypothesis. I suspect that a lot of people would regard that as a simple but perfectly valid way to do “science.”
it’s a social science, which is ‘scientific’ in that it is empirical.