Since last year, a half-dozen researchers have been having a debate: Should the humanities focus on replication? No, said Sarah de Rijcke and Bart Penders in Nature last August: “Resist calls for replicability in the humanities.” In the most recent piece on this subject, de Rijcke and Penders were joined by J. Britt Holbrook to again say “no.” Here, Rik Peels, Lex Bouter, and René van Woudenberg, who have been in the “yes” camp, respond.
In their recent contribution to the Centre for Science and Technology Studies (CWTS) blog, entitled “The Humanities Do Not Need a Replication Drive,” Holbrook, Penders, and de Rijcke argue that two of us, namely Rik Peels and Lex Bouter, have been mistaken in defending the possibility and desirability of replication in the humanities. It seems to us, though, that they have crucially misunderstood us on a couple of points, so we would like to use this opportunity to clarify our position and respond to their objections.
Their main thesis is that “although replication might be possible in some (parts of) fields in the humanities, replicability is not obviously possible in all humanities fields.” The problem with this thesis is that it is perfectly compatible with everything we have argued. In fact, we wholeheartedly embrace this position ourselves. We argued that replication is possible and desirable in the humanities, not that all studies in the humanities should be replicated or even be replicable. There are at least two kinds of studies that come to mind for which replication does not seem to be an obvious desideratum:
- There are humanistic studies that are so wide-ranging in their scope and all encompassing in their sources – think of biographies – that replication of the study as a whole is neither possible nor desirable. Of course, this leaves plenty of room for replication of specific results within the biography in question, for instance, a controversial hypothesis regarding where the person at issue was born or whether she was steered to a particular course of action because she was influenced by, say, Marx’ doctrine of alienation.
- Moreover, there are some fields of the humanities that are not empirical but a priori and completely deductive. Here, we think of large parts of metaphysics, logic, epistemology, ethics, meta-ethics, and so on. Since many studies in these fields do not involve gathering data, statistical analysis, and inductive or abductive lines of reasoning, it is not at all clear that replication in the regular sense of the word is possible for them. We agree that we could and should have been more explicit on this, and it seems to us that this takes away the main worry of Holbrook et al.
In the remainder of this article, we address three further worries that they raise. First, according to Holbrook et al, we require that replication in the humanities “meets all the criteria that have been identified for biomedical, natural and social science research.” They point out that “this is a strong requirement, suggesting that replication studies also need to use the same protocols, methods, and data as the original study.”
Fortunately, we require no such thing. We have argued that replication in the sciences on the one hand and in the humanities on the other is essentially the same thing: “to carry out a replication study is basically to do an independent repetition of an earlier study, answering the same study question by using the same or similar methods under the same or similar circumstances.” Unquestionably, what a replication study will exactly look like will differ from field to field and might even differ among various studies within a specific field.
Moreover, we deny expressis verbis that a replication study needs to use the same protocols, methods, and data as the original study, because we want to leave room for a so-called conceptual replication. The essence of a conceptual replication is that the protocol is not the same. In fact, we suggest that this is often preferable in the humanities.
Second, our opponents argue that diversity of views and arguments is actually a good thing and that our arguments are based on an empiricist and positivist epistemology. As to the latter point, we are not exactly sure what they have in mind here. However, let us nail our colors to the mast. We reject empiricism on its standard definition, because we believe in the possibility of a priori knowledge. And we reject positivist epistemology on its standard definition, because we believe there are many meaningful and knowable propositions that are neither tautological nor empirically verifiable or falsifiable, such as moral propositions. (For standard definitions of empiricism and positivism, see The Cambridge Dictionary of Philosophy.) We fail to see how anything we say commits us to these problematic epistemological positions.
As to diversity of arguments, we agree that this can be a good thing. The more sound arguments for a position, the better. And: the more arguments on both sides of an issue, the better for identifying what the problems are and what is needed to resolve them. What is not a good thing though – not something to be pursued because it is deemed good in itself – is the existence of opposing answers backed up by various arguments to exactly the same question. It is better to find the right answer to a question or to find a set of right answers that are compatible to a question than to accept and even applaud a multitude of different competing answers to that question, at least when it comes to academic questions or hypotheses.
To deny this amounts to denying the value of the joint pursuit of truth and of knowing the truth, which is one of the core values of the humanities as one of us has argued elsewhere. Of course, pursuing the truth may not always be successful, for instance due to dearth of evidence, but it should at least be the aim of humanistic (as well as of scientific) inquiry.
Third and finally, they say that many humanistic studies do not allow of replication and not even of replicability because they rely on interpretation. This is a remarkable but, taken at face value, unconvincing statement. For any study in any field whatsoever relies on interpretation: interpretation of texts, interpretation of data, interpretation of figures and tables, and so on. One cannot engage in a replication study without performing acts of interpretation.
Also, why would a study that involves interpretation not be replicable? Unfortunately, they don’t say. However, interpretative studies surely seem to be replicable. For example, the claim that St. Augustine was influenced by the Gnostics has been confirmed time and again by repeated, and independent, studies of his work.
We conclude that the objections leveled by Holbrook et al fail and that our conclusion stands: for many empirical studies in the humanities, replication is both possible and desirable. But if we are right, then we should like to move on from possibility and desirability to actuality, that is, we should like to see a couple of crucial studies in the humanities replicated, so as to test what we have been arguing for.
Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
humanities is not science, it is whatever the inventor thinks it to be, so this entire discussion is pointless
My suspicion is that “replication” is used here improperly; scientists attempt to replicate the results of laboratory experiments to see if the same results are obtained (within some agreed upon error). There is nothing quite equivalent to “repeating experiments to confirm results” in the humanities. Historical accounts need to be consistent within reason; no two observers will report the same happening identically. But surely some collaborative effort to arrive at the truth of an event is worth the effort. Otherwise nothing of much value for the future is realized. No foundation gets built, not even a temporary one that may be buttressed by some future discovery. Witness the present sad state of world history as exemplified by titles such as “The End of History”
“Historical accounts need to be consistent within reason; no two observers will report the same happening identically. ”
I was also thinking about history. As the article notes, there is a lot of philosophy of science behind the scenes, of course, but in a sense I think history (as an academic field) has always more or less conformed with some notion of “replications”. The cycle may be much longer than in the sciences (i.e., decades or even more) but many things still get “replicated” in history. Previously closed historical archives open. Falsehoods are corrected. The knowledge gets sharper as time passes. To some extent, it even cumulates just like in the sciences.
It would be interesting to read a book about World Political History in the 20th Century written collaboratively by Soviet era and free world historians. Would falsehoods be corrected and knowledge sharpened over time? Particularly after most of those alive at the time have died and can no longer be asked to support their conclusions. One would like think it was possible but I m not holding my breath. Recall the sorry story of the Katyn massacre.
It would be interesting to read a book about World Political History in the 20th Century written collaboratively by Soviet era and free world historians. Would they agree on the Katyn massacre?
It would be interesting to read a book about World Political History in the 20th Century written collaboratively by Soviet era and free world historians. Would they agree on the Katyn massacre?
So you managed to completely reverse my point: in 2019 we do have a much sharper picture about the 20th century than we did before. Through “replications” falsehoods such as Soviet era history writing have been corrected! (Most of academic history also relies on archival material; in many ways, the serious study of history can only start when there are no contemporaries left.)
It seems like academics in the humanities want to use scientific language without committing to scientific principles. For example, when a layperson hears about “a study,” he thinks of, at minimum, data collection and, more likely, something approximating the scientific method, i.e., an argument (hypothesis), a way of collecting data to prove that argument (methodology), data collection (experiment), analysis, and conclusion; but what academics in the humanities call “a study” is, essentially, a glorified persuasive essay. Their reasoning is obvious: scientific language denotes objectivity and credibility; their work can be taken more seriously.
I don’t know when the humanities adopted this language. My hunch is that it followed the advent of postmodernism when philosophers and social scientists began to “deconstruct” language. A good example of this phenomenon is “racism.” For decades, the term implied something like racial prejudice based on a personal belief of racial superiority. Relatively recently, some scholars reformulated “racism” to include an asymmetrical power dynamic (“prejudice plus power”). Preached from the ivory towers, this new definition has found an audience among social justice activists who insist that it isn’t new at all and that the popular consensus is (and presumably always has been) wrong. To the rest of us—the overwhelming majority—it feels like gaslighting. (Another good example is the putative distinction between “sex” and “gender.”)
I believe the humanities apologists are engaging in a similar kind of manipulation, and that the debate between de Rijcke, Penders, and Holbrook and Peels, Bouter, and van Woudenberg boils down to language. The humanities should still publish what it publishes, but it should drop the pretense that it’s equivalent to the work of scientists.
I seriously doubt that anything like this is what a typical “layperson” “thinks of” when he “hears about ‘a study’”. But of course you are making an empirical claim here (about some population of “layperson”s), and by marshaling some evidence (as contrasted with speculation) about a sample of members of such a population of “laypersons” actually do think when they hear about “a study”, you might convince me of the truth of your claim. Have you any such evidence?
I have some such evidence (although it concerns merely a convenience sample of English-speaking persons whose status as “layperson”s is unclear) which seems to me to undercut, rather than to support, your claim.
My evidence consists of the written-English usage examples for the relevant definition (5b) of the English noun study in the Oxford English Dictionary. That definition is
specifically the part following “frequently:”. Here are the examples (shorn of italics, for my convenience):
Of these, only the last completely fits your claim about what a “layperson” “thinks of”; and several of the others are (compelling) evidence that if “what academics in the humanities call ‘a study’ is, essentially, a glorified persuasive essay”, then at least they they have been using the word like that for several hundred years.
Now, one of the OED’s lexicographers’ principles is that, to the extent possible, they try to find the earliest printed citation of any given shade of meaning (and they have a huge crew of volunteer readers who compete informally to find earlier examples). According to the OED on-line, the entry for “study, n.” that I have just excerpted was updated for the Third Edition, as of September, 2015—strongly suggesting that at least those laypersons like me, whose uses of “study”, definition 5b, began to become calcified rather closer in time to 1957 than 2008, do not have such an elaborate para-scientific model in mind when we “hear about ‘a study’”.
The rest of your comment is itself a “persuasive essay”, as is all of this comment (except that I am actually trying to use some data).
I agree that research articles in domains of human knowledge considered “humanities”, like sociology, history and anthropology, are useless mysticism without giving the readers the option to replicate at least the thinking processes of their writers.
Another issue in humanities that almost nobody talks about, is article references. An obvious example is research into the relatively recent and politically contested history.
Hi Robert,
Not sure if Medical Education counts as one of the humanities but this study on questionable practices in academic medical education might be of interest to you.
Artino AR Jr, Driessen EW, Maggio LA. Ethical shades of gray: international frequency of scientific misconduct and questionable research practices in health professions education. Acad Med. 2019;94(1):76–84.