Doing the right thing: Scientists reward authors who report their own errors, says study
We’ve always like to highlight cases in which scientists do the right thing and retract problematic papers themselves, rather than being forced to by editors and publishers. Apparently, according to a new paper by economists and management scholars, scientists reward that sort of behavior, too.
The study by Benjamin Jones of the Kellogg School of Management at Northwestern University and the National Bureau of Economic Research and colleagues, “The Retraction Penalty: Evidence from the Web of Science,” was published yesterday in Scientific Reports, a Nature Publishing Group title.
The authors lay out what they do:
In this paper, we draw on all retraction notices in the Web of Science (WOS) database. We focus on the post-2000 period when WOS indexing of retractions appears relatively complete (see supporting information for detailed discussion of the database) and use the WOS to expand our analysis across the known universe of fields. Our analysis can thus provide a more comprehensive cross-field view of retractions than the existing literature. Most importantly, we examine a new dimension: We analyze the effect of retraction on scientists’ prior work, thus quantifying a potentially critical consequence, and disincentive, for being associated with false scientific results. Our analysis further shows how chain reactions to retraction hinge on whether authors self-report errors.
(Speaking of reporting our own errors, we tweeted yesterday that we’d covered this paper last month as a working paper about which members of a team suffered most when studies were retracted. The new one was actually a precursor to that paper, Jones tells us.)
The subject-by-subject comparisons are illuminating. Retraction was rare across all disciplines, but was incredibly rare in the arts and humanities — .01 retractions per 10,000 papers — and the social sciences — .02 per 10,000. Biology and medicine had .14 retractions per 10,000 papers.
About 22% of retractions were self-reported, while about 71% were not self-reported, with the rest unknown.
Not surprisingly, given previous analyses, the authors found that citations of a retracted paper declined after it was withdrawn. One particular figure from the paper, however, really highlights the difference in citations of an authors’ prior papers based on who reported the errors (click to enlarge):
The authors conclude:
…retractions can create substantial citations penalties well beyond the retracted paper itself. Citation penalties spread across publication histories, measured both by the temporal distance and the degrees of separation from the retracted paper. These broad citation penalties for an author’s body of work come in those cases, the large majority, where authors do not self-report the problem leading to the retraction. By contrast, self-reporting mistakes is associated with no citation penalty and possibly positive citation benefits among prior work. The lack of citation losses for self-reported retractions may reflect more innocuous or explainable errors, while any tendency toward positive citation reactions in these cases may reflect a reward for correcting one’s own mistakes.
“A reward for correcting one’s own mistakes” — we’re smiling.