When journals retract a paper but don’t explain why, what should readers think? Was the problem as simple as an administrative error by the publisher, or more concerning, like fraud? In a recent paper in Research Policy, economists led by Adam Cox at the University of Portsmouth, UK, analyzed 55 retractions from hundreds of economics journals, mostly issued between 2001 and 2016. (Does that number sound low? It should — a 2012 analysis of retractions in business and economics found they are a relatively rare occurrence.) In the new paper, Cox and his colleagues analyzed how many notices failed to provide detailed information, the potential costs of these information gaps, and what journals should do about it.
Retraction Watch: You used “rational crime theory” to analyze retraction notices and their consequence to offenders in economics. Could you explain briefly how rational crime theory works in this context?
Adam Cox: Rational crime theory is a framework for explaining why an individual may commit a crime. This involves an (implicit) cost-benefit analysis by (prospective) perpetrators of crime, or in our case, (prospective) perpetrators of research impropriety. If the benefits exceed the costs then a rational individual may be tempted to participate in the crime.
Benefits to individuals of research impropriety include obtaining relief from the time and costs involved in data collection and analysis. It may also help to pave a quick-route to publication, relieving pressures on researchers to publish for career enhancement. Costs are based on the individual’s moral values and predisposition towards malpractice, and, the costs (including reputational harm) associated with the possibility of being caught and convicted.
We argue that the cost of engaging in research fraud is lowered by the reluctance of journals to publish replication studies. Replications hold a strong prospect of confirming the strength of a field or illustrating problems within it. The infrequency of replication allows poorly supported or erroneous findings to remain undetected, lowering the cost of engaging in research malpractice. Other things being equal, an increase in the probability of being apprehended and the impact of the punishment would increase the costs and thus help promote research propriety. Our endeavours as a research community should be directed to increase the costs and reduce the benefits of engaging in research impropriety.
RW: You note: “We conclude that the frequent vagueness of retraction statements, and a reluctance to signal research malpractice, generally results in little damage to the reputation of caught, and known, offenders.” Can you say more about that, and why that shows “a key deterrent to engaging in research malpractice is lacking?”
AC: If a retraction statement is vague, then the message that there are huge consequences from engaging in malpractice is not communicated effectively, and it may go unnoticed. It becomes impossible to identify the cause of the retracted article and to separate malpractice from other less serious reasons, like “administration errors.”
For example, we identified 12 retractions from the same author, Khalid Zaman. His actions suggest that the cost of reputational damage arising from published retraction notifications (combined with the probability of being apprehended) were considered sufficiently low compared to the benefits of committing research fraud.
Vague retraction statements may be frequently used due to the editors of journals being “more rational” and less willing to risk legal action from aggrieved authors. Retraction Watch has documented many cases of authors taking legal action to prevent retraction of their papers, including a recent case (not in economics).
RW: You say your analysis draws “attention to problems of poor research practice in the field.” How so? Does economics research suffer from this more than other fields?
AC: Reasons for retraction in economics include plagiarism, self-plagiarism, manipulating the peer review process, submitting the same article to multiple journals, and flawed analysis. This collection of retractions shows that some researchers in this field have been undertaking poor research practices, and that research malpractice exists.
To answer this reliably, we would have to separate the effectiveness of detection procedures from the actual level of misconduct. It is akin to attempting to measure crime by the number of arrests, we just don’t observe the full picture. Also, some odious (to some persons at least) practices remain unreported because they are condoned: for example, HARKing (effectively, the practice of writing papers as pieces of chronological fiction) and p-hacking (massaging data and selecting analysis techniques until statistical significance is achieved).
However, from the retraction statements we studied (in economics), there appears to be a higher frequency of cases of fake peer reviews and fewer cases of data fraud compared to other fields.
RW: You note: “Disturbingly, 28 of the 55 retraction notices (in 18 separate journals) did not provide a reason for retraction. Of these, the highest frequency of ‘no reason’ retraction occurred in Statistics and Probability Letters (n = 8) and the International Review of Law and Economics (n = 3), both published by Elsevier.” Were these online-only papers? If so, Elsevier has an explanation for why the notices are so opaque. What is your opinion about it?
AC: All except two are online only versions of the paper. We believe that there is little cost incurred by a publisher for labelling an administrative error as such. The scientific field benefits greatly by being able to separate genuine errors from research malpractice. We would like to trust that what we read is genuine. Further, we can’t find a reasonable argument for attributing a higher value or level of importance to a printed version over an online version.
RW: You note that “several publishers refer with approbation to” guidelines provided by the Committee on Publication Ethics (COPE) and the International Committee of Medical Journal Editors (ICMJE). “These require a statement of the reasons for retraction. But our analysis points to this rarely occurring.” Does this suggest to you that journals are giving lip-service to recommended guidelines?
AC: Yes, some appear to be offering lip service to these guidelines. Some statements are tantamount to false pretences or reflect ignorance of the COPE guidelines. Some journals appear to labour under a misconception that stating (vaguely) who is responsible for retracting a paper is sufficient, and constitutes a reason for retraction. Other journals, in effect, present readers with a shopping list of possible reasons by saying that the article is withdrawn, consistent with the publisher’s editorial policies. The reader has to go hunting for a vague list of possible reasons and work out which one applies.
RW: You offer some recommendations to publishers and editors. Can you summarize some here?
AC: Proposal 1: Retracting publishers should alert all other publishers of the author concerned of their retraction. In egregious cases (e.g., of data fabrication) the other publishers should ensure that the retracted author’s works, published by them, are subject to a research propriety audit.
Proposal 2: All journals should issue clear statements of the reasons for retraction, consistent with recommendations of COPE and ICMJE.
Proposal 3: The text of a retracted paper should be clearly watermarked as retracted. Otherwise, defective work will continue to be cited and influence scholarly thinking.
Proposal 4: If they have not already done so, publishers should require authors to make their data available in a way that facilitates inspection, re-analysis and replication.
Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].