Some peer reviews evidently are tempted to ask authors to cite their work, perhaps as a way to boost their own influence. But a recent episode at the journal Bioinformatics suggests, the risk can outweigh the reward.
We’ll let the editors — Jonathan Wren, Alfonso Valencia and Janet Kelso — tell the tale, which they did in “Reviewer-coerced citation: Case report, update on journal policy, and suggestions for future prevention:”
A case was recently brought to the journal’s attention regarding a reviewer who had requested a large number of citations to their own papers as part of their review. After investigation of their most recent reviews, we found that in every review this reviewer requested an average of 35 citations be added, approximately 90% of which were to their own papers and the remainder to papers that both cited them extensively and mentioned them by name in the title. The reviewer’s phrasing strongly suggested that inclusion of these citations would influence their recommendation to the editor to accept or reject the paper. The reviewer was unable to provide a satisfactory justification for these requests and Bioinformatics has therefore banned them as a reviewer. Our investigation also suggests that the reviewer has behaved similarly in reviewing for other journals.
We asked why the editorial didn’t name the offender, given that he could be doing the same thing for lots of journals. Wren told Retraction Watch that the publisher of the journal — Oxford University Press — had other ideas:
I felt the simplest solution was to mention him by name, but OUP’s legal department made an effective argument that I/we don’t have the legal right to do that, given that journals commit to peer-reviewer anonymity as a condition of peer-review. It’s new territory and that’s why I think some increased awareness of the nature of this problem is important. I personally think that journals should adopt a new clause that says something along the lines of “reviewers found to have engaged in misconduct, after a fair and impartial investigation, will be banned from reviewing from that journal for X years and the findings of the investigation will be made public”. Obviously, such investigations should be held to a high standard to protect people from false accusations, and I think everyone eventually deserves a second chance, but it would be an inexpensive and simple disincentive. And, of course, there are other possible solutions, but that’s what I think merits some thought and discussion in the scientific community – what are the pros and cons of the different possible solutions to this type of problem, specifically when a pattern of behavior (i.e., compelling, not anecdotal, evidence) is hard to detect?
Take our poll below to let us know what you think.
Still, Wren and his colleagues could contact other journals for whom the perpetrator might be reviewing. However, he told us:
Well, that turns out to be the hard part. I did compile some citation metrics for this reviewer and a list of journals he likely reviewed for based on non-self and self-citation patterns. I contacted #1 on that list, they confirmed he was doing the same thing there and banned him from reviewing as well. I contacted one more (which turned out to be an interesting and complicated story, which I can’t go into yet, but is still ongoing) and planned to contact others, but there are a lot of journals to contact and even if I managed to contact them all, he can still review for other journals that haven’t been contacted. This is one of the reasons I wanted to publish the editorial – not only can we not effectively solve this problem for this specific reviewer, the peer-review system is not currently amenable to solving this type of problem in general.
Citation boosting is of course nothing new in scientific publishing.
Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
Requesting citation of an excessive number of your own manuscripts is of course a problem.
A simple solution would be for the editor, whose job it is to look out for theses things, to return the review to the reviewer for revision.
That being said there are examples where requesting one’s own manuscript to be cited is appropriate.
If the paper under review is a partial or full replication or built heavily on the prior publication then a request for citation is in my view perfectly acceptable and maintains continuity in the scientific literature.
“the reviewer requested an average of 35 citations be added, approximately 90% of which were to their own papers”
On the AVERAGE? Good grief, where were the handling editors? Did they even look at the reviews or just click off on the recommendations in the editorial management software? Wondering what passes for normal in this ‘Bioinformatics’ journal community, I popped open the first two open access articles in the current issue, and they both had about ~40 citations total. So having a reviewer suggest 30 more to themselves didn’t raise any red flags to the editor? You would think this would get picked off the first time it happened.
I was one of the few in the poll that agreed with the lawyers it probably is inappropriate for the journal to out the miscreant. Anonymous promises must be kept if they want to keep getting future reviewers. However, the manuscript authors who received that anonymous review could post it.
Rather than disclosing the name of the reviewer publicly, one might consider confidentially informing the institution of the reviewer of the appearance of a breach of scholarly integrity; if a respectable research institution, they should have a procedure for handling such concerns. One might even ask the reviewer and his/her institution to inform other journals for which such reviews might have been produced.
From the paper, it appears that they did take this step and (not surprisingly) met with some resistance:
“…the first step is to contact the reviewer for an explanation. If it is unsatisfactory, then bring the matter to the attention of their immediate supervisor. This second step is only effective in a well-organized academic system, which was not the case here.”
There are occasions when requesting citations can be perfectly justified, even citations of one’s own work. Not 35 though.
Once I suggested a list of additional citations on a paper I was reviewing and just 2/10 were my own work (Q1 materials science journal and the paper already had ~80 references). The handling editor returned the review to me and not only asked me to justify suggesting my own work but also wanted to know my relationship with the authors of the other papers (none). This journal sends me requests regularly (1-2 a month) and they know me well. So editors do take it seriously in many fields and do look out for this. (The paper ended up getting rejected, btw).
For a bioinformaticiam it would probably not be too difficult to perform a quick mining of Bioinformatics papers over the last 10 years that contain >30 times the same name in the references list…
The OUP editor says: “I personally think that journals should adopt a new clause that says something along the lines of “reviewers found to have engaged in misconduct, after a fair and impartial investigation, will be banned from reviewing from that journal for X years and the findings of the investigation will be made public”. Obviously, such investigations should be held to a high standard to protect people from false accusations …”
Highly profitable journals ask reviewers to work for free. It usually takes me a couple hours to study a manuscript, and another half hour to write the review. And now as part of doing that work I’m told I’m going to be subject to an investigation (“obviously … held to a high standard”) if an author accuses me of “misconduct”? That could include saying I recommended rejection or major revision because I was biased or conflicted. Editors just need to do their job – which should include evaluating the reviews. From this RW article, it is apparent some don’t even read them. Let the “high standards” start with the journal.
I dislike commercial publishers of all stripes as much as the next guy but this is a bit unfair. This is a society journal edited by academics that’s trying to raise their standards.
Part of this involves asking authors and reviewers to meet high standards too.
But yes, I agree that a handling editor should have noticed a reviewer suggesting 35 citations.
If the “misconduct” to be subject to investigation and public disclosure is limited to coersion of citations, why not have all reviewers check a box indicating whether they requested their own work be cited, when they submit their reviews to the editor. If they checked yes, the editor can look to see if the request was self-serving and coersive. If it was, or if the reviewer was dishonest when checking “no”, then blacklist the reviewer. Once is enough. The checkbox will serve as notification to reviewers that the issue is being taken seriously and acted on. It doesn’t seem like rocket science.
There are other solutions I’m sure. I just don’t think it’s necessary or a good idea to threaten reviewers with investigations and public denouncement.
…but I think it is a fair point that reviewers are working for free for the (often commercial) journal. Getting citations could be seen as one (unfortunate, of course) form of payback – and risking investigations, loss of reputation and perhaps worse will reduce incentive to do an job that probably is already marginal.
‘Let the “high standards” start with the journal.’
Yes, indeed. I was asked by an editor to add two of his papers as references as a condition for acceptance.
Two is of course not a lot but they were not only irrelevant, these were the only citations demanded, and complying would also almost double the citation counts for those papers(!)
The demand was overruled with an appeal to upstairs and the paper published.
Oxford University Press is part of the University of Oxford, which is a UK public institution subject to the Freedom of Information Act 2000. So, you could submit a Freedom of Information request to ask for the reviewer’s name. It’s extremely easy: just send an e-mail to anyone at OUP, saying that you’re making a Freedom of Information request. If you prefer, you can write directly to the university’s centralized system for handling requests https://www.admin.ox.ac.uk/councilsec/compliance/foi/ , but that’s not mandatory.
OUP is legally required to disclose the information within 20 working days or withhold it by applying an exemption. If they withhold the information, you can ask for an internal review within an extra 40 working days. If they continue to withhold the information, you can submit a complaint to the Information Commissioner’s Office (ICO), which might take up to a year. If the matter goes to the ICO, then the decision to disclose or withhold is effectively taken out of the OUP lawyers’ control.
OUP might claim an exemption under Section 43, prejudice to commercial interests, arguing that OUP would be placed at a competitive disadvantage by being forced to name the reviewer, leading to difficulties in obtaining willing reviewers, which commercial rivals would avoid. The first step to apply the exemption is to determine whether or not Section 43 is engaged. I think this would be difficult for OUP, since it would need to provide a causal argument that prejudice to its commercial interests is likely, not merely conjecture that there would be prejudice. Also, one might argue that OUP would benefit by being seen to be upholding high standards of public integrity. Because Section 43 is a qualified exemption, if it’s determined that Section 43 is engaged, then a public interest test must be carried out as the second step. Roughly speaking, to apply the exemption, the public interest for withholding the information should be greater than (equal to isn’t enough) the public interest for disclosing the information. The public interest factor for withholding the information would be the integrity of the peer-review process. Public interest factors for disclosing the information might include public trust in science, investigating unethical behavior and deterrence of unethical behavior.
OUP might claim an exemption under Section 41, that disclosure of the information would constitute an actionable breach of confidence. The information is probably confidential. The confidentiality of peer-review is implicit, without any explicit confidentiality agreement being signed, but perhaps one might argue that it’s implicit that in exceptional circumstances one’s identity might be released. Although Section 41 is an absolute exemption that doesn’t require a public interest test, the possibility of a public interest defence must be considered. The public interest in maintaining confidentiality is always high, but one might still be able to argue that the above public interest factors in favour of disclosure are greater.
As it happens I’m currently reviewing a paper that cites 3 of my papers – in a report with only 8 references (so that caught my eye)
I was going to suggest they drop one (as its’ rather redundant) but on reflection – I may perhaps suggest if they added 2 more (of mine) they would have a more robustly referenced
Its all about the giving :o)
[its’ rather redundant…]
I hope the paper is better than the review. At least in terms of the grammar.
It seems to me that an easy way to solve this problem would be to ban making requests for extra citations directly to the authors. All the requests could be made via the confidential note to the editor. The editor could then assess the pertinence of the request.
Correct way to increase your h-index is as follows:
1/ write in your report that there is an egregious lack of citations.
2/ Just avoid to ask for more than 5 citations of your own work.
3/ Also demand 5 other, additional citations.
If you are a recognized leader: increase up to 10 and devote much time to reviewing papers. But never 35. Unless you review a review article.
Another editorial has been written by the Journal of Theoretical Biology on the same subject regarding similar behaviour in their journal (by the same individual):
https://www.sciencedirect.com/science/article/pii/S0022519320300278
And this individual has been obliquely named by one of the editors:
https://twitter.com/mcapellanus/status/1222467180425629697
I’ve now written up and named this person, Kuo-Chen Chou, in a report for Nature. https://www.nature.com/articles/d41586-020-00335-7
Some authors only mention the names of their contributors in the article so that they can do the same and increase the number of citations. For example, author A only mentions the works of authors B and C; author B mentions the works of authors A and C, and so on. Now the question is what should the referee do if the referee’s articles should have been cited but were not?
P.S.> “should have been cited” means the papers of the reviewer are also in the same field of the manuscript under review.
I think the author of this article must write another article about “The case of well-known authors as peer reviewers steal research ideas from the little known authors of the paper they reviewed”. I recently refereed a manuscript in which a well-known author had used (‘stole’ is appropriate word) the method of a young author without any citation and claimed about novelty of his idea! It was really disgusting.