If you know Prairie Home Companion, you that that in fictional Lake Wobegon, “all the women are strong, all the men are good looking, and all the children are above average.”
That’s a bit like what Harvard’s Nir Eyal found when he was asked to review a paper for Dove Medical Press. Here’s what he saw when he looked at Dove’s peer reviewer form:
The worst possible rating — a 9 — was that “Paper has major flaws and requires considerable modification to make it acceptable.”
At most other journals, some version of “reject” would be available. As Eyal noted:
Failing to even offer peer reviewers the option of rejecting a paper mocks the peer review system. It is a brazen violation of basic, definitive ethical norms in scientific review.
Tim Hill, publisher of Dove Medical Press — which was expelled from the Open Access Scholarly Publishers Association (OASPA) last fall following the John Bohannon sting published in Science — considered Eyal’s comments, and made a change. The new peer review form reads:
Hill tells Retraction Watch:
I should explain that we have operated since we started publishing journals on the basis of a minimum of two sets of peer-reviewer comments for each paper. Narrative comments plus the numeric rating and any Conflicts of Interest disclosures are supplied to each journal’s Editor-in-Chief. The Editor-in-Chief then evaluates the reviewer responses along with their own reading of the paper and they then decide what happens to each paper.
To help the reviewers and Editors-in-Chief we have a numeric rating scale on each reviewer response form for all our journals. These are intended to give a quick ranking of the paper that supplements the narrative as a help to the Editor-in-Chief in making their decisions. We put a brief legend on a score of 1, 5, and 9 to give the reviewers some idea of what the scale means.
We have never discouraged our reviewers from rejecting papers. When a peer-reviewer contacted me recently I was interested to receive his comments as it was the first time that anyone had raised such an issue. We are always eager to improve how our processes operate and the peer-reviewer had identified something that was not previously considered. We wanted to improve our process and make the numeric scale as straightforward as possible for users.
When the Bohannon scam paper occurred and we were made aware of it we immediately began to modify our internal processes to prevent anything similar happening in future. These changes were in place well before we heard of the OASPA decision. Our experience with the Bohannon paper taught us that we need to continually improve and adapt our systems and continue to keep them as up-to-date and secure as possible. We think that adopting the recent peer-reviewer suggestion may avoid anyone else taking the view on that numeric scale that he had.
I’m not sure that we need a “reject” option… assuming the paper fits the broad aims of the journal and didn’t violate any explicit rules (e.g. is not plagiarism, all authors contributed and agreed to submission, the authors own the data, no fraud) the old option 9 seems just find to me. Specific further comments could include: prior to accepting this publication, 1) all experiments need to be completely redone in line with scientific norms 2) appropriate controls need to be performed 3) a discussion merited by the results should be included 4) more analysis needs to be done 5) or even…. experiments that would make this finding somewhat interesting to someone in the world should be performed before publishing this paper.
Basically, as long as the comments can say “You really need to start over from scratch…. but we aren’t rejecting you per se” I see no reason why “Paper has major flaws and requires considerable modification to make it acceptable” is not OK. I guess the problems are 1) if the author keeps re-submitting without actually addressing the concerns fully it would be a waste of time 2) if the author quits the review processes without the paper being formally rejected, it could be accepted by a less stringent reviewer.
TL;DR: I don’t know that a review should need to have the option to reject a paper, so long as they are allowed to not endorse its acceptance.
The options are just recommendations, but I prefer that these recommendations are aligned with the options that the editor has. So as the editor can reject a manuscript, I would like to be able to explicitly recommend that course of action. The few journals that I have reviewed for are like this; as a reviewer I have had no more than four options: accept without revision, minor revision, major revision/reject with option to resubmit, reject/reject without the option to resubmit. It certainly makes it easier to make the recommendation because you know exactly what these things map to.
I guess I meant that, more inline with the PLOS ONE et al. philosophy, since journals have virtually unlimited space to store/share articles, once it gets passed the screening editor as legit (and unless it later violates a rule), I don’t understand why an article ever needs to be rejected, either by an editor or reviewer. Presumably there is always an amount of better experiments and writing that can make an article meet the scientific standards of any journal. Rejection only becomes an issue when a journal needs to select the most “interesting” articles to increase its impact.
Which journal is more legit?
The Hypothetical Journal of Excellent Science, where no paper is ever rejected, but only papers with the absolute highest level of rigorous science and thorough discussions are accepted.
or
The Journal of Nature, where papers with low citation potential are rejected regardless of quality and papers with high citation potential are given some leeway.
It matters much more that there is a stringent requirement for acceptance than an option for rejection.
Agreed it is nice to have the reviewer summary boxes in line with editors’ options. But the idea that this was a “brazen violation” of norms that “mocks the peer review system” by not allowing reviewers to reject a paper, does not accord with my experience with reviewers, or academics in general, who are an independent lot whose living is made by presenting/arguing their views to the world. Of course reviewers will give their honest assessment without holding back (sometimes one wishes they would hold back just a little! or at least be more polite about it), regardless of the wording of some button option. To me, the phrase “major flaws and requires considerable modification to make it acceptable” is more or less just a polite way of phrasing “sorry Bub, doesn’t pass muster.” (Is Dove Publishers England-based, by any chance? UK form tends to more polite, less-blunt wording, imho.) That doesn’t mean they actually have lower standards for what makes it into their journals (I have no idea what their standards are.)
if I can be a little blunt here…of course Dove Press has not a “reject” button..it is a potentially predatory open access publisher! as Beall says in his 2012 list: “Dove Press
This New Zealand-based medical publisher boasts high-quality
appearing journals and articles, yet it demands a very high author fee for
publishing articles. Its fleet of journals is large, bringing into question how it
can properly fulfill its promise to quickly deliver an acceptance decision on
submitted articles. ” (here is the link: http://carbon.ucdenver.edu/~jbeall/Beall%27s%20List%20of%20Predatory,%20Open-Access%20Publishers%202012.pdf
so…this is why is quick…
I recently reviewed a paper for some reasonably prestigious journal that did not in fact have any options for the reviewer to choose. It just asked for the reviewer’s comments on the article but not for an opinion on the final disposition of the article. In fact, I wonder if BMJ actually asks for the reviewer’s opinion as to acceptance or rejection or whether that is decided by the editorial group.