After two decades of submitting papers to journals, and more than 10 years of serving on an editorial board or editing journals, geography researcher Kevin Ward knows a thing or two about peer review.
Recently, as the editor of Urban Geography, he received a particularly “grumpy” and “obnoxious” review in his inbox, which got him thinking. Although, he says, the review raised “professionally appropriate issues,” it went well beyond the widely accepted content and tone. Ward, therefore, decided to reflect on his two decades of experience, and decipher the different types of reviewers and their characteristics.
In all, Ward — from the University of Manchester in the UK — says he’s encountered six types of referees.
Here’s the first, according to his recent editorial published in Urban Geography:
1.The “bitter and twisted” reviewer
Think of the unpopular kid at school, who used to sulk in the corner. Ward describes this type of reviewer as:
…this review reveals a reviewer’s sense of being out of place in their field, a situation with which they are not happy and that comes through in the review.
2. The “enthusiastic and supportive” reviewer
This type of reviewer is a cheerleader — of the best kind. It’s the type of reviewer you want. Ward says
…this review takes the paper at face value. It avoids ending up arguing for the writing of a different paper. Rather it works with what has been written, detailing how given these parameters the paper can be improved.
3. The experienced reviewer (whom Ward calls “In my day…”)
Every school has at least one teacher who talks about how great things used to be back in the day, when life was simpler, students showed more respect, etc. Science has those too, says Ward:
…the paper under review is used to speak on behalf of wider trends in a particular field. There was once a time, according to this reviewer, when the field was better, fitter, stronger and so on. Now the field is worse and this paper is an example of this decline.
4. The “it’s all about me” reviewer
Watch out — you don’t want to get the next type of reviewer:
…for some reviewers, the review is a chance to showboat, to use it as means of generating citations for their own work. As Spiegel (2012, p. 1334) notes “Referees not only want authors to cite their work, but they also want the text to laud it as well. Whatever the motivation—and we can all easily think of several— referees cannot directly demand others affirm their publication record.” He is right, of course. However, this type of review can come close to demanding that author/s cite the work of the reviewer!
5. The annoying reviewer
This type of referee would be banned from the soccer team (pun intended); Ward says this reviewer likes to engage in “goalpost moving:”
Presented with a revised paper and a covering letter detailing how the author/s responded to the original set of reviews, this review writes “yes … but”. It asks for a second set of revisions based not on the original paper but on the revised paper. The review has moved the goalposts, potentially resetting the review process.
6. The brief reviewer
Think of this reviewer as the student who stops writing a paper the moment he reaches the minimum word length. It’s someone who leaves you with “not much to work with,” writes Ward:
These reviews are short and very to the point. Whether they are negative or positive, the content is so minimal that there is relatively little with which to work as an editor. Often no more than a few lines, there is little explanation or justification for the decision.
Jokes aside, Ward finishes on a serious note:
…the next time you write a review think about how it will be read by an editor as well as by an author, not just on its own but also along the reviews of others.
These are Ward’s picks — what are yours?
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
I’ve encountered two more types:
1. The jealous person whose own research in the same sub-field has not progressed much lately, so he’ll poke holes in the writer’s balloon.
2. The reviewer with very little English, who writes “lot of mistype and misspell [sic] on page 3, line 25”, when in fact, there aren’t any. Any attempt to state the truth concisely in the response (“There are no mistypes or misspellings in that paragraph”) will be seen as unkind or confrontational by the editor, when it is simply a fact.
Well written. As an author and reviewer, I have seen almost all these types of reviews. I can say that unfortunately the “it’s all about me” type is more common these days. I have seen two interesting cases recently. The first, I was serving as the second reviewer and during the second round of review I noticed that reviewer #1 has suggested 11 of his own papers (with minimum relevance) and asked the author to cite them in the paper. Fortunately, the author complained about this shamefully unprofessional behavior to the editor. The second case was even more funny. This time I was reviewer #1 and noticed that reviewer #2 has recommended 7 of his papers. To my surprise all these 7 papers were written by the associate editor of the journal, so I am sure he was reviewer #2. Although I wrote a comment to both the author and editor that this is not acceptable, you can guess that the author had no option but to add the papers to the list of references. I have seen many other unprofessional behaviors, but these two really stand out. I am now making an archive of these reviews and publish a commentary on this issue in the future.
I had one paper reviewed by type 4 who claimed that he had done it all, complete with references. I read them, couldn’t find it, did find one useful reference and decided not to cite it unless I was asked to after review at another journal. At next journal had no 2 who nicely let me know about some work that he had done that was in press and could be used to justify my models.
Missed the “I don’t have a clue” reviewer, who tends to miss the errors that the more competent reviewers find, and then makes comments that are clearly wrong showing a lack of expertise in the field.
The complaint about ‘moving the goalposts’ troubles me a little. If a reviewer finds something wrong in the revised paper that they overlooked the first time, isn’t it preferable from a scientific point of view to mention it?
Like most things it depends. Some reviewers will come back with requests for unimportant changes. Sometimes they do miss things and sometimes the changes to the other parts of the paper will require changes, so it is all OK.
One thing that does become annoying is when it appears that the reviewer wants the paper to be how they would have written it. Then they do start asking for lots of little changes.
I had a similar response. Sometimes it needs a first round of revise & resubmit to really see the forest for the trees. It’s rare that a very badly written paper ever the chance of a resubmit. But when that happens and everything is suddenly that much clearer, this may be the time that as a reviewer (standing in for future readers) finally understands the real holes and problems in a paper. I’ve seen that happen, both as an editor and as a reviewer.
You are right to be concerned about that, but the “moving goalposts” refers to a reviewer that asks for changes and when the author/s address/es these changes they ask for further changes to their original request (i.e. asking for further specific control data in the first review, then asking for more control experiments directly related to the first request in the second review). Of course sometimes things are missed and it is important to point these out, but more often it seems that a reviewer is just trying to make it difficult for the author/s to publish the work as they could have asked for all follow-up work in the first review. Specifically, this is annoying if the authors have already recapitulated the data by multiple methods and the reviewer is only causing delays to the publication by asking for additional work. Here editors should play a stronger role at making sure reviewers are not making gratuitous requests that simply delay or block publication. I had once submitted a manuscript where we had functionally characterized a large eukaryotic membrane transport protein. We created a homology model to help us hopefully create more intelligent mutants that would yield functional differences. In the SECOND review I was told that we should obtain the crystal structure because a model was insufficient (no eukaryotic structures of this transporter family currently exist). This was way outside the scope of the paper and if it was an issue it should have been addressed in the first review (not a small oversight). This is purely publication blocking and means the process was a waste of several people’s valuable time.
I had a reviewer like that for one of the first papers that I submitted. Time after time, I made the requested revisions only to get a review ask for more revisions. Finally, the reviewer wrote something to the effect that it was a very poorly written first draft. What could I say? I had made every change that he or she had requested so it was a rough first draft, it wasn’t mine but the reviewers. I gave up and went on to other things.
As a post-doc, I received a review of a paper that was largely a personal attack on my supervisor at the time, with a critique of the science in distant second place…
I’ve never been editor for a journal and I’ve written more reviews than I’ve had reviewed, but this typology (and the additions above [below?]) ring true. I’ve always tried to be #2, which once or twice was rather difficult to do. I also make sure to differentiate between the “this point really needs to be improved” and the “I’d like to see you to do this, but it’s up to you” bits. If anonymity for a reviewer is not mandatory, I almost always give my name.
I once received, instead of the text of a review, a note from the journal editor that the reviewer had proven to be hostile to the use of statistics, and the editor had suppressed the review. Thank you, journal editor! I’m sure it would have been bad for my blood pressure. (I am pretty sure now that my article got caught in the crossfire of the cladistics/phenetics debate, but at the time I was simply mystified.)
My personal most-hated review circumstance, which has happened to me twice, does not involve paper reviews but grant reviews. It’s the combination of a review that says “This is pretty decent, I have no complaints” and a granting agency conclusion that says “Your grant is rejected; If you meet all criticisms by the reviewer we may reconsider.” What do you DO with that? Just try writing the response to reviews!
Definitely had an extreme case of #4 reviewer for a paper.
Comments were about 3 pages long and almost every point they made had a reference to themselves, some in the most obscure journals possible (some not even in the journal’s native English).
In the followup, when we declined to cite them for the Nth time they rebutted with a sentence they had written themselves, citing themselves, that they suggested we use…
The editor stopped it at that point.
When I was applying for grants to fund my PhD, I had a real dick as reviewer #2. The guy was meant to review our research proposal as well as the quality of the institute, the applicant (me), and my supervisor.
When judging my prospective supervisor (who is the head of a large medical instution’s research department, and who leads several international collaborations in his field), reviewer #2 said ‘Don’t know him’. Where asked to judge me, he just left the form blank. And when asked whether our institution looks fit for the project, he just commented ‘No opinion’.
Coming to the research proposal itself, he told us that basically he doesn’t understand why we would want to study this, and why we wouldn’t want to repeat some experiments from the 1980s. We replied that these old studies had repeatedly been falsified (that’s a harsh word, but basically they were so fundamentally confounded that they are no longer considered in our field).
So in sum, our reviewer #2 didn’t know anything about the methods in the field (s)he was supposed to be an expert in, didn’t know to comment one of the biggest institutions in his country in that field, and likewise had never heard about one of the most active supervisors in that field.
Reviewer #1 on the other hand sent us a glowing review that even contained some practical suggestions for our proposed experiments. These came in handy when we finally prepared the experiments. But at the time it was said that in order to get this grant, you really need two glowing reviews, and accordingly we weren’t considered there. Fortunately, I had more luck next time (and even got a better deal).
No one likes reviewer #2.