Ever read a review where the editor or reviewer seems to be specifically looking for reasons to reject a paper? Neil Herndon, the editor-in-chief of the Journal of Marketing Channels, from the South China University of Technology in Guangzhou, has. In a recent editorial, Herndon calls this type of review “gotcha” peer reviewing, and presents an alternative.
Retraction Watch: What is “gotcha” reviewing? What is its purpose and who is practicing it for the most part?
Neil Herndon: Gotcha Reviewing occurs when the editors and reviewers at a journal emphasize finding what is wrong with a paper to provide a basis for rejection. They are thinking first of rejection rather than how to improve the paper given that there are no “fatal flaws” in the research, that is, flaws that cannot be fixed. I associate this practice with top-tier journals as they use it as a screening method to reduce editor and reviewer workload due in part to their tremendously large number of submissions. Unfortunately, a lot of really fine research with interesting ideas gets lost in the process.
Gotcha reviewing also allows a journal to inappropriately quickly screen out more junior researchers (among other groups) along with interesting and valuable research findings that may expand our understanding of marketing. I believe as do many others, but cannot prove, that paper acceptance in top-tier journals may revolve around more than the quality of the research and its presentation: gender bias, regional bias, seniority bias, and affiliation bias are all suspected by authors, reviewers, and editors according to a global study published in 2015 by Journal of Marketing Channels publisher Taylor and Francis.
RW: What is “developmental” reviewing? What are its key distinctions from the “gotcha” practice?
NH: Developmental Reviewing, which I believe is the gold standard for journals truly interested in helping guide quality research, occurs when the emphasis is on providing authors with additional insights and advice to improve the quality of their work in a double-blind, helpful way rather than thinking first of rejection (as is done in gotcha reviewing). This provides what we hope is welcome support, especially so for new researchers in a publish-or-perish world.
RW: I would imagine most journals say that they are, or would like to, practice “developmental” reviewing. How many actually are doing so, in your opinion?
NH: If a journal truly wants reviews to be helpful and collegial, then an editorial attitude and editorial review board culture of developmental reviewing will go a long way towards making that possible. If a journal truly wants to publish papers from all segments of the marketing community studying areas within that journal’s aims and scope, developmental reviewing will help open its pages to quality work in an unbiased way.
I can’t guess how many journals take a developmental approach, but I can say that my own experiences and those of my colleagues and research partners suggest that not many do so. I also find that some editors and reviewers are unkind, unhelpful, and certainly not collegial in their approach to writing reviews, a situation that can certainly discourage junior faculty new to the publish-or-perish world of academia.
RW: Do you think all journals should practice “developmental” reviewing all the time or do you think there’s also room for “gotcha” in some instances?
NH: I believe that everyone that does the work of conducting research and submitting a paper to a journal deserves a fair and unbiased evaluation of their work: this is the core of developmental reviewing. The core of gotcha reviewing is to reject as many papers as possible without regard for their possibilities: this is an unfortunate and unfair approach.
That said, there are papers that I desk reject all too often because they are outside of the aims and scope of the Journal of Marketing Channels, that are of extremely low quality such that developing them into an article is not possible, that have fatal flaws in their methodology (especially data collection and research design) that cannot be repaired with a different approach or statistical analysis, and papers that, unfortunately, have been previously published or otherwise engage in unethical research practices such as plagiarism.
RW: What tips would you give journals and reviewers to make their reviews more “developmental” in nature?
NH: First of all, a reviewer has to invest the time to really carefully read a paper, deeply understand its theoretical base and methodology, meticulously examine its hypotheses, statistics, and results, and see if the discussion insights, theoretical implications, managerial implications, public policy implications (if any), and future research suggestions are all supported and fit together as a unit. Only then should the reviewer begin to consider any criticism of the work and make suggestions for its improvement.
All too often editors and reviewers decide to reject a paper based on the first few pages of text, sometimes just from reading the abstract, and then look for gotcha rejection opportunities. Developmental suggestions must come from a deep understanding of the researcher’s intentions and approach with the reviewer making comments as though he or she were a coauthor sitting at the desk with the author(s). Only then can the reviews be truly developmental.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.