Weekend reads: How to fix “slow,” “unhelpful,” and “generally awful” peer review, where all the PhDs go

booksAnother busy week at Retraction Watch, but there was lots happening elsewhere, too:

16 thoughts on “Weekend reads: How to fix “slow,” “unhelpful,” and “generally awful” peer review, where all the PhDs go”

  1. Concerning the “terrible academic job market” it should be clear that there are always a number of brilliant professors sons who decide to follow their fathers – and some of those are really brilliant to deserve such an easy career, but I have seen more than questionable cases which jsut show how corrupt the whole academic system is when only the professors son has a chance to get research and working conditions 100 others in an institute don’t get. It is clear that each institute has to compete with other institute to get the sons of the most prestigious professors to increase the ranking of an institute or host professor.
    Concerning the “peer-review process”, I couldn’t beleive it until recently that some journals appear to accept contributions from those who previously published there even if the conclusions and experiments are totally wrong. It is shocking to see how many papers are based on questionable experiments or wrong conclusions and still the bad scientists will be promoted to an academic or entrepreneurial career since they were able to trick the journal editors…

    1. That ACS editorial is one of the precise reasons I believe why science publishing is rotting. Here we have 7 editors who come together to write a one-page editorial, probably half that size if you remove all the noise and references. I ask, how is it possible that such a superficial Q&A session by 7 editors require 7 individuals to write such a small piece? I would really like to know the exact “authorship” roles of these 7 editors. In fact, I have seen quite a few editorials that should, frankly speaking, be retracted, because they are either so superficial or so shallow that they don’t actually contribute much to adacing science’s knowledge. As for their question about their authorship, yes, I guess it might be restricted to their journal. That’s because I occasionally ask editors, as an author, particularly when there have been no peer reports or shallow peer review: “Why Did You Accept My Paper?” In fact, my record indicates that I have withdrawn at least a half dozen papers from journals, in some cases, even after accepted, because they did not conduct peer review, somewhat like Bohannon experienced in several cases of his sting, with the exception that my papers were real, not contrived, with real authors, e-mails and affiliations, and not fake ones.

      Separately, excellent to see the fighback by The Ecologist. Let’s hope that the NRA answers its pertinent questions. It is quite evident that the NRA, with its powerful legal-backed team, has no respect for nature. This aggressive superior attitude of “human” over nature needs individuals to fight back. Seeing attitudes like those expressed by the NRA (an din fact gunholders and hunters who misguidedly think that nature was created for them to kill, and to make sport and profit) should, in an evolutionary sense, make us feel embarassed to be humans on this planet. If for no other reason, I support The Ecologist’s resistance to heavy-handed, lawyer-doctored attempts to retract science- and evidence-based studies in defense of the condor.

      1. Eibl.
        There is no question that each and every academic has recognized the serious flaws in current peer review, but why do the majority deny it? why hasn’t there been any serious effort to reject it yet?

        JATdS.
        I love your logic and courage.

        1. To give you a real-case example. Just today, I withdrew a paper from submission to Botanica Serbica. I had submitted on March 26, 2014. The paper was acknowledged a month later (not so bad), but after re-submitting following some editorial requests, the lack of an acknowledgement e-mail, the failure to assign a manuscript number immediately had me concenred 2 months later. I immediately contacted the editor, Prof. Marko Sabovljević, who stated “do not expect to have MS published prior to late 2015 or early 2016 issue”. I found this astonishing. If so, then why did he not indicate this vital information upon submission? Displeased with this editorial handling, I withdrew my paper. I should note that Prof. Sabovljević is from the University of Belgrade – Faculty of Biology, the same location of another Serbian journal that failed to conduct peer review and then demanded an exhorbitant fee for producing a PDF file:
          http://retractionwatch.com/2014/07/07/serbian-journal-lands-in-hot-water-after-challenge-on-24-hour-peer-review-that-cost-1785-euros/
          Also, the academic concerns about Serbia have been highlighted at RW previously:
          http://retractionwatch.com/2012/12/17/serbian-scientists-decry-systematic-plagiarism-predatory-publishing/

          1. Why not upload your submissions to ResearchGate in order to protect and secure your ideas until your articles get published. I am sure many academics “including me” would love to follow your updates.

          2. Aceil, I am strongly anti ResearchGate and LinkedIn. Those two are separate cases that I also hope to publish to reveal my reasons why, but with over 160 folders of to-write papers, I am not sure how I am going to do it… I am now testing quite alot of open access publishers and some alternative publishing options, with alot of experimental stuff taking place, because I feel that I need to present ideas and alternatives at least to colleagues in the plant sciences. One could say that I am doing full-experiments on myself, with my own ideas and with some colleagues, that could either be highly successful, or a total disaster. Self-publishing is the last option, least attractive because it would be the most criticized (and perhaps least respected) by peers. That is why I still prefer, for now, to stay on a fairly conservative path with standard journal and publishing options because the great majority of the peer pool in plant science is too fixated with this model to be able to appreciate anything else among the alternatives. I feel that this ultra-conservative bunch will take decades to evolve.

          3. My main argument against traditional publishing is the lack of accountability; unethical reviewers can steal your ideas and plagiarize your work, by the time you have discovered that; people could hardly believe those words were yours in the first place, Therefore, people who have the courage to speak out the truth about what most academics suffer from silently, should find a vehicle to communicate and preserve their ideas. Not to mention that Ideas are commodities after all.

            Here is a discussion of some available tools:

            http://www.the-scientist.com/?articles.view/articleNo/36695/title/Preserving-Research/

          4. Aceil, I value your comments and suggestions. Self-publishing is definately the future, but even so, it may actually allow for more fraud to proliferate because if dishonest scientists or those without a backbone embrace this model, then we will have many fold more rubbish being read by Google. So, as far as I see it, self-publishing is only viable for a limited slice of academics, not for the masses. Centralization of ideas also has value, and risks. But many of these concepts are still experimental, and thus represent a danger to academics because of the time and effort required to write a paper, often several months for each paper. There are three clear issues, as I see them: a) preserving ideas and intellect; b) preserving research; c) preserving data. And, the preservation of each might very well require different approaches, and not a onefits-all approach. At the moment, patents, which I personally don’t like, serve the scientific community well for a). I believe that open access repositories like DOAJ serve well for preserving published ideas, but risk fluctuations in moods, e.g. removal of titles after the Bohannon sting. So, we are still extremely far off a solution, much less a solution for everyone. But, it’s good to talk about these issues because 100 years from now, when we are no longer here, we want to know, how has our intelect and effort been preserved? In my case, for example, a fairly fierce critic (and equal supporter) of some publshers like Elsevier and Springer, imagine in the future a management comes to power that adopts a policy of wiping out literature by critics of the company, i., retractions by force. I know it sounds unbelievable, but tyranny can affect any structure in society. So, imagine that happens? That would wipe out about a third of my publishing CV. These issues need to be considerdd in this debate, too.

    2. With double-blind peer review, neither the editor who makes the decision whether to send the paper out for review, nor the reviews, are told the names of the authors or their affiliations. They therefore have to make decisions based on the scientific content of the paper. It would reduce bias, conflicts of interest, nepotism, and collusion.

      1. Although reviewer bias is a legitimate concern, and double-blind peer review would seem to be more objective, most studies of double-blind peer review have not shown much impact on review quality (McNutt et al. JAMA 263:1371, 1990; Godlee et al. JAMA 280:237, 1998; Van Rooyen et al. JAMA 280:234, 1998; Justice et al. JAMA 280:240, 1998) and may not even be feasible in many cases (Yankauer. Am J Public Health 81:843, 1991; Cho et al. JAMA 280:243, 1998; Katz et al., AJR Am J Roentgenol 179:1415, 2002; Alam et al. Br J Dermatol 165:563, 2011). Nature looked at double-blind peer review a number of years ago and apparently decided against it (451:605, 2008). Nature Geoscience and Nature Climate Change are currently offering the option of double-blind review on a trial basis but so far most authors have not opted in (http://www.nature.com/news/journals-weigh-up-double-blind-peer-review-1.15564). In the economics field, Blank found that double-blind reviewers tend to be more critical (Am Econom Rev 81:1041, 1991). In my field of molecular biology, it is generally straightforward to identify the lab of origin of a paper based on the strains/constructs used and the cited studies on which the new work is based.

        1. I think that reviewer bias is the main reason so many irreproducible papers are published – the reviewers don’t even need to read papers coming from their cabal. In molecular biology it may well be that the lab of origin can be identified if you study the details of the paper, but then half the battle is won! Just because Nature decided against DBPR doesn’t mean it is not worthwhile. Maybe they don’t think they have a problem with bias in the papers they publish (just as they don’t think they need any advice from COPE). Having DBPR as an option would not work, because those who are part of the ‘club’ would just opt out. Given that most published findings are false, or can’t be reproduced, having more critical reviews might be a good thing.

  2. Concerning the printed matter from Veja magazine, it is interesting to note that over the last 10 days there has been a wave of articles on scientific misconduct in common media in Brazil. One thing shared among them that displeased me is a misconception of the word “Retraction” which is being presented in the sense of “pulling back”, claiming that fraudulent papers should be pulled or wiped out of scientific literature. Please someone correct me if I am wrong, but doesn’t the term “Retraction” come from the necessity of a public retraction from authors and editors on any printed material that later it proves invalid or inaccurate? From what I understand, it is actually another communication, and the original should remain, though making clear reference to this public retraction. I wonder if publishers try to manipulate the meaning of Retraction as basis to justify disappearing with the problematic paper without notice, which I have seen happening in many periodicals? The practice seems akin to burning the evidence.

    1. Last weekend, I posted my personal public complaint about editorial incompetence at Springer’s JSEE:
      http://retractionwatch.com/2014/07/12/weekend-reads-peer-review-unreliable-merck-retracts-legal-threats-over-criticism/#comments
      I had called publically for the resignation of Dr. Spier from her position as co-EIC of JSEE. One week later, the result has arrived. My manuscript was rejected. I sent a copy of that decision letter to RW, and I would hope that it could be added as an open access document. It is these incidents of inefficient peer review that embiter and anger scientists. Incidentally, the impact factor of Science and Engineering Ethics is 0.901.

      1. Self-publish it! If your results are sound and you have a good scientific reputation, your readers should be able to find and appreciate your manuscript and finds given they are open online.

        1. CR, see my link here: https://retractionwatch.files.wordpress.com/2014/07/jsee-r1-teixeira-da-silva-jan-17-2013.pdf . It is now open access through the RW portal, open for all to screen, comment and criticize at will. Note very carefully in the acknowledgements section that my paper had been pre-peer reviewed by three scientists, one known and two unknown, traditionally peer reviewed by three JSEE-selected peers (was the copy provided blind, or was the process double-blind?), and viewed by at least Dr. Bird and Dr. Spier. I even gave Elsevier Ltd., COPE, the ICMJE and Neurology an opportunity to respond. The former two did not. These peer-reviewed facts speak for themselves. Incidentally, regarding your criticisms about the system of retraction = completely erase in Brazil, can you provide a list of as many examples as possible here at RW? This is an important observation, and phenomenon that needs to be explored. Actualy, if the evidence is totally removed, then how does one know what literature was actually retracted from the Brazilian literature? I assume that you are referring primarily to Scielo-related papers, right?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.