After five years of operation, the Nature Publishing Group is will no longer accept submissions to its preprint server Nature Precedings, having found the experiment “unsustainable as it was originally conceived.”
Here’s the announcement sent to all Nature Precedings registrants this morning:
As you are an active user of Nature Precedings, we want to let you know about some upcoming changes to this service. As of April 3rd 2012, we will cease to accept submissions to Nature Precedings. Submitted documents will be processed as usual and hosted provided they are uploaded by midnight on April 3rd. Nature Precedings will then be archived, and the archive will be maintained by NPG, while all hosted content will remain freely accessible to all.
Be assured that Nature and the Nature research journals continue to permit the posting of preprints and there is no change to this policy, which is detailed here.
Nature Precedings was launched in 2007 as NPG’s preprint server, primarily for the Life Science community. Since that date, we have learned a great deal from you about what types of content are valued as preprints, and which segments of the research community most embrace this form of publication. While a great experiment, technological advances and the needs of the research community have evolved since 2007 to the extent that the Nature Precedings site is unsustainable as it was originally conceived.
Looking forward, NPG remains committed to exploring ways to help researchers, funders, and institutions manage data and best practices in data management, and we plan to introduce new services in this area. We have truly valued your contributions as authors and users to Nature Precedings and hope that you will actively participate in this research and development with us.
Nature Precedings — which has become home to thousands of manuscripts, posters, and presentations — came up a number of times in the comments about a post we ran earlier this year about F1000 Research, which, as we noted:
will publish all submissions immediately, “beyond an initial sanity check.”
So there were some similarities between the two efforts, but also some differences, as F1000’s Rebecca Lawrence pointed out.
It’s unclear how long Nature Publishing Group has been planning to cease accepting submissions. We have a message in to Nature and will update with anything we find out. In the meantime, here’s a one-year anniversary post from the Nature Precedings team, from 2008.
Update, 3 p.m. Eastern, 4/1/02: We asked F1000’s Rebecca Lawrence for comment:
I suspect that many people will use this as evidence that such a model doesn’t work in the biomedical community (as compared with ArXiv in physics-related communities) but I don’t believe that this announcement actually really does show that. I don’t know the full details behind the model that NPG had original planned for Nature Precedings and hence on what basis they have judged that it was not a success. A preprint server is certainly unlikely to be a service that will make money in itself for a commercial company, and of course ArXiv is not run by a commercial organisation. We certainly had no expectation of making money from our preprint service, F1000 Posters.
Furthermore, to build up F1000 Posters (and in fact with any such new venture), we have had to invest quite some time and effort in partnering with learned societies to inform their presenters of the options to deposit with us and to also clarify the positions of the various publishers with regards prior publication. We are now finding this is paying off as researchers become increasingly comfortable with the idea of submitting their work to such a service. With regards F1000 Research, this is quite different from a preprint server (as we have clarified previously), and it will be based on a now well-established and successful business model of gold open access, i.e. article processing charges, so I don’t think comparisons can be made between these two. In many ways, we see F1000 Posters as a precursor to F1000 Research and therefore feel that we do have a sustainable long-term plan which one must assume is somewhat different to that which NPG had envisioned for Nature Precedings.
Hat tip: Sierra Rayne. Jonathan Eisen has also posted on this, and in a comment on his post, Nature Publishing Group’s Grace Baynes confirms that the email is genuine.
Inevitable I’m afraid. The F1000 is doomed to failure as well I suspect and I think a lot of us were surprised when F1000 came out with their announcement. The whole post-publication peer-review thing is a great idea in THEORY but it just wont work, especially in biological sciences. I just wish publishers would stop trying to force feed it to us.
Agree with Dave. We had this discussion earlier on F1000 http://www.retractionwatch.com/2012/01/30/an-arxiv-for-all-of-science-f1000-launches-new-immediate-publication-journal/ – however, Rebecca Lawrence from F1000 did not agree…in that blog, Nature Precedings was also discussed.
It’s rather unfortunate, but not unexpected. Science is still in difficulty. The so-called “peer-reviewed” journal system (a.k.a., let’s all mark our friends’s exams) is too corrupt, and there are few other credible outlets for work,
Interesting point of view. I don’t agree ‘though. Nature Precedings didn’t work because there isn’t really a market for this approach, at least in the biomedical sciences. Maybe it can be considered an interesting foray towards novel publication approaches that might hit the mark in the future.
Is science in difficulty? I wouldn’t have said so in a fundamental way. Of course there are too many pressures to publish and this supports the prollferation of a whole slew of sub standard journals. But much of that is immediately ignorable, even if it is a bit of a time waster.
Th fact is that good science continues to be funded, to be performed and to be published, and peer review (in which I include the most important element a.k.a.”self peer review”) works, if somewhat messily. Here in the UK the problems revolve around a considerable cut-back in funding, misguided efforts to impose corporate-style managment structures onto academic research, the pressures involved in meeting government-imposed “research quality assessement” exercises…that sort of tedious stuff…
…but scientist continue to find out stuff and to publish their work. As for “other credible outlets for work”…what’s wrong with good scientific journals? Anything that’s worth publishing can be published…
Didn’t think you would agree. Those in the establishment never do. “Anything that’s worth publishing can be published.” This is nonsense. Plenty of excellent work goes unpublished in traditional peer-reviewed journals for no other reason than pure politics. As for peer-review working, if that was the case we wouldn’t see the proliferation of junk in the literature. Maybe your specific field doesn’t have a lot of junk being published in its top-ranked journals, but mine does – as do many other fields (as evidenced by the many stories on this website).
I should also note how amusing it is to see scientists publicly complaining about lack of funding, accountability, etc. Like we would expect those with direct vested interests to feel any different? Hmm … conflict of interest problem? Does anyone in govt think their budgets are too big? that they are not accountable enough? Academic and govt scientists are civil servants – nothing more, nothing less.
Your suggestion that “Plenty of excellent work goes unpublished in traditional peer-reviewed journals for no other reason than pure politics” seems an unsubstantiated one to me! Like everyone else I’ve had papers rejected from journals occasionally…but you just send them some where else and in the end it doesn’t matter that much (if the work is good enough it will make an impact wherever it ends up). Your suggestion seems a little conspirary theorist to me! I’ve been doing science for a while and I’ve never come across anyone that hasn’t been able to get their work published. Of course I can only speak for the broad Physical Chemistry-Biochemistry-Molecular Biophysics-Molecular Biology fileds that I have experience of…
“Proliferation of junk”…yes…but that’s largely down to the proliferation of junk journals and these can largely be ignored. There is some low level of scientific fraud as attested by the examples on this site! These can’t easily be picked up in peer review (that’s why I emphasised the importance of “self peer review” which most scientists conform to in my experience). The pressures to publish does mean that too many papers are submitted and this does reduce the standards a little on average I think.
…but I can’t help noticing that the good stuff that drives research fields forwards continues to be published! And I wouldn’t consider myself “establishment” – in fact I’m a researcher that currently struggles with funding, but still gets a kick out of tackling interesting problems and writing nice papers…
Sorry, Chris, your efforts seem either naive or ‘spewing the party line’ to me. I can give you many examples of absolute junk in the leading environmental science journals, and a number in leading chemistry journals, so for you to suggest that junk is primarily limited to so-called low-end journals is simply not consistent with reality. I assume you have quantitative evidence to back up your claims? Want to go through them one-by-one on this site?
As for conspiracy theories, absolutely. Science is a small community, and run very much like a non-democratic state. It needs to be made fully transparent and accountable. Sounds like you don’t like transparency and accountability?
As for picking up the fraud, the current peer-review system is fully to blame for most of those. As we’ve seen repeatedly on this site, the fraud is often obvious. Marking your friends’ exams (i.e., peer-review) as some determinant of career success, public policy, etc., is a joke – and logically indefensible. Sometimes it works, sometimes it doesn’t. Sounds like a poor approach to me.
You sound like the type of guy that will defend science regardless, so you’re effectively just a self-serving lobbyist, which translates into low credibility. Yes, I’m sure your taxpayer-derived funding levels are tragically low …
I’m happy that you’re amused Sierra! You did raise the notion that “science is in difficulty”, and it’s worth highlighting some of the real issues that make doing science rather difficult at the moment in the UK. If the UK research councils are only funding 10-15% of applications, and if you are fortunate enough to get a grant the limitations of funds means that any capital equipment expenditure is either not met or is only funded at 50% of costs, then that does make doing research very hard work.
The management structures being imposed on academic research are not related to “accountablility” which has always (at least in the last couple of decades) been a fundamental part of academic research. They’re more to do with attempts to manage a suddenly shruken funding base…that’s fair enough…the problem is that the mechanisms by which this is being implemented are widely felt to be making things worse…we’ll see how this all shakes out ‘though, and those that have the stomach to push on in difficult circumstances will continue to do good work (and publish good papers!)…
and academic and govt scientists aren’t civil servants. They’re scientists! I expect that as in the past publically-funded science and the knowledge base it builds will continue to make huge contributions to our economies and well-being….of course I imagine it can be satisfying to trash this effort!
Getting back to the point of this thread, there clearly isn’t much of a market for a repository of bits and pieces of this and that…I imagine it might be of some interest to some “depositors” (maybe for purposes of claiming priority; ‘though in the Biomedical Sciences there are some v. rapid publishing sources that support this purpose) but it doesn’t seem to be of much value otherwise…maybe the Nature Precedings models simply wasn’t quite right….however I can’t see how lowering the essential standard of quality helps very much in the process of scientific advance.
And so my challenge to the anonymous chemist Chris from the UK (Bristol?) is simple: let’s go through a bunch of specific examples that I’ll suggest one-by-one on this website (right here and now). We can deal with the peer review system (I’ve recently blogged on how an ISI Highly Cited Researcher suggests [and obtains] his own Ph.D. student and/or his colleagues as reviewers for a well-known journal), specific examples of junk science that a reviewer would have to either be blind or dumb not to pick up, etc. Care to accept?
Well yes, one should defend science! It’s the fundamental method for finding stuff out, advancing our knowledge, supporting our economies, giving us the potential to live healthier lives and so on…! There are some astonishing anti-science efforts that aim to suppress important knowledge in many fields, and on should be robust in defending an astonishingly productive process.
Is a lot of the fraud that is highlighted on these pages so obvious? I’m not so sure – sometimes it appears to be so in hindsight. It may be a mixed blessing, but peer-review does rather rely on the presumption of “good faith” on all sides. Sometimes this turns out not to be justified sadly. My feeling is that efforts such as this website will have a large effect in squashing the sort of frauds (image manipulation/plagiarism) that are largely associated with the newish electronic age. Once the cheats realize that this stuff is actually quite easy to identify using electronic surveillance, it may well shrink back. And some of the large-scale serial frauds are getting caught, and the fact that these are being increasingly highlighted may wll help to curtail tha sort of rubbish too. Can’t praise RetractionWatch too highly for highlighting this stuff…
My experience seems to be quite different from yours. I try to work on important and interesting problems and to publish good stuff. Pretty much everyone I know in science has a similar philosophy…to you that seems to be a major negative worthy of ridicule and insult!
Come now, Chris, you’re pretty keen to attack others as well. Spare us all the innocence of science stuff. Science is a business – nothing more, nothing less. It has the same level of corruption, politics, and general jibberish as any other sector of society. People get paid (rather well – in many cases, far too well) to do it. Accept my challenge and put your money where your mouth is. We’ll go through many examples one-by-one and see just how well science is performing. Up for it? or would you just prefer to sit in Bristol and offer pseudo-anonymous generalities with absolutely no supporting evidence? and an equally low level of credibility?
As for picking up the frauds and other deficiencies, funny how the blogosphere appears superior to – and well ahead of – the journal editors and reviewers in many cases? Perhaps not choosing the right reviewers? Keep making excuses …
Sierra, surely the evidence is that the scientific effort continues to produce outstanding advances. I don’t deny that rubbish gets funded and frauds occur (I said just that in my first post). I don’t really see what your fundamental problem is. You consider that “the peer review system is corrupt” and that there are “few other credible outlets for work”. I disagree, and surely the proof is in the pudding. If the peer review system is corrupt how come so much good research continues to be published? There’s bound to be some element of corruption in peer review just as there is in all aspects of life that otherwise function well enough(we know that’s the case – there are some particularly creepy examples in the antiscience elements that attempt to pervert understanding in climate science and evolutionary biology, and so on…if someone come up with better model then that will be great….the model provided by Nature Precedings seems not to be useful where it matters.
I do agree with you that the Internet is providing the major impetus to identify scientific fraud and getting funders and publishers to deal with this properly.
You have erroneously lumped together two issues into one: corruption and junk science. They often do not go hand-in-hand. There is nothing to prevent corrupt individuals from publishing good science themselves. Rather, these individuals just act to prevent others from publishing good science (must like some plants toxicify the soils around them so other plants cannot grow and compete – that is how science today is often structured). Thus, the presence of reasonably good science in many journals is in no way evidence for the absence of problematic and systemic corruption. Perhaps science would be proceeding much faster, and with far more benefits to society, if all the corrupt twits were publicly weeded out?
In general, I am willing to engage in a detailed assessment of the issues with you, using those novel concepts called real examples, but you need to accept the challenge. Funny how you’ve taken me on rather strongly in the past, but when the rubber hits the road, you fall back into propaganda. We could undoubtedly go through several illustrative examples in the time you’ve spent holding the party line today … maybe even some examples will touch on your mysterious home institution?
Have I “taken you on rather strongly in the past”?? I’m simply presenting my view of science and scientific publishing (lots of crap elements btw) and it seems not to accord fully with yours. You are right that science would proceed more productively if the corrupt twits were weeded out; I don’t have any disagreement with you there!
I don’t see the need to go through specific examples. I don’t doubt that you can raise some; I can too.
My point of disagreement is that the examples of corruption and fraud in science are not a problem of peer review but are a problem of corruption and fraud! So I also think you are erroneously lumping together two issues. Peer review isn’t corrupt. However there are some corrupt individuals that pervert their own contribution to the process and I have no problem with your highlighting of these.
In my opinion scientific research dissemination needs some screening for quality – imagine if all science publishing was based on a Nature Precedings dumping ground approach? Of course that’s a little unfair since the point of Nature Precedings is as a supplement to the standard means of scientific publishing, but some people consider this to be a broadly feasible approach where everything is published without much consideration, and left to sink or swim according to its usefulness. That would seem to require an almost saintly integrity and devotion to public service to be workable. The present system means that work has to be pretty good to be published (I know you don’t agree with this), and it does help to maintain standards. You only have to look at the huge outpourings of poor quality stuff in the vast welter of new journals to see what can happen when pretty much anything can be published. Not sure how that advances science at all.
If we can come up with a better method to peer review, then great. Perhaps losing the anonymity of peer review might help, to get round some of the problems you allude to, ‘though that has it’s own problems…
Sure, you’ve taken me on strongly in the past on this site, and I like it. So keep it up.
You are entirely incorrect when you state that “the examples of corruption and fraud in science are not a problem of peer review but are a problem of corruption and fraud.” That is absolute nonsense. Much of the fraud was readily uncovered in the blogosphere by simply looking at figures and tables and text critically. You know, what the reviewers should have done? And how can corruption not be a problem of peer review when much of the corruption arises during peer review?
As for the examples, of course you don’t want to go through them … and then maybe you shouldn’t be throwing around such jibberish as implying others have conspiracy theories.
“academic and govt scientists aren’t civil servants”? Nonsense. A perfect example of the non-accountability of science. If scientists want to be self-governing, they can find their own private funds to conduct their work. Self-governance never works – history (a subject that too few scientists know anything about) teaches us that. It leads to rampant corruption (i.e., the current situation in science). Of course you don’t agree, perhaps because you’re part of the problem? Maybe that why you refuse to use your real name?
Tell us, who do publicly funded scientists serve, if not the public? Themselves? That actually seems quite likely.
That’s a bit of a semantic game Sierra. There’s a large element of self-governance in publically-funded science. Part of the point is that scientists choose themselves what the important questions are and how to tackle these. Of course this has to overlap with certain more widely held ideas about what is potentially worth exploring; if your clever ideas aren’t of interest to public or industrial funders you don’t do the work unless you can fund it yourself (which does happen). In many cases publically-funded scientists are more directly serving issues of public concern (targetted funding to deal with foot and mouth outbreak, or bird flu, or ls specific public health or medical issues)…when the work is funded by industry then the scientist is directly serving industrial interests which hopefully broadly benefit the public more generally.
All of this is done under structures that ensure accountability.
Well, as I keep noting, there’s obviously a low level of accountability in the system. Let’s move through a large suite of examples one-by-one and see how they should be accounted for. As you should know, science is about the details (which you continually seem to want to avoid), not just feel-good propaganda. Let’s get into the details. Why waste your time writing generic propaganda, when you could show me up publicly by debunking my conspiracy theories?
For someone who apparently has such a massive problem with the peer-review publishing model, you place an astonishing emphasis on your peer-reviewed publication record… http://sierra-rayne.blogspot.ca/p/education.html
Astonishing emphasis? Apparently you haven’t seen the CVs of many other scientists?
Well, I’ve never seen a CV of anyone listing a “seventh most cited article”.
Oops, it’s actually “seventh most downloaded” not “cited”. My bad.
We’ve already got post-publication peer review – your peers read your paper & decide it’s good or crap. That’s always been around, but blogging etc. has made it much easier. It’s grown up (and still evolving) organically. I don’t think there’s any need, or much scope, for forcing it in a top-down way.
I disagree. Most blogs are crap, written by wannabes in need of attention. This blog is a rare exception.
One could make the same assessment of most journal articles.
Anyone wanting an inside view of the limitations of the peer review process should read The Hockey Stick Illusion by Andrew Montford. It’s about global warming, but a key component is the weakness of the peer review process. Comments ignored. No access to the underlying data. No attempt to even reperform the tests…
This is another story about the failure of the current system. 43 of 47 studies could not be replicated (and yet they passed peer review)
http://news.yahoo.com/cancer-science-many-discoveries-dont-hold-174216262.html
That Reuters story on cancer science is good.
Those of us with even basic critical thinking skills have long seen most cancer research for what it is – a money making machine for those involved. Too much money coming too fast leads to junk science, and that’s what this story shows.
Anything to do with climate science is a lesson in corruption. That field, and environmental science in general, is sickening from the purported “peer-review” perspective. It’s almost all politics and pushing political and economic agendas under the guise of real science.
Undoubtedly Chris and others on this page will cry horror at questioning climate science, but let’s face it, we have absolutely no real idea where our climate is headed (or to some degree, has been). Admitting what you don’t know is the first step to wisdom.
Medical science is equally problematic. It’s all money nowadays – nothing more, nothing less. And where there is big money, there are big temptations for unethical behavior and/or junk science, and that is exactly what we see in peer-reviewed literature.
How odd! From the point of view of an interested scientist, climate science has made extraordinary advances. It’s difficult to take your vague assertion “…no attempt to even reperform the tests…” seriously when reproduction of analyses is as much a part of climate science as any good science!
So the temperature record has been independently determined by GISS, NOAA, Hadcrut and BEST….. the high resolution Law Dome ice core was drilled three seperate times using three separate methods to ensure that the record wasn’t contaminated by artefacts resulting from drilling method….the essential features of the Mann/Bradley “hockey stick” analysis (late 20th century warming likely greatest in last millennium) has been reproduced around a dozen times by different groups….polar ice depeletion has been independently determined by direct observation, satellite altimetry and GRACE gravity measurements….sea levels and ocean heat accumulation has been independently determined by satellite altimetry and tide gauge measurements and ocean heat content determined by the ARGO system and the sea level data consistently assessed in terms of the separate mass and steric contributions.
These all-encompassing trashings: “anything to do with climate science is a lesson in corruption”…”peer review is corrupt”… are tedious and smack of “chips on shoulders” to me. The sad fact is there are some unpleasant antiscience things going on, and the disgraceful rubbish churned out by Montford is a typically sad example. If one is looking for examples of how a tiny element of corruption perverts the science one only needs to look at the pathetic attack on Dr. Mann by a serial plagiarist:
http://www.retractionwatch.com/2011/05/17/controversial-paper-critiquing-climate-change-science-set-to-be-retracted-because-of-plagiarism/
…two of who’s other papers containing large tracts of plagiarised material (in Wiley Interdisciplinary Reviews: Computational Statistics), have been allowed to be rewritten to cover up the plagiarism without any penalty whatsoever. That goes completely against the philosophy of ethical behaviour highlighted on RetractionWatch, but these characters get away with it, presumably because it’s politically expedient in some circles to ensure that the cheats aren’t subject to the same rules as everyone else..
It’s worth reemphasizing the relevance of this to the subject of this thread (the withdrawl of Nature Precedings). A tiny number of these antiscience characters attempt to sneak rubbish (like Wegman’s rubbish) into the pukka scientific literature in order to give the pretence of “scientific respectability” to dodgy ideas in service of creepy agendas. In a pervasive Nature Precedings style publishing philosophy where pretty much anything can be published with minimal gatekeeping, it becomes all too easy to give politically-motivated junk the appearance of equal validity as proper science done in good faith. An eye needs to be kept on that particular problem as publishing practices evolve…
Excellent, so now we’re getting to anonymous Chris’ real agenda – stifling any dissent. Thanks for making the case public. And there are many examples of the perversion of science in the climate science field. Let me guess, they are all wrong? By the way, what is your publication record like in climate science – can you refer us to any examples in the literature? We need a truly independent commission to re-examine absolutely everything (including all e-mails, raw data, etc.) in the climate science field. Let’s find the truth. And if the alarmists have nothing to hide, why not fully open up all the books and submit to a truly independent inquiry?
And I suspected there might be a political agenda behind the shutdown of Nature Precedings, which you appear to be strongly alluding to. Care to inform us further. Is “unsustainable” code for “against the party line”? Can’t be publishing anything that contradicts the party propaganda? Please elaborate.
Indeed an eye needs to be kept on these issues – as the perversion of science for political aims now potentially poses one of the greatest threats against freedom. About 90% of scientists are left-wing on the political spectrum. Hmm, wonder how that happened. Just random, I guess? And, of course, this wouldn’t lead to any political bias in resulting policy-based science, would it? Nope, nothing to see here, just move on everyone. Fall in line and blindly obey, eh?
Real scientists welcome continuous dissent and vigorous open debate based on transparent datasets. Pseudo-scientists spew the “do as we say” nonsense.
What a strange interpretation! I don’t think I suggested anywhere that dissent should be stifled. I’m suggesting that we should be savvy enough to recognise politically-motivated attacks on the science where these are obvious, and to focus on the science (rather than self-serving trash) if we want to understand, for example, climate science. Montford’s ignorant attacks on climate science are pretty astonishing; Wegman’s (less-ignorant) attacks are sufficiently blatant that his paper was retracted for plagiarism (rather astonishingly he was allowed to “correct” the plagiarism in two other papers).
The key to understanding science, Sierra, is the science. Increasing_skeptic may not like the results and implications of Dr. Mann’s work, but if the essential conclusions have been effectively confirmed around a dozen times since he published it, Rational interpretation suggets it’s likely to be correct, and at least not “fraudulent” as the Montfords and Wegmans try to insinuate. The problem with the “dissent” in this arena is that it is so disgracefully non-scientific.
I sort of get the impression that the science isn’t of much interest to you, and what you want is some kind of an “Inquisition” in which which the emails and records of everyon’s research you happen not to like is trawled through. And Sierra, do you really consider that the greenhouse effect, or solar radiation, or ocean circulation or the myriad of other physical parameters measured in the climate system has a “left wing bias”? Astonishing. I’m happy to address the science on its own terms thank you very much..
Who said anything or “alluded” anything about a “political agenda” behind the shutdown of Nature Precedings? Not me. I hardly knew about Nature Precedings before yesterday! The dumping ground approach to scientific publishing turns out not to be very useful in his instance so Nature decided not to continue with it. Don’t think you need to raise a conspiracy theory against what was presumably a pretty pragmatic decision!
One of the sad things about your post(s) is that you seem not to have much of a concept of objectivity and quality in relation to science. One can objectively assess the science in climatology if one is sufficiently informed and approaches it with a desire to find out what interpretations the evidence supports. One can generally assess the quality of a scientific manuscript according to the particular objectives of a journal to which it is sent. It’s silly to bitch about “political power” and “left wing bias” when you can actually focus down and address this stuff objectively. That’s one of the essential beauties of science!
Chris: One of the sad things about your posts is that you clearly have either not thought critically about the political nature of science and are naive regarding human nature, or you are aware of it and support it while trying to publicly cover it up. No wonder you don’t want to talk about specific examples, you’d rather keep up the generalities of propaganda instead.
Indeed, let us “objectively” go through the science. I look forward to seeing absolutely all the raw climate data, computer code, e-mails, etc., published in the open domain. I know of a number of individuals that would relish the opportunity to analyze it. If your real goal is objectivity, what can possibly be your objections to such complete transparency? And let’s apply this standard to absolutely every paper published in the so-called “peer-reviewed” literature.
Oh yeah, Chris, still waiting for your publication record in climate science. Otherwise, you just appear – once again – to be simply following the party line that someone told you. What specific climate studies have you engaged in that make your points worthy of attention?
If one actually takes the time to read The Hockey Stick Illusion, it is very difficult to see how the mathematical calculations of Mann are questionable. Graphs are printed side-by-side. Claims made about adjustments to data that were “insignificant” clearly were not. It’s unfortunate that Wegman fell into the same trap, but at least his work was retracted. Mann’s work is still out there and he’s still defending it. Obviously, there could be other studies that show the similar results in the time that has passed since the book came out. At the same time, McIntyre has pointed out important weaknesses in the data – like why in 2005 the proxy data was still from the 70s and 80s?
http://climateaudit.org/2005/02/20/bring-the-proxies-up-to-date/
Yet, despite all that, the critique of the peer review process holds whether you believe the story or not. The other article link about the cancer studies that I posted shows the dangers in the current system. In fact, in order to get access to some of the data, they had to agree not to publish whose studies turned out false. Where is the scientific ethic in that? Standards that allow these failures to hide behind non-disclosure benefits no one.
Think you meant “…are not questionable Increasing_skeptic. The problem I have with Montford is that he doesn’t tell the truth (he seems determined to pretend that the evidence that the Medieval Warm Period was cooler than now, is some sort of conspiracy!) and is fixated on demonising Mann’s early paleotemperature data without informing his readers that there is a whole slew of subsequent paleotemperature data that effectively validate the main conclusion of Mann et al 1998/1999 that current temperatures are anomalously warm in the context of the last 600 (Mann et al 1998) and 1000 years (Mann et al, 1999 [.g. Crowley 2000; Moberg 2005; Oerlmans 2005; Rutherford 2005; Hegerl 2006; Juckes 2007; Loehle 2007; Guirat 2010 just to list some of the papers I know of; most likely there are more].
Now one might actually wish to know the answer to the question “what does the scientific evidence say about current temperatures in the context of the last millenium?”, and as usual one can find this out by a perusal of the scientific literature. Montford pretends that that the way to address this rather interesting and important question is to fixate on a single paper from nearly 15 years ago and attempt to demonise the main author.
I only skimmed Montford’s unpleasant book and have to say I gave up quite quickly on it. One thing that jumped of the page in an early section (maybe 1st chapter if you have a copy at hand) is a fabrication on a subject I know something about. Montford asserts that paleotemperatures from borehole data by a Uni Michigan scientist (Shaopeng Huang) were submitted to a prestigious journal (Science/Nature??) and that his data showed that the MWP was warmer than current temperatures, and that (presumably due to the nasty corrupt per review system) the paper was never published. It doesn’t take very long to do a literature search, find that Huang did actually publish this data [Huang, S. P., H. N. Pollack, and P.-Y. Shen (1997), Late Quaternary temperature changes seen in world-wide continental heat flow measurements, Geophys. Res. Lett., 24(15), 1947–1950] and that it doesn’t have anything at all to say about current and MWP temperatures. In fact Huang states quite explicitly in his later compilation of a vast amount of borehole data that current temperatures are at least 0.5K warmer than during the MWP [S. P. Huang, H. N. Pollack, and P.-Y. Shen (2008); Late Quaternary climate reconstruction based on borehole heat flux data, borehole temperature data, and the instrumental record, Geophys. Res. Lett., VOL. 35, L13703].
It’s not particularly surprising that some ill-informed journalist chooses not to tell the whole story in pursuit of some self-serving narrative; that’s modern journalism for you. The sad thing is that much of the rubbish is very easy to spot. Montford, like all science misrepresenters is relying on our ignorance and willingness to engage in a bit of conspiracy theory! And note that pretty much all of this (palaeoreconstructions/borehole data) was sitting pretty in the scientific literature well before Montford’s book was published in 2010.
Perhaps if you had read all of Montford’s book, you would know that the problems don’t end with the original Mann paper. Out of that list of supposedly confirming studies, many of them used the same flawed data sets or suffered from other errors. All of this is chronicled in great detail. However, here’s one link that shows the overlapping data sets amongst some of the studies you cite. And yes, the data sets were particularly sensitive to just a few of them. Take them out and the hockey stick goes away.
http://climateaudit.org/2006/12/18/the-independent-2006-multiproxy-studies/
On the broader topic of why we have peer review or other systems that produce flawed results, we simply need to look for the incentives. In my career, I’ve confronted fraud, such as schemes to divert international aid to inappropriate uses. Fraud exists when there is Opportunity, Rationalization, and Pressure (standard fraud triangle). Even when organizations go to great lengths to create systems that prevent fraud, such as codes of conduct and codes of ethics, reporting hotlines, and separation of duties, fraud still occurs because collusion (two people working together) can always find way to cheat the system.
When I read about the limitations of the peer review system: no pay, limited time, potential inhibitions caused by not wanting to lose grant funding, closed networks of reviewers, etc. it seems like a system does not do a good job in preventing fraud. Opportunity is certainly there if the peer reviewers have no time or don’t have access to the underlying data. Therefore, while it may not be that there is an overt intent to commit fraud, we seem to have a system that gets us too close to the edge… and then they end up on Retraction Watch.
Of course, real science is quite hard, and the more I read, the more I wonder how much I can truly believe. Some of my heroes that seem to have something important to say are: Nasim Taleb, Denise Minger, Gary Taubes, Matt Ridley, Freeman Dyson, and Andrew Montford. The cover a wide range of topics, and in my view they provide an extremely valuable input to the system. They act to reduce the Opportunity that someone can pass off substandard work and have it turn into a law or regulation that affects millions or maybe even billions of people. So, I chose “increasing_skeptic” as my name. I often use my dog’s name (Rufus) when posting on the Internet as well. Rufus is just a black dog, but he is smart enough to see through many fallacies. In any case, my actual name is Kevin Stever.
One wonders if Chris is being fed information from another source? Chris writes above about the “broad Physical Chemistry-Biochemistry-Molecular Biophysics-Molecular Biology fileds [sic] that I have experience of,” and yet, here he/she writes extensively about some rather specific details of climate science.
Hmm … speaking of conspiracies. Perhaps he/she is, instead, just extremely widely read.
Well yes Sierra, I’m a biologist (in the broadest sense of the word) but am interested in climate science in a sort of hobby way. Hope that doesn’t make me appear a little nerdy!
It’s quite easy to keep up with climate science particularly as reading the scientific literature is part of my day job and it’s just as easy to do a Web of Science search on the latest borehole paleotemperature data (say) as it is to do one on p53 mutations and its effects on apoptosis. I do find the subject of misrepresentation of science particular fascinating in a car-crash sort of way, and so tend to notice when someone like Montford publishes dreary nonsense…
So, you might be Chris Paraskeva at the University of Bristol (http://www.bristol.ac.uk/cellmolmed/cancer/paraskeva.html)? He appears to be doing work in this field: http://cancerres.aacrjournals.org/content/60/1/22.short.
At any rate, we didn’t all fall off the bus yesterday … although your claims are most amusing.
O.K. fair enough Increasing_skeptic – this isn’t really the place to have a detailed discussion of Montford’s book and I’d have to borrow it again; something I have little desire to do!
I will make one point ‘though. You linked to a blog to illustrate your comments which seems problematic. The owner of that blog provided the data (original published in Geophys. Res. Lett. 32, L03710, doi:10.1029/2004GL021750, 2005) that was reproduced by Wegman in the report to the Barton committee. A main element was an attempt to show that the methods used by Mann et al, when applied to “red noise” produced “hockey-stick”-like signals. However what Wegman either neglected to show, or didn’t know, was that the “red noise” wasn’t red noise at all but was undetrended noise obtained from proxy data themselves and thus retained some of the trend in the proxy series. Having used a method that biased towards generating “hockey-sticks” from supposed “noise”, a program was used to generate 10,000 versions of “paleodata” from “noise”, the runs were ranked for degree of “hockeystickness” (using a hockey stick index or HSI), and the 100 most “hockeystick”-like were used to for preparing figures [in other words 99% of the data that more poorly supported the attempt to “construct” “hockey-sticks” from noise were thrown away].
In case my description of this dodgy procedure is interpreted as “trashing” I’ll point out that one can download the computer code from the AGU website (go to the paper cited in previous paragraph and download the code deposited under “Auxiliary Material”) and confirm for oneself. Much of this has been described in the scientific literature and that seems to me to be the appropriate place to go to understand the science rather than blogs. And since the subsequent paleoreconstructions essentially confirm the original interpretation that contemporary temperatures are anomalous in the context of the last millennium, I would conclude for now that that’s simply what the data shows. If someone comes up with convincing evidence otherwise, then fine.
Personally I think we should highlight dodgy practices in disseminating scientific research whatever their source, and not choose according to our particular agendas. Incidentally, I do agree with you that Matt Ridley’s “Genome” is very good…
SR-Criticizing peer review seems to be a hobby/specialty of yours. Surely, you must also be thinking of alternatives/solutions. Maybe it’s time to tone down the rhetoric and start letting us in on your proposals to revamp this corrupt system.
nothing more nothing less
MM: We can start with some basic changes to be implemented effectively at all journals, and then iteratively move our way up through the process as we see how the system responds/evolves.
No journal should have anonymous reviews. The names, affiliations, and actual reviews should be published as supporting information with each and every article (including the replies-to-reviewers by the authors, and any and all written editorial contact [i.e., letters, e-mails, etc.] between the editor(s) and either the authors or reviewers about or related to the article in question). Any non-written contact that will not be published with the article between the editor(s) and reviewers/authors regarding an article should be forbidden (i.e., made an ethical violation, leading to immediate editorial dismissal if uncovered).
All journals should have a mandatory 3-week peer-review maximum, and a mandatory 2-week peer-review maximum for any and all revisions. Articles that cannot be reviewed within this period will be automatically published by the journal as submitted (or including any revisions dictated by reviews that were completed in the required period). This will help cut down on the classic “rip-off” game whereby a paper in peer review gets sent around a group, and recipients attempt to steal the ideas via various routes. It also allows authors to rapidly screen through potential journals in their attempt to get their work published.
Over time, the best approach may be to move away from traditional journal formats entirely and the historic peer-review process altogether, and just have individuals publish their work independently on the web. This certainly would cut down on the wastefully high costs of publishing that are being paid for by taxpayers. Indeed, the taxpayers are already subsidizing the major publishers in a significant way.
MM: And the major difficulty with reforming the peer-review system is that the publishers are commercial entities with the rights and freedoms to engage in business as they see fit. They are legally entitled to publish what they want, and to not publish what they want. Ergo, any “basic rules” cannot realistically apply to the relations between authors from the private sector and the corresponding publishers. If the private sector is dumb enough to hire individuals with trumped-up publication records and substandard intellects – that’s their choice, and they can suffer the financial costs of their decisions alone.
The ground to tackle first is publishing relations between authors employed in the public sector and the publishers. These publishing relations and the corresponding publishing transparency can be readily regulated. Since journal publications form a core part of government hiring decisions, and since many government employees sit on the editorial boards of journals and/or act as reviewers, one can reasonably make some interesting arguments regarding the obligations of the editors, reviewers, and the journals themselves to be fully accountable and transparent for all transactions that they engage in.
The common law right to earn a living may also apply, and the corruption and/or incompetence at many journals may run afoul of this.
I agree with a lot of what you say, but as the biochemists say..”garbage in…garbage out”. We can vent about peer review until we are blue, but that doesn’t address the underlying problem that goes way beyond how papers are published.
I agree with Sierra Rayne – peer review system is not doing its justice at all. Why then papers of similar quality are published differently? Manuscripts from big guns are always going to be in high profile journals and manuscripts from unknown groups are always in low profile journals. Big guns keep on publishing in high impact journals consistently – sometimes every month….there is no equal treatment for quality…
Well there is one response to that – do great science and get yourself in to that elite group. It is not easy, but it can be done and happens all the time. It is simply not true to say that big groups publish “always” in big journals (they don’t) and it is also not true to state that smaller groups publish “always” in lower tier journals (they don’t). I admit that there is bias in the system, but I just don’t believe that journal peer review is where the major source of the bias is. Even if it was, open-access journals that publish everything following nothing more than a “sanity check” are clearly, clearly, clearly not the solution. If anything these journals are diluting quality science with work that is unimportant and of poor quality, and are widening the gap between the big guys and the little guys. One only has to look at the PLoS One website to get a sense of the utter nonsense that is published there. In my opinion, it is more damaging to ones career these days to publish something in these journals than it is to not publish anything at all. Remember if you have nothing interesting to say, don’t say it at all!!!
And who defines what is interesting? you? your friends? some editorial board? Sounds pretty much like an advocation for tyranny (for those that think this is an exaggeration, don’t laugh, many nutcase scientists in the environmental science movement have publicly advocated for restricting the freedom of speech for climate skeptics).
Ressci is correct. Often what determines the choice of journal is political power. Same goes for what is a goofy “hot topic” in science nowadays. Strangely enough, many (if not most) of the “hot topics” are the ones the most politically powerful groups in science are working on. No massive bias there, of course – like we were all born yesterday? The oldest immature game in science is to tell all your friends that the work of your competitors is not interesting, not significant, or just plain crap. Almost always this is done with no evidence. For those that make such statements and provide evidence, good on them. For the rest, show us your data, please!
Please give us some specific examples of articles in PLoS One that are “utter nonsense” and a brief explanation for each as to your rationalization.
Yup rayparlour that’s the key. Do good science, write good papers and you’ll get decent publications in good journals. It really is that simple. Of course it’s not easy, but it isn’t supposed to be easy. You can bitch about perceived injustices or you can knuckle down and do the work. Science does actually work; one of the delights is that there is a strong element of objectivity to the scientific enterprise.
In my experience, which differs apparently from at least one commentator on this thread, once a piece of work starts to near completion and you begin to put it together, you have a very good idea of where the paper will go and 75% of the time it gets accepted (usually after some addressing of reviewers comments). Occasionally you have to try your second choice journal. That’s never failed for me and for every colleague I’ve ever had the pleasure of working with…
..all this stuff about “political power” is silly. It’s easy to establish what’s interesting and important. If you know the field reasonably well, then what you find interesting is almost certainly what is objectively interesting and worth pursuing. After all the point of doing science is to find stuff out and to solve problems. If you find something out in a relevant field or solve a problem then it’s interesting pretty much by definition. And then you publish it…
As “Increasing_skeptic” did, I invite you to read the article of Drs. Glenn Begley and Lee M. Ellis published in Nature (http://www.nature.com/nature/journal/v483/n7391/full/483531a.html) and another one published by Reuters (http://www.reuters.com/article/2012/03/28/us-science-cancer-idUSBRE82R12P20120328) on the unreliability of 47/53 basic studies on cancer that have been previously published in recognized and high ranking journals, and think about the impact this might have on patients suffering from cancer.
Here, the problem seems to be located at several levels, but reviewers who follow the unrealistic demands of editors…(they publish only positive, new and perfect results that will make a scoop) are in some way also responsible of having bad (or junk) results published….
The claim that “we have absolutely no real idea where our climate is headed” is as mind-boggling as the statement that cancer research is merely “a money making machine for those involved”.
Fifty years ago when I was a child women routinely died of breast cancer. Today they don’t, if they have access to medical care. They get treated and they go on living for decades.
I also notice that for all the talk about top journals being full of junk, not a single concrete example was presented. Not that I don’t think there aren’t any, but it’s interesting how the broadsides oddly avoid specifics.
Silvio Levy
Excellent, a call for examples (I challenged Chris, but he/she refused to accept). Here are some to start with in some top-ranked environmental science journals:
– http://sierra-rayne.blogspot.ca/2011/05/comment-on-assessing-exposure-to.html
– http://sierra-rayne.blogspot.ca/2011/05/comment-on-prediction-of-soil-sorption.html
– http://sierra-rayne.blogspot.ca/2011/05/comment-on-prediction-of-soil-sorption_06.html
– http://sierra-rayne.blogspot.ca/2011/04/comment-on-polyparameter-linear-free.html
– http://sierra-rayne.blogspot.ca/2011/04/comment-on-development-and-exploration.html
– http://sierra-rayne.blogspot.ca/2011/04/comment-on-screening-chemicals-for.html
– http://sierra-rayne.blogspot.ca/2011/04/comment-on-food-web-specific.html
-http://sierra-rayne.blogspot.ca/2011/06/comment-on-visualising-equilibrium.html
Please give me your analysis. We can discuss the details of the one in the journal Science as you desire, as the fundamental issues with this article are related to some of the others posted above.
But Silvio, at what rate must examples be presented to actually mean something? I mean, we are scientists. We can all point to some paper in our field that was poorly done, or the data poorly interpreted. This is not a random sampling though! And looking at examples of where things have gone wrong, what does this tell us about the nature of peer review? I would contend that it tells us it is imperfect… but not HOW imperfect, i.e. by how much it is flawed. And isn’t that the question we want to address??
@ SR: I have one example to show you:
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0032988
Took me all of 20 seconds to find it.
Not being a skin pigmentation expert, what are the specific problems with it?
What is right with it?
I have no idea. It’s not in my area of expertise. Apparently it’s within yours, so please let us know specifically what the problems are.
A theory is something nobody believes, except the person who made it. An experiment is something everybody believes, except the person who made it.
Albert Einstein
1) Science. 2011 Jun 3;332(6034):1163-6. One example – later on this has ignited controversy.
Science 30 March 2012: 1638-1643 (just last friday’s paper) – very interesting paper. But time will tell how important this paper would be. Please browse through the earlier blog (comments as well) on Das for these authors contribution on resveratrol…http://www.retractionwatch.com/2012/01/12/so-how-peripheral-was-dipak-das-resveratrol-work-really/
At the risk of breaking rule 14, it is important to discuss some of the proposals Sierra has regarding peer review…
First, removing anonymity might sound good to you, but as someone who reviews >100 papers a year, this is not something I would welcome. There are probably 1000+ scientists whose papers I have rejected by now. For anyone who reviews in such large quantities, the political ramifications of having one’s identity revealed are horrendous… just one person who doesn’t agree with you and sits on an NIH study section, and goodbye grant funding! So grants and papers are linked – big surprise, you mess with one and the other will be affected. Excuse me if I don’t appear keen to jeopardize my grant funding.
Second, private editorial communications are essential. What if I have a perceived conflict of interest (let’s say the lead author on a paper recently came to give a seminar at my institution, so I know something that is not in the paper)? The editor should be aware of such things, in order to gauge the importance of my review vs. the others. Clearly disclosing it to the author is inappropriate because then they’d know who the reviewer is. What if the paper I’ve been sent has already been rejected from another journal (maybe even by me)? Again, the editor needs to know this, but the author does not have any right to know.
Third, I agree that timing should be improved. What you many do not realize is that a large part of the delay is not reviewers themselves, but assigning them. It can often take 2 weeks just to find reviewers for a paper. One reason is that reviewers suggested by authors are often inappropriate. Generally, only one of your suggested reviewers will ever be consulted, and the other reviewers will be pulled from the journal’s pool. At busy times of year, these folks may often have 10+ manuscripts backed up, so will turn down any new reviews, thus extending the search for reviewers. Then, 2 reviewers will have their reviews in on time, but the last one holds everything up. If journals start getting strict about deadlines, more reviewers will simply say no.
One way in which peer review can be improved (which has been discussed here numerous times), is to implement standard tools… DejaVu and other software to detect plagiarism and data manipulation, as well as demanding original data and images for certain types of experiment (e.g., original western blots). This needs more money.
Another way in which peer review can be improved, is to have each paper reviewed by more reviewers, so as to get a more balanced opinion and avoid the possibility for a single reviewer to hijack the process. This needs more time.
Unfortunately, the entire peer review system relies on FREE LABOR. Either you want fast turnaround, or more reviewers per paper, or improved fraud detection. You can’t have all 3 in an unpaid system. If the journals devote just a tiny fraction of their double-digit profit margins toward such efforts (say reimbursing reviewers $50 per manuscript, things would be imrpoved. Maybe even link the reivew fee to turnaround time (late reviews go unpaid), and maybe have it refundable if the paper gets retracted in future.
The other interesting discussion this brings up is the interdependency of the various systems that are in place here… Publishing / Funding / Job Security. Publshing is mostly private, Funding is mostly public, and Job Security is dependent on both. As with all complex systems, it is incredibly difficult to make wholesale changes in one part of the system without affecting the other branches.
Your arguments against anonymity are not sufficient. The large number of biased and/or indefensible reviews being generated nowadays would be largely curtailed if individuals had to publicly place their names next to what they have written. This is another good example of how science is likely one of the least democratic and transparent of all professions. And thanks for acknowledging that the granting system is a biased joke. If individuals on such committees can muster the power to cut off your research funding just because you gave them a harsh (but fair) review (and they knew it), that is Exhibit A as to the fundamental corruption of science. Science is socially structured in a throwback to the Middle Ages. 21st century instrumentation with 12th century psychologies. Time to modernize.
Statements such as these by you are most illustrative of the problem: “What if the paper I’ve been sent has already been rejected from another journal (maybe even by me)? Again, the editor needs to know this, but the author does not have any right to know.” And why – specifically – does the editor of the 2nd journal need to know this information? and why does the author not need to know that you have likely transgressed reviewer ethical guidelines by providing this detailed information? I keep all my reviews anonymous and confidential – indeed, I thought I was supposed to?
Indeed, you just proved my concerns. The purported anonymity, independence, and objectivity of the peer-review system is a farce. There are all kinds of unethical background communications going on about various papers (of course, lending credence to the Climategate 1 and 2 types of concerns). Apparently, all journals in a discipline are linked via background communications, and thus, if one journal goes corrupt, essentially it can bring down most of a discipline with it.
One can see how the public’s declining trust in science is fully justified.
I agree wholeheartedly with vhedwig. I review a ton of papers. Some journals no longer use anonymous peer review, and they do publish reviewer comments. When my name is attached, I must admit that how I review changes. I still say the things that need to be said, and thus I believe I am still acting with integrity. But it is different.
I can see that Sierra does not like anonymity. There is even a hint of bullying in his interactions with chris above, trying to ‘unmask’ him/her. But interactions like those on this forum, and in the peer review system, are (in my opinion) more honest when they can occur without fear of repercussions. What good does it do for you to know my name? Does it change the validity of my opinion? Must chris be an expert in climate science to comment on it? (If so, should we all attach our cv’s when commenting on a Western blot, so other commenters can count the number of blots we have published and then ‘rank’ our expertise in this area??)
I can’t help but get the impression that Sierra wants to tell us about some horrible incidence of peer review ‘gone wrong’ in his past. If so, just tell us! Provide links to your paper that was rejected from a top journal, and let us assess whether it was wrongly dismissed. There are certainly cases of this in the literature – amazing work that was published in sub-par journals, and I am sure those authors were unhappy about it.
I most certainly do not support any notion of Chris trashing other individuals in public while refusing to provide his identity. To say this type of behavior is unethical is an understatement (of course, it is very telling that many other scientists such as yourself find no problems with it). If you want anonymity, do not make negative public statements against other individuals. If you want to make such statements, use your real identity. Perhaps Retraction Watch should require all individuals to use their real names in order to make comments? Seems consistent with their calls for greater transparency among science.
Your belief that statements “are (in my opinion) more honest when they can occur without fear of repercussions” is entirely unsupported by a body of knowledge in western democracies. Indeed, we have a substantial body of law founded on quite the opposite.
Also very amusing how scientists call for greater transparency and accountability in governments and industry, but resist such requirements themselves. Hypocrisy, thy name is thy scientific establishment.
And LNV, I’ll provide you with some examples if you provide us with your real name. Why not rise to the challenge of accountability? rather than cowardice …
I’ve read through your blog. Your reference list, full of “comment on” papers, is telling enough. I don’t need your examples, and since we work in different fields, they wouldn’t be terribly enlightening. Watching your banter with chris, however, has shown me plenty. As a scientist, I would have to insist that cherry picking examples of where ‘science has done you wrong’ is, of course, not scientific. Have a good day.
The problem presented for eliminating anonymous review results from current model of the reviewer/submitter relationship as being an adversarial one. I agree with SR, but go further. Reviewers should not only be named, they should be included as authors on the paper. In many cases they probably spend more time with the paper than many actual authors. Also, this might turn the relationship from adversarial to one of joint work with the ultimate goal of publishing the best science possible.
Is the reviewer/submitter relationship an adversarial one? Of course it depends on what one means by “adversarial”, but I wouldn’t consider it adversarial! In my experience the reviewer/submitter relationship is more about constructive critique.
A good way of getting a feel for the reviewer/submitter relationship is to look at the reviews in a journal that posts reviews online. Atmospheric Chemistry and Physics is an example. Submitted papers are treated as “Discussion Papers” that are posted for anyone to submit reviews which themselves are posted. Note that the reviews can be anonymous which seems a vital component to me. I would say these give a good idea of the nature of paper reviewing as constructive critique. Occasionally one gets a dodgy review, but you can rebut that as an author, and the editor is supposed to make an informed decision based on a few reviews…
The idea that reviewers should be included as authors is totally unacceptable to me. Authors plan, raise funds for, and perform the work and do the troubleshooting; they assess, compile, prepare and present the data, define the context of the work and write the paper. The reviewer makes some assessment of the quality of the manuscript in the context of some particular journal; and makes suggestions for improvement. The idea that some arbitrary group of individuals become coauthors on my paper is not a happy one!
e.g. I’ve had single line reviews of the sort “I can’t see anything at all wrong with this paper” and “this paper shouldn’t be publised in xxxx”, in each case for papers that were published in the journal to which they were submitted. Do these people become coauthors on my paper?
It’s quite a difficult discussion to follow here. The anonymity of the reviewers is always a big discussion. I agree that there shouldn’t be any anonymity but there are serious problems there as well. Once the anonymity is lost, a less establish scientist will have difficulty being critical of a paper from an established lab with the fear of facing these people in grant applications or tenure reviews. Especially if a paper from a group leader serving also in funding bodies is received. If the anonymity of the reviewers is abolished then all the authors in any reviewed paper should be exempted from reviewing any of the work produced by the reviewer. Soon of course, no one can review anyone else’s work.
I think another main problem in peer reviewing lies with the editors. Editors should distinguish between a rightful review and unnecessarily critical reviews. For that to be done, reviewers should explain in detail what experiments they think need to be done and they should justify them. I haven’t yet had the chance to see many peer reviews but the ones I saw were either asking things to be done without stating why or saying it’s not good enough again without giving any reasons. Editors should ask the reviewers to clearly state the reasoning behind their critics. Unfortunately, many journal editors have very few publishing experience themselves.
One more point is the fact that scientist are a lot more critical to someone else’s work than theirs. I’ve seen people asking things in peer reviews that they themselves don’t do in their own research. It happens frequently that a small group of scientists that are pioneers in a certain field end up blocking publications from others to keep the field to themselves. This at the end kills that field, driving young researchers away.
Anyway, I shall stay anonymous as I don’t want to upset any possible future reviewer whom I can face in a paper publication or fellowship application. This is exactly why F1000 will never work either.