Late last year, we published an invited commentary in Nature calling for science to more formally embrace post-publication peer review, and stop fetishizing the published paper. One of the models we cited was Faculty of 1000 (F1000), “in which experts flag important papers in their field.”
So it’s not surprising that F1000 is announcing today that they’re launching a new journal, F1000 Research,
intended to address three major issues afflicting scientific publishing today: timely dissemination of research, peer review and sharing of data.
The journal will publish all submissions immediately, “beyond an initial sanity check:”
It no longer makes sense to wait months or years to read, comment, or build upon another lab’s work, and similarly to hold back your own data and insights until the archival version is released, without the benefit of wider peer feedback. All work at pre-review stage will be very clearly indicated as such.
I asked F1000’s Rebecca Lawrence what comprised a “sanity check:”
Sanity check is really just a check that it isn’t complete rubbish – it is a basic in-house editorial check, similar to what is done by most publishers (e.g. like PLoS One) before anything would be sent for referee.
Lawrence said that the new journal’s approach meshed with the ideas we — and others — have proposed for post-publication peer review:
What we are planning to do fits well with that general idea – i.e. inclusion of all comments, referee reports, author responses, corrections, updates etc, as well as trying to pull together metrics and relevant discussion around the paper that is hosted elsewhere e.g. blogs, tweets etc. The idea is that the paper never finishes refereeing – at the moment we have a largely arbitrary cut-off point when the paper is published but by making all the refereeing post-publication means that anyone can comment on it at any point in the future – and in fact the authors can correct/update it at any point in the future. What we will need to work out is what constitutes a minor update versus a linked but new manuscript.
The journal will work much as arXiv does for physics, in which papers are published as pre-publication drafts and usually published elsewhere after peer review. One question, however, is whether publishing a dataset and protocol on F1000 Research would count as a violation of the Ingelfinger Rule, in which journals say they won’t publish anything after it’s already run in the media or other journals. Lawrence has been looking into that, as she did with another project, F1000 Posters:
Everyone confirmed to me that it would not be considered prior publication (including the likes of NEJM, Science, J Clin Invest, Lancet, BMJ, PNAS, Nature-titled journals, Elsevier journals, BMC, PLoS etc and even the RSC journals) except for Cell Press and Ann Oncol, and possibly a couple more who are veering in that direction at the moment. A full list of responses will be on the f1000research.com site later today.
Naturally, at Retraction Watch we were interested in what the journal’s approach might mean for retractions:
In terms of retractions, it is an interesting question. The ability to continue to update your article following publication will help with some of the issues but if articles are plagiarised or turn out to be a fabrication then we still need to be able to retract in such circumstances. Of course the advantage we have is that all such retractions etc will be very obvious at source rather than needing a later publication that is unlinked to the original article.
We’ll certainly keep an eye on F1000 Research and see what develops.
Update, 3 p.m. Eastern, 1/30/12: See comments for a number of questions for F1000, particularly about how the journal will differ from Nature Precedings, along with responses from Rebecca Lawrence. Also, Nature‘s Richard van Noorden explains how arXiv deals with retractions.
Isn’t this just for biology and medicine? So why the headline “An arXiv for all of science”?
And what about this statement on the press release regarding future issues that need to be thought out?: “What author fees are appropriate for the different types of content?” Sounds like the other ‘open-access’ forums where if you pay your fee, your content appears online. Cheaper just to stick your finding on your webpage/blog, and they get good search coverage as well.
Re: headline, yes, it’s for biology and medicine, but there’s already an arXiv for physics.
arXiv is flawed. It has the silly requirement that in order to validate your account for submissions, you need existing members to ‘vouch’ for you. Nonsense that is, and the system can then exclude people based on the sole grounds they cannot get someone to ‘vouch’ for them. It is not a good model to follow. At least it is free, and appears to be staying that way (unlike perhaps where F1000 is going).
Nature Precedings will take submissions from a broad range of disciplines, and does so for free, and allows comments and linking to a journal-published future version. It is the best model to date, and accepts papers from the same disciplines (and more) as this F1000 thing.
I’ll also add that the name (Faculty of 1000) seems wannabe ultra-elitist. What if you are not a faculty member? Are you inferior? Why only 1000? why not Faculty of 1,000,000,000, or Faculty of 15,132? Why not come up with a more inclusive (and non-quantitative) name – like Community of Researchers?
I agree — renaming to Faculty Of 7,000,000,000 seems like a good plan for this journal.
BTW., I am very interested by this development, and I’ll be watching it closely. I’ll talk to my colleagues about submitting here.
So are biology, medicine, and physics the only branches of science now? As a chemist I’m kinda offended by that!
Certainly no offense intended, nor did we suggest that “biology, medicine, and physics [are] the only branches of science now.” The point of the headline was to ask whether the arXiv model would extend to all of science, now that F1000 seems to be going beyond physics.
Not sure what you mean by “The point of the headline was to ask whether the arXiv model would extend to all of science, now that F1000 seems to be going beyond physics.” I don’t think F1000 was ever in physics. Weren’t they always in just biology and medicine?
And arXiv does much beyond just physics (http://arxiv.org/). It handles physics, mathematics, nonlinear sciences, computer science, quantitative biology, quantitative finance, and statistics. Thus, in concert with Nature Precedings (which takes biology, medicine, chemistry, and the earth and environmental sciences), these two repositories pretty much cover most of science.
Ergo, it’s not clear what F1000 is proposing that is new.
I understand your point about F1000, and will ask them how they plan to differentiate themselves from Nature Precedings.
But I’m unclear on how you thought saying extending “the arXiv model” beyond physics (and whatever else it already covers) says F1000 was ever in physics. It wasn’t, and we never said it was. The arXiv model exists, and F1000 is taking it beyond what arXiv already covers, into biology and medicine.
No worries. I think we have an interpretative difference. You said that “now that F1000 seems to be going beyond physics”. I (probably incorrectly) interpreted this to mean you were suggesting F1000 was ever in physics, which they were not. As well, it’s important to note that arXiv is much more than just physics. Not important, we’re getting to the truth on this one.
At present, Nature Precedings and arXiv appear to have similar models (except arXiv has the dumb ‘need to be nominated to contribute’ requirement, and is thus not a truly open model), and collectively they appear to cover all of science, allow commenting, and allow linking to a future published version (if this is pursued). Thus, I have no idea what F1000 is adding that is new.
I’m wondering of submissions will be given a DOI or other identifier & how the revision history will be reflected in the metadata for citation purposes.
I would be interested to know (and should obviously just go and look it up) precisely what arXiv, Nature Precedings and F1000 mean by “open access” — in particular, which of them actually confirm to the Budapest definition of that term, and do not encumber publications with “non-commercial” clauses and similar.
I can certainly speak for F1000 Research and say that our default licence will be the most open of the Creative Commons licences (i.e. without the non-commercial clause).
Nature Precedings: http://precedings.nature.com/site/help#copyright
Under the Creative Commons license.
Same with arXiv: http://arxiv.org/help/license
Many thanks to you both. Excellent to know that F7x10^9 will use CC-BY, and following the Nature Precedings link shows that of the various CC licences, they too have gone with CC-BY. This is a big win, as it greatly simplifies the issues surrounding text-mining and other automated uses of posted material.
To my surprise, arXiv is the least good of the three. (My prejudice had led me to imagine that that title would go to Nature‘s offering!) Material posted on arXiv can be CC-BY (which is best), CC-NC-SA (which I have a lot of sympathy with, even though I think it’s probably not the best choice), or just “grant arXiv.org a non-exclusive and irrevocable license to distribute the article”. This unfortunately doesn’t seem to say anything about what uses are covered.
Good luck with that! Only outsiders can possibly think that anything of this sort might work! Anyone who is in academia knows too well that there is no time for commenting on a flood of mostly crappy articles… What you will see is empty commentaries sections. What do people gain from commenting? Absolutely zero. Therefore, they will not spend time on it.
I think that arXiv actually does well in this respect. There’s a reason that it’s been around for 10+ years.
On the other hand, it primarily serves (theoretical) physics, mathematics, and computer science. The culture of academic publishing in those fields is a bit different than in chemistry/biology/medicine, and the quantity of publications is a bit lighter than, say, PLoS One.
What about a sanity check for comments? You know – trolls?
Joachim,a good question; if we find that comments start to get out of hand, then we will need to look at whether something is required here beyond the normal approaches taken where others can report inappropriate comments if they see them.
Thank you for all your comments.
@Sierra Rayne: F1000 was actually started 10 years ago and is primairly a post-publication service which started with an Faculty of 1000 leading experts across biology (hence it’s original name), but has in fact since grown to 10,000 experts with the addition of medicine as well. The main evaluation service is an expert filtering tool where the Faculty highlight those papers they think are particularly important, irrespective of where they are published – and a high percentage of these key papers are not published in what are typically thought of as the ‘top journals’.
F1000 Research is a separate publishing initiative. It will use as many referees as is required to ensure that published articles are suitably refereed – there is no number limit. It is fully Open Access, i.e. all content will remain free to view but at the same time, we will ensure proper refereeing and other tools to enhance the data and associated metadata to make it mine-able and re-useable. We also have a related initiative, F1000 Posters (http://f1000.com/posters) which is just like Nature Precedings in that it is free to submit, and free to view posters and slides from a very wide range of meetings. F1000 Research will build off the back of that.
@Lorenz White: Like current pre-publication refereeing approaches, we will be formally inviting referees to review each manuscript so there is no reason why the level of refereeing in that respect should be any different from a more ‘standard’ article. Open commenting from others outside of the referees is optional and, as you say, may be variable in response between articles. There is also increasing evidence that papers are often discussed away from the publisher’s site e.g. blogs, tweets etc and so we plan to bring these conversations onto the site as well so you can view all the ongoing discussions about a paper.
@mrgunn: Yes we will be using DOIs; one thing we will need to identify is which is the best DOI system to use for the versioning of the article and what level of updates require a new DOI.
@Mike Taylor: Great, thank you – we look forward to your submissions!
Ivan, regarding how to cope with retractions and corrections: I’ve put a brief explanation of how ArXiv currently does this on my post about the F1000 project on the Nature news blog (http://blogs.nature.com/news/2012/01/f1000-launches-fast-open-science-publishing-for-biology-and-medicine.html). But if F1000 Research has its own staff (to invite post-pub refereeing) then maybe it’ll be more formal than ArXiv in this regard too.
I really wonder how this will work for biology and medicine…
arXiv works best for theoretical papers that primarily involve derivation. Once published on arXiv, if the derivation is correct, you really can’t be scooped. I’ve heard of more than one computational paper that was scooped by competitors after being published on arXiv.
I can already see it: naive junior faculty member posts something innovative on F1000, a-hole academy member reads it, puts two postdocs on it, “contributes” it to PNAS before the junior faculty member finishes responding to the first round of review.
Hell, it already happens with a bio preprint archive.
That should be “it already happens without a bio preprint archive”
This sounds like a strong argument why we need things like F1000 and Nature Precedings. You can’t be scooped if the world knows that you were there first.
That’s true in theory, but I’m pretty sure that it doesn’t work in practice.
In experimental work, large labs that have more resources can build a more complete story in short time. So, the academy member, with his posh NIH/HHMI/etc funding, can always claim “well, that junior guy showed a hint, but we really did the work to demonstrate it.”
As Sierra Rayne mentioned, even if the academy member is ultimately forced to cite the junior’s paper, this seriously reduces the impact of the junior member’s publication. Consequently, the junior member has a harder time getting grants. This is very detrimental to tenure, or, at soft-money places, continued employment. Why would a lab risk it?
sfs: I agree with you, but are we not between a rock and a hard place? The newby could send his paper for review at a good journal, have the manuscript get passed around the club (inevitably, many submitted manuscripts get seen by a far wider audience than just the editors, editorial boards, and the reviewers – the ‘fwd’ button in many editors and reviewers’ email accounts is used far too often), have one senior a-hole among this broad audience throw the two goonlike postdocs on the problem immediately, and then a-hole submits the work to PNAS before newby even gets the reviews back? Newby is screwed in all directions, it would seem.
Large labs that have more resources can build a more complete story in short time. So, the academy member, with his posh NIH/HHMI/etc funding, can always claim “well, that junior guy showed a hint, but we really did the work to demonstrate it.”
They can tell whatever story they like. The bottom line is that John Q. Newbie’s paper will be right there on arXiv/Precedings/F1000, datestamped in advance of Jack R. Plagiarist’s PNAS paper. The world can see. as usual with science, more openness gives better results!
@Sierra Rayne:
I don’t know. You raise good points. I’m deeply pessimistic that this will fix the problem.
@Mike Taylor:
The question is whether anyone will care about the work posted on F1000. After all, it’s not a “real journal,” and the work can be posted prior to undergoing peer review. I’m inclined to think that grant/paper reviewers will view this more like a conference poster than a prior publication.
Like I said, I’m deeply pessimistic about this. For what it’s worth, I hope that you’re right and that I’m wrong.
sfs asks, reasonably enough: will publications on F1000 “count”? Will grant committees care about them?
I suppose we’ll just have to wait and see. Can anyone with a background in maths/physics/etc. tell us what the situation is with regard to arXiv? Do physics tenure committees care about work published there?
With regard to the discussion we’re having (sfs, myself, and Mike) regarding whether to submit to an online preprint archive first to get the date stamp/public priority date and thereby risk ‘scooping’ via this route, or to chance it, keep your work pseudo-confidential, and submit directly to a traditional journal (and risk ‘scooping’ during an overly lengthy/corrupt review process), the one advantage of going the online preprint route (provided the archive is rapidly covered by Google Scholar or other indexing services) is that, if you get scooped, you will always be able to point a finger at the scoopers and say that they either did know of your work, or they were incompetent by not knowing of it. Granted, you may not be able to undo the scooping, but if you and your competitor attend the same conferences, read the same blogs, etc., you can continuously and justifiably tell the rest of your field just what the scoopers did. If everyone in your field ignores you, then you know your field of study is generally corrupt, and you were going to get screwed no matter what approach you took. Alternatively, maybe your field is generally full of decent folks, and they will listen (eventually) to your points, recognize the scoopers for the trash they are, and collectively the community marginalizes/weeds out the scoopers.
On the flip side, if you roll your dice with a traditional journal, you can be strung along for several months and then finally rejected, meanwhile your competitors scoop you, and you are left with no public priority date evidence (you can show everyone your submission and rejection e-mails from the journal as ‘informal’ priority dates, but then you have to prove the other group knew of your purportedly confidential journal submission).
Mike and sfs raise the point about whether or not grant/promotion/hiring committees care about submissions to F1000, Precedings, and/or arXiv. From my understanding, the history of publishing/hiring/promotions/grants in math and physics is much different than in other science disciplines. Math/physics have apparently long been disposed to giving credit for non-traditionally ‘peer-reviewed’ work. Perhaps in these disciplines, there is a culture of actually staying on top of the preprint literature, and the community reaches a consensus as to whether the preprint work is of quality (i.e., a different form of ‘peer review’ compared to a traditional journal approach with an editor, editorial board, and 2-3 reviewers for each work to pass over the hurdle).
Chemistry and many other disciplines are certainly not that way. No credit is generally given for non-journal work, so you are screwed either way if the establishment doesn’t want you in the club. Namely, the establishment doesn’t recognize the non-establishment publishing venues, and the establishment won’t let you publish in the establishment journals if they don’t want you to. Difficult to do an ‘end run’ around this type of nonsense. All these other disciplines should move towards following the better parts of the math/physics model.
In some sectors of applied science, many faculty have made a good career based on conference proceedings and seminar presentations alone (often because industry consulting experience counts in this sector of the academic system, and thus, technical reports and presentations thereon are sometimes accorded equal status, or even superior status, to a journal article). A number of engineering faculty have CVs that are very light on journal articles, and very long on presentations/reports. In Chemistry, for example, you generally find little credit awarded to conference proceedings/seminars/reports.
As a mathematical physicist, I just wanted to clarify the status of the arXiv in the mathematics and (primarily theoretical) physics communities. The arXiv system is a great way of establishing ironclad priority for new ideas. It also provides a convenient single location for all new papers in a given field; you can receive a daily mailing of abstracts to look over each evening and know that you aren’t missing any important new results.
However, arXiv pre-prints do not count for anything when it comes to tenure at an academic institution; that requires that the pre-prints subsequently be published in conventional refereed journals. In the early days of the arXiv system, some journals were hesitant to publish things that had previously been posted online, but the use of pre-prints became so ubiquitous that all theoretical physics journals had to relent. Some famous people don’t bother with journals at all and only put things on the arXiv, but that’s not an option for most people in the community.
Sounds like all the disciplines are in the same boat.
One concern with respect to the F1000 model, as I understand it, is how someone reviewing another person’s CV will know whether the F1000 paper is truly peer-reviewed and ‘viewed positively’ by the reviewers? With the assumption that hiring and promotions committees generally accord most weight to journal publications, and that (particularly for hiring committees viewing hundreds, potentially thousands, of applications) time is limited, how will someone know the status of a F1000 paper?
At least with Precedings and arXiv, everyone knows it is not peer-reviewed. With F1000, though, it sounds like all papers will have reviewers invited, but they’ll be published regardless of whether the reviews are completed and/or whether the reviews are favorable. Thus, someone could list a F1000 paper in the peer-reviewed journal article section of their CV even though the reviewers did not reply and/or the reviews were unfavorable, and in order for someone else to determine the truth, they would have to go to the F1000 site, look up the article(s), and then try to make an assessment regarding any reviews present and what they might mean. This seems onerous.
Perhaps it will become de rigeur when citing an F1000 publication to state its review score (or some other metric that expresses what kind of review it’s been through and with what success).
Unfortunately, if the reviewers don’t respond, you are stuck with a ‘no review’ designation. You may still be able to then submit to a traditional journal in an attempt to get a ‘peer-reviewed’ designation for the work on your CV, but (1) that’s an additional hassle, and (2) some journals will not consider the work if it has already appeared online somewhere else (thereby limiting your options below what you would have had if you had submitted straightaway to a traditional journal).
I’m still think that arXiv is in a very different position than F1000. Almost all of math, and a large fraction of both physics and CS are purely theoretical (that is, derivation-based as opposed to experimental or computational). This is a significant contrast with biology/medicine. In terms of demonstrating “ownership” of work, I’ve always thought that derivation is a little more black and white than experimental or even computational work.
That said, I haven’t browsed arXiv in a while, so I don’t know how much experimental work is posted, and whether this differs between large collaborations (like the accelerators) and small groups (e.g., most condensed matter groups) use it or not.
@Brett (or anyone else that’s knowledgeable):
Do you know how frequently experimental work, from both large collaborations and single groups, is posted on arXiv? What about work that is primarily computational/simulation?
@sfs: Virtually all experimental results in high-energy particle physics are posted on the arXiv, but that’s an area where establishing priority is rarely a concern; only one or two detectors in the world are generally going to be able to perform a particular cutting edge measurement. Most papers in astrophysics and astronomy also are posted online, although not all. Participation from other branches of experimental physics (nuclear, condensed matter, etc.) is much lower.
I agree this is a concern. The counter to this concern is that if you can find it online using any one of the major literature databases (including Google Scholar, which picks up Nature Precedings, arXiv, and presumably F1000, on the day after publication), and it is on-point, then you should be forced to cite it or be held to account for plagiarism. Ergo, if said a-hole throws it in PNAS after scooping the idea from F1000, then the naive newby only needs to write to the journal editor showing proof of priority date on his paper, and the PNAS editor should force the a-holes to retract or correct (by correct, I mean cite the prior work appropriately) – either way, put a mechanism in place so that priority in the science literature is given to the naive young buck who acted honorably. Granted, the newby is still at least partially screwed because clearly PNAS won’t now take his paper, nor likely will any other leading journal, as the novelty is all messed up. But such likely public embarrassments for the a-holes may serve as a deterrent. The a-holes may get one PNAS paper out of it, and the naive newby may get screwed once, but after that, there will be a permanent record in PNAS showing the world where the a-holes likely got their idea, and that newby is a lot brighter than his publication record perhaps shows.
I’m pretty sure F1000 has been doing something similar for a while. Months ago I saw “pre publication” manuscripts on the website. At least I thought I did, perhaps I am confused!! In any case, the place was a ghost town.
In general, this idea is just done to death now. It won’t work. Biology and physics are not the same and while the idea behind post-publication peer review is good, it has been shown to be an epic failure. Nature tried it = fail! PLoS One are trying it = massive fail. If F1000 think that people will actually be bothered to comment on the papers they “publish”, they should perhaps do their research first. I’m starting to really object to this kind of open access force feeding.
This will just become another dumping ground for boring/unimportant research that can’t get published elsewhere and, in many ways, these kind of initiatives only serve to enhance the allure of big-name journals these days.
Lot of publicity over nothing.
F1000 has a repository just like Nature Precedings called F1000 Posters (http://f1000.com/posters) – it launched only in June 2010 and has over 2500 posters and slides (many more than on Nature Precedings which is several years older). I should point out that on F1000 Research, we are not just going to be publishing articles and then waiting for someone to comment; we will be inviting formal reviews just like any other standard journal does. The ‘only’ difference here is that this process will be fully visible and transparent to all.
This is a very key difference from Nature Precedings and indeed Arxiv. Furthermore, articles will be linked where appropriate, there will be versioning, data articles will include structured metadata that is mine-able etc.
And a key point that is mentioned above is that priority will be established when the article is first submitted and hence in this case, published – this publication is the same as the article that is first submitted to a normal journal i.e. it is a complete article. So priority is established immediately. The paper may be improved following comments from the referees (and possibly others), but it is not a part-article so it can’t be scooped. Any further work that is done by other groups following the initial publication in F1000 Research and then published elsewhere is part of the normal positive progression of science, building upon others work.
Why not consider extending the model to be all-inclusive for all scientific disciplines? What happens if/when reviewers do not respond?
Will F1000 Research be indexed in PubMed, Web of Science, etc?
Just following up on Rebecca’s mention of Nature Precedings and F1000 Posters for clarity. Nature Precedings enables researchers to share many types of documents, including presentations and posters but also white papers, technical papers, supplementary findings, and non-peer-reviewed manuscripts. So the two offer related, overlapping services, but I hope its helpful to clarify that their scopes appear to be slightly different. Rebecca, do please correct me if I’ve misunderstood the scope of F1000 Posters.
Grace Baynes
Nature Publishing Group
@Grace – yes, you are indeed correct about the slightly different scopes between Nature Precedings and F1000 Posters. The point I was really trying to make is that they are both pre-print servers (like ArXiv) and so materials placed on all three services are not ‘published’ in the way many think of the term ‘published’ – they are certainly not formally peer reviewed, and this is therefore one of the key ways they contrast with the F1000 Research plans.
@Sierra Rayne – there is no specific reason other than our expertise is specifically in biology and medicine and given the field-specific issues as it is, I think certainly for now, that will be plenty for us to worry about just in these fields.
With regards a lack of response from reviewers, this is all part of our experiments over the coming months as we will need to work out a model that provides enough incentive to get peer review (just as is the case now with articles), and if we really cannot get anyone to referee a specific paper – well, that is a question in itself that we would welcome views on from the community – what indeed should be do with the paper? Of course the less that papers are refereed multiple times for different journals, strictly speaking the less pressure they will have to be refeering all the time but it will take some time for us to get t that point.
@LNV PubMed has a long process to apply for indexing which one can only start once you start publishing, but we shall certainly be applying. And we will also definitely be aiming for indexing with Scopus, Web of Science, Google Scholar, Microsoft Academic Research etc.
This seems like a charlatans charter, and a great boon to the politicisation of science. Just as cheats cite stuff “published” on the internet to massage a veneer of scientific respectability to pseudoscientific notions in pursuit of dodgy agendas, so we’re going to have an “everything is equally valid” publication source where cheats will find their agenda-led rubbish accorded equal status as pukka science.
One of the things that traditional scientific publishing practices protect is the notion of “quality”, however easy it is to be cynical about that old-fashioned idea. I believe that the vast majority of scientists work and publish in good faith, and send their manuscripts to the best and most appropriate journal in their field. Any working scientist knows the top quality journals in their field – they’re where important and influential research is published. That’s pretty much always been the case and I don’t see that much has changed in recent years. Call it “fetishisation” if you like or elitism but I’d call it a broad hierarchy of quality.
An example: Some US politicians are engaging in repellent hounding of climate scientists in the US because they happen not to like the outcomes of their scientific investigations; these (the outcomes) are at odds with their (the politicians’) political agendas. In order to support these attacks a supply of “supporting” “science” (aka pseudoscience) is useful, and thus attempts are made to “sneak” rubbish into the scientific literature with some limited success. The editor of Remote Sensing resigned last year when he realized that his journal had been used in this way; the editorial board of Climate Science resigned some years ago over a related instance of a paper that was massaged into the journal by a sympathetic editor that eased it around proper peer-review; interestingly the publisher of that journal editorialised that the paper shouldn’t have been published in his journal though the paper was never retracted. Happily, at least one attempt at smearing a climate scientist in this way has resulted in a retraction (by Computational Statistics and Data Analysis).
That unscrupulous individuals have difficulty getting agenda-led pseudoscience into journals results from what seems to becoming old-fashioned notions of scientific integrity and the protection of standards of quality. I don’t see how this new publishing model is going to deal with that nasty problem. To whatever extent knowledgeable and honest scientists critique rubbish papers in the comments section of journals of this model, so those that support whatever anti-science agenda is being pursued will counter-critique, and the comments section will take on the appearance of an internet message board.
Upon re-reading, my post seems rather more forthright than I might have meant to convey! So perhaps it should have a big question mark at the end. I think this is a potential problem for the new styles of scientific publication but maybe I’m being overly sensitive to a problem that has workable solutions withing the new model?
initially when Faculty 1000 started they chose people to become members, if I am correct, to have elite members. Now I see even my former graduate students are expert members – not me. It appears that the policy has changed. Moreover, Facuty1000 members are so elite that they comment on papers published in elite journals…therefore, i stopped browsing at Faculty 1000 as there is no point in wasting time – just back scratching as I wrote earlier. I browsed through the Nature Precedings just now – found out that there are lot of contributions from certain countries more often than some countries. Moreover, I could not find many contributions from the so called “prominent” researchers in the field. In addition, you can see hardly any comments on the contributions – which was its main intention. I presume it may turn out that Faculty 1000 new approach will head in that direction though proposer claims that it will do better with the help of its elite members? May be I will be proven wrong, that is alright.
What about “clinical” research ie research used for direct patient care (drug trials, Diagnostic Test Accuracy studies, prognostic studies etc)? Will that also be included? Many (if not most) doctors like the “security blanket” of treating patients based on “peer reviewed” papers. Never mind how smelly or holey the blanket is.
@Chris: You make some interesting points about a much wider problem we have particularly in some areas of science. What I would say is that almost everything gets published now anyway in a ‘peer reviewed’ journal – if you try hard enough as an author, you can usually find somewhere to take your paper. What I hope will be the advantage of our proposal is that we will very clearly mark when a paper is not yet refereed, and also clearly show at a glance whether the referees have approved the article or not. And others beyond the invited referees can also weigh in. This means that if someone subsequently takes a paper that has been deemed as scientifically unsound or has misleading analyses or conclusions, and tries to use it to support their own political view, it will be obvious to all that this is not a sound paper.
@Ressci Integrity: F1000 has always used a peer selection process for the Faculty Members (FMs). At the start we identified the top Heads of each Faculty (there are now about 40 Faculties) and tried to get a good balance between the different Heads of each Faculty. They then identified the Sections and appropriate Section Heads (again delibrately several per Section to get a balance), and these Section Heads then identify the FMs. FMs can also invite Associate FMs to help work with them. The majority of what these FMs pick are NOT published in what might be considered the top ‘elite’ journals. I should also reiterate that F1000 Research is not intended to be the same as ArXiv or Nature Precedings – we will have a formal peer review process, and hence comments from others are additional (i.e. we are not going to be solely relying upon them).
@MSN: Yes we do plan to include clinical research. There are a number of aspects on the clinical side that will may need to consider more closely e.g. data confidentiality etc. There will still be peer review and it will be clear when that peer review has happened so it won’t be any different in that regard to now. I think maybe an easier way to think of this initiative is like PLoS One but just with a similar type of peer review happening after publication and out in the open, rather than before.
@Rebecca Lawrence: Just a few points.
1) when F1000 started it was also free, now even for commentaries, we need subscription? Do you have any statistics how many people read these comments?
2) If Heads are chosen, then it appears that they are elite group of people. What is the difference then between editorial board of elite journals and F1000 faculty member? they all be there, right? If one looks at the speakers at Gordon Research Conferences – same speakers are there in many related conferences within a year…
3) members are also selected and then these members select associate members and chain continues.
4) I browsed through the associate member list. Many of them are within their third year of Ph.D. studies or early postdoctoral career. They list their duties as checking the quality of publications in journals like Science, Cell, Nature etc. May be they are brilliant and geniuses to criticise papers published in those journals (No offense – agree they might have published one or two papers in journals liek Nature and Science.
5) I don’t need to list those profiles as they are publicly available.
6) Hope we are not making elite groups within elite groups.
@ Resci Integrity @sfs @Sierra Ryane @Mike Taylor @Brett – I’m jumping in to clarify a few points about F1000’s core evaluation service, and our new initiative F1000 Research. First, the core subscription service has never been free; perhaps you are thinking of the open access F1000 Reports, which may be the only OA review articles in bio/med publishing. Or, until recently, we permitted browsing of articles rated and evaluated in each section, which also revealed the first couple lines of the evaluations, so you may be thinking of that level of access. We have about 400,000 researchers with institutional access, and about half that number currently visiting the site each month.
Regarding the comparison of our Section Heads (http://f1000.com/thefaculty/headsoffaculty) with editorial boards of journals, as an elite group – I suppose that’s a fair statement. They are certainly an impressive group of researchers and they’re responsible for the makeup of Faculty Members (FMs) and for overseeing the quality of their sections, as Rebecca noted earlier. However, we also now take nominations for new members from current FMs as a way to diminish cronyism and broaden our reach. And we’re pretty happy with the program of Associate Faculty Members –junior researchers who mentor with the senior members and co-author (never individually author) a small percentage of the evaluations. As post-docs and associate profs, they are closer to the bench and have a different take on their specialty areas that can provide a valuable perspective. And there’s a healthy turnover and crossover of researchers at all levels as we work each year to keep pace with the rise and fall and merging and splitting of areas of investigation.
We have a pretty good representation, too, of articles evaluated from more specialty journals. To get a sense of this, you can look at our journal rankings and Hidden Jewels listings: by far the majority of evaluated articles come from non-high-impact journals. For example, at the Section level, e.g., Virology, you’ll see the top three journals in 2010 (provisional) are PLoS Pathogens, Nature Immunology, and Cell Host & Microbe. If anything, we’ve found that some Faculty Members avoid highlighting articles in the high-impact journals, because they want to bring lesser known papers to the attention of their colleagues.
It’s true that apart from F1000, you don’t find much post-publication commenting on articles. And also that scientists may be loathe to publish incremental data and analyses in advance of the final tome. However, it’s my hope that the new generation of scientists raised on social media and open, collaborative science will help turn the tide, and that this will address Mike’s concern that a major driver in the current system is fear of being scooped. That, plus new technologies which integrate commenting with scientists’ workflow, and alt-metrics that make the comments discoverable and quantifiable, might finally disrupt journal publishing as we know it today. This in turn we hope will give funders and administrators a new means of factoring this type of activity toward advancing researchers’ careers. The number of experiments in this transformation currently out there are heartening — each promoting new elements and building upon innovations; but they are by no means mainstream. As quoted in my blog post on http://e-Patients.net earlier in this week, Richard Smith, a pioneer of disruption while at BMJ, noted:
“…experimentation inevitably means some failures. ‘In all science, error precedes the truth, and it is better it should go first than last,’ said Hugh Walpole…. It does, however, feel very bold for editors to abandon prepublication of peer review — like walking into the street naked. But if the emperor has no clothes, what’s to be lost? Nothing, but much is to be gained.”
Finally, to @Resci Integrity’s point that browsing F1000’s evaluations has no longer been helpful in his work, we acknowledge that inevitably there are weaker Sections within F1000 which we constantly scan and work to improve. We depend largely upon reader feedback, and the analyses of our Heads of Faculty and Section Heads to point to areas that need to be strengthened. We believe the absolute transparency that’s at core of our processes make these critiques more meaningful and actionable.
The comments in this blog have already been incredibly helpful as we tackle the next new thing.
@Sarah Greene: Can you guide me where I can get free access to read through the article comments? Is it still possible? In fact, I read the evaluation free of cost during 2000 to 2003. Now it is asking for subscription.
You have not touched upon the fact that graduate students (no Ph.D. yet) being associate member. How do you justify this? Do you want me to quote some names from the list? I don’t want to do this on a public blog. You will realise that those students may have a paper in Science or Nature as a main author or just a co-author. Does it mean that they are able to evaluate and check the quality of publications?
I know you are justifying the cause but I am still not convinced about the associate member selection. You mention that nominations of new members would made by current members. Once their co-worker or mentee gets Ph.D. or experienced he/she will be nominated to be a member. I don’t see much of a change any way.
Nature Immunology is not a small journal. I just glanced through the list of evaluations on your site, more than 70 % are papers published in high impact journals.
“… graduate students (no Ph.D. yet) being associate member. How do you justify this? … Does it mean that they are able to evaluate and check the quality of publications?”
Whyever not? Graduate students routinely do peer-review for journals, why would this be different?
@Sarah Grenee: Thanks for the details and clarifications. I think some of us, myself included, were initially confused as the article above (that we are commenting on) suggested that F1000 was functioning in a similar manner as that of Precedings and/or arXiv, both of which are free to post (although arXiv has the nominated member approach – which is unfortunate) and free to obtain sites that – to the best of my knowledge – are not commercially oriented. It sounds like F1000 is oriented more as a biology/medicine focused mid-point between Precedings/arXiv (which, collectively, cover essentially all science disciplines) and the traditional purely commercial entirely peer-reviewed journals.
@Mike Taylor: Please read the points I have listed above (Ressci Integrity
January 31, 2012 at 8:28 pm). Point number 4 i copy again. No offense.
“4) I browsed through the associate member list. Many of them are within their third year of Ph.D. studies or early postdoctoral career. They list their duties as checking the quality of publications in journals like Science, Cell, Nature etc. May be they are brilliant and geniuses to criticise papers published in those journals (No offense – agree they might have published one or two papers in journals liek Nature and Science)”
They list their duties as checking the quality of publications in journals (which F1000 covers). If they are the ones who check the quality…
Sorry, Ressci, I am still not getting it. Why would we assume that Ph.D students or postdocs would not be able to do this? In my experience, they often have their fingers more on the pulse of current literature than more senior academics who have other responsibilities on their plates.
@Mike Taylor:
that is one of the reason why only few people can publish in CSN journals…..and all others research is suitable for specialised journals…
@Sarah Greene: In the part of the world where I live and work, Associate Professors are not postdocs nor they are mentored by Professors. They are independent researchers who run laboratories and mentor Ph.D.s and postdocs..
Why not just use Reddit?
(also posted on F1000 and THES)
One major concern I have with F1000 is that it is a for-profit company. Quite aside from what it decides to charge for its services to start with, what guarantees do we have that if successful it won’t change its charging practices in the future?
This is a very real concern. If a large scale open access service becomes very successful and therefore influential and then decides to “cash in” on that success, it can be very difficult for the academic community to shift to an alternative model or service. We see this with the current difficulties in shifting away from traditional, very expensive but high reputation journals.
I believe that F1000 is owned by Science Navigation Group, which previously started a number of other journals and services including BioMed Central, Current Biology and others. They have sold these and similar journals/services to the likes of Thomson Reuters, Springer and Elsevier. What guarantees do we have that they won’t do the same with F1000 Research?
Why would academic researchers take the risk of investing in such a for-profit exercise if in the future it could be sold to exactly the type of publishing companies that we’re trying so desperately to escape?
Apparently Nature Precedings will not be accepting any new submissions after April 3, 2012. As a site user, I received an e-mail from Nature Publishing Group this morning. Apparently, the site will be archived and still accessible online. Rather unfortunate.
I believe that F1000 is owned by Science Navigation Group, which previously started a number of other journals and services including BioMed Central, Current Biology and others.