If there’s one consistent lesson of covering retractions, it’s that science doesn’t stop when researchers publish a paper. But what also seems true is that once a paper is published, lots of people — authors and editors, in particular — are often reluctant to say just what’s happened next, particularly if it casts the study or the journal in a negative light.
Some of this is understandable, given the weight given papers by tenure committees and granting agencies. Still, Retraction Watch readers will not be surprised to know we’d like that to change, so when Nature asked us to contribute an end-of-the-year commentary, we decided to focus on post-publication peer review. In our piece, which appears this week, titled “The paper is not sacred,” we argue:
What is needed, instead, is a system of publication that is more meritocratic in its evaluation of performance and productivity in the sciences. It should expand the record of a scientific study past an individual paper, including additional material such as worthy blog posts about the results, media coverage and the number of times the paper has been downloaded.
As it happens, there will soon be a way to do all of that: CrossMark, about to be launched by CrossRef. (Ivan recently spoke at CrossRef’s annual meeting, but just in case it’s a question, didn’t accept any travel expenses or honorarium, and wasn’t talking about CrossMark.) As we note in the piece:
The idea is for every piece of content to include a clickable logo that will let a reader know whether there have been any corrections, retractions or other revisions. It is a solution to the fact that such changes are at best difficult to find — and are sometimes not mentioned at all on ‘current’ versions of papers.
That is the ‘Status’ tab on CrossMark. But the platform will also have a ‘Record’ tab that gives publishers a way to take the idea even further. They will be able to include material they didn’t produce, such as blog posts, media coverage, letters, additional data and metrics such as downloads.
You can read our whole commentary, in which we also question the troubling practice of allowing authors to make new claims in retraction and correction notices, sans peer review — here. (We have it from a higher authority that the naked Adam pictured holding a scientific paper might, or might not, be based on Adam.)
A final note: As Retraction Watch readers know, we tend to beat up on lots of journals, Nature and its sister journals certainly among them, so we appreciate the space they’ve given us in this week’s issue. We also appreciate the kind words from Nature Medicine, which called us “key people who made headlines this year, either by standing up for what they saw as right or by stopping what they felt was wrong.”
Congrats on the paper and the well deserved words of praise.
This sounds really good. Not being a subscriber to NATURE, I’ve got to figure out a way to read this commentary.
For those not subscribed to Nature, my comment there: Technological developments passing science by – when has that happened in history before? Scientists essentially rely on a 17th century legacy communication system (see also Kravitz, D., & Baker, C. (2011). Toward a New Model of Scientific Publishing: Discussion and a Proposal. Frontiers in Computational Neuroscience, 5 DOI: 10.3389/fncom.2011.00055 ). This anachronism is only getting worse as the development of communication technology continues to hurl past seemingly dumbfounded scientists (and, apparently, editors) who are struck by the pace at which the general public, often quite scientifically literate, and some scientist colleagues are embracing this technology (and sometimes using it against them).
Science, almost by definition, constitutes the epitome of the cutting-edge of human development. Scientists need to be at the forefront of the intellectual endeavor. A scientific communication system that mentally binds scientists to an antiquated mindset can only be detrimental to science itself. If the general public is overtaking science in the race of innovation, something is embarrassingly rotten in the state of science. The popularity and success not only of retractionwatch.com but of many other innovations in the scholarly communication field are testament that we are in need of a massive overhaul and modernization of scholarly communication and not just some cosmetic tweaking around the edges.
Thanks, Ivan and Adam, for a fine article. Apologies for posting a couple of comments here since I can’t post them on Nature’s site.
“If journals aren’t willing to start reviewing and compiling additional content related to their papers, someone else will do it.”
By “journals” do you mean editors or publishers? Regardless, it seems that in many cases neither actor is highly motivated to do the extra work of correcting the record (by reviewing and publishing letters to the editor, for example, or dealing with corrigenda unless absolutely necessary) or reviewing and compiling (on CrossMark, for example) reactions that appear outside the purview of their own journals.
For all publishers’ and editors’ claims to adhere to ICMJE and COPE guidelines, there seem to be a number of cases out there in which journals have failed to do everything they could to correct the record. I’d guess that the cases that have been reported one way or another are the tip of another iceberg.
Editors and publishers, out of their own self-interest, would actually be more motivated *not* to submit links to criticisms. So for many articles, “someone else” would have to provide content for CrossMark’s ‘Record’ tab. But who? This would involve real work and real time, and the resources (enough people motivated to do this for nothing) don’t seem to be available.
CrossMark is a great idea since bringing criticisms together and making them easy to find, read and cite would be very helpful to readers of the original item. The idea that the scientific literature should be “self-correcting” is comforting but self-correction often does not succeed in practice. Best of luck to CrossMark in providing support for self-correction!
i have publish one journal .some problems is there.what is procedure for cancel the paper
I can’t read your piece – it’d hidden behind the Nature paywall. Did you hand over the copyright to them or can you reproduce it here?
Thanks for your interest. We did hand over copyright, after a great deal of thought. Our feeling was that reaching Nature’s audience was worth the lack of open access. We have the right to publish the version we submitted after six months.
Please be sure to post it at that time, as I’m sure it will still be of much interest to the readers of RW. Thanks.
Well it it really nothing new that science undergoes post publication peer review. It has been happening all along, well before internet. However, in the past, alterations to theories and concepts were made in the scientific literature, as were corrections to data and conclusions. It seems the internet only allows it to be hyped up and and blown out of all proportion when something happens. These corrections/ revisions and changes to theories and concepts are simply part of the process of science.
I am not so sure that the internet and blogging hype will improve the self correction process nor post publication peer review. What it probably will help is the journalistic ambitions of a few individuals, and not much else.
The media and the media wanna-be(s) have more to answer for than is superficially obvious.
After all, even Einstein effectively published a retraction to an erroneous hypothesis. Maybe if he was around in the internet age, he would have been censured by the public and a university investigation committee, and may have lost his job. Most Nobel prize winner’s work is on the edge, but now, who will take that risk ?
The world has only lost perspectiveand a proven scientific process, and gained hype. After all, 2000 years ago, hype said the earth was the centre of the universe, and 1000 years ago hype said the world was flat.
Perhaps, it could start with not fetishizing the scientific stars or star institutions at the expense of solid work done elsewhere. Nature is guilty of that, firstly.
The whole peer review process is so flawed as to be totally worthless. As these “so called” peers sit on boards that review their friends papers and will vote to publish based on the fact that the reviews will present their own paper to be reviewed by the first author who sits on another peer review board or whom they have collaborated with on other papers. In court this would be called incest. Also universities spend large amounts of cash to wine, dine and offer guest lecture fee etc, to people sitting on the peer review boards, in additon spending countless hours in providing background information to their professors on the personallities, likes, dislikes and a general profile on the personal habits of the peer review group, so that the paper can be adjusted to appeal to the ego of the reviewers. In addition these “peers” will refuse to publish papers that dispute their own beliefs or papers they have written.
scott, that’s simply not how peer review works. I’m curious to know where your notions came from! Scientific papers aren’t reviewed by “boards”; they’re reviewed by other individual scientists that act independently and make recommendations to editors. In the context of scientific publishing, your stuff about universities “wining and dining” people on review boards is silly, since there aren’t “review boards” for scientific papers and the choice of reviewers is up to the editor in any particular case and is usually anonymous.
In my experience peer review works pretty well even if it can be tedious if not downright maddening on occasion. Most of the papers I’ve published have been improved by peer review, and I take my role as a reviewer very seriously; that applies to the scientists I have personal experience of. Since the vast majority of publishing scientists submit their work in good faith, the vast majority of submitted papers are not problematic and as we know science and knowledge continues to advance productively. The fact that there are some dishonest or unnecessarily careless individuals that work as scientists doesn’t mean that there is anything inherently wrong with peer review.
Unfortunately peer review isn’t very good at identifying fraudulent practices. If someone makes a convincing fabrication and presents this as fact, it may not be easy to spot. Simply put peer-review is carried out with the presumption of good faith; in reviewing a paper one has to assume that the data presented is a faithful representation of the experimental output.
One of the excellent roles of RetractionWatch is to highlight the nature of contemporary scientific malpractice where it exists, as well as improving the practices of publishers and editors. As perusal of RetractionWatch shows, a considerable amount of contemporary malpractice resulting in retractions aligns with our electronic age: image manipulation and cut-n-paste plagiarism are both easy to do and a few dishonest individuals seem unable to resist this. The happy corrollary is that that these electronic frauds are similarly rather easy to identify. Once journals more widely routinely assess electronic manuscripts for these frauds, I expect these practices will reduce. RetractionWatch is doing a fabulous job in encouraging publishers and editors to be more responsible in this regard.
So as we come to the end of 2011, I would hope the efforts of RetractionWatch will be recognised as a resourse for spotlighting the tiny but damaging group of individuals that cheat in science, and encouraging better scientific publishing practices, rather than as a tool for broadside attacks on peer review (which has its faults but actually works quite well for scientific publishing IMHO).
chris i will attempt to explain myself further, in the grant writting process a professor will write a proposal to study a certian thing (grants are a very important source of pay for the professor (extra pay for the professor)and a hugh income producer for the university, as most universities take half of the grant money for administrative costs) the granting agency (can be either federal or private money). after the grant is approved and work is submitted to both the grantor and to various publishing companies or to various professional organizations (in the hopes that one will print the paper). this process is very important to both the university and the professor as once the professor is published it makes the next grant easier to obtain
having seen these for over 20 years, the process for publishing papers is exactly how i described, a sub committe of 2 to 3 people are chosen from a field (these postions are sought after buy professors and their univesities), these people then are assigned papers (and yes i am aware that the papers are submitted without names or universities, but most people in their field will identify the writters of the papers based on past experience and knowledge of the grants received, most papers submitted identify the grant the paper was written under which is then easily traced to the writter of the paper) these reviewers then submit a recomendation to the editor (or a board of editors) for their approval to be published. now anyone can get a paper out, via mail, internet etc. but to be published in a trade magazine, this is how the process works. you even agknowledge that you write papers and do peer review on others who submit paper, that is comparable to a criminal prosecuter sitting on a jury, i repeat that is incest.
your mistake belief that universities don’t wine and dine board members of professional organizations is flat out wrong, this can be confirmed by looking at the budget of any university and look at the pay incentives given to professors for belonging to these board members, universities will pay fees and expenses to their employees to attend conferences and will even pay the professional dues for them and the pay that is given to visiting professors or guest lectures (who of course do peer review). the writting of the papers is just a pay scam for professors, i would guess that even you got extra paid to write the vast majority of the papers you wrote of course over an above your regular university pay.
Scott, in what area of science do you work? You claims are so far away from my experiences, that it hardly can be any of the natural or life sciences.
In my field, I know of no staff members, professor or otherwise, who gets any money from his university to go to conferences and pay membership of professional organisations. Most have some money that can be used ‘freely’ (within loads of boundaries), but that’s not the same.
Also, I know of no colleagues who get paid extra for writing papers. It’s a requirement for me to write papers, but 5 or 20 doesn’t do much to my pay. At best I can try and get a little extra for extraordinary contribution (with the 20), but if that’s being paid extra, I would earn more, and use much less time/effort, having a paper route!
I also know of no colleagues who get extra money when they review papers. Heck, this year I reviewed about three times as many papers as I published, and my head of department called me nuts!
the university pays based on a 9 month year (this is the policy of the business department but is university wide) when a grant is written, it is based on a 12 months of pay of the professor and work, the grant work is conducted 12 months a year, and pay for professors salary (if more that one professor is on the grant the pay is spread out), grad students (if any), lab time and university administration cost are paid on this process.
the university thru it’s internal grant process will pay speaking fees, professional fees and board fees for active professionals. Most business professions retain professors to review papers on a cost basis. I am not familiar with the natural sciences, so my experience is limited to business.
both marco and chris missed my point of peer review, both of you write papers and both of you review papers, you both say you are neutral when you review papers but…. you both know the style and writting of the others in your profession and like all people have bias in your work,i’ve seen many excellent papers which never have seen the light of day based on the bias of the reader. most universities have assistants who will help write the papers based on the reviewers. i know the argument that only peers can judge the nuances of your work, but other professions have true outsiders review and approve work being done, a building inspector is one area (i mean would you live in a house that was certified as sound knowing the inspector was another builder and a friend of your builder) police have civilian review boards, and countless federal agencies like the EPA (would you want an other power plant operator to review another power plant operators emissions) just to name a few.
i have seen other institutions mired in corruption based on the internal review process similiar to higher learing
one point i did forget to mention and that the use of citations in papers submitted. if you look carefully into the back ground of the list of citation in a particular paper many times the papers citited are written by person who have worked with or been associated with the papers author. these other paper’s writters in turn cite each other in a never ending chain. more incest, but what is missed is a legal term call “fruits of the poision tree” which in regular terms mean if even one of the authors has doctored, missused, fudged or other wise screwed up. it requires the dismissing of ALL papers and authors connected with the author, something that is missing in the modern peer review process. chris you can not accept things on blind faith sorry but the earth is not flat and the earth is not the center of the universe, those things were faith too.
Scott, your analogies do not work at all. The building inspector is an expert in the field (if not, I’d rather take my chances with the guy who is a friend of the builder). The EPA uses experts in the field. The civilian review boards for the police are not just ordinary civilians either. These people are generally chosen based on their technical understanding of law enforcement. Experts.
The peer review process in science may be flawed, but you thus offer no alternative. It’s like democracy, as Churchill said: “Democracy is the worst system, except all others.” Bad papers go through, good papers do not, based on the subjective nature of peer review. That would also happen if the expert reviewers are ‘independent’. I would not necessarily trust that independent building inspector any more than that building inspector who knows the builder. It’s very well possible that the latter is much more of an expert than the former. I also know that I rather have my scientist ‘friends’ be honest about a bad paper of mine, than having it torn to shreds later when it is published.
Oh, and note that in many countries salary is not dependent on grants, and even in the US it is not an absolute. I myself have a fixed salary, and that’s it. The grants allow me to hire others to do most of the research.
Scott, I also must object your claim about the “poision tree”. I have co-authored papers with many different individuals (30 or so, if I remember correctly). I have likely also cited their own papers frequently. According to you, if one of those co-authors has fudged data or done something else that is immoral, my own papers should now also be considered fraudulent? Sorry, but that’s just plain paranoia. I think that you yourself would be pretty angry if the police pay you a visit and search your house and take a very close look at all your finances, just because one of your close colleagues has been caught stealing. Or are you OK with the idea of “guilty by association”?
maro the building inspector, the epa inspector all work for independent agencies. they do not advance or diminish their lives by passing or failing a project, they are NEUTRAL observers and have no skin in the game, peer reviewer do win or lose by advancing or rejecting papers of others, be they friends, associates or points of view that the reviewer supports. we allow judges in court to make case law, yet none of them are experts in the cases they are presiding over, and you and other seem to have no objection to that. Most civilian review boards in america are not made up of experts in the field of law enforcement (our local civilain review board has no current or former police officers on it, (if it were the peer review process used by the learning community and using your logic only police officer, who are the only real experts in law enforcement, could serve on the review board).
you also make the illogica, “guilt by association” argument. if a business associate of mine were to get arrested for fraud/theft and we have worked on project together the police would be obligated to look at my records and all the joint projects as well, that is not paranoia but a logical extention of the fraud/theft. and in the real world that is how it does work.
in a building if one of the supporting beams is substandard, weak or made from inferior steel the building process must stop and the beam replaced or the build, when completed will have issues and could fail, the same applies to your 30 cite paper, if one of your 30 cites is inferior than the whole paper will fail.
i will give you an example of the bias in the peer review process, you work in the life sciences, and if i were to present a creationism paper you might object to the merits (no matter how valid the work is) of it out of hand (and no i am not a creationist) based on your training and your bias. Like wise a paper supporting evolution you would not look at as criticial as the creationism paper. i will repeat myself I AM NOT A CREATIONIST. the issues is that the peer review process comes with built in biases that can not be policed internally but needs someone from the outside to look at all the thousands of papers we produce each year (the trouble is we produce so many papers that are only good as bird cage liners, i have seen many business papers out of date due to new laws, computers or practices, before the research work was finished and the revisions to the paper out of data before they were done as well).
all sites like “retration watch” is asking is that the general public be allowed to look at the work of experts and make comments about it and conduct fact checks independent of our learning community. i know it is kind of scary for you because you have lived in the cloistered world of colleges and universities for so long, but there is a real world out here that is becoming increasing sceptical of all scientific work before we lose our supporters, ceditability as we have more stories of fraud/theft we must open up the process or a least take the first step, i know your egos (and boy do some of you have hugh egos) will get bruised, but we will be better off in the end.
Scott, seriously, you should rethink your examples. Those ‘independent’ building inspectors are just as human as anyone else, and thus will be affected by personal bias. Judges are experts in law, and must evaluate a case based on laws. They are thus evaluating cases in which they are experts, even though they will also be dependent on expert evaluations on certain details. A creationist paper would indeed be more likely to be more closely scrutinised by me, but that’s because it contradicts the vast body of scientific literature, and because it is rooted in an a priori biased starting point (there is a supreme being who created the world). The civilian review boards essentially always have at least several people who know about law (not law enforcement), and there is a selection procedure: you’ll have to explain how you can contribute based on your experience/knowledge.
The guilty by association example you got wrong, too: the way you describe how papers should be dismissed if they cite a paper in which a fraudulent issue is present is the same as you being investigated by the police if your colleague has committed tax fraud, or stole something from a store. Unless there are indications you were involved (your colleague says you helped him file the taxes, or you drove him to the store), there isn’t a single police officer who would even consider going after you, too. And be happy, or you’d be under consistent scrutiny by the police, because there is very likely at least one person in your immediate vicinity (colleague, friend, family) who has done something bad, even if it is only speeding.
Also remember that many of the citations in papers are meant to put one’s research into perspective. Unless that fraudulent paper is the all-defining perspective, or the main source of your data, citing a fraudulent paper is unlikely to have an impact on your own paper. There will even be cases where your paper is actually strengthened by the paper you cited being fraudulent, e.g. when it contradicts your own findings. I have also cited heavily flawed papers, specifically because they were flawed. In some cases there may well be some data manipulation in those papers, too. In your analogy, my paper is now to be dismissed! As General McAuliffe said: NUTS!
You will also find that few scientists have problems with their papers being scrutinised by others that may not be experts in the field. The problems start when this scrutiny is biased, with the person doing the analysis insufficiently aware of the facts and the context.
And please get off your high horse: all scientists live in society, universities are not Ivory Towers, at least not more than any other part of society. People with large egos you will also find everywhere.
marco can we please stick to the words that i write and not make up straw men for you to knock down.
a building inspector is paid by a third party usually a government. these inspectors will and do base their inspections on enginerring principals ie load bearing i beams, locations of switches, wireing requirements the list can go on. a judge is an expert in law they apply the law as the 3rd party experts testity. as far as our metro civilian police review board we have a retired judge but we also have a welder and a house wife, none of those jobs are a disqualifier for the board and actually make the board stronger. in the university setting such reviewers would be looked upon with distain.
as far as the 3rd or 4th straw man yes of course if you friend committed a criminal act you would not be looked at, however if you had contract dealing with him/her and he had committed a fraud you would be looked at and to say you wouldn’t is laughable. to compare speeding with the fraud committed by the writters of these papers is yet another stupid straw man.
i do agree with you that the fraud that is committed and exposed does make other works more reliable. however if you use a fraud based paper that follows the same line then your paper is useless.
the reason we write papers should be to make the public aware of issues and not be afraid of the way they might misread them (it is our job not theirs to make them understandable) we must not get into the habbit of writting papers for each other
Scott, I think we will never get anywhere in our discussion, as it seems you are now moving the goalposts to accuse me of strawmen. You made huge black-and-white claims, and then get upset they are being challenged. You clearly stated that one author’s misconduct affects all co-authors (that is, they are all to be considered complicit in the misconduct), which is the same as accusing people of speeding because they have sat in the car of someone who was speeding at one point in time. Maybe you meant something different, but that’s not how your tirade against peer review sounds to me. Especially when you state that if I cite a fraudulent paper that works along the same line, my paper is to be dismissed, too. That is the same as being pronounced guilty by some type of association, without having any proof.
I also think you should reconsider that supposed objectivity of your building inspector, judge, or civilian review boards compared to the peer reviewer of scientific papers. You will find that building inspectors often have been employed or otherwise worked with construction companies (sometimes they still do). Judges in several fields have been lawyers previously, defending the same type of people whose case they are now to judge. In civilian review boards you will frequently find that they actually have professional investigators (who are likely to have some type of prior experience in the field).
A final request: since you are so negative about peer review, can you suggest an alternative?
once again i will attempt to show you the use of fraudlent work cited in a paper makes the paper invalid.
if a good mason/brick layer is building a wall. he will use footing to make the foundation solid, he will inspect each brick as he lays them (and if he is good will discard bad bricks) and will cap off the wall with cap stone. at the end he may point out the bricks that he discarded to the owner or the those who would build other things on top of his work, that does not make his wall stronger nor does it assists the other who follow but does give some assurances of its quality. if however he uses even one defective brink (thru no fault of his own, an internal fault) the structure is weakened and will be subject to failure, that is not my opinion but a fact, thus papers used that have fraud in supporting paper are also weak and subject to failure. (however that was not my point in the peer review process).
the peer review process should be open to the general public and all (i repeat all) supporing data released. (the gobal warming group should have released all their raw data and could have avoided a lot of issues if they had).
we as the learning community should trust in the regular people when we release papers (some assume that since regular people do not have letters behind their name (ms, phd) that they some how are unqualified to read and judge papers, i know and have met many people who are smarter and more intutive knowledge that some of the most senior tenured Phd. that is all i am asking for transparency and the willingness of the university collistered persons to accept review by so call non professionals.
if you can not even accept that. then your are like a person standing in the middle of a road and can not imangine a vehicle would hit you.
i can not make my argument any simpler than that.
Scott, once again your analogy is flawed. What you are saying is that one flawed brick makes the whole wall unstable. And that is, as any construction engineer will be able to tell you, incorrect. It may make a wall that is designed poorly from the start unstable, but not walls that are properly designed. There may be cases where that flawed brick is an important stone in some aspect of the wall, and then it will be problematic, but this is hardly ever the case.
Now, as to your suggestion on the involvement of the general public: no one prevents anyone from reviewing scientific papers! Your demand is not prevented by anyone to start with! But if you think that review is more objective, I think you are the one in the middle of the road. There are several fields of science where the results are not exactly to the liking of vested interests, and where you frequently see ‘reviews’ of scientific papers that are clearly ideologically biased. There’s a good chance Richard Lenski could not have published his work on bacterial evolution if he had been required to have his work reviewed by the general US public!
Regarding sharing data: are you willing to give me 50,000 dollar a year, so my group can put all raw data in publishable format? I need the money to maintain the data server and pay for the data traffic, and pay someone to make the raw data, like spectra, readable and understandable for everyone. That’s just my group, so please do set aside another few billions for all scientific groups around the world.
And is it OK for me to deny to give you data that I received from someone else with the explicit or implicit request not to share with others without prior consent. If yes, you may want to rethink your reference to ‘the global warming scientists’…
And while you are at it, I suggest you read Lenski’s second e-mail to Andy Schlafly, pointing out that sharing of certain ‘data’ requires the recipient to have certain skills, or otherwise that sharing would be outright dangerous. Or are living bacteria suddenly no longer “raw data”?
I’ll read the article in the library, thanks … I can understand why Nature wouldn’t give it away for free, but $32 is more than a bit excessive. Seems to me Scott has a point: a vigorous peer review process, in theory, should prevent many of these retractions. That it doesn’t suggests that review prior to publication has become something substantially less then impartial. Journals, universities, and laboratories have an obligation to ensure that their research is scrutinised BEFORE it’s published. The scrutineers shouldn’t be limited to the authors’ friends, colleagues, or ‘peers’ applying for similar grants. Had Michael Manne’s famous ‘hockey stick’ paper been subjected to review by a single qualified statistician (common sense, since statistics were central to his conclusions, and he’s clearly no statistician), a decade’s embarrassment would’ve been avoided — to everyone’s benefit, especially Manne’s. The errors in Wakefield’s paper were well known a decade before it was retracted. That peer review didn’t set off a single alarm bell prior to publication is, in itself, alarming. That it took so dreadfully long to discredit his faux conclusions is clear evidence that his ‘peers’ were unable or unwilling to speak out. Journal editors and research institutions need to be far more hard-headed about their output. It’s a whole lot cheaper and easier to prevent standards from slipping than it will be to re-establish them once they’ve been trashed.
Rob, what easily recognisable errors are there in Wakefield’s paper?
Rob, both your examples (Michael Mann and Andrew Wakefield) far from being examples of flawed peer review, are actually examples of something rather more sinister – the creation of false controversies by individuals and groups with non-science agendas.
Since Mann’s conclusions have been shown to be largely independent of statistical methodologies, and have been broadly confirmed in numerous studies in the intervening years, it’s difficult to argue that publication supported by peer review wasn’t entirely appropriate. It’s telling that the only objective frauds associated with Mann’s work has come from some of the characters that have engaged in the disgraceful efforts to trash his work as shown, for example, elsewhere on RetractionWatch ( http://www.retractionwatch.com/2011/05/17/controversial-paper-critiquing-climate-change-science-set-to-be-retracted-because-of-plagiarism/ ). As for Mann’s reputation, that’s scientifically intact; since the hockey-stick work he’s published around 100 papers with an astonishing level of citation, so he clearly continues to make an extraordinary contribution to the science.
The Wakefield debacle is different since the original work does seems to have been based on fraudulent practices. However as I described in my response to scott above fraud is not necessarily identifiable in peer review. The data in Wakefield’s paper showed that 12 autistic individuals that had the MMR vaccine (not surprisingly since almost everyone has this in the UK) also had a rare bowel disorder and in 8 cases there was a putative temporal link to the MMR vaccine. These essentially anecdotal observations were publishable even if there was already lots of evidence against a causal link between autism and MMR. I don’t think there was anything necessarily wrong with the peer review process.
The fact that this was whipped up into a ludicrous controversy was almost entirely due to ill-informed press campaigns. By 2004 10 of the 12 coauthors of the original Wakefield published a statement in the Lancet that they dissociated themselves from any interpretation of a casual link between the vaccine and autism, and by that time numerous studies by other groups failed to reproduce Wakefield’s findings. So the whole controversy should, especially if supposed science journalists had done their job, have been resolved at least by that time.
Sadly peer-review doesn’t apply to the press (or the internet). The astonishingly incompetent and dishonest behaviour on the part of many journalists with supposed expertise in issues of science with broader societal relevance is truly a wonder to behold. If we lived in a world with widespread competent and disinterested investigative journalism each of these contrived debacles would have been defused years ago.
coming at this from a legal point of view
1. lawsuits are won based on someone using a flawed brick, and inferior part on a car (a 50 cent spring), an I beam that could not support the weight, the list could go on. or are you trying to tell me that your are now a construction engineer and are willing to testify in court that using a faulty part is ok because the other parts are strong and the whole product is good because it was well engineered. our product should be subjected to the same scrutiny as any other product that is made, apple makes computer, ford makes cars, bp drills for oil, we produce papers, very little difference a good product will stand the test of time.i didn’t mean to move the goalpost for you but tried to make it simple
2. all reviewer and authors have a vested intrest, (whether or not you would like to admit it). let the public judge who has the bigger bias, simple.
3. just make the data viewable universities have the very large servers with the space(if the university of michigan can send and receive over 50,000 emails a day with attachment to include raw data, photo and every students personal email) ( i would suggest that you have all your data on a server now), make access available.
4. if a person makes data that can not be shared then, their paper should not be cited. or at the very least note that in the cite.
5. only higher learned people with a higher skill set can understand data, talk about high horse.
6 live virus, strawman.
you cited winston churchill in an earlier post about democracy, what i asked for is democracy, provide the population with all the information let them decide. you asked me in this thread what we are to do and I have done that, if you are either unwilling or unable to meet these small requests and make excuses why you can’t, then you open yourself to the slings and arrows that will come your way because of the frauds in the learning community. sorry for you
Scott, do I really have to point out again that at present the general public is in no way prevented to review papers?
Moreover:
1. No lawsuit is won when the inferior part is not the cause of the failure. even better, no litigation is successful if a part is inferior but there has been no failure! In other words, you will need to show a paper is flawed, and THEN you can possibly refer to the fraudulent paper its cites as a cause,
2. You’re asking the general public, of which the vast majority have no way of understanding what is going on, to determine who is biased. Good luck with that popularity contest, which will be completely governed by vested interest. I’m definitely not interested in that.
3. So, you are not willing to pay for the extra effort it takes to collect the data in one place, and make it accessible. You are also aware that those mail servers are regularly replaced and emptied (oops, there goes the data) ?
4. You are clearly not aware what you are asking for, since a lot of ‘data’ is not tangible in the sense that it can be entered into an excel worksheet. Nor does that prevent fraud in any way! Remember that many frauds involve manipulation of data, so who will be there to check that a researcher really puts the raw data in? And what exactly IS raw data? Is it the detector voltage of my instrument when I collect a spectrum, or am I allowed to show the absorbance (which involves a calculation) ?
5. Yes, to handle data you need higher skills in that area. Do you think you can handle all forms of statistics? If so, remember that 99,9% of the general public does not. In this case my example was one to live bacteria (not viruses), where the ‘data’ is the bacteria. No strawmen there at all, the data is directly in the bacteria, unless you blindly believe the authors when they report number of bacteria and growth rate. In that case the latter is the data, and you’ll need an understanding of how bacteria grow to understand whether the observed curves make any sense.
Finally, you still have not told me what the alternative to peer review is. You propose some changes (release all data!), without realising that it is not that easy, You also propose that the general public is allowed to decide, but you do not describe how this would have to work in practice. You also do not explain how this would counter the frauds in the learning community.
1. my son’s ipod, prime example, class action lawsuit ie the battery, some were made not up to apple standards (repeat no failure) result large settlement. the same goes for all products including our.
2. repeat everyone has a vested intrest, our community does not want to admit ours.
3 so you do not keep your raw data on a university server, or do you re inter your work everytime a server is changed (by the way how do you send your raw data to friends and associates in far away places the postal service, pony express)
4 so people are going to commit fraud and its getting worse and we should do nothing to try and stop it.
5 i may not know how to handle some data but i would guess that someone in the populations who is not connected to the learning community does.
5A. when you send out a paper on bacteria do you send the live bacteria to the reviewer or to you friends, your are being silly.
6 i did not say it would be easy (it is simple yes,easy no, people in our community are resistant to change), but an open and honest debate of our work and the exposing of work to sunlight would go a long way