About these ads

Retraction Watch

Tracking retractions as a window into the scientific process

Cell, Nature, Science boycott: What was Randy Schekman’s tenure at PNAS like?

with 49 comments

Randy Schekman, via eLife

Randy Schekman, via eLife

By now, Retraction Watch readers may have heard about new Nobel laureate Randy Schekman’s pledge to boycott Cell, Nature, and Science — sometimes referred to the “glamour journals” — because they damage and distort science. Schekman has used the bully pulpit of the Nobels to spark a conversation that science dearly needs to have about the cult of the impact factor.

The argument isn’t airtight. Schekman — now editor of eLife, an open access journal — says that open access journals are a better way to go, although he doesn’t really connect mode of publishing with the quality of what’s published. Others have pointed out that the move will punish junior members of his lab while likely having no effect on the career of someone who has published dozens of studies in the three journals he’s criticizing, and has, well, won a Nobel.

All that aside, it was Schekman’s reference to retractions that, not surprisingly, caught our eye:

In extreme cases, the lure of the luxury journal can encourage the cutting of corners, and contribute to the escalating number of papers that are retracted as flawed or fraudulent. Science alone has recently retracted high-profile papers reporting cloned human embryos, links between littering and violence, and the genetic profiles of centenarians. Perhaps worse, it has not retracted claims that a microbe is able to use arsenic in its DNA instead of phosphorus, despite overwhelming scientific criticism.

Although the first sentence is an inference — one that others have also argued — everything else Schekman says in this paragraph is true. But just how many retractions have these journals had? And how does that compare to the number in the Proceedings of the National Academy of Sciences (PNAS) while one Randy Schekman was editor? Here’s the 2006-2011 data, analyzed according to Ferric Fang and Arturo Casadevall’s Retraction Index, which calculates the rate of retraction per 1,000 papers published:

Journal

Retractions

Articles

Retraction Index

Impact Factor

Science

10

5702

1.754

32.452

Nature

7

5403

1.296

36.235

Cell

11

2899

3.794

34.774

PNAS

23

21614

1.064

10.472

So yes, PNAS had a lower Retraction Index than the other journals, but not really that much lower than Nature. Put another way, however, PNAS retracted 23 papers from 2006 to 2011, while Cell, Nature, and Science retracted 28. And perhaps even more important, there were 1,300 retractions in journals other than those four.

“Wait,” you’re saying, “are more retractions really a bad thing? Didn’t you just publish a post about a study that said the opposite?” Well yes, yes we did. But Schekman is suggesting retractions are a mark against a journal, which we think makes PNAS’s record of retractions fair game.

When Schekman refers to the fact that Science hasn’t retracted the arsenic life paper, he invites readers to review the history of requests for retractions at PNAS. And that doesn’t look all that different from the journals he’s criticizing.

We covered this retraction of a cheetah fossil find, for example, which took four years, three of which were during Schekman’s tenure. Along the way, PNAS wouldn’t even publish a letter of critique. Compare that with the multiple letters — and two replication attemptsScience published about the arsenic life paper.

Or take this example, chronicled by UCLA researcher Andrew Diener. Diener raised concerns about a paper in late 2006, and after an exchange of emails with the authors and the journal, managing editor Daniel Salsbury told him:

We have now received the authors’ response to your comments.  A member of the Editorial Board has re-read the paper and the comments from you and the authors.  I have included the authors’ reply and encourage you to contact them directly regarding any additional concerns you may have with their work.  The Editorial Board has concluded that no additional action is required by the authors at this time.

Diener wasn’t satisfied with that, so he wrote the journal again. Then two years went by, and he found the paper retracted. So PNAS eventually did the right thing. But in those two years, the paper picked up ten citations.

None of this proves that PNAS under Schekman had a worse record than did Cell, Science, or Nature. It does, however, suggest the picture may be a bit more complicated than his Guardian piece let on.

In the end, though, it’s hard to disagree with Schekman’s conclusion:

Funders and universities, too, have a role to play. They must tell the committees that decide on grants and positions not to judge papers by where they are published. It is the quality of the science, not the journal’s brand, that matters.

In fact, even Nature editor in chief Philip Campbell agrees.

About these ads

Written by Ivan Oransky

December 11, 2013 at 9:30 am

49 Responses

Subscribe to comments with RSS.

  1. He’s right, but there is the stink of hypocrisy since it is no doubt his successful history of publishing in exactly those journals that has put him where he is today.

    The problem as I see it (as I posted in a previous thread) is that those journals (particularly Science and Nature) are really magazines that consider themselves gatekeepers of the Hot New Thing. They are discriminators of fashion and I compared them to Vogue. The issue is not one of quality science vs poor science, but one of self-validation through publication in so-called elite journals. In other words, something is ‘important’ simply because it is published in Science/Nature/Vogue.

    Science should not be a fashion show, nor a popularity contest, and there should not be such gatekeeper journals.

    Dan Zabetakis

    December 11, 2013 at 10:52 am

    • These journals aren’t gatekeepers, there are literally thousands of credible alternatives. But they may be fashion gurus, to better fit your analogy.

      UNOUNO

      December 12, 2013 at 7:35 am

      • you ever try to get a faculy position without a glamour article?

        yonemoto

        December 26, 2013 at 10:16 pm

    • Nature Cell and Science look at publications as products based on acceptable risk/fame and glory theory: High impact factor journals are willing to publish riskier, but potentially higher-impact claims ASAP – more retractions are the price for getting high-impact science out early. The more negative version of this theory is that high impact factor journals CARE more about a high impact factor than about the integrity of what they publish. Go on feed them garbage and you will get garbage.

      Edgar Wilde

      December 24, 2013 at 1:17 pm

  2. I am curious about the consequences of having ‘luxury’ funding agencies like the HHMI and Welcome trust in charge of a journal. Have the funders run a journal seems to present a conflict of interest. Will the editors of Elife be under any pressure to review HHMI funded research? Or will they refuse to consider research that contradicts a member of their prestigious body? Will people without HHMI funding feel under pressure to publish in Elife to help get HHMI funding?

    JW

    December 11, 2013 at 11:29 am

  3. This is an excellent analysis of Schekman’s views. It’s a complex issue. On one hand, we sorely need to have this discussion, as the glamor rags of science really do distort the whole process of impact and assessment, and its only going to get worse as the stakes get higher and the makeup of what constitutes a minimal publishable unit in these journals increases. On the other hand, and with the greatest respect to Dr. Schekman, we have to seriously ask whether he has the credibility to lead this discourse. As someone who has more than 40 Cell, Science, and Nature publications to his name, has built a career with such high impact credits, and who has a vested interest in the success of a new journal, I don’t think he fits the bill.

    Peer007

    December 11, 2013 at 11:46 am

    • But if his papers had been rejected by those journals, he would also be accused of a conflict of interest; and had he not been an editor at PNAS, it would be said that he had no experience with editing a relatively high-visibility journal.

      Another aspect of the problem (just because no one else has mentioned it) is that the glamor rags have a constituency far beyond the pool of potential authors: e.g., scientific administrators, educators, policy-makers, and a slice of the general public. The interests of these non-authors are diverse. However, seeing that deserving researchers get a fair (measured how?) amount of public exposure for their work is probably not often a critical consideration for non-authors. It *is* a factor. I don’t necessarily disagree with you. But, integrating over the whole constituency of those journals, fairness to researchers and rewarding merit (assuming we can measure them) are not always the dominant variables — and probably shouldn’t be.

      Toby White

      December 12, 2013 at 12:57 pm

  4. Unfortunately, there are at least a few directors of big research institutions who never really did in my view important science, but only politics. They are even proud of choosing new laboratory heads just by the impact factors the applicants got, combined with the citations. Of course, nobody can really be able to understand and evaluate all the research capabilities an applicant for some prestigious full professorships has, but some institutions appear to be proud of having an administrator to get a lot of money and to make the decision on who will get the new positions. It is clear for me that there is no possibility to fill such positions with real basic scientists, but more with those in currently popular medical fields. Not many people really understood my (unpublished) work.

    Eibl

    December 11, 2013 at 12:05 pm

  5. One might add to the dataset and put in a column for “corrections”, where data have been used “inappropriately”, including use of the same data for multiple experiments describing different conditions. My own experience at PNAS was now entirely wonderful, detailed here http://ferniglab.wordpress.com/2013/02/06/responses-to-evidence-of-self-plagiarism/

    ferniglab

    December 11, 2013 at 12:49 pm

    • An interesting read and the point is taken. Perhaps this is the tip of the iceberg.

      It is not only retractions, its all the dodgy data in the glamour journals or shall I say “errors”

      Of course, as we all know too well, dodgy data is not limited to the glamour journals, but if glamour journals could take a lead, perhaps others will follow.

      A simple solution is to make science-fraud-linked grant awards a criminal offence. Not only should monies be repaid to the grant bodies in question, but fair and just punishments dispensed.

      Stewart

      December 11, 2013 at 2:04 pm

  6. Publishing papers in “elite” journals surely brought Schekman to the career point where he is now (and the Nobel Prize of course). However, whatever are his personal or professional reasons for boycott, he is right. And also, if a scientist who still needs to prove himself would publicly declare something like this, his/hers future career would be certainly damaged and ridiculed because of failure to publish in this journals. So the only way for things to be corrected in a deep-rooted system, is for someone who finally gains the power of being untouchable, to speak out! And to gain such a position you firstly need to be uplifted by the same system you are opposing. System can only be destroyed by the inside, especially the wrong one. So, Schekman got it right. And of course, there is always a way to make accomplishments relative, by using appropriate rhetorical and logical skills. But it any case – he got it right. Criticism should not be reserved only for the “week” and “pour” (scientific journals)! Good job, congrats to him!!!

    Zorana

    December 11, 2013 at 3:03 pm

    • ” System can only be destroyed by the inside, especially the wrong one. ”

      One problem is that Schekman did not propose a plan for overturning the system. He merely said he was not going to publish in high-impact journals. I don’t think even he will succeed in this. His post-docs and junior collaborators will insist on submitting to Cell, Science, and Nature as much as they can.

      It’s hard to see what to do about all this. Department Chairs, Tenure Committees, and the all-important funding agencies all look mainly at one thing: your publication record. How many papers? And Where? That’s the beginning and the end right there. And as far as I can tell, the only alternatives to this are favoritism and nepotism. I can foresee no mechanism for the objective evaluation of scientific effort, skill, or quality.

      Dan Zabetakis

      December 11, 2013 at 4:53 pm

      • “How many papers? And Where? That’s the beginning and the end right there.”

        Yes, heaven forbid they actually “read the papers”. (Or bring in an outside expert with no COIs to read/evaluate. Easier said than done in narrow fields, I suppose.)

        Allison (@DrStelling)

        December 11, 2013 at 6:05 pm

      • Double-blind peer review would not eliminate publication bias, but it would reduce it. At minimum it would force reviewers to read the papers so that they can guess who the authors are. eLife missed an opportunity by not having double-blind review.

        michaelhbriggs

        December 11, 2013 at 8:37 pm

        • A couple weeks ago, I reviewed a manuscript for a journal with a double-blind review process. A quote from the first page: “We previously reported… [citation].” Boom. Anonymity gone. Double-blind peer review is a nice idea, but rarely seems to work in practice.

          Mitch

          December 12, 2013 at 7:29 am

      • I agree. However, I cannot escape the feeling that raising the voice can be good for a start. At least, to promote public awareness about the issue. In practice, it will probably not make any significant change.

        Zorana

        December 12, 2013 at 3:46 am

    • In other words, Schekman is a Gorbachev.

      Akhlesh

      December 11, 2013 at 9:01 pm

  7. This is RW. Isn’t there a statistician in the house? I calculate that PNAS had a significantly lower retraction rate than the other three combined (p < 0.05), but not significantly lower than Science or Nature, taken individually. There's also something to be said for using the impact factor as a very rough proxy for resulting damage to science (just as it is sometimes used as a proxy for the scientific importance of a non-retracted article). In that case, PNAS is significantly better than any of the others, individually or in combination.

    What those numbers mean — assuming I haven't botched the calculation — is ultimately up to y'all, but there's no sense in tossing retraction statistics around unless we're willing to accept that they might mean something.

    Toby White

    December 11, 2013 at 4:03 pm

    • in some ways, would the even better metric be citations of retracted papers?

      say fake journal Natural Cell Science retracts ten papers, but all within a week of publication with no citations… little harm, little foul… but The Science of Nature’s Cells retracts only 1 paper, but only after 5 years and 1987629 citations.

      Then again, a paper with a splashy finding that leaves mechanisms wide open is the best way to generate citations… but not the best way to do science…

      qaq

      December 11, 2013 at 6:27 pm

    • I think it’s worth following the link to the Fang/Casadevall paper, which demonstrates the correlation between impact factor and retraction index.

      Joel

      December 13, 2013 at 1:42 pm

  8. People keep confidently asserting that Schekman would not be where he is today without having published in glamour journals. There is plenty of reason to believe that this can help, but where is the evidence for his particular case? Who is to say that he would not have made the same accomplishments and won the Nobel prize if he had published his papers in other journals?

    frank

    December 11, 2013 at 7:49 pm

    • will eLife be turned into another elite journal category – similar to Nature, Science, Cell. Looking at the editorial board – it appears to be in that direction. This is true for other subsidary journals from the above magazines/journals. Cancer Discovery is not far behind becoming one of those. So ordinary scientists will not have any chance there. It would be interesting to check how many of these elite journals have the same names as members of editorial boards?

      KK

      December 11, 2013 at 8:46 pm

  9. That’s it! I’m joining Schekman! I simply WON’T publish all of my awesome papers in Cell, Science or Nature! I refuse out of principle! (which of course, is the only reason why I won’t have any when I’m on the job market)

    qaq

    December 11, 2013 at 10:54 pm

    • On a more serious note… why not boycott PNAS(S) too? While not quite as high impact, unlike SNC, it grants some scientists different peer review options than it allows for other scientists. Selective for the masses and fodder for making unworthy post-docs from big name labs have prettier resumes.

      Finally, he calls on funding agencies to not judge CVs by their impact factor publications… but this whole thing could be ended rather quickly: If the NIH were to only allow funded researches to publish in journals owned and published by not-for-profit companies or government agencies. Boom, SCN would be done in a flash and, while some publishers would likely rise up as the high impact replacements, it would be better having the $$$ out of it…

      qaq

      December 12, 2013 at 12:24 am

      • no need to add an extra S, the name is already embarassing enough.

        yonemoto

        December 26, 2013 at 10:20 pm

        • “If the NIH were to only allow funded researches to publish in journals owned and published by not-for-profit companies or government agencies… it would be better having the $$$ out of it…”

          right. So it would be infinitely better for it to be all politics, instead. Have you seen politicians worldwide? I feel like I would prefer the most money-grubbing capitalist making decisions strictly on profit motive versus the capricious dingbats who run governments. At least you know where the one is coming from.

          yonemoto

          December 26, 2013 at 10:22 pm

  10. Schekman’s comment “It is the quality of the science, not the journal’s brand, that matters” is noble, and he’s not first to say it, but no-one to my knowledge has posited a workable and fair approach for doing so at large scale, hence popularity of the Impact Factor.

    UNOUNO

    December 12, 2013 at 7:31 am

  11. It seems Schekman is reinventing the proverbial wheel. Inevitably, eLife would end up the same as the very journals he deems to be the problem, I suspect. Couldn’t he use his stature as a Nobelist/scientist/editor to try to improve the process for publication and peer review? Open access or conventional publication are just the formats and a reflection of technology, I would think.

    YS

    December 12, 2013 at 8:56 am

    • Nature had a perfect summary of the problem in their press release:
      “The research community tends towards an over-reliance in assessing research by the journal in which it appears, or the impact factor of that journal.”

      It seems a bit hypocritical because they could do something about it if the wanted to:
      blog.pubpeer.com/?p=77

      PubPeer

      December 12, 2013 at 10:11 am

  12. If he were sincere about fixing the scientific publishing he would have stopped the PNAS contributed by articles, where the editor is the last author as well and the paper is reviewed by their friends and former students or postdocs.

    louf sun

    December 12, 2013 at 12:32 pm

  13. Sheckman can put his money where his mouth is: he can crucify his own post-docs by having them publish their great work (that they thought of, not Scheckman–does an advisor ever help?) in eLife.

    LOL!

    NMH

    December 12, 2013 at 12:42 pm

    • not quite. the recommendation letters the postdocs from his lab (or any big name PI) will get will likely compensate for the lack of NCS papers. however, it would be interesting to see if Schekman will now find it harder to attract good postdocs. my guess is he won’t, for the reason stated above.

      bertberg

      May 10, 2014 at 11:25 am

  14. I find Schekman’s comment “It is the quality of the science, not the journal’s brand, that matters” interesting. If he truly believes in this, I wonder if he would allow any of his students/Post docs to publish in any of the “predatory” journals.

    Daniel Medina

    December 12, 2013 at 2:16 pm

  15. I have found that glamor journals like Nature have become advocacy journals that promote questionable research solely because it fits their prevailing bias while refusing to publish more valid research when it contradicts their advocacy. As a conservationist I found the most egregious example to be their coverage of the spreading chytrid fungus that was causing amphibian extinctions. Education would have helped to prevent the spread of that novel diseases by researchers and the pet trade but Nature only published papers that blamed climate change and the epidemiologists had to publish elsewhere. Nature’s detrimental actions are well documented in the essay “Contrasting Good and Bad Science: Disease, Climate Change and the Case of the Golden Toad” available at http://landscapesandcycles.net/contrasting-good-and-bad-science–disease–climate.html

    Jim Steele

    December 14, 2013 at 1:23 pm

  16. He’s a little late on his boycott. I’ve been boycotting Science, Nature, and “Post-Science and Nature” for years now. Not by choice of course.

    Roscoe Takamura

    December 16, 2013 at 7:00 pm

    • I am starting to do the same. Science is not for sale anymore

      Edgar Wilde

      December 24, 2013 at 1:14 pm

  17. Well, it is about time somebody speaks up. These are magnazines and not journals. They feed off teh impact factor and want you to think that thhey are top journals. Over the past two years, I do not cite any works of from these ‘magazines’ . We should all do the same. If they are good, then they do not have to use teh impact factor metric to push the work.

    A. Elezzabi

    December 24, 2013 at 12:35 pm

  18. You are correct. Here are two articles published by Nature last year.
    Martin Schultze, et al. “ Controlling Dielectrics with the Electric Field of Light” Nature 493, 75-78 (2013) and
    Agustin Schiffrin et al. “Optical Field-Induced Current in Dielectrics”, Nature 493, 70-74 (2013)
    Jacob B Khurgin replied to these articles in his “Optically induced currents in dielectrics as a nonlinear optical effect) published in http://arxiv.org/abs/1303.3994.
    A gain, the physics is wrong as you can see by reading the reply. Expect retraction anytime!

    Edgar Wilde

    December 24, 2013 at 1:02 pm

    • I agree, this was a hot topic by the nanoplasmonics community in the SPIE SanDiego conference. Simply Nature refused to retract the work despite strong evidence from researchers in Germany (Prof. R. Claus).

      F. Levy

      January 3, 2014 at 12:23 pm

  19. Nature Cell and Science look at publications as products based on acceptable risk/fame and glory theory: High impact factor journals are willing to publish riskier, but potentially higher-impact claims ASAP – more retractions are the price for getting high-impact science out early. The more negative version of this theory is that high impact factor journals CARE more about a high impact factor than about the integrity of what they publish. Go on feed them garbage and you will get garbage.

    Edgar Wilde

    December 24, 2013 at 1:13 pm

  20. see facebook: Boycott Nature Photonics

    Edgar Wilde

    December 24, 2013 at 1:36 pm

  21. Everyone says he’s a nice guy, but I reported problems with papers to PNAS during his tenure and problems with eLife papers during his time there already. I have an academic email address, they are papers in my field, yet I receive not a single reply. He was cc’ed on all second attempts. Nothing.

    eLife is the new glamour mag

    December 27, 2013 at 1:49 pm

  22. Dr Randy Schekman is 100% right with his reasoning that addresses how magazines such as Nature Magazine undertake their business with regard to highly important work which in many ways they suppress. In this respect Nature Magazine we have found is basically a pawn in the game of big business. They do their biding when the bottom-line is threatened was our finding. The fallacy that a vaccine will come in time to prevent the world’s future most deadly pandemic in terms of Bird Flu et al is just a single example of their power over such magazines such as ‘Nature’. They stop the truth emerging in terms of alternative strategies and scientific solutions when that work affects n particular the big pharma’s vast profit making machine.

    In this respect we have first hand experience of how Nature Magazine operates behind closed doors. A few of countless articles that may be of interest and mind opening are as follows –

    http://worldinnovationfoundation.blogspot.co.uk/2013/12/possibly-most-important-keynote-speech.html

    http://worldinnovationfoundation.blogspot.co.uk/2013/12/vaccines-will-never-save-us-from-deadly.html

    http://worldinnovationfoundation.blogspot.co.uk/2013/12/global-pharmaceutical-giants-have-made.html

    Dr David Hill
    World Innovation Foundation

    bettyhill

    December 30, 2013 at 12:46 pm

  23. Science Nature and Cell have already boycotted me and I don’t have to boycott them. That explains why I don’t publish there. Any chance of me winning a Nobel?

    Glenn Dobson

    April 22, 2014 at 11:58 pm

    • Why do you want to publish in these magazines?

      J. Cooper

      May 9, 2014 at 5:38 pm

  24. The physics behind these papers is wrong. I challenge anyone who can reproduce the results from these papers
    (1) Seungchul Kim et al., “High-harmonic generation by resonant plasmon field enhancement” Nature 453, Nature 453, 757-760 (5 June 2008)
    (2) In-Yong Park et al “Plasmonic generation of ultrashort extreme-ultraviolet light pulses”, Nature Photonics 5, 677–681 (2011).

    Nature watch

    December 24, 2013 at 12:52 pm

  25. Way to go Nature, screw science again and again. Bring us to the dark ages . You can tell us what to research , what to read and what to publish.

    stop NPG now

    January 2, 2014 at 12:44 pm

  26. The goal of any publication is to share knowledge . If nature magazine does not want to the job for you because they cannot make money on their impact factor and report profit, then, he’ll with them. Do not even cite their published work and see who will come begging for your work. Both can play this game. The loser is the one who wants to make money.

    J. Cooper

    May 9, 2014 at 5:43 pm


We welcome comments. Please read our comments policy at http://retractionwatch.wordpress.com/the-retraction-watch-faq/ and leave your comment below.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 34,829 other followers

%d bloggers like this: