About these ads

Retraction Watch

Tracking retractions as a window into the scientific process

Why write a blog about retractions?

with 23 comments

Photo courtesy scriptingnews via flickr http://www.flickr.com/photos/scriptingnews/

Post by Ivan Oransky and Adam Marcus

The unfolding drama of Anil Potti — a Duke researcher who posed as a Rhodes Scholar and appears to have invented key statistical analyses in a study of how breast cancer responds to chemotherapy — has sent ripples of angst through the cancer community. Potti’s antics prompted editors of The Lancet Oncology to issue an “expression of concern” — a Britishism that might be better expressed as “Holy Shit!” — about the validity of a 2007 paper in their journal by Potti and others.

Unlike newspapers, which strive for celerity as much as accuracy, science journals have the luxury of time. Thorough vetting, through editorial boards, peer reviewers and other filters, is the coin of the realm.

And yet mistakes happen. Sometimes these slips are merely technical, requiring nothing more than an erratum notice calling attention to a backwards figure or an incorrect address for reprints. Less often but far more important are the times when the blunders require that an entire article be pulled. For a glossary of the spectrum between erratum and retraction — including expression of concern — see this piece, commissioned by one of us, Ivan, while he was at The Scientist.

Retractions are born of many mothers. Fraud is the most titillating reason, and mercifully the most rare, but when it happens the results can be devastating. Consider the case of Scott Reuben, a prodigiously dishonest anesthesiologist whose fabrications led to the retraction of more than a score of papers and deeply rattled an entire medical specialty. (One of us, Adam, broke that story.)

So why write a blog on retractions?

First, science takes justifiable pride in the fact that it is self-correcting — most of the time. Usually, that just means more or better data, not fraud or mistakes that would require a retraction. But when a retraction is necessary, how long does that self-correction take? The Wakefield retraction, for example, was issued 12 years after the original study, and six years after serious questions had been raised publicly by journalist Andrew Brian Deer. (Thanks to commenter Tutak for letting us know about this error.) Retractions are therefore a window into the scientific process.

Second, retractions are not often well-publicized. Sure, there are the high-profile cases such as Reuben’s and Wakefield’s. But most retractions live in obscurity in Medline and other databases. That means those who funded the retracted research — often taxpayers — aren’t particularly likely to find out about them. Nor are investors always likely to hear about retractions on basic science papers whose findings may have formed the basis for companies into which they pour dollars. So we hope this blog will form an informal repository for the retractions we find, and might even spur the creation of a retraction database such as the one called for here by K.M Korpela.

Third, they’re often the clues to great stories about fraud or other malfeasance, as Adam learned when he chased down the Reuben story. The reverse can also be true. The Cancer Letter’s expose of Potti and his fake Rhodes Scholarship is what led his co-authors to remind The Lancet Oncology of their concerns, and then the editors to issue their expression of concern. And they can even lead to lawsuits for damaged reputations. If highlighting retractions will give journalists more tools to uncover fraud and misuse of funds, we’re happy to help. And if those stories are appropriate for our respective news outlets, you’ll only read about them on Retraction Watch once we’ve covered them there.

Finally, we’re interested in whether journals are consistent. How long do they wait before printing a retraction? What requires one? How much of a public announcement, if any, do they make? Does a journal with a low rate of retractions have a better peer review and editing process, or is it just sweeping more mistakes under the rug?

These are the sorts of things we’ll cover when we write about a particular retraction, and we hope they’ll form the basis of larger discussions of the obligations of journals. The two of us — both with experience covering science and medicine for the consumer as well as trade press — seem to come across these issues often. The experience one of us, Ivan, has had with Embargo Watch suggests that a blog is a great forum for such conversations. While any particular Retraction Watch post will only carry one of our bylines, we will both contribute to all posts.

A few researchers have studied the issues we plan to raise here. For example, a 2008 paper in the Journal of Medical Ethics found that “the rate of retractions remains low but is increasing” and that:

Although retractions are on average occurring sooner after publication than in the past, citation analysis shows that they are not being recognised by subsequent users of the work. Findings suggest that editors and institutional officials are taking more responsibility for correcting the scientific record but that reasons published in the retraction notice are not always reliable. More aggressive means of notification to the scientific community appear to be necessary.

Those sound like great arguments for Retraction Watch.

About these ads

23 Responses

Subscribe to comments with RSS.

  1. This sounds excellent and I look forward to the posts. Although when Ivan first mentioned this, I assumed that you would be covering retractions/corrections of science-reporting rather than science itself. The latter is probably more interesting but any thoughts on covering the former too?

    Ed Yong

    August 3, 2010 at 10:16 am

    • Thanks Ed! We will certainly keep our eye on retractions and corrections of science reporting as well. They are more rare, but just as interesting. Our core interest will be retractions of the science itself.

      ivanoransky

      August 3, 2010 at 12:22 pm

    • Covering the oopsies of science reporting is more difficult/intensive because, as we all know, there’s not the same level of prominence in disclosing such errors compared to that of primary scientific journals.

      One thought, though: smart version-comparing robots could help with such an endeavor. In particular, I’m pondering Scott Rosenberg’s reporting on changes to a story that Politico (yeah, not science but still) tried to sneak in: http://www.wordyard.com/2010/07/21/politico-slate-and-story-versioning/ Seems to me that someone with the right know-how and resources could (easily?) pull this off. Perhaps interface the “hey this story was changed” data with something like MediaBugs.org (if it ever gains mainstream popularity) for crowd-sourced analysis.

      Dave Mosher

      August 4, 2010 at 10:26 am

  2. I applaud you both for starting this important blog. Hopefully your putting what you find in the public domain will lead journal editors and the scientific community to create the database K.M. Korpela called for. Meanwhile your blog can serve as an informal collection of retractions. Since this is a blog, you may, along the way write suggestions for reforms. Ivan, I am amazed at how you find time for these new ventures, but I sure am glad that you do! Congratulations to you both for thinking of this and for actually doing it. I know your posts will be well read.

    Mary Knudson

    August 3, 2010 at 6:40 pm

  3. Should be fascinating. Librarians are interested, too, in how the retraction shows up in the online version of the journal (is the article still there but marked clearly? is the article gone? is there no marking) and in research databases.

    Christina Pikas

    August 3, 2010 at 7:55 pm

  4. Sorry to be utterly pedantic, but I don’t think ‘expression of concern’ is the least bit a Britishism.

    See, for instance, these ‘expressions of editorial concern’ in Science: 2005 DOI: 10.1126/science.1124185 and 2009 DOI: 10.1126/science.1186078

    If they’d said ‘a concern about a spot of bother over a bit of a rum article’, I’d be inclined to see the Britishismness more!

    A Brit

    August 9, 2010 at 7:10 am

  5. Go to Elsevier’s ScienceDirect site and search for “retracted.” Most results are to articles that have been retracted rather than legit articles that use the keyword. The folks at Elsevier are scrupulous about noting who retracted the article, and why. You can also find “retracted” articles from Wiley and Springer.

    Denise Bennett

    September 23, 2010 at 11:18 am

  6. Hi,

    Also check out http://econjwatch.org/ . They do similiar things for economics. In economics papers are not often retracted but “mistakes” are sometimes reported on Econ Journal Watch and the authors are invited to respond.

    Cheers

    Elliot

    Elliot

    September 24, 2010 at 8:10 pm

  7. This is an interesting piece but please correct the name of the journalist who did sterling investigative work on the Wakefield paper (and has been under huge personal attacks for it by the pro-vaccine lobby) – it’s Brian Deer, not Andrew.

    Tutak

    October 7, 2010 at 4:48 am

  8. Boy, I betcha Google’s cache attic could easily do the equivalent of Word’s “Compare Versions.” Wouldn’t that be interesting!

    e-Patient Dave

    November 27, 2010 at 6:57 pm

    • interesting thought Dave! moreover, I am a proponent of open source/publication medical papers like the JMIR. When it is open to the public, it will be easier to track fraud

      epatientgr

      April 8, 2011 at 11:07 am

  9. I love the RW blog.
    It is such important work. Another case study (where I wrote about “retraction ethics” illustrates why:

    Eysenbach G, Kummervold PE
    “Is Cybermedicine Killing You?” – The Story of a Cochrane Disaster
    J Med Internet Res 2005;7(2):e21
    URL: http://www.jmir.org/2005/2/e21/
    doi: 10.2196/jmir.7.2.e21

    Gunther Eysenbach

    December 3, 2010 at 10:24 am

  10. Great blog!

    Here’s a recent case to watch:

    http://www.nature.com/news/2011/110107/full/news.2011.703.html

    Michael

    January 20, 2011 at 12:56 pm

    • Will do — thanks!

      ivanoransky

      January 20, 2011 at 1:08 pm

    • This case emphasizes how necessary it is for investigations of fraud claims to be carried out by people completely independent of the institution where the individual works, independent of the granting agencies and government bodies that supported the individual and independent of the journals that published the work.

      Only then can we be sure of the “results” of such investigations.

      sceptic

      January 29, 2011 at 5:07 pm

  11. Does anyone know something similar to this exists for chemistry?

    As an aside, few retractions can beat “Manuscript withdrawn for scientific reasons” in the ‘being so vague as to tell you nothing category’, see here for an example: http://pubs.acs.org/doi/abs/10.1021/ja904224y

    In this case, I can only assume ‘withdrawn for scientific reasons’ translates to ‘they didn’t bother to check a general chemistry textbook before publishing this’, ‘impossible’, and ‘the referee’s have egg on their face for letting this pass’.

    atompusher

    February 4, 2011 at 3:58 pm

  12. I found this blog while exploring the breadth of plagiarism within academia. I have a unique insight into just how widespread plagiarism is considering I wrote for the custom writing industry for quite a few years and eventually started my own company in this industry. I actually tried to offer my services to several major universities in terms of aiding them in developing strategies to identify certain characteristics of papers that students turned in who had not actually written those papers but who had, in fact, utilized a custom writing service. As I quickly discovered, colleges and universities tend to beat their drums when confronted with a particular instance of plagiarism by one of their professors or students but when given the opportunity to address the systemic pervasiveness of plagiarism they tend to bury their heads in the sand. I have published an expose of this industry that you might find interesting: “Putting America Through College One Paper at a Time: The Outsourcing of Our American Intellect.” It is available on Amazon for a pittance because I have avoided the traditional publishing route. If nothing else, it will give you an idea of just how widespread and influential that the custom writing service has become and how most colleges and universities just ignore it because to address it appropriately would inhibit enrolments, graduation rates and, ultimately, international student fees as well.

    Mark

    August 21, 2011 at 12:49 pm

  13. Just found U. Excellent job

    Brenda

    September 14, 2012 at 12:06 am

  14. Ivan, have you seen Dr. John Krueger’s article in the Office of Research Integrity Newsletter (December 2012) on “what can retractions tell us?” It might be of interest to your readers. Best, Alan Price

    http://ori.hhs.gov/images/ddblock/dec_vol21_no1.pdf

    Alan Price

    February 25, 2013 at 8:58 pm

  15. Fixed — thanks!

    ivanoransky

    January 15, 2011 at 11:52 am


We welcome comments. Please read our comments policy at http://retractionwatch.wordpress.com/the-retraction-watch-faq/ and leave your comment below.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 33,826 other followers

%d bloggers like this: