Over the summer, while searching for some studies and evidence for various treatments, my wife, a television writer and producer, noticed something she thought unusual enough to flag for me. The titles of a number of Cochrane Library reviews started with “WITHDRAWN.”
The Cochrane Library is the world’s leading publisher of systematic reviews, which gather all of the high-quality evidence on a given subject and offer a rigorous analysis of whether a given test or treatment works. It’s an invaluable resource. (Shameless plug: Join the Association of Health Care Journalists, where I’m treasurer, and access to the $285-per-year Cochrane subscription is included.)
Retraction Watch was curious about what “WITHDRAWN” meant, since “withdrawal” is often used synonymously with retraction. Cochrane updates its reviews regularly, as new evidence surfaces, of course. But these abstracts didn’t say anything about new reviews.
We asked Jen Beal, who handles media relations for Wiley, the Cochrane Library’s publisher. She responded:
I’ve spoken to the editor in chief who explained that these aren’t updates, they are reviews which have been withdrawn because they are considered out of date – i.e. it’s felt that there is more evidence out there which should be included.
For example, Beal, went on, the review from one Medline citation I sent her of a homeopathic remedy for flu “says that a new author team has taken on the task of getting it up to date .”
In other words, even before there’s an update ready to publish — at which point you could argue the Library would “retract” the earlier review — the publishers flag the fact that a review is out of date.
It’s not surprising that the Cochrane Library is this transparent; after all, they stand for the most rigorous standards of evidence. And critics might say that by taking reviews out of circulation — they’re unavailable while they are in “WITHDRAWN” status — doctors and patients have less information rather than more. But the original studies the review was based on are still available, so it’s a question of whether an out-of-date review is better than no review at all. Clearly, Cochrane thinks not.
In any case, there’s a nice lesson here about how journals could do a better job of letting readers know that a given study has been eclipsed by new evidence. Withdrawing such primary spapers isn’t necessarily reasonable or appropriate. Retractions aren’t in order either, unless there are flaws in a paper, or its findings can’t be replicated. The scientific process, after all, expects such self-correction as new evidences comes to light. But somehow noting that new studies have rendered earlier ones less relevant would be great.