In precedent break, BMJ explains why it rejected controversial “weekend effect” paper

After the reviewer of a rejected paper was publicly outed, the BMJ has taken the unusual step of explaining why it chose not to publish the paper.

The paper — eventually published in another journal — raised hackles for suggesting that there is no “weekend effect,” or a higher mortality rate in hospitals on Saturday and Sunday. This caught the attention of UK policy makers, who have proposed changing policies to compensate for any supposed “weekend effect.”

Amidst the heated discussion about the research, one of the reviewers was identified, along with suggestions that he may have been conflicted because he had published a study showing the opposite finding. Yesterday, the BMJ posted a blog explaining that it was the editors — and not one sole reviewer — who decided to reject the paper:

The idea of an all-powerful reviewer who can control what we publish is impossible to reconcile with the way our process actually works. During our weekly manuscript meeting, a team of research editors and a consulting statistician decide which papers to publish. We take reviewer comments into account, but also consider how relevant the paper will be for our readers, the quality of the study methods, its novelty and timeliness, and how it compares to other papers we have to consider.

The decision to reject the Meacock paper was made by The BMJ editors and not by the four external peer reviewers who saw the original paper and a subsequent revision.

In another unusual move, the blog — signed by Elizabeth Loder, Head of research, José G. Merino, US clinical research editor, and Rubin Minhas, research editor — also published all the reviewer comments about the paper.

Once readers see all the information, some may believe the journal made a mistake in rejecting the paper, the editors concede:

In agreeing to our request three of the reviewers, including [identified reviewer Nicholas] Freemantle, told us, unprompted, that they had hoped we would publish the paper and were surprised that we didn’t. We suspect that many people who read the reviewer comments may also think we made a mistake.

“Higher mortality rates amongst emergency patients admitted to hospital at weekends reflect a lower probability of admission” was ultimately published in the Journal of Health Services Research & Policy.

The editors add that they believe Freemantle — based at University College London — did not have a major conflict because he had published in the same area. Indeed, the opposite is true:

We agree with those who suggest that he might have included in his competing interests declaration a statement that he had published a paper on the topic in question that reached a different conclusion. But this made no material difference to the review process; indeed, he was selected precisely for this expertise because we seek opinions from people who are active in the relevant field.

Almost everyone who is invited to be a peer reviewer has published papers on related topics, and frequently they have reached different conclusions from the authors whose paper they review. We do not think this disqualifies someone as a reviewer. In fact, such reviews help us identify the criticism a paper might face if it is published.

The journal explains that the incident hasn’t diminished its faith in open peer review:

We believe that open review promotes constructive conversation while still providing comments that help authors improve their manuscript and editors make a decision about publication. The system gives credit to reviewers while making them accountable for their comments. Reviewers for The BMJ agree to have their signed comments posted if a paper is published, but not otherwise.

One argument against open peer review has always been the potential for reprisals against reviewers. Since we instituted open review in 1999 we have heard of few instances of retaliation. This incident is one of them, and we are grateful to Professor Freemantle for his resilience and professionalism in the face of the unfounded criticism he has received on social media. We continue to think that open peer review is a more responsible system than any alternative.

However, the editors note that they are going to clarify the instructions to authors:

Our instructions to reviewers make it clear that papers under review are confidential and should never be disclosed to anybody else. We realise that our instructions to authors have not been similarly explicit about confidentiality and the considerate behaviour we expect towards reviewers.

Despite the sharp focus paid to the paper and the topic, in light of heated discussions in the UK about whether to adjust policies to any supposed “weekend effect” — such as creating a new contract for junior doctors, prompting a recent strike — the incident at the BMJ over this particular paper ended well, the editors add:

Readers might like to know that all of this has ended amicably: the authors of the Meacock paper have apologised for releasing information about the previous review and the identity of one of the reviewers. When we contacted them, we learned they had already been in touch with Professor Freemantle, who had graciously accepted the apology they offered. Let’s hope things end as well with the junior doctors’ dispute.

Update 5/18/16 9:24 p.m. eastern: We’ve heard from Elizabeth Loder, acting head of research at the BMJ:

Thanks for your interest in this matter. We broke with precedent in this case because Professor Freemantle, the reviewer whose identity was revealed, was the target of unpleasant accusations about his role as a peer reviewer of the paper. This was only possible because of our open peer review process, in which authors and reviewers are known to each other. We thought it was important to make it clear to everyone that he was just one of four reviewers, that his review had been fair and balanced, and that the decision to reject the Meacock paper was made by the research editors and not by the reviewers.

It is interesting you say this might suggest that peer review should be even more transparent. Some might argue the opposite: that this case should prompt a return to the secretive world of blinded peer review. With blinded peer review systems reprisals like this are unlikely to occur because authors aren’t told who reviewed their paper.

The strongest case for the even more transparent process you describe is that the effort put into producing an excellent peer review report should not be wasted if the paper is rejected. We already encourage authors who submit papers to The BMJ to attach reviews from any journals that have previously rejected the paper, along with an explanation of their response to these reviews. In practice, few do. We often learn through other channels that a paper has been rejected elsewhere before it was sent to us. I believe many authors value the idea that they will get a fresh chance at a different journal. Requiring that all reviews be attached, rather than suggesting it as we do now, might have the paradoxical effect of encouraging even more reprisals and resentment against reviewers. Authors might then feel that a bad peer review would follow them forever.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.