Rheumatology explains what went wrong with meta-analysis

Earlier this week we wrote about how Rheumatology, the official journal of the British Society for Rheumatology, was retracting an error-beset meta-analysis on the association between lupus and cervical cancer.

As the notice explained:

Errors were identified in the publication involving several parts of the study including extraction of data, statistical analyses and interpretation of results.

Cervical neoplasia should have been interpreted as abnormal pap smears.

Unfortunately, such extensive changes cannot be addressed in a corrigendum and warrant a new manuscript. It is important to note that this is not considered to be scientific misconduct, but rather an honest error by authors.

We’d posed a few questions for the journal about the notice, to which we have received replies, on behalf of the editor, Robert Moots.

Q. Who brought these errors to your attention and when?

A. Two readers submitted a letter pointing out potential errors in the paper following its publication online. One of the readers pointed out an incorrect citation and observed that cervical neoplasia had been incorrectly classified and that the paper actually looked at abnormal pap smears.

Q. Why were they not detected during the review process? Did the authors not make their data available?

A. The fact that the errors were not identified by two experienced independent peer reviewers underlines the fact that peer review is not a perfect system. We try to ensure an appropriately rigorous peer review, but, in this case, the errors were not identified. We did not ask for the original data to be looked at. The original manuscript was assessed by two experienced independent reviewers, who had rated this paper quite highly, recommending publication. They did not suggest there was a reason to suspect the data.

When the errors were discovered we considered the selection of papers for the study to be suboptimal and took advice from an Associate Editor experienced in meta-analysis and from our Publisher and adhering to COPE guidelines we communicated our reservations to the authors.

Q. The part in the notice about honest error — did that come from you or [the corresponding author, Hongli] Liu?

A. We added honest error as we believed that to be the case. The authors understandably expressed concerns that a retraction tended to always suggest research misconduct or questions integrity. As they freely admitted that mistakes had been made with their first meta-analyses, we felt that the journal had acted appropriately and responsibly.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.