Anonymous blog comment suggests lack of confidentiality in peer review — and plays role in a new paper
The new paper is about another paper, a December 2012 study, “Fractionating Human Intelligence,” published in Neuron by Adam Hampshire and colleagues in December 2012. The Neuron study has been cited 16 times, according to Thomson Scientific’s Web of Knowledge.
Richard Haier and colleagues write in Intelligence that
The main purpose of this report is to invite Hampshire and colleagues to respond to our specific scientific concerns that aim to clarify their work and contribute a constructive discussion about the meaning of their findings.
But it is their discussion of what happened during the peer review process and after the study was published that caught our attention. On July 13, 2012, Haier was asked to write a Preview to the paper, to be published along with the study in Neuron:
RH found many aspects of the paper quite difficult to understand and, more troubling, he worried that some main conclusions could be based on erroneous application and interpretation of factor analysis.
He had been given a short deadline, so he shared the manuscript with four colleagues — now co-authors of the Intelligence paper — “with considerable expertise in brain imaging and psychometrics, especially factor analysis.” They shared his concerns, and wrote the Preview together, submitting it on August 6:
… in a cover letter we informed Neuron that our concerns were so serious, that had any of us been original reviewers, we would not have recommended acceptance without major clarifications.
The journal wrote back the next day, and asked Haier and his colleagues to submit a more detailed critique to which Hampshire and his colleagues could respond. Haier et al did that on August 16, but on October 31, Neuron
informed us that publication of the Hampshire et al. manuscript would go forward with some minor changes. We were also informed that, after considerable internal discussion, the editorial board had decided that our Preview would not be published; no reason was given. We objected and asked if we could submit a modified Preview based on the modified manuscript (which was not shared with us). Neuron declined. One editor asked to have a confidential phone call with RH and that call took place on December 2nd. RH respects that confidentiality and can only say that he found the editorial process and decision-making hard to understand.
When Haier saw an embargoed copy of the paper on December 18, he and his colleagues
were surprised to see that the final version did not address our concerns in any substantial way.
The paper — which was press-released by Hampshire et al’s university, The University of Western Ontario — earned some press coverage, including from Neuroskeptic. And it was in the comments on Neuroskeptic’s post where things got interesting:
There were several comments that suggested knowledge of our unpublished Preview. We determined that a graduate student had overheard a relevant discussion and decided to comment on the blog anonymously without our knowledge. One commenter on the blog responded to some of the scientific critiques with a lengthy detailed technical argument (see Appendix D for the full comment). This detailed comment also concluded in part with these sentences: “Finally, a critical comment was submitted to Neuron however, there was no ‘conspiracy’. It was decided, based on feedback from an independent reviewer, that the author of the comment was heavily biased and that the criticisms raised were lacking in substance. Also, the authors of the article demonstrated that they were both willing and able to address all of those criticisms point by point if the journal chose to publish them.”
Obviously someone with inside knowledge of the review process wrote this comment. We sent this comment to Neuron and asked if it were true that our 20 detailed concerns were communicated only to one of the original reviewers who then determined our concerns did not have substance and were biased. We also requested that Neuron provide any written responses to our 20 points made by the original reviewers or the authors. Neuron replied that discussions were all by phone and there were no written responses. Neuron would not confirm that only one original reviewer determined that our concerns were biased or that they had not required a point-by-point response. Finally, we asked Neuron if we could submit comments on the Hampshire et al. paper under the category of “Viewpoint” or “Perspective” and allow the authors to respond. We felt that this would be constructive and educational. Neuron declined.
Worth noting: Both Neuron and Intelligence are published by Elsevier.
The whole situation seems almost like open peer review, only without the transparency.
Hat tip: Neil Martin
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post.