When an independent replication isn’t really independent

My laboratory at Boston Children’s Hospital and Harvard Medical School studies genetic diseases that affect the skeletal system.  We became interested in the protein osteocalcin after Gerard Karsenty at Columbia University reported in several papers using knockout mice – mice lacking the genes which produce osteocalcin – that osteocalcin is a bone-derived hormone that affects glucose metabolism, insulin production, male fertility, muscle mass, and cognition.  If osteocalcin functions similarly in humans, then osteocalcin becomes an exciting and clinically important protein. 

To independently confirm these findings, we created our own osteocalcin knockout mouse strain. We examined glucose metabolism and male fertility in our mice and found none of the effects reported by Karsenty and colleagues; we reported our findings in May 2020.  A group in Japan created a third osteocalcin knockout mouse strain which also failed to confirm Karsenty and colleagues’ findings.  

In earlier years my laboratory also could not independently confirm other results reported by the Karsenty group: a paper I co-authored in 2011 found no evidence of the Wnt co-receptor LRP5 affecting blood serotonin levels, contrary to what Karsenty’s lab published.

As I puzzled over our inability to independently confirm Karsenty’s osteocalcin findings, I contacted Thomas Clemens at the John Hopkins Medical Institute, whose lab appeared to have more success validating Karsenty’s research. In fact, Clemens was the corresponding author on a 2010 paper published in Cell that appeared to confirm one of Karsenty’s major findings: osteocalcin’s hormonal effect on blood glucose.  When I wrote to Clemens in June 2020 asking for his detailed experimental methods, I made a curious discovery which increased my concern about Karsenty’s published findings. 

I specifically asked about the experiment reported in Figure 5 of Clemens’ paper, which showed that osteocalcin infusions reduced hyperglycemia in insulin receptor (IR) knockout mice.  Clemens’ initial email reply pointed out that the experiments were done about a decade earlier. He wrote that the detailed methods might be difficult to find in “old notebooks” and he would “speak with the post-doc who was working on this.”

Subsequently, he wrote to me to indicate that the experiment in Figure 5 was not actually performed by him, any of his co-authors, or any individual acknowledged in his 2010 paper. Instead, the data I inquired about was provided by the Karsenty lab: 

Matt,

Sorry for the delay in getting back to you.

I reviewed our study of osteocalcin infusion in the IR KO mice (Figure 5, Cell 142, 309–319).  This study was done in collaboration with Gerard’s group who made and validated the recombinant peptide.  I suggest you speak with him about the details if you need them.

Regards,

Tom

and

Matt,

Apologies for not being clear on my last email.  This study was done in Gerard’s lab by Mathieu Ferron (copied above).  I am sorry I cannot provide further details.

Regards,

Tom

Therefore, Clemens’ paper did not independently confirm Karsenty’s findings, but instead reported data obtained from the Karsenty lab.  No mention of this collaboration was made in Clemens’ paper or in Karsenty’s own article published in the very same issue of Cell.  As a reader who had seen no overlapping individuals included as co-authors or listed in the acknowledgements, I had inferred the work was done independently, rather than collaboratively. 

An accompanying Cell editorial bolstered my impression by describing the two papers as coming from “independent laboratories,” with Clemens’ group performing the osteocalcin infusions.  A review article the following year, co-authored by Clemens and Karsenty, also stated their work had been performed independently and, in 2016, Clemens published a review article stating his osteocalcin findings were independent of Karsenty’s.

In my nearly 40 years as a scientist, I have not previously encountered a situation where an investigator published consequential experiments while failing to acknowledge a collaborator responsible for part of the work and without that collaborator apparently raising any objection. 

Since both scientists and funding agencies take claims of independent validation of research into consideration when designing experiments and providing funding, I shared my concern that the authors had not been sufficiently transparent about their collaboration with the U.S. Office of Research Integrity (ORI), the Johns Hopkins Medical Institute’s Office of Policy Compliance, Columbia University’s Vice President for Research Compliance, Training and Policy, and the editor of Cell.

The email response I received from the ORI – which has oversight of research integrity in work funded by the NIH, which has supported both Clemens and Karsenty – indicated it does not consider an investigator publishing another group’s data without attribution to be a problem if the group that provided the data does not object.  The email I received from the Johns Hopkins Medical Institute appeared to consider the matter an authorship dispute rather than scientific impropriety.  I never received a response from Columbia University. 

My correspondence with Cell appears to have prompted a correction by Clemens earlier this year.  But that correction does not clarify the lack of independent validation.  In his correction, Clemens wrote that Mathieu Ferron “assisted in performing the osteocalcin infusion studies described in Figure 5.” However, Clemens did not disclose that these studies were performed entirely in Karsenty’s lab.  

Nor does Clemens’ correction disclose that Karsenty was at the time of publication, a founder and member of the scientific advisory board of Escoublac, Inc, a company whose business was based on Karsenty’s claim that osteocalcin plays a role in energy metabolism and related disorders. 

A more complete correction to the Clemens’ Cell paper: should include Ferron and Karsenty as collaborators and co-authors; make it clear that Clemens’ lab did not independently show that osteocalcin affects blood glucose levels; disclose collaborator Karsenty’s conflict of interest.  

The Clemens’ correction should also indicate, consistent with standard disclosures, whether the Institutional Animal Care and Use Committee at Columbia University approved the importation of the insulin receptor knockout animals from Clemens’ lab, if this indeed occurred, and the infusion of osteocalcin into those animals. If the importation of Clemens’ mice did not occur, then Clemens’ paper would appear to be misrepresenting the actual mouse strain on which the data in Figure 5 were acquired. That would further undermine scientific confidence in the robustness of Karsenty’s findings across mouse strains.

A larger issue this matter raises for science is whether authorship/attribution guidelines themselves should require, to avoid any doubt, that all material sources of data in papers are disclosed.  This would avoid reviewers, readers, and funders wrongly inferring results had been independently validated when they were not.

Matt Warman is director of the Orthopedic Research Laboratories at Boston Children’s Hospital and a Professor of Genetics and Orthopedic Surgery at Harvard Medical School. He wishes to thank Eugenie Reich, a former science journalist and author of Plastic Fantastic, an account of the Jan Hendrik Schön fraud case, and an attorney at Greene LLP in Boston, for helpful input.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].