PNAS retracts two papers on osmolytes after researchers discover crucial measurement errors

A good carpenter never blames his tools. But for scientists, sometimes machines do go bad–with disastrous results. Consider the following:

The Proceedings of the National Academy of Sciences has retracted two papers by researchers in the United Kingdom and the United States after the scientists learned that their results were based largely on a problem with their highly sensitive instruments.

Both notices appeared this month in the journal. Here’s the first, for a study cited 44 times, according to Thomson Scientific Web of Knowledge:

Retraction for “Solvent molecules bridge the mechanical un-folding transition state of a protein,” by Lorna Dougan, Gang Feng, Hui Lu, and Julio M. Fernandez, which appeared in issue 9, March 4, 2008, of Proc Natl Acad Sci USA (105:3185–3190; first published February 27, 2008; 10.1073/pnas.0706075105). The authors note the following: “We wish to retract our article because interpretation of the experimental data was based on incorrect calibration of the apparatus in viscous solutions and there is no basis for the major conclusions of the study on the structure of the transition state for mechanical unfolding.”

The second paper, “Probing osmolyte participation in the unfolding transition state of a protein,” has been cited twice and has an identical notice:

Retraction for “Probing osmolyte participation in the unfolding transition state of a protein,” by Lorna Dougan, Georgi Z. Genchev, Hui Lu, and Julio M. Fernandez, which appeared in issue 24, June 14, 2011, of Proc Natl Acad Sci USA (108:9759– 9764; first published May 25, 2011; 10.1073/pnas.1101934108). The authors note the following: “We wish to retract our article because interpretation of the experimental data was based on incorrect calibration of the apparatus in viscous solutions and there is no basis for the major conclusions of the study on the structure of the transition state for mechanical unfolding.”

Funding for the study came from UK Engineering and Physical Sciences Research Council, the NIH and the China National Basic Research Program. The first author of both papers, Lorna Dougan, was a post-doc in the Fernandez lab in the late 2000s, before moving to the University of Leeds, where she runs her own research group.  Her CV shows three papers in PNAS (all published in 2009), but not the two retracted articles.

Ann Griswold, a PNAS spokesperson told us that:

These retractions were fairly straightforward. The author contacted us and expressed sincere concern about the inadvertent experimental errors in the papers. A member of the editorial board reviewed the concerns and approved the authors’ retraction statement.

That’s pretty much what we learned from Fernandez, who walked us through the problems in detail (we emailed Dougan for comment but have yet to hear back from her).

Fernandez said the lab conducted the first experiments in 2006, at a time when the instruments for measuring forces on proteins in the presence of osmolytes — chemicals that help cells regulate their volume and that can stabilize proteins — using a technique he helped pioneer called force-clamp spectroscopy that allows scientists to determine key features of a folding protein. By dipping a glass cantilever into a solution containing proteins, Fernandez’ group could measure how the probe deflected, giving them an idea about the kinetics of protein unfolding.  Fernandez’ group used that information to determine both the mechanical stability of the proteins they wanted to study and the distance to the transition state of for unfolding, an important feature of the free energy.

The problem, however, was that the early cantilevers were less reliable than today’s, Fernandez said. Their readings could vary significantly under the more challenging conditions of the highly viscous solutions that contained the osmolytes. As a result, he said,

 We weren’t sure if the variations were problems with the calibration or the manufacturing.

The answer appears to be a bit of both.

This summer, after publishing the second paper in PNAS, Fernandez and his colleagues found themselves struggling to replicate their earlier results using different proteins. Unfortunately, he said, although the data on the stability of the proteins seemed robust, other data looked shaky.

Dougan returned to New York City, where she and Fernandez reviewed her raw data for the 2008 paper — it had all been preserved, Fernandez said.  They quickly realized that the findings were largely, although not entirely, unreliable. The reason, they realized was that

When you calibrate a cantilever in a viscous solution, you underestimate the spring constant and increase the slope of the force dependency [a way to assess folding].

However, Fernandez initially considered asking for corrections, because the other half of the study, showing the effects of osmolytes on stability, held up.

In a sense, the principal conclusion is still valid. The solvents participate in the mechanical transition state of the structure — the unfolding of the protein — and you can greatly slow that down in the presence of these viscous osmolytes.   The distance from the native to the transition state, that’s the part that we got wrong in the paper; it does not change.

But in the end, after “agonizing” over the decision, he and his group agreed that retractions would be a better route.

I felt it was better to make a clean retraction and then publish a paper later on with the correct measurements.

The current generation of cantilevers are much more stable, Fernandez added, and now that his group knows how to deal with the challenges posed by highly viscous solutions, they’re better able to calibrate the instruments.

George Somero, of Stanford, refereed the 2008 article and was surprised to hear that it had been retracted:

That’s one of the nightmares of experimental science.

Somero said it’s hard to know what went wrong with the machine. Some are calibrated at the factory, others by the researches themselves or lab technicians. Somero added that detecting an instrument problem in raw data is extremely difficult, if not impossible.

If a reviewer is really familiar with an instrument, you could offer a caveat about calibration. But when you’re analyzing data sets, you assume [that the researchers have addressed those issues].

Leeds made hay out of the 2011 PNAS article, issuing a lengthy press release — “Scientists a step closer to understanding ‘natural antifreeze’ molecules” —  in which Dougan was quoted extensively about the significance of her group’s findings on osmolytes:

If you put something like human tissue straight in the freezer, ice crystals start to grow in the freezing water and solutes – solid particles dissolved in the water – get forced out into the remaining liquid. This can result in unwanted high concentrations of solutes, such as salt, which can be very damaging to the tissue,” said Dr Lorna Dougan from the University of Leeds, who led the study. “The addition of cryoprotectants, such as glycerol, lowers the freezing temperature of water and prevents crystallisation by producing a ‘syrupy’ semi-solid state. The challenge is to know which cryoprotectant molecule to use and how much of it is necessary.

We want to get this right so that we recover as much of the biological material as possible after re-thawing. This has massive cost implications, particularly for the pharmaceutical industry because at present they lose a large proportion of their viable drug every time they freeze it.

Perhaps not. Fernandez said all’s not lost, though.

 While much of these observations remain true, it is unfortunate that some of the mechanistic details turned out to be wrong and must wait for further studies.

0 thoughts on “PNAS retracts two papers on osmolytes after researchers discover crucial measurement errors”

  1. Tough break for the PI, but a great example of how retractions should work. I think many other groups would choose an alternative route here but I think they chose the right one. Looks like Dr Dougan has plenty of other decent pubs to her name and this will surely have little impact on her reputation and rightly so.

  2. (Disclosure statement: the author of this comment, Lewyn Li, has conducted research and published on the topic of single-molecule protein folding using atomic force microscopy, when he was a post-doctoral fellow in the laboratory of Julio M. Fernandez between 2003 and 2006.)

    Adam Marcus should be applauded for an excellent report. His follow-up communications with the principal investigator in question (Julio M. Fernandez, Columbia University) have greatly helped to foster a critical understanding of what might have gone wrong in these two retracted papers.

    I found the retractions and subsequent explanations by Fernandez wanting in three aspects.

    First, no explanation was given why a clarification or retraction had not been issued on the reliability of a third paper from the same laboratory, which utilized strikingly similar experimental conditions, set-up and measurement instrument as the first two retracted papers to study a different protein. The third paper is “Osmolyte-induced separation of the mechanical folding phases of ubiquitin” by Sergi Garcia-Manyes, Lorna Dougan and Julio M. Fernández, Proc Natl Acad Sci USA, 106(26):10540-10545, 2009.

    The third paper, similar to the two retracted papers, reported an increase in the distance from the native to the transition state, when a protein (ubiquitin in the third paper, I27 in the two retracted papers) was mechanically unfolded in 30% glycerol solution. This was described in Figure 2 and Table S1 of the third paper.

    On the other hand, the third paper reported that the average distance from the “ensemble of collapsed conformations” to the transition state did not significantly change upon the addition of 30% glycerol. This was described in Figure 4 and Table S1 of the third paper.

    These different effects from glycerol were cited and discussed by Garcia-Manyes, Dougan and Fernandez as critical experimental support for the main conclusion of the third paper: namely, that their results “open the way for a detailed analysis of the transition state structures that form along the folding trajectory of a mechanically extended protein” (verbatim quote from the abstract of the third paper).

    Contrary to the main conclusion of the third paper, the transition state structures cannot be analyzed in details by these methods. As Fernandez has now admitted, the instrument and experimental set-up, as described in these publications, cannot determine the distance to the transition state accurately in high-viscosity glycerol solutions.

    It is difficult to imagine that the conclusions of the third paper will survive in their entirety unmodified by the retraction of the first two papers. At the very least, Garcia-Manyes, Dougan and Fernandez owe the scientific community a clarification, which will explain whether or not they still stand behind all or part of the results and conclusions of the third paper, and why.

    Second, the retractions lack details and specific explanations. One of the main functions of a retraction is to explain why the authors have reached the rather drastic decision to retract. Whenever possible, the authors should strive to provide additional data and specific explanations for their decision, in the interest of scientific integrity and full disclosure. For two examples of a detailed explanation, see the retractions by Case et al. (Science, 307:1409, 2005) and Johnston and Siderovski (Proc Natl Acad Sci USA, 109(5):1808, 2012).

    The retraction by Fernandez consists of a single sentence, which contains no data and no specifics. How will any scientist begin to understand what might have happened, and how can any scientist independently begin to evaluate the reliability of similar research in light of the retractions?

    Third, the retractions left important questions unanswered. Both retracted papers included results from independent computer simulations, which were reportedly in agreement with the experimentally observed distances to the transition state. The retractions gave no reason for the now apparent inconsistency between experiment (no effects from glycerol on the distance to the transition state) and computer simulations (a clear effect from glycerol on the distance to the transition state).

    This lack of an intellectually coherent explanation is liable to cause confusion, because a reader can legitimately start to wonder: how can two rigorous and independent methods begin by agreeing, only to disagree completely at a later time, without proper explanations? To a larger point: under what conditions should an agreement between experiment and computer simulations be considered non-fortuitous and therefore mutually supportive?

Leave a Reply

Your email address will not be published. Required fields are marked *