A suggestion: If you’re going to use the words “overestimated accuracy” in the title of your paper, you’d better make sure you aren’t guilty of the same yourself.
A group of authors in China has lost their June 2013 paper in the European Biophysics Journal because they appear to have misinterpreted their data. The paper, “Overestimated accuracy of circular dichroism in determining protein secondary structure,” came from chemists at Fudan University in Shanghai, and purported to find that:
Circular dichroism (CD) is a spectroscopic technique widely used for estimating protein secondary structures in aqueous solution, but its accuracy has been doubted in recent work. In the present paper, the contents of nine globular proteins with known secondary structures were determined by CD spectroscopy and Fourier transform infrared spectroscopy (FTIR) in aqueous solution. A large deviation was found between the CD spectra and X-ray data, even when the experimental conditions were optimized. The content determined by FTIR was in good agreement with the X-ray crystallography data. Therefore, CD spectra are not recommended for directly calculating the content of a protein’s secondary structure.
But now comes this:
After publication of this article, concerns were raised regarding accuracy and interpretation of the presented Circular Dichroism data. To investigate these claims, the article was checked again by two additional referees, and was also discussed by members of the European Circular Dichroism Society. The investigation revealed that the Circular Dichroism aspects of the article are invalid, because the measured protein Circular Dichroism spectra are highly inaccurate, and contradict numerous published studies. Hence, the conclusions of the authors regarding the unreliability of using Circular Dichroism for protein secondary structure determination are unjustified and misleading. When confronted with these findings, the authors were unable to dispel the doubts in their experimental methods and declined the invitation to withdraw the article themselves. It could also not be clarified, why the Circular Dichroism spectra presented in this paper have anomalously low amplitudes.
As a result of these findings and the subsequent discussions, the Managing Editor of the European Biophysics Journal, Anthony Watts, has now retracted this article.
The paper has yet to be cited. We commend the EBJ on its informative retraction notice. But the editors might not want to use the same batch of reviewers again — or at least limit them to areas in which they have more expertise.
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post.
This retraction may be worth a second look. The position taken in the abstract is more or less what I was told by protein structure people many years ago, and a very brief look at some more recent literature suggests that the basic story may not have changed all that much. Briefly (and I’ll bow out to anyone with genuine expertise) the message was that CD was good for small peptides and, under the right circumstances, for measuring a conformation change; but not too reliable for baseline protein conformation studies in solution.
So, it was slightly (and only slightly) eyebrow-raising to read that the journal’s retraction relied in part on a discussion with “members of the European Circular Dichroism Society.” Did they suggest the new referees? How much of a professional investment do society members have in the CD technique? This seems to be one of those retractions which is probably righteous, but with a little question mark.
Well, I think the European Circular Dichroism Society will not claim that X-ray and CD should always give the same conformation, but this paper gave such large deviations for well known proteins already analyzed by many others, with notably different results, that any good scientist would start with the question: why are my results so different? Plenty of ‘stupid’ reasons for that, such as erroneous calculations (CD is notorious for its many different units, and ‘odd’ calculations to go from one to the other), or a problem with the cell (pathlength is crucial for a proper calculation).
“this paper gave such large deviations for well known proteins already analyzed by many others, with notably different results, that any good scientist would start with the question: why are my results so different?”
I have heard that repeatability is difficult using the same researchers, protein and instrument; am I mis-informed?
It depends on what you consider “difficult”. In my lab we regularly do CD, and I cannot remember anything worse than 10% variability that we could not easily pin to a mistake somewhere; repeatability usually is well within a few %. However, some people may consider that “few %” already too much and therefore call repeatability “difficult”.
The most common mistakes that introduce variability are practical issues, such as diluting the sample further after determining its concentration (introducing the error of the pipetting), pathlength variability, wrong or poor baseline subtraction, using a cell that is not as clean as it should be, or the funniest one: subtracting a noisy baseline and then use a strong smoothing to get the spectra to look nice.
I have no idea whether any of these played a role here, but the differences with many other papers are so large that they at least should have mentioned that their CD analysis differed significantly from that of many others,and that their conclusion should have been that *in their hands* CD was not a robust method.
I agree with Toby. This retractions seems to come very close to the Argument from Authority.
“These people are experts. This dissenter should be ignored because the experts don’t agree. We know the dissenter is wrong because we asked the experts.”
Wouldn’t it have been far better to publish a letter to the editor, or a rebutting paper, rather than a forced retraction that only claims that there are unidentified problems with the paper?
The moral of the story: people who live in glass houses should be very careful before casting stones. In other words, if you’re going to question the validity of widely used techniques, you better be damn sure of the validity of your own techniques.
According to google scholar, this paper has been cited twice (I ignored the reference in a thesis).
Being in China, they most mlikely cashed in on the impact factor, and quite handsomely considering that this journal has an IF of 2.274: http://www.springer.com/life+sciences/biochemistry+%26+biophysics/journal/249
So, will the authors reimburse the money?
This again sounds like another great case for “Epic Science Battles.” Two blind scientists (the authors and someone from the CD society), one of the nine proteins randomly given to both. Let them determine the structure with CD. If one of them does it better than was done in this paper… eternal shame for the authors. Otherwise, the CD society has to change its name to the European CD (Except for determining secondary structure) Society.
I think that epic battle has already been had: just google the names of the proteins used in this study and search for CD spectral analysis. You’ll find plenty of papers for most of these proteins.
A few issues in the comparison may be the use of different units and different analysis programs, but even then it is likely that the results in this particular paper are a very clear outlier.
Marco is right – as long as you control your protein quality, buffers and cell cleanliness, then reproducibility is very good over a timescale of years. This reproducibility includes deviations from structure predicted from X-ray diffraction, which is not surprising, as only the most rigid conformer of a protein in solution will crystalise.
The question of deconvolution of spectra is entirely different and again, as Marco points out, this area has many traps for the unwary. The issue here is a comparison of spectra, rather than a comparison of structural features deduced from a spectrum with those determined by X-ray crystallography. It would appear that they rushed to publish, rather than look at their experiments with a critical eye and track the quality of their protein samples, buffers, etc.. Perhaps the lure of IF was indeed too much, who knows.