Last week, we reported that Craig Hill, a prominent chemist at Emory University, and his colleagues at six other institutions are retracting three papers they published in the mid-2000s, two in the Journal of the American Chemical Society and one in Science.
We have now spoken with Hill, who walked us through the history of the research. According to Hill, the international team of researchers, after “unusually extensive experiments” felt they had enough evidence to publish their original articles
but all authors (and others) remained skeptical given the unprecedented nature of these compounds.
Hill’s lab continued to conduct experiments and probe the original data after the publications, he said. (Hill wrote a piece for Nature in 2008 explaining the significance of the research, which, among other things, might lead to better ways of harnessing solar energy.)
About a year and a half ago, we got we a result that was not compatible with the original papers. We did additional experiments, contacted the rest of the team, and everybody agreed with our new formulation of these compounds based on the new data.
Of course, Hill and his colleagues wanted to publish their new findings, but that raised the question of what to do about the existing papers. After speaking with the editors of JACS and Science, the researchers agreed that retraction would be the cleanest solution despite the fact that all the data in the original papers are correct.
All the data were in the original papers are 100% correct. The only error was in the interpretation. As a consequence of having zero bad science, we didn’t have to withdraw these papers. However, the JACS Editor and we felt it was best to retract them for 2 reasons: (1) we are publishing a new paper with much new data in it and the new paper summarizes the data in the earlier manuscripts; (2) I don’t want to be replying to newcomers to this field for the next 10 years or more who that don’t know this history. That’s what would happen if we kept the original papers in published status.
The new article is the paper in Inorganic Chemistry that is cited in the JACS retraction notices, but which still doesn’t seem to have gone live [updated 6/18/12]. We’ve asked the journal’s editor for the link, and will update when we hear back. In the meantime, said Hill:
This is really an example of science working. I spoke with all the people involved with the papers and also non-author colleagues and they’ve all strongly concurred with that. It’s really the system working.
We did wonder about one thing, though: If all the data in the old papers were correct, would the new publication refer to them in any way even though the papers were retracted?
Hill said he had the same question.
The editors thought the new manuscript and associated information would take care of this. So I haven’t worried further about it.
Still, it does raise an interesting issue: If a paper is retracted but the data are presumed to be intact, should the researchers reference it with some sort of asterisk-like notation?
Extra: Chemical & Engineering News weighed in on the retractions today.
This statement makes no sense: “All the data were in the original papers are 100% correct. The only error was in the interpretation. As a consequence of having zero bad science, we didn’t have to withdraw these papers.” Science is as much – if not more – about interpreting data as it is about generating data. Ergo, errors in interpretation qualify as “bad science.”
Others have made the point that if all misinterpretations were withdrawn from the literature, there’d be no literature left. A university library would fit in a broom closet!
This is a terrible precedent. I think we need a second category of official rectification for errors in good faith; to lump them in with data fakery is simply wrong. After all, how do we know that the current interpretation of the data is correct? There is no surety in science, nor in life either.
The statement I quoted was – and is – nonsensical. It implies that generating solid data is the only aspect that contributes towards good science, which is clearly inaccurate. The interpretational components are equally – if not more – important. So if you mess up the interpretation of your data, you have conducted bad science. Indeed, the interpretation of your data often determines whether or not it is publishable, and if it is publishable, the type and level of journal it is published in.
I never made any statements as to whether I thought this was sufficient grounds for a retraction, or whether or not all scientists had made such mistakes (undoubtedly, if you publish regularly, there are both data and interpretation errors in your body of work), only that to make a statement like “All the data were in the original papers are 100% correct. The only error was in the interpretation. As a consequence of having zero bad science” is jibberish. Does this mean that most – if not effectively all – scientists have conducted (and potentially published) some “bad science” based on erroneous interpretations of otherwise good data at some point(s) in their careers? Absolutely. Time to get over the attempts at spin doctoring as appears to be evident on this particular case.
more true in biology, may be…
Perhaps what we really know does “fit into a broom closet”. More than that is not really useful.
Certainly not as useful as brooms.
You have hit on a good analogy for science. Brooms. Quite right! Kepp sweeping away!
Proper scientific inquiry is actually about as little interpretation of objective facts as possible, i.e. a good experiment should yield objective facts which either support or reject a hypothesis. This is not something I am making up, but rather a principle that has been established since the 17th century by the first “true” natural philosophers.
I have become rather weary of publications which contain discussions that make up 50% of the paper and which draw upon dozens of references in order to make a point.
In this particular case I would argue, that if the science/experimentation was 100% correct, then perhaps errata should have been published to make the necessary changes to the interpretation of the results.
Most of chemistry is founded on interpretation. We teach NMR interpretation, IR interpretation, and on the list goes. Interpretational components are fundamental to the discipline, despite what some purists may want to believe. We have various layers of interpretation. We interpret what the interpreted NMR spectra tell us about the particular system we are investigating, and so on. As such, in almost all cases in chemistry, the “science” involves some degree of interpretation, and if any of that interpretation is incorrect, the science is faulty. To claim otherwise is nonsensical.
As for errata vs. retractions, that depends on how the corrected material impacts the original claims in the work. If a work was published in a particular journal based on extraordinary claims made in the original submission (i.e., the high impact standard that JACS, PNAS, Science, Nature, etc., apply to reject many papers that are submitted to these journals), and these extraordinary claims are no longer supported by a new – and correct – interpretation of the original data, and if – based on the new and correct data interpretation – the work now stands with a level of impact below that commonly accepted for the journal in question, then the work should be retracted. To find otherwise sets up a substantial moral hazard, whereby authors are encouraged – even rewarded – by making extraordinary and unsupported claims off their data in the hopes of obtaining a high-rank publication that will not be retracted even if the claims are later found to be incorrect.
“Bad science” has a bad connotation. I think errors in interpretation are not as bad as, you know, faked western blots. Do we have weak science from errors in interpretation? Better than having bad science.
Faked western blots is fraud, not science.
Serious errors in interpretation is ‘bad science’ (especially the ones that could have been avoided at a time being).
If all papers where the original interpretation did not stand the test of time were retracted, there would be almost no papers left. Retracting papers which are claimed to be 100% good science on these grounds does a massive disservice to other researchers in the field, removing part of the history of work in the field for no good reason.
The Chemical & Engineering News commentary notes “crystal structures can fool you”, but also “the complexes violated basic transition-metal bonding principles”, did they report crystal structure than violate basic transition-metal bonding principles? Or did they model the complexes wrong? In the latter case, it should have read “they interpreted the crystal diffraction data wrong”, crystal structure do not fool you, the models of the scientist may…
They modeled the complex wrong, I think. The complex had two axial oxo ligands and based on the lengths of the metal-oxygen bonds, they assigned one ligand as -OH and the other as =O.
This is an abuse of the mechanism of retraction. You’d probably have to retract half the scientific literature if the standard is that your interpretation of your results turns out to be wrong. If their description and analysis of their data are correct, then those papers should stand. This sets a horrible precedent.
So I admit I haven’t read the papers, but reinterpreting data in the light of new knowledge contributes to the robustness of science, and is not only perfectly acceptable, it is required. I suspect retracting these papers on those grounds is more to do with ego than science.
This is very much how science should work. The interpretation was very exciting and also a reasonable possibility. The authors continued to study their compounds and eventually concluded that the original interpretation was wrong.
Bad science would then build a defensive wall to protect the original papers and then publish more and more papers on their back. Good science looks hard at the accumulating evidence and states bluntly that the world isn’t how we thought it was. The authors should be held up as an example of how science is and should be done. My view is that they are correct to retract the papers.
The question arises how do we differentiate between scientists and charlatans? Charlatans make up their data. Simple, we read Retraction Watch! More seriously, the retraction notice in the the journal should provide the information as to the cause of retraction. In this instance it does, but, as noted frequently on this blog, often journals are rather coy.
For Comradde PhysioProffe
“If their description and analysis of their data are correct, then those papers should stand.”
I don’t think their description and analysis is correct – I think there’s a difference in what people mean by ‘interpretation’. The whole point of the papers was that they were reporting “We have made this compound – and this compound is extremely unusual”. They have not made that compound, so the whole premise of the paper(s) is wrong.
From the Chem & Eng News article linked above:
“His group continued its experiments and eventually found evidence that a tungsten atom is at the heart of two of the butterfly-shaped polytungstate complexes, not platinum or gold as originally reported.”
I guess they could have corrected all three papers, but it would have meant re-writing them entirely and would have turned them into reports of run-of-the-mill compounds, rather than ones that challenge conventional understanding of transition metal chemistry and provide insights into potentially useful/biologically relevant catalysis.
Neil, I agree. This is not a matter of collecting good data on a well-characterized system and then drawing conclusions that prove to be wrong, which is something all of us have done and indeed does not warrant retraction. What some of the commenters are missing is that the entire point of these papers was the characterization of such-and-such a compound, and indeed the reason they were accepted in top journals was precisely that the compounds were so unusual and novel. The best analogy I could think of would be an ancient-DNA specialist, back in the early days of the field when less was known, isolating DNA thought to be from an extinct species of archaic human, doing all kinds of painstaking sequencing, lots of comparative analysis with other species, tons of phylogenetic modeling, doing everything right, writing it up, and discovering later that the sample was instead contamination from the laborer who had helped collect the material. All of the data is indeed right; but all of the science is wrong, and the interest level falls through the floor. That comparison is a little bit unfair to the Hill group, since there may well be some interest out there in the compounds they prove to have actually made, but it’s closer to what is meant by the separation of “data” and “interpretation” in this case. I still consider retracting the papers very much erring on the side of conservatism, but the precedent is not nearly as wide-ranging as might be thought.
I saw Prof. Hill present some of this work in Bern a few years back and he made it clear to us at the time that the bonding was unusual, so I really don’t think there was any ill-faith in there at all. It’s a recurring problem with crystallography and non-bridging oxygen ligands as to whether it’s oxo- hydroxo- or aqua- on a site. If it’s bridging, you can tell the state of the oxygen by bond valence sums, but otherwise, it’s very difficult and imprecise.
You can bet that there are a number of reviewers who will now look a little more closely at complexes with hanging oxygen atoms attached…
they indeed publish in big journals usually. if the structure is not clear why mislead people. mostly such discoveries are published in Cell, Nature and Science with potential implications on drug targeting or something like that….if it is ambiguous in nature – as you say – why make it big……
The concern about the new paper summarizing data in the previous paper seems like a made-up concern. Hill could include a reference to the original paper to avoid any worry that he was claiming new results that are a duplicate publication of the published results. And the excuse that he wants to save himself the inconvenience of explaining to newcomers that his earlier paper was wrong — big raspberry to that one.
Publications might consider having an equivalent to drawing a single line through an entry in a lab notebook. You don’t scribble over the mistaken item to make it unreadable. You put a single line through it, so you can read it in case you ever want to come back to it. Maybe you really were right the first time and were wrong when you decided your entry was incorrect. Any paper that went through as much checking as that one did can serve as an example of how easy it is to make the wrong conclusion from the right data. Are we going to remove Phoebus Levene’s tetranucleotide conclusion because he was wrong about DNA? The history of science includes all the wrong paths scientists have followed, not just the right ones.
The whole point of the papers was that they were reporting “We have made this compound – and this compound is extremely unusual”. They have not made that compound, so the whole premise of the paper(s) is wrong.
I didn’t understand that was what was meant by “interpretation”, and now I agree that retraction is appropriate.
I think that we are now getting a handle on this… Even if the raw crystallographic data (intensities, unit cell, space group) were/are “100% correct” (like almost all similar data), the issue is of course the modelling of the structure(s), which unfortunately seem to be completely wrong, in the sense that the key components of the structures contain the expected tungsten atoms and not the very unexpected gold or platinum atoms [it seems that the issue of “oxo” versus “hydroxo” (Au=O or Au-OH) bonds was a bit of a red herring]. Thus, as stated above, these are interesting (and well worth publishing) but relatively incremental, rather than revolutionary, advances. Whether this error of interpretation constitutes “zero bad science” is a matter for debate…
There is still a puzzle for me: key supporting evidence for the original structures were the elemental analysis data, which seemed to agree almost perfectly with the incorrect compositions. I hope the authors resolve this issue in the Inorg. Chem. paper. As for the cryptic comment that “crystal structures can fool you,” markj (see above) has put it perfectly…
The Inorg. Chem. paper is live now: http://pubs.acs.org/doi/abs/10.1021/ic2008914?ref=tw
Woops, having read the paper now, yes, they got it wrong, and retraction is appropriate. I retract my previous comment!