On Tuesday, we covered a retraction in the Proceedings of the National Academy of Sciences (PNAS) involving zalutumumab, a compound once being developed for treatments of head and neck cancer. As we noted at the time, the authors decided to retract the paper because they no longer trusted the method they used. One of the authors didn’t sign the notice, and we’ve now heard from her about why.
First, a comment from lead author Paul W.H.I Parren, who tells Retraction Watch:
As explained in the retraction notice, concerns with respect to the technical validity of Sidec Protein Tomography (PT) came to our attention following our publication in PNAS. We discussed the potential issues with several experts in the field. To settle the issue, we decided to perform a study to validate the Sidec PT approach. We chose a staggered approach in which we would first perform a double-blinded experiment using PT on vitrified soluble (unlabeled) proteins, which represented the technique best documented in the scientific literature. This would be followed by a validation experiment using fixated immunolabeled samples (as used in our study in PNAS), which was considered a less-established method. However, the first validation study unexpectedly failed with none of the samples identified correctly. Because no apparent mistakes had been made, we performed a second experiment using different sample composition. Again the validation failed. When these data became available, Sidec AB’s board decided to end Sidec operations. Thus, we were unable to perform the intended second part of our study. This however left us with a problem as the experiments performed still did not tell us whether Sidec PT analyses of fixated and immunolabeled protein (as used in the PNAS paper) were also unreliable. To still address this point, we decided to have our original data reanalyzed by an independent expert. Using a weighted backprojection reconstruction method with IMOD rather than COMET, he concluded our previous analysis to be unreliable.
We also re-examined the supportive biological data reported in our manuscript as well as the model proposed. Here, we are confident these data to be correct and the model plausible. However, all things considered we felt that we needed to draw the conclusion that we can no longer stand behind the Sidec PT data reported. Retracting the paper therefore appeared the only right thing to do and I contacted PNAS.
Almost all authors, including two of the authors originally at Sidec AB, agreed and decided to sign the retraction notice.
Dr. von Euler disagreed and takes the standpoint that we have not formally proven our original data to be incorrect. De facto she is right, as we could not complete the full validation because Sidec AB went out of business as explained above. Nevertheless, we feel that we have diligently pursued validating the technique with the end result that we need to conclude it to be unreliable.
von Euler tells Retraction Watch that she sent this comment to PNAS:
The retraction of this article seems to be due to a disbelief in the accuracy of the analysis method COMET. Our original data have been re-evaluated by another EM group who has not been able to repeat our results. We, who performed the now questioned tomography study, have not seen any of the re-evaluated data nor do we know how the re-evaluation study was performed regarding settings of parameters etc. We do not know either whether the re-evaluation provided a different result or failed to give any result at all. This point is important in order to establish the significance of the re-evaluation. Tomography is a complex methodology and there are many possible reasons why a dataset fails to provide a satisfactory result.
I do not want to sign the retraction since I believe that the correct way to establish whether a method is trustworthy or not is to conduct a new study and publish the results, or at least discuss the new results with the authors of the original paper in order to clarify and understand the discrepancies.
Parren:
We agree with Dr. von Euler that additional studies to establish whether PT is trustworthy would be of interest. Nevertheless, we felt that retraction of our publication in PNAS was the best way forward at this point as it unambiguously conveys our uncertainty with respect to the PT data to other workers in the field.
We’re with Parren. While we appreciate von Euler sharing her rationale — a step that many who decline to sign notices don’t take — her justification sounds an awful lot like an inability to prove a negative. Conducting a new study and publishing the results sounds just fine, but “we can’t prove it’s right, but we can’t prove it’s wrong” doesn’t seem like a great reason not to retract a paper.
To paraphrase the late Johnnie Cochran: If the proteins don’t interact, you must retract!
Its easier to get blood out of a stone, than to get an “expert” to admit
that they could be wrong.
An interesting one, and probably an unusual case – a retraction that seems purely voluntary. Authors who make retractions like this are essentially saying that they want to develop scientific lines of investigation with strong foundations. For such people, quality of scientific work, life and reputation are important. If you persist with a story that is wrong, you will waste your time and that of others, and sooner or later someone will prove you wrong. Your reputation in the field could be destroyed.
It’s not surprising there are authors who don’t want to sign; indeed not every aspect of the story has become clear. But I completely disagree that the correct way to establish whether a method is trustworthy or not is to conduct a new study and publish the results. The only correct approach is to make sure your method is trustworthy is to ensure it is trustworthy before it enters a publication. Otherwise you expose the whole scientific community, ranging from experts who are sceptical to students who accept it at face value. If you find out after publication that the method was probably unreliable, then you’ve broken that rule and you need to retract – you can’t keep the paper there exposed to the public while complex, potentially endless discrepancies are discussed. If you retract, you can start with a clean slate and set about doing the work properly, or doing something else.
The flip side of all this is the retraction letters which should be written but aren’t. The moment people start down that road, the closer they get to scientific fraud. But there are probably many times more such cases than voluntary retractions of this sort.
Thanks for the transcripts. I think everybody can see now, who of the two correspondents is a scientist, and who is just a Dr.
I got a laugh that von Euler wants to squeeze another publication from a debate. Ive seen this happen: one group will say something, another group will say its wrong, a third group will say both are wrong. Inevitably someone or more is wrong, but the publications are not retracted. This happens in good journal like PNAS.
Depending on where/what the “wrong” is, such publications can be really good: a vigorous debate in the scientific literature to find out what is the most appropriate description/most likely mechanism. i would not want the “wrong” paper(s) to be retracted in this case, as they may well have been instrumental in finding out what is going on. Moreover, retracting such papers may then lead to the argument being removed from memory, meaning it may well be taken up again many years later.
I also got a good laugh out of Dr. von Euler wanting to get another publication out of the debacle of the first one.
A veteran professor of my acquaintance took the position that when you have doubts about your data “you put it out there and let someone else prove it wrong.” The university is such an amusing place to work.
It would appear that Dr. van Euler believes the other authors have essentially gone behind her back, have ordered alternate testing and have refused to share the results with her so that she is prevented from having meaningful participation in the discussion about what the alternate test results mean, whether they are valid, and whether the paper should be retracted. This makes me wonder.
I also wonder why the researchers could not continue with the research just because the company went out of business. Don’t the researchers own the machine? If they were getting a free loaner and the company took it back when it folded, is there a question about the ethics of the arrangement from the very start? Dr. van Euler refers to “We, who performed the now questioned tomography study” and “the authors of the original paper” meaning herself and presumably at least one additional person who did the work and have been shut out of the decision to retract. So is this a case of several company employees doing the experiment and writing it up, and letting several professors tag along as co-authors, thus lending the prestige and supposed independence of the university to a manuscript that is fundamentally an advertisement for the company’s product? Where have I seen this before?
Off topic:-
Plagiarism
http://www.telegraphindia.com/1120323/jsp/frontpage/story_15285429.jsp#.T2weq9n9zvo
and
http://nanopolitan.blogspot.jp/2012/03/plagiarism-snares-big-gun-at-iit-k.html
A good plagiarism and aftermath story for Retraction Watch…
Perhaps van Euler could refound Sidec and use Protein Tomography to study how XMRV causes CFS and prostate cancer.
For the cancer studies they’ll need an oncologist – I recommend reknowned clinician and researcher, Dr Anil Potti. Then they can work on a cure for XMRV. Maybe resveratrol would work – it works on everything, apparantly – so they need to call Prof. Das and get him on the team.
And if they need a press officer, I hear Mike Daisey has a lot of journalistic experience and is currently free!
Yawn…
I’m curious to know how the technique was actually implemented, that is, what equipment was used? I looked at the schematic on the method paper for Protein Tomography (TM). Did the company actually bring that equipment in, then come and take it away when the board decided to close up? Or something else?
The only unique part of the schematic I could see was the program to interpret data input by an electron microscope. The program generated a 3-D image from the data input. How did Sidec control access to their technique? (and the hard question: did they know it was “unreliable” beforehand?)
von Euler’s resistance to a retraction is incomprehensible. Does she have a hidden motive?
So, will this Science paper be retracted because the authors recently realized the conclusions were based on an artifact?
Dosenbach N.U. et al. Science 329, 1358-1361 (2010)
http://www.ncbi.nlm.nih.gov/pubmed/20829489
http://sfari.org/news-and-opinion/news/2012/movement-during-brain-scans-may-lead-to-spurious-patterns
Petersen, one of the authors, and biggie in the field, says: “Let me tell you, denial was big. We had every explanation in the world other than that it was an artifact. But it’s an artifact.”