On Tuesday, we covered a retraction in the Proceedings of the National Academy of Sciences (PNAS) involving zalutumumab, a compound once being developed for treatments of head and neck cancer. As we noted at the time, the authors decided to retract the paper because they no longer trusted the method they used. One of the authors didn’t sign the notice, and we’ve now heard from her about why.
First, a comment from lead author Paul W.H.I Parren, who tells Retraction Watch:
As explained in the retraction notice, concerns with respect to the technical validity of Sidec Protein Tomography (PT) came to our attention following our publication in PNAS. We discussed the potential issues with several experts in the field. To settle the issue, we decided to perform a study to validate the Sidec PT approach. We chose a staggered approach in which we would first perform a double-blinded experiment using PT on vitrified soluble (unlabeled) proteins, which represented the technique best documented in the scientific literature. This would be followed by a validation experiment using fixated immunolabeled samples (as used in our study in PNAS), which was considered a less-established method. However, the first validation study unexpectedly failed with none of the samples identified correctly. Because no apparent mistakes had been made, we performed a second experiment using different sample composition. Again the validation failed. When these data became available, Sidec AB’s board decided to end Sidec operations. Thus, we were unable to perform the intended second part of our study. This however left us with a problem as the experiments performed still did not tell us whether Sidec PT analyses of fixated and immunolabeled protein (as used in the PNAS paper) were also unreliable. To still address this point, we decided to have our original data reanalyzed by an independent expert. Using a weighted backprojection reconstruction method with IMOD rather than COMET, he concluded our previous analysis to be unreliable.
We also re-examined the supportive biological data reported in our manuscript as well as the model proposed. Here, we are confident these data to be correct and the model plausible. However, all things considered we felt that we needed to draw the conclusion that we can no longer stand behind the Sidec PT data reported. Retracting the paper therefore appeared the only right thing to do and I contacted PNAS.
Almost all authors, including two of the authors originally at Sidec AB, agreed and decided to sign the retraction notice.
Dr. von Euler disagreed and takes the standpoint that we have not formally proven our original data to be incorrect. De facto she is right, as we could not complete the full validation because Sidec AB went out of business as explained above. Nevertheless, we feel that we have diligently pursued validating the technique with the end result that we need to conclude it to be unreliable.
von Euler tells Retraction Watch that she sent this comment to PNAS:
The retraction of this article seems to be due to a disbelief in the accuracy of the analysis method COMET. Our original data have been re-evaluated by another EM group who has not been able to repeat our results. We, who performed the now questioned tomography study, have not seen any of the re-evaluated data nor do we know how the re-evaluation study was performed regarding settings of parameters etc. We do not know either whether the re-evaluation provided a different result or failed to give any result at all. This point is important in order to establish the significance of the re-evaluation. Tomography is a complex methodology and there are many possible reasons why a dataset fails to provide a satisfactory result.
I do not want to sign the retraction since I believe that the correct way to establish whether a method is trustworthy or not is to conduct a new study and publish the results, or at least discuss the new results with the authors of the original paper in order to clarify and understand the discrepancies.
We agree with Dr. von Euler that additional studies to establish whether PT is trustworthy would be of interest. Nevertheless, we felt that retraction of our publication in PNAS was the best way forward at this point as it unambiguously conveys our uncertainty with respect to the PT data to other workers in the field.
We’re with Parren. While we appreciate von Euler sharing her rationale — a step that many who decline to sign notices don’t take — her justification sounds an awful lot like an inability to prove a negative. Conducting a new study and publishing the results sounds just fine, but “we can’t prove it’s right, but we can’t prove it’s wrong” doesn’t seem like a great reason not to retract a paper.
To paraphrase the late Johnnie Cochran: If the proteins don’t interact, you must retract!