“Devastating:” Authors retract paper in Nature journal upon discovering error

Several years ago, Chris Dames thought he had made an exciting discovery, a “secret sauce” that would allow him to design a device using a novel mechanism.

In a 2014 Nature Communications paper, Dames—who works at the University of California at Berkeley—and his team described the first experimental results for the device, a photon thermal diode. A thermal diode conducts heat in one direction but not in the other, and in theory, could have broad applications—for example, provide barriers that shield buildings from excess heat or use heat to power computers.

But two years later, in August 2016, a colleague thought he had discovered a fundamental error in the design of the experiments. Bair Budaev, who also works at the University of California at Berkeley, believed that the authors made a “a fundamental symmetry error” which invalidated their results.

Budaev discussed his concern with Dames and submitted a commentary to Nature Communications, outlining the flaw he had observed.

Dames told us he was aware of the symmetry issue Budaev raised, but believed he and his team had already factored that point into the experimental design. Still, Dames revisited the results and analysis and found a subtle error in the original model, which ultimately confirmed Budaev’s belief: The authors had not designed a thermal diode. Dames and his co-authors “concluded that our original concept wouldn’t work” and decided to retract the paper. The process of re-analyzing the study from various perspectives, drafting the correspondence, and moving it through peer review took close to a year.

On Aug. 21 2017, Nature Communications published the retraction and correspondence from Budaev and the authors.

Here’s the retraction notice for “A photon thermal diode:”

Because two of its three major findings have been invalidated, the authors wish to retract this Article1. Budaev2 correctly identifies a fundamental symmetry error in the way the ‘inelastic thermal collimation’ was configured in several crucial experiments of ref. 1, specifically the results presented in Fig. 3c (the three filled and four striped bars, labeled ‘Col. 1’ and ‘Col. 2’, respectively) and Fig. 4. As detailed in the accompanying Correspondence3, this error originated from a faulty thermal estimate4, and further modelling3,4 now confirms the problem. Although we continue to believe the heat flow measurements in ref. 1 were accurate for all of the configurations presented, due to the symmetry error none of those experimental configurations were actually relevant for two of the most important findings of the paper, which are therefore retracted: firstly, that a photon thermal diode was experimentally demonstrated, and secondly, that the ‘inelastic thermal collimation’ mechanism is a suitable nonlinearity for realizing thermal rectification when combined with asymmetric scattering structures (e.g., copper pyramids or etched triangular pores in silicon).

The symmetry error2 does not apply to the experiments without thermal collimation, specifically the results presented in Fig. 3c for photons (the six leftmost, unfilled bars) and Supplementary Figure 12 for phonons. Therefore, the last major conclusion of the Article1 remains well-supported by those original experiments: Asymmetric scattering alone is insufficient to achieve thermal rectification.

The 2014 paper has been cited 35 times, according to Clarivate Analytics’ Web of Science, three of which are for the correspondences and retraction notice.

Not a diode: Inside the design issue

Budaev told Retraction Watch why he believes the proposed device—composed of, what the authors call, a collimator and a test section—is not a thermal diode:

… for the structure be a diode, it should be a “sealed device” that could be used only as a whole, without a possibility to make any changes inside it.

But, according to Budaev, in the experiments the authors flipped the position of the collimator and test section, which changed the device’s internal structure and created the fundamental symmetry error he describes in his commentary.

Dames told us that he and his team had not ignored the fundamental symmetry requirements of a thermal diode, but:

…as we did the detailed experimental design, practical challenges got us thinking about simplifying the apparatus.

Dames added:

At this stage, two of us made a key calculation from which we concluded that omitting one of the two perforated graphite plates (the thermal collimators) would introduce only small errors.

Based on the calculation, Dames told us, the team decided to proceed with an experiment with imperfect symmetries, but “believed the missing symmetry piece would introduce an acceptably small error.”

In other words, Dames explained:

It’s not that we completely missed the core symmetry issue, it’s that we made a simplification that we believed, at the time, was justified…to my deep regret this was not explained in the paper.

However, when he went back over the experiments in 2016 and 2017, Dames discovered a subtle but critical mistake: An error in the thermal estimate calculation used to justify the missing symmetry piece.

Dames told us that this realization “was devastating.”

Dames said that perhaps a broader lesson for him is the value of carefully explaining the thought processes behind each decision and step, “since this makes it easier for oneself and others to critically assess the arguments.”

But Dames said:

Unfortunately, there was no way to go back.  After the devastating discovery, I had to think about what I could do from that day forward.

Hat tip: Rolf Degen

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.