In February of this year, the Joy of Cooking launched what you could call an epic Twitter stream. Inspired by Stephanie Lee’s reporting in BuzzFeed on Brian Wansink — the food marketing researcher at Cornell who later resigned following findings of misconduct by the university — the legendary cookbook pointed out all that was wrong with a 2009 study claiming that their recipes added calories over the years. Those tweets led to coverage in The Verge, The New Yorker, and elsewhere.
This week, the Annals of Internal Medicine retracted that paper, along with another. That brings Wansink’s tally of retracted papers to 17, with one of the papers retracted twice. (And no, 17 is nowhere near a record; he’s not even among the 30 authors with the most retractions in the world.) As Retraction Watch readers will likely recall, his work began to unravel when, after a 2016 blog post in which Wansink seemed to endorse p-hacking, four researchers joined forces to analyze his work.
Here’s the notice for 2009’s “The Joy of Cooking Too Much: 70 Years of Calorie Increases in Classic Recipes,” a paper that has been cited 20 times, according to Clarivate Analytics’ Web of Science:
The editors of Annals of Internal Medicine are retracting, “The Joy of Cooking Too Much: 70 Years of Calorie Increases in Classic Recipes,” published in 2009 (1). In response to Annals’ query regarding a Cornell University investigation of Brian Wansink’s work, we received a letter from the Office of the Vice Provost for Research dated 27 September 2018 that stated, “This investigation has concluded that Professor Wansink committed academic misconduct in his research and scholarship.” Because this letter did not state whether the institution found the specific work reported in this article to be scientifically valid, Annals contacted the authors to inquire whether they had concerns about its validity. The contact information we had for Dr. Payne was no longer current, and we were unable to locate current contact information. In response, Dr. Wansink reported that he was also unable to contact Dr. Payne, but Dr. Wansink provided a reanalysis of the data and reported that “The files we reran gave the same conclusions, but different numbers in the table.” In fact, almost every number was different from those in the published article, many substantially so. In light of the inability to reproduce the published results, the editors cannot be confident in the integrity of the work reported in this article.
And here is the notice for “Meal Size, Not Body Size, Explains Errors in Estimating the Calorie Content of Meals,” a paper published in 2006 and cited 77 times since:
The editors of Annals of Internal Medicine are retracting the article, “Meal Size, Not Body Size, Explains Errors in Estimating the Calorie Content of Meals,” published in 2006 (1). In response to Annals’ query regarding a Cornell University investigation of Brian Wansink’s work, we received a letter from the Office of the Vice Provost for Research dated 27 September 2018 that stated, “This investigation has concluded that Professor Wansink committed academic misconduct in his research and scholarship.” Because this letter did not state whether the institution found the specific work reported in this article to be scientifically valid, Annals contacted the authors to inquire whether they had concerns about its validity. In his response, Dr. Chandon reported that age was not a variable collected during the study. Yet, the article reports mean age for male and female participants. In addition, the editors identified no age variable in the data files nor on the sample paper data forms that provided in response to our query. In light of the reporting of a variable (age) that seems not to have been collected, the editors cannot be confident in the integrity of the work reported in this article.
It’s worth noting that the journal undertook its own investigation here, rather than limiting itself to what Cornell sent them. While other journals have taken that kind of initiative, it’s not necessarily the norm — and should, we think, be championed.
Update, 1100 UTC, 12/6/18: We had asked Wansink yesterday whether he agreed with the retractions. He tells us:
With both of these papers, the data is easy to collect and the general findings are robust and useful. Like my other work, this makes them easy to replicate and easy to confirm by other research teams (like I think they both already have).
Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
This is certainly an instance where the journal notifications following the institutional investigation were inadequate. Particularly given that the many concerns surrounding Dr. Wansink’s papers are already widely known, it’s unclear what purpose this notification serves if it doesn’t speak directly to the integrity of specific paper the journal published.
It appears they’re towing a different legal line by calling it “academic misconduct” rather than “research misconduct;” to me implying that the notification here likely follows from a narrow reading of their institutional policy rather than the federal regs. But that still doesn’t really explain why the provost couldn’t be more specific. I’m on the record as not being a fan of institutions releasing all the dirty details willy-nilly, but I think we can all agree that at minimum, publishers should be told about the specific findings impacting the validity of individual papers published in their journals following any investigation. While it was good that they followed up, it really shouldn’t have been necessary.
I think journal follow-up should be necessary. At a minimum, it makes up for the journal’s lax vetting of the paper in the first place. Praising journals that do follow up, and shaming journals that do not follow up, is a necessary component to sorting out the publishing mess currently in place with regards to poorly run analyses being published far too often.
Since institutions are so loathe to police themselves in proper accord with reasonable scientific conduct, journals are rarely going to get a thorough institutional report of the institution’s “scientific” staff submitting shoddy papers. Journal editors really need to step up here, and the editors of Annals of Internal Medicine deserve thanks and praise for following through here and properly assessing the validity of the materials in these now retracted papers.
For the “The Joy of Cooking Too Much: 70 Years of Calorie Increases in Classic Recipes” paper:
In response, Dr. Wansink reported that he was also unable to contact Dr. Payne, but Dr. Wansink provided a reanalysis of the data and reported that “The files we reran gave the same conclusions, but different numbers in the table.” In fact, almost every number was different from those in the published article, many substantially so. In light of the inability to reproduce the published results, the editors cannot be confident in the integrity of the work reported in this article.
For the “Meal Size, Not Body Size, Explains Errors in Estimating the Calorie Content of Meals” paper:
In his response, Dr. Chandon reported that age was not a variable collected during the study. Yet, the article reports mean age for male and female participants. In addition, the editors identified no age variable in the data files nor on the sample paper data forms that provided in response to our query. In light of the reporting of a variable (age) that seems not to have been collected, the editors cannot be confident in the integrity of the work reported in this article.
This level of extra effort by the editors of Annals of Internal Medicine demonstrate that this journal is still taking its scientific mission seriously.
It’s perhaps notable that the articles from the Cornell Food and Brand Lab that have been retracted so far have disproportionately been those published in medical journals. The psychology and marketing journals are somewhat lagging behind.
I don’t think I agree that we can conclude that the journals’ vetting of the papers was particularly lax. While Dr. Wansink’s various journal submissions surely benefited from his prolificacy and recognition in the lay media, I don’t see any indication that the peer review and initial editorial processes were notably different for his work than for any other paper. Readers of RW will be very aware that peer reviewers fail to recognize “red flags” in papers more often than not, and that uncovering these sorts of errors in data analysis, cherry-picking and statistical shenanigans generally requires a much deeper dive into the primary data than a typical peer reviewer would ever perform. My remark about how the journal follow up should have been unnecessary pertains primarily to my suspicion that if the institution did all their due diligence in their investigation to actually determine the nature of the issues in each paper and then notified the respective journals in detail about their specific findings, then the journals would not have had to do so for themselves.