A marketing journal is taking heat on social media for issuing an expression of concern over a 2019 paper that many readers believe should have been retracted — and correcting another instead of retracting it.
The article now subject to an expression of concern, “Role of Ambient Temperature in Influencing Willingness to Pay in Auctions and Negotiations,” was written by Jayati Sinha, who holds the Macy’s Retailing Professorship at Florida International University and Rajesh Bagchi, of Virginia Tech University.
According to the abstract:
While temperature’s effects on human physiology have been well studied, its effects in decision-making contexts are still relatively unknown. The authors investigate the role of ambient temperature in one important decision-making context: consumer purchase. More specifically, they examine how ambient temperature influences consumers’ willingness to pay in different kinds of purchase contexts, such as in auctions and in negotiations. The authors show that whereas higher (vs. moderate) temperatures elicit higher willingness to pay in auctions, they lead to a lower willingness to pay in negotiations, and temperature-induced discomfort and aggression underlie these effects. The authors also study the effects of lower temperatures and extend these findings to more general competitive settings. They report findings from six studies and discuss theoretical, managerial, and policy implications.
But as the journal states, a reader contacted the editors with concerns that the article — which drew comments on PubPeer about questionable effect sizes in the studies — contained fabricated data:
based on the identification of unusual patterns in three studies reported in the article. After receipt of the concerns, the journal arranged for a review by a panel of independent scholars. The authors fully cooperated with the independent review by providing copies of the original completed questionnaires and associated spreadsheet files used in the analysis for studies 2a, 2b, and 2c – the three studies under question. The independent reviewers were unable to confirm the third-party’s concerns about data fabrication.
However, in comparing the original questionnaires and Excel files, four anomalies were identified: (1) data appearing in the spreadsheet files when no values were present in the original surveys; (2) data in the spreadsheet files having different values from those that appear in the original surveys; (3) large outliers (outside the valid response range) whose values were changed in the entered data; and (4) entries in the original survey being scratched over with new results circled. Anomalies were found in all three studies, with Study 2b having the largest number.
The anomalies described in points (2) and (4) were deemed less serious as the anomalies described in point (2) could be attributed to random data entry errors, and the changed entries described in point (4) may happen in surveys and experiments.
Point (1), the entry of data in the spreadsheet files that was not present in the raw files, and point (3), the failure to identify the occurrence of outliers and the action of changing the outlier data to results more consistent with other reported data without disclosing these changes in the article, were deemed more serious infractions of data management.
The number of instances of (1) and (3) are very small relative to the number of data points checked in Studies 2a–2c. Upon independently reanalyzing the data, it was found that the results do not change when correcting for these anomalies. However, given the nature of the errors, an Expression of Concern was deemed prudent.
The first author, Jayati Sinha, had informed the journal that the data were entered exclusively by the first author.
Sinha was the senior author of a 2012 paper in Marketing Letters that the journal retracted in 2016 because of questions about “troubling oddities” with the data:
This article has been retracted at the suggestion of journal Editors-in-Chief, Peter N. Golder and Joel H. Steckel. The article’s authors unanimously requested retraction of Study 3 based on unexplained anomalies in the data and coding errors. As a result, the editors deem it appropriate to retract the entire article.
Sinha said:
I accept responsibility for making a small number of inadvertent errors. For example, there were two instances of large outliers ($7500) and I missed the last digit and mistakenly entered $750, perhaps because the response was required to be below $1000. As the EOC states, the errors were very small in number and even after correcting the errors, the results hold, and the conclusions do not change. I sincerely regret these errors but am consoled by the fact that they did not change the conclusions that we report in the paper.
The expression of concern triggered its own concerns among readers, including reproducibility guru Brian Nosek, who called it the “weirdest expression of concern I have ever read.”
Aaron Charlton, who teaches marketing at Illinois State University, noted that the Journal of Marketing does not appear to have retracted a paper (which our database confirms) — even one by Ping Dong, a former Northwestern Unviersity professor who has had three papers retracted and about whom we’ve written several times.
As Charlton points out, the journal corrected Dong’s paper — by correcting her name out of the list of authors and allowing her co-authors to revise and resubmit the article.
Christine Moorman, the editor in chief of the JoM, told us:
The investigation performed revealed data entry errors of the four types described in the EOC. However, two of these types were viewed as possible true errors, all types of errors were small in number, and reanalysis of the data with corrected inputs showed that the results held. Given this, a retraction did not seem necessary. Instead, it seemed more suitable to share an honest description of the data problems and their implications to the discipline, but to allow the original article to remain published. On the spectrum of reasons for article retraction, this infraction did not seem to meet the threshold for retraction.
The Journal of Marketing is committed to publishing research with a high standard of integrity and is entirely open to retractions as a tool for maintaining such a standard. The case you cite in your email was brought to the Journal of Marketing by two of the authors who raised concerns about Ping Dong’s behaviors with regard to some of the data in the accepted article. Given that the authors did not have access to the original data in these studies, they sought to replicate the studies in question before the article was published in the paper version of the journal, which they did. The new studies are reported in a corrected paper and online version of the paper that was published by the Journal of Marketing. Given this inquiry, correction, and replication of the original study, a retraction was not necessary in this case.
As a reminder, expressions of concern are not meant to be the final word on a paper’s status. Perhaps this just illustrates that, for the JoM, the difference between a correction and retraction is … all in the marketing.
Like Retraction Watch? You can make a one-time tax-deductible contribution or a monthly tax-deductible donation to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
Thanks for a great article! The editor, Christine Moorman, seems sincere and I appreciate her willingness to address our concerns. If the two papers really are problematic, maybe with more evidence in the future the right thing will be done (either to retract or not depending on what the evidence shows).