Is having money linked to bad behavior?
A high profile paper published in 2012 in the Proceedings of the National Academy of Sciences (PNAS) set out to answer that question — and found that yes, the more money people have, the more likely they are to lie, cheat, and steal. And the greedier they are, the worse they behave. But when a more recent paper tried to replicate some of those findings, it couldn’t.
It turns out, both the original paper and the paper that tried to replicate it contained errors. Although neither appear to affect the main conclusions, the authors of the 2016 replication recently issued a correction; the error in the 2012 paper was initially deemed too insignificant to correct, but the journal has decided to revisit the idea of issuing a correction.
A representative of PNAS told us that the replication paper — and reporting by Retraction Watch — is the reason why:
We reached out to the author about the Correction on September 20 after receiving your note regarding the [replication] article…Thank you for bringing this matter to our attention.
Is greed good?
The replication study, published Jan. 31 in Scientific Data, focused on the effect of greed on the relationship between wealth and good behavior, and tried to replicate the findings of the 2012 PNAS paper, “Higher social class predicts increased unethical behavior.”
The PNAS paper found that the socioeconomic elite were more likely to engage in or endorse lying, cheating, or stealing. The paper attributed that propensity to a belief, in the words of fictional corporate raider Gordon Gekko, from the 1987 film “Wall Street,” that “greed is good.” And by priming people to think that way, the researchers reported that they could induce their subjects to answer questions more unethically — more like rich people.
The report caught hold in the popular press around the world — Scientific American, Der Spiegel, and The Guardian all ran stories — and has received ongoing mentions in stories about similar studies concerning the moral lacunae of the economic elite. The paper has also gained traction in the field of psychology — it has been cited 174 times since 2012, according to Clarivate Analytics’ Web of Science.
Joshua Patenaude, an author on the replication attempt and a doctoral student at the University of Western Ontario (Western), told us:
It seemed like a sound study to replicate. It had a well done methodology and we thought we were primed to replicate it by increasing its sample size.
But while trying to replicate the 2012 paper’s findings on the effect of greed, the authors — all from Western — found a bit of math in the PNAS paper that didn’t quite look right. Patenaude told us:
We were running the numbers calculating everything from our study and their study and one [value] didn’t quite add up.
None of the values changed, it was just that one of the numbers was put in wrong.
In September 2016, the Western team contacted the authors of the 2012 study, including first author Paul Piff, now a professor at the University of California, Irvine, to let them know about the error.
Piff told us his team waited until April 2017 to contact PNAS about the error, attributing the delay to “the perceived extremely minor nature of the standard error typo.” He said the journal didn’t push for a correction:
they indicated that PNAS issues corrections only for changes that significantly impact the scientific conclusions of an article such that an informed reader would be misled without the correction.
In consultation with PNAS, and given their guidelines and the nature of the correction, we decided not to proceed with the correction at that time.
That’s since changed, and Piff and PNAS are again in contact to discuss publication of a correction to the paper.
The snake eats its own tail
On March 8, Piff returned the favor of pointing out a mistake. He found the authors of the replication study had inadvertently inverted their analysis to show that acting unethically was not associated with being rich. In fact, it was — weakly. Piff identified the rich-poor mix-up while reviewing the methods, data, and analysis Patenaude had shared online using the Open Science Framework.
Piff told us the errors in the replication study were “not a big deal.”
For a number of reasons, especially the inverted analysis mistake, the situation reminds us of a set of papers, now corrected, that originally linked conservative political beliefs to signs of “psychoticism.”
Though the Western researchers felt their results held, they decided to pursue a correction. Last author Lorne Campbell, a professor at Western, told us they did so “in the interest of accurate reporting.”
The Western team worked on the corrigendum, with Piff’s help, over the next month before requesting a correction on April 17. The journal offered a few suggestions on how to draft the notice, Patenaude said, and received the final draft of the corrigendum on July 25.
So on Sept. 5, the authors of “A 4-study replication of the moderating effects of greed on socioeconomic status and unethical behaviour” issued a corrigendum to their paper, admitting to making a mistake in their statistical analysis. They wrote:
there was an error in the SPSS syntax for the regression models analyzing the main effects.
In the notice, the authors explain that, even after correcting their data, they were still unable to validate the 2012 paper’s finding that greed had any effect on the relationship between wealth and unethical behavior.
Patenaude said the episode has been “highly informative” and characterized the episode as a win for the open science model. Concerning his team’s statistical mistake, he said:
I prefer it be found now than years later. It was a good thing.
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
So what can be concluded, if anything, from these retractions? Which of these possible conclusions can be drawn? The original study’s conclusion about the positive relationship between having money and lying, cheating, and stealing holds up, or the replication study’s conclusion that there is no relationship is correct, or there is a positive relationship but not as strong as stated in the original study?
It is important to make known which is the likely case for Retraction Watch and other audiences.
Then why the witch hunt? They should collaborate. This is an example of the narcissism of small differences. RE: “Although neither appear to affect the main conclusions…”
In the 2012 paper, the wrong standard error (SE) was typed in the text for the interaction term testing the main hypothesis. The regression coefficient and t value were correct, so the error was a typo. It was important for us to know the correct SE as we were using it (along with the regression coefficient) in a meta-analysis. The corresponding author of the paper was very helpful in this regard, and provided feedback about our replication efforts as we developed the project.
In the 2016 paper we essentially ran 2 models: (1) a main effects only model (with no interaction term) in order to document the coefficients of the 2 predictor variables (these values are conditional main effects when the interaction term is entered), and (2) the model of interest testing the interaction term (it was the interaction between the two variables that we were attempting to replicate). The two primary variables of interest were (1) a continuous measure of socioeconomic status (SES), and (b) a code indicating membership in the control or experimental condition. In the main effects model only, the non-reverse coded SES variable was entered as the predictor variable; therefore, the sign of main effect was interpreted in a manner not consistent with the original paper. In the model testing the interaction term (the coefficient of interest in terms of replicating the prior research), the reverse coded SES variable was used and the results reported were consistent with the interpretation of original research (though we did not find evidence supporting a significant interaction between SES and condition). That is why neither error affected the main conclusions of either paper.
Thanks, Lorne. This explanation tells me what I need to know. These exchanges on Retraction Watch are very helpful.
From what I read, likely neither. Once all corrected, it all looks like good science to me. (And to satisfy your curiosity, the relevant in the corrigendum is: ” In the first sentence, the relationship between SES and unethical behaviour was incorrectly reported with a negative sign (b=−0.023, s.e.=0.028, t(305)=−0.804). The correct values are b=0.023, s.e.=0.028, t(305)=0.804. The relationship remained non-significant.” You could have executed this one-click task yourself.)
As an author on one of the articles, I think this write-up makes it look like there was a witch hunt, when in fact there was not. Everything was done collaboratively. They made a small error, and were happy to correct it. We made a small error, and were happy to correct it. That’s it.
⋲[Making Money ∴ Rich].[!Mean] is the problem space to explore! There’s a strong bias in that phrasing putting it on the last trade (divesting however much) to have represented nn years of investing in enterprise (or at least their trading frontages) and protecting the value of all shareholders when chicanery is afoot (with or without a distribution to clock in,) which is unfair; at least measure some deep market chop (looming recession, city shrinkage, etc.) across some useful MT work.
As I noted in a letter to PNAS shortly after the original study appeared. The findings by Piff et al. (2012) seem “too good to be true.” If we (optimistically) take their original results seriously, then the probability of all seven of their experiments showing statistically significant results is estimated to be around 0.02. Details are at
http://www.pnas.org/content/109/25/E1587.full
Piff et. al provided a response to my letter
http://www.pnas.org/content/109/25/E1588.full
but it basically suggests that they do not understand what was done, or why their own results are unbelievable. I provide a response at
http://www2.psych.purdue.edu/~gfrancis/Publications/FrancisRebuttal2012.pdf
Scientists should not trust the findings reported in Piff et al. (2012). The replication study seems to validate this conclusion.
The ramifications of an article about a very hot button issue are immense in the pop news world where so many busy people get their info. Authors may believe the article stays within the “in group”. But the title makes clear that the article would be widely read in the pop press and the suspicion remains that this was indeed the original intention of the authors.