…the authors predict that a ratio of positive to negative affect at or above 2.9 will characterize individuals in flourishing mental health.
The paper made quite a splash. It has been cited 360 times, according to Thomson Scientific’s Web of Knowledge, and formed the basis of a 2009 book by Fredrickson, Positivity: Top-Notch Research Reveals the 3 to 1 Ratio That Will Change Your Life.
But something didn’t sit right with Nick Brown, a psychology grad student at the University of East London. He found the paper’s claims wanting, and contacted Alan Sokal — yes, that Alan Sokal, who published a fake paper in Social Text in 1996. Sokal agreed, and he, Brown, and Harris Friedman published a critique of the paper in July of this year in American Psychologist. Its abstract:
We examine critically the claims made by Fredrickson and Losada (2005) concerning the construct known as the “positivity ratio.” We find no theoretical or empirical justification for the use of differential equations drawn from fluid dynamics, a subfield of physics, to describe changes in human emotions over time; furthermore, we demonstrate that the purported application of these equations contains numerous fundamental conceptual and mathematical errors. The lack of relevance of these equations and their incorrect application lead us to conclude that Fredrickson and Losada’s claim to have demonstrated the existence of a critical minimum positivity ratio of 2.9013 is entirely unfounded. More generally, we urge future researchers to exercise caution in the use of advanced mathematical tools, such as nonlinear dynamics, and in particular to verify that the elementary conditions for their valid application have been met.
Fredrickson responded in another paper in American Psychologist:
…I draw recent empirical evidence together to support the continued value of computing and seeking to elevate positivity ratios. I also underscore the necessity of modeling nonlinear effects of positivity ratios and, more generally, the value of systems science approaches within affective science and positive psychology. Even when scrubbed of Losada’s now-questioned mathematical modeling, ample evidence continues to support the conclusion that, within bounds, higher positivity ratios are predictive of flourishing mental health and other beneficial outcomes.
But in the wake of the critique, Discover blogger NeuroSkeptic called for the Fredrickson-Losada paper to be retracted. The Chronicle of Higher Education covered the story in some detail in early August.
Now, the Fredrickson-Losada paper has been partially withdrawn. Here’s the notice, which appeared on September 16:
Reports an error in “Positive Affect and the Complex Dynamics of Human Flourishing” by Barbara L. Fredrickson and Marcial F. Losada (American Psychologist, 2005[Oct], Vol 60, 678-686). The hypothesis tested in this article was motivated, in part, by the nonlinear dynamic model introduced in Losada (1999) and advanced in Losada and Heaphy (2004) and herein (Fredrickson & Losada, 2005). This model has since been called into question (Brown, Sokal, & Friedman, 2013). Losada has chosen not to defend his nonlinear dynamic model in light of the Brown et al. critique. Fredrickson’s (2013) published response to the Brown et al. critique conveys that although she had accepted Losada’s modeling as valid, she has since come to question it. As such, the modeling element of this article is formally withdrawn as invalid and, along with it, the model-based predictions about the particular positivity ratios of 2.9 and 11.6. Other elements of the article remain valid and are unaffected by this correction notice, notably (a) the supporting theoretical and empirical literature, (b) the data drawn from two independent samples, and (c) the finding that positivity ratios were significantly higher for individuals identified as flourishing relative to those identified as nonflourishing.
We asked Sokal what he thought of the move:
I would say that it is a positive step, but it still leaves some key issues unresolved. That is because Fredrickson’s response to our paper fails to make clear which claims of the Fredrickson-Losada paper she is withdrawing and which ones she is reaffirming; and the brief withdrawal notice partially compounds the confusion.
In the 2005 paper and in her 2009 book, Fredrickson asserted that a discontinuous phase transition — analogous to the phase transition between liquid water and ice — occurs when the positivity ratio passes through the value 2.9013 (in her book she most often rounded this off to 3). The only reason for entertaining such a radical claim was the nonlinear-dynamics model, which Fredrickson and Losada have now officially withdrawn. Nevertheless, Fredrickson insists in her 2013 response that “Whether the outcomes associated with positivity ratios show discontinuity and obey one or more specific change points, however, merits further test.”
Now, one could conceivably argue that any hypothesis, no matter how implausible a priori, “merits further test”; so this sentence may simply be an unobjectionable way of saving face. But Fredrickson is apparently not yet prepared even to abandon the attempt to model the time evolution of human emotions using the Lorenz equations: “Whether the Lorenz equations — the nonlinear dynamic model we’d adopted — and the model estimation technique that Losada utilized can be fruitfully applied to understanding the impact of particular positivity ratios merits renewed and rigorous inquiry.”
It would therefore be valuable to know whether Fredrickson’s published response to our paper represents her current opinion, or whether she has now abandoned also the attempt to model the time evolution of human emotions using the Lorenz equations and/or the attempt to find discontinuous phase transitions.
Finally, in their 2005 paper (p. 684) Fredrickson and Losada claim that their two samples provide empirical support for a phase transition at 2.9:
“More critical to our hypothesis, however, in each sample, these mean ratios flanked the 2.9 ratio.”
“Supporting the hypothesis derived from Losada’s (1999) nonlinear dynamics model, we found in two independent samples that flourishing mental health was associated with positivity ratios above 2.9. … The relationship between positivity ratios and flourishing appears robust …”
Now, it is very easy to demonstrate that their empirical data do not show anything of the kind — and indeed that, given their experimental design and method of data analysis, no data whatsoever could possibly give any evidence of any nonlinearity in the relationship between “flourishing” and the positivity ratio — much less evidence for a sharp discontinuity. For lack of space we omitted from our paper such a discussion, but we will be happy to provide it if the editors of American Psychologist give us the opportunity to respond to Fredrickson’s comment on our paper (thus far they have refused).
The partial withdrawal notice says that
“Other elements of the article remain valid and are unaffected by this correction notice, notably … (b) the data drawn from two independent samples, and (c) the finding that positivity ratios were significantly higher for individuals identified as flourishing relative to those identified as nonflourishing.”
But what about the claim that their data provide empirical evidence for a phase transition at or near 2.9? Are they withdrawing this claim as well, or not? Both Fredrickson’s response and the withdrawal notice are unclear on this point.
Brown also gave us some thoughts about “some very basic issues with the empirical part of the Fredrickson and Losada (2005) paper:”
- The datasets analysed were, by Fredrickson’s own admission, taken from studies that had been conducted for some other purpose and were analysed post hoc; see http://www.youtube.com/watch?v=jvPHF3u5zL8 from about 29:30 for about four minutes (although the whole video is worth watching to get an insight into Fredrickson’s understanding of the math here). Now, this doesn’t have to be a problem on its own, but it would have been good practice for the authors to declare it; if you read the relevant passage from the paper, you will see that it is very carefully worded.
- Of the two datasets, one (Study 2) does not, in fact, achieve significance; the authors state a result of “t(99) = 1.62, p=.05″, but in fact the (one-tailed) p-value of that t-test is .0542, and you don’t get to round down and then state that your number doesn’t exceed .05!
- Continuing the above, the authors give no justfication for using a one-tailed test. It seems to me to be rather unusual to use a one-tailed test to determine whether the means of two groups are different, unless there are extremely powerful theoretical or logical reasons for believing that one mean must be greater than the other (for example, if you’re measuring a population that starts off fixed and declines over time). In this case, however, it appears that the only possible justification for such a belief, and hence for using a one-tailed test, is the assumption that the theory of the critical positivity ratio is correct, which is the very theory being tested by the experiment.
We will be addressing these issues in our reply to Fredrickson, whether that appears in AP or another journal. Even if the empirical work were completely spotless, however, there would remain the question of just how a paper can be allowed to stand when approximately 60% of it, by word count, has been marked by the first author as “invalid”. I’m sure you’ve seen plenty of articles completely retracted for erroneous content occupying one-tenth as much of the whole. Clearly the 340+ scholarly citations of this article are not because of two small empirical studies of undergraduates that produced the unsurprising result that people who have flourishing lives also express more positive emotions; its popularity results from its substantial (not to say grandiose) claims to general truth. Fredrickson’s correction does little to address this, unless AP is planning to re-issue the PDF of the article with “Corrected” stamped across every page.
Sokal told us he thinks the case raises other issues:
Last but not least, there is a huge open question, which concerns not Fredrickson and Losada but the entire psychology community, and particularly those people working in “positive psychology”. How could such a loony paper have passed muster with the reviewers at the most prestigious American journal of psychology, netted 350 scholarly citations, and been repeatedly hyped by the “father of positive psychology” (and past president of the APA), without anyone calling it into question before a first-term part-time Masters’ student in Applied Positive Psychology at the University of East London came along and expressed his doubts? Where were all the leaders in the field of positive psychology? The leaders in the application of nonlinear-dynamics models to psychology? Was everyone really so credulous? Or were some people less credulous but politely silent, for reasons of internal politics?
Hat tip: Dale Barr