Oft-quoted paper on spread of fake news turns out to be…fake news

*See update at end of post

The authors of an much-ballyhooed 2017 paper about the spread of fake news on social media have retracted their article after finding that they’d botched their analysis.

The paper, “Limited individual attention and online virality of low-quality information,” presented an argument for why bogus facts seem to gain so much traction on sites such as Facebook. According to the researchers — — from Shanghai Institute of Technology, Indiana University and Yahoo — the key was in the sheer volume of bad information, which swamps the brain’s ability to discern the real from the merely plausible or even the downright ridiculous, competing with limited attention spans and time.

As they reported:

Our main finding is that survival of the fittest is far from a foregone conclusion where information is concerned.

Put another way:

quality and popularity of information are weakly correlated …

Not surprisingly, the article, which appeared in Nature Human Behaviour, received considerable media attention, including coverage in Smithsonian,  Ars Technica and many other outlets. And it has continued to serve as fodder for articles about the spread of lies in cyberspace.

But as the retraction notice, dated January 7, 2019, indicates, the study had major flaws. It turns out that fake news does not spread as wildfire-y as the real McCoy:

The authors wish to retract this Letter as follow-up work has highlighted that two errors were committed in the analyses used to produce Figs 4d and 5.

In Fig. 4d, a software bug led to an incorrect value of the discriminative power represented by the blue bar. The correct value is τ = 0.17, as opposed to the value τ = 0.15 reported in the Letter.

In Fig. 5, the model plot was produced with erroneous data. Produced with the correct data, the authors’ model does not account for the virality of both high- and low-quality information observed in the empirical Facebook data (inset). In the revised figure shown below (Fig. 5), the distribution of high-quality meme popularity predicted by the model is substantially broader than that of low-quality memes, which do not become popular. Thus, the original conclusion, that the model predicts that low-quality information is just as likely to go viral as high-quality information, is not supported. All other results in the Letter remain valid.

The retraction appears to be the first for the journal, which debuted two years ago this month.

Gordon Pennycook, a behavior researcher at the University of Regina in Saskatchewan, Canada, who has studied the phenomenon of fake news, tells Retraction Watch:

If there is a lesson it’s that we should be mindful of double (and triple… and quadruple) checking our analyses. The pressure to publish is very strong, but no scientist wants to find themselves in this position. Having said that, I think the authors should be commended for retracting the paper.

So, while it may indeed be “true” that a lie is halfway around the world before the truth gets its pants on, we still don’t know why.

Update, 1715 UTC, 1/10/19: We have replaced the image on this post. Originally, and consistent with images we’ve used since our launch in 2010, the image was the cover of the current issue of Nature Human Behaviour. However, authors of a study in that issue objected and asked us to replace the image, with one author saying that they “would rather not be associated with fake news.” Although we do not feel that the use of the original image did that, we have replaced the image as a courtesy.

Hat tip: Rolf Degen

Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at team@retractionwatch.com.

6 thoughts on “Oft-quoted paper on spread of fake news turns out to be…fake news”

  1. Surely the paper is not retracted because of an error that makes 0.15 into a 0.17. By making this the only quotable numbers in the retraction, the author makes it appear as if this is the main problem with the paper. Frankly speaking, it looks like spin.
    It would be more honest to give the numbers for what appears to be the main problem.

  2. Hi, I just wanted to let you know that you are using an unrelated image for this article. This is the cover illustration for one of my papers on exploration and generalization in human learning (https://www.nature.com/articles/s41562-018-0467-4). We would rather not be associated with fake news.

    Would you consider our reasonable request to replace it with a more appropriate image? We feel that it is rather irresponsible to use the first random picture you find on the internet for an article about fake news. Especially since we have done our best to follow open science practices by openly published the data and code.

    – Charley Wu
    Max Planck Institute for Human Development

    1. Thanks for the feedback. Please see our update at the end of the post:

      Update, 1715 UTC, 1/10/19: We have replaced the image on this post. Originally, and consistent with images we’ve used since our launch in 2010, the image was the cover of the current issue of Nature Human Behaviour. However, authors of a study in that issue objected and asked us to replace the image, with one author saying that they “would rather not be associated with fake news.” Although we do not feel that the use of the original image did that, we have replaced the image as a courtesy.

  3. As an author of the retracted study, I would like to point out that the following statement in this post is inaccurate or misleading:

    “It turns out that fake news does not spread as wildfire-y as the real McCoy”

    This suggests that there was a problem with the empirical data presented in the paper, which is not the case. In fact, the empirical finding about the distribution of popularity for fake news articles on Facebook holds.

    The error that led to the retraction was in the analysis of our model, not the empirical data. Our model does not reproduce those empirical curves, despite the fact that the model does predict low correlation between popularity and quality. Therefore, while finite attention and information overload may play a role in the virality of low-quality information, according to our revised results, they alone are not sufficient to explain the patterns of virality observed in the empirical data. The literature has since pointed to other factors such as novelty and social bots, which also play important roles.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.