Widely cited COVID-19-masks paper under scrutiny for inaccurate stat

You probably read a story or heard a news report over the past few days saying that if nearly all Americans wore masks to prevent COVID-19 spread, 130,000 lives could be saved by the end of February. That’s what a paper published on Friday says.

But it turns out that figure sounds twice as good as reality. Here’s the story:

On October 6, a group at the Institute for Health Metrics Evaluation (IHME) — a frequently cited source of COVID-19 data — submitted a manuscript to Nature Medicine. The paper was accepted on October 13, and published on October 23. It concluded:

We find that achieving universal mask use (95% mask use in public) could be sufficient to ameliorate the worst effects of epidemic resurgences in many states. Universal mask use could save an additional 129,574 (85,284–170,867) lives from September 22, 2020 through the end of February 2021, or an additional 95,814 (60,731–133,077) lives assuming a lesser adoption of mask wearing (85%), when compared to the reference scenario.

That conclusion led to more than 100 headlines around the world, including “The Price for Not Wearing Masks: Perhaps 130,000 Lives” (New York Times), “Universal Mask Wearing Could Save Some 130,000 Lives In The U.S., Study Suggests” (NPR), and “Universal mask wearing could save almost 130,000 lives from coronavirus, epidemiologist says” (CNBC).

There was only one problem: The figures that projection was based on were already out of date by the time the paper was published. The paper refers to 49% of Americans saying they wore masks as of September 21, meaning that a change to 95% would be a near doubling. But as Phil Magness, an economic historian, wrote in a letter to the journal yesterday: 

A review of this source indicates, however, that public mask use for the United States sat at a significantly higher rate of 68% as of 21 September, the stated date. This higher number is also consistent with more recent survey data, suggesting U.S. mask usage in public spaces has consistently hovered between 75 and 80% since mid-July 2020 – a figure much closer to the IHME’s own targeted mask compliance rates. [4]

That would mean the difference between current mask use and 95% would be much lower, and the additional benefit of near-universal mask wearing — while still substantial — would mean fewer than 130,000 lives saved.

Even an IHME spokesperson, in comments to Retraction Watch, acknowledged that 63,000 was a more up-to-date projection:

The paper you reference is based on data from the third week of September and does not include the most recent findings based on our research and modeling. That is because of time required for the peer-review process.

IHME’s most current release of data – from 22 October – finds that “approximately two-thirds of Americans are wearing a mask outside the home, and this level has remained constant over the last two weeks.” For more information, please see our most recent policy brief, also dated 22 October: http://www.healthdata.org/sites/default/files/files/Projects/COVID/briefing_US_20201022.pdf

From that brief:

Expanding mask use to 95%, the level seen in Singapore, can greatly delay the imposition of mandates and save 63,000 lives.

While that October 22 briefing came out the day before the study did, news reports about the study — many of which quote IHME scientists — quote the 130,000 figure, as it appears in the paper.

Early this morning, Nature Medicine editor in chief Joao Monteiro told Retraction Watch he had not received Magness’ letter yet but would reach out to the IHME authors to assess the situation.

In the meantime, Magness told Retraction Watch:

I’m honestly even more baffled now by their reply. Even though their newest release shows about 68% mask use, mask rates hovering around at least that level has been the case for several months now. The YouGov survey, which they list among their sources, first topped 70% on June 22, and has been in the high 70s/low 80s consistently since July 14.

If they were still using 49% for a paper dated around September 21, then it appears to be an error of using numbers that were already more than 3 months out of date at that time.

It would still seem to warrant a correction or update of some sort to reflect the new data, as the 49% statistic is receiving massive media coverage at the moment based on the October 22 paper release, which erroneously presents that number as accurate as of September 21. I can appreciate the challenges created by the slow pace of peer review in a fast moving pandemic, but that also requires scientists to use an abundance of care when making empirical claims before the public about ever-changing data.

In any case, a public acknowledgement from the IHME group and the journal is probably necessary to temper the widespread reporting about the older statistic in the press.

You might say that a bad stat traveled halfway around the world before the truth got its mask on.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].

24 thoughts on “Widely cited COVID-19-masks paper under scrutiny for inaccurate stat”

  1. Questionable math is becoming all to common. What these calculations, even when accurate, failed to take into account is that contracting SARS-CoV-2 poses no danger to most of the population.

    1. Not true.

      Current near term projections say that about 10% of all who contract COVID will require hospitalization, and nearly all of those who contract COVID suffer long term organ damage. Just this week it came out that 63% of all persons hospitalized with COVID suffer permanent heart damage – when most thought (wrongly) that lung damage was the only effect of COVID.

      Any activity or disease that causes thousands of cases per day of permanent lifetime heart or lung damage, above and beyond the death rate of about 800 per day as of this week – would be considered by any sane person as a danger to them.

      1. Parroting unverified claims by the same media that will most likely not correct the errors this very article points out is nothing but fear-mongering.

        1. Then please provide your better data and sources. Trashing best available data without supplying equal or better is simply denial or worse, actively promoting an agenda, not objective science.

      2. “Current near term projections say that about 10% of all who contract COVID will require hospitalization”

        What do you mean by “contract COVID”? If we are to listen to the mass media, every positive test is a case. But it’s ridiculous to think that one of every 10 people with a positive test end up in the hospital.

        Even if all the stats you are citing are technically true, it would be misleading if they don’t account for the learning that has taken place with regard to how to prevent and treat COVID-19. Some doctors are ignoring the apparent illegality of prescribing HCQ plus zinc and azithromycin, and are experiencing extremely high cure rates as a result. That’s right, I wrote CURE rates — not just the typical FDA drug pusher TREATMENT rate, which keeps the patient coming back for another fix. Ivermectin and other drugs as well have an extremely high success rate if given early. Other learning has to do with NOT putting patients on ventilators — defying the perverse incentive that was set up months ago of being given $39,000 additional per “vent”.

        No doubt someone here will dismiss all this, calling it anecdotal. Knock yourself out. Thousands of collective patients walking away happy after seeing five or 10 collective doctors is NOT anecdotal. On the other hand, “anecdotal” IS an accurate description of the MSM scare tactic of focusing on one 24-year-old woman with no obvious comorbidities who had serious problems from COVID.

        1. This is a surprisingly large figure. Where do you get it from?

          I note that ‘projections’ is a word frequently used by people using the most extreme outputs from an unproven model for political purposes…..

      3. “Any activity or disease that causes thousands of cases per day of permanent lifetime heart or lung damage, above and beyond the death rate of about 800 per day as of this week – would be considered by any sane person as a danger to them.”

        You ignored the part about “most of the population”. Why did you do that?

      4. “Projections”????? Projections are not scientific facts. They are just opinions dressed up as such. Educated people know this. Uneducated people don’t.

        “63% of those hospitalized suffer long term organ damage”? Ummmm, since this disease has not been around for a full year this statement is a joke.

      5. As I recently posted in WSJ forum:

        Based on current population of 7,825,000,000 (almost 8 billion), the recent WHO infection rate estimate of about 10% or 782,500,000 (over 3/4ths of a billion), and fatalities attributed to COVID-19 to date of about 1,283,000, the IFR (Infection Fatality Rate) is at about 16 basis points, or 16 one-hundredths of one percent.

        A 16 basis point IFR is non-zero, non-trivial, and far lower than widely understood.

        Importantly, both the CFR (Case Fatality Rate) and IFR (Infection Fatality Rate) are declining as familiarity with and ability to limit the spread of , and treat the disease, increase through time.

    2. Playing Russian Roulette poses no danger to most of the population. 5/6 of players are absolutely fine afterwards.

      1. Wow—seriously? I should probably not decimate this absurd conflation, but if some people find it compelling then you’ve contributed to misinformation on an issue that requires clarity. So here we go.

        1. Your gun has waaay too few chambers.

        I’m not pulling up the actual numbers because I don’t need to do so to make my point.

        Let’s assume a hypothetical population—let’s call it the Nation of Covidia. In Covidia, nobody has comorbidities and is uniformly aged such that each individual’s risk of dying is 99.7%. There are 300,000,000 people living there, and each person is exposed to that risk in turn in a weird game of “COVID Roulette”, where an auto-injecting syringe dart is randomly chambered into a revolver-style syringe dart shooter, and everyone passes it around.

        So let’s define some numbers:

        300,000,000–(.997 • 300,000,000) = ~900,000 deaths in 300,000,000 trigger pulls is what we can expect—or simplified, about 1 attempt in every 333 will result in a fatal shot, on average.

        That means the gun, in order to have just one “live” chamber, must have 333 chambers. So your turn comes up, and you have a 1-in-332 chance of surviving to pass it to your neighbor.

        Or, put as you put it, 332/333 players are absolutely fine afterwards.

        On the other side of that are the negative consequences of “avoiding” the risk.

        None of the measures are perfect, but you’ve got a choice between living in a world where you are isolated from human contact, you can’t go out without first covering your face and imposing face coverings on all of your fellow Covidians, even those who really don’t like wearing them for one reason or another. The arts are on permanent hold, there are no sports, and businesses the Covidian ruler decides are non-essential must close.

        Covidia’s finest, recently under intense scrutiny for inappropriately violent enforcement behavior, go armed with guns that contain traditional metal-slug-type bullets in semiautomatic clips of 12 to enforce Covid Roulette safety measures.

        Additionally, hospitals only accept people who who have already been shot dead with the Covid Revolver, so if you get cancer, or need heart surgery, or need to have your stomach pumped because of a drug/alcohol overdose, or because your spouse is abusive, or because you’re so depressed you can’t go on any longer, you’ll need to suck it up buttercup, because the Covid Revolver’s victims or possible victims are prioritized….need I go on?

        Oh, and eventually everyone has to play, no matter what, so all that stuff is sort of maybe delaying your inevitable turn with the Covid Revolver—nobody knows for sure, or if they do know, you wouldn’t know, because the media and politicians in Covidia don’t want to risk you learning that nothing they’ve done accomplished anything but the piling on of misery beyond that imposed by the game itself, so they censor any information that might clue you into the fact that they haven’t increased your likelihood of surviving Covid Roulette or prevent the certainty that you will ultimately have to play the game.

        So the real question is, would you rather the government intervene with more misery-creation to enhance your inevitable turn with the 333-chambered revolver, or would you rather just get it out of the way quickly with as little disruption to society as possible?

        Personally, I prefer my Covid Roulette sans the side of heaped-on misery. Happily, viruses are nothing like single-loaded-chamber-revolvers, society is made up of unique individuals with different risk tolerances and values, and most people who encounter the virus actually don’t get sick at all, while the vast majority that do survive it.

        I hope that clears up how bad your comparison was, and that you’re sufficiently ashamed to stop using it to troll people.

        To readers: please feel free to use this response without attribution if you want to respond to anyone else who raises this feeble, pernicious little weasel of an excuse for an argument.

        1. “Andzac-Inurfaise”
          Nice nym.

          “I’m not pulling up the actual numbers because I don’t need to do so to make my point.”
          Your argument on relative risks and costs/benefits would work better if you looked up real statistics instead of making some up.

  2. It’s hard to say that a paper based on self-reported data is reliable. Next, you’ll try to tell me that no one has ever lied about their diet.

  3. Sounds like yet another GIGO mathematical model..

    Now here is the interesting thing about the published literature from Asian researchers over the last few decades on face-masks and airborne upper respiratory infectious diseases. Apart from one very specific case with N95 masks and URI’s that cause sneezing and wet coughs there was no strong statistical effect measured for wearing facemasks outside of a home which had infected people in it. Ever.

    Now as SARs CoV2 infections cause dry coughs so mostly < 2 micron respiratory aerosols are transmission mode and almost everyone is wearing a N0 (cloth) or N20 (when dry) mask how can something that is a zero effectiveness physical barrier against airborne human corona-viruses "save" hundreds of thousands of lives. When the IFR is around 0.2% with risk only to those with high PSI/PORT scores.

    If this is supposed to be high quality scientific research I would not like to see the really low quality stuff.

  4. give me a break-there is no way 68% of US is wearing masks. even in the northeast–i would say this is close to 50% at best. who ran these crap surveys–do these people say yes when they are carrying a mask in their pocket?

  5. “But it turns out that figure sounds twice as good as reality.” This to me is the main problem of the entire Corona story. There are layers upon layers in this story consisting of nothing but shoddy (or absent) science and of unanswered questions. The endless debate about the effectiveness of masks is just one symptom of the situation. It is for example impossible to receive an official response to questions re test conditions of the Corona PCR test. Threshold values/ cycle numbers/verification of results by PCR independent methods are closely guarded secrets. The same goes for questions re patients with Covid. How many other viruses potentially relevant for their symptoms apart from SARS-CoV2 do these patients have? Has this ever been tested routinely? Apart from care in ICUs what exactly is the care a hospital provides in a ‘normal’ ward to a person with symptoms of Covid? In what way does this go beyond the care such a person could receive at home? In order to be able to understand reality with regards to Corona we would need answers to these and many more questions. But answers are not forthcoming. The entire story is shrouded in mystery and instead we get presented with a bunch of models with claims that cannot be tested.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.