A 2017 paper, when originally published, had a fairly clear message: People who got the flu vaccine every year were no less protected than someone who had skipped last year’s dose. But now that it’s been retracted, the picture is somewhat less clear.
The retraction notice in BMC Medicine doesn’t provide much information — it simply says the authors included and omitted some information that affects the conclusions.
Last author Bryna Warshawsky, the medical director of communicable diseases at Public Health Ontario in Canada, provided some additional explanation to Retraction Watch — namely, that after addressing errors in the analysis, the researchers found that there were actually slight differences; specifically, for some strains of the flu, the new analysis suggests that the flu shot was slightly more effective in people who’d skipped last year’s dose.
Warshawsky told Retraction Watch she and her team have submitted the corrected paper to the journal, and the bottom line message stays the same:
From the patient perspective, getting vaccinated in the current season generally confers equal or greater protection against all influenza types/subtypes than not getting vaccinated that season, regardless of one’s past vaccination history.
“The impact of repeated vaccination on influenza vaccine effectiveness: a systematic review and meta-analysis” has been cited nine times since it appeared last year, according to Clarivate Analytics’ Web of Science.
There is some legitimate debate over the flu vaccine — although it doesn’t offer 100% protection against the flu, many experts still recommend getting it each year, since it does have some effect, with little risk. Here’s from a good backgrounder piece in Vox from 2015:
“Depending on the season,” explained Tom Jefferson, an author on these Cochrane reviews [on the topic], “you need to vaccinate anywhere between 33 and 100 people to avoid one set of symptoms.” In a good year, when the [World Health Organization] guesses correctly and the flu shot matches the strains in circulation, you need to give 33 adults flu shots, on average, to prevent one case of illness. In a year when the WHO guesses badly, you need to vaccinate 100 people to prevent one flu case….On the other hand, the evidence suggests that there’s no harm in getting a flu shot, either. The meta-analyses found no evidence of serious side effects beyond the odd case of soreness at the injection site.
Given the ongoing discussion of the topic, the original BMC Medicine paper received some press coverage; Warshawsky has also spoken to media about the benefits of getting the flu shot every year. She was also the chair of the Canadian National Advisory Committee on Immunization.
“The fact that this article was about vaccine effectiveness did not play into our decision”
Warshawsky told us an outside researcher alerted them to the issues in the paper:
We were contacted by a colleague who was reviewing the available evidence on the impact of repeated influenza vaccination. The colleague asked us about a few of the estimates that had been included in our meta-analysis. This prompted us to review our findings again….we detected four errors. Although the absolute differences were small, incorporating the corrected results impacted the conclusions of two analyses.
She shared more about the nature of the errors:
As per accepted practice for systematic reviews, we had two co-authors review and abstract data for each study. Nonetheless, on subsequent review, we detected three instances from three different publications where we misinterpreted the findings and included an estimate that should have been excluded from the meta-analysis, and one instance of an estimate to include which was reported in a supplementary graph.
When the authors compared the vaccine’s effectiveness between people who got it last year and people who didn’t, they originally saw that, for some flu strains, the people who skipped last year’s shot had slightly more protection, but the difference was not statistically significant. But upon reanalysis, that slight difference became statistically significant:
Instead of publishing an erratum, the journal has invited us to resubmit our article with these new results after retracting our original publication.
Researching vaccine safety can be fraught; as a recent op-ed in the New York Times noted, researchers often hesitate to speak critically about the field out of fear people will forgo vaccines altogether:
Scientists are so terrified of the public’s vaccine hesitancy that they are censoring themselves, playing down undesirable findings and perhaps even avoiding undertaking studies that could show unwanted effects. Those who break these unwritten rules are criticized.
We asked Warshawsky if this concern prompted her to hesitate over retracting her paper:
The decision to retract the article was based only on reporting the most correct version of results and ensuring an accurate application of the stated methodology. The fact that this article was about vaccine effectiveness did not play into our decision; it was solely based on acknowledging the error and correcting it in order to maintain the integrity of the published literature.
Hat tip: Rolf Degen
Like Retraction Watch? You can make a tax-deductible contribution to support our growth, follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up for an email every time there’s a new post (look for the “follow” button at the lower right part of your screen), or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].