A journal has issued expressions of concern for a pair of 2021 meta-analyses purporting to find that ivermectin is an effective treatment for Covid-19 after data sleuths raised questions about some of the research in the studies.
As we reported last fall, one of the two papers – “Ivermectin for Prevention and Treatment of COVID-19 Infection: A Systematic Review, Meta-analysis, and Trial Sequential Analysis to Inform Clinical Guidelines” – began to wobble when data central to its conclusion were retracted from the journal Viruses. That article has been cited 37 times, according to Clarivate Analytics’ Web of Science, making it a highly-cited, “hot” paper.
The other article was titled “Review of the Emerging Evidence Demonstrating the Efficacy of Ivermectin in the Prophylaxis and Treatment of COVID-19” and was written by a group led by Pierre Kory. Kory is a controversial Wisconsin physician whose ideas about how to treat the infection, and particularly ivermectin, have made him a darling of ivermectin proponents like Joe Rogan.
Kory’s group lost a different Covid-19 paper last November over problems with the data, and a paper similar to the one now subject to an expression of concern was removed from a Frontiers journal last year.
The two meta-analyses were the subject of an editorial in the November/December 2021 issue of the journal by its editor, Peter Manu, who cautioned that:
The ivermectin reviews published in the American Journal of Therapeutics do not contain anything that should keep people from vaccination.
The expressions of concern – both of which cite a paper in Nature Medicine by Jack M. Lawrence, Gideon Meyerowitz-Katz, James A. J. Heathers, Nicholas J. L. Brown, and Kyle A. Sheldrick detailing the issues with work on ivermectin – are nearly identical, save for a critical difference. The one for “Review of the Emerging Evidence,” by Kory and colleagues, reads:
The decision is based on the evaluation of allegations of inaccurate data collection and/or reporting in at least one of the primary sources of the meta-analysis contained in the article. These allegations were first made after the publication of this article. The exclusion of the suspicious data appears to raise questions regarding ivermectin’s potential to decrease the mortality of COVID-19 infection. Currently, the investigation of these allegations is incomplete and inconclusive.
The primary source in question here is a 2021 study in the Asian Pacific Journal of Tropical Medicine by researchers in Iran titled “Ivermectin as an adjunct treatment for hospitalized adult COVID-19 patients: A randomized multi-center clinical trial.” Lawrence and his fellow sleuths said those findings appear to be marred by “unexpected stratification across baseline variables” – another way of calling them “implausible.”
The notice for the other meta-analysis, from a group in the United Kingdom led by Andrew Bryant and including Theresa Lawrie, goes a step further, stating:
The exclusion of the suspicious data appears to invalidate the findings regarding ivermectin’s potential to decrease the mortality of COVID-19 infection …
Both notices, which at the moment are not mentioned on the original articles, go on to state that the journal’s action:
does not imply that the methodology used [by the authors] was incorrect. The use of summary data published by others is a generally accepted approach in biomedical metanalytic research.
To be sure, meta-analyses are important tools for scientists – but need to be updated in light of new evidence or evidence that underlying data are flawed.
Bryant did not immediately respond to a request for comment. However, in a letter to the editor of the journal in the September/October 2021 issue, he and his colleagues said they would:
issue a comprehensive correction to our meta-analyses will be issued if the data are found to be unreliable
But they insisted that the loss of the Elgazzar data would not be fatal to their conclusions:
The claim that conclusions are “entirely reversed”2 cannot be sustained on the evidence.
We asked Manu why he issued expressions of concern rather than retract the articles outright:
My decisions were justified and referenced in the text of the EoCs. The investigations of the allegations regarding work made public by Elgazzar et al and Niaee et al are currently incomplete and inconclusive. I am ready to revise my decisions as soon as final adjudications become available.
Manu added that he couldn’t give a timeline for the verdict:
I have no idea when the investigations of these allegations will be completed. As far as I know, the Asian Pacific Journal of Tropical Medicine has not retracted the Niaee article. With regard to Elgazzar et al I am still waiting for the report of the investigation conducted at Benha University in Egypt. I have not been contacted by anyone with access to complete Individual Participant Data used in these reports.
Heathers told us:
Taking any action at all when clear and obvious unreliabilities exist within a paper is a good start, at least. So often, this sort of second-order problem – a paper that relies crucially on other sources later found unreliable – are simply ignored. They’re also less prominent. Everyone remembers the Wakefield retraction being consequential and taking more than a decade, but few remember the Rimland and McGinnis paper on the same topic, which cited it prominently and remained un-retracted for even longer.
Both of these papers now with EOCs obviously rely on evidence that is demonstrably unreliable. But journals often do not see themselves as arbiters of a paper’s status of reliability. You can make the argument that this is because editors start from a baseline of being scrupulous, officious, and overworked, and are then thrust into a difficult and consequential process. You can also make the argument that editors are lazy cowards who take no scientific or moral responsibility for their output, and would prefer to slow-walk or ignore obvious problems because it’s easier than dealing with them.
And while both are obviously true on occasion, you rarely get to find out which applies to any given scenario.
Like Retraction Watch? You can make a one-time tax-deductible contribution by PayPal or by Square, or a monthly tax-deductible donation by Paypal to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
Where were the expressions of concern published? I would like to see the primary articles. Thanks.
Links to the expressions of concern and original articles are all included in the post.
The funny part at this reviews, is that hit hard on one specific medicine but never at vaccines.
Where is all peers reviews of side effects, deaths, or th so called “acceptable risks” data?
It looks like peers backed by lab$$$.
Big pharma is very effective at stopping cheap drugs.
Whereas, alas, data never stops conspiracy theorists…
No one is “stopping” Ivermectin. Millions doses of Ivermectin are prescribed every year to humans and other animals. It’s well known to be effective against various parasites and is encouraged for that purpose. It just doesn’t happen to be effective against Covid. There are a lot of things to criticize “big pharma” for, but the conspiracy theory that they’re downplaying the efficacy of their own popular product is pretty silly and not at all supported by evidence.
Dr Pierre Kory is a hero of the pandemic. IVM was just one part of a comprehensive early treatment program that could have saved hundreds of thousands of lives. Science and medicine has a reckoning coming for losing objectivity in favor of profits and politics.
Pierre Kory encourages people to take a medicine that doesn’t work instead of a vaccine that does. He is the exact opposite of a hero and is doing the exact opposite of saving lives.
Matt, you are very right: IVM could have saved millions, only if it was an effective antiviral. Kory and friends have shown an inability to conduct a clinical trial at an almost comical scale. The only thing more comical is the continued belief by so many in the potential of a drug, in the face of so little data.
Big Pharma is paying big $$ tomshill for them, as apparent by this article.
And how is that apparent exactly?
Or maybe Pharma is paying Kory to run amateurish clinical trials? It can’t be that IVM has no antiviral activity…
@Paul, ghost stories about vaccines can simply be disproved by looking at excess mortality rates in countries with a high vaccination rate. Vaccines have been very successful (which certainly cannot be said of all medicines).
mRNA vaccines have been in development for 30 years. Twenty-two months ago, the first 50,000 volunteers received the Pfizer vaccine. How many more years do you want to be afraid of long-term consequences? Those are very unlikely.
mRNA vaccines have been in development for 30 years. Twenty-two months ago, the first 50,000 volunteers received the Pfizer vaccine. 4.24 billion people are now vaccinated. A vaccine manufacturer does have interest in keeping track of any harmful consequences. Based on this experience a pre-selection takes place. That is why you have to fill in a long questionnaire, at least here in the Netherlands, before you are vaccinated.
How many more years do you want to be afraid of the long-term consequences? Those are very unlikely.
The chances of ivermectin being a successful drug against covid are in my opinion very slim. Maybe it helps a little, who knows.
I am getting the impression that not reading this post is the main requirement for commenting on it. Since I have actually read the post, I guess I am excluded from commenting.
I am a retired pediatrician who has given hundreds of doses COVID vaccines and worked many vaccine PODS as medical consult as an unpaid volunteer in Ulster and Dutchess Counties in NY State. I am also a big fan of Retraction Watch and appreciate how difficult and important your work is. Yet I wish you would reconsider the sentence “Kory is a controversial Wisconsin physician whose ideas about how to treat the infection, and particularly ivermectin, have made him a darling of ivermectin proponents like Joe Rogan.” in the otherwise excellent article “Ivermectin papers slapped with expressions of concern” from Feb 11.
It is hard for me to imagine who listened to Rogan’s entire podcast and his discussions of Ivermectin after he was criticized would ever call him an “Ivermectin proponent”. Yet it is easy to see how people who heard small parts of certain podcasts might think that he is.
Rogan clearly and repeatedly states he is not a medical or COVID expert, and that he took Ivermectin because it was prescribed by his physician.
It is true that Rogan engaged in long conversation with Pierre Kory and Robert Malone (who I think are both generally mostly wrong about COVID treatments and vaccine, although they are not always wrong). Rogan also had long discussions with many mainstream well respected researchers such as Michael Osterholm. No, I doubt that Ivermectin is helpful, but at the time of his discussions the metaanalyses showing Ivermectin did not work had not yet been published.
We need to maintain credibility of our institutions and science in general. Please consider addressing whether the sentence calling Rogan an “ivermectin proponent” was accurate.
Respectfully,
Roger Green, MD