Earlier this week, we reported that a paper claiming that two deaths resulted from COVID-19 vaccination for every three cases that were prevented had earned an expression of concern.
[Please see an update on this post; the paper has been retracted.]
The authors, including Harald Walach, who was also co-author of a just-published paper in JAMA Pediatrics questioning the safety of masks in children, had used data from the Dutch national registry of side effects. That registry carries a warning label about its use. The editors of Vaccines, which published the study last month, wrote that there were concerns over “misrepresentation of the COVID-19 vaccination efforts and misrepresentation of the data.”
Walach told us earlier this week that he would send us a detailed response to the criticisms. He did that today, and we have made the entire response available here.
Walach and his co-authors are responding to criticisms leveled by Eugène van Puijenbroek of the University of Groningen. In it, they write:
These vaccines have had an emergency approval without the necessary safety data. Although we would agree with Prof. van Puijenbroek that the self-reporting system of side-effects for vaccines and other drugs is far from foolproof, it is the only data we have. So why should it not be put to use?
They add:
We are happy to concede that the data we used – the large Israeli field study to gauge the number needed to vaccinate and the LAREB data to estimate side-effects and harms – are far from perfect, and we said so in our paper. But we did not use them incorrectly. We used imperfect data correctly. We are not responsible for the validity and correctness of the data, but for the correctness of the analysis. We contend that our analysis was correct. We agree with LAREB that their data is not good enough. But this is not our fault, nor can one deduce incorrect use of data or incorrect analysis.
Like Retraction Watch? You can make a one-time tax-deductible contribution or a monthly tax-deductible donation to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
Garbage in garbage out
The only thing we learn from this response is that prof. Walach doesn’t know anything about pharmacovigilance…
COI: I am employed by a European national drug agency and have done work for WHO, EMA, MHRA (UK) and TGA (Australia) regarding regulatory processes and pharmacovigilance for herbal drugs.
No, we have learnt something else. There shall be no criticism of any of the COVID-19 vaccines. Criticism will result in immediate sanction. Those who suffer serious side effects from the vaccines should suffer in silence. The vaccine manufacturers, making billions of dollars from the vaccines, know best.
Now we have it in plain sight. The institutional science as we knew it is dead. But I am afraid it has been dead for a while. Pandemic has only focused our attention on one issue.
How does data collection in pharmacovigilance work? From what I have found: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1873545/
I quote:
“Recording and reporting clinical observations of a suspected ADR with a marketed drug is known as spontaneous or voluntary reporting. The national system in the UK is the ‘yellow card’ scheme where doctors, dentists, and recently, hospital pharmacists are encouraged to report all suspected reactions to new medicines and serious suspected reactions to established medicines. Pharmaceutical companies also collect and collate such reports with their licensed products [8]. Reports to companies often come initially as a question from a prescribing physician or pharmacist to Medical Information or a sales representative about whether a product could be the cause of a patient’s problem. After providing such information, pharmacovigilance staff will seek details of the case to add to the database of reports, this relies on the goodwill and continued interest of reporters. Companies must report suspected ADRs to the MCA and other authorities; some authorities, including MCA, make anonymised data available to licence holders. There is also a move towards electronic exchange of data between authorities and companies.”
The data in this system is based on self reporting (e.g. Yellowcard) and relying “on the goodwill and continued interest of reporters”.
If I haven’t missed something (?) about the described data collection process and if its all there is, well… it is basically a similar kind of data as Walach et al used. Some in the discussion here calling it not adequate, because signal detection is hard due to lacking structure in data collection.
Prof. Van Puijenbroek is Head Science and Research at Lareb (the Netherlands Pharmacovigilance Centre), which is probably more relevant in this case than his position at the University of Groningen, as his criticism is most likely the official complaint by Lareb to Vaccines about the misuse of their data by Walach et al.
Walach writes: “These vaccines have had an emergency approval without the necessary safety data.” This is absolutely incorrect, in the EU the vaccines were explicitly NOT granted an EUA, but a stronger Conditional Markething Authorization (CMA).
who can see data correct or incorrect?
reader? editor? reviewer?
all figures not from the Article Intelligence (AI).
According to the human brain? Or eyes.
I can not agree with this method.
This is a professional skill, not can see or not?
On the other hand, the author has to review two times from editors. I can’t understand why to do that.
Walach et al seem to be satisfied with using crappy data to make sweeping, unwarranted conclusions that jeopardize lives.
It’s a problem that plagues alternative medicine in general.
The article has now been retracted: https://www.mdpi.com/2076-393X/9/7/729
We are not responsible for the validity and correctness of the data, but for the correctness of the analysis. We contend that our analysis was correct. We agree with LAREB that their data is not good enough. But this is not our fault, nor can one deduce incorrect use of data or incorrect analysis.
“Apples are fruit. Oranges are fruit. When you say it isn’t valid to compare them, you are saying that fruit are invalid, and I’m not prepared to stand here and let the healthiness of fruit be questioned.”
They don’t have a confidence interval for the adverse event rates from LAREB, so their analysis isn’t correct and the CI on their final result is useless.
So the article is retracted because data quality is not good enough ?
But is there any available better data ?
Has any article done the same kind of calculations with better data ?
If no, the aticle should have not been retracted as it is the best we can do in the current situation.
It is better to be one-eyed than blind.
“So the article is retracted because data quality is not good enough ?”
No.
“But we did not use them incorrectly. We used imperfect data correctly. We are not responsible for the validity and correctness of the data, but for the correctness of the analysis. We contend that our analysis was correct. We agree with LAREB that their data is not good enough. But this is not our fault, nor can one deduce incorrect use of data or incorrect analysis.”
WTF? What’s the point to make a paper based on wrong data? Even worse, what’s the point if you knowingly used wron data? That’s one of the stupidest justification I ever read.
The data were fine, within the contexts and for the purposes they were collected for. The authors chose to ignore those contexts, and combined the data in a way that is about as meaningful as multiplying potatoes by custard.
The authors’ insistence on blaming the data displays the same integrity as their analysis.
I agree, this is a super sketchy thing to do and it shows how knowledgeable and integer the author is that he even dares to give this justification if you can even call it that. You can also have monkeys enter random data and than analyze it, as an analyst you can’t be held accountable for the validity and correctness of the data so who cares right. What does this clown even think data analysis is, if you don’t even analyze if you have the correct source….absolutely insane that this can even get published in a peer-reviewed paper.
Would it be correct to say that data should represent the phenomenon under scientific investigation, and data analysis help researchers to make their conclusions based on the results of the analysis?
It’s worse than that. The paper says:
“we discovered that the reporting of side effects varies by a factor of 47 (Figure 1). While the European average is 127 individual case safety reports (ICSRs), i.e., cases with side effect reports, per 100,000 vaccinations, the Dutch authorities have registered 701 reports per 100,000 vaccinations, while Poland has registered only 15 ISCRs per 100,000 vaccinations. Assuming that this difference is not due to differential national susceptibility to vaccination side effects, but due to different national reporting standards, we decided to use the data of the Dutch national register”
A responsible researcher would at least investigate the reasons for the variance, and possibly even use it to estimate a confidence interval. Instead, they make an assumption, cherry-pick the country with the highest rate by far, and use that as a point estimate with NO confidence interval at all. They just assumed the Dutch number was perfect.
Has the covid 19 vaccination program followed the standard procedure for vaccinations approval like below?
Exploratory stage
Pre-clinical stage
Clinical development
Regulatory review and approval
Manufacturing
Quality control
Yes. HTH.
It’s worth noting that the only study or review that I’m aware of that has studied causality is a Norwegian study into 100 deaths of residents in care. It found that 10 deaths were probably caused by the COVID-19 vaccine, while another 26 deaths were possibly caused by the vaccine.
Clearly, what is needed is more research into serious side effects from the vaccines and the cause of those side effects.
https://legemiddelverket.no/nyheter/expert-group-has-assessed-deaths-amongst-the-frail-elderly-following-covid-19-vaccination
So, everyone criticising the analysis and that one is responsible for the data; fair enough! However, ask yourselves why vaccine producers of an experimental vaccine were not obliged to set up infrastructure under an EUA/CMA in order to ensure the collection of structured data so that signals and causalities can be detected? And more importantly, from where shall we get this data? And from where do the regulators get such data?
This is a not a rhetorical question. I would highly appreciate it, if someone could point me in a direction. Thank you!
Each country has its own infrastructure for reporting incidents. The problem is not the infrastructure.
The problem is that counts are meaningless unless they are compared to the background level of incidents in a comparable unvaccinated population. This author chose to ignore that, as well as cherry-picking the sets of data that best support his pre-defined conclusion.
If incidents are underreported in vaccinated individuals, where there’s a system in place, they must be even more underreported in the unvaccinated, where there’s no centralised reporting system whatsoever.