Update on Fujii: Anesthesia journal finds overwhelming statistical evidence of data fabrication

There’s a bit more this afternoon on the story of Yoshitaka Fujii, the Japanese anesthesiologist accused of fraud and other misconduct that we reported on yesterday.

The British journal Anaesthesia, which has been looking into Fujii’s research record, has posted four articles and editorials about the case and related issues on its website. One in particular is remarkable for its conclusions. Written by a UK anesthesiologist named John Carlisle, the article claims to have analyzed 169 randomized controlled trials that Fujii conducted between 1991 and 2011.

According to the abstract (which we formatted for readability, and which should be online shortly, we’re told):

I extracted rates for categorical variables and means  (SDs) for continuous variables, and compared these published distributions with distributions that would be expected by chance. The published distributions of 28 ⁄ 33 variables (85%) were inconsistent with the expected distributions, such that the likelihood of their occurring ranged from 1 in 25 to less than 1 in 1 000 000 000 000 000 000 000 000 000 000 000 (1 in 10^33), equivalent to p values of 0.04 to < 1 · 10^33, respectively. In 142 human studies, 13 ⁄ 13 published continuousvariable distributions were inconsistent with expected, their likelihoods being: weight < 1 in 10^33; age < 1 in 1033; height < 1 in 1033; last menstrual period 1 in 4.5 · 10^15; baseline blood pressure 1 in 4.2 · 10^5; gestational age 1 in 28; operation time < 1 in 10^33; anaesthetic time < 1 in 10^33; fentanyl dose 1 in 6.3 · 10^8; operative blood loss 1 in 5.6 · 10^9; propofol dose 1 in 7.7 · 10^7; paracetamol dose 1 in 4.4 · 10^2; uterus extrusion time 1 in 33. The published distributions of 7 ⁄ 11 categorical variables in these 142 studies were inconsistent with the expected, their likelihoods being: previous postoperative nausea and vomiting 1 in 2.5 · 10^6; motion sickness 1 in 1.0 · 10^4; male or female 1 in 140; antihypertensive drug 1 in 25; postoperative headache 1 in 7.1 · 10^10; postoperative dizziness 1 in 1.6 · 10^6; postoperative drowsiness 1 in 3.8 · 10^4.

Distributions for individual RCTs were inconsistent with the expected in 97 ⁄ 135 human studies by Fujii et al. that reported more than two continuous variables, their likelihood ranging from 1 in 22 to 1 in 140 000 000 000 (1 in 1.4 · 10^11), compared with 12 ⁄ 139 RCTs by other authors. In 26 canine studies, the distributions of 8 ⁄ 9 continuous variables were inconsistent with the expected, their likelihoods being: right atrial pressure < 1 in 10^33; diaphragmatic stimulation (100 Hz) < 1 in 10^33; pulmonary artery occlusion pressure < 1 in 10^33; diaphragmatic stimulation (20 Hz) < 1 in 10^33; heart rate 1 in 6.3 · 10^10; mean pulmonary artery pressure 1 in 2.2 · 10^14; mean arterial pressure 1 in 6.3 · 10^7; cardiac output 1 in 110. Distributions were inconsistent with the expected in 21 ⁄ 24 individual canine studies that reported more than two continuous variables, their likelihood ranging from 1 in 345 to 1 in 51 000 000 000 000 (1 in 5.1 · 10^13).

Meanwhile, editors involved in the case expressed some chagrin about the statement Fujii’s former institution, Toho University, released about the scandal. The statement, which cites lack of ethics approval for some studies, is mute on the issue of whether Fujii’s results were, in fact, reliable. (The statement now includes a promised list of nine papers that are being retracted.)

But Steven Shafer, editor-in-chief of Anesthesia & Analgesia, which published 24 of Fujii’s articles over the years — and, as we reported yesterday, a 2000 letter expressing incredulity about the validity of his results — said he has

grave concerns about the possibility of manipulation or fabrication in Dr. Fujii’s published research. This will be investigated, and compromised papers in Anesthesia & Analgesia will be retracted.

The journal has issued a statement of concern about the papers.

Steve Yentis, editor-in-chief of Anaesthesia, had a similar comment:

As far as the head of the hospital knows, there was only one clinical study listed by Dr. Fujii as having been conducted at the hospital” – but the conclusion of the investigation was that the studies were conducted without ethical approval, not that they were fabricated. I’d have liked to have seen a statement about the investigating committee’s conclusions as to whether these studies took place at all, or whether they took place but without approval – especially given the sentence I’ve quoted.

Needless to say, we’ll be following all this closely.

0 thoughts on “Update on Fujii: Anesthesia journal finds overwhelming statistical evidence of data fabrication”

  1. Seriously, how does one evaluate papers – manuscripts – like clinical trials reports, where there is often no objective, material data, just arbitrary clinical outcomes, discomforts, improvements, self-reports of side effects, and other verbs and nouns? Where “raw data” is questionnaires and such?

    1. You could start by actually looking at the questionnaires. Do they exist at all? The implication is that the questionnaires, databases of answers, maybe even the patients(!) never existed.

      1. One could, but it’s not that easy. It’s not a tiff file, not a pdb, or a abi. If there is a doubt on reviewers’ part, they can through Editor request say original image files, or raw mass spec data, or a pdb. What are they going to ask for in this case? Bankers boxes full of questionnaires? At best all they can get in this case is some kinda excel file…

    2. If the raw data are questionnaires and/or patient files, this is always difficult. Usually study participants are granted anonymity. So you can’t just show patient files around as raw data. Only in the case of suspected fraud, you can hope for internal review, and for such purposes there usually is internal access to the patient files.

      But the power of proof of so called raw data should not be overestimated anyway. If you want to forge a conclusion, you can always generate raw data under specific circumstances that you simply don’t report. Checking the raw data raises the hurdle for fraud, but certainly doesn’t make it impossible.

  2. The quote would be clearer if you wrote the scientific notation numbers as (e.g.) “< 1 in 10^33" rather than as "< 1 in 1033"…

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.