A paper suggesting that scientists may want to rethink what levels of benzene are carcinogenic has led to a sharp exchange in the journal that originally published it.
In 2006, Stephen Rappaport, of UNC-Chapel Hill, and colleagues, published a paper that began by saying that benzene
is an important industrial chemical that is also emitted into the air from gasoline, engine exhausts and combustion of organic materials (including cigarette smoke) (1,2). Occupational exposures to benzene at air levels greater than ∼10 p.p.m., have long been linked to hematotoxicity and to acute myelogenous leukemia (3–5). A recent report of hematotoxic effects in workers exposed to benzene <1 p.p.m. (6) has raised additional concerns regarding the health consequences of low exposures to this contaminant.
The authors conclude:
If our conjecture that exposures to benzene <1 p.p.m. favor production of HQ and MA is correct, there could be important implications for risk assessment. Certainly, HQ is the precursor of 1,4-benzoquinone, which is generally regarded as most hematotoxic of the benzene metabolites (18,22,24, 36,56), and MA is derived from the extremely reactive and toxic muconaldehydes (14,15,17). We are currently applying various non-linear models to these data to more fully examine the dose-related metabolism of benzene and to estimate effects of physiological and genetic factors upon benzene metabolites.
In other words, we may not really understand how toxic benzene is. Some scientists have paid attention, with the paper earning 49 citations, according to Thomson Scientific’s Web of Knowledge. But last year, Dow Chemical’s Paul Price and colleagues repeated the analysis, and disagreed, writing in a paper in Carcinogenesis that
findings of increased metabolism do not provide a basis for questioning current estimates of the health risks from low level benzene exposures.
Often in the scientific literature, that’s the end of things, or the original authors publish a letter thanking the critics of their work and saying where they agree and disagree, and that’s the end of things. But not this time. In December, Rappaport and colleagues published their response, also in Carcinogenesis:
A series of articles by Kim et al. used air and biomonitoring data from workers in Tianjin, China, to investigate the dose-specific metabolism (DSM) of benzene over a wide range of air concentrations (0.03–88.9 p.p.m.). Kim et al. concluded that DSM of benzene is greatest at air concentrations <1 p.p.m. This provocative finding motivated the American Petroleum Institute to fund a study by Price et al. to reanalyze the original data. Although their formal ‘reanalysis’ reproduced Kim’s finding of enhanced DSM at sub-p.p.m. benzene concentrations, Price et al. argued that Kim’s methods were inappropriate for assigning benzene exposures to low exposed subjects (based on measurements of urinary benzene) and for adjusting background levels of metabolites (based on median values from the 60 lowest exposed subjects). Price et al. then performed uncertainty analyses under alternative approaches, which led them to conclude that ‘… the Tianjin data appear to be too uncertain to support any conclusions …’ regarding the DSM of benzene. They also argued that the apparent low-dose metabolism of benzene could be explained by ‘lung clearance.’ In addressing these criticisms, we show that the methods and arguments presented by Price et al. are scientifically unsound and that their results are unreliable.
Rappaport and colleagues — one of whom, it should be noted, has also received funding from the American Petroleum Institute, a trade group for the oil and gas industry — outline their issues with Price et al’s paper, then conclude:
These shortcomings raise questions whether Price’s reanalysis of Kim’s work was motivated by scientific skepticism or by an effort to obfuscate the low-dose metabolism of benzene. In either case, we regard the above shortcomings as sufficient to justify retraction of Price et al. (34) from Carcinogenesis (http://publicationethics.org/).
Price and colleagues weren’t happy with the tone of that paragraph. In a letter responding to the critique, they conclude:
We support the right of Dr. Rappaport and his coauthors to question the technical merits of our work or any work in the public domain and to defend Kim et al. (2, 3), but we strongly object to the authors’ wording in the title and in the paper’s concluding paragraph. While the analyses in the Price et al. paper may have been complex, they were clearly presented and subject to the peer review process. Furthermore, we published the raw data from the study so that experts could replicate the analyses and come to their own conclusions, which is something Kim et al. (2, 3) did not do. We included factors in our reanalysis that favor a finding of increased metabolism at low doses as well as others that do not. Thus we clarified and made transparent legitimate technical issues with the analyses of Kim et al. (2, 3) and did not “obfuscate” the original findings. It is unfortunate that Rappaport et al. attempted to disparage our analyses with allegations regarding both our motivations and our professional ethics in expressing our concerns with the Kim et al. datasets and analyses.
That didn’t convince Rappaport et al, whose letter in response was titled “Ignoring and adding errors do not improve the science“:
In closing, we reiterate that Price et al. have not clarified their use of spline and uncertainty models so as to counter obvious errors and permit independent confirmation of their results (2). Here we have identified additional errors in Price et al.’s Letter-to-the-Editor that further diminish the validity of their arguments. As for the tone of our discourse, it is fair to say that Price et al. questioned the integrity of our workmanship and we responded in turn. Perhaps the most important message from this exchange is that investigators should carefully examine competing results derived from complex data under different analytical approaches and assumptions. We will rely upon the discerning reader to judge the strengths and weaknesses of arguments presented in these letters and the publications that preceded them.
Price tells Retraction Watch:
While I have great admiration for much of Dr. Rappaport’s other works particularly on the issue of the Exposome, I believe that he and his research group simply made an honest mistake in their original analysis of the benzene worker data (Kim et al 2006 a,b). Our 2012 publication (A reanalysis of the evidence for increased efficiency in benzene metabolism at airborne exposure levels below 3 ppm, Carcinogenesis doi: 10.1093/carcin/bgs257) pointed to evidence of this mistake (inconsistencies with other published findings on benzene) and offered a possible alternative explanation for the worker data. It was very unfortunate that Dr. Rappaport interpreted our reanalysis as an attempt to obfuscate science and we were unaware of his concerns until his paper appeared in Carcinogenesis.
In our letter to the editor, I and my coauthors confirm that we have no intention of withdrawing our 2012 publication and that we stand on our findings. We further show that despite the title and concluding language, the findings in Dr. Rappaport’s publication actually support our concerns over the original conclusions reached in the Kim et al. 2006a,b.
Neither Rappaport nor the editor of Carcinogenesis responded to our requests for comment. As Price notes:
This debate on the interpretation of this set of worker data will undoubtedly go on for some time. It is my hope that the tone of the discussion will improve in the future.
Hat tip: David Egilman
Both Dr. Rappaport and the second author of the 2006 paper had received expert fees from plaintiffs’ law firms representing benzene plaintiffs. They quite properly disclosed this fact at the time. What is more interesting is Rappaport’s use of the terms “scientifically unsound” and “unreliable” in the 2012 call for retraction of Price et al. These phrases may have no special significance in a scientific publication. They are, however, both legal terms of art — two of the key factors which a court may use to reject the admission of expert scientific evidence. Note that either a retraction or a finding that a study is “unreliable” or “scientifically unsound” would have the effect of making the Price et al. study unavailable to defense experts. Indeed, even the existence of the 2012 call for retraction, when phrased in those exact terms, may have that effect. All the more so if, for instance, the plaintiff’s expert just happens to be a co-author of the 2012 letter.
If we insist that any ties to business make a scientist’s findings suspect, then we need to take an even-handed look at what business may be involved and how that might have influenced a publication.