Environmental group alleges scientific fraud in disputed methane studies

Note: We are reprinting below an article originally published at InsideClimate News.

The inspector general of the U.S. Environmental Protection Agency has been asked to examine whether a significant recent study of greenhouse gas emissions from oil and gas fields was technically flawed—and whether researchers brushed aside concerns that methane pollution was being understated.

The emission of methane, a much more potent greenhouse gas in the short term than carbon dioxide, has proven difficult to measure. The latest complaint is a volley in a long-running skirmish among academics, advocacy groups and regulators over how tightly methane should be regulated.

On Wednesday, a North Carolina environmental advocacy group, NC Warn, alleged that this dispute has risen to the level of fraud. The group petitioned Arthur A. Elkins Jr, the agency’s inspector general. The IG is an independent watchdog office that resides within each federal agency.

NC Warn said that David Allen, a University of Texas engineering professor who led the industry-funded study, had ignored flaws in the study after they were pointed out by another expert, Touché Howard. Howard had invented the technology behind one of the devices that Allen and colleagues used in their 2013 and 2014 studies. Soon after Allen’s 2013 study came out, Howard alerted him of possible underreporting of methane emissions due to sensor malfunction.

“Our study team strongly asserts that the instrument we used and the measurements we made were not impacted by the claimed failure,” Allensaid in a statement responding to the NC Warn allegations.

Allen was chairman of the EPA’s outside science advisory board from 2012-15, at the time the two contested studies were published. The work was part of an extensive series of studies organized by the Environmental Defense Fund in collaboration with the oil and gas industry. EDF, one of the largest national green groups, has advocated for tougher regulations of methane emissions, a hotly debated climate policy issue. But it has also been criticized for working closely with the industry in conducting the studies.

NC Warn, an anti-fracking group that advocates a swift transition away from fossil fuels, wants the EPA to conduct a full investigation. It also wants any flawed studies to be withdrawn and the agency to revise any regulations based on flawed studies. The group says the agency should adopt a zero-emissions standard for oil and gas operations.

“What we are alleging is a cover-up, scientific fraud and possibly criminal misconduct by a high-ranking EPA official and perhaps others,” said Jim Warren, executive director of NC Warn.  “Due to this cover-up, humanity has been robbed of several years during a critical period of the climate crisis where we should have been working cooperatively to greatly reduce emissions.”

Despite that strident claim, this technical debate has been fully aired in peer reviewed publications. Neither side has budged; so an inspector general’s report might help resolve the technical questions, as well as settling whether there was any impropriety.

If the agency’s IG chooses to conduct a formal investigation, it will bring the matter to the forefront of policy debates in Washington and beyond at a time when methane regulations are being put into place amid a great deal of technical and legal argument.

Knowing exactly how much methane is emitted by the oil and gas industry is of crucial importance for meeting the Paris climate agreement, said Bradley Campbell, president of the environmental advocacy group Conservation Law Foundation.

“Whatever the merits of the allegations in this case, it touches on a critical issue of both science and public policy,” he said. “This is the data that EPA is essentially relying on to regulate.”

Disputed Readings Spark Debate

Howard has been sparring with Allen over the technical merits of his study ever since it came out. His complaints focus on what he calls the improper calibration of the instrument Allen’s team used to measure methane emissions.

Howard may be uniquely qualified to challenge Allen’s results. In 1996, he patented the technology later used to create the Bacharach Hi-Flow Sampler that Allen used in his 2013 study and, to a lesser extent, his 2014 study. Working as a consultant for industry and government clients since 1988, Howard, who is based in North Carolina, has conducted measurement and training programs at more than 500 natural gas facilities in North America, Europe, and the former Soviet Union. He collaborated on a related EDF study looking at methane emissions from the oil and gas industry.

The 2013 study by Allen and colleagues was a survey of hundreds of hydraulically fractured natural gas wells. It was the first in an ambitious, ongoing series of 16 studies launched by EDF to determine how much methane the oil and gas industry is leaking or emitting. The findings, which have since been cited by the oil and gas industry, suggested that methane emissions from drilling sites were about 10 percent lower than previous EPA estimates.

Those results ran counter to findings of a number of studies published before and after that found the industry’s methane emissions were higher than EPA estimates. Environmentalists criticized the Allen study because of industry backing and collaboration. Ninety percent of the study’s $2.3 million in funding came from oil and gas operators, which also provided researchers access to well sites.

Warren, NC Warn’s director, said that Allen and colleagues had no way of knowing in 2013 that their measuring device, a Bacharach Hi-Flow Sampler (BHFS), was underreporting methane emission rates.

Howard said he repeatedly raised concerns with Allen about the 2013 study, beginning soon after its publication. Howard, who was already investigating possible errors with the device, examined data from the 2013 study and quickly suspected the device was underreporting. NC Warn released emails between Howard and Allen discussing the issue.

Confronted with Howard’s evidence that data from the device might be flawed, Warren said, Allen failed to acknowledge the problem and did not correct the data in his study.

“That study threw a lot of confusion in the air and continues to be relied on by the fossil fuel industry to argue that the problem is not so bad and the EPA should back off trying to curtail emissions even more,” Warren said.

In March 2014, Howard and researchers from the University of Texas, tested one of the devices used by Allen and colleagues in their 2013 study. According to Howard, those tests showed the device was still underreporting emissions by a small amount.

At issue was the gas detector’s automatic transition between two operating modes. One mode measures methane when concentrations are at or below 5 percent. The other mode measures concentrations above 5 percent. When concentrations exceeded 5 percent, the device can fail to initiate its second mode.

Errors during the March 2014 field test occurred even though new software developed to fix the problem had been added to the device after the 2013 study.

Allen and colleagues published a follow-up study in December 2014 using a BHFS and other sampling devices which, according to Howard, were also giving erroneous readings during the March 2014 field test. The study didn’t mention the underreporting errors detected by Howard.

“He went ahead and published that data and did not disclose at all that tests had been run showing his instruments were far out of calibration,” Howard said.

Howard challenged the Allen group’s findings in March 2015 in the academic journal Environmental Science and Technology, where Allen’s 2014 study was published.

Allen responded to each of Howard’s concerns in the same edition of the journal.

Howard then published a pair of papers in peer-reviewed academic journals in late March and August 2015, describing how the device used by Allen in his 2013 paper underestimated methane emissions. Howard said the issues he raised still haven’t been addressed, leading to the formal complaint with the EPA’s inspector general.

“I’ve gone to Dr. Allen repeatedly and asked him to address these issues, and since they haven’t been addressed, unfortunately, at this point, I think that is the only solution,” Howard said in an interview.

Measurements from the disputed device were cross checked by an infrared camera, and downwind sampling by independent investigators, Allen said in a statement.  Leaks are also often detected by operator observations by their noise and smell, Allen said.

“None of these parallel systems indicated a problem with our [Bacharach] HiFlow instrument,” he said.  “All of these systems would have had to fail, simultaneously, and only at certain types of sites with the conditions that are claimed to produce the equipment failure, for our measurements to have been impacted.”

EDF Defends Allen’s Work

According to Wednesday’s filing, Allen had acknowledged potential concerns about underreporting of emissions.

In April 2016, Howard obtained a confidential letter from Allen to EDF’s “production committee,” scientists from academia and industry who advise EDF on its methane studies, according to Wednesday’s filing. The memo is undated, but according to the filing was sent sometime between July and October 2014. In it Allen acknowledged concerns about “cross-over malfunction” raised by “some investigators.” The memo then proceeds to provide a detailed explanation for why such malfunction “was not a major cause of measurement error” in Allen’s 2013 study.

Howard said in an interview that the information in the memo was incorrect, and that Allen knew it.

EDF said issues raised in this week’s petition had already been aired in academic journals and added that even if the BHFS did malfunction during the 2013 study, the potential inaccuracy was not that significant.

“Even if you assume the worse-case scenario for inaccuracy of the technique, it would have only changed the outcome of the [2013] UT [University of Texas] paper by about 12 to 24 percent,” said Mark Brownstein, vice president of EDF’s climate and energy program. “If you extrapolated it to what it means for emissions from the total natural gas supply chain, it would change your assessment of emissions by about 2 to 5 percent.”

EDF is now working on a final analysis of methane emissions from across the oil and gas industry. When asked by InsideClimate News if they would still include Allen’s 2013 study in that analysis given its potential inaccuracy, Brownstein said “absolutely.”

“Touché has raised some legitimate questions about how the [Bacharach] Hi-Flow meters would have worked in 2 of the 5 sources in which the technique was used, but the work that we are doing now to understand the totality of what the literature is telling us, I don’t think is affected by this at all,” Brownstein said.

The response from the EPA’s office of the inspector general could range from declining to investigate the allegations to a full investigation with subpoena power. The IG has investigative powers and can refer criminal matters to the Justice Department. In 2014, the IG criticized the EPA for poor work on methane emissions. Early this year, the EPA revised upward its estimates of how much methane is being emitted.

Jennifer Kaplan, deputy assistant inspector general for congressional and public affairs, said she cannot comment on any complaint made to the inspector general’s office.

Phil McKenna is a Boston-based reporter for InsideClimate News. Before joining ICN in 2016, he was a freelance writer covering energy and the environment for publications including The New York Times, Smithsonian, Audubon and WIRED. Uprising, a story he wrote about gas leaks under U.S. cities, won the AAAS Kavli Science Journalism Award and the 2014 NASW Science in Society Award. Phil has a master’s degree in science writing from the Massachusetts Institute of Technology and was an Environmental Journalism Fellow at Middlebury College.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.