Noteworthy: Journal posts all the corrections it wanted in a climate change paper after authors refuse most
In a case of refreshing transparency, a journal has published a detailed list of corrections it requested from authors of a paper on the costs of climate change, even though the authors declined to make most of them.
Earlier this year, the journal Ecological Economics published a paper that cast some doubt on the FUND model, which, as the article explains:
The FUND model of climate economics, developed by Richard Tol and David Anthoff, is widely used, both in research and in the development of policy proposals. It was one of three models used by the U.S. government’s Interagency Working Group on the Social Cost of Carbon in 2009 (Interagency Working Group on Social Cost of Carbon, 2010). The Working Group’s “central estimate” 1 of the social cost of carbon (SCC), i.e. the monetary value of the incremental damages from greenhouse gas emissions, was $21 per ton of CO2.
The paper concluded:
In FUND’s agricultural modeling, the temperature-yield equation comes close to dividing by zero for high-probability values of a Monte Carlo parameter. The range of variation of the optimal temperature exceeds physically plausible limits, with 95% conﬁdence intervals extending to 17 °C above and below current temperatures. Moreover, FUND’s agricultural estimates are calibrated to research published in 1996 or earlier.
Use of estimates from such models is arguably inappropriate for setting public policy. But as long as such models are being used in the policymaking process, an update to reﬂect newer research and correct modeling errors is needed before FUND’s damage estimates can be relied on.
In other (simplified) words, FUND needs some work if we’re going to use it to make policy.
Tol, a professor of economics at the University of Sussex, and Anthoff were happy to have constructive criticism, but they also thought the paper got a number of things wrong. And they were a bit baffled by some of those errors, given that they had helped the authors through their analysis.
So they wrote a letter to the editor of the journal, as David Stern notes in a letter published online in the journal last week. Elsevier asked Stern, a professor at the Crawford School of Public Policy, part of Australian National University, Canberra, to adjudicate because the journal’s editor in chief, Richard Howarth, had published with Ackerman.
Along with the Stern letter, the journal has now published a commentary from Anthoff and Tol, and a response from Frank Ackerman and Charles Munitz. From Stern’s letter:
The main point of contention is around Section 4.1 of the paper, which claims that the results of the FUND model could be affected by a division by zero problem. In my investigation, I had access to correspondence between Anthoff and Tol and Frank Ackerman prior to publication of the paper. In this exchange, Anthoff and Tol had told Frank Ackerman that the apparent division by zero problem was in fact addressed by the FUND model and the results were not substantially affected by it. I also relayed Tol’s concerns to Ackerman and received a reply from him. Based on the responses I received and the previous correspondence, I determined that some statements in the paper were problematic and that Ackerman and Munitz did not report in their paper the information they had received from the model developers about the division by zero issue.
Richard Tol stated that the minimum set of corrections that would address his concerns is the following:
In the abstract:
 In place of: “We examine the treatment of climate damages in the FUND model.” substitute: “We examine the treatment of climate damages in a modiﬁed version of the FUND model.”
In section 2:
 In place of: “The analysis described here begins with the Working Group’s modiﬁed version of FUND (fn 2).” substitute: “The analysis described here begins with the Working Group’s modiﬁed version of FUND, to which further changes were made (fn 2).” and add the following to footnote 2:
 “We made changes to the FUND model code as described in this paper. These changes were not validated by the model developers, David Anthoff and Richard Tol, and they did not vet the results. David Anthoff and Richard Tol are, therefore, not responsible for any of the model results presented below.”
In section 4:
 In place of: “4.1. Risk of division by zero” substitute: “4.1. Apparent risk of division by zero”
 In place of: “A ﬁx for the optimum temperature equation bug is planned for the next version of FUND.” substitute: “Changes to the optimum temperature equation are planned for the next version of FUND.”
Ackerman and Munitz were willing to accept  and a modiﬁed version of  but were not willing to accept the other changes. Given this, Ecological Economics could not publish a formal correction to the article. Therefore, I decided to include the full set of requested corrections in this Editor’s note, along with the commentary of Anthoff and Tol, and the response from Ackerman and Munitz.
I trust that with the publication of both the commentary and response, along with this note, the journal has provided all parties the opportunity to express their concerns and opinions.
For their part, Tol and Anthoff conclude their own letter, published online today:
We have been in repeated contact with AM on this matter. We helped with conﬁguring the code to run on their machines. Mr Ackerman then contacted us because he thought he had found a division-by-zero error. We explained why his tests are inconclusive; AM’s “not an appropriate way” (p. 222) echoes this. Well before the AM paper was submitted, we shared the results of our standard diagnostic test and the one specifically tailored to the alleged problem. Neither test reveals a problem.
We are surprised that AM nevertheless chose to publish their division-by-zero claim, while remaining silent on the results of these diagnostic tests.
This all seems like a good example of post-publication peer-review.
The letter from Ackerman and Munitz doesn’t appear to be online yet, but we’ll keep an eye out for it and update.
Update, 6:15 p.m. Eastern, 7/9/12: Please see Frank Ackerman’s comment below, which includes the text of his and Munitz’s letter.