The Journal of Economic Perspectives has published a second correction for a 2009 paper that argued that some amount of global warming could lead to economic gains.
The author of “The Economic Effects of Climate Change,” Richard Tol, a professor of economics at the University of Sussex, blamed earlier problems with the paper on “gremlins.” In a notice posted last year, Tol wrote that “minus signs were dropped”; he also added a pair of “overlooked estimates” and several recently published studies.
After the first correction was published, several people contacted the JEP to point out more issues with the paper. Editors worked with Tol and outside researchers to update the paper again.
Here’s some text from the newest correction notice:
In the Spring 2009 issue, this journal published “The Economic Effects of Climate Change” by Richard S. J. Tol (vol. 23, no. 2, pp. 29–51). The paper included a figure summarizing the results of a number of studies, showing their estimates of how the economic costs of climate change varied with the predicted change in global temperatures. In early 2014, the editors received a complaint pointing out errors in the paper: specifically, several estimates had not been accurately transferred from the original studies. In the Spring 2014 issue, we published a “Correction and Update: The Economic Effects of Climate Change” (vol. 28, no. 2, pp. 221–26) by Richard Tol. However, this version also contained errors that were soon pointed out by various researchers. The editors discussed the situation with Richard Tol and with outside reviewers at some length.
The paper’s central figure was replaced with one from a chapter in the 2014 IPCC report that Tol helped assemble. (He later withdrew his name from the summary document for policymakers when early drafts carried a message that “does not support the political agenda for greenhouse gas emission reduction,” he said in a 2014 blog post.)
This correction offers a final revision and update to the figure in question. This figure is republished from the most recent report of the International Panel on Climate Change (IPCC), in Chapter 10 of the volume Climate Change 2014: Impacts, Adaptation, and Vulnerability. Richard Tol is one of the two “Coordinating Lead Authors” of this chapter, along with Douglas J. Arent, but the chapter also draws on efforts by a group of other lead authors, contributing authors, and review editors. Figure 1 reproduces Figure 10-1 from p. 690 of the IPCC report. The IPCC discussion of this figure offers some useful cautions about interpretation:
Estimates agree on the size of the impact (small relative to economic growth), and 17 of the 20 impact estimates shown in Figure 10-1 are negative. Losses accelerate with greater warming, and estimates diverge.
The new estimates have slightly widened the uncertainty about the economic impacts of climate. Welfare impacts have been estimated with different methods, ranging from expert elicitation to econometric studies and simulation models. Different studies include different aspects of the impacts of climate change, but no estimate is complete; most experts speculate that excluded impacts are on balance negative. Estimates across the studies reflect different assumptions about inter-sectoral, inter-regional, and inter-temporal interactions, about adaptation, and about the monetary values of impacts. Aggregate estimates of costs mask significant differences in impacts across sectors, regions, countries, and populations. Relative to their income, economic impacts are higher for poorer people.
The original figure in the 2009 JEP article estimated a best-fit line passing through the points on this kind of figure, along with confidence intervals for that estimate. Given the differences across the studies as mentioned in the IPCC report, several outside reviewers involved in our editorial process expressed a concern that such estimates were not meaningful. As shown, the figure in the IPCC report does not seek to estimate a best-fit line or confidence intervals, but only offers a summary of the results from existing studies. Tol offers further discussion of the curve-fitting issues with this kind of data in “Bootstraps for Meta-Analysis with an Application to the Impact of Climate Change,” forthcoming in Computational Economics(doi: 10.1007/s10614-014-9448-5).
The journal’s managing editor, Timothy Taylor, declined to discuss details of the second correction but explained that using the IPCC figure was a choice made in the spirit of the journal.
I’m not comfortable offering details about our internal conversations with author, editors, reviewers, and other commenters. But perhaps this general background is useful.
The Journal of Economic Perspectives isn’t a standard research journal. For a sense of what we do, you can check the JEP website at <https://www.aeaweb.org/jep/
submissions.php>. As it says there: “The focus of JEP articles should be on understanding the central economic ideas of a question, what is fundamentally at issue, why the question is particularly important, what the latest advances are, and what facets remain to be examined.” Authors of JEP articles often refer to a range of studies on a given subject. Referring to the IPCC figure–that is, a calculation from a mainstream consensus document–is the kind of reference that would be standard in our pages. From our point of view, it’s like a JEP author citing estimates from the IMF or the Congressional Budget Office. We know that there are critiques of those estimates, too. However, for a JEP essay, pointing out credible mainstream sources that the author finds important will suffice. Those who want details on these calculations would then need to turn to the research literature or working papers or other reports.
Tol’s contribution to the 2014 IPCC report was also the subject of controversy: In 2014, Bob Ward of The London School of Economics and Political Science raised concerns of calculation errors in Tol’s 2013 article, “Targets for global climate policy: An overview,” that were subsequently used in the IPCC report. According to Ward, the final report also omitted a late-inserted statement, based on Tol’s research, that climate change “may be beneficial for moderate climate change.”
The Journal of Economic Developments and Control posted a correction for the 2013 article last year.
The latest correction probably wouldn’t be the end of the argument. In the final section of the latest correction, the JEP has provided an index of methods and materials in anticipation of further debate:
For a list of studies that were included, what methods were used in these studies, what economic sectors were covered, and the like, we would point interested readers to the “Supplementary Material” table for Chapter 10. The full report, Chapter 10, and the Supplementary Material are all available at http://www.ipcc.ch/report/ar5/wg2/. Controversy over these estimates seems likely to continue. We recommend that readers interested in these questions use the figure and data from the IPCC report as their starting point.
When asked for a statement on the latest correction, Tol offered a brief response.
I am grateful to the editors for their constructive attitude in a painful episode.
Readers may also want to look at the latest assessment here: https://ideas.repec.org/p/sus/susewp/7515.html
The paper has been cited 133 times, according to Thomson Scientific’s Web of Knowledge.
We’ve reached out to the publisher, the American Economic Association, and we’ll update with any reply.
Like Retraction Watch? Consider supporting our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, and sign up on our homepage for an email every time there’s a new post. Click here to review our Comments Policy.
The corrections did not affect the conclusion that the initial impacts of climate change are beneficial.
Because that conclusion is based on a single outlying data point – your study, which no-one else comes close to agreeing with.
Tol’s recent paper states in its first paragraph, “One cannot have cheap energy without carbon dioxide emissions.” This assumes that, for example, solar photovoltaic power cannot be made “cheap” (cheaper than coal or natural gas, say.) This assumption is contradicted by the recent drops in the cost of solar photovoltaic panels: 20 % this year alone, with no indication that costs will rise again and every indication that they will continue to drop. This is particularly true when the economy of scale is considered. There are credible calculations indicating that solar photovoltaic may already be cheaper than coal, and if it is not already so, it will soon be.
It is said that it takes a year for a solar photovoltaic panel to pay off the energy costs of its manufacture; after the first year, the panels continue to produce electricity for at least twenty to twenty five years, essentially free. (Assuming that the transmission costs are similar to those of coal-derived electricity, and ignoring the opportunity cost of the land shadowed by the panels.)
Then why can’t one have “cheap energy” without carbon dioxide emissions?
The reaction of the renewables industry to the withdrawal of subsidies in the UK neatly illustrates my point.
Climate change and all its repercussions, are important matters for future generations. To make an error in this and correct it, is fine. However, to issue a second correction makes this study incredible. With the first correction, the authors should have thoroughly reconsidered the manuscript. That was not sufficiently done. If there were errors now, it should be withdrawn. Whether, the conclusions are not affected is unimportant. It shows that there was sloppiness in the research, and so the conclusions are questionable.
Tol was also the co-author of a 2012 working paper when he was at the Irish Economic and Social Research Institute which was withdrawn for ‘serious methodological issues’:
http://www.thejournal.ie/richard-tol-esri-working-paper-withdrawn-484845-Jun2012/
The paper claimed that 44 per cent of families with young children would be better off on the dole than working, which raised quite a stink at the time. There were accusations of political interference, although these recent corrections seem to indicate sloppy research.
You gotta love a paper that gets so much wrong, yet the author continues to state that the conclusions remain the same. That’s some fascinating math.
This is rather late in coming, as the second correction has been commented on many places in the last half year. Be that as it may
Prof. Tol’s contention is anchored in one disputed data point, which, of course is his. However, even beyond this there are issues that have been raised by many about the nature of the data, with many of the points based on basically the same evolving model (Nordhaus).
Moreover, given the nature of the data set, a forced fit with one early positive point (there is another but it is only positive by a very small amount), will always allow Prof. Tol to say that there is a benefit to warming, but one must wonder if the fit and reality are even distantly related.
To go even a bit further, the title of the figure is
which does not reference a baseline. Is the baseline pre-industrial (in which case the world is now past Tol’s positive point), 1860 or so when instrumental records start in which case the world is pretty close to it), or some more recent time, in which case, another couple of degrees would, at least according to Tol, have little effect. Eli notes that Chris Field’s studies appear to be differential, which, also raises some interesting questions.
The “latest assessment” to which Richard Tol refers at the end of the article above, a working paper posted on the University of Sussex website with the title ‘Economics of climate change’, appears to contain a major error in Table 1 on page 11. It lists a paper by Nordhaus and Yang (1996) as finding that global warming of 2.5ºC would result in a change in welfare-equivalent income of +1.4%. In fact, Table 2 in Nordhaus and Yang (1996) shows that each of the six regions has a climate-damage function that is negative for warming of 2.5ºC: http://www.jstor.org/stable/2118303?seq=1#page_scan_tab_contents
It is difficult to see how any sensible aggregation could result in a positive global impact on GDP. Furthermore, the value listed in Tol’s new working paper for Nordhaus and Yang (1996) is at odds with Chapter 10 of the contribution of Working Group II to the IPCC Fifth assessment Report, which lists Nordhaus and Yang as finding a total loss of -1.7% of GDP due to warming of 2.5ºC (Table SM10.2): http://www.ipcc.ch/pdf/assessment-report/ar5/wg2/supplementary/WGIIAR5-Chap10_OLSM.pdf
This error has major implications for the analysis in Tol’s new working paper and its conclusions, including the following claim on page 2: “The 11 estimates for 2.5ºC show that researchers disagree on the sign of the net impact: 4 are positive, and 7 negative”. In fact, only one of the 11 estimates is positive – Mendelsohn et al (2000) which found values of 0 and 0.1%.
I see that Professor Tol has now replaced his working paper with an amended version in which the result attributed to Nordhaus and Yang (1996) is now -1.4% GDP. But other discrepancies remain in the amended paper.