The author of a controversial 2009 paper arguing that at least some amount of global warming could lead to economic gains has corrected the paper, along with a later article in a different journal. We confess to be baffled by the implications of the mix-up, although others appear to be less confused.
The 2009 article, “The Economic Effects of Climate Change,” was written by Richard Tol, of the University of Sussex, and appeared in the Journal of Economic Perspectives. It has been cited 91 times, according to Thomson Scientific’s Web of Science.
Here’s the abstract:
I review the literature on the economic impacts of climate change, an externality that is unprecedentedly large, complex, and uncertain. Only 14 estimates of the total damage cost of climate change have been published, a research effort that is in sharp contrast to the urgency of the public debate and the proposed expenditure on greenhouse gas emission reduction. These estimates show that climate change initially improves economic welfare. However, these benefits are sunk. Impacts would be predominantly negative later in the century. Global average impacts would be comparable to the welfare loss of a few percent of income, but substantially higher in poor countries. Still, the impact of climate change over a century is comparable to economic growth over a few years. There are over 200 estimates of the marginal damage cost of carbon dioxide emissions. The uncertainty about the social cost of carbon is large and right-skewed. For a standard discount rate, the expected value is $50/tC, which is much lower than the price of carbon in the European Union but much higher than the price of carbon elsewhere. Current estimates of the damage costs of climate change are incomplete, with positive and negative biases. Most important among the missing impacts are the indirect effects of climate change on economic development; large-scale biodiversity loss; low-probability, high-impact scenarios; the impact of climate change on violent conflict; and the impacts of climate change beyond 2100. From a welfare perspective, the impact of climate change is problematic because population is endogenous, and because policy analyses should separate impatience, risk aversion, and inequity aversion between and within countries.
But according to the correction notice:
Gremlins intervened in the preparation of my paper “The Economic Effects of Climate Change” published in the Spring 2009 issue of this journal. In Table 1 of that paper, titled “Estimates of the Welfare Impact of Climate Change,” minus signs were dropped from the two impact estimates, one by Plambeck and Hope (1996) and one by Hope (2006). In Figure 1 of that paper, titled “Fourteen Estimates of the Global Economic Impact of Climate Change,” and in the various analyses that support that figure, the minus sign was dropped from only one of the two estimates. The corresponding Table 1 and Figure 1 presented here correct these errors. Figure 2 titled,”Twenty-One Estimates of the Global Economic Impact of Climate Change” adds two overlooked estimates from before the time of the original 2009 paper and five more recent ones.
The second article, “Targets for global climate policy: An overview,” appeared in the May 2013 issue of the Journal of Economic Developments and Control. It attempted to correct the 2009 paper, and its abstract states:
A survey of the economic impact of climate change and the marginal damage costs shows that carbon dioxide emissions are a negative externality. The estimated Pigou tax and its growth rate are too low to justify the climate policy targets set by political leaders. A lower discount rate or greater concern for the global distribution of income would justify more stringent climate policy, but would imply an overhaul of other public policies. Catastrophic risk justifies more stringent climate policy, but only to a limited extent.
Its correction reads:
The author regrets that three errors crept into “Targets for global climate policy: An overview”. In Table 1 of that paper, the author dropped the minus sign on Chris Hope׳s 2006 estimate; his best guess is that a global warming by 2.5 °C would cause a welfare loss equivalent to losing 0.9% of income, with a confidence interval spanning −2.7% to 0.2%. This error also affects Fig. 1. Fig. 1 had a coding error in the upper bound of the confidence interval, and an error in the caption. Corrected Fig. 1 is given below.
17 Estimates of the global economic impact of climate change, expressed as the welfare-equivalent income loss, as a function of the increase in global mean temperature relative to pre-industrial times. The blue dots represent the estimates. The central line is the least squares fit to the 17 observations: D=3.99(1.39)T−1.82(0.52)T2, R2=0.67, where D denotes impact and T denotes temperature. The outer lines are the boundaries of the 95% confidence interval, where the standard deviation is the least squares fit to the five reported standard deviations or half confidence intervals: Soptimistic=0.43(0.16)T, R2=0.65 and Spessimistic=1.82(0.86)T, R2=0.53, where S is the standard deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
The blog of the London School of Economics has an interesting piece on the implications of the mistake. It reads, in part:
A correction to a paper quietly published by an economics journal last week has put pressure on the Intergovernmental Panel on Climate Change (IPCC) to amend a section of its new report that claimed that moderate amounts of global warming have overall positive benefits. The correction by Richard Tol, a professor of economics at the University of Sussex, has updated a paper he published in the Journal of Economic Perspectives in 2009 which collected together the results of studies of the economic impacts of climate change.
The original paper suggested that there were net beneficial impacts up to about 2.2 centigrade degrees of warming, and has been widely cited by climate change ‘sceptics’, including Viscount Ridley who used it as the basis of an article in The Spectator in October 2013 which claimed that “climate change has done more good than harm so far and is likely to continue doing so for most of this century”.
However, after I raised concerns with Professor Tol and with the journal about a number of errors in the article, a correction was quietly posted on the journal’s website last week. The correction points out that the original paper concluded that “there were net benefits of climate change associated with warming below about 2°C”, but the updated analysis shows “impacts are always negative”.
This is significant because a more recent paper by Professor Tol, which was published by the Journal of Economic Dynamics and Control in 2013 but contained many of the same errors as the 2009 paper, was used as the basis for a section on the economic damage caused by climate change in the report of the IPCC on ‘Impacts, Adaptation and Vulnerability’.
The final version of the Summary for Policymakers of the IPCC report was published on 31 March, along with the underlying final drafts of the chapters. Section 10.9.2 of Chapter 10, on which Professor Tol is a Coordinating Lead Author, draws on his 2013 paper and states: “Climate change may be beneficial for moderate climate change but turn negative for greater warming”. The chapter also includes Figure 10-1 which plots 20 of the 21 data points included in the correction by Professor Tol to his 2009 paper.
I have now drawn the attention of the IPCC to Professor Tol’s correction and suggested that it needs to amend the text of Chapter 10 before the final version is published later this year. The amendment is also important to ensure that the error does not appear in the IPCC Synthesis Report, which is due to be published at the end of October. I have also asked Viscount Ridley, via Twitter, to amend his article in The Spectator now that Professor Tol has published a correction. Lord Ridley has so far refused.
Tol, for his part, tells us that the errors don’t undermine his conclusions:
Although the numbers have changed, the conclusions have not. The difference between the new and old results is not statistically significant. There is no qualitative change either.
Update, 12 p.m. Eastern, 5/21/14: Tol added more comments in response to the LSE post below.
Hat tip: Thomas Baguley
The full pdf of the correction explains Tol’s opinion on the consequences of the correction
http://www.aeaweb.org/atypon.php?return_to=/doi/pdfplus/10.1257/jep.28.2.221
In particular Tol explains that the change is policy-irrelevant, as the ‘benefits’ happen(ed) regardless of policy, and the new graph suggests a slower increase in cost as global warming increases.
It should also be noted that according to the disclaimer of the bottom of the LSE blog, the content of that post are Bob Ward’s opinions only. Ward and Tol have a history of disagreement and that might explain why one considers small an error the other sees as very important.
The article contains a long quote by Mr Robert ET Ward BSc, without comment. Unfortunately, it contains errors.
The Spectator piece Dr Matthew W. Ridley FMedSci draws on a paper I published last year in Climatic Change. It is a separate piece of analysis, unaffected by the errors discussed here.
Mr Ward cites rather selectively from the JEP corrigendum, which states explicitly that the assessment for the short run has not changed. This was omitted by Mr Ward.
Mr Ward draws undue conclusions from a variable that changed from positive-but-insignificant to negative-but-insignificant.
Mr Ward also omits that the assessment of the impacts of profound climate change has been revised: We are now less pessimistic than we used to be.
Mr Ward suggests that the IPCC would be reluctant to correct these errors. This is not the case. The IPCC does not issue errata on drafts. The version to-be-published will not contain these errors.
Mr Ward was aware of the above points well before 9 May 2014.
Mistakes happen and it is great when they are corrected. What is unforgivable is the way that the AEA journals handle corrected and retracted papers. There is no notice that there is a problem with the Tol paper at the JEP website for the original article or a 2002 AER article that was retracted in 2007. Researchers can still do a literature search, find the paper, read it and cite it in their own research without any idea that there is a problem. It seems like an easy thing to post a correction or retraction notice at the website for the original article and put a watermark on the original PDF.
What Tol is saying is that if one includes his outlier even with corrections the curve is rather banana like with a net positive benefit at modest warming. As Frank Ackerman has shown this is a result of the curious way in which Tol’s FUND model calculates agricultural effects, the positive benefit is large and completely different from every other study.
If you do not include the Tol and Anthoff result, then you pretty much get a simple aX^2 fit which is negative everywhere, rather than what Tol shows (see And Then There is Physics for a long discussion of this). If you force the fit using Tol’s data, then to encompass the other results the curve has to descend more rapidly past 3 C of warming taking into account the other models including the questionable Tol mode. This means that Tol’s fit predicts more damage at higher warming than the aX^2 fit however, as the IPCC AR5 points out, Integrated Assessment models (IAMs) past 3C warming are economic fiction, because the damage would be so great and from so many directions not included in the IAMs that the models lose validity.
Ackerman’s “contribution” is entirely unrelated to this issue. Anthoff has nothing to do with it either.
FUND is Anthoff and Tol. Ackerman was criticizing the large positive excursion in Anthoff and Tol due to supposed agricultural benefits. The result anchors your fits in both the mathematical and personal sense.
None of that has anything to do with the correction. If interested open a new blog TolWatch or FUNDWatch.
Leave Anthoff out of it. He is not cited. He had nothing to do with it.
Neither has FUND.
Ackerman claims to criticize FUND, but its strawmen all the way.
Richard, you wrote
“Mr Ward also omits that the assessment of the impacts of profound climate change has been revised: We are now less pessimistic than we used to be.”
This is, at best, a rather misleading statement. And yes, that outlier at 1.0 C is from FUND and specifically from FUND as developed by Tol and Anthoff mentioning your collaborator is in no way pejorative.
However, the real issue with that rather high estimate is where it comes from. According to Ackerman it is as a result of very high benefits FUND calculates for the agricultural sector at 1.0 C warming. Is this the case? If not, where does the majority of benefits come from? There are two extreme outliers in Fig 1. Yours at the beginning and the hugely negative one at the end. Outlers at extrema distort regressions.
The estimate for 1K is by Tol; rather than by Anthoff and Tol; and it is not a result from FUND. The benefits have little to do with agriculture, and a lot with health and energy.
The blog reports that Dr. Tol found that the new results are not statistically different than the original results. I question whether this is an appropriate test since there is no a priori reason to select the original results as the null hypothesis and the use of a null hypothesis generated from the same data changes the alpha level and degrees of freedom. A more interesting question (and one that can be tested more robustly) is whether the new result differs significantly from a result that is negative at all temperatures where results are available.
The original post rather unhelpfully left out the graph and table that demonstrate insignificance. They can be found here: http://www.aeaweb.org/articles.php?doi=10.1257/jep.28.2.221
Dr. Tol, thanks for pointing to the graph and table. Do you have a response to the points that I made?
I’m not sure I get your first point. I would think that the null hypothesis is whether there is a different pattern between the old and new results. There is not.
The second point is answered in the paper. Impacts are not significantly different from zero until about 2K warming.
forgot to say: the differences in the old and new results are driven by new observations, rather than corrected old ones …
Bob Ward ( http://blogs.lse.ac.uk/politicsandpolicy/archives/41855 ) wrote: “Despite numerous requests, Professor Tol has so far failed to make available the details of these aggregations so that I might check them for further errors.”
Please note that the 2009 paper (Tol, Richard S J. 2009. “The Economic Effects of Climate Change.” Journal of Economic Perspectives, 23(2): 29-51) lists Free University Amsterdam, The Netherlands as one of the affiliated institutes.
Please note that all researchers of Free University Amsterdam must strictly work according to “The Netherlands Code of Conduct for Scientific Practice” ( http://www.vsnu.nl/files/documenten/Domeinen/Onderzoek/The_Netherlands_Code_of_Conduct_for_Scientific_Practice_2012.pdf ).
Page 7 of the Code: “III. Verifiability. Presented information is verifiable. Whenever research results are publicized, it is made clear what the data and the conclusions are based on, where they were derived from and how they can be verified. (…). III.3 Raw research data are stored for at least five years. These data are made available to other scientific practitioners at request.”
I fail to understand why Bob Ward is unable to get these details.
The data are all in the public domain.
Klaas, is the reason “why Bob Ward is unable to get these details” due to intransigence on Tol’s part (not sending them) or that Ward is unable to find them himself?
Carrick, no idea what’s the real reason why Bob Ward is unable to get these details. Richard Tol does not provide insight where these data can be found in the public domain. The blog of Bob Ward reveals that both had contact with each other, so I don’t understand what’s going on here. It seems to me that it must be not too tough to find the online source of these raw data. Please note that the main topic (economics and climate change) is way ahead from my field of research.
The significance of the correction notwithstanding, I’m amazed that any journal would allow such near-Sartrean examples of displacement as “gremlins intervened in the preparation of my paper” and “errors crept into” to pass.
I have a colleague, with whom I’ve coauthored a number of papers, who has a similar taste for metaphor in his papers. Generally it’s been tolerated by reviewers.
Concern About papers of Azusa Okagawa(National Institute for Environmental Studies)
http://bestbetter.up.seesaa.net/image/matome.pdf
http://nieskaizan.doorblog.jp
On his blog, Andrew Gelman friendly but firmly deconstructs the use of “statistics” by Tol until nothing is left of the arguments, revised or not.
http://www.washingtonpost.com/blogs/monkey-cage/wp/2015/05/23/the-gremlins-did-it-iffy-curve-fit-drives-strong-policy-conclusions/
and more comments here:
http://andrewgelman.com/2014/05/23/gremlins-iffy-statistics-drive-strong-policy-recommendations/
Gelman writes one more post on it:
http://andrewgelman.com/2014/05/27/whole-fleet-gremlins-looking-carefully-richard-tols-twice-corrected-paper-economic-effects-climate-change/
Tol respons in comments
There is a subtle but meaningful difference between the labels of the x-axis in the original Figure 1 of Tol (2009) and the updated Figures 1 and 2 and these changes presumably carry implications throughout the text and analysis. In the original Figure 1, the x-axis is labelled temperature change relative to ‘today.’ In the new Figures, the x-axis is labelled as temperature change relative to pre-industrial temperatures. The term ‘pre-industrial’ derives from climate scientist’s interest in anthropogenic influences on CO2 levels and they usually assign a date of 1750 as the last time natural CO2 levels were observed. A temperature change of 0 relative to pre-industrial temperatures, existed in 1750. On the newly labelled x-axis, we are ‘today’ at +0.8C.
Were the economic analyses that form the source of these plots making comparisons to pre-industrial (i.e. 1750) GDP or that of ‘today?’ It would not be surprising for economists to suggest the GDP is higher now (at +0.8C) than in pre-industrial times yet GDP% changes presented in Figures 1 and 2 are nearly always negative. Whereas it is correct to assign a GDP% change of 0.0 at the x-axis value of zero if both represent pre-industrial (1750) values. On an x-axis scale of temperature change relative to 1750 we expect a positive dGDP% at +0.8C. Thus, points may be mis-plotted on the x-axis.
It would be difficult to imagine economists interested in (or reporting) the effect of climate change on GDP in comparison to the GDP at pre-industrial times, temperatures, and CO2 levels. More likely the economic analyses were performed relative to the GDP of ‘today’ at the time of each publication (e.g. at +0.8C relative to pre-industrial levels in 2014) but calibrated with climate models that expressed results relative to 1750 levels (usually taken at 285ppm).
Climate scientists commonly compare climate model results to those they compute for pre-industrial levels of CO2. Rarely do climate scientists express results of temperature change relative to CO2 levels of ‘today’ since that is a rapidly moving target but some model results may be reported for CO2 levels like those we see now (400ppm). Climate model results available in the early ’90’s used values like 300ppm or 330ppm to describe ‘today.’ Thus, the meaning of ‘today’ is a potential source of confusion between the two disciplines.
Due to non-linearities and lags in the earth system, a +1C change relative to 1750 has a different impact than a +1C change relative to today’s temperature (already at +0.8C relative to 1750). Presumably, economists calibrate their analyses using available climate model results. In that case (and given the 20-year span of studies cited), each study must be considered with respect to whether the underlying climate model results were expressed relative to some ‘today’ or to pre-industrial levels. This can shift the x-axis location of the data.
Given the shift in labelling of the x-axes, with no adjustments in the position of the data points between the 2009 and current versions of the plots, it is not clear that the author considered the importance of these distinctions so caution in interpreting the relation of the underlying data to published plots is in order.
I am not an economist but would love for an economist to reassure me that the y-axes are compatible in the various studies. For example, I note that Nordhaus (1994a in the original Tol 2009 paper) interprets most of his interviewees estimates as change in ‘Gross World Product’ (apparently some respondents were working in different measures) not GDP. Since he found results to be highly skewed he reported median as well as mean. In this case median may be a better maximum likelihood estimator and is the variable Professor Tol lists in his newer analysis although it seems he reported a misreading of the mean in his original paper. Are the results reported for other authors uniformly the median? Nordhaus (1994a) also reports two estimates for +6C (-6.7% and -10.4%). These are not included in Prof. Tol’s plots although they certainly seem relevant to the new Figure 2.
I am not sure I am following you Rachael. The estimates in each study concern the effect of a specific increase in surface temperatures (“benchmark warming”). If the costs depend on the initial conditions and historical situation, those studies are meaningless apart than as historical curiosities, and impossible to compare. In a sense, those studies are about the impacts of a marginal change in temperatures (there is of course an upper limit above which these marginal changes aren’t marginal any longer – Tol .
Likewise there isn’t anybody in the world that would affirm that the GDP increase since pre-industrial times is attributable to the +0.8C increase in global air and sea surface temperatures. Following Tol’s review, one could try to identify how many growth has been perhaps lost due that increase, but I do not imagine there’s much interest in that field of research.
Finally I am surprised by all this discussion on details of a paper that concludes “The policy implication
is that reduction of greenhouse gas emissions should err on the ambitious side” (my emphasis). Would the new all-negative results have had a different policy implication and if so how.
sorry…that should be – (there is of course an upper limit above which these marginal changes aren’t marginal any longer – Tol says as much himself)
My point is that if one changes the x-axis labels as was done, then the locations of points on the x-axis change, which was not done. Without that change in the locations of points in the x-axis, the interpretation of the y-axis value of those same points is called into question.
As to the ‘surprise’ of the extent of discussion, I can only speak for myself and my interest is trying to understand what I read. I am grateful for this forum since I cannot afford a subscription to the Journal of Economic Perspectives which would allow me to post there and which I appreciate would be a more appropriate forum for academic discussion.
An FYI comment … this 5 May post highling Richard Tol’s problems with gremlins had some reasonable traffic … http://getenergysmartnow.com/2014/05/05/the-power-of-passive/
The identification and classification of the Gremlins as a species would be at least a nature or science paper!
Worth noting, Tol has now corrected his own correction with further corrections.
Gremlins are apparently pretty tough creatures, there now is yet another (and proclaimed “final” correction), but not from the author himself.
http://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.29.1.217