A new paper from Hausfather and colleagues (incl. me) has just been published with the most comprehensive assessment of climate model projections since the 1970s. Bottom line? Once you correct for small errors in the projected forcings, they did remarkably well.
Climate models are a core part of our understanding of our future climate. They also have been frequently attacked by those dismissive of climate change, who argue that since climate models are inevitably approximations they have no predictive power, or indeed, that they aren’t even scientific.
In an upcoming paper in Geophysical Research Letters, Zeke Hausfather, Henri Drake, Tristan Abbott and I took a look at how well climate models have actually been able to accurately project warming in the years after they were published. This is an extension of the comparisons we have been making on RealClimate for many years, but with a broader scope and a deeper analysis. We gathered all the climate models published between 1970 and the mid-2000s that gave projections of both future warming and future concentrations of CO2 and other climate forcings – from Manabe (1970) and Mitchell (1970) through to CMIP3 in IPCC 2007.
We found that climate models – even those published back in the 1970s – did remarkably well, with 14 out of the 17 projections statistically indistinguishable from what actually occurred.
We evaluated these models both on how well modeled warming compared with observed warming after models were published, and how well the relationship between warming and CO2 (and other climate forcings) in models compares to observations (the implied transient climate response) (see Figure). The second approach is important because even if an old model had gotten all the physics right, the future projected warming would be off if they assumed we would have 450 ppm CO2 in 2020 (which some did!). Future emissions depend on human societal behavior, not physical systems, and we can usefully distinguish evaluation of climate models physics from paths of future concentrations.
However, it is not totally obvious how one should correct for the forcing assumptions because of subtle issues related to the different efficacy of different forcings and, of course, the remaining uncertainty in the real value of the actual forcings (driven predominantly by the aerosol component). For forcing projections that were close to linear, this didn’t make that much difference, but for scenarios that weren’t (notably scenario C in Hansen et al (1988)), the correction does not work well.
There are a few other results that stand out, notably the (infamous?) low sensitivity result in Rasool and Schneider (1971), which was mainly due to a lack of stratospheric adjustment and water vapor short wave absorption in their formulation. This was noted by Schneider (1975) and the calculation redone by Schneider and Thompson (1981) which turned out to be far more accurate. On the other hand, only Mitchell (1970) appears to have substantially overestimated the TCR – even while he predicted the temperature rise quite accurately (due to a compensation between a too large sensitivity and an underestimate of the forcings). [Amusing aside, both Manabe’s and Mitchell’s 1970 projections appeared in a special volume on the Global Effects of Environmental Pollution, reporting on an 1968 AAAS workshop and edited by (the now-notorious) S. Fred Singer before he went off the deep end].
It’s worth noting that this comparison includes two kinds of climate model – those published prior to 1988 which are energy balance models of varying complexity, and those published afterwards which are true GCMs and include atmospheric (and eventually, ocean) dynamics. Of the early models, the work of Sawyer (1972) stands out as being the most accurate in terms of both temperature trends and forcings, though this must be considered somewhat fortuitous.
The fact that both classes of climate model did so well in projecting future warming should increase our confidence that current climate models are getting things right for mostly the right reasons. While there are still real uncertainties in future warming associated with climate sensitivity, we can confidently state that the rate of surface warming we are experiencing today is pretty much what past climate models projected it would be.
Gosh, maybe we know something about climate after all!
Note: all the data and code for this study are available here.
- Z. Hausfather, H.F. Drake, T. Abbott, and G.A. Schmidt, "Evaluating the Performance of Past Climate Model Projections", Geophysical Research Letters, vol. 47, 2020. http://dx.doi.org/10.1029/2019GL085378
- S.I. Rasool, and S.H. Schneider, "Atmospheric Carbon Dioxide and Aerosols: Effects of Large Increases on Global Climate", Science, vol. 173, pp. 138-141, 1971. http://dx.doi.org/10.1126/science.173.3992.138
- S.H. Schneider, "On the Carbon Dioxide–Climate Confusion", Journal of the Atmospheric Sciences, vol. 32, pp. 2060-2066, 1975. http://dx.doi.org/10.1175/1520-0469(1975)032<2060:OTCDC>2.0.CO;2
- S.H. Schneider, and S.L. Thompson, "Atmospheric CO2 and climate: Importance of the transient response", Journal of Geophysical Research: Oceans, vol. 86, pp. 3135-3147, 1981. http://dx.doi.org/10.1029/JC086iC04p03135
- "Global Effects of Environmental Pollution", 1970. http://dx.doi.org/10.1007/978-94-010-3290-2
- J.S. SAWYER, "Man-made Carbon Dioxide and the “Greenhouse” Effect", Nature, vol. 239, pp. 23-26, 1972. http://dx.doi.org/10.1038/239023a0