{"id":23769,"date":"2021-08-09T03:08:26","date_gmt":"2021-08-09T08:08:26","guid":{"rendered":"https:\/\/www.realclimate.org\/?p=23769"},"modified":"2021-08-09T03:08:29","modified_gmt":"2021-08-09T08:08:29","slug":"notallmodels","status":"publish","type":"post","link":"https:\/\/www.realclimate.org\/index.php\/archives\/2021\/08\/notallmodels\/","title":{"rendered":"#NotAllModels"},"content":{"rendered":"
\n\n

The biggest contribution scientists can make to #scicomm related to the newly released IPCC Sixth Assessment report<\/a>, is to stop talking about the multi-model mean.<\/p>\n\n\n\n\n\n\n\n

We’ve discussed the issues in the CMIP6 multi-model ensemble many times over the last couple of years – for instance here<\/a> and here<\/a>. There are two slightly contradictory features of this ensemble that feature in the new IPCC report – first is the increase in skill seen in CMIP6 compared to CMIP5 models. Biases in the Southern Ocean are less, similarly with sea ice extent or rainfall etc. but also the models as a whole are doing better in representing key modes of variability and their teleconnections. This is good news. The second issue is that the spread of the models’ climate sensitivity is much wider than in CMIP5, and specifically, much wider than the new IPCC assessed range. This is not so good – at least at first glance.<\/p>\n\n\n\n

The climate sensitivity constraints are discussed in the SPM:<\/p>\n\n\n\n

A.4.4 <\/strong>The equilibrium climate sensitivity is an important quantity used to estimate how the climate responds to radiative forcing. Based on multiple lines of evidence, the very likely <\/em>range of equilibrium climate sensitivity is between 2\u00b0C (high confidence<\/em>) and 5\u00b0C (medium confidence<\/em>). The AR6 assessed best estimate is 3\u00b0C with a likely <\/em>range of 2.5\u00b0C to 4\u00b0C (high confidence<\/em>), compared to 1.5\u00b0C to 4.5\u00b0C in AR5, which did not provide a best estimate.<\/p><\/blockquote>\n\n\n\n

I’ve plotted the CMIP6 climate sensitivities before, but here I have updated it to the latest compilation (from Mark Zelinka<\/a>) and added the likely<\/em> and very likely<\/em> assessed ranges from AR6.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

For reference, out of 50 models, 40 are within the very likely AR6 range, and 23 within the likely range. Nonetheless of the 8 models with ECS > 5\u00baC, most are from very highly respected groups whose models do very well against comparisons of the climatology, including the Hadley Center (3 models), NCAR (2 models), DoE and the Canadian Climate Center (1 model each). My personal assessment of the likely range would be more closely based on Sherwood et al (2020)<\/a><\/span> and so is a little narrower (as discussed here<\/a>) – but it will need a deeper dive into the main report to see why the ranges differ and that isn’t the main point here. I would note though that given the distribution of ECS seen here, it is marginal whether this is even consistent with sampling from a straightforward distribution.<\/p>\n\n\n\n

What does this mean in practice? It\u2019s well known and accepted that CMIP is an ensemble of opportunity and not a structured exploration of model parameter space, that models are not independent of each other (some common assumptions, common modules and very close variants from some model groups) and so it makes no sense to treat the ensemble as if it was a pdf with a nice properties. And yet in CMIP5 (and previously), this was done almost ubiquitously (even by me on the model-data comparison<\/a> page). Despite the many excellent reasons why ‘model democracy’ shouldn’t be the best thing to do, it often was.<\/p>\n\n\n\n

With CMIP5, the coincidence of the model range of sensitivity (2.1\u00b0C to 4.6\u00b0C) with a reasonable assessed range of 2 to 4.5\u00b0C meant that, even if it wasn’t strictly kosher, practically it didn’t make much difference. And even in the AR5 report, the only constrained projection (for sea ice area) turned out to be overfitted and did not properly account for internal variability. <\/p>\n\n\n\n

But now with CMIP6, the situation is different. There are now large differences between the constrained projections and ‘model democracy’. For the historical period, the differences are there, but the the unweighted model mean does ok when compared to the observations:<\/p>\n\n\n\n

\"\"
AR6 Fig SPM1b<\/figcaption><\/figure><\/div>\n\n\n\n

However, they are only plotting the 5-95% envelope and at least one of the high ECS models (NCAR CESM2) used the historical trends as a tuning target. Despite that, it’s clear that the model spread is excessive in the post 1990 period. Were the raw CMIP6 data to be extended into the future – particularly for the higher emissions scenarios, the differences would be even starker. Thus for temperature projections, the IPCC authors (sensibly) effectively screen the models for coherence with observed temperatures (following Tokarska et al. (2020)<\/a><\/span> and downweight models with ECS values outside the assessed range:<\/p>\n\n\n\n

\"\"
AR6 Figure SPM 8a<\/em><\/figcaption><\/figure><\/div>\n\n\n\n

The high (and low) ECS models to make a contribution in the maps associated with the impacts at specific Global Warming Levels (GWLs) of 1.5\u00baC, 2\u00baC and 4\u00baC (in figure SPM 5) so the useful information they contain is not lost. <\/p>\n\n\n\n

How then should we talk about these models? In my opinion, describing the properties of the multi-model mean or generalizing about the models as a whole is not sensible. Claims such as those made recently<\/a> that the CMIP6 ensemble ‘runs hot’ are very easily misconstrued to imply that all CMIP6 models have too high ECS values (or indeed all models in general), when really it is only a subset. Discussions of the mean CMIP6 sensitivity is, to my mind, pointless, not least because the ‘CMIP6 mean’ is based on a somewhat arbitrary selection of models that doesn’t take into account model independence nor the fact that CMIP6 itself is a moving target as more models are still being added to the database. And given that all the temperature projections in IPCC are constrained projections, the raw CMIP6 mean and its properties are simply irrelevant for any of the AR6 conclusions. <\/p>\n\n\n\n

It is true that *some* models have high ECS beyond what can be reconciled with our understanding of paleoclimate change, and in those models the cloud feedback particularly in the Southern Oceans is more positive than previously. But it is not the case that all the CMIP6 models ‘run hot’, nor is true that the model projections in AR6 are affected by these high ECS values. We should therefore avoid giving that impression.<\/p>\n\n\n\n

Many people have previously declared that ‘model democracy’ was flawed, but this is the report that has finally buried it. <\/p>\n

References<\/h2>\n
    \n
  1. <\/a>\nS.C. Sherwood, M.J. Webb, J.D. Annan, K.C. Armour, P.M. Forster, J.C. Hargreaves, G. Hegerl, S.A. Klein, K.D. Marvel, E.J. Rohling, M. Watanabe, T. Andrews, P. Braconnot, C.S. Bretherton, G.L. Foster, Z. Hausfather, A.S. von der Heydt, R. Knutti, T. Mauritsen, J.R. Norris, C. Proistosescu, M. Rugenstein, G.A. Schmidt, K.B. Tokarska, and M.D. Zelinka, \"An Assessment of Earth's Climate Sensitivity Using Multiple Lines of Evidence\", Reviews of Geophysics<\/i>, vol. 58, 2020. http:\/\/dx.doi.org\/10.1029\/2019RG000678<\/a>\n\n\n<\/li>\n
  2. <\/a>\nK.B. Tokarska, M.B. Stolpe, S. Sippel, E.M. Fischer, C.J. Smith, F. Lehner, and R. Knutti, \"Past warming trend constrains future warming in CMIP6 models\", Science Advances<\/i>, vol. 6, 2020. http:\/\/dx.doi.org\/10.1126\/sciadv.aaz9549<\/a>\n\n\n<\/li>\n<\/ol>\n\n<\/div> ","protected":false},"excerpt":{"rendered":"

    The spread of climate sensitivity in the CMIP6 models is wider than previous model ensembles and wider than the assessed range. But #notallmodels are running hot.<\/p>\n","protected":false},"author":2,"featured_media":23885,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[5,1,23,2],"tags":[],"class_list":{"0":"post-23769","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-climate-modelling","8":"category-climate-science","9":"category-ipcc","10":"category-paleoclimate","11":"entry"},"aioseo_notices":[],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/23769","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/comments?post=23769"}],"version-history":[{"count":7,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/23769\/revisions"}],"predecessor-version":[{"id":23902,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/23769\/revisions\/23902"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/media\/23885"}],"wp:attachment":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/media?parent=23769"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/categories?post=23769"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/tags?post=23769"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}