More accurate quantification of model-to-model agreement

In a new study scientists Dr Nicola Maher and Prof Jochem Marotzke from the Max Planck Institute for Meteorology (MPI-M) in collaboration with Prof Scott Power from Monash University (Melbourne) have more accurately quantified model-to-model agreement in strongly forced long-term projections of temperature, precipitation, and their temporal variability.

They quantified the relative roles of model-to-model differences and internal variability in causing uncertainty in climate model projections. They find that uncertainty across the globe in long-term projections of temperature and precipitation is dominated by model-to-model differences, but that this is not true for temporal temperature and precipitation variability. When assessing model-to-model agreement they can identify differences in the forced response to external forcing across the land surface, and find that although all models show warming under a strong forcing scenario, the magnitude of that warming varies by 4oC over Europe, Australia and Asia and 10oC over the Arctic.

Individual simulations of climate models differ when exposed to external forcing, such as that from increasing greenhouse gases, due to both structural differences between the models and their phase of chaotic internal climate variability. Which of the two causes is at work matters a great deal – if it is the structural differences, model-to-model agreement might increase in the future causing the uncertainty in the response to external forcing to decrease. If, however, it is the internal variability, the spread in projections will not decrease. Determining why model simulations differ is limited in the traditional use of multi-model ensembles of climate models due to small numbers of realisations. This is further complicated because the models are not independent, sharing both components and code. This can lead to overconfidence in projections. 

Figure 1: Percentage variance contribution of model-to-model differences to the sum of the variance due to model-to-model differences and internal variability. Shown for the SMILE estimates in the top row and the CMIP5 atmospheric sub-ensemble estimates in the bottom row. Shown for long-term projections of the response to external forcing for: a,e) mean-state temperature, b,f) mean-state precipitation, c,g) temporal temperature variability and d,h) temporal precipitation variability. Projections are taken for annual mean quantities for the period 2050-2099 (under strong forcing) compared to the period 1950-1999 (historical). (adapted from Maher et al, 2021 Fig. 1-4)

In this study, Maher and her co-authors use a new set of six largely independent single model initial-condition large ensembles (SMILEs) to separate the uncertainties due to internal variability and model differences for temperature, precipitation, and their temporal variability. By using SMILEs they can easily quantify both the externally forced response in each model (ensemble mean) and its internal variability (the spread of ensemble members). They additionally implement a new method for a multi-model ensemble (CMIP5), where they create small sub-ensembles of models that share an atmospheric component, which are hence not independent. They find that the uncertainty in temperature and precipitation projections is dominated by model-to-model differences (Figure 1; left columns). However, for the temporal variability of both quantities, the uncertainty due to internal variability is generally larger than model differences in the extra-tropics (Figure 1; right columns). This has an important implication; that increasing model-to-model agreement in this region for these quantities may not improve the spread of projections.

Figure 2: a) Agreement in the sign of the forced response in temperature (ΔTF), shown in colour for the SMILEs and using the '+' symbol for the CMIP5 atmospheric sub-ensembles. Forced response in each individual SMILE is shown for b) Europe and c) Australia and south-east Asia. Projections in each SMILE are taken as the ensemble mean annual temperature for the period 2050-2099 (RCP8.5) compared to the period 1950-1999 (historical). (adapted from Maher et al, 2021 Fig. 5+6)

Maher and her co-authors then ask whether climate models agree not only on the sign, but also on the magnitude of the forced response in the projections. For the simple example of mean-state temperature they confirm that there is model agreement that it will warm everywhere except the northern North Atlantic and the Southern Ocean in both the SMILEs and the CMIP5 sub-ensembles (Figure 2a). When focusing on the magnitude of the change in specific regions in the SMILEs, they find that over Europe, Australia and south-east Asia the projections disagree by up to 4oC, while over the Arctic this disagreement is up to 10oC (Figure 2b, c). This difference in magnitude can now be quantified much more accurately using the SMILEs and is assessed for precipitation, and temporal temperature and precipitation variability as well as temperature for three areas that are policy relevant (i.e. sections of the land surface, the Arctic, and the tropical Pacific) to illustrate how the SMILE results can be used in the original publication.

Original publication:
Maher, N., Power, S.B., and Marotzke, J. (2021) More accurate quantification of model-to-model agreement in externally forced climatic responses over the coming century. Nature Communications.


Dr Nicola Maher
Now at CIRES, University of Colorado, Boulder, USA
Email: nicola.maher@we dont want

Prof Dr Jochem Marotzke
Max Planck Institute for Meteorology
Phone: +49 (0) 40 41173 311 (Assistant Kornelia Müller)
Email: jochem.marotzke@we dont want