Does Model Calibration Reduce Uncertainty in Climate Projections?

Simon F. B. Tett aSchool of Geosciences, University of Edinburgh, Edinburgh, United Kingdom

Search for other papers by Simon F. B. Tett in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0001-7526-560X
,
Jonathan M. Gregory bNational Centre for Atmospheric Science, University of Reading, Reading, United Kingdom
cMet Office Hadley Centre, Exeter, United Kingdom

Search for other papers by Jonathan M. Gregory in
Current site
Google Scholar
PubMed
Close
,
Nicolas Freychet aSchool of Geosciences, University of Edinburgh, Edinburgh, United Kingdom

Search for other papers by Nicolas Freychet in
Current site
Google Scholar
PubMed
Close
,
Coralia Cartis dMathematical Institute, University of Oxford, Oxford, United Kingdom

Search for other papers by Coralia Cartis in
Current site
Google Scholar
PubMed
Close
,
Michael J. Mineter aSchool of Geosciences, University of Edinburgh, Edinburgh, United Kingdom

Search for other papers by Michael J. Mineter in
Current site
Google Scholar
PubMed
Close
, and
Lindon Roberts eMathematical Sciences Institute, Australian National University, Canberra, Australian Capital Territory, Australia

Search for other papers by Lindon Roberts in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Uncertainty in climate projections is large as shown by the likely uncertainty ranges in equilibrium climate sensitivity (ECS) of 2.5–4 K and in the transient climate response (TCR) of 1.4–2.2 K. Uncertainty in model projections could arise from the way in which unresolved processes are represented, the parameter values used, or the targets for model calibration. We show that, in two climate model ensembles that were objectively calibrated to minimize differences from observed large-scale atmospheric climatology, uncertainties in ECS and TCR are about 2–6 times smaller than in the CMIP5 or CMIP6 multimodel ensemble. We also find that projected uncertainties in surface temperature, precipitation, and annual extremes are relatively small. Residual uncertainty largely arises from unconstrained sea ice feedbacks. The more than 20-year-old HadAM3 standard model configuration simulates observed hemispheric-scale observations and preindustrial surface temperatures about as well as the median CMIP5 and CMIP6 ensembles while the optimized configurations simulate these better than almost all the CMIP5 and CMIP6 models. Hemispheric-scale observations and preindustrial temperatures are not systematically better simulated in CMIP6 than in CMIP5 although the CMIP6 ensemble seems to better simulate patterns of large-scale observations than the CMIP5 ensemble and the optimized HadAM3 configurations. Our results suggest that most CMIP models could be improved in their simulation of large-scale observations by systematic calibration. However, the uncertainty in climate projections (for a given scenario) likely largely arises from the choice of parameterization schemes for unresolved processes (“structural uncertainty”), with different tuning targets another possible contributor.

Significance Statement

Climate models represent unresolved phenomena controlled by uncertain parameters. Changes in these parameters impact how well a climate model simulates current climate and its climate projections. Multiple calibrations of a single climate model, using an objective method, to large-scale atmospheric observations are performed. These models produce very similar climate projections at both global and regional scales. An analysis that combines uncertainties in observations with simulated sensitivity to observations and climate response also has small uncertainty showing that, for this model, current observations constrain climate projections. Recently developed climate models have a broad range of abilities to simulate large-scale climate with only some improvement in their ability to simulate this despite a decade of model development.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Simon Tett, simon.tett@ed.ac.uk

Abstract

Uncertainty in climate projections is large as shown by the likely uncertainty ranges in equilibrium climate sensitivity (ECS) of 2.5–4 K and in the transient climate response (TCR) of 1.4–2.2 K. Uncertainty in model projections could arise from the way in which unresolved processes are represented, the parameter values used, or the targets for model calibration. We show that, in two climate model ensembles that were objectively calibrated to minimize differences from observed large-scale atmospheric climatology, uncertainties in ECS and TCR are about 2–6 times smaller than in the CMIP5 or CMIP6 multimodel ensemble. We also find that projected uncertainties in surface temperature, precipitation, and annual extremes are relatively small. Residual uncertainty largely arises from unconstrained sea ice feedbacks. The more than 20-year-old HadAM3 standard model configuration simulates observed hemispheric-scale observations and preindustrial surface temperatures about as well as the median CMIP5 and CMIP6 ensembles while the optimized configurations simulate these better than almost all the CMIP5 and CMIP6 models. Hemispheric-scale observations and preindustrial temperatures are not systematically better simulated in CMIP6 than in CMIP5 although the CMIP6 ensemble seems to better simulate patterns of large-scale observations than the CMIP5 ensemble and the optimized HadAM3 configurations. Our results suggest that most CMIP models could be improved in their simulation of large-scale observations by systematic calibration. However, the uncertainty in climate projections (for a given scenario) likely largely arises from the choice of parameterization schemes for unresolved processes (“structural uncertainty”), with different tuning targets another possible contributor.

Significance Statement

Climate models represent unresolved phenomena controlled by uncertain parameters. Changes in these parameters impact how well a climate model simulates current climate and its climate projections. Multiple calibrations of a single climate model, using an objective method, to large-scale atmospheric observations are performed. These models produce very similar climate projections at both global and regional scales. An analysis that combines uncertainties in observations with simulated sensitivity to observations and climate response also has small uncertainty showing that, for this model, current observations constrain climate projections. Recently developed climate models have a broad range of abilities to simulate large-scale climate with only some improvement in their ability to simulate this despite a decade of model development.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Simon Tett, simon.tett@ed.ac.uk
Save
  • Caldwell, P. M., M. D. Zelinka, and S. A. Klein, 2018: Evaluating emergent constraints on equilibrium climate sensitivity. J. Climate, 31, 39213942, https://doi.org/10.1175/JCLI-D-17-0631.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cartis, C., J. Fiala, B. Marteau, and L. Roberts, 2019: Improving the flexibility and robustness of model-based derivative-free optimization solvers. ACM Trans. Math. Software, 45, 32, https://doi.org/10.1145/3338517.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Charney, J. G., and Coauthors, 1979: Carbon dioxide and climate: A scientific assessment. Report of an ad hoc study group on carbon dioxide and climate, Woods Hole, Massachusetts, July 23–27, 1979, to the Climate Research Board, Assembly of Mathematical and Physical Sciences, National Research Council. National Academies Press, 34 pp., http://www.nap.edu/catalog/12181.html.

    • Search Google Scholar
    • Export Citation
  • Gillett, N. P., V. K. Arora, D. Matthews, and M. R. Allen, 2013: Constraining the ratio of global warming to cumulative CO2 emissions using CMIP5 simulations. J. Climate, 26, 68446858, https://doi.org/10.1175/JCLI-D-12-00476.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gordon, C., C. Cooper, C. A. Senior, H. Banks, J. M. Gregory, T. C. Johns, J. F. B. Mitchell, and R. A. Wood, 2000: The simulation of SST, sea ice extents and ocean heat transports in a version of the Hadley Centre coupled model without flux adjustments. Climate Dyn., 16, 147168, https://doi.org/10.1007/s003820050010.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gregory, J. M., and Coauthors, 2004: A new method for diagnosing radiative forcing and climate sensitivity. Geophys. Res. Lett., 31, L03205, https://doi.org/10.1029/2003GL018747.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gregory, J. M., T. Andrews, and P. Good, 2015: The inconstancy of the transient climate response parameter under increasing CO2. Philos. Trans. Roy. Soc., A373, 20140417, https://doi.org/10.1098/rsta.2014.0417.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grose, M. R., J. Gregory, R. Colman, and T. Andrews, 2018: What climate sensitivity index is most useful for projections? Geophys. Res. Lett., 45, 15591566, https://doi.org/10.1002/2017GL075742.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hall, A., and X. Qu, 2006: Using the current seasonal cycle to constrain snow albedo feedback in future climate change. Geophys. Res. Lett., 33, L03502, https://doi.org/10.1029/2005GL025127.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hourdin, F., and Coauthors, 2017: The art and science of climate model tuning. Bull. Amer. Meteor. Soc., 98, 589602, https://doi.org/10.1175/BAMS-D-15-00135.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IPCC, 2013: Climate Change 2013: The Physical Science Basis. T. F. Stocker et al., Eds., Cambridge University Press, 1535 pp.

  • IPCC, 2021: Climate Change 2021: The Physical Science Basis. Cambridge University Press, in press.

  • Jones, A., D. L. Roberts, M. J. Woodage, and C. E. Johnson, 2001: Indirect sulphate aerosol forcing in a climate model with an interactive sulphur cycle. J. Geophys. Res., 106, 20 29320 310, https://doi.org/10.1029/2000JD000089.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knight, C. G., and Coauthors, 2007: Association of parameter, software, and hardware variation with large-scale behavior across 57,000 climate models. Proc. Natl. Acad. Sci. USA, 104, 12 25912 264, https://doi.org/10.1073/pnas.0608144104.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knutti, R., M. A. A. Rugenstein, and G. C. Hegerl, 2017: Beyond equilibrium climate sensitivity. Nat. Geosci., 10, 727736, https://doi.org/10.1038/ngeo3017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Loeb, N. G., B. A. Wielicki, D. R. Doelling, G. L. Smith, D. F. Keyes, S. Kato, N. Manalo-Smith, and T. Wong, 2009: Toward optimal closure of the Earth’s top-of-atmosphere radiation budget. J. Climate, 22, 748766, https://doi.org/10.1175/2008JCLI2637.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lowe, J. A., and Coauthors, 2019: UKCP18 science overview report. Met Office Hadley Centre, 73 pp., https://www.metoffice.gov.uk/pub/data/weather/uk/ukcp18/science-reports/UKCP18-Overview-report.pdf.

    • Search Google Scholar
    • Export Citation
  • Mauritsen, T., and Coauthors, 2012: Tuning the climate of a global model. J. Adv. Model. Earth Syst., 4, M00A01, https://doi.org/10.1029/2012MS000154.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pope, V. D., M. L. Gallani, P. R. Rowntree, and R. A. Stratton, 2000: The impact of new physical parametrizations in the Hadley Centre climate model: HadAM3. Climate Dyn., 16, 123146, https://doi.org/10.1007/s003820050009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ringer, M., 2019: Global-mean data and analysis of CMIP6 experiments. Accessed March 2022, https://github.com/mark-ringer/cmip6.

  • Roach, L. A., S. F. B. Tett, M. J. Mineter, K. Yamazaki, and C. D. Rae, 2018: Automated parameter tuning applied to sea ice in a global climate model. Climate Dyn., 50, 5165, https://doi.org/10.1007/s00382-017-3581-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rowlands, D., and Coauthors, 2012: Broad range of 2050 warming from an observationally constrained large climate model ensemble. Nat. Geosci., 5, 256260, https://doi.org/10.1038/ngeo1430.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanderson, B. M., 2011: A multimodel study of parametric uncertainty in predictions of climate response to rising greenhouse gas concentrations. J. Climate, 24, 13621377, https://doi.org/10.1175/2010JCLI3498.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanderson, B. M., A. G. Pendergrass, C. D. Koven, F. Brient, B. B. B. Booth, R. A. Fisher, and R. Knutti, 2021: The potential for structural errors in emergent constraints. Earth Syst. Dyn., 12, 899918, https://doi.org/10.5194/esd-12-899-2021.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schlund, M., A. Lauer, P. Gentine, S. C. Sherwood, and V. Eyring, 2020: Emergent constraints on equilibrium climate sensitivity in CMIP5: Do they hold for CMIP6? Earth Syst. Dyn., 11, 12331258, https://doi.org/10.5194/esd-11-1233-2020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sexton, D. M. H., and Coauthors, 2021: A perturbed parameter ensemble of HadGEM3-GC3.05 coupled model projections: Part 1: Selecting the parameter combinations. Climate Dyn., 56, 33953436, https://doi.org/10.1007/s00382-021-05709-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sherwood, S. C., and Coauthors, 2020: An assessment of Earth’s climate sensitivity using multiple lines of evidence. Rev. Geophys., 58, e2019RG000678, https://doi.org/10.1029/2019RG000678.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stainforth, D. A., and Coauthors, 2005: Uncertainty in predictions of the climate response to rising levels of greenhouse gases. Nature, 433, 403406, https://doi.org/10.1038/nature03301.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Taylor, K. E., 2001: Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res., 106, 71837192, https://doi.org/10.1029/2000JD900719.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tett, S. F. B., M. J. Mineter, C. Cartis, D. J. Rowlands, and P. Liu, 2013a: Can top of atmosphere radiation measurements constrain climate predictions? Part 1: Tuning. J. Climate, 26, 93489366, https://doi.org/10.1175/JCLI-D-12-00595.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tett, S. F. B., D. J. Rowlands, M. J. Mineter, and C. Cartis, 2013b: Can top of atmosphere radiation measurements constrain climate predictions? Part 2: Climate sensitivity. J. Climate, 26, 93679383, https://doi.org/10.1175/JCLI-D-12-00596.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tett, S. F. B., K. Yamazaki, M. J. Mineter, C. Cartis, and N. Eizenberg, 2017: Calibrating climate models using inverse methods: Case studies with HadAM3, HadAM3P and HadCM3. Geosci. Model Dev., 10, 35673589, https://doi.org/10.5194/gmd-10-3567-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Williamson, D., M. Goldstein, L. Allison, A. Blaker, P. Challenor, L. Jackson, and K. Yamazaki, 2013: History matching for exploring and reducing climate model parameter space using observations and a large perturbed physics ensemble. Climate Dyn., 41, 17031729, https://doi.org/10.1007/s00382-013-1896-4.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yamazaki, K., and Coauthors, 2013: Obtaining diverse behaviors in a climate model without the use of flux adjustments. J. Geophys. Res. Atmos., 118, 27812793, https://doi.org/10.1002/jgrd.50304.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yamazaki, K., D. M. H. Sexton, J. W. Rostron, C. F. McSweeney, J. M. Murphy, and G. R. Harris, 2021: A perturbed parameter ensemble of HadGEM3-GC3.05 coupled model projections: Part 2: Global performance and future changes. Climate Dyn., 56, 34373471, https://doi.org/10.1007/s00382-020-05608-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zelinka, M. D., T. A. Myers, D. T. McCoy, S. Po-Chedley, P. M. Caldwell, P. Ceppi, S. A. Klein, and K. E. Taylor, 2020: Causes of higher climate sensitivity in CMIP6 models. Geophys. Res. Lett., 47, e2019GL085782, https://doi.org/10.1029/2019GL085782.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 610 0 0
Full Text Views 758 386 33
PDF Downloads 734 334 24