• Abramowitz, G., and Coauthors, 2019: Model dependence in multi-model climate ensembles: Weighting, sub-selection and out-of-sample testing. Earth Syst. Dyn., 10, 91105, https://doi.org/10.5194/esd-10-91-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Annan, J. D., and J. C. Hargreaves, 2010: Reliability of the CMIP3 ensemble. Geophys. Res. Lett., 37, L02703, https://doi.org/10.1029/2009GL041994.

  • Annan, J. D., and J. C. Hargreaves, 2011: Understanding the CMIP3 multimodel ensemble. J. Climate, 24, 45294538, https://doi.org/10.1175/2011JCLI3873.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Annan, J. D., and J. C. Hargreaves, 2017: On the meaning of independence in climate science. Earth Syst. Dyn., 8, 211224, https://doi.org/10.5194/esd-8-211-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bishop, C., 2007: Pattern Recognition and Machine Learning. 2nd ed. Springer-Verlag, 738 pp.

  • Bishop, C. H., and G. Abramowitz, 2013: Climate model dependence and the replicate Earth paradigm. Climate Dyn., 41, 885900, https://doi.org/10.1007/s00382-012-1610-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blum, A., J. Hopcroft, and R. Kannan, 2018: Foundations of Data Science. Cornell University, 454 pp., https://www.cs.cornell.edu/jeh/book.pdf.

  • Boé, J., 2018: Interdependency in multimodel climate projections: Component replication and result similarity. Geophys. Res. Lett., 45, 27712779, https://doi.org/10.1002/2017GL076829.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bretherton, C. S., M. Widmann V. P. Dymnikov, J. M. Wallace, and I. Bladé, 1999: The effective number of spatial degrees of freedom of a time-varying field. J. Climate, 12, 19902009, https://doi.org/10.1175/1520-0442(1999)012<1990:TENOSD>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chazottes, J.-R., 2015: Fluctuations of observables in dynamical systems: From limit theorems to concentration inequalities. Nonlinear Dynamics New Directions, Springer, 47–85.

    • Crossref
    • Export Citation
  • Cherkassky, V. S., and F. Mulier, 2007: Learning from Data: Concepts, Theory, and Methods. 2nd ed. John Wiley and Sons, 560 pp.

    • Crossref
    • Export Citation
  • Christiansen, B., 2015: The role of the selection problem and non-Gaussianity in attribution of single events to climate change. J. Climate, 28, 98739891, https://doi.org/10.1175/JCLI-D-15-0318.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christiansen, B., 2018a: Ensemble averaging and the curse of dimensionality. J. Climate, 31, 15871596, https://doi.org/10.1175/JCLI-D-17-0197.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christiansen, B., 2018b: Reply to “Comment on ‘Ensemble averaging and the curse of dimensionality.’” J. Climate, 31, 90179019, https://doi.org/10.1175/JCLI-D-18-0416.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Christiansen, B., 2019: Analysis of ensemble mean forecasts: The blessings of high dimensionality. Mon. Wea. Rev., 147, 16991712, https://doi.org/10.1175/MWR-D-18-0211.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clusel, M., and E. Bertin, 2008: Global fluctuations in physical systems: A subtle interplay between sum and extreme value statistics. Int. J. Mod. Phys., 22B, 33113368, https://doi.org/10.1142/S021797920804853X.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Donoho, D. L., 2000: High-dimensional data analysis: The curses and blessings of dimensionality. Conf. on Math Challenges of the 21st Century, Los Angeles, CA, American Mathematical Society.

  • Ehrendorfer, M., 1994: The Liouville equation and its potential usefulness for the prediction of forecast skill. Part I: Theory. Mon. Wea. Rev., 122, 703713, https://doi.org/10.1175/1520-0493(1994)122<0703:TLEAIP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Evans, J. P., F. Ji, G. Abramowitz, and M. Ekström, 2013: Optimally choosing small ensemble members to produce robust climate simulations. Environ. Res. Lett., 8, 044050, https://doi.org/10.1088/1748-9326/8/4/044050.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ferguson, T. S., 1996: A Course in Large Sample Theory. Chapman and Hall, 245 pp.

    • Crossref
    • Export Citation
  • Flato, G., and Coauthors, 2013: Evaluation of climate models. Climate Change 2013: The Physical Science Basis, T. F. Stocker et al., Eds., Cambridge University Press, 741–866.

  • Fraedrich, K., C. Ziehmann, and F. Sielmann, 1995: Estimates of spatial degrees of freedom. J. Climate, 8, 361369, https://doi.org/10.1175/1520-0442(1995)008<0361:EOSDOF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gelaro, R., and Coauthors, 2017: The Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2). J. Climate, 30, 54195454, https://doi.org/10.1175/JCLI-D-16-0758.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gleckler, P., K. Taylor, and C. Doutriaux, 2008: Performance metrics for climate models. J. Geophys. Res., 113, D06104, https://doi.org/10.1029/2007JD008972.

    • Search Google Scholar
    • Export Citation
  • Gorban, A. N., I. Y. Tyukin, and I. Romanenko, 2016: The blessing of dimensionality: Separation theorems in the thermodynamic limit. IFAC-PapersOnLine, 49, 6469, https://doi.org/10.1016/j.ifacol.2016.10.755.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gorban, A. N., V. A. Makarov, and I. Y. Tyukin, 2020: High-dimensional brain in a high-dimensional world: Blessing of dimensionality. Entropy, 22, 82, https://doi.org/10.3390/e22010082.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hall, P., J. S. Marron, and A. Neeman, 2005: Geometric representation of high dimension, low sample size data. J. Roy. Stat. Soc., 67B, 427444, https://doi.org/10.1111/j.1467-9868.2005.00510.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550560, https://doi.org/10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Haughton, N., G. Abramowitz, A. Pitman, and S. J. Phipps, 2014: On the generation of climate model ensembles. Climate Dyn., 43, 22972308, https://doi.org/10.1007/s00382-014-2054-3.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herger, N., G. Abramowitz, R. Knutti, O. Angélil, K. Lehmann, and B. M. Sanderson, 2018: Selecting a climate model subset to optimise key ensemble properties. Earth Syst. Dyn., 9, 135151, https://doi.org/10.5194/esd-9-135-2018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hersbach, H., and Coauthors, 2019: Global reanalysis: Goodbye ERA-Interim, hello ERA5. ECMWF Newsletter, No. 159, ECMWF, Reading, United Kingdom, 17–24, https://www.ecmwf.int/en/elibrary/19027-global-reanalysis-goodbye-era-interim-hello-era5.

  • Hourdin, F., and Coauthors, 2017: The art and science of climate model tuning. Bull. Amer. Meteor. Soc., 98, 589602, https://doi.org/10.1175/BAMS-D-15-00135.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jones, P. D., and B. R. Briffa, 1996: What can the instrumental record tell us about longer timescale paleoclimatic reconstructions? Climatic Variations and Forcing Mechanisms of the Last 2000 Years, Global Environmental Change, Vol. 41, Springer-Verlag, 625–644.

    • Crossref
    • Export Citation
  • Jun, M., R. Knutti, and D. W. Nychka, 2008: Spatial analysis to quantify numerical model bias and dependence. J. Amer. Stat. Assoc., 103, 934947, https://doi.org/10.1198/016214507000001265.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77, 437471, https://doi.org/10.1175/1520-0477(1996)077<0437:TNYRP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knutti, R., R. Furrer, C. Tebaldi, J. Cermak, and G. A. Meehl, 2010: Challenges in combining projections from multiple climate models. J. Climate, 23, 27392758, https://doi.org/10.1175/2009JCLI3361.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knutti, R., D. Masson, and A. Gettelman, 2013: Climate model genealogy: Generation CMIP5 and how we got there. Geophys. Res. Lett., 40, 11941199, https://doi.org/10.1002/grl.50256.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Knutti, R., J. Sedláček, B. M. Sanderson, R. Lorenz, E. Fischer, and V. Eyring, 2017: A climate model projection weighting scheme accounting for performance and interdependence. Geophys. Res. Lett., 44, 19091918, https://doi.org/10.1002/2016GL072012.

    • Search Google Scholar
    • Export Citation
  • Kontorovich, L., and K. Ramanan, 2008: Concentration inequalities for dependent random variables via the martingale method. Ann. Probab., 36, 21262158, https://doi.org/10.1214/07-AOP384.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kumar, S., J. L. Kinter, Z. Pan, and J. Sheffield, 2016: Twentieth century temperature trends in CMIP3, CMIP5, and CESM-LE climate simulations: Spatial-temporal uncertainties, differences, and their potential sources. J. Geophys. Res. Atmos., 121, 95619575, https://doi.org/10.1002/2015JD024382.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leduc, M., R. Laprise, R. de Elía, and L. Šeparović, 2016: Is institutional democracy a good proxy for model independence? J. Climate, 29, 83018316, https://doi.org/10.1175/JCLI-D-15-0761.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1969: Atmospheric predictability as revealed by naturally occurring analogues. J. Atmos. Sci., 26, 636646, https://doi.org/10.1175/1520-0469(1969)26<636:APARBN>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Maher, N., and Coauthors, 2019: The Max Planck Institute Grand Ensemble: Enabling the exploration of climate system variability. J. Adv. Model. Earth Syst., 11, 20502069, https://doi.org/10.1029/2019MS001639.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Masson, D., and R. Knutti, 2011: Climate model genealogy. Geophys. Res. Lett., 38, L08703, https://doi.org/10.1029/2011GL046864.

  • Mauritsen, T., and Coauthors, 2012: Tuning the climate of a global model. J. Adv. Model. Earth Syst., 4, M00A01, https://doi.org/10.1029/2012MS000154.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • North, G. R., J. Wang, and M. G. Genton, 2011: Correlation models for temperature fields. J. Climate, 24, 58505862, https://doi.org/10.1175/2011JCLI4199.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., 2000: Predicting uncertainty in forecasts of weather and climate. Rep. Prog. Phys., 63, 71116, https://doi.org/10.1088/0034-4885/63/2/201.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., R. Buizza, R. Hagedorn, A. Lorenze, M. Leutbecher, and S. Lenny, 2006: Ensemble prediction: A pedagogical perspective. ECMWF Newsletter, No. 106, ECMWF, Reading, United Kingdom, 10–17, https://www.ecmwf.int/sites/default/files/elibrary/2006/18024-ensemble-prediction-pedagogical-perspective.pdf.

  • Pennell, C., and T. Reichler, 2011: On the effective number of climate models. J. Climate, 24, 23582367, https://doi.org/10.1175/2010JCLI3814.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pirtle, Z., R. Meyer, and A. Hamilton, 2010: What does it mean when climate models agree? A case for assessing independence among general circulation models. Environ. Sci. Policy, 13, 351361, https://doi.org/10.1016/j.envsci.2010.04.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Potempski, S., and S. Galmarini, 2009: Est modus in rebus: Analytical properties of multi-model ensembles. Atmos. Chem. Phys., 9, 94719489, https://doi.org/10.5194/acp-9-9471-2009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanderson, B. M., and R. Knutti, 2012: On the interpretation of constrained climate model ensembles. Geophys. Res. Lett., 39, L16708, https://doi.org/10.1029/2012GL052665.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanderson, B. M., R. Knutti, and P. Caldwell, 2015: A representative democracy to reduce interdependency in a multimodel ensemble. J. Climate, 28, 51715194, https://doi.org/10.1175/JCLI-D-14-00362.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanderson, B. M., M. Wehner, and R. Knutti, 2017: Skill and independence weighting for multi-model assessments. Geosci. Model Dev., 10, 23792395, https://doi.org/10.5194/gmd-10-2379-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sansom, P. G., D. B. Stephenson, C. A. T. Ferro, G. Zappa, and L. Shaffrey, 2013: Simple uncertainty frameworks for selecting weighting schemes and interpreting multimodel ensemble climate change experiments. J. Climate, 26, 40174037, https://doi.org/10.1175/JCLI-D-12-00462.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sillmann, J., V. V. Kharin X. Zhang, F. W. Zwiers, and D. Bronaugh, 2013: Climate extremes indices in the CMIP5 multimodel ensemble: Part I. Model evaluation in the present climate. J. Geophys. Res. Atmos., 118, 17161733, https://doi.org/10.1002/JGRD.50203.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Steinschneider, S., R. McCrary, L. O. Mearns, and C. Brown, 2015: The effects of climate model similarity on probabilistic climate projections and the implications for local, risk-based adaptation planning. Geophys. Res. Lett., 42, 50145044, https://doi.org/10.1002/2015GL064529.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephenson, D. B., M. Collins, J. C. Rougier, and R. E. Chandler, 2012: Statistical problems in the probabilistic prediction of climate change. Environmetrics, 23, 364372, https://doi.org/10.1002/env.2153.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Swart, N. C., J. C. Fyfe, E. Hawkins, J. E. Kay, and A. Jahn, 2015: Influence of internal variability on Arctic sea-ice trends. Nat. Climate Change, 5, 8689, https://doi.org/10.1038/nclimate2483.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Talagrand, M., 1995: Concentration of measure and isoperimetric inequalities in product spaces. Inst. Hautes Études Sci. Publ. Math., 81, 73205, https://doi.org/10.1007/BF02699376.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Talagrand, O., R. Vautard, and B. Strauss, 1998: Evaluation of probabilistic prediction systems. Proc. Seminar on Predictability, Reading, United Kingdom, ECMWF.

  • Taylor, K. E., R. J. Stouffer, and G. A. Meehl, 2012: An overview of CMIP5 and the experiment design. Bull. Amer. Meteor. Soc., 93, 485498, https://doi.org/10.1175/BAMS-D-11-00094.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tebaldi, C., and R. Knutti, 2007: The use of the multi-model ensemble in probabilistic climate projections. Philos. Trans. Roy. Soc., 365A, 20532075, https://doi.org/10.1098/rsta.2007.2076.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., 1995: Degrees of freedom in Northern Hemisphere circulation data. Tellus, 47A, 457472, https://doi.org/10.3402/tellusa.v47i4.11531.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Loon, M., and Coauthors, 2007: Evaluation of long-term ozone simulations from seven regional air quality models and their ensemble. Atmos. Environ., 41, 20832097, https://doi.org/10.1016/j.atmosenv.2006.10.073.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, X., and S. S. Shen, 1999: Estimation of spatial degrees of freedom of a climate field. J. Climate, 12, 12801291, https://doi.org/10.1175/1520-0442(1999)012<1280:EOSDOF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yokohata, T., J. D. Annan, M. Collins, C. S. Jackson, M. Tobis, M. J. Webb, and J. C. Hargreaves, 2012: Reliability of multi-model and structurally different single-model ensembles. Climate Dyn., 39, 599616, https://doi.org/10.1007/s00382-011-1203-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 39 39 22
Full Text Views 11 11 4
PDF Downloads 18 18 5

Understanding the Distribution of Multimodel Ensembles

View More View Less
  • 1 Danish Meteorological Institute, Copenhagen, Denmark
© Get Permissions
Restricted access

Abstract

When analyzing multimodel climate ensembles it is often assumed that the ensemble is either truth centered or that models and observations are drawn from the same distribution. Here we analyze CMIP5 ensembles focusing on three measures that separate the two interpretations: the error of the ensemble mean relative to the error of individual models, the decay of the ensemble mean error for increasing ensemble size, and the correlations of the model errors. The measures are analyzed using a simple statistical model that includes the two interpretations as different limits and for which analytical results for the three measures can be obtained in high dimensions. We find that the simple statistical model describes the behavior of the three measures in the CMIP5 ensembles remarkably well. Except for the large-scale means we find that the indistinguishable interpretation is a better assumption than the truth centered interpretation. Furthermore, the indistinguishable interpretation becomes an increasingly better assumption when the errors are based on smaller temporal and spatial scales. Building on this, we present a simple conceptual mechanism for the indistinguishable interpretation based on the assumption that the climate models are calibrated on large-scale features such as, e.g., annual means or global averages.

Denotes content that is immediately available upon publication as open access.

Corresponding author: Bo Christiansen, boc@dmi.dk

Abstract

When analyzing multimodel climate ensembles it is often assumed that the ensemble is either truth centered or that models and observations are drawn from the same distribution. Here we analyze CMIP5 ensembles focusing on three measures that separate the two interpretations: the error of the ensemble mean relative to the error of individual models, the decay of the ensemble mean error for increasing ensemble size, and the correlations of the model errors. The measures are analyzed using a simple statistical model that includes the two interpretations as different limits and for which analytical results for the three measures can be obtained in high dimensions. We find that the simple statistical model describes the behavior of the three measures in the CMIP5 ensembles remarkably well. Except for the large-scale means we find that the indistinguishable interpretation is a better assumption than the truth centered interpretation. Furthermore, the indistinguishable interpretation becomes an increasingly better assumption when the errors are based on smaller temporal and spatial scales. Building on this, we present a simple conceptual mechanism for the indistinguishable interpretation based on the assumption that the climate models are calibrated on large-scale features such as, e.g., annual means or global averages.

Denotes content that is immediately available upon publication as open access.

Corresponding author: Bo Christiansen, boc@dmi.dk
Save