• Abramowitz, G., , and H. Gupta, 2008: Towards a model space and independence metric. Geophys. Res. Lett., 35, L05705, doi:10.1029/2007GL032834.

    • Search Google Scholar
    • Export Citation
  • Annan, J. D., , and J. C. Hargreaves, 2010: Reliability of the CMIP3 ensemble. Geophys. Res. Lett., 37, L02703, doi:10.1029/2009GL041994.

  • Bretherton, C. S., , M. Widmann, , V. P. Dymnikov, , J. M. Wallace, , and I. Bladé, 1999: The effective number of spatial degrees of freedom of a time-varying field. J. Climate, 12, 19902009.

    • Search Google Scholar
    • Export Citation
  • Delworth, T. L., and Coauthors, 2006: GFDL’s CM2 global coupled climate models. Part I: Formulation and simulation characteristics. J. Climate, 19, 643674.

    • Search Google Scholar
    • Export Citation
  • Furrer, R., , R. Knutti, , S. R. Sain, , D. W. Nychka, , and G. A. Meehl, 2007: Spatial patterns of probabilistic temperature change projections from a multivariate Bayesian analysis. Geophys. Res. Lett., 34, L06711, doi:10.1029/2006GL027754.

    • Search Google Scholar
    • Export Citation
  • Gleckler, P. J., , K. E. Taylor, , and C. Doutriaux, 2008: Performance metrics for climate models. J. Geophys. Res., 113, D06104, doi:10.1029/2007JD008972.

    • Search Google Scholar
    • Export Citation
  • Gnanadesikan, A., and Coauthors, 2006: GFDL’s CM2 Global Coupled Climate Models. Part II: The baseline ocean simulation. J. Climate, 19, 675697.

    • Search Google Scholar
    • Export Citation
  • Hasumi, H., , and S. Emori, 2004: K-1 coupled GCM (MIROC) description. K-1 Tech. Rep. 1, Center for Climate System Research, University of Tokyo, 34 pp. [Available online at http://www.ccsr.u-tokyo.ac.jp/kyosei/hasumi/MIROC/tech-repo.pdf.]

    • Search Google Scholar
    • Export Citation
  • Jun, M., , R. Knutti, , and D. W. Nychka, 2008a: Local eigenvalue analysis of CMIP3 climate model errors. Tellus, 60A, 9921000.

  • Jun, M., , R. Knutti, , and D. W. Nychka, 2008b: Spatial analysis to quantify numerical model bias and dependence: How many climate models are there? J. Amer. Stat. Assoc., 103, 934947.

    • Search Google Scholar
    • Export Citation
  • Knutti, R., 2008: Should we believe model predictions of future climate change? Philos. Trans. Roy. Soc., 366, 46474664.

  • Knutti, R., , R. Furrer, , C. Tebaldi, , J. Cermak, , and G. A. Meehl, 2010: Challenges in combining projections from multiple climate models. J. Climate, 23, 27392758.

    • Search Google Scholar
    • Export Citation
  • Meehl, G. A., and Coauthors, 2007a: Global climate projections. Climate Change 2007: The Physical Science Basis, S. Solomon et al., Eds, Cambridge Press University, 747–846.

    • Search Google Scholar
    • Export Citation
  • Meehl, G. A., , C. Covey, , K. E. Taylor, , T. Delworth, , R. J. Stouffer, , M. Latif, , B. McAvaney, , and J. F. B. Mitchell, 2007b: The WCRP CMIP3 multimodel dataset: A new era in climate change research. Bull. Amer. Meteor. Soc., 88, 13831394.

    • Search Google Scholar
    • Export Citation
  • Pierce, D. W., , T. P. Barnett, , B. D. Santer, , and P. J. Gleckler, 2009: Selecting global climate models for regional climate change studies. Proc. Natl. Acad. Sci. USA, 106, 84418446.

    • Search Google Scholar
    • Export Citation
  • Pincus, R., , C. P. Batstone, , R. J. P. Hofmann, , K. E. Taylor, , and P. J. Gleckler, 2008: Evaluating the present-day simulation of clouds, precipitation, and radiation in climate models. J. Geophys. Res., 113, D14209, doi:10.1029/2007JD009334.

    • Search Google Scholar
    • Export Citation
  • Reichler, T., , and J. Kim, 2008a: How well do coupled models simulate today’s climate? Bull. Amer. Meteor. Soc., 89, 303311.

  • Reichler, T., , and J. Kim, 2008b: Uncertainties in the climate mean state of global observations, reanalyses, and the GFDL climate model. J. Geophys. Res., 113, D05106, doi:10.1029/2007JD009278.

    • Search Google Scholar
    • Export Citation
  • Reifen, C., , and R. Toumi, 2009: Climate projections: Past performance no guarantee of future skill? Geophys. Res. Lett., 36, L13704, doi:10.1029/2009GL038082.

    • Search Google Scholar
    • Export Citation
  • Schmidt, G. A., and Coauthors, 2005: Present-day atmospheric simulations using GISS ModelE: Comparison to in situ, satellite, and reanalysis data. J. Climate, 19, 153192.

    • Search Google Scholar
    • Export Citation
  • Tebaldi, C., , and R. Knutti, 2007: The use of the multi-model ensemble in probabilistic climate projections. Philos. Trans. Roy. Soc., 365A, 20532075.

    • Search Google Scholar
    • Export Citation
  • Tebaldi, C., , R. L. Smith, , D. Nychka, , and L. O. Mearns, 2005: Quantifying uncertainty in projections of regional climate change: A Bayesian approach to the analysis of multimodel ensembles. J. Climate, 18, 15241540.

    • Search Google Scholar
    • Export Citation
  • Van den Dool, H., 2007: Empirical Methods in Short-Term Climate Prediction. Oxford University Press, 215 pp.

  • Van den Dool, H., , and R. M. Chervin, 1986: A comparison of month-to-month persistence of anomalies in a general circulation model and in the earth’s atmosphere. J. Atmos. Sci., 43, 14541466.

    • Search Google Scholar
    • Export Citation
  • Wang, X., , and S. Shen, 1999: Estimation of spatial degrees of freedom of a climate field. J. Climate, 12, 12801291.

  • Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Elsevier, 627 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 137 137 25
PDF Downloads 105 105 21

On the Effective Number of Climate Models

View More View Less
  • 1 Department of Atmospheric Sciences, University of Utah, Salt Lake City, Utah
© Get Permissions
Restricted access

Abstract

Projections of future climate change are increasingly based on the output of many different models. Typically, the mean over all model simulations is considered as the optimal prediction, with the underlying assumption that different models provide statistically independent information evenly distributed around the true state. However, there is reason to believe that this is not the best assumption. Coupled models are of comparable complexity and are constructed in similar ways. Some models share parts of the same code and some models are even developed at the same center. Therefore, the limitations of these models tend to be fairly similar, contributing to the well-known problem of common model biases and possibly to an unrealistically small spread in the outcomes of model predictions.

This study attempts to quantify the extent of this problem by asking how many models there effectively are and how to best determine this number. Quantifying the effective number of models is achieved by evaluating 24 state-of-the-art models and their ability to simulate broad aspects of twentieth-century climate. Using two different approaches, the amount of unique information in the ensemble is calculated and the effective ensemble size is found to be much smaller than the actual number of models. As more models are included in an ensemble, the amount of new information diminishes in proportion. Furthermore, this reduction is found to go beyond the problem of “same center” models and systemic similarities are seen to exist across all models. The results suggest that current methodologies for the interpretation of multimodel ensembles may lead to overly confident climate predictions.

Corresponding author address: Thomas Reichler, Dept. of Atmospheric Sciences, University of Utah, 135 S 1460 E, Rm. 819 (WBB), Salt Lake City, UT 84112-0110. E-mail: thomas.reichler@utah.edu

Abstract

Projections of future climate change are increasingly based on the output of many different models. Typically, the mean over all model simulations is considered as the optimal prediction, with the underlying assumption that different models provide statistically independent information evenly distributed around the true state. However, there is reason to believe that this is not the best assumption. Coupled models are of comparable complexity and are constructed in similar ways. Some models share parts of the same code and some models are even developed at the same center. Therefore, the limitations of these models tend to be fairly similar, contributing to the well-known problem of common model biases and possibly to an unrealistically small spread in the outcomes of model predictions.

This study attempts to quantify the extent of this problem by asking how many models there effectively are and how to best determine this number. Quantifying the effective number of models is achieved by evaluating 24 state-of-the-art models and their ability to simulate broad aspects of twentieth-century climate. Using two different approaches, the amount of unique information in the ensemble is calculated and the effective ensemble size is found to be much smaller than the actual number of models. As more models are included in an ensemble, the amount of new information diminishes in proportion. Furthermore, this reduction is found to go beyond the problem of “same center” models and systemic similarities are seen to exist across all models. The results suggest that current methodologies for the interpretation of multimodel ensembles may lead to overly confident climate predictions.

Corresponding author address: Thomas Reichler, Dept. of Atmospheric Sciences, University of Utah, 135 S 1460 E, Rm. 819 (WBB), Salt Lake City, UT 84112-0110. E-mail: thomas.reichler@utah.edu
Save