• Buizza, R., and T. N. Palmer, 1998: Impact of ensemble size on ensemble prediction. Mon. Wea. Rev., 126 , 25032518.

  • Cover, T. M., and J. A. Thomas, 1991: Elements of Information Theory. Wiley, 542 pp.

  • DelSole, T., 2004: Predictability and information theory. Part I: Measures of predictability. J. Atmos. Sci., 61 , 24252440.

  • DelSole, T., 2005: Predictability and information theory. Part II: Imperfect forecasts. J. Atmos. Sci., 62 , 33683381.

  • Derome, J., H. Lin, and G. Brunet, 2005: Seasonal forecasting with a simple general circulation model: Predictive skill in the AO and PNA. J. Climate, 18 , 597609.

    • Search Google Scholar
    • Export Citation
  • Epstein, E. S., 1969: Stochastic dynamic prediction. Tellus, 21 , 739759.

  • Farrell, B. F., and P. J. Ioannou, 1993: Stochastic forcing of perturbation variance in unbounded shear and deformation flows. J. Atmos. Sci., 50 , 200211.

    • Search Google Scholar
    • Export Citation
  • Feddersen, H., A. Navarra, and M. N. Ward, 1999: Reduction of model systematic error by statistical correction for dynamical seasonal predictions. J. Climate, 12 , 19741989.

    • Search Google Scholar
    • Export Citation
  • Gleeson, T. A., 1970: Statistical-dynamical predictions. J. Appl. Meteor., 9 , 333344.

  • Kharin, V. V., and F. W. Zwiers, 2001: Skill as a function of time scale in ensembles of seasonal hindcasts. Climate Dyn., 17 , 127141.

    • Search Google Scholar
    • Export Citation
  • Kleeman, R., 2002: Measuring dynamical prediction utility using relative entropy. J. Atmos. Sci., 59 , 20572072.

  • Kleeman, R., 2008: Limits, variability, and general behavior of statistical predictability of the midlatitude atmosphere. J. Atmos. Sci., in press.

    • Search Google Scholar
    • Export Citation
  • Kleeman, R., and A. M. Moore, 1997: A theory for the limitation of ENSO predictability due to stochastic atmospheric transients. J. Atmos. Sci., 54 , 753767.

    • Search Google Scholar
    • Export Citation
  • Kumar, A., A. B. Barnston, P. Peng, M. P. Hoerling, and L. Goddard, 2000: Changes in the spread of the variability of the seasonal mean atmospheric states associated with ENSO. J. Climate, 13 , 31393151.

    • Search Google Scholar
    • Export Citation
  • Madec, G., P. Delecluse, M. Imbard, and C. Levy, 1998: OPA 8.1 Ocean General Circulation Model reference manual. Institut Pierre Simon Laplace (IPSL), 91 pp.

  • Molteni, R., R. Buizza, and T. N. Palmer, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122 , 73119.

    • Search Google Scholar
    • Export Citation
  • Moore, A. M., and R. Kleeman, 1998: Skill assessment for ENSO using ensemble prediction. Quart. J. Roy. Meteor. Soc., 124 , 557584.

  • Moore, A. M., and Coauthors, 2006: Optimal forcing patterns for coupled models of ENSO. J. Climate, 19 , 46834699.

  • Murphy, J. M., 1988: The impact of ensemble forecasts on predictability. Quart. J. Roy. Meteor. Soc., 114 , 463493.

  • Scherrer, S. C., C. Appenzeller, P. Eckert, and D. Cattani, 2004: Analysis of the spread–skill relations using the ECMWF Ensemble prediction system over Europe. Wea. Forecasting, 19 , 552565.

    • Search Google Scholar
    • Export Citation
  • Schneider, T., and S. M. Griffies, 1999: A conceptual framework for predictability studies. J. Climate, 12 , 31333155.

  • Stern, W. F., and K. Miyakoda, 1995: The feasibility of seasonal forecast inferred from multiple GCM simulations. J. Climate, 8 , 10711085.

    • Search Google Scholar
    • Export Citation
  • Tang, Y., R. Kleeman, A. M. Moore, A. Weaver, and J. Vialard, 2003: The use of ocean reanalysis products to initialize ENSO predictions. Geophys. Res. Lett., 30 .1694, doi:10.1029/2003GL017664.

    • Search Google Scholar
    • Export Citation
  • Tang, Y., R. Kleeman, and R. Moore, 2005: On the reliability of ENSO dynamical predictions. J. Atmos. Sci., 62 , 17701791.

  • Tang, Y., H. Lin, and A. Moore, 2007a: Measuring the potential predictability of ensemble climate predictions. J. Geophys. Res., in press.

    • Search Google Scholar
    • Export Citation
  • Tang, Y., H. Lin, J. Derome, and M. K. Tippett, 2007b: A predictability measure applied to seasonal predictions of the Arctic Oscillation. J. Climate, 20 , 47334750.

    • Search Google Scholar
    • Export Citation
  • Tippett, M. K., R. Kleeman, and Y. Tang, 2004: Measuring the potential utility of seasonal climate predictions. Geophys. Res. Lett., 31 .L22201, doi:10.1029/2004GL021575.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74 , 23172330.

  • Whitaker, J. S., and A. F. Loughe, 1998: The relationship between ensemble spread and ensemble mean skill. Mon. Wea. Rev., 126 , 32923302.

    • Search Google Scholar
    • Export Citation
  • Xue, Y., M. A. Cane, S. E. Zebiak, and T. N. Palmer, 1997: Predictability of a coupled model of ENSO using singular vector analysis. Part II: Optimal growth and forecast skill. Mon. Wea. Rev., 125 , 20572073.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 137 57 3
PDF Downloads 94 45 4

Comparison of Information-Based Measures of Forecast Uncertainty in Ensemble ENSO Prediction

View More View Less
  • 1 Environmental Science and Engineering, University of Northern British Columbia, Prince George, British Columbia, Canada
  • | 2 Courant Institute of Mathematical Sciences, New York University, New York, New York
  • | 3 Ocean Sciences Department, University of California, Santa Cruz, Santa Cruz, California
Restricted access

Abstract

In this study, ensemble predictions of the El Niño–Southern Oscillation (ENSO) were conducted for the period 1981–98 using two hybrid coupled models. Several recently proposed information-based measures of predictability, including relative entropy (R), predictive information (PI), predictive power (PP), and mutual information (MI), were explored in terms of their ability of estimating a priori the predictive skill of the ENSO ensemble predictions. The emphasis was put on examining the relationship between the measures of predictability that do not use observations, and the model prediction skills of correlation and root-mean-square error (RMSE) that make use of observations. The relationship identified here offers a practical means of estimating the potential predictability and the confidence level of an individual prediction.

It was found that the MI is a good indicator of overall skill. When it is large, the prediction system has high prediction skill, whereas small MI often corresponds to a low prediction skill. This suggests that MI is a good indicator of the actual skill of the models. The R and PI have a nearly identical average (over all predictions) as should be the case in theory.

Comparing the different information-based measures reveals that R is a better predictor of prediction skill than PI and PP, especially when correlation-based metrics are used to evaluate model skill. A “triangular relationship” emerges between R and the model skill, namely, that when R is large, the prediction is likely to be reliable, whereas when R is small the prediction skill is quite variable. A small R is often accompanied by relatively weak ENSO variability. The possible reasons why R is superior to PI and PP as a measure of ENSO predictability will also be discussed.

Corresponding author address: Youmin Tang, Environmental Science and Engineering, University of Northern British Columbia, Prince George, BC V2N 4Z9, Canada. Email: ytang@unbc.ca

Abstract

In this study, ensemble predictions of the El Niño–Southern Oscillation (ENSO) were conducted for the period 1981–98 using two hybrid coupled models. Several recently proposed information-based measures of predictability, including relative entropy (R), predictive information (PI), predictive power (PP), and mutual information (MI), were explored in terms of their ability of estimating a priori the predictive skill of the ENSO ensemble predictions. The emphasis was put on examining the relationship between the measures of predictability that do not use observations, and the model prediction skills of correlation and root-mean-square error (RMSE) that make use of observations. The relationship identified here offers a practical means of estimating the potential predictability and the confidence level of an individual prediction.

It was found that the MI is a good indicator of overall skill. When it is large, the prediction system has high prediction skill, whereas small MI often corresponds to a low prediction skill. This suggests that MI is a good indicator of the actual skill of the models. The R and PI have a nearly identical average (over all predictions) as should be the case in theory.

Comparing the different information-based measures reveals that R is a better predictor of prediction skill than PI and PP, especially when correlation-based metrics are used to evaluate model skill. A “triangular relationship” emerges between R and the model skill, namely, that when R is large, the prediction is likely to be reliable, whereas when R is small the prediction skill is quite variable. A small R is often accompanied by relatively weak ENSO variability. The possible reasons why R is superior to PI and PP as a measure of ENSO predictability will also be discussed.

Corresponding author address: Youmin Tang, Environmental Science and Engineering, University of Northern British Columbia, Prince George, BC V2N 4Z9, Canada. Email: ytang@unbc.ca

Save