Improved Combination of Multiple Atmospheric GCM Ensembles for Seasonal Prediction

Andrew W. Robertson International Research Institute for Climate Prediction, The Earth Institute at Columbia University, Palisades, New York

Search for other papers by Andrew W. Robertson in
Current site
Google Scholar
PubMed
Close
,
Upmanu Lall International Research Institute for Climate Prediction, The Earth Institute at Columbia University, Palisades, New York

Search for other papers by Upmanu Lall in
Current site
Google Scholar
PubMed
Close
,
Stephen E. Zebiak International Research Institute for Climate Prediction, The Earth Institute at Columbia University, Palisades, New York

Search for other papers by Stephen E. Zebiak in
Current site
Google Scholar
PubMed
Close
, and
Lisa Goddard International Research Institute for Climate Prediction, The Earth Institute at Columbia University, Palisades, New York

Search for other papers by Lisa Goddard in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

An improved Bayesian optimal weighting scheme is developed and used to combine six atmospheric general circulation model (GCM) seasonal hindcast ensembles. The approach is based on the prior belief that the forecast probabilities of tercile-category precipitation and near-surface temperature are equal to the climatological ones. The six GCMs are integrated over the 1950–97 period with observed monthly SST prescribed at the lower boundary, with 9–24 ensemble members. The weights of the individual models are determined by maximizing the log likelihood of the combination by season over the integration period. A key ingredient of the scheme is the climatological equal-odds forecast, which is included as one of the “models” in the multimodel combination. Simulation skill is quantified in terms of the cross-validated ranked probability skill score (RPSS) for the three-category probabilistic hindcasts. The individual GCM ensembles, simple poolings of three and six models, and the optimally combined multimodel ensemble are compared.

The Bayesian optimal weighting scheme outperforms the pooled ensemble, which in turn outperforms the individual models. In the extratropics, its main benefit is to bring much of the large area of negative-precipitation RPSS values up to near-zero values. The skill of the optimal combination is almost always increased (in the large spatial averages considered) when the number of models in the combination is increased from three to six, regardless of which models are included in the three-model combination.

Improvements are made to the original Bayesian scheme of Rajagopalan et al. by reducing the dimensionality of the numerical optimization, averaging across data subsamples, and including spatial smoothing of the likelihood function. These modifications are shown to yield increases in cross-validated RPSS skills. The revised scheme appears to be better suited to combining larger sets of models, and, in the future, it should be possible to include statistical models into the weighted ensemble without fundamental difficulty.

Corresponding author address: Andrew W. Robertson, IRI, Monell 230, 61 Route 9W, Palisades, NY 10964. Email: awr@iri.columbia.edu

Abstract

An improved Bayesian optimal weighting scheme is developed and used to combine six atmospheric general circulation model (GCM) seasonal hindcast ensembles. The approach is based on the prior belief that the forecast probabilities of tercile-category precipitation and near-surface temperature are equal to the climatological ones. The six GCMs are integrated over the 1950–97 period with observed monthly SST prescribed at the lower boundary, with 9–24 ensemble members. The weights of the individual models are determined by maximizing the log likelihood of the combination by season over the integration period. A key ingredient of the scheme is the climatological equal-odds forecast, which is included as one of the “models” in the multimodel combination. Simulation skill is quantified in terms of the cross-validated ranked probability skill score (RPSS) for the three-category probabilistic hindcasts. The individual GCM ensembles, simple poolings of three and six models, and the optimally combined multimodel ensemble are compared.

The Bayesian optimal weighting scheme outperforms the pooled ensemble, which in turn outperforms the individual models. In the extratropics, its main benefit is to bring much of the large area of negative-precipitation RPSS values up to near-zero values. The skill of the optimal combination is almost always increased (in the large spatial averages considered) when the number of models in the combination is increased from three to six, regardless of which models are included in the three-model combination.

Improvements are made to the original Bayesian scheme of Rajagopalan et al. by reducing the dimensionality of the numerical optimization, averaging across data subsamples, and including spatial smoothing of the likelihood function. These modifications are shown to yield increases in cross-validated RPSS skills. The revised scheme appears to be better suited to combining larger sets of models, and, in the future, it should be possible to include statistical models into the weighted ensemble without fundamental difficulty.

Corresponding author address: Andrew W. Robertson, IRI, Monell 230, 61 Route 9W, Palisades, NY 10964. Email: awr@iri.columbia.edu

Save
  • Barnston, A. G., and T. M. Smith, 1996: Specification and prediction of global surface temperature and precipitation from global SST using CCA. J. Climate, 9 , 26602697.

    • Search Google Scholar
    • Export Citation
  • Barnston, A. G., S. J. Mason, L. Goddard, D. G. DeWitt, and S. E. Zebiak, 2003: Multi-model ensembling in seasonal climate forecasting at IRI. Bull. Amer. Meteor. Soc, 84 , 17831796.

    • Search Google Scholar
    • Export Citation
  • Doblas-Reyes, F. J., M. Deque, and J-P. Piedelievre, 2000: Multi-model spread and probabilistic seasonal forecasts in PROVOST. Quart. J. Roy. Meteor. Soc, 126 , 20692088.

    • Search Google Scholar
    • Export Citation
  • Epstein, E. S., 1969: A scoring system for probability forecasts of ranked categories. J. Appl. Meteor, 8 , 985987.

  • Fraedrich, K., and N. R. Smith, 1989: Combining predictive schemes in long range forecasting. J. Climate, 2 , 291294.

  • Gelman, A., J. B. Carlin, H. S. Stern, and D. B. Rubin, 1995: Bayesian Data Analysis. Chapman and Hall, 526 pp.

  • Goddard, L., 2005: El Niño: Catastrophe or opportunity? J. Climate, in press.

  • Goddard, L., and S. J. Mason, 2002: Sensitivity of seasonal climate forecasts to persisted SST anomalies. Climate Dyn, 19 , 619631.

  • Goddard, L., A. G. Barnston, and S. J. Mason, 2003: Evaluation of the IRI's “net assessment” seasonal climate forecasts: 1997–2001. Bull. Amer. Meteor. Soc, 84 , 17611781.

    • Search Google Scholar
    • Export Citation
  • Gong, X., A. G. Barnston, and M. N. Ward, 2003: The effect of spatial aggregation on the skill of seasonal precipitation forecasts. J. Climate, 16 , 30593071.

    • Search Google Scholar
    • Export Citation
  • Graham, R. J., A. D. L. Evans, K. R. Mylne, M. S. J. Harrison, and K. B. Robertson, 2000: An assessment of seasonal predictability using atmospheric general circulation models. Quart. J. Roy. Meteor. Soc, 126 , 22112240.

    • Search Google Scholar
    • Export Citation
  • Hack, J. J., J. T. Kiehl, and J. W. Hurrell, 1998: The hydrological and thermodynamic characteristics of the NCAR CCM3. J. Climate, 11 , 11791206.

    • Search Google Scholar
    • Export Citation
  • Hagedorn, R., 2001: Development of a multi-model ensemble system for seasonal to interannual prediction. Proc. XXVI General Assembly of the EGS, Nice, France, European Geophysical Society.

    • Search Google Scholar
    • Export Citation
  • Kanamitsu, M., and K. C. Mo, 2003: Dynamical effect of land surface processes on summer precipitation over the southwestern United States. J. Climate, 16 , 496509.

    • Search Google Scholar
    • Export Citation
  • Kanamitsu, M., and Coauthors, 2002: NCEP dynamical seasonal forecast system 2000. Bull. Amer. Meteor. Soc, 83 , 10191037.

  • Kumar, A., M. P. Hoerling, M. Ji, A. Leetmaa, and P. Sardeshmukh, 1996: Assessing a GCM's suitability for making seasonal predictions. J. Climate, 9 , 115129.

    • Search Google Scholar
    • Export Citation
  • Kumar, A., A. G. Barnston, and M. P. Hoerling, 2001: Seasonal predictions, probabilistic verifications, and ensemble size. J. Climate, 14 , 16711676.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci, 20 , 130148.

  • Mason, S., 2004: On using “climatology” as a reference strategy in the Brier and ranked probability skill scores. Mon. Wea. Rev, 132 , 18911895.

    • Search Google Scholar
    • Export Citation
  • New, M., M. Hulme, and P. D. Jones, 1999: Representing twentieth-century space–time climate variability. Part I: Development of a 1961–90 mean monthly terrestrial climatology. J. Climate, 12 , 829856.

    • Search Google Scholar
    • Export Citation
  • New, M., M. Hulme, and P. D. Jones, 2000: Representing twentieth-century space– time climate variability. Part II: Development of a 1961–90 monthly grid of terrestrial surface climate. J. Climate, 13 , 22172238.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., C. Brankovic, and D. S. Richardson, 2000: A probability and decision-model analysis of PROVOST seasonal multi-model ensemble integrations. Quart. J. Roy. Meteor. Soc, 126 , 20132034.

    • Search Google Scholar
    • Export Citation
  • Pavan, V., and F. J. Doblas-Reyes, 2000: Multi-model seasonal hindcasts over the Euro-Atlantic: Skill scores and dynamic features. Climate Dyn, 16 , 611625.

    • Search Google Scholar
    • Export Citation
  • Peng, P., A. Kumar, H. Van den Dool, and A. G. Barnston, 2002: An analysis of multimodel ensemble predictions for seasonal climate anomalies. J. Geophys. Res.,107, 4710, doi:10.1029/2002JD002712.

    • Search Google Scholar
    • Export Citation
  • Rajagopalan, B., U. Lall, and S. E. Zebiak, 2002: Categorical climate forecasts through regularization and optimal combination of multiple GCM ensembles. Mon. Wea. Rev, 130 , 17921811.

    • Search Google Scholar
    • Export Citation
  • Reynolds, R. W., 1988: A real-time global sea surface temperature analysis. J. Climate, 1 , 7587.

  • Reynolds, R. W., and T. M. Smith, 1994: Improved global sea surface temperature analyses using optimum interpolation. J. Climate, 7 , 929948.

    • Search Google Scholar
    • Export Citation
  • Roeckner, E., and Coauthors, 1996: The atmospheric general circulation model ECHAM4: Model description and simulation of present-day climate. Max-Planck-Institüt für Meteorologie Rep. 218, 90 pp.

    • Search Google Scholar
    • Export Citation
  • Ropelewski, C. F., and M. S. Halpert, 1987: Global and regional scale precipitation patterns associated with the El Niño/Southern Oscillation. Mon. Wea. Rev, 115 , 16061626.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. International Geophysical Series, Vol. 59, Academic Press, 464 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 672 125 9
PDF Downloads 378 70 10