Editorial Type:
Article
Article Type:
Research Article

Ensemble Regression

David A. Unger NOAA/NWS/NCEP/Climate Prediction Center, Camp Springs, Maryland

Search for other papers by David A. Unger in
Current site
Google Scholar
PubMed Close
,
Huug van den Dool NOAA/NWS/NCEP/Climate Prediction Center, Camp Springs, Maryland

Search for other papers by Huug van den Dool in
Current site
Google Scholar
PubMed Close
,
Edward O’Lenic NOAA/NWS/NCEP/Climate Prediction Center, Camp Springs, Maryland

Search for other papers by Edward O’Lenic in
Current site
Google Scholar
PubMed Close
, and
Dan Collins NOAA/NWS/NCEP/Climate Prediction Center, Camp Springs, Maryland

Search for other papers by Dan Collins in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Jul 2009
DOI:
https://doi.org/10.1175/2008MWR2605.1
Page(s):
2365–2379
Restricted access

Abstract

A regression model was developed for use with ensemble forecasts. Ensemble members are assumed to represent a set of equally likely solutions, one of which will best fit the observation. If standard linear regression assumptions apply to the best member, then a regression relationship can be derived between the full ensemble and the observation without explicitly identifying the best member for each case. The ensemble regression equation is equivalent to linear regression between the ensemble mean and the observation, but is applied to each member of the ensemble. The “best member” error variance is defined in terms of the correlation between the ensemble mean and the observations, their respective variances, and the ensemble spread. A probability density function representing the ensemble prediction is obtained from the normalized sum of the best-member error distribution applied to the regression forecast from each ensemble member. Ensemble regression was applied to National Centers for Environmental Prediction (NCEP) Climate Forecast System (CFS) forecasts of seasonal mean Niño-3.4 SSTs on historical forecasts for the years 1981–2005. The skill of the ensemble regression was about the same as that of the linear regression on the ensemble mean when measured by the continuous ranked probability score (CRPS), and both methods produced reliable probabilities. The CFS spread appears slightly too high for its skill, and the CRPS of the CFS predictions can be slightly improved by reducing its ensemble spread to about 0.8 of its original value prior to regression calibration.

Corresponding author address: David A. Unger, NOAA/NWS/NCEP/Climate Prediction Center, 5200 Auth Rd., Camp Springs, MD 20746. Email: david.unger@noaa.gov

Abstract

A regression model was developed for use with ensemble forecasts. Ensemble members are assumed to represent a set of equally likely solutions, one of which will best fit the observation. If standard linear regression assumptions apply to the best member, then a regression relationship can be derived between the full ensemble and the observation without explicitly identifying the best member for each case. The ensemble regression equation is equivalent to linear regression between the ensemble mean and the observation, but is applied to each member of the ensemble. The “best member” error variance is defined in terms of the correlation between the ensemble mean and the observations, their respective variances, and the ensemble spread. A probability density function representing the ensemble prediction is obtained from the normalized sum of the best-member error distribution applied to the regression forecast from each ensemble member. Ensemble regression was applied to National Centers for Environmental Prediction (NCEP) Climate Forecast System (CFS) forecasts of seasonal mean Niño-3.4 SSTs on historical forecasts for the years 1981–2005. The skill of the ensemble regression was about the same as that of the linear regression on the ensemble mean when measured by the continuous ranked probability score (CRPS), and both methods produced reliable probabilities. The CFS spread appears slightly too high for its skill, and the CRPS of the CFS predictions can be slightly improved by reducing its ensemble spread to about 0.8 of its original value prior to regression calibration.

Corresponding author address: David A. Unger, NOAA/NWS/NCEP/Climate Prediction Center, 5200 Auth Rd., Camp Springs, MD 20746. Email: david.unger@noaa.gov

Save
Cover Monthly Weather Review
Monthly Weather Review

  • Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9 , 1518–1530.

    • Search Google Scholar
    • Export Citation

  • Barnett, T. P., 1995: Monte Carlo climate forecasting. J. Climate, 8 , 1005–1022.

  • Barnston, A. G., and Coauthors, 1994: Long lead seasonal forecasts—Where do we stand? Bull. Amer. Meteor. Soc., 75 , 2097–2114.

  • Barnston, A. G., M. Chelliah, and S. B. Goldberg, 1997: Documentation of a highly ENSO-related SST region in the equatorial Pacific. Atmos.–Ocean, 35 , 367–383.

    • Search Google Scholar
    • Export Citation

  • Barnston, A. G., Y. He, and D. A. Unger, 2000: A forecast product that maximizes utility for state-of-the-art seasonal climate prediction. Bull. Amer. Meteor. Soc., 81 , 1271–1279.

    • Search Google Scholar
    • Export Citation

  • Benzi, R., and A. Speranza, 1989: Statistical properties of low frequency variability in the Northern Hemisphere. J. Climate, 2 , 367–379.

    • Search Google Scholar
    • Export Citation

  • Draper, N., and H. Smith, 1981: Applied Regression Analysis. John Wiley and Sons, 709 pp.

  • Eckel, F. A., and M. K. Walters, 1998: Calibrated probabilistic quantitative precipitation forecasts based on the MRF ensemble. Wea. Forecasting, 13 , 1132–1147.

    • Search Google Scholar
    • Export Citation

  • Epstein, E., 1969a: Stochastic dynamic prediction. Tellus, 21 , 739–759.

  • Epstein, E., 1969b: A scoring system for probability forecasts of ranked categories. J. Appl. Meteor., 8 , 985–987.

  • Fortin, V., A. C. Favre, and M. Said, 2006: Probabilistic forecasting from ensemble prediction systems: Improving on the best member method by using a different weight and dressing kernel for each member. Quart. J. Roy. Meteor. Soc., 132 , 1349–1369.

    • Search Google Scholar
    • Export Citation

  • Glahn, B., and Coauthors, 2009: MOS uncertainty estimates in an ensemble framework. Mon. Wea. Rev., 137 , 246–268.

  • Glahn, H. R., and D. Lowry, 1972: The use of model output statistics (MOS) in objective weather forecasts. J. Appl. Meteor., 11 , 1203–1211.

    • Search Google Scholar
    • Export Citation

  • Gneiting, T., A. Raftery, A. H. Westveld III, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133 , 1098–1118.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., 2000: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129 , 550–560.

  • Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev., 125 , 1312–1327.

  • Hamill, T. M., and S. J. Colucci, 1998: Evaluation of Eta–RSM ensemble probabilistic precipitation forecasts. Mon. Wea. Rev., 126 , 711–724.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., and J. S. Whitaker, 2006: Quantitative precipitation forecasts based on reforecast analogs: Theory and applications. Mon. Wea. Rev., 134 , 3209–3229.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., J. S. Whitaker, and X. Wei, 2004: Ensemble reforecasting: Improving medium-range forecasting using retrospective forecasts. Mon. Wea. Rev., 132 , 1434–1447.

    • Search Google Scholar
    • Export Citation

  • Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15 , 559–570.

    • Search Google Scholar
    • Export Citation

  • Hoffman, R. N., and E. Kalnay, 1983: Lagged average forecasting, an alternative to Monte-Carlo forecasting. Tellus, 35A , 100–118.

  • Klein, W. H., F. Lewis, and I. Enger, 1959: Objective prediction of 5-day mean temperature during winter. J. Meteor., 16 , 672–682.

  • Kumar, A., and M. P. Hoerling, 1995: Prospects and limitations of seasonal atmospheric GCM predictions. Bull. Amer. Meteor. Soc., 76 , 335–345.

    • Search Google Scholar
    • Export Citation

  • Leith, C. E., and Coauthors, 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102 , 409–418.

  • Livezey, R. E., 1990: Variability of skill of long range forecasts and implications for their use and value. Bull. Amer. Meteor. Soc., 71 , 300–309.

    • Search Google Scholar
    • Export Citation

  • Livezey, R. E., and M. M. Timofeyeva, 2008: The first decade of long-lead U.S. seasonal forecasts: Insights from a skill analysis. Bull. Amer. Meteor. Soc., 89 , 843–855.

    • Search Google Scholar
    • Export Citation

  • Matheson, J. E., and R. L. Winkler, 1976: Scoring rules for continuous probability distributions. Manage. Sci., 22 , 1087–1096.

  • Michaelsen, J., 1987: Cross-validation in statistical climate forecast models. J. Climate Appl. Meteor., 26 , 1589–1600.

  • Murphy, A. H., 1970: The ranked probability score and the probability score: A comparison. Mon. Wea. Rev., 98 , 917–924.

  • O’Lenic, E. A., 2008: Developments in operational long-range climate prediction at CPC. Wea. Forecasting, 23 , 496–515.

  • Phelps, M. W., A. Kumar, and J. J. O’Brien, 2004: Potential predictability in the NCEP CPC seasonal forecast system. J. Climate, 17 , 3775–3785.

    • Search Google Scholar
    • Export Citation

  • Raftery, A. E., and Coauthors, 2005: Using Bayesian model averaging to calibrate forecast model ensembles. Mon. Wea. Rev., 133 , 1155–1174.

    • Search Google Scholar
    • Export Citation

  • Roads, J. O., 1988: Lagged averaged predictions in a predictability experiment. J. Atmos. Sci., 45 , 147–162.

  • Roulston, M. S., and L. A. Smith, 2003: Combining dynamic and statistical ensembles. Tellus, 55A , 16–30.

  • Rowell, D. P., 1998: Assessing seasonal predictability with an ensemble of multidecadal GCM simulations. J. Climate, 11 , 109–120.

  • Saha, S., and Coauthors, 2006: The NCEP Climate Forecast System. J. Climate, 19 , 3483–3517.

  • Silverman, B. W., 1986: Density Estimation for Statistics and Data Analysis. Chapman and Hall, 175 pp.

  • Sivillo, J. K., J. E. Ahlquist, and Z. Toth, 1997: An ensemble forecasting primer. Wea. Forecasting, 12 , 809–818.

  • Stern, W., and K. Miyakoda, 1995: Feasibility of seasonal forecasts inferred from multiple GCM simulations. J. Climate, 8 , 1071–1085.

  • Talagrand, O., R. Vauland, and B. Strauss, 1997: Evaluation of probabilistic predictions systems. Proc. Workshop on Predictions, Reading, United Kingdom, ECMWF, 1–25. [Available from ECMWF, Shinfield Park, Reading, Berkshire RG29AX, United Kingdom.].

    • Search Google Scholar
    • Export Citation

  • Thiébaux, H. J., and F. W. Zwiers, 1984: The interpretation and estimation of effective sample size. J. Climate Appl. Meteor., 23 , 800–811.

    • Search Google Scholar
    • Export Citation

  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74 , 2317–2330.

  • van den Dool, H. M., 1994: Searching for analogues, how long must we wait? Tellus, 46A , 314–324.

  • Vialard, J., and Coauthors, 2005: An ensemble generation technique for seasonal forecasting with an ocean–atmosphere coupled model. Mon. Wea. Rev., 133 , 441–453.

    • Search Google Scholar
    • Export Citation

  • Wang, X., and C. H. Bishop, 2005: Improvements of ensemble reliability using a new dressing kernel. Quart. J. Roy. Meteor. Soc., 131 , 965–986.

    • Search Google Scholar
    • Export Citation

  • Wilks, D. S., 2006: Comparison of ensemble MOS methods in the Lorenz 96 setting. Meteor. Appl., 13 , 243–256.

  • Wilks, D. S., and T. M. Hamill, 2007: Comparison of MOS-ensemble methods using GFS reforecasts. Mon. Wea. Rev., 135 , 2379–2390.

  • Wilson, L. J., and Coauthors, 2007: Calibrated surface temperature forecasts from the Canadian Ensemble Prediction System using Bayesian model averaging. Mon. Wea. Rev., 135 , 1365–1385.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 4746 3799 187
PDF Downloads 862 154 30