Postprocessing of Ensemble Precipitation Predictions with Extended Logistic Regression Based on Hindcasts

Emmanuel Roulin Institut Royal Météorologique de Belgique, Brussels, Belgium

Search for other papers by Emmanuel Roulin in
Current site
Google Scholar
PubMed
Close
and
Stéphane Vannitsem Institut Royal Météorologique de Belgique, Brussels, Belgium

Search for other papers by Stéphane Vannitsem in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Extended logistic regression is used to calibrate areal precipitation forecasts over two small catchments in Belgium computed with the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System (EPS) between 2006 and 2010. The parameters of the postprocessing are estimated from the hindcast database, characterized by a much lower number of members (5) than the EPS (51). Therefore, the parameters have to be corrected for predictor uncertainties. They have been fitted on the 51-member EPS ensembles, on 5-member subensembles drawn from the same EPS, and on the 5-member hindcasts. For small ensembles, a simple “regression calibration” method by which the uncertain predictors are corrected has been applied. The different parameter sets have been compared, and the corresponding extended logistic regressions have been applied to the 51-member EPS. The forecast probabilities have then been validated using rain gauge data and compared with the raw EPS. In addition, the calibrated distributions are also used to modify the ensembles of precipitation traces.

The postprocessing with the extended logistic regression is shown to improve the continuous ranked probability skill score relative to the raw ensemble, and the regression calibration to remove a large portion of the bias in parameter estimation with small ensembles. With a training phase limited to a 5-week moving window, the benefit lasts for the first 2 forecast days in winter and the first 5 or 6 days in summer. In general, substantial improvements of the mean error and of the continuous ranked probability score have been shown.

Corresponding author address: Emmanuel Roulin, Institut Royal Météorologique de Belgique, 3 Avenue Circulaire, B-1180 Brussels, Belgium. E-mail: emmanuel.roulin@meteo.be

Abstract

Extended logistic regression is used to calibrate areal precipitation forecasts over two small catchments in Belgium computed with the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System (EPS) between 2006 and 2010. The parameters of the postprocessing are estimated from the hindcast database, characterized by a much lower number of members (5) than the EPS (51). Therefore, the parameters have to be corrected for predictor uncertainties. They have been fitted on the 51-member EPS ensembles, on 5-member subensembles drawn from the same EPS, and on the 5-member hindcasts. For small ensembles, a simple “regression calibration” method by which the uncertain predictors are corrected has been applied. The different parameter sets have been compared, and the corresponding extended logistic regressions have been applied to the 51-member EPS. The forecast probabilities have then been validated using rain gauge data and compared with the raw EPS. In addition, the calibrated distributions are also used to modify the ensembles of precipitation traces.

The postprocessing with the extended logistic regression is shown to improve the continuous ranked probability skill score relative to the raw ensemble, and the regression calibration to remove a large portion of the bias in parameter estimation with small ensembles. With a training phase limited to a 5-week moving window, the benefit lasts for the first 2 forecast days in winter and the first 5 or 6 days in summer. In general, substantial improvements of the mean error and of the continuous ranked probability score have been shown.

Corresponding author address: Emmanuel Roulin, Institut Royal Météorologique de Belgique, 3 Avenue Circulaire, B-1180 Brussels, Belgium. E-mail: emmanuel.roulin@meteo.be
Save
  • Applequist, S., G. E. Gahrs, R. L. Pfeffer, and X.-F. Niu, 2002: Comparison of methodologies for probabilistic quantitative precipitation forecasting. Wea. Forecasting, 17, 783799.

    • Search Google Scholar
    • Export Citation
  • Carroll, R. J., and L. A. Stefanski, 1990: Approximate quasilikelihood estimation in models with surrogate predictors. J. Amer. Stat. Assoc., 85, 652663.

    • Search Google Scholar
    • Export Citation
  • Carroll, R. J., D. Ruppert, L. A. Stefanski, and C. M. Crainiceanu, 2006: Measurement Error in Nonlinear Models: A Modern Perspective. Chapman & Hall/CRC Press, 455 pp.

    • Search Google Scholar
    • Export Citation
  • Casella, G., and R. L. Berger, 2002: Statistical Inference. 2nd ed. Duxbury Press, 660 pp.

  • Folland, C., and C. Anderson, 2002: Estimating changing extremes using empirical ranking methods. J. Climate, 15, 29542960.

  • Gleser, L. J., 1990: Improvements of the naive approach to estimation in nonlinear errors-in-variables regression models. Contemp. Math., 112, 99114.

    • Search Google Scholar
    • Export Citation
  • Hagedorn, R., 2008: Using the ECMWF reforecast dataset to calibrate EPS forecasts. ECMWF Newsletter, No. 117, ECMWF, Reading, United Kingdom, 9–13.

    • Search Google Scholar
    • Export Citation
  • Hagedorn, R., T. M. Hamill, and J. S. Whitaker, 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part I: Two-meter temperatures. Mon. Wea. Rev., 136, 26082619.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and S. J. Colucci, 1998: Evaluation of Eta-RSM ensemble probabilistic precipitation forecasts. Mon. Wea. Rev., 126, 711724.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and J. S. Whitaker, 2006: Probabilistic quantitative precipitation forecasts based on reforecast analogs: Theory and application. Mon. Wea. Rev., 134, 32093229.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. S. Whitaker, and X. Wei, 2004: Ensemble reforecasting: improving medium-range forecast skill using retrospective forecasts. Mon. Wea. Rev., 132, 14341447.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., R. Hagedorn, and J. S. Whitaker, 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part II: Precipitation. Mon. Wea. Rev., 136, 26202632.

    • Search Google Scholar
    • Export Citation
  • Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15, 559570.

    • Search Google Scholar
    • Export Citation
  • Katz, R. W., and M. Ehrendorfer, 2006: Bayesian approach to decision making using ensemble weather forecasts. Wea. Forecasting, 21, 220231.

    • Search Google Scholar
    • Export Citation
  • Leutbecher, M., and T. N. Palmer, 2008: Ensemble forecasting. J. Comput. Phys., 227, 35153539.

  • McCollor, D., and R. Stull, 2008: Hydrometeorological accuracy enhancement via postprocessing of numerical wheather forecasts in complex terrain. Wea. Forecasting, 23, 131144.

    • Search Google Scholar
    • Export Citation
  • Peel, S., and L. J. Wilson, 2008: Modeling the distribution of precipitation forecasts from the Canadian ensemble prediction system using kernel density estimation. Wea. Forecasting, 23, 575595.

    • Search Google Scholar
    • Export Citation
  • Reggiani, P., and A. H. Weerts, 2008: Probabilistic quantitative precipitation forecast for flood prediction: An application. J. Hydrometeor., 9, 7695.

    • Search Google Scholar
    • Export Citation
  • Rodwell, M., 2006: Comparing and combining deterministic and ensemble forecasts: How to predict rainfall occurrence better. ECMWF Newsletter, No. 106, ECMWF, Reading, United Kingdom, 17–23.

    • Search Google Scholar
    • Export Citation
  • Rosner, B., W. C. Willett, and D. Spiegelman, 1989: Correction of logistic regression relative risk estimates and confidence intervals for systematic within-person measurement error. Stat. Med., 8, 10511069.

    • Search Google Scholar
    • Export Citation
  • Rosner, B., D. Spiegelman, and W. C. Willett, 1990: Correction of logistic regression relative risk estimates and confidence intervals for measurement error: The case of multiple covariate measured with error. Amer. J. Epidemiol., 132, 734745.

    • Search Google Scholar
    • Export Citation
  • Rosner, B., D. Spiegelman, and W. C. Willett, 1992: Correction of logistic regression relative risk estimates and confidence intervals for random within-person measurement error. Amer. J. Epidemiol., 136, 14001413.

    • Search Google Scholar
    • Export Citation
  • Roulin, E., and S. Vannitsem, 2005: Skill of medium-range hydrological ensemble predictions. J. Hydrometeor., 6, 729744.

  • Roulston, M. S., and L. A. Smith, 2002: Evaluating probabilistic forecasts using information theory. Mon. Wea. Rev., 130, 16531660.

  • Schmeits, M. J., and K. J. Kok, 2010: A comparison between raw ensemble output, (modified) Bayesian model averaging, and extended logistic regression using ECMWF ensemble precipitation reforecasts. Mon. Wea. Rev., 138, 41994211.

    • Search Google Scholar
    • Export Citation
  • Sloughter, J. M., A. E. Raftery, T. Gneiting, and C. Fraley, 2007: Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Mon. Wea. Rev., 135, 32093220.

    • Search Google Scholar
    • Export Citation
  • Stefanski, L. A., and R. J. Carroll, 1985: Covariate measurement error in logistic regression. Ann. Stat., 13, 13351351.

  • Thoresen, M., and P. Laake, 2000: A simulation study of measurement error correction methods in logistic regression. Biometrics, 56, 868872.

    • Search Google Scholar
    • Export Citation
  • Thoresen, M., and P. Laake, 2003: The use of replicates in logistic measurement error modeling. Scand. J. Stat., 30, 625636.

  • Van den Bergh, J., and E. Roulin, 2009: Hydrological ensemble prediction and verification for the Meuse and Scheldt basins. Atmos. Sci. Lett., 11, 6471.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2009: Extending logistic regression to provide full-probability-distribution MOS forecasts. Meteor. Appl., 16, 361368.

  • Wilks, D. S., and T. M. Hamill, 2007: Comparison of ensemble-MOS methods using GFS reforecasts. Mon. Wea. Rev., 135, 23792390.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 3707 3080 66
PDF Downloads 463 69 10