• Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78 , 13.

  • Collett, D., 2002: Modelling Binary Data. 2nd ed. Chapman and Hall, 408 pp.

  • Ehrendorfer, M., 1997: Predicting the uncertainty of numerical weather forecasts: A review. Meteor. Z., 6 , 147183.

  • Epstein, E. S., 1985: Statistical Inference and Prediction in Climatology: A Bayesian Approach. Meteor. Monogr., No. 42, Amer. Meteor. Soc., 199 pp.

  • Ferranti, L., E. Klinker, A. Hollingsworth, and B. J. Hoskins, 2002: Diagnosis of systematic forecast errors dependent on flow pattern. Quart. J. Roy. Meteor. Soc., 128 , 16231640.

    • Search Google Scholar
    • Export Citation
  • Glahn, H. R., 2004: Discussion of verification concepts in Forecast Verification: A Practitioner's Guide in Atmospheric Science. Wea. Forecasting, 19 , 769775.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., A. E. Raftery, A. Westveld, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133 , 10981118.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., F. Balabdaoui, and A. E. Raftery, 2007: Probabilistic forecasts, calibration and sharpness. J. Roy. Stat. Soc., 69B , 243268.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., C. Snyder, and R. E. Morss, 2000: A comparison of probabilistic forecasts from bred, singular vector, and perturbed observation ensembles. Mon. Wea. Rev., 128 , 18351851.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., C. Snyder, and J. S. Whitaker, 2003: Ensemble forecasts and the properties of flow-dependent analysis-error covariance singular vectors. Mon. Wea. Rev., 131 , 17411758.

    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., and D. B. Stephenson, 2005: Comments on “Discussion of verification concepts in Forecast Verification: A Practitioner’s Guide in Atmospheric Science.”. Wea. Forecasting, 20 , 796800.

    • Search Google Scholar
    • Export Citation
  • Katz, R. W., and M. Ehrendorfer, 2006: Bayesian approach to decision making using ensemble weather forecasts. Wea. Forecasting, 21 , 220231.

    • Search Google Scholar
    • Export Citation
  • Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102 , 409418.

  • McCullagh, P., and J. Nelder, 1989: Generalized Linear Models. 2nd ed. Chapman and Hall, 532 pp.

  • Molteni, E., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122 , 73119.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1973: A new vector partition of the probability score. J. Appl. Meteor., 12 , 595600.

  • Rajagopalan, B., U. Lall, and S. E. Zebiak, 2002: Categorical climate forecasts through regularization and optimal combination of multiple GCM ensembles. Mon. Wea. Rev., 130 , 17921811.

    • Search Google Scholar
    • Export Citation
  • Robertson, A. W., U. Lall, S. E. Zebiak, and L. Goddard, 2004: Improved combination of multiple atmospheric GCM ensembles for seasonal prediction. Mon. Wea. Rev., 132 , 27322744.

    • Search Google Scholar
    • Export Citation
  • Stephenson, D. B., 2008: An introduction to probability forecasting. Seasonal Climate: Forecasting and Managing Risk, A. Trocolli et al., Eds., Nato Science Series, Vol. 82, Springer Academic, 235–258.

    • Search Google Scholar
    • Export Citation
  • Stephenson, D. B., and F. J. Doblas-Reyes, 2000: Statistical methods for interpreting Monte Carlo ensemble forecasts. Tellus, 52A , 300322.

    • Search Google Scholar
    • Export Citation
  • Wang, X., and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60 , 11401158.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2006a: Comparison of ensemble-MOS methods in the Lorenz ’96 settings. Meteor. Appl., 78 , 28512857.

  • Wilks, D. S., 2006b: Statistical Methods in the Atmospheric Sciences. 2nd ed. International Geophysical Series, Vol. 91, Academic Press, 648 pp.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., and T. M. Hamill, 2007: Comparison of ensemble-MOS methods using GFS reforecasts. Mon. Wea. Rev., 135 , 23792390.

  • Wilson, L. J., and M. Valle, 2002: The Canadian Updateable Model Output Statistics (UMOS) system: Design and development tests. Wea. Forecasting, 17 , 206222.

    • Search Google Scholar
    • Export Citation
  • Winkler, R. L., 1968: “Good” probability assessors. J. Appl. Meteor., 7 , 751758.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 199 107 5
PDF Downloads 149 81 2

Calibration of Probabilistic Forecasts of Binary Events

View More View Less
  • 1 European Centre for Medium-Range Weather Forecasts, Reading, United Kingdom
  • | 2 School of Engineering, Computing and Mathematics, University of Exeter, Exeter, United Kingdom
Restricted access

Abstract

Probabilistic forecasts of atmospheric variables are often given as relative frequencies obtained from ensembles of deterministic forecasts. The detrimental effects of imperfect models and initial conditions on the quality of such forecasts can be mitigated by calibration. This paper shows that Bayesian methods currently used to incorporate prior information can be written as special cases of a beta-binomial model and correspond to a linear calibration of the relative frequencies. These methods are compared with a nonlinear calibration technique (i.e., logistic regression) using real precipitation forecasts. Calibration is found to be advantageous in all cases considered, and logistic regression is preferable to linear methods.

Corresponding author address: Dr. Cristina Primo, European Centre for Medium-Range Weather Forecasts, Shinfield Park, Reading RG2 9AX, United Kingdom. Email: cristina.primo@ecmwf.int

Abstract

Probabilistic forecasts of atmospheric variables are often given as relative frequencies obtained from ensembles of deterministic forecasts. The detrimental effects of imperfect models and initial conditions on the quality of such forecasts can be mitigated by calibration. This paper shows that Bayesian methods currently used to incorporate prior information can be written as special cases of a beta-binomial model and correspond to a linear calibration of the relative frequencies. These methods are compared with a nonlinear calibration technique (i.e., logistic regression) using real precipitation forecasts. Calibration is found to be advantageous in all cases considered, and logistic regression is preferable to linear methods.

Corresponding author address: Dr. Cristina Primo, European Centre for Medium-Range Weather Forecasts, Shinfield Park, Reading RG2 9AX, United Kingdom. Email: cristina.primo@ecmwf.int

Save