Cover Monthly Weather Review
Monthly Weather Review

  • Doswell, C. A. I. I. I., , R. Davies-Jones, , and D. L. Keller, 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting, 5 , 576–585.

    • Search Google Scholar
    • Export Citation

  • Epstein, E. S., 1969: A scoring system for probability forecasts of ranked categories. J. Appl. Meteor, 8 , 985–987.

  • Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14 , 155–167.

  • Hanssen, A. W., , and W. J. A. Kuipers, 1965: On the relationship between the frequency of rain and various meteorological parameters. Meded. Verh, 81 , 2–15.

    • Search Google Scholar
    • Export Citation

  • Katz, R. W., , A. H. Murphy, , and R. L. Winkler, 1982: Assessing the value of frost forecasts to orchardists: A dynamic decision-making approach. J. Appl. Meteor, 21 , 518–531.

    • Search Google Scholar
    • Export Citation

  • Mason, I., 1982: A model for assessment of weather forecasts. Austr. Meteor. Mag, 30 , 291–303.

  • Murphy, A. H., 1971: A note on the ranked probability score. J. Appl. Meteor, 10 , 155–156.

  • Murphy, A. H., 1985: Decision making and the value of forecasts in a generalized model of the cost-loss ratio situation. Mon. Wea. Rev, 113 , 362–369.

    • Search Google Scholar
    • Export Citation

  • Murphy, A. H., 1991: Forecast verification: Its complexity and dimensionality. Mon. Wea. Rev, 119 , 1590–1601.

  • Murphy, A. H., 1996: The Finley affair: A signal event in the history of forecast verification. Wea. Forecasting, 11 , 3–20.

  • Richardson, D. S., 2000: Skill and economic value of the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc, 126 , 649–668.

  • Strauss, B., , and A. Lanzinger, 1995: Validation of the ECMWF EPS. Proc. ECMWF Seminar on Predictability, Vol. 2, Shinfield Park, Reading, United Kingdom, ECMWF, 157–166.

    • Search Google Scholar
    • Export Citation

  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 467 pp.

  • Wilks, D. S., , and T. M. Hamill, 1995: Potential economic value of ensemble-based surface weather forecasts. Mon. Wea. Rev, 123 , 3564–3575.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 41 41 11
PDF Downloads 32 32 6
Editorial Type:
Article

Accuracy and Potential Economic Value of Categorical and Probabilistic Forecasts of Discrete Events

Roberto Buizza1
View More View Less
  • 1 European Centre for Medium-Range Weather Forecasts, Reading, Berkshire, United Kingdom
Print Publication:
01 Sep 2001
DOI:
https://doi.org/10.1175/1520-0493(2001)129<2329:AAPEVO>2.0.CO;2
Page(s):
2329–2345
© Get Permissions
Restricted access

Abstract

The accuracy and the potential economic value of categorical and probabilistic forecasts of discrete events are discussed. Accuracy is assessed applying known measures of forecast accuracy, and the potential economic value is measured by a weighted difference between the system probability of detection and the probability of false detection, with weights function of the cost–loss ratio and the observed ratio and the observed relative frequency of the event.

Results obtained using synthetic forecast and observed fields document the sensitivity of accuracy measures and of the potential forecast economic value to imposed random and systematic errors. It is shown that forecast skill cannot be defined per se but depends on the measure used to assess it: forecasts judged to be skillful according to one measure can show no skill according to another measures. More generally, it is concluded that the design of a forecasting system should follow the definition of its purposes, and should be such that the ensemble system maximizes its performance as assessed by the accuracy measures that best quantify the achievement of its purposes.

Results also indicate that independently from the model error (random or systematic) ensemble-based probabilistic forecasts exhibit higher potential economic values than categorical forecasts.

Corresponding author address: Dr. Roberto Buizza, European Centre for Medium-Range Weather Forecasts, Shinfield Park, Reading, Berkshire RG2 9AX, United Kingdom. Email: buizza@ecmwf.int

Abstract

The accuracy and the potential economic value of categorical and probabilistic forecasts of discrete events are discussed. Accuracy is assessed applying known measures of forecast accuracy, and the potential economic value is measured by a weighted difference between the system probability of detection and the probability of false detection, with weights function of the cost–loss ratio and the observed ratio and the observed relative frequency of the event.

Results obtained using synthetic forecast and observed fields document the sensitivity of accuracy measures and of the potential forecast economic value to imposed random and systematic errors. It is shown that forecast skill cannot be defined per se but depends on the measure used to assess it: forecasts judged to be skillful according to one measure can show no skill according to another measures. More generally, it is concluded that the design of a forecasting system should follow the definition of its purposes, and should be such that the ensemble system maximizes its performance as assessed by the accuracy measures that best quantify the achievement of its purposes.

Results also indicate that independently from the model error (random or systematic) ensemble-based probabilistic forecasts exhibit higher potential economic values than categorical forecasts.

Corresponding author address: Dr. Roberto Buizza, European Centre for Medium-Range Weather Forecasts, Shinfield Park, Reading, Berkshire RG2 9AX, United Kingdom. Email: buizza@ecmwf.int

Save