• Brooks, H., and C. A. Doswell III, 1995: A comparison of measures- oriented and distributions-oriented approaches to forecast verification. Wea. Forecasting,11, 288–303.

  • Donaldson, R. J., R. M. Dyer, and M. J. Krauss, 1975: An objective evaluator of techniques for predicting severe weather events. Preprints, Ninth Conference on Severe Local Storms, Norman, OK, Amer. Meteor. Soc., 321–326.

  • Doswell, C. A., III, R. Davies-Jones, and D. Keller, 1990: On summary measures of skill in rare event forecasting based on contingency tables. Wea. Forecasting,5, 576–585.

  • Gandin, L. S., and A. Murphy, 1992: Equitable skill scores for categorical forecasts, Mon. Wea. Rev.,120, 361–370.

  • Hamill, T. M., 1997: Reliability diagrams for multi-category probabilistic forecasts, Wea. Forecasting,12, in press.

  • Kendall, M. G., and A. Stuart, 1969: The Advanced Theory of Statistics. Vol. 1. Hafner Publishing, 439 pp.

  • Mason, I., 1982: A model for assessment of weather forecasts. Aust. Meteor. Mag.,30, 291–303.

  • Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting,8, 281–293.

  • ——, 1996: The Finley affair: A signal event in the history of forecast verification. Wea. Forecasting,11, 3–20.

  • ——, and R. L. Winkler, 1987: A general framework for forecast verification, Mon. Wea. Rev.,115, 1330–1338.

  • ——, and ——, 1992: Diagnostic verification of probability forecasts. Int. J. Forecasting,7, 435–455.

  • ——, B. G. Brown, and Y.-S. Chen, 1989: Diagnostic verification of temperature forecasts. Wea. Forecasting,4, 485–501.

  • O’Hagan, A., 1994: Kendall’s Advanced Theory of Statistics. Vol. 2B, Bayesian Inference, Halsted Press, 330 pp.

  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill, Wea. Forecasting,5, 570–575.

  • Wilks, D. S., 1995: Statistical Methods in the atmospheric Sciences, Academic Press, 467 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 117 114 2
PDF Downloads 26 25 5

Bayesian Probability and Scalar Performance Measures in Gaussian Models

View More View Less
  • 1 National Severe Storms Laboratory, Cooperative Institute for Mesoscale and Meteorological Studies, and Department of Physics, University of Oklahoma, Norman, Oklahoma
© Get Permissions Rent on DeepDyve
Restricted access

Abstract

The transformation of a real, continuous variable into an event probability is reviewed from the Bayesian point of view, after which a Gaussian model is employed to derive an explicit expression for the probability. In turn, several scalar (one-dimensional) measures of performance quality and reliability diagrams are computed. It is shown that if the optimization of scalar measures is of concern, then prior probabilities must be treated carefully, whereas no special care is required for reliability diagrams. Specifically, since a scalar measure gauges only one component of performance quality—a multidimensional entity—it is possible to find the critical value of prior probability that optimizes that scalar measure; this value of “prior probability” is often not equal to the “true” value as estimated from group sample sizes. Optimum reliability, however, is obtained when prior probability is equal to the estimate based on group sample sizes. Exact results are presented for the critical value of “prior probability” that optimize the fraction correct, the true skill statistic, and the reliability diagram, but the critical success index and the Heidke skill statistic are treated only graphically. Finally, an example based on surface air pressure data is employed to illustrate the results in regard to precipitation forecasting.

Corresponding author address: Dr. Caren Marzban, National Severe Storms Laboratory, 1313 Halley Circle, Norman, OK 73069.

marzban@gump.nssl.noaa.gov

Abstract

The transformation of a real, continuous variable into an event probability is reviewed from the Bayesian point of view, after which a Gaussian model is employed to derive an explicit expression for the probability. In turn, several scalar (one-dimensional) measures of performance quality and reliability diagrams are computed. It is shown that if the optimization of scalar measures is of concern, then prior probabilities must be treated carefully, whereas no special care is required for reliability diagrams. Specifically, since a scalar measure gauges only one component of performance quality—a multidimensional entity—it is possible to find the critical value of prior probability that optimizes that scalar measure; this value of “prior probability” is often not equal to the “true” value as estimated from group sample sizes. Optimum reliability, however, is obtained when prior probability is equal to the estimate based on group sample sizes. Exact results are presented for the critical value of “prior probability” that optimize the fraction correct, the true skill statistic, and the reliability diagram, but the critical success index and the Heidke skill statistic are treated only graphically. Finally, an example based on surface air pressure data is employed to illustrate the results in regard to precipitation forecasting.

Corresponding author address: Dr. Caren Marzban, National Severe Storms Laboratory, 1313 Halley Circle, Norman, OK 73069.

marzban@gump.nssl.noaa.gov

Save