• Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9 , 15181530.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clemen, R. T., 1996: Making Hard Decisions: An Introduction to Decision Analysis. 2d ed. Duxbury, 688 pp.

  • Ehrendorfer, M., 1997: Predicting the uncertainty of numerical weather forecasts: A review. Meteor. Z., 6 , 147183.

  • Ehrendorfer, M., , and Murphy A. H. , 1988: Comparative evaluation of weather forecasting systems: Sufficiency, quality, and accuracy. Mon. Wea. Rev., 116 , 17571770.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Epstein, E. S., 1985: Statistical Inference and Prediction in Climatology: A Bayesian Approach Meteor. Monogr., No. 42, Amer. Meteor. Soc., 199 pp.

    • Crossref
    • Export Citation
  • Hamill, T. M., , and Colucci S. J. , 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev., 125 , 13121327.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., , and Stephenson D. B. , 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. John Wiley, 240 pp.

    • Search Google Scholar
    • Export Citation
  • Katz, R. W., , and Murphy A. H. , 1987: Quality/value relationship for imperfect information in the umbrella problem. Amer. Stat., 41 , 187189.

    • Search Google Scholar
    • Export Citation
  • Katz, R. W., , and Murphy A. H. , 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.

  • Kharin, V. V., , and Zwiers F. W. , 2003: Improved seasonal probability forecasts. J. Climate, 16 , 16841701.

  • Krzysztofowicz, R., , and Long D. , 1991: Forecast sufficiency characteristic: Construction and application. Int. J. Forecasting, 7 , 3945.

  • Lindley, D. V., , and Phillips L. D. , 1976: Inference for a Bernoulli process (a Bayesian view). Amer. Stat., 30 , 112119.

  • Palmer, T. N., 2002: The economic value of ensemble forecasts as a tool for risk assessment: From days to decades. Quart. J. Roy. Meteor. Soc., 128 , 747774.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., and Coauthors, 2004: Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER). Bull. Amer. Meteor. Soc., 85 , 853872.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Raftery, A. E., , Gneiting T. , , Balabdaoui F. , , and Polakowski M. , 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133 , 11551174.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Richardson, D. S., 2000: Skill and relative economic value of the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 126 , 649667.

  • Richardson, D. S., 2001: Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size. Quart. J. Roy. Meteor. Soc., 127 , 24732489.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., , and Smith L. A. , 2002: Evaluating probabilistic forecasts using information theory. Mon. Wea. Rev., 130 , 16531660.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sivillo, J. K., , Ahlquist J. E. , , and Toth Z. , 1997: An ensemble forecasting primer. Wea. Forecasting, 12 , 809818.

  • Thompson, M. L., , and Zucchini W. , 1990: Assessing the value of probability forecasts. Mon. Wea. Rev., 118 , 26962706.

  • van Tiel, J., 1984: Convex Analysis: An Introductory Text. John Wiley, 125 pp.

  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 464 pp.

  • Wilks, D. S., 2002: Smoothing forecast ensembles with fitted probability distributions. Quart. J. Roy. Meteor. Soc., 128 , 28212836.

  • Wilks, D. S., , and Hamill T. M. , 1995: Potential economic value of ensemble-based surface weather forecasts. Mon. Wea. Rev., 123 , 35653575.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Winkler, R. L., 2003: Introduction to Bayesian Inference and Decision. 2d ed. Probabilistic Publishing, 452 pp.

  • Winkler, R. L., , Smith J. E. , , and Fryback D. G. , 2002: The role of informative priors in zero-numerator problems: Being conservative versus being candid. Amer. Stat., 56 , 14.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhu, Y., , Toth Z. , , Wobus R. , , Richardson D. , , and Mylne K. , 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83 , 7383.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 62 62 13
PDF Downloads 56 56 15

Bayesian Approach to Decision Making Using Ensemble Weather Forecasts

View More View Less
  • 1 Institute for the Study of Society and Environment, National Center for Atmospheric Research, * Boulder, Colorado
  • | 2 Institute for Meteorology and Geophysics, University of Innsbruck, Innsbruck, Austria
© Get Permissions
Restricted access

Abstract

The economic value of ensemble-based weather or climate forecasts is generally assessed by taking the ensembles at “face value.” That is, the forecast probability is estimated as the relative frequency of occurrence of an event among a limited number of ensemble members. Despite the economic value of probability forecasts being based on the concept of decision making under uncertainty, in effect, the decision maker is assumed to ignore the uncertainty in estimating this probability. Nevertheless, many users are certainly aware of the uncertainty inherent in a limited ensemble size. Bayesian prediction is used instead in this paper, incorporating such additional forecast uncertainty into the decision process. The face-value forecast probability estimator would correspond to a Bayesian analysis, with a prior distribution on the actual forecast probability only being appropriate if it were believed that the ensemble prediction system produces perfect forecasts. For the cost–loss decision-making model, the economic value of the face-value estimator can be negative for small ensemble sizes from a prediction system with a level of skill that is not sufficiently high. Further, this economic value has the counterintuitive property of sometimes decreasing as the ensemble size increases. For a more plausible form of prior distribution on the actual forecast probability, which could be viewed as a “recalibration” of face-value forecasts, the Bayesian estimator does not exhibit this unexpected behavior. Moreover, it is established that the effects of ensemble size on the reliability, skill, and economic value have been exaggerated by using the face-value, instead of the Bayesian, estimator.

* The National Center for Atmospheric Research is sponsored by the National Science Foundation

Corresponding author address: Dr. Richard W. Katz, Institute for the Study of Society and Environment, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO 80307-3000. Email: rwk@ucar.edu

Abstract

The economic value of ensemble-based weather or climate forecasts is generally assessed by taking the ensembles at “face value.” That is, the forecast probability is estimated as the relative frequency of occurrence of an event among a limited number of ensemble members. Despite the economic value of probability forecasts being based on the concept of decision making under uncertainty, in effect, the decision maker is assumed to ignore the uncertainty in estimating this probability. Nevertheless, many users are certainly aware of the uncertainty inherent in a limited ensemble size. Bayesian prediction is used instead in this paper, incorporating such additional forecast uncertainty into the decision process. The face-value forecast probability estimator would correspond to a Bayesian analysis, with a prior distribution on the actual forecast probability only being appropriate if it were believed that the ensemble prediction system produces perfect forecasts. For the cost–loss decision-making model, the economic value of the face-value estimator can be negative for small ensemble sizes from a prediction system with a level of skill that is not sufficiently high. Further, this economic value has the counterintuitive property of sometimes decreasing as the ensemble size increases. For a more plausible form of prior distribution on the actual forecast probability, which could be viewed as a “recalibration” of face-value forecasts, the Bayesian estimator does not exhibit this unexpected behavior. Moreover, it is established that the effects of ensemble size on the reliability, skill, and economic value have been exaggerated by using the face-value, instead of the Bayesian, estimator.

* The National Center for Atmospheric Research is sponsored by the National Science Foundation

Corresponding author address: Dr. Richard W. Katz, Institute for the Study of Society and Environment, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO 80307-3000. Email: rwk@ucar.edu

Save