The Effect of Probabilistic Information on Threshold Forecasts

Susan Joslyn Department of Psychology, University of Washington, Seattle, Washington

Search for other papers by Susan Joslyn in
Current site
Google Scholar
PubMed
Close
,
Karla Pak Department of Psychology, University of Washington, Seattle, Washington

Search for other papers by Karla Pak in
Current site
Google Scholar
PubMed
Close
,
David Jones Applied Physics Laboratory, University of Washington, Seattle, Washington

Search for other papers by David Jones in
Current site
Google Scholar
PubMed
Close
,
John Pyles Department of Cognitive Sciences, University of California, Irvine, Irvine, California

Search for other papers by John Pyles in
Current site
Google Scholar
PubMed
Close
, and
Earl Hunt Department of Psychology, University of Washington, Seattle, Washington

Search for other papers by Earl Hunt in
Current site
Google Scholar
PubMed
Close
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

The study reported here asks whether the use of probabilistic information indicating forecast uncertainty improves the quality of deterministic weather decisions. Participants made realistic wind speed forecasts based on historical information in a controlled laboratory setting. They also decided whether it was appropriate to post an advisory for winds greater than 20 kt (10.29 m s−1) during the same time intervals and in the same geographic locations. On half of the forecasts each participant also read a color-coded chart showing the probability of winds greater than 20 kt. Participants had a general tendency to post too many advisories in the low probability situations (0%–10%) and too few advisories in very high probability situations (90%–100%). However, the probability product attenuated these biases. When participants used the probability product, they posted fewer advisories when the probability of high winds was low and they posted more advisories when the probability of high winds was high. The difference was due to the probability product alone because the within-subjects design and counterbalancing of forecast dates ruled out alternative explanations. The data suggest that the probability product improved threshold forecast decisions.

Corresponding author address: Susan Joslyn, Dept. of Psychology, University of Washington, Box 351525, Seattle, WA 98125. Email: susanj@u.washington.edu

Abstract

The study reported here asks whether the use of probabilistic information indicating forecast uncertainty improves the quality of deterministic weather decisions. Participants made realistic wind speed forecasts based on historical information in a controlled laboratory setting. They also decided whether it was appropriate to post an advisory for winds greater than 20 kt (10.29 m s−1) during the same time intervals and in the same geographic locations. On half of the forecasts each participant also read a color-coded chart showing the probability of winds greater than 20 kt. Participants had a general tendency to post too many advisories in the low probability situations (0%–10%) and too few advisories in very high probability situations (90%–100%). However, the probability product attenuated these biases. When participants used the probability product, they posted fewer advisories when the probability of high winds was low and they posted more advisories when the probability of high winds was high. The difference was due to the probability product alone because the within-subjects design and counterbalancing of forecast dates ruled out alternative explanations. The data suggest that the probability product improved threshold forecast decisions.

Corresponding author address: Susan Joslyn, Dept. of Psychology, University of Washington, Box 351525, Seattle, WA 98125. Email: susanj@u.washington.edu

Save
  • Anderson, O., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9 , 15181530.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baars, J. A., Mass C. , and Albright M. , 2004: Performance of National Weather Service forecasts versus model output statistics. Preprints, 20th Conf. on Weather Analysis and Forecasting and 16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., CD-ROM, 20.5.

  • Gonzalez, R., and Wu G. , 1999: On the form of the probability weighting function. Cogn. Psychol., 38 , 129166.

  • Green, D. M., and Swets J. Z. , 1966: Signal Detection Theory and Psychophysics. Wiley, 455 pp.

  • Grimit, E. P., and Mass C. F. , 2002: Initial results of a mesoscale short-range ensemble forecasting system over the Pacific Northwest. Wea. Forecasting, 17 , 192205.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grimit, E. P., and Mass C. F. , 2003: Toward short-range ensemble prediction of mesoscale forecast skill. Preprints, 10th Conf. on Mesoscale Processes, Portland, OR, Amer. Meteor. Soc., 22–27.

  • Kahneman, D., and Tversky A. , 1979: Prospect theory: An analysis of decision under risk. Econometrica, 47 , 263291.

  • Keith, R., 2003: Optimization of value of aerodrome forecasts. Wea. Forecasting, 18 , 808824.

  • Mason, I. B., 1982: A model for assessment of weather forecasts. Aust. Meteor. Mag., 30 , 291303.

  • Mason, I. B., 2003: Binary events. Forecast Verification: A Practitioner’s Guide in Atmospheric Science, I. T. Jolliffe and D. B. Stephenson, Eds., Wiley, 37–76.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and Winkler R. L. , 1974a: Credible interval temperature forecasting: Some experimental results. Mon. Wea. Rev., 102 , 784794.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and Winkler R. L. , 1974b: Subjective probability forecasting experiments in meteorology: Some preliminary results. Bull. Amer. Meteor. Soc., 55 , 12061216.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and Winkler R. L. , 1977: Can weather forecasters formulate reliable probability forecasts of precipitation and temperature? Natl. Wea. Dig., 2 , 2. 29.

    • Search Google Scholar
    • Export Citation
  • National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. Committee on Estimating and Communicating Uncertainty in Weather and Climate Forecasts, The National Academies Press, 124 pp.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., 2002: The economic value of ensemble forecasts as a tool for risk assessment: From days to decades. Quart. J. Roy. Meteor. Soc., 128 , 747774.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Richardson, D. S., 2001: Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size. Quart. J. Roy. Meteor. Soc., 127 , 24732489.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., and Smith L. A. , 2004: The boy who cried wolf revisited: The impact of false alarm intolerance on cost–loss scenarios. Wea. Forecasting, 19 , 391397.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., Bolton G. E. , Kleit A. N. , and Sears-Collins A. L. , 2006: A laboratory study of the benefits of including uncertainty information in weather forecasts. Wea. Forecasting, 21 , 116122.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weber, E. U., 1994: From subjective probabilities to decision weights: The effect of asymmetric loss function. Psychol. Bull., 115 , 228242.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Windschitl, P. D., and Weber E. , 1999: The interpretation of “likely” depends on the context, but “70%” is 70%—Right? The influence of associative processes on perceived certainty. J. Exp. Psychol. Learn. Mem. Cogn., 25 , 15141533.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 718 277 48
PDF Downloads 294 64 9