What Is the True Value of Forecasts?

Antony Millner Oxford University Centre for the Environment, Oxford, United Kingdom

Search for other papers by Antony Millner in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Understanding the economic value of weather and climate forecasts is of tremendous practical importance. Traditional models that have attempted to gauge forecast value have focused on a best-case scenario, in which forecast users are assumed to be statistically sophisticated, hyperrational decision makers with perfect knowledge and understanding of forecast performance. These models provide a normative benchmark for assessing forecast value, but say nothing about the value that actual forecast users realize. Real forecast users are subject to a variety of behavioral effects and informational constraints that violate the assumptions of normative models. In this paper, one of the normative assumptions about user behavior is relaxed—users are no longer assumed to be in possession of a perfect statistical understanding of forecast performance. In the case of a cost–loss decision, it is shown that a model of users’ forecast use choices based on the psychological theory of reinforcement learning leads to a behavioral adjustment factor that lowers the relative value score that the user achieves. The dependence of this factor on the user’s decision parameters (the ratio of costs to losses) and the forecast skill is deduced. Differences between the losses predicted by the behavioral and normative models are greatest for users with intermediate cost–loss ratios, and when forecasts have intermediate skill. The relevance of the model as a tool for directing user education initiatives is briefly discussed, and a direction for future research is proposed.

Corresponding author address: Antony Millner, Oxford University Centre for the Environment, South Parks Road, Oxford OX1 3QY, United Kingdom. Email: antony.millner@magd.ox.ac.uk

Abstract

Understanding the economic value of weather and climate forecasts is of tremendous practical importance. Traditional models that have attempted to gauge forecast value have focused on a best-case scenario, in which forecast users are assumed to be statistically sophisticated, hyperrational decision makers with perfect knowledge and understanding of forecast performance. These models provide a normative benchmark for assessing forecast value, but say nothing about the value that actual forecast users realize. Real forecast users are subject to a variety of behavioral effects and informational constraints that violate the assumptions of normative models. In this paper, one of the normative assumptions about user behavior is relaxed—users are no longer assumed to be in possession of a perfect statistical understanding of forecast performance. In the case of a cost–loss decision, it is shown that a model of users’ forecast use choices based on the psychological theory of reinforcement learning leads to a behavioral adjustment factor that lowers the relative value score that the user achieves. The dependence of this factor on the user’s decision parameters (the ratio of costs to losses) and the forecast skill is deduced. Differences between the losses predicted by the behavioral and normative models are greatest for users with intermediate cost–loss ratios, and when forecasts have intermediate skill. The relevance of the model as a tool for directing user education initiatives is briefly discussed, and a direction for future research is proposed.

Corresponding author address: Antony Millner, Oxford University Centre for the Environment, South Parks Road, Oxford OX1 3QY, United Kingdom. Email: antony.millner@magd.ox.ac.uk

Save
  • Arthur, W., 1991: Designing economic agents that act like human agents: A behavioral approach to bounded rationality. Amer. Econ. Rev., 81 , 353359.

    • Search Google Scholar
    • Export Citation
  • Behrens, T., Woolrich M. , Walton M. , and Rushworth M. , 2007: Learning the value of information in an uncertain world. Nat. Neurosci., 10 , 12141221.

  • Brenner, T., 2006: Agent learning representation: Advice on modelling economic learning. Agent-Based Computational Economics, L. Tesfatsion and K. Judd, Eds., Vol. 2, Handbook of Computational Economics, Elsevier, 895–947.

    • Search Google Scholar
    • Export Citation
  • Bush, R., and Mosteller F. , 1955: Stochastic Models for Learning. Wiley, 365 pp.

  • Camerer, C., 2003: Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University Press, 550 pp.

  • Cross, J., 1973: A stochastic learning model of economic behavior. Quart. J. Econ., 87 , 239266.

  • Dennett, D., 1975: Why the law of effect will not go away. J. Theory Soc. Behav., 5 , 169188.

  • Erev, I., and Roth A. , 1998: Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. Amer. Econ. Rev., 88 , 848881.

    • Search Google Scholar
    • Export Citation
  • Erev, I., Ert E. , and Yechiam E. , 2008: Loss aversion, diminishing sensitivity, and the effect of experience on repeated decisions. J. Behav. Decis. Making, 21 , 575597.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gigerenzer, G., Hertwig R. , van den Broek E. , Fasolo B. , and Katsikopoulos K. , 2005: “A 30% chance of rain tomorrow”: How does the public understand probabilistic weather forecasts? Risk Anal., 25 , 623629.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jolliffe, I., and Stephenson D. , 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. Wiley, 240 pp.

  • Kahneman, D., and Tversky A. , 1979: Prospect theory: An analysis of decision under risk. Econometrica, 47 , 263292.

  • Kahneman, D., and Tversky A. , 2000: Choices, Values, and Frames. Cambridge University Press, 840 pp.

  • Katz, R., and Murphy A. , 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.

  • Katz, R., and Ehrendorfer M. , 2006: Bayesian approach to decision making using ensemble weather forecasts. Wea. Forecasting, 21 , 220223.

  • Khamsi, M., and Kirk W. , 2001: An Introduction to Metric Spaces and Fixed Point Theory. Wiley, 302 pp.

  • Lieberman, D., 1999: Learning: Behavior and Cognition. 3rd ed. Wadsworth Publishing, 595 pp.

  • Mazur, J., 2006: Learning and Behavior. 6th ed. Prentice Hall, 444 pp.

  • Millner, A., 2008: Getting the most out of ensemble forecasts: A valuation model based on user–forecast interactions. J. Appl. Meteor. Climatol., 47 , 25612571.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A., 1977: The value of climatological, categorical and probabilistic forecasts in the cost–loss ratio situation. Mon. Wea. Rev., 105 , 803816.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A., and Ehrendorfer M. , 1987: On the relationship between the accuracy and value of forecasts in the cost–loss ratio situation. Wea. Forecasting, 2 , 243251.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A., and Winkler R. , 1987: A general framework for forecast verification. Mon. Wea. Rev., 115 , 13301338.

  • National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. National Academies Press, 112 pp.

    • Search Google Scholar
    • Export Citation
  • Nicholls, N., 1999: Cognitive illusions, heuristics, and climate prediction. Bull. Amer. Meteor. Soc., 80 , 13851397.

  • Norman, M., 1968: Some convergence theorems for stochastic learning models with distance diminishing operators. J. Math. Psychol., 5 , 61101.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Oreskes, N., Shrader-Frechette K. , and Belitz K. , 1994: Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263 , 641646.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Patt, A., and Gwata C. , 2002: Effective seasonal climate forecast applications: Examining constraints for subsistence farmers in Zimbabwe. Global Environ. Change, 12 , 185195.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Patt, A., Suarez P. , and Gwata C. , 2005: Effects of seasonal climate forecasts and participatory workshops among subsistence farmers in Zimbabwe. Proc. Natl. Acad. Sci. USA, 102 , 1267312678.

    • Search Google Scholar
    • Export Citation
  • Patt, A., Ogallo L. , and Hellmuth M. , 2007: Learning from 10 years of climate outlook forums in Africa. Science, 318 , 4950.

  • Pavlov, I., 1928: Lectures on Conditioned Reflexes: Twenty-Five Years of Objective Study of the Higher Nervous Activity (Behavior) of Animals. International Publishers, 414 pp.

    • Search Google Scholar
    • Export Citation
  • Pielke R. Jr., , and Carbone R. E. , 2002: Weather impacts, forecasts, and policy: An integrated perspective. Bull. Amer. Meteor. Soc., 83 , 393403.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Platt, M., and Huettel S. , 2008: Risky business: The neuroeconomics of decision making under uncertainty. Nat. Neurosci., 11 , 398403.

  • Rayner, S., Lach D. , and Ingram H. , 2005: Weather forecasts are for wimps: Why water resource managers do not use climate forecasts. Climatic Change, 69 , 197227.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Richardson, D., 2001: Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size. Quart. J. Roy. Meteor. Soc., 127 , 2473.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roncoli, C., 2006: Ethnographic and participatory approaches to research on farmer responses to climate predictions. Climate Res., 33 , 8199.

  • Roulston, M. S., and Smith L. A. , 2004: The boy who cried wolf revisited: The impact of false alarm intolerance on cost–loss scenarios. Wea. Forecasting, 19 , 391397.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rushworth, M., and Behrens T. , 2008: Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci., 11 , 389397.

  • Skinner, B., 1933: The rate of establishment of a discrimination. J. Gen. Psychol., 9 , 302350.

  • Sonka, S., Changnon S. , and Hofing S. , 1988: Assessing climate information use in agribusiness. Part II: Decision experiments to estimate economic value. J. Climate, 1 , 766774.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stewart, T., 1997: Forecast value: Descriptive decision studies. Economic Value of Weather and Climate Forecasts, R. Katz and A. Murphy, Eds., Cambridge University Press, 147–181.

    • Search Google Scholar
    • Export Citation
  • Stewart, T., Pielke R. Jr., and Nath R. , 2004: Understanding user decision making and the value of improved precipitation forecasts: Lessons from a case study. Bull. Amer. Meteor. Soc., 85 , 223235.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thorndike, E., 1911: Animal Intelligence: Experimental Studies. Macmillan, 297 pp.

  • Thorndike, E., 1932: The Fundamentals of Learning. AMS Press, 638 pp.

  • Thorndike, E., 1933: A theory of the action of the after-effects of a connection upon it. Psychol. Rev., 40 , 434439.

  • Vogel, C., and O’Brien K. , 2006: Who can eat information? Examining the effectiveness of seasonal climate forecasts and regional climate-risk management strategies. Climate Res., 33 , 111122.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D., 2001: A skill score based on economic value for probability forecasts. Meteor. Appl., 8 , 209219.

  • Wilks, D., 2006: Statistical Methods in the Atmospheric Sciences. 2nd ed. Academic Press, 627 pp.

  • Yu, A., and Dayan P. , 2005: Uncertainty, neuromodulation, and attention. Neuron, 46 , 681692.

  • Zhu, Y., Toth Z. , Wobus R. , Richardson D. , and Mylne K. , 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83 , 7383.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 360 120 12
PDF Downloads 275 78 6