• Arguez, A., , I. Durre, , S. Applequist, , R. S. Vose, , M.F. Squires & , and X. Yin, 2012: NOAA's 1981–2010 U.S. Climate Normals: An overview. Preprints, 24th Conf. on Climate Variability and Change, New Orleans, LA, Amer. Meteor. Soc., 11A.4. [Available online at http://ams.confex.com/ams/92Annual/webprogram/Manuscript/Paper195247/Arguez-Extended-Normals-AMS2012.pdf.]

    • Search Google Scholar
    • Export Citation
  • Bosart, L. F., 1983: An update on trends in skill of daily forecasts of temperature and precipitation at the State University of New York at Albany. Bull. Amer. Meteor. Soc., 64, 346354.

    • Search Google Scholar
    • Export Citation
  • Colucci, S. J., , P. C. Knappenberger & , and T. K. Ceppa, 1992: Evaluation of a nonprobabilistic weather forecast experiment. Wea. Forecasting, 7, 507514.

    • Search Google Scholar
    • Export Citation
  • Croft, P. J. & , and J. D. Milutinovic, 1991: The Rutgers University forecasting contest: Forecaster performance versus model guidance. Natl. Wea. Dig., 16, 212.

    • Search Google Scholar
    • Export Citation
  • de Elía, R. & , and R. Laprise, 2005: Diversity in interpretations of probability: Implications for weather forecasting. Mon. Wea. Rev., 133, 11291143.

    • Search Google Scholar
    • Export Citation
  • Dunsmore, I. R., 1968: A Bayesian approach to calibration. J. Roy. Stat. Soc., 30B, 396405.

  • Glahn, H. R. & , and D. P. Ruth, 2003: The new digital forecast database of the National Weather Service. Bull. Amer. Meteor. Soc., 84, 195201.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T. & , and A. E. Raftery, 2007: Strictly proper scoring rules, prediction, and estimation. J. Amer. Stat. Assoc., 102, 359378.

    • Search Google Scholar
    • Export Citation
  • Gyakum, J. R., 1986: Experiments in temperature and precipitation forecasting for Illinois. Wea. Forecasting, 1, 7788.

  • Hamill, T. M. & , and D. S. Wilks, 1995: A probabilistic forecast contest and the difficulty in assessing short-range forecast uncertainty. Wea. Forecasting, 10, 620631.

    • Search Google Scholar
    • Export Citation
  • Illston, B. G., , J. B. Basara, , M. Voss & , and C. C. Weiss, 2009: An overview of the WxChallenge forecasting competition and its use as an educational tool. Preprints, 18th Symp. on Education, Phoenix, AZ, Amer. Meteor. Soc., 4.4. [Available online at https://ams.confex.com/ams/89annual/techprogram/paper_143680.htm.]

    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., 2007: Uncertainty and inference for verification measures. Wea. Forecasting, 22, 637650.

  • Joslyn, S., , L. Nadav-Greenberg & , and R. M. Nichols, 2009: Probability of precipitation: Assessment and enhancement of end-user understanding. Bull. Amer. Meteor. Soc., 90, 185193.

    • Search Google Scholar
    • Export Citation
  • Mason, S. J. & , and N. E. Graham, 1999: Conditional probabilities, relative operating characteristics, and relative operating levels. Wea. Forecasting, 14, 713725.

    • Search Google Scholar
    • Export Citation
  • Morss, R. E., , J. L. Demuth & , and J. K. Lazo, 2008a: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974991.

    • Search Google Scholar
    • Export Citation
  • Morss, R. E., , J. K. Lazo, , B. G. Brown, , H. E. Brooks, , P. T. Ganderton & , and B. N. Mills, 2008b: Societal and economic research and applications for weather forecasts: Priorities for the North American THORPEX program. Bull. Amer. Meteor. Soc., 89, 335346.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H. & , and R. L. Winkler, 1974: Credible interval temperature forecasting: Some experimental results. Mon. Wea. Rev., 102, 784794.

    • Search Google Scholar
    • Export Citation
  • National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. The National Academies Press, 124 pp.

    • Search Google Scholar
    • Export Citation
  • Newman, S. B., 2003: Encouraging student involvement through use of basic probability forecasting games. Extended Abstracts, 12th Symp. on Education, Long Beach, CA, Amer. Meteor. Soc., P2.5. [Available online at https://ams.confex.com/ams/annual2003/techprogram/paper_48188.htm.]

    • Search Google Scholar
    • Export Citation
  • Norman, G., 2010: Likert scales, levels of measurement, and the “laws” of statistics. Adv. Health Sci. Educ., 15, 625632.

  • Roebber, P. J., , L. F. Bosart & , and G. S. Forbes, 1996: Does distance from the forecast site affect skill? Wea. Forecasting, 11, 582589.

    • Search Google Scholar
    • Export Citation
  • Sanders, F., 1986: Trends in skill of Boston forecasts made at MIT, 1966–84. Bull. Amer. Meteor. Soc., 67, 170176.

  • Wernli, H., , M. Paulat, , M. Hagan & , and C. Frei, 2008: SAL—A novel quality measure for the verification of quantitative precipitation forecasts. Mon. Wea. Rev., 136, 44704487.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences: An Introduction. Academic Press, 467 pp.

  • Winkler, R. L., 1972: A decision-theoretic approach to interval estimation. J. Amer. Stat. Assoc., 67, 187191.

  • Winkler, R. L. & , and A. H. Murphy, 1979: The use of probabilities in forecasts of maximum and minimum temperatures. Meteor. Mag., 108, 317329.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 48 48 4
PDF Downloads 27 27 4

Development and Analysis of a Probabilistic Forecasting Game for Meteorology Students

View More View Less
  • 1 Department of Environmental Sciences, Rutgers, The State University of New Jersey, New Brunswick, New Jersey
© Get Permissions
Restricted access

Calls for moving from a deterministic to a probabilistic view of weather forecasting have become increasingly urgent over recent decades, yet the primary national forecasting competition and many in-class forecasting games are wholly deterministic in nature. To counter these conflicting trends, a long-running forecasting game at Rutgers University has recently been modified to become probabilistic in nature. Students forecast high- and low-temperature intervals and probabilities of precipitation for two locations: one fixed at the Rutgers cooperative observing station, the other chosen for each forecast window to maximize difficulty. Precipitation errors are tabulated with a Brier score, while temperature errors contain a sharpness component dependent on the width of the forecast interval and an interval miss component dependent on the degree to which the verification falls within the interval.

The inclusion of a probabilistic forecasting game allows for the creation of a substantial database of forecasts that can be analyzed using standard probabilistic approaches, such as reliability diagrams, relative operating characteristic curves, and histograms. Discussions of probabilistic forecast quality can be quite abstract for undergraduate students, but the use of a forecast database that students themselves help construct motivates these discussions and helps students make connections between their forecast process, their standing in class rankings, and the verification diagrams they use. Student feedback on the probabilistic game is also discussed.

CORRESPONDING AUTHOR: Steven G. Decker, Dept. of Environmental Sciences, 14 College Farm Rd., New Brunswick, NJ 08901, E-mail: decker@envsci.rutgers.edu

Calls for moving from a deterministic to a probabilistic view of weather forecasting have become increasingly urgent over recent decades, yet the primary national forecasting competition and many in-class forecasting games are wholly deterministic in nature. To counter these conflicting trends, a long-running forecasting game at Rutgers University has recently been modified to become probabilistic in nature. Students forecast high- and low-temperature intervals and probabilities of precipitation for two locations: one fixed at the Rutgers cooperative observing station, the other chosen for each forecast window to maximize difficulty. Precipitation errors are tabulated with a Brier score, while temperature errors contain a sharpness component dependent on the width of the forecast interval and an interval miss component dependent on the degree to which the verification falls within the interval.

The inclusion of a probabilistic forecasting game allows for the creation of a substantial database of forecasts that can be analyzed using standard probabilistic approaches, such as reliability diagrams, relative operating characteristic curves, and histograms. Discussions of probabilistic forecast quality can be quite abstract for undergraduate students, but the use of a forecast database that students themselves help construct motivates these discussions and helps students make connections between their forecast process, their standing in class rankings, and the verification diagrams they use. Student feedback on the probabilistic game is also discussed.

CORRESPONDING AUTHOR: Steven G. Decker, Dept. of Environmental Sciences, 14 College Farm Rd., New Brunswick, NJ 08901, E-mail: decker@envsci.rutgers.edu
Save