• Barnston, A. G., Li S. , Mason S. J. , DeWitt D. G. , Goddard L. , and Gong X. , 2010: Verification of the first 11 years of IRI's seasonal climate forecasts. J. Appl. Climatol. Meteor., 49, 493520.

    • Search Google Scholar
    • Export Citation
  • Epstein, E. S., and Murphy A. H. , 1965: A note on the attributes of probabilistic predictions and the probability score. J. Appl. Meteor., 4, 297299.

    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., and Stephenson D. B. , 2012: Introduction. Forecast Verification: A Practitioner's Guide in Atmospheric Sciences, I. T. Jolliffe and D. B. Stephenson, Eds., Wiley-Blackwell, 1–9.

  • Jupp, T. E., Lowe R. , Coelho C. A. S. , and Stephenson D. B. , 2012: On the visualization, verification and recalibration of ternary probabilistic forecasts. Philos. Trans. Roy. Soc. London, A370, 11001120.

    • Search Google Scholar
    • Export Citation
  • Katz, R. W., and Murphy A. H. , Eds., 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, 222 pp.

  • Livezey, R. E., and Timofeyeva M. M. , 2008: The first decade of long-lead U.S. seasonal forecasts—Insights from a skill analysis. Bull. Amer. Meteor. Soc., 89, 843854.

    • Search Google Scholar
    • Export Citation
  • Livezey, R. E., Vinnikov K. Y. , Timofeyeva M. M. , Tinker R. , and Van den Dool H. M. , 2007: Estimation and extrapolation of climate normals and climatic trends. J. Appl. Meteor. Climatol., 46, 17591776.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1972: Scalar and vector partitions of the probability score: Part II. N-state situation. J. Appl. Meteor., 11, 11831192.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1991: Forecast verification: Its complexity and dimensionality. Mon. Wea. Rev., 119, 15901601.

  • Murphy, A. H., 1997: Forecast verification. Economic Value of Weather and Climate Forecasts, R.W. Katz and A.H. Murphy, Eds., Cambridge University Press, 19–74.

  • Murphy, A. H., and Winkler R. L. , 1977: Reliability of subjective probability forecasts of precipitation and temperature. Appl. Stat., 26, 4147.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and Daan H. , 1985: Forecast evaluation. Probability, Statistics, and Decision Making in the Atmospheric Sciences, A. H. Murphy and R. W. Katz, Eds., Westview, 379–437.

  • Murphy, A. H., and Hsu W.-R. , 1986: The attributes diagram: A geometrical framework for assessing the quality of probability forecasts. Int. J. Forecasting, 2, 285293.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., and Winkler R. L. , 1987: A general framework for forecast verification. Mon. Wea. Rev., 115, 13301338.

  • Murphy, A. H., and Winkler R. L. , 1992: Diagnostic verification of probability forecasts. Int. J. Forecasting, 7, 435455.

  • Murphy, A. H., Brown B. G. , and Chen Y.-S. , 1989: Diagnostic verification of temperature forecasts. Wea. Forecasting, 4, 485501.

  • Van den Dool, H., 2007: Empirical Methods in Short-Term Climate Prediction. Oxford University Press, 215 pp.

  • Van den Dool, H., and Toth Z. , 1991: Why do forecasts for “near normal” often fail? Wea. Forecasting, 6, 7685.

  • Wilks, D. S., 2000: Diagnostic verification of the Climate Prediction Center long-lead outlooks, 1995–98. J. Climate, 13, 23892403.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2001: A skill score based on economic value for probability forecasts. Meteor. Appl., 8, 209219.

  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Academic Press, 676 pp.

  • Wilks, D. S., 2013: Projecting “normals” in a nonstationary climate. J. Appl. Meteor. Climatol., 52, 289302.

  • Wilks, D. S., and Godfrey C. M. , 2002: Diagnostic verification of the IRI net assessment forecasts, 1997–2000. J. Climate, 15, 13691377.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 201 96 6
PDF Downloads 155 88 6

The Calibration Simplex: A Generalization of the Reliability Diagram for Three-Category Probability Forecasts

Daniel S. WilksDepartment of Earth and Atmospheric Sciences, Cornell University, Ithaca, New York

Search for other papers by Daniel S. Wilks in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Full exposition of the performance of a set of forecasts requires examination of the joint frequency distribution of those forecasts and their corresponding observations. In settings involving probability forecasts, this joint distribution has a high dimensionality, and communication of its information content is often best achieved graphically. This paper describes an extension of the well-known reliability diagram, which displays the joint distribution for probability forecasts of dichotomous events, to the case of probability forecasts for three disjoint events, such as “below,” “near,” and “above normal.” The resulting diagram, called the calibration simplex, involves a discretization of the 2-simplex, which is an equilateral triangle. Characteristics and interpretation of the calibration simplex are illustrated using both idealized verification datasets, and the 6–10- and 8–14-day temperature and precipitation forecasts produced by the U.S. Climate Prediction Center.

Corresponding author address: Daniel S. Wilks, Dept. of Earth and Atmospheric Sciences, Bradfield Hall, Rm. 1113, Cornell University, Ithaca, NY 14853. E-mail: dsw5@cornell.edu

Abstract

Full exposition of the performance of a set of forecasts requires examination of the joint frequency distribution of those forecasts and their corresponding observations. In settings involving probability forecasts, this joint distribution has a high dimensionality, and communication of its information content is often best achieved graphically. This paper describes an extension of the well-known reliability diagram, which displays the joint distribution for probability forecasts of dichotomous events, to the case of probability forecasts for three disjoint events, such as “below,” “near,” and “above normal.” The resulting diagram, called the calibration simplex, involves a discretization of the 2-simplex, which is an equilateral triangle. Characteristics and interpretation of the calibration simplex are illustrated using both idealized verification datasets, and the 6–10- and 8–14-day temperature and precipitation forecasts produced by the U.S. Climate Prediction Center.

Corresponding author address: Daniel S. Wilks, Dept. of Earth and Atmospheric Sciences, Bradfield Hall, Rm. 1113, Cornell University, Ithaca, NY 14853. E-mail: dsw5@cornell.edu
Save