This paper was funded in part by a grant/cooperative agreement from the National Oceanic and Atmospheric Administration (NOAA), Contract NA17RJ1231, with the University of California, San Diego, and Contract NA050AR4311004 with the trustees of Columbia University. The views expressed herein are those of the authors and do not necessarily reflect the views of NOAA or any of its subagencies. The anonymous comments of referees, and helpful discussions with A. G. Barnston, L. Clarke, I. T. Jolliffe, G. V. Kass, B. Rajagopalan, L. A. Smith, R. L. Smith, U. Lall, and D. S. Wilks are gratefully acknowledged.
Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9 , 1518–1530.
Atger, F., 2004: Estimation of the reliability of ensemble-based probabilistic forecasts. Quart. J. Roy. Meteor. Soc., 130 , 627–646.
Dix, M. R., and B. G. Hunt, 1995: Chaotic influences and the problem of deterministic seasonal predictions. Int. J. Climatol., 15 , 729–752.
Elmore, K. L., 2005: Alternatives to the chi-squared test for evaluating rank histograms from ensemble forecasts. Wea. Forecasting, 20 , 789–795.
Gneiting, T., A. E. Raftery, A. H. Westveld, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133 , 1098–1113.
Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15 , 559–570.
Hsu, W-R., and A. H. Murphy, 1986: The attributes diagram: A geometrical frame work for assessing the quality of probability forecasts. Int. J. Forecasting, 2 , 285–293.
Mason, S. J., 2004: On using “climatology” as a reference strategy in the Brier and ranked probability skill scores. Mon. Wea. Rev., 137 , 1891–1895.
Mason, S. J., and N. E. Graham, 1999: Conditional probabilities, relative operating characteristics and relative operating levels. Wea. Forecasting, 14 , 713–725.
Mason, S. J., and G. M. Mimmack, 2002: A comparison of statistical methods of probabilistic seasonal climate forecasting. J. Climate, 15 , 8–29.
Mitchell, T. D., T. R. Carter, P. D. Jones, M. Hulme, and M. New, 2003: A comprehensive set of high-resolution grids of monthly climate for Europe and the globe: The observed record (1901–2000) and 16 scenarios (2001–2100). Working Paper 55, Tyndall Centre for Climate Change Research, Norwich, United Kingdom, 30 pp.
Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8 , 281–293.
Murphy, A. H., 1997: Forecast verification. Economic Value of Weather and Climate Forecasts, R. W. Katz and A. H. Murphy, Eds., Cambridge University Press, 19–74.
Murphy, A. H., and D. S. Wilks, 1998: A case study for the use of statistical models in forecast verification: Precipitation probability forecasts. Wea. Forecasting, 13 , 795–810.
Roeckner, E., and Coauthors, 1996: The atmospheric circulation model ECHAM-4: Model description and simulation of present-day climate. MPI-Rep. 218, Max-Planck-Institut für Meteorologie, Hamburg, Germany, 90 pp.
Rowell, D. P., 1998: Assessing potential seasonal predictability using an ensemble of multidecadal GCM simulations. J. Climate, 11 , 109–120.
Seillier-Moiseiwitsch, F., and A. P. Dawid, 1993: On testing the validity of sequential probability forecasts. J. Amer. Stat. Assoc., 88 , 355–359.
Talagrand, O., R. Vautard, and B. Strauss, 1998: Evaluation of probabilistic prediction systems. Proc. ECMWF Workshop on Predictability, Reading, United Kingdom, ECMWF, 17–28.
Teigen, K. H., and M. Jørgensen, 2005: When 90% confidence intervals are 50% certain: On the credibility of credible intervals. Appl. Cognit. Psychol., 19 , 455–475.
Toth, Z., O. Talagrand, G. Candille, and Y. Zhu, 2003: Probability and ensemble forecasts. Forecast Verification: A Practitioner’s Guide in Atmospheric Science, I. Jolliffe and D. Stephenson, Eds., Wiley, 137–163.