• Arpe, K., A. Hollingsworth, M. S. Tracton, A. C. Lorenc, S. Uppala, and P. Kallberg, 1985: The response of numerical weather prediction systems to FGGE level IIb data. Part II: Forecast verifications and implications for predictability. Quart. J. Roy. Meteor. Soc.,111, 67–101.

    • Crossref
    • Export Citation
  • Brier, G. W., and R. A. Allen, 1951: Verification of weather forecasts. Compendium of Meteorology, T. Malone, Ed., Amer. Meteor. Soc., 841–848.

    • Crossref
    • Export Citation
  • Briggs, W. M., and R. A. Levine, 1997: Wavelets and field forecast verification. Mon. Wea. Rev.,125, 1329–1341.

    • Crossref
    • Export Citation
  • Buizza, R., 1997: Potential forecast skill of ensemble prediction and spread and skill distributions of the ECMWF ensemble prediction system. Mon. Wea. Rev.,125, 99–119.

    • Crossref
    • Export Citation
  • Colby, F. P., Jr., 1984: Convective inhibition as a predictor of convection during AVE-SESAME II. Mon. Wea. Rev.,112, 2239–2252.

    • Crossref
    • Export Citation
  • Donaldson, R. J., R. M. Dyer, and R. M. Kraus, 1975: An objective evaluator of techniques for predicting severe weather events. Preprints, Ninth Conf. on Severe Local Storms, Norman, OK, Amer. Meteor. Soc., 321–326.

  • Du, J., S. L. Mullen, and F. Sanders, 1997: Short-range ensemble forecasting of quantitative precipitation. Mon. Wea. Rew.,125, 2427–2459.

    • Crossref
    • Export Citation
  • Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev.,125, 1312–1327.

    • Crossref
    • Export Citation
  • ——, and ——, 1998: Evaluation of Eta–RSM ensemble probabilistic precipitation forecasts. Mon. Wea. Rev.,126, 711–724.

    • Crossref
    • Export Citation
  • Hollingsworth, A., K. Arpe, M. Tiedtke, M. Capaldo, and H. Savijaervi, 1980: The performance of a medium range forecast model in winter—Impact of physical parameterizations. Mon. Wea. Rev.,108, 1736–1773.

    • Crossref
    • Export Citation
  • Houtekamer, P. L., and L. Lefaivre, 1997: Using ensemble forecasts for model verification. Mon. Wea. Rev.,125, 2416–2426.

    • Crossref
    • Export Citation
  • Janjić, Z. I., 1994: The step-mountain Eta coordinate model: Further developments of the convection, viscous sublayer, and turbulence closure schemes. Mon. Wea. Rev.,122, 927–945.

    • Crossref
    • Export Citation
  • Juang, H.-M., and M. Kanamitsu, 1994: The NMC nested regional spectral model. Mon. Wea. Rev.,122, 3–26.

    • Crossref
    • Export Citation
  • Kalnay, E., M. Kanamitsu, and W. E. Baker, 1990: Global numerical weather prediction at NMC. Bull. Amer. Meteor. Soc.,71, 1410–1428.

    • Crossref
    • Export Citation
  • Lorenz, E. N., 1965: A study of the predictability of a 28-variable atmospheric model. Tellus,17, 321–333.

    • Crossref
    • Export Citation
  • Mason, I., 1982: A model for assessment of weather forecasts. Aust. Meteor. Mag.,30, 291–303.

  • ——, 1989: Dependence of the critical success index on sample climate and threshold probability. Aust. Meteor. Mag.,37, 75–81.

  • Miyakoda, K., G. D. Hembree, R. F. Strickler, and I. Shulman, 1972:Cumulative results of extended forecast experiments. Part I: Model performance for winter cases. Mon. Wea. Rev.,100, 836–855.

    • Crossref
    • Export Citation
  • Molteni, R., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc.,122, 73–119.

    • Crossref
    • Export Citation
  • Mullen, S. L., and D. P. Baumhefner, 1988: Sensitivity to numerical simulations of explosive oceanic cyclogenesis to changes in physical parameterizations. Mon. Wea. Rev.,116, 2289–2329.

    • Crossref
    • Export Citation
  • ——, and ——, 1989: The impact of initial condition uncertainty on numerical simulations of large-scale explosive cyclogenesis. Mon. Wea. Rev.,117, 2800–2821.

    • Crossref
    • Export Citation
  • Murphy, A. H., 1988: Skill scores based on the mean square error and their relationships to the correlation coefficient. Mon. Wea. Rev.,116, 2417–2424.

    • Crossref
    • Export Citation
  • ——, 1995: The coefficients of correlation and determination as measures of performance in forecast verification. Wea. Forecasting,10, 681–688.

  • ——, and H. Daan, 1985: Forecast evaluation. Probability, Statistics, and Decision Making in the Atmospheric Sciences, A. H. Murphy and R. W. Katz, Eds., Westview Press, 379–437.

  • ——, and R. L. Winkler, 1987: A general framework for forecast verification. Mon. Wea. Rev.,115, 1330–1338.

    • Crossref
    • Export Citation
  • ——, and E. S. Epstein, 1989: Skill scores and correlation coefficients in model verification. Mon. Wea. Rev.,117, 572–581.

    • Crossref
    • Export Citation
  • Panofsky, H. A., and G. W. Brier, 1958: Some Applications of Statistics to Meteorology. The Pennsylvania State University Press, 224 pp.

  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting,5, 570–575.

    • Crossref
    • Export Citation
  • Stensrud, D. J., H. E. Brooks, J. Du, M. S. Tracton, and E. Rogers, 1999: Using ensembles for short-range forecasting. Mon. Wea. Rev.,127, 433–446.

    • Crossref
    • Export Citation
  • Takacs, L. L., 1985: A two-step scheme for the advection equation with minimized dissipation and dispersion errors. Mon. Wea. Rev.,113, 1050–1065.

    • Crossref
    • Export Citation
  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc.,74, 2317–2330.

    • Crossref
    • Export Citation
  • Tracton, S., and E. Kalnay, 1993: Ensemble forecasting at NMC: Operational implementation. Wea. Forecasting,8, 379–398.

  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences:An Introduction. Academic Press, 467 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 36 36 5
PDF Downloads 20 20 3

The Correspondence Ratio in Forecast Evaluation

View More View Less
  • 1 NOAA/National Severe Storms Laboratory, Norman, Oklahoma
  • | 2 The University of Arizona, Tucson, Arizona
© Get Permissions Rent on DeepDyve
Restricted access

Abstract

The correspondence ratio is developed to evaluate output from an ensemble of numerical weather prediction models. This measure is a simple extension of the threat score, or critical success index, to more than two fields and is used to measure the divergence of the forecast fields. The ratio is compared with two commonly used measures: the anomaly correlation, and the mean square error. Results indicate that the correspondence ratio is sensitive to the bias and, when calculated for several threshold values, can provide information beyond that supplied by the mean-square error and anomaly correlation measures. The correspondence ratio is particularly useful in evaluating discontinuous fields, such as precipitation. While no one measure can provide a complete assessment of forecast success, this ratio provides useful information that can increase our understanding of model forecast quality.

Corresponding author address: Dr. David J. Stensrud, National Severe Storms Laboratory, 1313 Halley Circle, Norman, OK 73069.

Email: David.Stensrud@nssl.noaa.gov

Abstract

The correspondence ratio is developed to evaluate output from an ensemble of numerical weather prediction models. This measure is a simple extension of the threat score, or critical success index, to more than two fields and is used to measure the divergence of the forecast fields. The ratio is compared with two commonly used measures: the anomaly correlation, and the mean square error. Results indicate that the correspondence ratio is sensitive to the bias and, when calculated for several threshold values, can provide information beyond that supplied by the mean-square error and anomaly correlation measures. The correspondence ratio is particularly useful in evaluating discontinuous fields, such as precipitation. While no one measure can provide a complete assessment of forecast success, this ratio provides useful information that can increase our understanding of model forecast quality.

Corresponding author address: Dr. David J. Stensrud, National Severe Storms Laboratory, 1313 Halley Circle, Norman, OK 73069.

Email: David.Stensrud@nssl.noaa.gov

Save