The Use of Ensembles to Identify Forecasts with Small and Large Uncertainty

Zoltan Toth National Centers for Environmental Prediction, Camp Springs, Maryland

Search for other papers by Zoltan Toth in
Current site
Google Scholar
PubMed
Close
,
Yuejian Zhu National Centers for Environmental Prediction, Camp Springs, Maryland

Search for other papers by Yuejian Zhu in
Current site
Google Scholar
PubMed
Close
, and
Timothy Marchok National Centers for Environmental Prediction, Camp Springs, Maryland

Search for other papers by Timothy Marchok in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

In the past decade ensemble forecasting has developed into an integral part of numerical weather prediction. Flow-dependent forecast probability distributions can be readily generated from an ensemble, allowing for the identification of forecast cases with high and low uncertainty. The ability of the NCEP ensemble to distinguish between high and low uncertainty forecast cases is studied here quantitatively. Ensemble mode forecasts, along with traditional higher-resolution control forecasts, are verified in terms of predicting the probability of the true state being in 1 of 10 climatologically equally likely 500-hPa height intervals. A stratification of the forecast cases by the degree of overall agreement among the ensemble members reveals great differences in forecast performance between the cases identified by the ensemble as the least and most uncertain. A new ensemble-based forecast product, the “relative measure of predictability,” is introduced to identify forecasts with below and above average uncertainty. This measure is standardized according to geographical location, the phase of the annual cycle, lead time, and also the position of the forecast value in terms of the climatological frequency distribution. The potential benefits of using this and other ensemble-based measures of predictability is demonstrated through synoptic examples.

* Additional affiliation: General Sciences Corporation, Beltsville, Maryland.

Corresponding author address: Zoltan Toth, Environmental Modeling Center, NCEP, 5200 Auth Rd., Room 207, Camp Springs, MD 20746.Email: Zoltan.Toth@noaa.gov

Abstract

In the past decade ensemble forecasting has developed into an integral part of numerical weather prediction. Flow-dependent forecast probability distributions can be readily generated from an ensemble, allowing for the identification of forecast cases with high and low uncertainty. The ability of the NCEP ensemble to distinguish between high and low uncertainty forecast cases is studied here quantitatively. Ensemble mode forecasts, along with traditional higher-resolution control forecasts, are verified in terms of predicting the probability of the true state being in 1 of 10 climatologically equally likely 500-hPa height intervals. A stratification of the forecast cases by the degree of overall agreement among the ensemble members reveals great differences in forecast performance between the cases identified by the ensemble as the least and most uncertain. A new ensemble-based forecast product, the “relative measure of predictability,” is introduced to identify forecasts with below and above average uncertainty. This measure is standardized according to geographical location, the phase of the annual cycle, lead time, and also the position of the forecast value in terms of the climatological frequency distribution. The potential benefits of using this and other ensemble-based measures of predictability is demonstrated through synoptic examples.

* Additional affiliation: General Sciences Corporation, Beltsville, Maryland.

Corresponding author address: Zoltan Toth, Environmental Modeling Center, NCEP, 5200 Auth Rd., Room 207, Camp Springs, MD 20746.Email: Zoltan.Toth@noaa.gov

Save
  • Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9 , 15181530.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Atger, F., 1999: The skill of ensemble prediction systems. Mon. Wea. Rev, 127 , 19411953.

  • Atger, F., 2001: Verification of intense precipitation forecasts from single models and ensemble prediction systems. Nonlinear Processes Geophys., in press.

    • Search Google Scholar
    • Export Citation
  • Eckel, F. A., and Walters M. K. , 1998: Calibrated probabilistic quantitative precipitation forecasts based on the MRF ensemble. Wea. Forecasting, 13 , 11321147.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ehrendorfer, M., 1997: Predicting the uncertainty of numerical weather forecasts: A review. Meteor. Z, 6 , 147183.

  • Hamill, T. M., and Colucci S. J. , 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev, 125 , 711724.

  • Houtekamer, P. L., 1993: Global and local skill forecasts. Mon. Wea. Rev, 121 , 18341846.

  • Houtekamer, P. L., Lefaivre L. , Derome J. , Ritchie H. , and Mitchell H. L. , 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev, 124 , 12251242.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalnay, E., and Coauthors. 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc, 77 , 437471.

  • Kobayashi, C., Yoshimatsu K. , Maeda S. , and Takano K. , 1996: Dynamical one-month forecasting at JMA. Preprints, 11th Conf. on Numerical Weather Prediction, Norfolk, VA, Amer. Meteor. Soc., 13–14.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., Buizza R. , Palmer T. N. , and Petroliagis T. , 1996: The ECMWF ensemble system: Methodology and validation. Quart. J. Roy. Meteor. Soc, 122 , 73119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mylne, K. R., 1999: The use of forecast value calculations for optimal decision making using probability forecasts. Preprints, 17th Conf. on Weather Analysis and Forecasting, Denver, CO, Amer. Meteor. Soc., 235–239.

    • Search Google Scholar
    • Export Citation
  • NWS, 1999: Vision 2005: National Weather Service Strategic Plan for Weather, Water, and Climate Services 2000–2005. 24 pp. [Available from NWS, 1315 East–West Highway, Silver Spring, MD, 20910-3282.].

    • Search Google Scholar
    • Export Citation
  • Rennick, M. A., 1995: The ensemble forecast system (EFS). Models Department Tech. Note 2-95, 19 pp. [Available from Models Department, Fleet Numerical Meteorology and Oceanography Center, 7 Grace Hopper Ave., Monterey, CA 93943.].

    • Search Google Scholar
    • Export Citation
  • Richardson, D. S., 2000: Skill and economic value of the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc, 126 , 649668.

  • Stanski, H. R., Wilson L. J. , and Burrows W. R. , 1989: Survey of common verification methods in meteorology. WMO World Weather Watch Tech. Rep. 8, WMO/TD. No. 358, 114 pp.

    • Search Google Scholar
    • Export Citation
  • Talagrand, O., Vautard R. , and Strauss B. , 1997: Evaluation of probabilistic prediction systems. Proc. Workshop on Predictability, Reading, United Kingdom, ECMWF, 1–25.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., 1992: Quasi-stationary and transient periods in the Northern Hemisphere circulation series. J Climate, 5 , 12351247.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., and Kalnay E. , 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc, 74 , 23172330.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., 1995: Ensemble forecasting at NCEP. Proc. Seminar on Predictability, Reading, United Kingdom, ECMWF, 39–60.

  • Toth, Z., 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev, 125 , 32973319.

  • Toth, Z., Zhu Y. , Marchok T. , Tracton S. , and Kalnay E. , 1998: Verification of the NCEP global ensemble forecasts. Preprints, 12th Conf. on Numerical Weather Prediction, Phoenix, AZ, Amer. Meteor. Soc., 286–289.

    • Search Google Scholar
    • Export Citation
  • van den Dool, H. M., and Toth Z. , 1991: Why do forecasts for “near normal” often fail? Wea. Forecasting, 6 , 7685.

  • Whitaker, J. S., and Loughe A. F. , 1998: The relationship between ensemble spread and ensemble mean skill. Mon. Wea. Rev, 126 , 32923302.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 467 pp.

  • Zhu, Y., Iyengar G. , Toth Z. , Tracton M. S. , and Marchok T. , 1996: Objective evaluation of the NCEP global ensemble forecasting system. Preprints, 15th Conf. on Weather Analysis and Forecasting, Norfolk, VA, Amer. Meteor. Soc., J79–J82.

    • Search Google Scholar
    • Export Citation
  • Zhu, Y., Toth Z. , and Wobus R. , 2001: On the economic value of ensemble based weather forecasts. Bull. Amer. Meteor. Soc., in press.

  • Ziehmann, C., 2001: Skill prediction of local weather forecasts based on the ECMWF ensemble. Nonlinear Processes Geophys., in press.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 537 101 16
PDF Downloads 271 81 14