Ensemble Calibration of 500-hPa Geopotential Height and 850-hPa and 2-m Temperatures Using Reforecasts

Thomas M. Hamill Physical Sciences Division, NOAA/Earth System Research Laboratory, Boulder, Colorado

Search for other papers by Thomas M. Hamill in
Current site
Google Scholar
PubMed
Close
and
Jeffrey S. Whitaker Physical Sciences Division, NOAA/Earth System Research Laboratory, Boulder, Colorado

Search for other papers by Jeffrey S. Whitaker in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

An examination of the benefits of ensemble forecast calibration was performed for three variables: 500-hPa geopotential height (Z500), 850-hPa temperature (T850), and 2-m temperature (T2M). A large reforecast dataset was used for the calibration. Two calibration methods were examined: a correction for a gross bias in the forecast and an analog method that implicitly adjusted for bias, spread, and applied a downscaling where appropriate. The characteristics of probabilistic forecasts from the raw ensemble were also considered. Forecasts were evaluated using rank histograms and the continuous ranked probability skill score. T2M rank histograms showed a high population of extreme ranks at all leads, and a correction for model bias alleviated this only slightly. The extreme ranks of Z500 rank histograms were slightly underpopulated at short leads, though slightly overpopulated at longer leads. T850 had characteristics in between those of T2M and Z500. Accordingly, Z500 was the most skillful variable without calibration and the variable least improved by calibration, and the bias correction achieved most of the improvement in skill. For T850, there was a more substantial additional increase in skill relative to the bias correction when the analog technique was applied. For T2M forecasts, probabilistic forecasts from the raw ensemble were the least skillful, the application of a bias correction substantially increased the skill, and the application of the analog technique produced the largest further increase in skill relative to the bias correction. Hence, reforecast datasets may be particularly helpful in the improvement of probabilistic forecasts of the variables that are most directly relevant to many forecast users (i.e., the sensible surface-weather variables).

Corresponding author address: Dr. Thomas M. Hamill, Physical Sciences Division, NOAA/Earth System Research Laboratory, R/PSD 1, 325 Broadway, Boulder, CO 80305. Email: tom.hamill@noaa.gov

Abstract

An examination of the benefits of ensemble forecast calibration was performed for three variables: 500-hPa geopotential height (Z500), 850-hPa temperature (T850), and 2-m temperature (T2M). A large reforecast dataset was used for the calibration. Two calibration methods were examined: a correction for a gross bias in the forecast and an analog method that implicitly adjusted for bias, spread, and applied a downscaling where appropriate. The characteristics of probabilistic forecasts from the raw ensemble were also considered. Forecasts were evaluated using rank histograms and the continuous ranked probability skill score. T2M rank histograms showed a high population of extreme ranks at all leads, and a correction for model bias alleviated this only slightly. The extreme ranks of Z500 rank histograms were slightly underpopulated at short leads, though slightly overpopulated at longer leads. T850 had characteristics in between those of T2M and Z500. Accordingly, Z500 was the most skillful variable without calibration and the variable least improved by calibration, and the bias correction achieved most of the improvement in skill. For T850, there was a more substantial additional increase in skill relative to the bias correction when the analog technique was applied. For T2M forecasts, probabilistic forecasts from the raw ensemble were the least skillful, the application of a bias correction substantially increased the skill, and the application of the analog technique produced the largest further increase in skill relative to the bias correction. Hence, reforecast datasets may be particularly helpful in the improvement of probabilistic forecasts of the variables that are most directly relevant to many forecast users (i.e., the sensible surface-weather variables).

Corresponding author address: Dr. Thomas M. Hamill, Physical Sciences Division, NOAA/Earth System Research Laboratory, R/PSD 1, 325 Broadway, Boulder, CO 80305. Email: tom.hamill@noaa.gov

Save
  • Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9 , 15181530.

    • Search Google Scholar
    • Export Citation
  • Eckel, F. A., and M. K. Walters, 1998: Calibrated probabilistic quantitative precipitation forecasts based on the MRF ensemble. Wea. Forecasting, 13 , 11321147.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., A. E. Raftery, A. H. Westveld III, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble Model Output Statistics and minimum CRPS estimation. Mon. Wea. Rev., 133 , 10981118.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129 , 550560.

  • Hamill, T. M., and S. J. Colucci, 1998: Evaluation of Eta-RSM ensemble probabilistic precipitation forecasts. Mon. Wea. Rev., 126 , 711724.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and J. Juras, 2006: Measuring forecast skill: Is it real skill or is it the varying climatology? Quart. J. Roy. Meteor. Soc., 132 , 29052923.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and J. S. Whitaker, 2006: Probabilistic quantitative precipitation forecasts based on reforecast analogs: Theory and application. Mon. Wea. Rev., 134 , 32093229.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. S. Whitaker, and X. Wei, 2004: Ensemble reforecasting: Improving medium-range forecast skill using retrospective forecasts. Mon. Wea. Rev., 132 , 14341447.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. S. Whitaker, and S. L. Mullen, 2006: Reforecasts: An important dataset for improving weather predictions. Bull. Amer. Meteor. Soc., 87 , 3346.

    • Search Google Scholar
    • Export Citation
  • Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15 , 559570.

    • Search Google Scholar
    • Export Citation
  • Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77 , 437471.

  • Liu, Z-Q., and F. Rabier, 2002: The interaction between model resolution, observation resolution and observation density in data assimilation: A one-dimensional study. Quart. J. Roy. Meteor. Soc., 128 , 13671386.

    • Search Google Scholar
    • Export Citation
  • Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133 , 11551174.

    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., and L. A. Smith, 2003: Combining dynamical and statistical ensembles. Tellus, 55A , 1630.

  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74 , 23172330.

  • Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125 , 32973319.

  • Wang, X., and C. H. Bishop, 2005: Improvement of ensemble reliability with a new dressing kernel. Quart. J. Roy. Meteor. Soc., 131 , 965986.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2006a: Comparison of ensemble-MOS methods in the Lorenz ’96 setting. Meteor. Appl., 13 , 243256.

  • Wilks, D. S., 2006b: Statistical Methods in the Atmospheric Sciences. 2d ed. Academic Press, 627 pp.

  • Wilks, D. S., and T. M. Hamill, 2007: Comparison of ensemble-MOS methods using GFS reforecasts. Mon. Wea. Rev., 135 , 23792390.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 391 138 11
PDF Downloads 319 107 11