Using Evolutionary Programs to Maximize Minimum Temperature Forecast Skill

Paul J. Roebber University of Wisconsin–Milwaukee, Milwaukee, Wisconsin

Search for other papers by Paul J. Roebber in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Evolutionary program ensembles are developed and tested for minimum temperature forecasts at Chicago, Illinois, at forecast ranges of 36, 60, 84, 108, 132, and 156 h. For all forecast ranges examined, the evolutionary program ensemble outperforms the 21-member GFS model output statistics (MOS) ensemble when considering root-mean-square error and Brier skill score. The relative advantage in root-mean-square error widens with forecast range, from 0.18°F at 36 h to 1.53°F at 156 h while the probabilistic skill remains positive throughout. At all forecast ranges, probabilistic forecasts of abnormal conditions are particularly skillful compared to the raw GFS guidance.

The evolutionary program reliance on particular forecast inputs is distinct from that obtained from considering multiple linear regression models, with less reliance on the GFS MOS temperature and more on alternative data such as upstream temperatures at the time of forecast issuance, time of year, and forecasts of wind speed, precipitation, and cloud cover. This weighting trends away from current observations and toward seasonal (climatological) measures as forecast range increases.

Using two different forms of ensemble member subselection, a Bayesian model combination calibration is tested on both ensembles. This calibration had limited effect on evolutionary program ensemble skill but was able to improve MOS ensemble performance, reducing but not eliminating the skill gap between them. The largest skill differentials occurred at the longest forecast ranges, beginning at 132 h. A hybrid, calibrated ensemble was able to provide some further increase in skill.

Corresponding author address: Paul J. Roebber, Atmospheric Science Group, Department of Mathematical Sciences and School of Freshwater Sciences, University of Wisconsin–Milwaukee, 3200 North Cramer Ave., Milwaukee, WI 53211. E-mail: roebber@uwm.edu

Abstract

Evolutionary program ensembles are developed and tested for minimum temperature forecasts at Chicago, Illinois, at forecast ranges of 36, 60, 84, 108, 132, and 156 h. For all forecast ranges examined, the evolutionary program ensemble outperforms the 21-member GFS model output statistics (MOS) ensemble when considering root-mean-square error and Brier skill score. The relative advantage in root-mean-square error widens with forecast range, from 0.18°F at 36 h to 1.53°F at 156 h while the probabilistic skill remains positive throughout. At all forecast ranges, probabilistic forecasts of abnormal conditions are particularly skillful compared to the raw GFS guidance.

The evolutionary program reliance on particular forecast inputs is distinct from that obtained from considering multiple linear regression models, with less reliance on the GFS MOS temperature and more on alternative data such as upstream temperatures at the time of forecast issuance, time of year, and forecasts of wind speed, precipitation, and cloud cover. This weighting trends away from current observations and toward seasonal (climatological) measures as forecast range increases.

Using two different forms of ensemble member subselection, a Bayesian model combination calibration is tested on both ensembles. This calibration had limited effect on evolutionary program ensemble skill but was able to improve MOS ensemble performance, reducing but not eliminating the skill gap between them. The largest skill differentials occurred at the longest forecast ranges, beginning at 132 h. A hybrid, calibrated ensemble was able to provide some further increase in skill.

Corresponding author address: Paul J. Roebber, Atmospheric Science Group, Department of Mathematical Sciences and School of Freshwater Sciences, University of Wisconsin–Milwaukee, 3200 North Cramer Ave., Milwaukee, WI 53211. E-mail: roebber@uwm.edu
Save
  • Bakhshaii, A., and R. Stull, 2009: Deterministic ensemble forecasts using gene-expression programming. Wea. Forecasting, 24, 14311451, doi:10.1175/2009WAF2222192.1.

    • Search Google Scholar
    • Export Citation
  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 13, doi:10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., 1997: Potential forecast skill of prediction and spread and skill distributions of the ECMWF prediction system. Mon. Wea. Rev., 125, 99119, doi:10.1175/1520-0493(1997)125<0099:PFSOEP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., M. Milleer, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 28872908, doi:10.1002/qj.49712556006.

    • Search Google Scholar
    • Export Citation
  • Cooksey, R. W., 1996: Judgment Analysis: Theory, Methods and Applications. Academic Press, 407 pp.

  • Coulibaly, P., 2004: Downscaling daily extreme temperatures with genetic programming. Geophys. Res. Lett., 31, L16203, doi:10.1029/2004GL020075.

    • Search Google Scholar
    • Export Citation
  • Cui, B., Z. Toth, Y. Zhu, and D. Hou, 2012: Bias correction for global ensemble forecast. Wea. Forecasting, 27, 396410, doi:10.1175/WAF-D-11-00011.1.

    • Search Google Scholar
    • Export Citation
  • Fogel, L. J., 1999: Intelligence through Simulated Evolution: Forty Years of Evolutionary Programming. John Wiley, 162 pp.

  • Hamill, T. M., and J. S. Whitaker, 2007: Ensemble calibration of 500-hPa geopotential height and 850-hPa and 2-m temperatures using reforecasts. Mon. Wea. Rev., 135, 32733280, doi:10.1175/MWR3468.1.

    • Search Google Scholar
    • Export Citation
  • Harrison, M. S. J., T. N. Palmer, D. S. Richardson, and R. Buizza, 1999: Analysis and model dependencies in medium-range: Two transplant case studies. Quart. J. Roy. Meteor. Soc., 125, 24872516, doi:10.1002/qj.49712555908.

    • Search Google Scholar
    • Export Citation
  • Haupt, S. E., G. S. Young, and C. T. Allen, 2006: Validation of a receptor–dispersion model coupled with a genetic algorithm using synthetic data. J. Appl. Meteor. Climatol., 45, 476490, doi:10.1175/JAM2359.1.

    • Search Google Scholar
    • Export Citation
  • Hillis, W. D., 1990: Co-evolving parasites improve simulated evolution as an optimization procedure. Physica D, 42, 228234, doi:10.1016/0167-2789(90)90076-2.

    • Search Google Scholar
    • Export Citation
  • Hoeting, J. A., D. Madigan, A. E. Raftery, and C. T. Volinsky, 1999: Bayesian model averaging: A tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors). Stat. Sci., 14, 382417, doi:10.1214/ss/1009212519.

    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., 2000: Using a genetic algorithm to tune a bounded weak echo region detection algorithm. J. Appl. Meteor., 39, 222230, doi:10.1175/1520-0450(2000)039<0222:UAGATT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130141, doi:10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.

  • Monteith, K., J. Carroll, K. Seppi, and T. Martinez, 2011: Turning Bayesian model averaging into Bayesian model combination. Proc. Int. Joint Conf. on Neural Networks IJCNN'11, San Jose, CA, IEEE, 26572663, doi:10.1109/IJCNN.2011.6033566.

  • Novak, D. R., D. R. Bright, and M. J. Brennan, 2008: Operational forecaster uncertainty needs and future roles. Wea. Forecasting, 23, 10691084, doi:10.1175/2008WAF2222142.1.

    • Search Google Scholar
    • Export Citation
  • O’Steen, L., and D. Werth, 2009: The application of an evolutionary algorithm to the optimization of a mesoscale meteorological model. J. Appl. Meteor. Climatol., 48, 317329, doi:10.1175/2008JAMC1967.1.

    • Search Google Scholar
    • Export Citation
  • Orrell, D., 2005: Ensemble forecasting in a system with model error. J. Atmos. Sci., 62, 16521659, doi:10.1175/JAS3406.1.

  • Roebber, P. J., 1998: The regime dependence of degree day forecast technique, skill and value. Wea. Forecasting, 13, 783794, doi:10.1175/1520-0434(1998)013<0783:TRDODD>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., 2010: Seeking consensus: A new approach. Mon. Wea. Rev., 138, 44024415, doi:10.1175/2010MWR3508.1.

  • Roebber, P. J., 2013: Using evolutionary programming to generate skillful extreme value probabilistic forecasts. Mon. Wea. Rev., 141, 31703185, doi:10.1175/MWR-D-12-00285.1.

    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., 2015: Evolving ensembles. Mon. Wea. Rev., 143, 471–490, doi:10.1175/MWR-D-14-00058.1.

  • Shutts, G. J., and T. N. Palmer, 2007: Convective forcing fluctuations in a cloud-resolving model: Relevance to the stochastic parameterization problem. J. Climate, 20, 187202, doi:10.1175/JCLI3954.1.

    • Search Google Scholar
    • Export Citation
  • Stensrud, D. J., J.-W. Bao, and T. T. Warner, 2000: Using initial condition and model physics perturbations in short-range ensemble simulations of mesoscale convective systems. Mon. Wea. Rev., 128, 20772107, doi:10.1175/1520-0493(2000)128<2077:UICAMP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Teisberg, T. J., R. F. Weiher, and A. Khotanzad, 2005: The economic value of temperature forecasts in electricity generation. Bull. Amer. Meteor. Soc., 86, 17651771, doi:10.1175/BAMS-86-12-1765.

    • Search Google Scholar
    • Export Citation
  • Yang, H.-T., C. M. Huang, and C. L. Huang, 1996: Identification of ARMAX model for short-term load forecasting: An evolutionary programming approach. IEEE Trans. Power Syst., 11, 403408, doi:10.1109/59.486125.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1261 1047 89
PDF Downloads 123 30 2