Using Evolutionary Programming to Add Deterministic and Probabilistic Skill to Spatial Model Forecasts

Paul J. Roebber University of Wisconsin–Milwaukee, Milwaukee, Wisconsin

Search for other papers by Paul J. Roebber in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Evolutionary programming is applied to the postpocessing of ensemble forecasts of temperature on a spatial domain. These forecasts are obtained from the 11-member Reforecast V2 ensemble over the region from 24°–53°N to 125°–66°W for the period 1 January 1985–14 May 2011. The evolution is based upon a static ecosystem model that holds constant the number of individuals (algorithms), using a fixed rate of introduction of new algorithms and removal of existing algorithms. Each algorithm adheres to a specific underlying genetic architecture, and the selection pressure on the “species” is according to deterministic performance (root-mean-square error) on a training dataset. On a 2325-case, independent test dataset, the method improved root-mean-square error and ranked probability score relative to the Reforecast ensemble by 0.31°F (8.7%) and 3.3%, respectively, across the domain, with 96% of the grid points showing simultaneous improvements in both measures. The use of input information by the evolutionary programming algorithms varied by region; while the algorithm forecasts at all locations are fundamentally tied to the Reforecast ensemble forecast, northeastern locations found snow cover to be the next most useful input, whereas southwestern locations preferentially employed precipitable water. An adaptive form of the approach, developed to be readily implemented into operations, is tested in the absence of improving inputs but is found to only slightly degrade performance (1.2% in root-mean-square error and 0.6% in ranked probability skill score). A number of future extensions are discussed.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Paul J. Roebber, roebber@uwm.edu

Abstract

Evolutionary programming is applied to the postpocessing of ensemble forecasts of temperature on a spatial domain. These forecasts are obtained from the 11-member Reforecast V2 ensemble over the region from 24°–53°N to 125°–66°W for the period 1 January 1985–14 May 2011. The evolution is based upon a static ecosystem model that holds constant the number of individuals (algorithms), using a fixed rate of introduction of new algorithms and removal of existing algorithms. Each algorithm adheres to a specific underlying genetic architecture, and the selection pressure on the “species” is according to deterministic performance (root-mean-square error) on a training dataset. On a 2325-case, independent test dataset, the method improved root-mean-square error and ranked probability score relative to the Reforecast ensemble by 0.31°F (8.7%) and 3.3%, respectively, across the domain, with 96% of the grid points showing simultaneous improvements in both measures. The use of input information by the evolutionary programming algorithms varied by region; while the algorithm forecasts at all locations are fundamentally tied to the Reforecast ensemble forecast, northeastern locations found snow cover to be the next most useful input, whereas southwestern locations preferentially employed precipitable water. An adaptive form of the approach, developed to be readily implemented into operations, is tested in the absence of improving inputs but is found to only slightly degrade performance (1.2% in root-mean-square error and 0.6% in ranked probability skill score). A number of future extensions are discussed.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Paul J. Roebber, roebber@uwm.edu
Save
  • Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9, 15181530, https://doi.org/10.1175/1520-0442(1996)009<1518:AMFPAE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bakhshaii, A., and R. Stull, 2009: Deterministic ensemble forecasts using gene- expression programming. Wea. Forecasting, 24, 14311451, https://doi.org/10.1175/2009WAF2222192.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Basak, A., S. Pal, S. Das, A. Abraham, and V. Snasel, 2010: A modified Invasive Weed Optimization algorithm for time-modulated linear antenna array synthesis. Proc. 2010 IEEE Congress on Evolutionary Computation (CEC), Barcelona, Spain, IEEE, https://doi.org/10.1109/CEC.2010.5586276.

    • Crossref
    • Export Citation
  • Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The rapid refresh. Mon. Wea. Rev., 144, 16691694, https://doi.org/10.1175/MWR-D-15-0242.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Biswas, R., C. Ofria, D. M. Bryson, and A. P. Wagner, 2014: Causes vs benefits in the evolution of prey grouping. Proc. ALIFE 14: 14th Int. Conf. on the Synthesis and Simulation of Living Systems, New York, NY, International Society for Artificial Life, 641–648.

    • Crossref
    • Export Citation
  • Buizza, R., P. Houtekamer, G. Pellerin, Z. Toth, Y. Zhu, and M. Wei, 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133, 10761097, https://doi.org/10.1175/MWR2905.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carja, O., U. Lieberman, and M. W. Feldman, 2014: Evolution in changing environments: Modifiers of mutation, recombination, and migration. Proc. Natl. Acad. Sci. USA, 111, 17 93517 940, https://doi.org/10.1073/pnas.1417664111.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cooksey, R. W., 1996: Judgment Analysis: Theory, Methods and Applications. Academic Press, 407 pp.

  • Cui, B., Z. Toth, Y. Zhu, and D. Hou, 2012: Bias correction for global ensemble forecast. Wea. Forecasting, 27, 396410, https://doi.org/10.1175/WAF-D-11-00011.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Delle Monache, L., T. Nipen, Y. Liu, G. Roux, and R. Stull, 2011: Kalman filter and analog schemes to postprocess numerical weather predictions. Mon. Wea. Rev., 139, 35543570, https://doi.org/10.1175/2011MWR3653.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deng, X., and R. Stull, 2005: A mesoscale analysis method for surface potential temperature in mountainous and coastal terrain. Mon. Wea. Rev., 133, 389408, https://doi.org/10.1175/MWR-2859.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deng, X., and R. Stull, 2007: Assimilating surface weather observations from complex terrain into a high-resolution numerical weather prediction model. Mon. Wea. Rev., 135, 10371054, https://doi.org/10.1175/MWR3332.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Epstein, E., 1969: A scoring system for probability forecasts of ranked categories. J. Appl. Meteor., 8, 985987, https://doi.org/10.1175/1520-0450(1969)008<0985:ASSFPF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Glahn, H. R., and D. A. Lowry, 1972: The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11, 12031211, https://doi.org/10.1175/1520-0450(1972)011<1203:TUOMOS>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gneiting, T., A. E. Raftery, A. H. Westveld, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133, 10981118, https://doi.org/10.1175/MWR2904.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., G. T. Bates, J. S. Whitaker, D. R. Murray, M. Fiorino, T. J. Galarneau Jr., Y. Zhu, and W. Lapenta, 2013: NOAA’s second-generation global medium-range ensemble reforecast dataset. Bull. Amer. Meteor. Soc., 94, 15531565, https://doi.org/10.1175/BAMS-D-12-00014.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and S. J. Colucci, 1997: Verification of Eta-RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 13121327, https://doi.org/10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hemri, S., M. Scheuerer, F. Pappenberger, K. Bogner, and T. Haiden, 2014: Trends in the predictive performance of raw ensemble weather forecasts. Geophys. Res. Lett., 41, 91979205, https://doi.org/10.1002/2014GL062472.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hennon, C. C., C. Marzban, and J. S. Hobgood, 2005: Improving tropical cyclogenesis statistical model forecasts through the application of a neural network classifier. Wea. Forecasting, 20, 10731083, https://doi.org/10.1175/WAF890.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796811, https://doi.org/10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koizumi, K., 1999: An objective method to modify numerical model forecasts with newly given weather data using an artificial neural network. Wea. Forecasting, 14, 109118, https://doi.org/10.1175/1520-0434(1999)014<0109:AOMTMN>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kuligowski, R. J., and A. P. Barros, 2001: Combined IR-microwave satellite retrieval of temperature and dewpoint profiles using artificial neural networks. J. Appl. Meteor., 40, 20512067, https://doi.org/10.1175/1520-0450(2001)040<2051:CIMSRO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lugo, C. A., and A. J. McKane, 2008: Quasicycles in a spatial predator-prey model. Phys. Rev. E, 78, 51 91151 925, https://doi.org/10.1103/PhysRevE.78.051911.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mehrabian, A. R., and C. Lucas, 2006: A novel numerical optimization algorithm inspired from weed colonization. Ecol. Inform., 1, 355366, https://doi.org/10.1016/j.ecoinf.2006.07.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Messner, J. W., G. J. Mayr, and A. Zeileis, 2017: Nonhomogeneous boosting for predictor selection in ensemble postprocessing. Mon. Wea. Rev., 145, 137147, https://doi.org/10.1175/MWR-D-16-0088.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Monteith, K., J. Carroll, K. Seppi, and T. Martinez, 2011: Turning Bayesian model averaging into Bayesian model combination. Proc. Int. Joint Conf. on Neural Networks (IJCNN’11), San Jose, CA, IEEE, 2657–2663, https://doi.org/10.1109/IJCNN.2011.6033566.

    • Crossref
    • Export Citation
  • Murphy, A. H., 1969: On the “ranked probability score.” J. Appl. Meteor., 8, 988989, https://doi.org/10.1175/1520-0450(1969)008<0988:OTPS>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1971: A note on the ranked probability score. J. Appl. Meteor., 10, 155156, https://doi.org/10.1175/1520-0450(1971)010<0155:ANOTRP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133, 11551174, https://doi.org/10.1175/MWR2906.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., 1998: The regime dependence of degree day forecast technique, skill and value. Wea. Forecasting, 13, 783794, https://doi.org/10.1175/1520-0434(1998)013<0783:TRDODD>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., 2010: Seeking consensus: A new approach. Mon. Wea. Rev., 138, 44024415, https://doi.org/10.1175/2010MWR3508.1.

  • Roebber, P. J., 2015a: Evolving ensembles. Mon. Wea. Rev., 143, 471490, https://doi.org/10.1175/MWR-D-14-00058.1.

  • Roebber, P. J., 2015b: Ensemble MOS and evolutionary program minimum temperature forecast skill. Mon. Wea. Rev., 143, 15061516, https://doi.org/10.1175/MWR-D-14-00096.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roebber, P. J., 2015c: Adaptive evolutionary programming. Mon. Wea. Rev., 143, 14971505, https://doi.org/10.1175/MWR-D-14-00095.1.

  • Roebber, P. J., M. R. Butt, S. J. Reinke, and T. J. Grafenauer, 2007: Real-time forecasting of snowfall using a neural network. Wea. Forecasting, 22, 676684, https://doi.org/10.1175/WAF1000.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Scheuerer, M., and T. M. Hamill, 2015: Statistical postprocessing of ensemble precipitation forecasts by fitting censored, shifted Gamma distributions. Mon. Wea. Rev., 143, 45784596, https://doi.org/10.1175/MWR-D-15-0061.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Siegert, S., J. Bröcker, and H. Kantz, 2011: Predicting outliers in ensemble forecasts. Quart. J. Roy. Meteor. Soc., 137, 18871897, https://doi.org/10.1002/qj.868.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sprott, J. C., 2008: Predator-prey dynamics for rabbits, trees and romance. Unifying Themes in Complex Systems IV, A. A. Minai and Y. Bar-Yam, Eds., Springer, 231–238, https://doi.org/10.1007/978-3-540-73849-7_26.

    • Crossref
    • Export Citation
  • Taillardat, M., O. Mestre, M. Zamo, and P. Naveau, 2016: Calibrated ensemble forecasts using quantile regression forests and ensemble model output statistics. Mon. Wea. Rev., 144, 23752393, https://doi.org/10.1175/MWR-D-15-0260.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Talagrand, O., R. Vautard, and B. Strauss, 1997: Evaluation of probabilistic prediction systems. Proc. ECMWF Workshop on Predictability, Reading, United Kingdom, ECMWF, 1–25. [Available from ECMWF, Shinfield Park, Reading, Berkshire RG2 9AX, United Kingdom.]

  • Teisberg, T. J., R. F. Weiher, and A. Khotanzad, 2005: The economic value of temperature forecasts in electricity generation. Bull. Amer. Meteor. Soc., 86, 17651771, https://doi.org/10.1175/BAMS-86-12-1765.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • U.S. Energy Information Administration, 2014: Northeast natural gas spot prices particularly sensitive to temperature swings. Today in Energy, 11 August 2014, https://www.eia.gov/todayinenergy/detail.php?id=17491.

  • Weigel, A. P., M. A. Liniger, and C. Appenzeller, 2007: The discrete Brier and ranked probability skill scores. Mon. Wea. Rev., 135, 118124, https://doi.org/10.1175/MWR3280.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yang, H. T., C. M. Huang, and C. L. Huang, 1996: Identification of ARMAX model for short-term load forecasting: An evolutionary programming approach. IEEE Trans. Power Syst., 11, 403408, https://doi.org/10.1109/59.486125.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 609 45 3
PDF Downloads 187 41 3