Editorial Type:
Article
Article Type:
Research Article

Evolving Ensembles

Paul J. Roebber University of Wisconsin–Milwaukee, Milwaukee, Wisconsin

Search for other papers by Paul J. Roebber in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Feb 2015
DOI:
https://doi.org/10.1175/MWR-D-14-00058.1
Page(s):
471–490
Restricted access

Abstract

An ensemble forecast method using evolutionary programming, including various forms of genetic exchange, disease, mutation, and the training of solutions within ecological niches, is presented. A 2344-member ensemble generated in this way is tested for 60-h minimum temperature forecasts for Chicago, Illinois.

The ensemble forecasts are superior in both ensemble average root-mean-square error and Brier skill score to those obtained from a 21-member operational ensemble model output statistics (MOS) forecast. While both ensembles are underdispersive, spread calibration produces greater gains in probabilistic skill for the evolutionary program ensemble than for the MOS ensemble. When a Bayesian model combination calibration is used, the skill advantage for the evolutionary program ensemble relative to the MOS ensemble increases for root-mean-square error, but decreases for Brier skill score. Further improvement in root-mean-square error is obtained when the raw evolutionary program and MOS forecasts are pooled, and a new Bayesian model combination ensemble is produced.

Future extensions to the method are discussed, including those capable of producing more complex forms, those involving 1000-fold increases in training populations, and adaptive methods.

Corresponding author address: Paul J. Roebber, Atmospheric Science Group, Department of Mathematical Sciences and School of Freshwater Sciences, University of Wisconsin–Milwaukee, 3200 North Cramer Ave., Milwaukee, WI 53211. E-mail: roebber@uwm.edu

Abstract

An ensemble forecast method using evolutionary programming, including various forms of genetic exchange, disease, mutation, and the training of solutions within ecological niches, is presented. A 2344-member ensemble generated in this way is tested for 60-h minimum temperature forecasts for Chicago, Illinois.

The ensemble forecasts are superior in both ensemble average root-mean-square error and Brier skill score to those obtained from a 21-member operational ensemble model output statistics (MOS) forecast. While both ensembles are underdispersive, spread calibration produces greater gains in probabilistic skill for the evolutionary program ensemble than for the MOS ensemble. When a Bayesian model combination calibration is used, the skill advantage for the evolutionary program ensemble relative to the MOS ensemble increases for root-mean-square error, but decreases for Brier skill score. Further improvement in root-mean-square error is obtained when the raw evolutionary program and MOS forecasts are pooled, and a new Bayesian model combination ensemble is produced.

Future extensions to the method are discussed, including those capable of producing more complex forms, those involving 1000-fold increases in training populations, and adaptive methods.

Corresponding author address: Paul J. Roebber, Atmospheric Science Group, Department of Mathematical Sciences and School of Freshwater Sciences, University of Wisconsin–Milwaukee, 3200 North Cramer Ave., Milwaukee, WI 53211. E-mail: roebber@uwm.edu
Save
Cover Monthly Weather Review
Monthly Weather Review

  • Bakhshaii, A., and R. Stull, 2009: Deterministic ensemble forecasts using gene-expression programming. Wea. Forecasting, 24, 1431–1451, doi:10.1175/2009WAF2222192.1.

    • Search Google Scholar
    • Export Citation

  • Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 1–3, doi:10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Cooksey, R. W., 1996: Judgment Analysis: Theory, Methods and Applications. Academic Press, 407 pp.

  • Coulibaly, P., 2004: Downscaling daily extreme temperatures with genetic programming. Geophys. Res. Lett., 31, L16203, doi:10.1029/2004GL020075.

    • Search Google Scholar
    • Export Citation

  • Crochet, P., 2004: Adaptive Kalman filtering of 2-metre temperature and 10-metre wind speed forecasts in Iceland. Meteor. Appl., 11, 173–187, doi:10.1017/S1350482704001252.

    • Search Google Scholar
    • Export Citation

  • Cui, B., Z. Toth, Y. Zhu, and D. Hou, 2012: Bias correction for global ensemble forecast. Wea. Forecasting, 27, 396–410, doi:10.1175/WAF-D-11-00011.1.

    • Search Google Scholar
    • Export Citation

  • Darwin, C., 1859: On the Origin of Species. Bantam Classic Edition, 495 pp.

  • Darwin, C., 1871: The Descent of Man. Penguin Classics, 791 pp.

  • Fogel, L. J., 1999: Intelligence through Simulated Evolution: Forty Years of Evolutionary Programming. John Wiley, 162 pp.

  • Fogel, L. J., A. J. Owens, and M. J. Walsh, 1966: Artificial Intelligence through Simulated Evolution. John Wiley, 170 pp.

  • Gibbons, J. F., and S. Mylroie, 1973: Estimation of impurity profiles in ion-implanted amorphous targets using joined half-Gaussian distributions. Appl. Phys. Lett., 22, 568–569, doi:10.1063/1.1654511.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., and J. S. Whitaker, 2007: Ensemble calibration of 500-hPa geopotential height and 850-hPa and 2-m temperatures using reforecasts. Mon. Wea. Rev., 135, 3273–3280, doi:10.1175/MWR3468.1.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., J. S. Whitaker, and S. Mullen, 2006: Reforecasts: An important dataset for improving weather predictions. Bull. Amer. Meteor. Soc., 87, 33–46, doi:10.1175/BAMS-87-1-33.

    • Search Google Scholar
    • Export Citation

  • Haupt, R. L., and S. E. Haupt, 2000: Optimum population size and mutation rate for a simple real genetic algorithm that optimizes array factors. Appl. Comput. Electromag. Soc. J., 15, 94–102.

    • Search Google Scholar
    • Export Citation

  • Haupt, S. E., G. S. Young, and C. T. Allen, 2006: Validation of a receptor–dispersion model coupled with a genetic algorithm using synthetic data. J. Appl. Meteor. Climatol., 45, 476–490, doi:10.1175/JAM2359.1.

    • Search Google Scholar
    • Export Citation

  • Hénon, M., 1976: A two-dimensional mapping with a strange attractor. Commun. Math. Phys., 50, 69–77, doi:10.1007/BF01608556.

  • Hillis, W. D., 1990: Co-evolving parasites improve simulated evolution as an optimization procedure. Physica D, 42, 228–234, doi:10.1016/0167-2789(90)90076-2.

    • Search Google Scholar
    • Export Citation

  • Hoeting, J. A., D. Madigan, A. E. Raftery, and C. T. Volinsky, 1999: Bayesian model averaging: A tutorial. Stat. Sci., 14, 382–417, doi:10.1214/ss/1009212519.

    • Search Google Scholar
    • Export Citation

  • Homleid, M., 1995: Diurnal corrections of short-term surface temperature forecasts using the Kalman filter. Wea. Forecasting, 10, 689–707, doi:10.1175/1520-0434(1995)010<0689:DCOSTS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • John, S., 1982: The three-parameter two-piece normal family of distributions and its fitting. Comm. Stat. Theory Methods, 11, 879–885, doi:10.1080/03610928208828279.

    • Search Google Scholar
    • Export Citation

  • Kong, and Coauthors, 2012: Rate of de novo mutations and the importance of father’s age to disease risk. Nature, 488, 471–475, doi:10.1038/nature11396.

    • Search Google Scholar
    • Export Citation

  • Lakshmanan, V., 2000: Using a genetic algorithm to tune a bounded weak echo region detection algorithm. J. Appl. Meteor., 39, 222–230, doi:10.1175/1520-0450(2000)039<0222:UAGATT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130–141, doi:10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E. N., 2005: Designing chaotic models. J. Atmos. Sci., 62, 1574–1587, doi:10.1175/JAS3430.1.

  • Mercer, A. E., C. M. Shafer, C. A. Doswell III, L. M. Leslie, and M. B. Richman, 2012: Synoptic composites of tornadic and nontornadic outbreaks. Mon. Wea. Rev., 140, 2590–2608, doi:10.1175/MWR-D-12-00029.1.

    • Search Google Scholar
    • Export Citation

  • Mirus, K. A., and J. C. Sprott, 1999: Controlling chaos in low- and high-dimensional systems with periodic parametric perturbations. Phys. Rev. E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics, 59, 5313–5324.

    • Search Google Scholar
    • Export Citation

  • Monteith, K., J. Carroll, K. Seppi, and T. Martinez, 2011: Turning Bayesian model averaging into Bayesian model combination. Proc. Int. Joint Conf. on Neural Networks (IJCNN'11), San Jose, CA, IEEE, 2657–2663.

  • Murphy, A. H., 1973: A new vector partition of the probability score. J. Appl. Meteor., 12, 595–600, doi:10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • O’Steen, L., and D. Werth, 2009: The application of an evolutionary algorithm to the optimization of a mesoscale meteorological model. J. Appl. Meteor. Climatol., 48, 317–329, doi:10.1175/2008JAMC1967.1.

    • Search Google Scholar
    • Export Citation

  • Persson, A., 1991: Kalman filtering—A new approach to adaptive statistical interpretation of numerical meteorological forecasts. WMO Training Workshop on the Interpretation of NWP Products in Terms of Local Weather Phenomena and Their Verification, Wageningen, Netherlands, WMO, WMO PSMP Rep. Series 34, XX-27–XX-32.

  • Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133, 1155–1174, doi:10.1175/MWR2906.1.

    • Search Google Scholar
    • Export Citation

  • Roebber, P. J., 1998: The regime dependence of degree day forecast technique, skill, and value. Wea. Forecasting, 13, 783–794, doi:10.1175/1520-0434(1998)013<0783:TRDODD>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Roebber, P. J., 2010: Seeking consensus: A new approach. Mon. Wea. Rev., 138, 4402–4415, doi:10.1175/2010MWR3508.1.

  • Roebber, P. J., 2013: Using evolutionary programming to generate skillful extreme value probabilistic forecasts. Mon. Wea. Rev., 141, 3170–3185, doi:10.1175/MWR-D-12-00285.1.

    • Search Google Scholar
    • Export Citation

  • Roebber, P. J., and L. F. Bosart, 1996: The contributions of education and experience to forecast skill. Wea. Forecasting, 11, 21–40, doi:10.1175/1520-0434(1996)011<0021:TCOEAE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Ross, G. H., 1987: An updateable model output statistics scheme. Programme on Short- and Medium-Range Series, WMO Rep. 25, World Meteorological Organization, 25–28.

  • Simonsen, C., 1991: Self adaptive model output statistics based on Kalman filtering. WMO Training Workshop on the Interpretation of NWP Products in Terms of Local Weather Phenomena and Their Verification, Wageningen, Netherlands, WMO, WMO PSMP Rep. Series 34, XX-33–XX-37.

  • Stevens, M. H. H., M. Sanchez, J. Lee, and S. E. Finkel, 2007: Diversification rates increase with population size and resource concentration in an unstructured habitat. Genetics, 177, 2243–2250, doi:10.1534/genetics.107.076869.

    • Search Google Scholar
    • Export Citation

  • Stewart, T., P. J. Roebber, and L. F. Bosart, 1997: The importance of the task in analyzing expert judgment. Organ. Behav. Hum. Decis. Processes, 69, 205–219, doi:10.1006/obhd.1997.2682.

    • Search Google Scholar
    • Export Citation

  • Teisberg, T. J., R. F. Weiher, and A. Khotanzad, 2005: The economic value of temperature forecasts in electricity generation. Bull. Amer. Meteor. Soc., 86, 1765–1771, doi:10.1175/BAMS-86-12-1765.

    • Search Google Scholar
    • Export Citation

  • Wilson, L. J., and M. Vallée, 2002: The Canadian Updateable Model Output Statistics (UMOS) system: Design and development tests. Wea. Forecasting, 17, 206–222, doi:10.1175/1520-0434(2002)017<0206:TCUMOS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Yamagiwa, J., J. Kahekwa, and A. K. Basabose, 2003: Intra-specific variation in social organization of gorillas: Implications for their social evolution. Primates, 44, 359–369, doi:10.1007/s10329-003-0049-5.

    • Search Google Scholar
    • Export Citation

  • Yang, H. T., C. M. Huang, and C. L. Huang, 1996: Identification of ARMAX model for short-term load forecasting: An evolutionary programming approach. IEEE Trans. Power Syst., 11, 403–408, doi:10.1109/59.486125.

    • Search Google Scholar
    • Export Citation

  • Yuval, and W. W. Hsieh, 2003: An adaptive nonlinear MOS scheme for precipitation forecasts using neural networks. Wea. Forecasting,18, 303–310, doi:10.1175/1520-0434(2003)018<0303:AANMSF>2.0.CO;2.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 312 56 1
PDF Downloads 87 32 1