Predictability and Forecast Skill in NMME

Emily Becker NOAA/NWS/NCEP/Climate Prediction Center, College Park, Maryland

Search for other papers by Emily Becker in
Current site
Google Scholar
PubMed
Close
,
Huug van den Dool NOAA/NWS/NCEP/Climate Prediction Center, College Park, Maryland

Search for other papers by Huug van den Dool in
Current site
Google Scholar
PubMed
Close
, and
Qin Zhang NOAA/NWS/NCEP/Climate Prediction Center, College Park, Maryland

Search for other papers by Qin Zhang in
Current site
Google Scholar
PubMed
Close
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

Forecast skill and potential predictability of 2-m temperature, precipitation rate, and sea surface temperature are assessed using 29 yr of hindcast data from models included in phase 1 of the North American Multimodel Ensemble (NMME) project. Forecast skill is examined using the anomaly correlation (AC); skill of the bias-corrected ensemble means (EMs) of the individual models and of the NMME 7-model EM are verified against the observed value. Forecast skill is also assessed using the root-mean-square error. The models’ representation of the size of forecast anomalies is also studied. Predictability was considered from two angles: homogeneous, where one model is verified against a single member from its own ensemble, and heterogeneous, where a model’s EM is compared to a single member from another model. This study provides insight both into the physical predictability of the three fields and into the NMME and its contributing models.

Most of the models in the NMME have fairly realistic spread, as represented by the interannual variability. The NMME 7-model forecast skill, verified against observations, is equal to or higher than the individual models’ forecast ACs. Two-meter temperature (T2m) skill matches the highest single-model skill, while precipitation rate and sea surface temperature NMME EM skill is higher than for any single model. Homogeneous predictability is higher than reported skill in all fields, suggesting there may be room for some improvement in model prediction, although there are many regional and seasonal variations. The estimate of potential predictability is not overly sensitive to the choice of model. In general, models with higher homogeneous predictability show higher forecast skill.

Corresponding author address: Emily Becker, NOAA/Climate Prediction Center, 5830 University Research Court, College Park, MD 20740. E-mail: emily.becker@noaa.gov

Abstract

Forecast skill and potential predictability of 2-m temperature, precipitation rate, and sea surface temperature are assessed using 29 yr of hindcast data from models included in phase 1 of the North American Multimodel Ensemble (NMME) project. Forecast skill is examined using the anomaly correlation (AC); skill of the bias-corrected ensemble means (EMs) of the individual models and of the NMME 7-model EM are verified against the observed value. Forecast skill is also assessed using the root-mean-square error. The models’ representation of the size of forecast anomalies is also studied. Predictability was considered from two angles: homogeneous, where one model is verified against a single member from its own ensemble, and heterogeneous, where a model’s EM is compared to a single member from another model. This study provides insight both into the physical predictability of the three fields and into the NMME and its contributing models.

Most of the models in the NMME have fairly realistic spread, as represented by the interannual variability. The NMME 7-model forecast skill, verified against observations, is equal to or higher than the individual models’ forecast ACs. Two-meter temperature (T2m) skill matches the highest single-model skill, while precipitation rate and sea surface temperature NMME EM skill is higher than for any single model. Homogeneous predictability is higher than reported skill in all fields, suggesting there may be room for some improvement in model prediction, although there are many regional and seasonal variations. The estimate of potential predictability is not overly sensitive to the choice of model. In general, models with higher homogeneous predictability show higher forecast skill.

Corresponding author address: Emily Becker, NOAA/Climate Prediction Center, 5830 University Research Court, College Park, MD 20740. E-mail: emily.becker@noaa.gov
Save
  • Arribas, A., and Coauthors, 2011: The GloSea4 ensemble prediction system for seasonal forecasting. Mon. Wea. Rev., 139, 18911910, doi:10.1175/2010MWR3615.1.

    • Search Google Scholar
    • Export Citation
  • Barnston, A. G., and H. M. Van den Dool, 1993: A degeneracy in cross-validated skill in regression-based forecasts. J. Climate, 6, 963977, doi:10.1175/1520-0442(1993)006<0963:ADICVS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Barnston, A. G., M. Chelliah, and S. B. Goldenberg, 1997: Documentation of a highly ENSO-related SST region in the equatorial Pacific. Atmos.–Ocean, 35, 367383, doi:10.1080/07055900.1997.9649597.

    • Search Google Scholar
    • Export Citation
  • Becker, E. J., H. M. Van den Dool, and M. Peña, 2013: Short-term climate extremes: Prediction skill and predictability. J. Climate, 26, 512531, doi:10.1175/JCLI-D-12-00177.1.

    • Search Google Scholar
    • Export Citation
  • DeWitt, D. G., 2005: Retrospective forecasts of interannual sea surface temperature anomalies from 1982 to present using a directly coupled atmosphere–ocean general circulation model. Mon. Wea. Rev., 133, 29722995, doi:10.1175/MWR3016.1.

    • Search Google Scholar
    • Export Citation
  • Doblas-Reyes, F. J., R. Hagedorn, and T. N. Palmer, 2005: The rationale behind the success of multi-model ensembles in seasonal forecasting—II. Calibration and combination. Tellus, 57A, 234252, doi:10.1111/j.1600-0870.2005.00104.x.

    • Search Google Scholar
    • Export Citation
  • Fan, Y., and H. Van den Dool, 2008: A global monthly land surface air temperature analysis for 1948–present. J. Geophys. Res., 113, D01103, doi:10.1029/2007JD008470.

    • Search Google Scholar
    • Export Citation
  • Feng, X., T. DelSole, and P. Houser, 2013: Comparison of statistical estimates of potential seasonal predictability. J. Geophys. Res. Atmos., 118, 60026016, doi:10.1002/jgrd.50498.

    • Search Google Scholar
    • Export Citation
  • Gates, W. L., 1992: AMIP: The Atmospheric Model Intercomparison Project. Bull. Amer. Meteor. Soc., 73, 1962–1970. doi:10.1175/1520-0477(1992)073<1962:ATAMIP>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hagedorn, R., F. J. Doblas-Reyes, and T. N. Palmer, 2005: The rationale behind the success of multi-model ensembles in seasonal forecasting—I. Basic concept. Tellus, 57A, 219233, doi:10.1111/j.1600-0870.2005.00103.x.

    • Search Google Scholar
    • Export Citation
  • Huang, J., and H. M. Van den Dool, 1993: Monthly precipitation–temperature relations and temperature prediction over the United States. J. Climate, 6, 11111132, doi:10.1175/1520-0442(1993)006<1111:MPTRAT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Johnson, C., and N. Bowler, 2009: On the reliability and calibration of ensemble forecasts. Mon. Wea. Rev., 137, 17171720, doi:10.1175/2009MWR2715.1.

    • Search Google Scholar
    • Export Citation
  • Kirtman, B. P., and D. Min, 2009: Multimodel ensemble ENSO prediction with CCSM and CFS. Mon. Wea. Rev., 137, 2908–2930, doi:10.1175/2009MWR2672.1.

    • Search Google Scholar
    • Export Citation
  • Kirtman, B. P., and Coauthors, 2014: The North American Multi-Model Ensemble (NMME): Phase-1 seasonal to interannual prediction, phase-2 toward developing intra-seasonal prediction. Bull. Amer. Meteor. Soc., 95, 585–601, doi:10.1175/BAMS-D-12-00050.1.

    • Search Google Scholar
    • Export Citation
  • Kumar, A., 2009: Finite samples and uncertainty estimates for skill measures for seasonal prediction. Mon. Wea. Rev., 137, 26222631, doi:10.1175/2009MWR2814.1.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1969: Three approaches to atmospheric predictability. Bull. Amer. Meteor. Soc., 50, 345349.

  • Lorenz, E. N., 1982: Atmospheric predictability experiments with a large numerical model. Tellus, 34A, 505513, doi:10.1111/j.2153-3490.1982.tb01839.x.

    • Search Google Scholar
    • Export Citation
  • Madden, R. A., and D. J. Shea, 1978: Estimates of the natural variability of time-averaged temperatures over the United States. Mon. Wea. Rev., 106, 16951703, doi:10.1175/1520-0493(1978)106<1695:EOTNVO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Merryfield, W. J., and Coauthors, 2013: The Canadian Seasonal to Interannual Prediction System. Part I: Models and initialization. Mon. Wea. Rev., 141, 2910–2945, doi:10.1175/MWR-D-12-00216.1.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., and Coauthors, 2004: Development of a European Multimodel Ensemble System for Seasonal-to-Interannual Prediction (DEMETER). Bull. Amer. Meteor. Soc., 85, 853872, doi:10.1175/BAMS-85-6-853.

    • Search Google Scholar
    • Export Citation
  • Reynolds, R. W., N. A. Rayner, T. M. Smith, D. C. Stokes, and W. Wang, 2002: An improved in situ and satellite SST analysis for climate. J. Climate, 15, 16091625, doi:10.1175/1520-0442(2002)015<1609:AIISAS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Rodwell, M. J., and F. J. Doblas-Reyes, 2006: Medium-range, monthly, and seasonal prediction for Europe and the use of forecast information. J. Climate, 19, 60256046, doi:10.1175/JCLI3944.1.

    • Search Google Scholar
    • Export Citation
  • Saha, S., and Coauthors, 2006: The NCEP Climate Forecast System. J. Climate, 19, 34833517, doi:10.1175/JCLI3812.1.

  • Saha, S., and Coauthors, 2014: The NCEP Climate Forecast System version 2. J. Climate, 27, 2185–2208, doi:10.1175/JCLI-D-12-00823.1.

  • Scaife, A. A., T. Woollings, J. R. Knight, G. Martin, and T. Hinton, 2010: Atmospheric blocking and mean biases in climate models. J. Climate, 23, 61436152, doi:10.1175/2010JCLI3728.1.

    • Search Google Scholar
    • Export Citation
  • Smith, D. M., and Coauthors, 2013: Real-time multi-model decadal climate predictions. Climate Dyn., 41, 2875–2888, doi:10.1007/s00382-012-1600-0.

    • Search Google Scholar
    • Export Citation
  • Smith, T. M., and R. E. Livezey, 1999: GCM systematic error correction and specification of the seasonal mean Pacific–North America region atmosphere from global SSTs. J. Climate,12, 273–288, doi:10.1175/1520-0442-12.1.273.

  • Van den Dool, H. M., 2007: Empirical Methods in Short-Term Climate Prediction. Oxford University Press, 215 pp.

  • Van den Dool, H. M., 2009: Methods of multi-model consolidation, with emphasis on the recommended three-year-out cross validation approach. Extended Abstracts, NOAA CTB Joint Seminar Series, Camp Springs, MD, NOAA, 75–77. [Available online at http://www.nws.noaa.gov/ost/climate/STIP/fy09jsctb.htm.]

  • Van den Dool, H. M., and R. M. Chervin, 1986: A comparison of month-to-month persistence of anomalies in a general circulation model and in the earth's atmosphere. J. Atmos. Sci., 43, 14541466, doi:10.1175/1520-0469(1986)043<1454:ACOMTM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Vernieres, G., M. M. Rienecker, R. Kovach, and C. L. Keppenne, 2012: The GEOS-iODAS: Description and evaluation. NASA Tech. Rep. NASA/TM-2012-104606, Vol 30, 61 pp. [Available online at http://gmao.gsfc.nasa.gov/pubs/docs/Vernieres589.pdf.]

  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences: An Introduction. Academic Press, 467 pp.

  • Wu, R., B. P. Kirtman, and H. M. Van den Dool, 2009: An analysis of ENSO prediction skill in the CFS retrospective forecasts. J. Climate, 22, 18011818, doi:10.1175/2008JCLI2565.1.

    • Search Google Scholar
    • Export Citation
  • Zhang, S., M. J. Harrison, A. Rosati, and A. Wittenberg, 2007: System design and evaluation of coupled ensemble data assimilation for global oceanic climate studies. Mon. Wea. Rev., 135, 35413564, doi:10.1175/MWR3466.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 4310 1256 231
PDF Downloads 1813 408 49