Verification of Two Years of CNR-ISAC Subseasonal Forecasts

Daniele Mastrangelo CNR-ISAC, Bologna, Italy

Search for other papers by Daniele Mastrangelo in
Current site
Google Scholar
PubMed
Close
and
Piero Malguzzi CNR-ISAC, Bologna, Italy

Search for other papers by Piero Malguzzi in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

The monthly forecasting system of the Institute of Atmospheric Sciences and Climate of the National Research Council (CNR-ISAC) of Italy is operationally run on a weekly basis in the framework of the Subseasonal-to-Seasonal (S2S) project to produce 41-member ensemble forecasts. The first two years of forecasts, covering 106 weeks from April 2015, are verified against ERA-Interim as weekly averages starting from the first forecast day. Nonprobabilistic scores of 500-hPa geopotential height and 850-hPa temperature anomalies are computed for the extratropical hemispheres. The anomaly correlation coefficient shows enhanced predictive skill during the cold months, when favorable values are occasionally obtained beyond week 2. The root-mean-square error saturates toward the climatological value between weeks 2 and 3. Reliability diagrams are used to evaluate the probabilistic forecast skill of 2-m temperature over Northern Hemisphere extratropical land points, in terms of above- and below-normal events. The forecasting system loses reliability and resolution beyond week 2, but well reproduces the observed 2-yr mean frequency up to week 4, proving to be unbiased. The reliability of the forecasting system systematically outperforms that obtained by persisting the previous week forecast. Beyond week 2, the forecast distribution of below-normal events shows low confidence. However, a reliability diagram based on equally populated bins of forecast probabilities highlights residual resolution up to week 4 at low probabilities. ROC diagrams confirm that the modeling system has greater discrimination capability for below-normal events. The reliability analysis of accumulated precipitation shows minor differences between below- and above-normal events, with lower skill than 2-m temperature.

© 2019 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Daniele Mastrangelo, d.mastrangelo@isac.cnr.it

Abstract

The monthly forecasting system of the Institute of Atmospheric Sciences and Climate of the National Research Council (CNR-ISAC) of Italy is operationally run on a weekly basis in the framework of the Subseasonal-to-Seasonal (S2S) project to produce 41-member ensemble forecasts. The first two years of forecasts, covering 106 weeks from April 2015, are verified against ERA-Interim as weekly averages starting from the first forecast day. Nonprobabilistic scores of 500-hPa geopotential height and 850-hPa temperature anomalies are computed for the extratropical hemispheres. The anomaly correlation coefficient shows enhanced predictive skill during the cold months, when favorable values are occasionally obtained beyond week 2. The root-mean-square error saturates toward the climatological value between weeks 2 and 3. Reliability diagrams are used to evaluate the probabilistic forecast skill of 2-m temperature over Northern Hemisphere extratropical land points, in terms of above- and below-normal events. The forecasting system loses reliability and resolution beyond week 2, but well reproduces the observed 2-yr mean frequency up to week 4, proving to be unbiased. The reliability of the forecasting system systematically outperforms that obtained by persisting the previous week forecast. Beyond week 2, the forecast distribution of below-normal events shows low confidence. However, a reliability diagram based on equally populated bins of forecast probabilities highlights residual resolution up to week 4 at low probabilities. ROC diagrams confirm that the modeling system has greater discrimination capability for below-normal events. The reliability analysis of accumulated precipitation shows minor differences between below- and above-normal events, with lower skill than 2-m temperature.

© 2019 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Daniele Mastrangelo, d.mastrangelo@isac.cnr.it
Save
  • Bauer, P., A. Thorpe, and G. Brunet, 2015: The quiet revolution of numerical weather prediction. Nature, 525, 4755, https://doi.org/10.1038/nature14956.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bröcker, J., and L. A. Smith, 2007: Increasing the reliability of reliability diagrams. Wea. Forecasting, 22, 651661, https://doi.org/10.1175/WAF993.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brunet, G., and Coauthors, 2010: Collaboration of the weather and climate communities to advance subseasonal-to-seasonal prediction. Bull. Amer. Meteor. Soc., 91, 13971406, https://doi.org/10.1175/2010BAMS3013.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cassou, C., 2008: Intraseasonal interaction between the Madden–Julian oscillation and the North Atlantic Oscillation. Nature, 455, 523527, https://doi.org/10.1038/nature07286.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, M., W. Wang, and A. Kumar, 2010: Prediction of monthly-mean temperature: the roles of atmospheric and land initial conditions and sea surface temperature. J. Climate, 23, 717725, https://doi.org/10.1175/2009JCLI3090.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, https://doi.org/10.1002/qj.828.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Déqué, M., and J. F. Royer, 1992: The skill of extended-range extratropical winter dynamical forecasts. J. Climate, 5, 13461356, https://doi.org/10.1175/1520-0442(1992)005<1346:TSOERE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ferrone, A., D. Mastrangelo, and P. Malguzzi, 2017: Multimodel probabilistic prediction of 2 m-temperature anomalies on the monthly timescale. Adv. Sci. Res., 14, 123129, https://doi.org/10.5194/asr-14-123-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hagedorn, R., T. M. Hamill, and J. S. Whitaker, 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part I: Two-meter temperatures. Mon. Wea. Rev., 136, 26082619, https://doi.org/10.1175/2007MWR2410.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoskins, B., 2013: The potential for skill across the range of the seamless weather-climate prediction problem: A stimulus for our science. Quart. J. Roy. Meteor. Soc., 139, 573584, https://doi.org/10.1002/qj.1991.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hsu W., and A. H. Murphy, 1986: The attributes diagram: A geometrical framework for assessing the quality of probability forecasts. Int. J. Forecast., 2, 285293, https://doi.org/10.1016/0169-2070(86)90048-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hudson, D., O. Alves, H. H. Hendon, and A. G. Marshall, 2011: Bridging the gap between weather and seasonal forecasting: Intraseasonal forecasting for Australia. Quart. J. Roy. Meteor. Soc., 137, 673689, https://doi.org/10.1002/qj.769.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hudson, D., A. G. Marshall, O. Alves, G. Young, D. Jones, and A. Watkins, 2016: Forewarned is forearmed: Extended-range forecast guidance of recent extreme heat events in Australia. Wea. Forecasting, 31, 697711, https://doi.org/10.1175/WAF-D-15-0079.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • L’Heureux, M. L., and Coauthors, 2017: Observing and predicting the 2015/16 El Niño. Bull. Amer. Meteor. Soc., 98, 13631382, https://doi.org/10.1175/BAMS-D-16-0009.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, S., and A. W. Robertson, 2015: Evaluation of submonthly precipitation forecast skill from global ensemble prediction systems. Mon. Wea. Rev., 143, 28712889, https://doi.org/10.1175/MWR-D-14-00277.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, H., G. Brunet, and J. Derome, 2009: An observed connection between the North Atlantic Oscillation and the Madden–Julian oscillation. J. Climate, 22, 364380, https://doi.org/10.1175/2008JCLI2515.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, H., N. Gagnon, S. Beauregard, R. Muncaster, M. Markovic, B. Denis, and M. Charron, 2016: GEPS-based monthly prediction at the Canadian Meteorological Centre. Mon. Wea. Rev., 144, 48674883, https://doi.org/10.1175/MWR-D-16-0138.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N.: 1975: Climate predictability: The physical basis of climate modeling. WMO GARP Publ. Ser. 16, 132–136.

  • Malguzzi, P., A. Buzzi, and O. Drofa, 2011: The meteorological global model GLOBO at the ISAC-CNR of Italy: Assessment of 1.5 years of experimental use for medium range weather forecasts. Wea. Forecasting, 26, 10451055, https://doi.org/10.1175/WAF-D-11-00027.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marzban, C., 2004: The ROC curve and the area under it as performance measures. Wea. Forecasting, 19, 11061114, https://doi.org/10.1175/825.1.

  • Mason, S. J., and N. E. Graham, 2002: Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quart. J. Roy. Meteor. Soc., 128, 21452166, https://doi.org/10.1256/003590002320603584.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mastrangelo, D., and P. Malguzzi, 2017: CNR-ISAC 2 m temperature monthly forecasts: a first probabilistic evaluation. Adv. Sci. Res., 14, 8588, https://doi.org/10.5194/asr-14-85-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mastrangelo, D., P. Malguzzi, C. Rendina, O. Drofa, and A. Buzzi, 2012: First outcomes from the CNR-ISAC monthly forecasting system. Adv. Sci. Res., 8, 7782, https://doi.org/10.5194/asr-8-77-2012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reichler, T. J., and J. O. Roads, 2003: The role of boundary and initial conditions for dynamical seasonal predictability. Nonlinear Processes Geophys., 10, 211232, https://doi.org/10.5194/npg-10-211-2003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Robertson, A. W., A. Kumar, M. Peña, and F. Vitart, 2015: Improving and promoting subseasonal to seasonal prediction. Bull. Amer. Meteor. Soc., 96, ES49ES53, https://doi.org/10.1175/BAMS-D-14-00139.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thomas, J. A., A. A. Berg, and W. J. Merryfield, 2016: Influence of snow and soil moisture initialization on sub-seasonal predictability and forecast skill in boreal spring. Climate Dyn., 47, 4965, https://doi.org/10.1007/s00382-015-2821-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thompson, D. W. J., M. P. Baldwin, and J. M. Wallace, 2002: Stratospheric connection to Northern Hemisphere wintertime weather: Implications for prediction. J. Climate, 15, 14211428, https://doi.org/10.1175/1520-0442(2002)015<1421:SCTNHW>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tripathi, O. P., A. Charlton-Perez, M. Sigmond, and F. Vitart, 2015: Enhanced long-range forecast skill in boreal winter following stratospheric strong vortex conditions. Environ. Res. Lett., 10, 104007, https://doi.org/10.1088/1748-9326/10/10/104007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • UCAR/NCAR/CISL/TDD, 2019: NCAR Command Language (version 6.6.2). UCAR/NCAR/CISL/TDD, Boulder, CO, https://doi.org/10.5065/D6WD3XH5.

    • Crossref
    • Export Citation
  • Vitart, F., 2004: Monthly forecasting at ECMWF. Mon. Wea. Rev., 132, 27612779, https://doi.org/10.1175/MWR2826.1.

  • Vitart, F., 2014: Evolution of ECMWF sub-seasonal forecast skill scores. Quart. J. Roy. Meteor. Soc., 140, 18891899, https://doi.org/10.1002/qj.2256.

  • Vitart, F., and F. Molteni, 2010: Simulation of the Madden–Julian Oscillation and its teleconnections in the ECMWF forecast system. Quart. J. Roy. Meteor. Soc., 136, 842855, https://doi.org/10.1002/qj.623.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vitart, F., A. W. Robertson, and D. L. T. Anderson, 2012: Subseasonal to Seasonal Prediction Project: Bridging the gap between weather and climate. WMO Bull., 61, 2328.

    • Search Google Scholar
    • Export Citation
  • Vitart, F., and Coauthors, 2017: The Sub-seasonal to Seasonal Prediction (S2S) Project Database. Bull. Amer. Meteor. Soc., 98, 163173, https://doi.org/10.1175/BAMS-D-16-0017.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Waliser, D. E., 2011. Predictability and forecasting. Intraseasonal Variability of the Atmosphere-Ocean Climate System. 2nd ed. W. K.-M. Lau and D. E. Waliser, Eds., Springer, 433–468.

    • Crossref
    • Export Citation
  • Wang, S., A. Anichowski, M. K. Tippett, and A. H. Sobel, 2017: Seasonal noise versus subseasonal signal: Forecasts of California precipitation during the unusual winters of 2015–2016 and 2016–2017. Geophys. Res. Lett., 44, 95139520, https://doi.org/10.1002/2017GL075052.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weigel, A. P., D. Baggenstos, M. A. Liniger, F. Vitart, and C. Appenzeller, 2008: Probabilistic verification of monthly temperature forecasts. Mon. Wea. Rev., 136, 51625182, https://doi.org/10.1175/2008MWR2551.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • White, C. J., D. Hudson, and O. Alves, 2014: ENSO, the IOD and intraseasonal prediction of heat extremes across Australia using POAMA-2. Climate Dyn., 43, 17911810, https://doi.org/10.1007/s00382-013-2007-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • White, C. J., and Coauthors, 2017: Potential applications of subseasonal-to-seasonal (S2S) predictions. Met. Apps, 24, 315325, https://doi.org/10.1002/met.1654.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. International Geophysics Series, Vol. 100, Academic Press, 704 pp.

  • World Meteorological Organization, 2017: Manual on the Global Data-processing and Forecasting System: Annex IV to the WMO Technical Regulations. WMO-485, 119 pp., https://library.wmo.int/doc_num.php?explnum_id=4246.

  • Xue, Y., and A. Kumar, 2017: Evolution of the 2015/16 El Niño and historical perspective since 1979. Sci. China Earth Sci., 60, 15721588, https://doi.org/10.1007/s11430-016-0106-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhou, X., Y. Zhu, D. Hou, Y. Luo, J. Peng, and R. Wobus, 2017: Performance of the New NCEP Global Ensemble Forecast System in a parallel experiment. Wea. Forecasting, 32, 19892004, https://doi.org/10.1175/WAF-D-17-0023.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1950 1373 41
PDF Downloads 455 86 10