Verification of Multimodel Ensemble Forecasts of North Atlantic Tropical Cyclones

Nicholas M. Leonardo School of Marine and Atmospheric Sciences, Stony Brook University, State University of New York, Stony Brook, New York

Search for other papers by Nicholas M. Leonardo in
Current site
Google Scholar
PubMed
Close
and
Brian A. Colle School of Marine and Atmospheric Sciences, Stony Brook University, State University of New York, Stony Brook, New York

Search for other papers by Brian A. Colle in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

North Atlantic tropical cyclone (TC) forecasts from four ensemble prediction systems (EPSs) are verified using the National Hurricane Center’s (NHC) best tracks for the 2008–15 seasons. The 1–5-day forecasts are evaluated for the 21-member National Centers for Environmental Prediction (NCEP) Global Ensemble Forecast System (GEFS), the 23-member UKMO ensemble (UKMET), and the 51-member European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble, as well as a combination of these ensembles [Multimodel Global (MMG)]. Several deterministic models are also evaluated, such as the Global Forecast System (GFSdet), Hurricane Weather Research and Forecasting Model (HWRF), the deterministic ECMWF model (ECdet), and the Geophysical Fluid Dynamical Laboratory model (GFDL).The ECdet track errors are the smallest on average at all lead times, but are not significantly different from the GEFS and ECMWF ensemble means. All models have a slow bias (90–240 km) in the along-track direction by 120 h, while there is little bias in the cross-track direction. Much of this slow bias is attributed to TCs undergoing extratropical transition (ET). All EPSs are underdispersed in the along-track direction, while the ECMWF is slightly overdispersed in the cross-track direction. The MMG and ECMWF track forecasts have more probabilistic skill than the ECdet and comparable skill to the NHC climatology-based cone forecast. TC intensity errors for the HWRF and GFDL are lower than the coarser models within the first 24 h, but are comparable to the ECdet at longer lead times. The ECMWF and MMG have comparable or better probabilistic intensity forecasts than the ECdet, while the GEFS’s weak bias limits its skill.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Dr. Brian A. Colle, brian.colle@stonybrook.edu

Abstract

North Atlantic tropical cyclone (TC) forecasts from four ensemble prediction systems (EPSs) are verified using the National Hurricane Center’s (NHC) best tracks for the 2008–15 seasons. The 1–5-day forecasts are evaluated for the 21-member National Centers for Environmental Prediction (NCEP) Global Ensemble Forecast System (GEFS), the 23-member UKMO ensemble (UKMET), and the 51-member European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble, as well as a combination of these ensembles [Multimodel Global (MMG)]. Several deterministic models are also evaluated, such as the Global Forecast System (GFSdet), Hurricane Weather Research and Forecasting Model (HWRF), the deterministic ECMWF model (ECdet), and the Geophysical Fluid Dynamical Laboratory model (GFDL).The ECdet track errors are the smallest on average at all lead times, but are not significantly different from the GEFS and ECMWF ensemble means. All models have a slow bias (90–240 km) in the along-track direction by 120 h, while there is little bias in the cross-track direction. Much of this slow bias is attributed to TCs undergoing extratropical transition (ET). All EPSs are underdispersed in the along-track direction, while the ECMWF is slightly overdispersed in the cross-track direction. The MMG and ECMWF track forecasts have more probabilistic skill than the ECdet and comparable skill to the NHC climatology-based cone forecast. TC intensity errors for the HWRF and GFDL are lower than the coarser models within the first 24 h, but are comparable to the ECdet at longer lead times. The ECMWF and MMG have comparable or better probabilistic intensity forecasts than the ECdet, while the GEFS’s weak bias limits its skill.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Dr. Brian A. Colle, brian.colle@stonybrook.edu
Save
  • Aberson, S. D., 2010: 10 years of hurricane synoptic surveillance (1997–2006). Mon. Wea. Rev., 138, 15361549, https://doi.org/10.1175/2009MWR3090.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arakawa, A., and W. H. Schubert, 1974: Interaction of a cumulus cloud ensemble with the large-scale environment, Part I. J. Atmos. Sci., 31, 674701, https://doi.org/10.1175/1520-0469(1974)031<0674:IOACCE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bassill, N. P., 2014: Accuracy of early GFS and ECMWF Sandy (2012) track forecasts: Evidence for a dependence on cumulus parameterization. Geophys. Res. Lett., 41, 32743281, https://doi.org/10.1002/2014GL059839.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blake, E. S., C. W. Landsea, and E. J. Gibney, 2011: The deadliest, costliest, and most intense United States tropical cyclones from 1851 to 2010 (and other frequently requested hurricane facts). NOAA Tech. Memo. NWS NHC-6, 47 pp., http://www.nhc.noaa.gov/pdf/nws-nhc-6.pdf.

  • Blake, E. S., T. B. Kimberlain, R. J. Berg, J. P. Cangialosi, and J. L. Beven II, 2013: Tropical cyclone report: Hurricane Sandy (AL182012) 22–29 October 2012. National Hurricane Center, 157 pp., http://www.nhc.noaa.gov/data/tcr/AL182012_Sandy.pdf.

  • Bourke, W., R. Buizza, and M. Naughton, 2004: Performance of the ECMWF and the BoM ensemble systems in the Southern Hemisphere. Mon. Wea. Rev., 132, 23382357, https://doi.org/10.1175/1520-0493(2004)132<2338:POTEAT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brennan, M. J., and S. J. Majumdar, 2011: An examination of model track forecast errors for Hurricane Ike (2008) in the Gulf of Mexico. Wea. Forecasting, 26, 848867, https://doi.org/10.1175/WAF-D-10-05053.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brennan, M. J., D. T. Kleist, K. Howard, and S. J. Majumdar, 2015: The impact of supplemental dropwindsonde data on the structure and intensity of Tropical Storm Karen (2013) in the NCEP Global Forecast System. Wea. Forecasting, 30, 683691, https://doi.org/10.1175/WAF-D-15-0002.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buckingham, C., T. Marchok, I. Ginis, L. Rothstein, and D. Rowe, 2010: Short- and medium-range prediction of tropical and transitioning cyclone tracks within the NCEP Global Ensemble Forecasting System. Wea. Forecasting, 25, 17361754, https://doi.org/10.1175/2010WAF2222398.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buizza, R., P. L. Houtekamer, Z. Toth, G. Pellerin, M. Wei, and Y. Zhu, 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133, 10761097, https://doi.org/10.1175/MWR2905.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cangialosi, J. P., and J. Franklin, 2014: 2013 National Hurricane Center Forecast Verification Report. NOAA/National Hurricane Center, 84 pp., http://www.nhc.noaa.gov/verification/pdfs/Verification_2013.pdf.

  • Carr, L. E., III, and R. L. Elsberry, 2000: Dynamical tropical cyclone track forecast errors. Part II: Midlatitude circulation influences. Wea. Forecasting, 15, 662681, https://doi.org/10.1175/1520-0434(2000)015<0662:DTCTFE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Colby, F. P., Jr., 2015: Global ensemble forecast tracks for Tropical Storm Debby. Wea. Forecasting, 30, 668682, https://doi.org/10.1175/WAF-D-14-00083.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • DeMaria, M., 2010: Tropical cyclone intensity predictability estimates using a statistical-dynamical model. 29th Conf. on Hurricanes and Tropical Meteorology, Tucson, AZ, Amer. Meteor. Soc., 9C.5, https://ams.confex.com/ams/29Hurricanes/techprogram/paper_167916.htm.

  • DeMaria, M., C. R. Sampson, J. A. Knaff, and K. D. Musgrave, 2014: Is tropical cyclone intensity guidance improving? Bull. Amer. Meteor. Soc., 95, 387398, https://doi.org/10.1175/BAMS-D-12-00240.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dupont, T., M. Plu, P. Caroff, and G. Faure, 2011: Verification of ensemble-based uncertainty circles around tropical cyclone track forecasts. Wea. Forecasting, 26, 664676, https://doi.org/10.1175/WAF-D-11-00007.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Emanuel, K., and F. Zhang, 2016: On the predictability and error sources of tropical cyclone intensity forecasts. J. Atmos. Sci., 73, 37393747, https://doi.org/10.1175/JAS-D-16-0100.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Froude, L. S. R., L. Bengtsson, and K. I. Hodges, 2007: The predictability of extratropical storm tracks and the sensitivity of their prediction to the observing system. Mon. Wea. Rev., 135, 315333, https://doi.org/10.1175/MWR3274.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gall, R., F. Toepfer, F. Marks, and E. Rappaport, 2013: The Hurricane Forecast Improvement Project. Bull. Amer. Meteor. Soc., 94, 329343, https://doi.org/10.1175/BAMS-D-12-00071.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goerss, J. S., 2000: Tropical cyclone track forecasts using an ensemble of dynamical models. Mon. Wea. Rev., 128, 11871193, https://doi.org/10.1175/1520-0493(2000)128<1187:TCTFUA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goerss, J. S., C. R. Sampson, and J. M. Gross, 2004: A history of western North Pacific tropical cyclone track forecast skill. Wea. Forecasting, 19, 633638, https://doi.org/10.1175/1520-0434(2004)019<0633:AHOWNP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goldenberg, S. B., S. G. Gopalakrishnan, V. Tallapragada, T. Quirio, F. Marks, S. Trahan, X. Zhang, and R. Atlas, 2015: The 2012 triply nested, high-resolution operational version of the Hurricane Weather Research and Forecasting Model (HWRF): Track and intensity forecast verifications. Wea. Forecasting, 30, 710728, https://doi.org/10.1175/WAF-D-14-00098.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gregory, D., and P. R. Rowntree, 1990: A mass-flux convection scheme with representation of cloud ensemble characteristics and stability-dependent closure. Mon. Wea. Rev., 118, 14831506, https://doi.org/10.1175/1520-0493(1990)118<1483:AMFCSW>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550560, https://doi.org/10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. S. Whitaker, M. Fiorino, and S. G. Benjamin, 2011: Global ensemble predictions of 2009’s tropical cyclones initialized with an ensemble Kalman filter. Mon. Wea. Rev., 139, 668688, https://doi.org/10.1175/2010MWR3456.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harr, P. A., D. Anwender, and S. C. Jones, 2008: Predictability associated with the downstream impacts of the extratropical transition of tropical cyclones: Methodology and a case study. Mon. Wea. Rev., 136, 32053225, https://doi.org/10.1175/2008MWR2248.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jones, S. C., and Coauthors, 2003: The extratropical transition of tropical cyclones: Forecast challenges, current understanding, and future directions. Wea. Forecasting, 18, 10521092, https://doi.org/10.1175/1520-0434(2003)018<1052:TETOTC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and J. M. Fritsch, 1990: A one-dimensional entraining/detraining plume model and its application in convective parameterization. J. Atmos. Sci., 47, 27842802, https://doi.org/10.1175/1520-0469(1990)047<2784:AODEPM>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kain, J. S., and J. M. Fritsch, 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models, Meteor. Monogr., No. 46, Amer. Meteor. Soc., 165–170.

    • Crossref
    • Export Citation
  • Kaplan, J., M. DeMaria, and J. A. Knaff, 2010: A revised tropical cyclone rapid intensification index for the Atlantic and eastern North Pacific basins. Wea. Forecasting, 25, 220241, https://doi.org/10.1175/2009WAF2222280.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kerns, B. W., and S. S. Chen, 2014: ECMWF and GFS model forecast verification during DYNAMO: Multiscale variability in MJO initiation over the equatorial Indian Ocean. J. Geophys. Res. Atmos., 119, 37363755, doi:10.1002/2013JD020833.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Munsell, E. B., and F. Zhang, 2014: Prediction and uncertainty of Hurricane Sandy (2012) explored through a real-time cloud-permitting ensemble analysis and forecast system assimilating airborne Doppler radar observations. J. Adv. Model. Earth Syst., 6, 3858, https://doi.org/10.1002/2013MS000297.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Munsell, E. B., J. A. Sippel, S. A. Braun, Y. Weng, and F. Zhang, 2015: Dynamics and predictability of Hurricane Nadine (2012) evaluated through convection-permitting ensemble analysis and forecasts. Mon. Wea. Rev., 143, 45144532, https://doi.org/10.1175/MWR-D-14-00358.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1973: A new vector partition of the probability score. J. Appl. Meteor., 12, 595600, https://doi.org/10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nixon, C., 2012: Evaluating tropical cyclone forecast track uncertainty using a grand ensemble of ensemble prediction systems. M.S. thesis, Naval Postgraduate School, 60 pp., https://calhoun.nps.edu/handle/10945/5465.

  • Pan, H. L., and W. S. Wu, 1995: Implementing a mass flux convection parameterization package for the NMC Medium-Range Forecast Model. NMC Office Note 409, 43 pp., http://www2.mmm.ucar.edu/wrf/users/phys_refs/CU_PHYS/Old_SAS.pdf.

  • Payne, K. A., R. L. Elseberry, and M. A. Boothe, 2007: Assessment of western North Pacific 96- and 120-h track guidance and present forecastability. Wea. Forecasting, 22, 10031015, https://doi.org/10.1175/WAF1030.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rappaport, E. N., and Coauthors, 2009: Advances and challenges at the National Hurricane Center. Wea. Forecasting, 24, 395419, https://doi.org/10.1175/2008WAF2222128.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, Y., Z. Zhong, and W. Lu, 2015: Sensitivity of tropical cyclone feedback on the intensity of the western Pacific subtropical high to microphysics schemes. J. Atmos. Sci., 72, 13461368, https://doi.org/10.1175/JAS-D-14-0051.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tiedtke, M., 1989: A comprehensive mass flux scheme for cumulus parameterization in large-scale models. Mon. Wea. Rev., 117, 17791800, https://doi.org/10.1175/1520-0493(1989)117<1779:ACMFSF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Van der Grijn, G. A., and F. Grazzini, 2003: Tropical cyclone forecasting at ECMWF: New products and validation. Geophysical Research Abstracts, Vol. 9, Abstract 11103, http://www.cosis.net/abstracts/EAE03/11103/EAE03-J-11103.pdf.

  • Weissmann, M., and Coauthors, 2011: The influence of assimilating dropsonde data on typhoon track and midlatitude forecasts. Mon. Wea. Rev., 139, 908920, https://doi.org/10.1175/2010MWR3377.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Elsevier, 676 pp.

    • Crossref
    • Export Citation
  • Zwiers, F. W., 1990: The effect of serial correlation on statistical inferences made with resampling procedures. J. Climate, 3, 14521461, https://doi.org/10.1175/1520-0442(1990)003<1452:TEOSCO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1611 853 53
PDF Downloads 791 145 12