Ensemble-Based Error and Predictability Metrics Associated with Tropical Cyclogenesis. Part I: Basinwide Perspective

William A. Komaromi Rosenstiel School of Marine and Atmospheric Science, University of Miami, Miami, Florida

Search for other papers by William A. Komaromi in
Current site
Google Scholar
PubMed
Close
and
Sharanya J. Majumdar Rosenstiel School of Marine and Atmospheric Science, University of Miami, Miami, Florida

Search for other papers by Sharanya J. Majumdar in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Several metrics are employed to evaluate predictive skill and attempt to quantify predictability using the ECMWF Ensemble Prediction System during the 2010 Atlantic hurricane season, with an emphasis on large-scale variables relevant to tropical cyclogenesis. These metrics include the following: 1) growth and saturation of error, 2) errors versus climatology, 3) predicted forecast error standard deviation, and 4) predictive power. Overall, variables that are more directly related to large-scale, slowly varying phenomena are found to be much more predictable than variables that are inherently related to small-scale convective processes, regardless of the metric. For example, 850–200-hPa wind shear and 200-hPa velocity potential are found to be predictable beyond one week, while 200-hPa divergence and 850-hPa relative vorticity are only predictable to about one day. Similarly, area-averaged quantities such as circulation are much more predictable than nonaveraged quantities such as vorticity. Significant day-to-day and month-to-month variability of predictability for a given metric also exists, likely due to the flow regime. For wind shear, more amplified flow regimes are associated with lower predictive power (and thereby lower predictability) than less amplified regimes. Relative humidity is found to be less predictable in the early and late season when there exists greater uncertainty of the timing and location of dry air. Last, the ensemble demonstrates the potential to predict error standard deviation of variables averaged in 10° × 10° boxes, in that forecasts with greater ensemble standard deviation are on average associated with greater mean error. However, the ensemble tends to be underdispersive.

Corresponding author address: William Komaromi, Rosenstiel School of Marine and Atmospheric Science, Division of Meteorology and Physical Oceanography, 4600 Rickenbacker Causeway, Miami, FL 33149. E-mail: wkomaromi@rsmas.miami.edu

Abstract

Several metrics are employed to evaluate predictive skill and attempt to quantify predictability using the ECMWF Ensemble Prediction System during the 2010 Atlantic hurricane season, with an emphasis on large-scale variables relevant to tropical cyclogenesis. These metrics include the following: 1) growth and saturation of error, 2) errors versus climatology, 3) predicted forecast error standard deviation, and 4) predictive power. Overall, variables that are more directly related to large-scale, slowly varying phenomena are found to be much more predictable than variables that are inherently related to small-scale convective processes, regardless of the metric. For example, 850–200-hPa wind shear and 200-hPa velocity potential are found to be predictable beyond one week, while 200-hPa divergence and 850-hPa relative vorticity are only predictable to about one day. Similarly, area-averaged quantities such as circulation are much more predictable than nonaveraged quantities such as vorticity. Significant day-to-day and month-to-month variability of predictability for a given metric also exists, likely due to the flow regime. For wind shear, more amplified flow regimes are associated with lower predictive power (and thereby lower predictability) than less amplified regimes. Relative humidity is found to be less predictable in the early and late season when there exists greater uncertainty of the timing and location of dry air. Last, the ensemble demonstrates the potential to predict error standard deviation of variables averaged in 10° × 10° boxes, in that forecasts with greater ensemble standard deviation are on average associated with greater mean error. However, the ensemble tends to be underdispersive.

Corresponding author address: William Komaromi, Rosenstiel School of Marine and Atmospheric Science, Division of Meteorology and Physical Oceanography, 4600 Rickenbacker Causeway, Miami, FL 33149. E-mail: wkomaromi@rsmas.miami.edu
Save
  • Abramov, R., A. Majda, and R. Kleeman, 2005: Information theory and predictability for low-frequency variability. J. Atmos. Sci., 62, 6587, doi:10.1175/JAS-3373.1.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., and W. F. Stern, 1996: Evaluating the potential predictive utility of ensemble forecasts. J. Climate, 9, 260269, doi:10.1175/1520-0442(1996)009<0260:ETPPUO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Berner, J., G. J. Shutts, M. Leutbecher, and T. N. Palmer, 2009: A spectral stochastic kinetic energy backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system. J. Atmos. Sci., 66, 603626, doi:10.1175/2008JAS2677.1.

    • Search Google Scholar
    • Export Citation
  • Bijlsma, J. S., L. M. Hafkenscheid, and P. Lynch, 1986: Computation of the streamfunction and velocity potential and reconstruction of the wind field. Mon. Wea. Rev., 114, 15471551, doi:10.1175/1520-0493(1986)114<1547:COTSAV>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble. Bull. Amer. Meteor. Soc., 91, 10591072, doi:10.1175/2010BAMS2853.1.

    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim Reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, doi:10.1002/qj.828.

    • Search Google Scholar
    • Export Citation
  • DelSole, T., 2004: Predictability and information theory. Part I: Measures of predictability. J. Atmos. Sci., 61, 24252440, doi:10.1175/1520-0469(2004)061<2425:PAITPI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • DeMaria, J., A. Knaff, and B. H. Connell, 2001: A tropical cyclone genesis parameter for the tropical Atlantic. Wea. Forecasting, 16, 219233, doi:10.1175/1520-0434(2001)016<0219:ATCGPF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Dunion, J. P., 2011: Rewriting the climatology of the tropical North Atlantic and Caribbean Sea atmosphere. J. Climate, 24, 893908, doi:10.1175/2010JCLI3496.1.

    • Search Google Scholar
    • Export Citation
  • Giannakis, D., and A. J. Majda, 2012: Quantifying the predictive skill in long-range forecasting. Part I: Coarse-grained predictions in a simple ocean model. J. Climate, 25, 17931813, doi:10.1175/2011JCLI4143.1.

    • Search Google Scholar
    • Export Citation
  • Golub, G. H., and C. F. Van Loan, 1996: Matrix Computations. 3rd ed. Johns Hopkins University Press, 694 pp.

  • Gray, W. M., 1979: Hurricanes: Their formation, structure and likely role in the tropical circulation. Meteorology over the Tropical Oceans, D. B. Shaw, Ed., Royal Meteorological Society, 155–218.

  • Griffies, S. M., and K. Bryan, 1997: A predictability study of simulated North Atlantic multidecadal variability. Climate Dyn., 13, 459488, doi:10.1007/s003820050177.

    • Search Google Scholar
    • Export Citation
  • Grimit, E. P., 2004: Probabilistic mesoscale forecast error prediction using short-range ensembles. Ph.D. dissertation, University of Washington, 146 pp.

  • Grimit, E. P., and C. F. Mass, 2007: Measuring the ensemble spread–error relationship with a probabilistic approach: Stochastic ensemble results. Mon. Wea. Rev., 135, 203221, doi:10.1175/MWR3262.1.

    • Search Google Scholar
    • Export Citation
  • Halperin, D. J., H. E. Fuelberg, R. E. Hart, J. H. Cossuth, P. Sura, and R. J. Pasch, 2013: An evaluation of tropical cyclone genesis forecasts from global numerical models. Wea. Forecasting, 28, 1423–1445, doi:10.1175/WAF-D-13-00008.1.

    • Search Google Scholar
    • Export Citation
  • Hayashi, Y., 1986: Statistical interpretation of ensemble-time mean predictability. J. Meteor. Soc. Japan, 64, 167181.

  • Kleeman, R., 2002: Measuring dynamical prediction utility using relative entropy. J. Atmos. Sci., 59, 20572072, doi:10.1175/1520-0469(2002)059<2057:MDPUUR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Kleeman, R., 2007: Information flow in ensemble weather predictions. J. Atmos. Sci., 64, 10051016, doi:10.1175/JAS3857.1.

  • Lorenz, E. N., 1963: Deterministic non-periodic flow. J. Atmos. Sci., 20, 130141, doi:10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.

  • Lorenz, E. N., 1965: A new study of the predictability of a 28-variable atmospheric model. Tellus, 17, 321333, doi:10.1111/j.2153-3490.1965.tb01424.x.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1969: Predictability of a flow which possesses many scales of motion. Tellus, 21, 289387, doi:10.1111/j.2153-3490.1969.tb00444.x.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1982: Atmospheric predictability experiments with a large numerical-model. Tellus, 34, 505513, doi:10.1111/j.2153-3490.1982.tb01839.x.

    • Search Google Scholar
    • Export Citation
  • Madden, R. A., and P. R. Julian, 1971: Detection of a 40–50 day oscillation in the zonal wind in the tropical Pacific. J. Atmos. Sci., 28, 702708, doi:10.1175/1520-0469(1971)028<0702:DOADOI>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • McMurdie, L. A., and B. Ancell, 2014: Predictability characteristics of landfalling cyclones along the North American west coast. Mon. Wea. Rev., 142, 301319, doi:10.1175/MWR-D-13-00141.1.

    • Search Google Scholar
    • Export Citation
  • Montgomery, M. T., and Coauthors, 2012: The Pre-Depression Investigation of Cloud Systems in the Tropics (PREDICT) experiment: Scientific basis, new analysis tools, and some first results. Bull. Amer. Meteor. Soc., 93, 153172, doi:10.1175/BAMS-D-11-00046.1.

    • Search Google Scholar
    • Export Citation
  • Murphy, A. H., 1988: Skill scores based on the mean square error and their relationships to the correlation coefficient. Mon. Wea. Rev., 116, 24172424, doi:10.1175/1520-0493(1988)116<2417:SSBOTM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., 1996: Predictability of the atmosphere and oceans: From days to decades. Decadal Climate Variability: Dynamics and Predictability, D. L. T. Anderson and J. Willebrand, Eds., NATO ASI Series, Vol. I, No. 44, Springer, 83–155.

  • Palmer, T. N., R. Gelaro, J. Barkmeijer, and R. Buizza, 1998: Singular vectors, metrics, and adaptive observations. J. Atmos. Sci., 55, 633653, doi:10.1175/1520-0469(1998)055<0633:SVMAAO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., R. Buizza, F. Doblas-Reyes, T. Jung, M. Leutbecher, G. J. Shutts, M. Steinheimer, and A. Weisheimer, 2009: Stochastic parametrization and model uncertainty. ECMWF Tech. Memo. 598, ECMWF, Reading, United Kingdom, 42 pp.

  • Schneider, T., and S. M. Griffies, 1999: A conceptual framework for predictability studies. J. Climate, 12, 31333155, doi:10.1175/1520-0442(1999)012<3133:ACFFPS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Schneider, T., and I. M. Held, 2001: Discriminants of twentieth-century changes in earth surface temperatures. J. Climate, 14, 249254, doi:10.1175/1520-0442(2001)014<0249:LDOTCC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Schumacher, A. B., M. DeMaria, and J. A. Knaff, 2009: Objective estimation of 24-h probability of tropical cyclone formation. Wea. Forecasting, 24, 456471, doi:10.1175/2008WAF2007109.1.

    • Search Google Scholar
    • Export Citation
  • Shieh, O. H., and S. J. Colucci, 2010: Local minimum of tropical cyclogenesis in the eastern Caribbean. Bull. Amer. Meteor. Soc., 91, 185196, doi:10.1175/2009BAMS2822.1.

    • Search Google Scholar
    • Export Citation
  • Shukla, J., 1981: Dynamical predictability of monthly means. J. Atmos. Sci., 38, 25472572, doi:10.1175/1520-0469(1981)038<2547:DPOMM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Shukla, J., 1985: Predictability. Advances in Geophysics, Vol. 28b, Academic Press, 87–122.

  • Snyder, A., Z. Pu, and Y. Zhu, 2010: Tracking and verification of the east Atlantic tropical cyclone genesis in NCEP global ensemble: Case studies during NASA African monsoon multi-disciplinary analyses. Wea. Forecasting, 25, 13971411, doi:10.1175/2010WAF2222332.1.

    • Search Google Scholar
    • Export Citation
  • Snyder, A., Z. Pu, and C. A. Reynolds, 2011: Impact of stochastic convection on ensemble forecasts of tropical cyclone development. Mon. Wea. Rev., 139, 620626, doi:10.1175/2010MWR3341.1.

    • Search Google Scholar
    • Export Citation
  • Stephenson, D., and F. Doblas-Reyes, 2000: Statistical methods for interpreting Monte Carlo ensemble forecasts. Tellus, 52A, 300322, doi:10.1034/j.1600-0870.2000.d01-5.x.

    • Search Google Scholar
    • Export Citation
  • Tsai, H.-C., R. L. Elsberry, M. S. Jordan, and F. Vitart, 2013: Objective verifications and false alarm analyses of western North Pacific tropical cyclone event forecasts by the ECMWF 32-day ensemble. Asia-Pac. J. Atmos. Sci., 49, 409420, doi:10.1007/s13143-013-0038-6.

    • Search Google Scholar
    • Export Citation
  • Ventrice, M. J., M. C. Wheeler, H. H. Hendon, C. J. Schreck III, C. D. Thorncroft, and G. N. Kiladis, 2013: A modified multivariate Madden Julian Oscillation index using velocity potential. Mon. Wea. Rev., 141, 4197–4210, doi:10.1175/MWR-D-12-00327.1.

    • Search Google Scholar
    • Export Citation
  • Waliser, D. E., W. Stern, S. Schubert, and K. M. Lau, 2003: Dynamic predictability of intraseasonal variability associated with the Asian summer monsoon. Quart. J. Roy. Meteor. Soc., 129, 28972925, doi:10.1256/qj.02.51.

    • Search Google Scholar
    • Export Citation
  • Wang, X. G., and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 11401158, doi:10.1175/1520-0469(2003)060<1140:ACOBAE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Zhang, F., N. Bei, R. Rotunno, C. Snyder, and C. C. Epifanio, 2007: Mesoscale predictability of moist baroclinic waves: Convection-permitting experiments and multistage error growth dynamics. J. Atmos. Sci., 64, 35793594, doi:10.1175/JAS4028.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 687 104 8
PDF Downloads 204 46 5