Advancing Tropical Cyclone Precipitation Forecast Verification Methods and Tools

Kathryn M. Newman aNational Center for Atmospheric Research, Boulder, Colorado
cDevelopmental Testbed Center, Boulder, Colorado

Search for other papers by Kathryn M. Newman in
Current site
Google Scholar
PubMed
Close
,
Barbara Brown aNational Center for Atmospheric Research, Boulder, Colorado
cDevelopmental Testbed Center, Boulder, Colorado

Search for other papers by Barbara Brown in
Current site
Google Scholar
PubMed
Close
,
John Halley Gotway aNational Center for Atmospheric Research, Boulder, Colorado
cDevelopmental Testbed Center, Boulder, Colorado

Search for other papers by John Halley Gotway in
Current site
Google Scholar
PubMed
Close
,
Ligia Bernardet bGlobal Systems Laboratory, National Oceanic and Atmospheric Administration, Boulder, Colorado
cDevelopmental Testbed Center, Boulder, Colorado

Search for other papers by Ligia Bernardet in
Current site
Google Scholar
PubMed
Close
,
Mrinal Biswas aNational Center for Atmospheric Research, Boulder, Colorado
cDevelopmental Testbed Center, Boulder, Colorado

Search for other papers by Mrinal Biswas in
Current site
Google Scholar
PubMed
Close
,
Tara Jensen aNational Center for Atmospheric Research, Boulder, Colorado
cDevelopmental Testbed Center, Boulder, Colorado

Search for other papers by Tara Jensen in
Current site
Google Scholar
PubMed
Close
, and
Louisa Nance aNational Center for Atmospheric Research, Boulder, Colorado
cDevelopmental Testbed Center, Boulder, Colorado

Search for other papers by Louisa Nance in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Tropical cyclone (TC) forecast verification techniques have traditionally focused on track and intensity, as these are some of the most important characteristics of TCs and are often the principal verification concerns of operational forecast centers. However, there is a growing need to verify other aspects of TCs as process-based validation techniques may be increasingly necessary for further track and intensity forecast improvements as well as improving communication of the broad impacts of TCs including inland flooding from precipitation. Here we present a set of TC-focused verification methods available via the Model Evaluation Tools (MET) ranging from traditional approaches to the application of storm-centric coordinates and the use of feature-based verification of spatially defined TC objects. Storm-relative verification using observed and forecast tracks can be useful for identifying model biases in precipitation accumulation in relation to the storm center. Using a storm-centric cylindrical coordinate system based on the radius of maximum wind adds additional storm-relative capabilities to regrid precipitation fields onto cylindrical or polar coordinates. This powerful process-based model diagnostic and verification technique provides a framework for improved understanding of feedbacks between forecast tracks, intensity, and precipitation distributions. Finally, object-based verification including land masking capabilities provides even more nuanced verification options. Precipitation objects of interest, either the central core of TCs or extended areas of rainfall after landfall, can be identified, matched to observations, and quickly aggregated to build meaningful spatial and summary verification statistics.

© 2023 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Kathryn M. Newman, knewman@ucar.edu

Abstract

Tropical cyclone (TC) forecast verification techniques have traditionally focused on track and intensity, as these are some of the most important characteristics of TCs and are often the principal verification concerns of operational forecast centers. However, there is a growing need to verify other aspects of TCs as process-based validation techniques may be increasingly necessary for further track and intensity forecast improvements as well as improving communication of the broad impacts of TCs including inland flooding from precipitation. Here we present a set of TC-focused verification methods available via the Model Evaluation Tools (MET) ranging from traditional approaches to the application of storm-centric coordinates and the use of feature-based verification of spatially defined TC objects. Storm-relative verification using observed and forecast tracks can be useful for identifying model biases in precipitation accumulation in relation to the storm center. Using a storm-centric cylindrical coordinate system based on the radius of maximum wind adds additional storm-relative capabilities to regrid precipitation fields onto cylindrical or polar coordinates. This powerful process-based model diagnostic and verification technique provides a framework for improved understanding of feedbacks between forecast tracks, intensity, and precipitation distributions. Finally, object-based verification including land masking capabilities provides even more nuanced verification options. Precipitation objects of interest, either the central core of TCs or extended areas of rainfall after landfall, can be identified, matched to observations, and quickly aggregated to build meaningful spatial and summary verification statistics.

© 2023 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Kathryn M. Newman, knewman@ucar.edu
Save
  • AghaKouchak, A., A. Behrangi, S. Sorooshian, K. Hsu, and E. Amitai, 2011: Evaluation of satellite-retrieved extreme precipitation rates across the central United States. J. Geophys. Res., 116, D02115, https://doi.org/10.1029/2010JD014741.

    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., and K. E. Mitchell, 1997: The NCEP hourly multisensor U.S. precipitation analysis for operations and GCIP research. Preprints, 13th Conf. on Hydrology, Long Beach, CA, Amer. Meteor. Soc., 5455.

  • Beck, H. E., and Coauthors, 2019: Daily evaluation of 26 precipitation datasets using Stage-IV gauge-radar data for the CONUS. Hydrol. Earth Syst. Sci., 23, 207224, https://doi.org/10.5194/hess-23-207-2019.

    • Search Google Scholar
    • Export Citation
  • Biswas, M., and Coauthors, 2018: Hurricane Weather Research and Forecasting (HWRF) Model: 2017 scientific documentation. NCAR Tech. Note NCAR/TN-544+STR, 111 pp., https://doi.org/10.5065/D6MK6BPR.

  • Brown, B., and Coauthors, 2021: The Model Evaluation Tools (MET): More than a decade of community-supported forecast verification. Bull. Amer. Meteor. Soc., 102, E782E807, https://doi.org/10.1175/BAMS-D-19-0093.1.

    • Search Google Scholar
    • Export Citation
  • Bullock, R. G., B. G. Brown, and T. L. Fowler, 2016: Method for object-based diagnostic evaluation. NCAR Tech. Note NCAR/TN-532+STR, 84 pp., https://doi.org/10.5065/D61V5CBS.

  • Chen, Y., E. E. Ebert, N. E. Davidson, and K. J. E. Walsh, 2018: Application of Contiguous Rain Area (CRA) methods to tropical cyclone rainfall forecast verification. Earth Space Sci., 5, 736752, https://doi.org/10.1029/2018EA000412.

    • Search Google Scholar
    • Export Citation
  • Cheung, K., and Coauthors, 2018: Recent advances in research and forecasting of tropical cyclone rainfall. Trop. Cyclone Res. Rev., 7, 106127, https://doi.org/10.6057/2018TCRR02.03.

    • Search Google Scholar
    • Export Citation
  • Clark, A. J., R. G. Bullock, T. L. Jensen, M. Xue, and F. Kong, 2014: Application of object-based time-domain diagnostics for tracking precipitation systems in convection–allowing models. Wea. Forecasting, 29, 517542, https://doi.org/10.1175/WAF-D-13-00098.1.

    • Search Google Scholar
    • Export Citation
  • Clark, P., N. Roberts, H. Lean, S. P. Ballard, and C. Charlton‐Perez, 2016: Convection‐permitting models: A step‐change in rainfall forecasting. Meteor. Appl., 23, 165181, https://doi.org/10.1002/met.1538.

    • Search Google Scholar
    • Export Citation
  • Davis, C. A., B. G. Brown, and R. G. Bullock, 2006: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 17721784, https://doi.org/10.1175/MWR3145.1.

    • Search Google Scholar
    • Export Citation
  • DeMaria, M., J. A. Knaff, and B. H. Connell, 2001: A tropical cyclone genesis parameter for the tropical Atlantic. Wea. Forecasting, 16, 219233, https://doi.org/10.1175/1520-0434(2001)016<0219:ATCGPF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and J. McBride, 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239, 179202, https://doi.org/10.1016/S0022-1694(00)00343-7.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and W. A. Gallus, 2009: Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Wea. Forecasting, 24, 14011415, https://doi.org/10.1175/2009WAF2222252.1.

    • Search Google Scholar
    • Export Citation
  • ECMWF, 2022: Implementation of IFS Cycle 48r1. ECMWF, accessed 8 March 2022, https://confluence.ecmwf.int/display/FCST/Implementation+of+IFS+Cycle+48r1.

  • Franklin, J. L., C. J. McAdie, and M. B. Lawrence, 2003: Trends in track forecasting for tropical cyclones threatening the United States, 1970–2001. Bull. Amer. Meteor. Soc., 84, 11971204, https://doi.org/10.1175/BAMS-84-9-1197.

    • Search Google Scholar
    • Export Citation
  • Gall, R., J. Franklin, F. Marks, E. N. Rappaport, and F. Toepfer, 2013: The Hurricane Forecast Improvement Project. Bull. Amer. Meteor. Soc., 94, 329343, https://doi.org/10.1175/BAMS-D-12-00071.1.

    • Search Google Scholar
    • Export Citation
  • Gilleland, E., D. Ahijevych, B. Brown, B. Casati, and E. Ebert, 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 14161430, https://doi.org/10.1175/2009WAF2222269.1.

    • Search Google Scholar
    • Export Citation
  • Goerss, J. S., 2000: Tropical cyclone track forecasts using an ensemble of dynamical models. Mon. Wea. Rev., 128, 11871193, https://doi.org/10.1175/1520-0493(2000)128<1187:TCTFUA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Haddad, Z. S., E. A. Smith, C. D. Kummerow, T. Iguchi, M. R. Farrar, S. L. Durden, M. Alves, and W. S. Olson, 1997: The TRMM “day-1” radar/radiometer combined rain-profiling algorithm. J. Meteor. Soc. Japan, 75, 799809, https://doi.org/10.2151/jmsj1965.75.4_799.

    • Search Google Scholar
    • Export Citation
  • Haiden, T., B. Casati, C. Cohelo, E. Gilleland, R. Ashrit, M. Dorninger, and C. Marsigli, 2019: Process-oriented verification. WMO WWRP Joint Working Group for Forecast Verification Research, 18 pp., http://wgne.meteoinfo.ru/wp-content/uploads/2019/05/ProcessVerif_JWGFVR_final.pdf.

  • Halley Gotway, J., and Coauthors, 2021: The MET version 10.0.1 user’s guide. Developmental Testbed Center, accessed 7 April 2022, https://github.com/dtcenter/MET/releases.

  • Halperin, D. J., H. E. Fuelberg, R. E. Hart, J. H. Cossuth, P. Sura, and R. J. Pasch, 2013: An evaluation of tropical cyclone genesis forecasts from global numerical models. Wea. Forecasting, 28, 14231445, https://doi.org/10.1175/WAF-D-13-00008.1.

    • Search Google Scholar
    • Export Citation
  • Halperin, D. J., R. E. Hart, H. E. Fuelberg, and J. H. Cossuth, 2017: The development and evaluation of a statistical–dynamical tropical cyclone genesis guidance tool. Wea. Forecasting, 32, 2746, https://doi.org/10.1175/WAF-D-16-0072.1.

    • Search Google Scholar
    • Export Citation
  • Heming, J. T., 2017: Tropical cyclone tracking and verification techniques for Met Office numerical weather prediction models. Meteor. Appl., 24, 18, https://doi.org/10.1002/met.1599.

    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., 2006: Satellite-based estimation of precipitation using microwave sensors. Encyclopedia of Hydrological Sciences, M. G. Anderson, Ed., John Wiley & Sons, 965–980.

  • Huffman, G. J., and Coauthors, 2020: Integrated multi-satellite retrievals for the Global Precipitation Measurement (GPM) Mission (IMERG). Satellite Precipitation Measurement, M. Stoffel and W. Cramer, Eds., Advances in Global Change Research, Vol. 67, Springer, 343–353, https://doi.org/10.1007/978-3-030-24568-9_19.

  • Jarvinen, B. R., C. J. Neumann, and M. A. S. Davis, 1984: A tropical cyclone data tape for the North Atlantic basin, 1886–1983: Contents, limitations, and uses. NOAA Tech. Memo. NWS NHC 22, 24 pp., https://repository.library.noaa.gov/view/noaa/7069.

  • Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Kim, D., and Coauthors, 2018: Process-oriented diagnosis of tropical cyclones in high-resolution GCMs. J. Climate, 31, 16851702, https://doi.org/10.1175/JCLI-D-17-0269.1.

    • Search Google Scholar
    • Export Citation
  • Landsea, C. W., and J. L. Franklin, 2013: Atlantic hurricane database uncertainty and presentation of a new database format. Mon. Wea. Rev., 141, 35763592, https://doi.org/10.1175/MWR-D-12-00254.1.

    • Search Google Scholar
    • Export Citation
  • Lin, Y., and K. E. Mitchell, 2005: The NCEP Stage II/IV hourly precipitation analyses: Development and applications. Preprints, 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 1.2, https://ams.confex.com/ams/pdfpapers/83847.pdf.

  • Luitel, B., G. Villarini, and G. A. Vecchi, 2018: Verification of the skill of numerical weather prediction models in forecasting rainfall from U.S. landfalling tropical cyclones. J. Hydrol., 556, 10261037, https://doi.org/10.1016/j.jhydrol.2016.09.019.

    • Search Google Scholar
    • Export Citation
  • Marchok, T., 2021: Important factors in the tracking of tropical cyclones in operational models. J. Appl. Meteor. Climatol., 60, 12651284, https://doi.org/10.1175/JAMC-D-20-0175.1.

    • Search Google Scholar
    • Export Citation
  • Marchok, T., R. Rogers, and R. Tuleya, 2007: Validation schemes for tropical cyclone quantitative precipitation forecasts: Evaluation of operational models for U.S. landfalling cases. Wea. Forecasting, 22, 726746, https://doi.org/10.1175/WAF1024.1.

    • Search Google Scholar
    • Export Citation
  • Meléndez‐Landaverde, E. R., M. Werner, and J. Verkade, 2020: Exploring protective decision‐making in the context of impact‐based flood warnings. J. Flood Risk Manage., 13, e12587, https://doi.org/10.1111/jfr3.12587.

    • Search Google Scholar
    • Export Citation
  • Miller, R. J., A. J. Schrader, C. R. Sampson, and T. L. Tsui, 1990: The Automated Tropical Cyclone Forecasting System (ATCF). Wea. Forecasting, 5, 653660, https://doi.org/10.1175/1520-0434(1990)005<0653:TATCFS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Morrow, B. H., and J. K. Lazo, 2015: Effective tropical cyclone forecast and warning communication: Recent social science contributions. Trop. Cyclone Res. Rev., 4, 3848, https://doi.org/10.6057/2015TCRR01.05.

    • Search Google Scholar
    • Export Citation
  • National Hurricane Center, 2022: National Hurricane Center verification. Accessed 8 March 2022, https://www.nhc.noaa.gov/verification/.

  • Qi, W., B. Yong, and J. Gourley, 2021: Monitoring the super typhoon Lekima by GPM-based near-real-time satellite precipitation estimates. J. Hydrol., 603, 126968, https://doi.org/10.1016/j.jhydrol.2021.126968.

    • Search Google Scholar
    • Export Citation
  • Rappaport, E. N., 2014: Fatalities in the United States from Atlantic tropical cyclones: New data and interpretation. Bull. Amer. Meteor. Soc., 95, 341346, https://doi.org/10.1175/BAMS-D-12-00074.1.

    • Search Google Scholar
    • Export Citation
  • Rappaport, E. N., and Coauthors, 2009: Advances and challenges at the National Hurricane Center. Wea. Forecasting, 24, 395419, https://doi.org/10.1175/2008WAF2222128.1

    • Search Google Scholar
    • Export Citation
  • Roberts, N. M., 2008: Assessing the spatial and temporal variation in the skill of precipitation forecasts from an NWP model. Meteor. Appl., 15, 163169, https://doi.org/10.1002/met.57.

    • Search Google Scholar
    • Export Citation
  • Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 7897, https://doi.org/10.1175/2007MWR2123.1.

    • Search Google Scholar
    • Export Citation
  • Sampson, C. R., J. L. Franklin, J. A. Knaff, and M. DeMaria, 2008: Experiments with a simple tropical cyclone intensity consensus. Wea. Forecasting, 23, 304312, https://doi.org/10.1175/2007WAF2007028.1.

    • Search Google Scholar
    • Export Citation
  • Villarini, G., J. A. Smith, M. L. Baeck, T. Marchok, and G. A. Vecchi, 2011: Characterization of rainfall distribution and flooding associated with U.S. landfalling tropical cyclones: Analyses of Hurricanes Frances, Ivan, and Jeanne (2004). J. Geophys. Res., 116, D23116, https://doi.org/10.1029/2011JD016175.

    • Search Google Scholar
    • Export Citation
  • Wilks, D., 2019: Statistical Methods in the Atmospheric Sciences. 4th ed. Elsevier, 840 pp.

  • Win-Gildenmeister, M., and Coauthors, 2021: The METplus version 4.0.0 user’s guide. Developmental Testbed Center, 1405 pp., https://metplus.readthedocs.io/_/downloads/en/v4.0.0/pdf/.

  • Wolff, J. K., M. Harrold, T. Fowler, J. H. Gotway, L. Nance, and B. G. Brown, 2014: Beyond the basics: Evaluating model-based precipitation forecasts using traditional, spatial, and object-based methods. Wea. Forecasting, 29, 14511472, https://doi.org/10.1175/WAF-D-13-00135.1.

    • Search Google Scholar
    • Export Citation
  • Wright, D. B., D. B. Kirschbaum, and S. Yatheendradas, 2017: Satellite precipitation characterization, error modeling, and error correction using censored shifted gamma distributions. J. Hydrometeor., 18, 28012815, https://doi.org/10.1175/JHM-D-17-0060.1.

    • Search Google Scholar
    • Export Citation
  • Yang, F., and Coauthors, 2020: GFSV16: Further advancements to the UFS medium range weather application in 2021. Unified Forecast System (UFS) Users’ Workshop, online, Developmental Testbed Center (DTC), 17 pp., https://dtcenter.org/sites/default/files/events/2020/1-fanglin-yang.pdf.

  • Yu, Z., Y. Wang, and H. Xu, 2015: Observed rainfall asymmetry in tropical cyclones making landfall over China. J. Appl. Meteor. Climatol., 54, 117136, https://doi.org/10.1175/JAMC-D-13-0359.1.

    • Search Google Scholar
    • Export Citation
  • Yu, Z., Y. J. Chen, B. Ebert, N. E. Davidson, Y. Xiao, H. Yu, and Y. Duan, 2020: Benchmark rainfall verification of landfall tropical cyclone forecasts by operational ACCESS-TC over China. Meteor. Appl., 27, e1842, https://doi.org/10.1002/met.1842.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 369 369 14
Full Text Views 247 247 20
PDF Downloads 207 207 17