Editorial Type:
Article
Article Type:
Research Article

Validation Schemes for Tropical Cyclone Quantitative Precipitation Forecasts: Evaluation of Operational Models for U.S. Landfalling Cases

Timothy Marchok NOAA/Geophysical Fluid Dynamics Laboratory, Princeton, New Jersey

Search for other papers by Timothy Marchok in
Current site
Google Scholar
PubMed Close
,
Robert Rogers NOAA/AOML/Hurricane Research Division, Miami, Florida

Search for other papers by Robert Rogers in
Current site
Google Scholar
PubMed Close
, and
Robert Tuleya SAIC at NOAA/NWS/Environmental Modeling Center, Norfolk, Virginia

Search for other papers by Robert Tuleya in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Aug 2007
DOI:
https://doi.org/10.1175/WAF1024.1
Page(s):
726–746
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

A scheme for validating quantitative precipitation forecasts (QPFs) for landfalling tropical cyclones is developed and presented here. This scheme takes advantage of the unique characteristics of tropical cyclone rainfall by evaluating the skill of rainfall forecasts in three attributes: the ability to match observed rainfall patterns, the ability to match the mean value and volume of observed rainfall, and the ability to produce the extreme amounts often observed in tropical cyclones. For some of these characteristics, track-relative analyses are employed that help to reduce the impact of model track forecast error on QPF skill. These characteristics are evaluated for storm-total rainfall forecasts of all U.S. landfalling tropical cyclones from 1998 to 2004 by the NCEP operational models, that is, the Global Forecast System (GFS), the Geophysical Fluid Dynamics Laboratory (GFDL) hurricane model, and the North American Mesoscale (NAM) model, as well as the benchmark Rainfall Climatology and Persistence (R-CLIPER) model. Compared to R-CLIPER, all of the numerical models showed comparable or greater skill for all of the attributes. The GFS performed the best of all of the models for each of the categories. The GFDL had a bias of predicting too much heavy rain, especially in the core of the tropical cyclones, while the NAM predicted too little of the heavy rain. The R-CLIPER performed well near the track of the core, but it predicted much too little rain at large distances from the track. Whereas a primary determinant of tropical cyclone QPF errors is track forecast error, possible physical causes of track-relative differences lie with the physical parameterizations and initialization schemes for each of the models. This validation scheme can be used to identify model limitations and biases and guide future efforts toward model development and improvement.

Corresponding author address: Timothy Marchok, NOAA/Geophysical Fluid Dynamics Laboratory, 201 Forrestal Rd., Princeton, NJ 08540-6649. Email: timothy.marchok@noaa.gov

Abstract

A scheme for validating quantitative precipitation forecasts (QPFs) for landfalling tropical cyclones is developed and presented here. This scheme takes advantage of the unique characteristics of tropical cyclone rainfall by evaluating the skill of rainfall forecasts in three attributes: the ability to match observed rainfall patterns, the ability to match the mean value and volume of observed rainfall, and the ability to produce the extreme amounts often observed in tropical cyclones. For some of these characteristics, track-relative analyses are employed that help to reduce the impact of model track forecast error on QPF skill. These characteristics are evaluated for storm-total rainfall forecasts of all U.S. landfalling tropical cyclones from 1998 to 2004 by the NCEP operational models, that is, the Global Forecast System (GFS), the Geophysical Fluid Dynamics Laboratory (GFDL) hurricane model, and the North American Mesoscale (NAM) model, as well as the benchmark Rainfall Climatology and Persistence (R-CLIPER) model. Compared to R-CLIPER, all of the numerical models showed comparable or greater skill for all of the attributes. The GFS performed the best of all of the models for each of the categories. The GFDL had a bias of predicting too much heavy rain, especially in the core of the tropical cyclones, while the NAM predicted too little of the heavy rain. The R-CLIPER performed well near the track of the core, but it predicted much too little rain at large distances from the track. Whereas a primary determinant of tropical cyclone QPF errors is track forecast error, possible physical causes of track-relative differences lie with the physical parameterizations and initialization schemes for each of the models. This validation scheme can be used to identify model limitations and biases and guide future efforts toward model development and improvement.

Corresponding author address: Timothy Marchok, NOAA/Geophysical Fluid Dynamics Laboratory, 201 Forrestal Rd., Princeton, NJ 08540-6649. Email: timothy.marchok@noaa.gov

Save
Cover Weather and Forecasting
Weather and Forecasting

  • Aberson, S. D., 1998: Five-day tropical cyclone track forecasts in the North Atlantic basin. Wea. Forecasting, 13 , 1005–1015.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Aberson, S. D., 2001: The ensemble of tropical cyclone track forecasting models in the North Atlantic Basin (1976–2000). Bull. Amer. Meteor. Soc., 82 , 1895–1904.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Accadia, C., Mariani S. , Casaioli M. , Lavagnini A. , and Speranza A. , 2003: Sensitivity of precipitation forecast skill scores to bilinear interpolation and a simple nearest-neighbor average method on high-resolution verification grids. Wea. Forecasting, 18 , 918–932.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Atallah, E. H., and Bosart L. F. , 2003: The extratropical transition and precipitation distribution of Hurricane Floyd (1999). Mon. Wea. Rev., 131 , 1063–1081.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Baldwin, M. E., and Mitchell K. E. , 1997: The NCEP hourly multi-sensor U.S. precipitation analysis for operations and GCIP research. Preprints, 13th Conf. on Hydrology, Long Beach, CA, Amer. Meteor. Soc., 54–55.

  • Bender, M. A., 1997: The effect of relative flow on the asymmetric structure in the interior of hurricanes. J. Atmos. Sci., 54 , 703–724.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Black, M. L., Gamache J. F. , Marks F. D. , Samsury C. E. , and Willoughby H. E. , 2002: Eastern Pacific Hurricanes Jimana of 1991 and Olivia of 1994: The effect of vertical shear on structure and intensity. Mon. Wea. Rev., 130 , 2291–2312.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bosart, L. F., and Lackmann G. M. , 1995: Postlandfall tropical cyclone reintensification in a weakly baroclinic environment: A case study of Hurricane David (September 1979). Mon. Wea. Rev., 123 , 3268–3291.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Braun, S. A., 2002: A cloud-resolving simulation of Hurricane Bob (1991): Storm structure and eyewall buoyancy. Mon. Wea. Rev., 130 , 1573–1592.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Colle, B. A., 2003: Numerical simulations of the extratropical transition of Floyd (1999): Structural evolution and responsible mechanisms for the heavy rainfall over the northeast United States. Mon. Wea. Rev., 131 , 2905–2926.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Corbosiero, K. L., and Molinari J. , 2002: The effects of vertical wind shear on the distribution of convection in tropical cyclones. Mon. Wea. Rev., 130 , 2110–2123.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • DeMaria, M., and Tuleya R. E. , 2001: Evaluation of quantitative precipitation forecasts from the GFDL hurricane model. Preprints, Symp. on Precipitation Extremes: Prediction, Impacts, and Responses, Albequerque, NM, Amer. Meteor. Soc., 340–343.

  • DeMaria, M., and Gross J. M. , 2003: Evolution of tropical cyclone forecast models. Hurricane! Coping with Disaster, R. Simpson, Ed., Amer. Geophys. Union, 103–126.

    • Search Google Scholar
    • Export Citation

  • DeMaria, M., Mainelli M. , Shay L. K. , Knaff J. A. , and Kaplan J. , 2005: Further improvements to the Statistical Hurricane Intensity Prediction Scheme (SHIPS). Wea. Forecasting, 20 , 531–543.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • DiMego, G. J., and Bosart L. F. , 1982: The transformation of Tropical Storm Agnes into an extratropical cyclone. Part I: The observed fields and vertical motion computations. Mon. Wea. Rev., 110 , 385–411.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ebert, E. E., and McBride J. L. , 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239 , 179–202.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ebert, E. E., Kusselson S. , and Turk M. , 2005: Validation of Tropical Rainfall Potential (TRaP) forecasts for Australian tropical cyclones. Aust. Meteor. Mag., 54 , 121–136.

    • Search Google Scholar
    • Export Citation

  • Ferraro, R., and Coauthors, 2005: The Tropical Rainfall Potential (TRaP) Technique. Part II: Validation. Wea. Forecasting, 20 , 465–475.

  • Frank, W. M., and Ritchie E. A. , 2001: Effects of vertical wind shear on the intensity and structure of numerically simulated hurricanes. Mon. Wea. Rev., 129 , 2249–2269.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Franklin, J. L., McAdie C. J. , and Lawrence M. B. , 2003: Trends in track forecasting for tropical cyclones threatening the United States, 1970–2001. Bull. Amer. Meteor. Soc., 84 , 1197–1203.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Fulton, R. A., Breidenbach J. P. , Seo D-J. , Miller D. A. , and O’Bannon T. , 1998: The WSR-88D rainfall algorithm. Wea. Forecasting, 13 , 377–395.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Gallus W. A. Jr., , 1999: Eta simulations of three extreme precipitation events: Sensitivity to resolution and convective parameterization. Wea. Forecasting, 14 , 405–426.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Gallus W. A. Jr., , 2002: Impact of verification grid-box size on warm-season QPF skill measures. Wea. Forecasting, 17 , 1296–1302.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Grell, G. A., Dudhia J. , and Stauffer D. R. , 1994: A description of the fifth-generation Penn State/NCAR Mesoscale Model (MM5). NCAR Tech. Note NCAR/TN-398+STR, 138 pp.

  • Janjić, Z., 1994: The step-mountain Eta coordinate model: Further developments of the convection, viscous sublayer, and turbulence closure schemes. Mon. Wea. Rev., 122 , 927–945.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Jarvinen, B. R., and Neumann C. J. , 1979: Statistical forecasts of tropical cyclone intensity for the North Atlantic basin. NOAA Tech. Memo. NWS NHC-10, 22 pp. [Available from National Technical Information Service, 5285 Port Royal Rd., Springfield, VA 22161.].

  • Jones, S. C., 2000: The evolution of vortices in vertical shear: III: Baroclinic vortices. Quart. J. Roy. Meteor. Soc., 126 , 3161–3185.

  • Kidder, S. Q., Kusselson S. J. , Knaff J. A. , Ferraro R. R. , Kuligowski R. J. , and Turk M. , 2005: The Tropical Rainfall Potential (TRaP) technique. Part I: Description and examples. Wea. Forecasting, 20 , 456–464.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Klein, P. M., Harr P. A. , and Elsberry R. L. , 2000: Extratropical transition of western North Pacific tropical cyclones: An overview and conceptual model of the transformation stage. Wea. Forecasting, 15 , 373–396.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Knaff, J. A., DeMaria M. , Sampson C. R. , and Gross J. M. , 2003: Statistical, 5-day tropical cyclone intensity forecasts derived from climatology and persistence. Wea. Forecasting, 18 , 80–92.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lin, Y-L., Chiao S. , Wang T-A. , Kaplan M. L. , and Weglarz R. P. , 2001: Some common ingredients for heavy orographic rainfall. Wea. Forecasting, 16 , 633–660.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lonfat, M., Marks F. D. Jr., and Chen S. S. , 2004: Precipitation distribution in tropical cyclones using the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager: A global perspective. Mon. Wea. Rev., 132 , 1645–1660.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Marks, F. D., Kappler G. , and DeMaria M. , 2002: Development of a tropical cyclone rainfall climatology and persistence (R-CLIPER) model. Preprints, 25th Conf. on Hurricanes and Tropical Meteorology, San Diego, CA, Amer. Meteor. Soc., 327–328.

  • Mesinger, F., 1998: Quantitative precipitation forecasts of the “early” Eta model: An update. Preprints, 16th Conf. on Weather Analysis and Forecasting, Phoenix, AZ, Amer. Meteor. Soc., 184–186.

  • Moorthi, S., Pan H-L. , and Caplan P. , 2001: Changes to the 2001 NCEP operational MRF/AVN Global Analysis/Forecast System. NCEP Tech. Procedures Bull. 484, 14 pp. [Available from NCEP/EMC, W/NP23, World Weather Building, Washington, DC 20233; also available online at http://www.nws.noaa.gov/om/tpb/484.htm.].

  • Neumann, C. J., 1972: An alternate to the Hurran (hurricane analog) tropical cyclone forecasting system. NOAA Tech. Memo. NWS SR-62, 23 pp. [Available from National Technical Information Service, 5285 Port Royal Rd., Springfield, VA 22161.].

  • Pan, H-L., and Wu W-S. , 1995: Implementing a mass flux convection parameterization package for the NMC Medium Range Forecast Model. NMC Office Note 409, 40 pp. [Available from NCEP/EMC, W/NP23, World Weather Building, Washington, DC 20233.].

  • Pfost, R. L., 2000: Operational tropical cyclone quantitative precipitation forecasting. Natl. Wea. Dig., 24 (1–2) , 61–66.

  • Rappaport, E. N., 2000: Loss of life in the United States associated with recent Atlantic tropical cyclones. Bull. Amer. Meteor. Soc., 81 , 2065–2074.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ritchie, E. A., and Elsberry R. L. , 2001: Simulations of the transformation stage of the extratropical transition of tropical cyclones. Mon. Wea. Rev., 129 , 1462–1480.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Rogers, R. F., Chen S. S. , Tenerelli J. E. , and Willoughby H. E. , 2003: A numerical study of the impact of vertical shear on the distribution of rainfall in Hurricane Bonnie (1998). Mon. Wea. Rev., 131 , 1577–1599.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Rogers, R. F., Black M. , Chen S. S. , and Black R. A. , 2007: An evaluation of microphysics fields from mesoscale model simulations of tropical cyclones. Part I: Comparisons with observations. J. Atmos. Sci., 64 , 1811–1834.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5 , 570–575.

  • Shapiro, L. J., 1983: The asymmetric boundary layer flow under a translating hurricane. J. Atmos. Sci., 40 , 1984–1998.

  • Tuleya, R. E., DeMaria M. , and Kuligowski R. , 2007: Evaluation of GFDL and simple statistical model rainfall forecasts for U.S. landfalling tropical storms. Wea. Forecasting, 22 , 56–70.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Tustison, B., Harris D. , and Foufoula-Georgiou E. , 2001: Scale issues in verification of precipitation forecasts. J. Geophys. Res., 106 , 11775–11784.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Wu, C-C., Yen T-H. , Kuo Y-H. , and Wang W. , 2002: Rainfall simulation associated with Typhoon Herb (1996) near Taiwan. Part I: The topographic effect. Wea. Forecasting, 17 , 1001–1015.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Zhang, D-L., Liu Y. , and Yau M. K. , 2000: A multiscale numerical study of Hurricane Andrew (1992). Part III: Dynamically induced vertical motion. Mon. Wea. Rev., 128 , 3772–3788.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1153 510 167
PDF Downloads 686 193 20