• Burnash, R. J. C., 1995: The National Weather Service River Forecast System—Catchment modeling. Computer Models of Watershed Hydrology, V. P. Singh, Ed., Water Resources Publications, 311–366.

  • Clark, R. A., J. J. Gourley, Z. L. Flamig, Y. Hong, and E. Clark, 2014: CONUS-wide evaluation of National Weather Service flash flood guidance products. Wea. Forecasting, 29, 377392, https://doi.org/10.1175/WAF-D-12-00124.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gourley, J. J., and R. Clark III, 2018: Real-time flash flood forecasting. Oxford Encyclopedia of Natural Hazard Science, Oxford University Press, https://doi.org/10.1093/acrefore/9780199389407.013.298.

    • Crossref
    • Export Citation
  • Gourley, J. J., and Coauthors, 2017: The FLASH project: Improving the tools for flash flood monitoring and prediction across the United States. Bull. Amer. Meteor. Soc., 98, 361372, https://doi.org/10.1175/BAMS-D-15-00247.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herman, G. R., and R. S. Schumacher, 2018: Flash flood verification: Pondering precipitation proxies. J. Hydrometeor., 19, 17531776, https://doi.org/10.1175/JHM-D-18-0092.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koren, V., S. Reed, M. Smith, Z. Zhang, and D.-J. Seo, 2004: Hydrology laboratory Research Modeling System (HL-RMS) of the US National Weather Service. J. Hydrol., 291, 297318, https://doi.org/10.1016/j.jhydrol.2003.12.039.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lincoln, W. S., and R. F. L. Thomason, 2018: A preliminary look at using rainfall average recurrence interval to characterize flash flood events for real-time warning forecasting. J. Oper. Meteor., 6, 1322, https://doi.org/10.15191/nwajom.2018.0602.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Martinaitis, S. M., and Coauthors, 2017: The HMT multi-radar multi-sensor hydro experiment. Bull. Amer. Meteor. Soc., 98, 347359, https://doi.org/10.1175/BAMS-D-15-00283.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Perica, S., S. Pavlovic, M. St. Laurent, C. Trypaluk, D. Unruh, and O. Wilhite, 2018: Version 2.0: Texas. Vol. 11, Precipitation-Frequency Atlas of the United States, NOAA Atlas 14, NOAA/National Weather Service, 283 pp., https://www.weather.gov/media/owp/oh/hdsc/docs/Atlas14_Volume11.pdf.

  • Ryzhkov, Z., M. Diederich, P. Zhang, and C. Simmer, 2014: Potential utilization of specific attenuation for rainfall estimation, mitigation of partial beam blockage, and radar networking. J. Atmos. Oceanic Technol., 31, 599619, https://doi.org/10.1175/JTECH-D-13-00038.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stovern, D. R., J. A. Nelson, S. Czyzyk, M. Klein, K. Landry-Guyton, K. Mattarochia, E. Nipper, and J. W. Zeitler, 2020: The extreme precipitation forecast table: Improving situational awareness when heavy rain is a threat. J. Operat. Meteor., 8, 93104, https://doi.org/10.15191/nwajom.2020.0807.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, J., and Coauthors, 2011: The Coupled Routing and Excess Storage (CREST) distributed hydrological model. Hydrol. Sci. J., 56, 8498, https://doi.org/10.1080/02626667.2010.543087.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, Y., S. Cocks, L. Tang, A. Ryzhkov, P. Zhang, J. Zhang, and K. Howard, 2019: A prototype quantitative precipitation estimation algorithm for operational S-band polarimetric radar utilizing specific attenuation and specific differential phase. Part I: Algorithm description. J. Hydrometeor., 20, 985997, https://doi.org/10.1175/JHM-D-18-0071.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, J., L. Tang, S. Cocks, P. Zhang, A. Ryzhkov, K. Howard, C. Langston, and B. Kaney, 2020: A dual-polarization radar synthetic QPE for operations. J. Hydrometeor., 21, 25072521, https://doi.org/10.1175/JHM-D-19-0194.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    Heat map of exceedances of MRMS QPEs over the 1-yr ARI for an AI of 3 h. The color table and symbology match that of Fig. 5b in HS18, with blue lines outlining River Forecast Center boundaries, black lines indicating NWS county warning area boundaries, and green circles representing radar locations.

  • View in gallery
    Fig. 2.

    Mean ETSs for fixed rainfall thresholds using FFRs. The rainfall source is MRMS QPE and is evaluated across AIs for (a) all regions and all seasons, (b) all regions and cool season, (c) Southwest region and all seasons, and (d) Southeast region and all seasons. Following HS18, the top number of each row label corresponds to the threshold for the 1-h QPE exceedances, the middle number applies to the 3- and 6-h accumulation comparisons, and the bottom number to the 24-h QPEs.

  • View in gallery
    Fig. 3.

    Mean ETSs for ARIs using FFRs. The rainfall source is MRMS QPE and is evaluated across ARIs for (a) all regions and all seasons, (b) all regions and cool season, (c) Southwest region and all seasons, and (d) Southeast region and all seasons.

  • View in gallery
    Fig. 4.

    Mean ETSs for Ratios using FFRs. The rainfall source is MRMS QPE and is evaluated across Ratios for (a) all regions and all seasons, (b) all regions and cool season, (c) Southwest region and all seasons, and (d) Southeast region and all seasons.

  • View in gallery
    Fig. 5.

    Mean ETSs for unit streamflow values from each distributed hydrologic model in FLASH using FFRs. The rainfall source is MRMS QPE and is evaluated across unit streamflow values for (a) all regions and all seasons, (b) all regions and cool season, (c) Southwest region and all seasons, and (d) Southeast region and all seasons.

  • View in gallery
    Fig. 6.

    Subjective guidance developed based on experience from NWS forecasters on the use of the MRMS and FLASH products for operational flash flood warning. Note the unit streamflow values are presented above in units of ft3 s−1 mi2, whereas they are presented in m3 s−1 km−2 in the article.

All Time Past Year Past 30 Days
Abstract Views 385 1 0
Full Text Views 367 252 12
PDF Downloads 373 228 7

Comments on “Flash Flood Verification: Pondering Precipitation Proxies”

Jonathan J. Gourley NOAA/OAR/National Severe Storms Laboratory, Norman, Oklahoma

Search for other papers by Jonathan J. Gourley in
Current site
Google Scholar
PubMed
Close
and
Humberto Vergara Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, Norman, Oklahoma
NOAA/OAR/National Severe Storms Laboratory, Norman, Oklahoma

Search for other papers by Humberto Vergara in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

New operational tools for monitoring flash flooding based on radar quantitative precipitation estimates (QPEs) have become available to U.S. National Weather Service forecasters. Herman and Schumacher examined QPE exceedance thresholds for several tools and compared them to each other, to flash flood reports (FFRs), and to flash flood warnings. The Next Generation Radar network has been updated with dual-polarization capabilities since the publication of Herman and Schumacher, which has changed the characteristics of the derived QPEs. Updated thresholds on Multi-Radar Multi-Sensor version 12 products that are associated to FFRs are provided and thus can be used as guidance by the operational forecasting community and other end-users of the products.

Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/JHM-D-20-0215.s1.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

The original article that was the subject of this comment/reply can be found at http://journals.ametsoc.org/doi/abs/10.1175/JHM-D-18-0092.1.

Corresponding author: Jonathan Gourley, jj.gourley@noaa.gov

Abstract

New operational tools for monitoring flash flooding based on radar quantitative precipitation estimates (QPEs) have become available to U.S. National Weather Service forecasters. Herman and Schumacher examined QPE exceedance thresholds for several tools and compared them to each other, to flash flood reports (FFRs), and to flash flood warnings. The Next Generation Radar network has been updated with dual-polarization capabilities since the publication of Herman and Schumacher, which has changed the characteristics of the derived QPEs. Updated thresholds on Multi-Radar Multi-Sensor version 12 products that are associated to FFRs are provided and thus can be used as guidance by the operational forecasting community and other end-users of the products.

Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/JHM-D-20-0215.s1.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

The original article that was the subject of this comment/reply can be found at http://journals.ametsoc.org/doi/abs/10.1175/JHM-D-18-0092.1.

Corresponding author: Jonathan Gourley, jj.gourley@noaa.gov

1. Introduction

The study of Herman and Schumacher (2018, hereafter HS18) used radar-based quantitative precipitation estimates (QPEs) from different algorithms and compared exceedance thresholds to flash flood reports (FFRs) and flash flood warnings (FFWs). One objective of HS18 was to provide the best practices for using the tools in an operational flash flood monitoring and forecasting environment. The commentary presented herein is motivated by the following points:

  1. QPE algorithms in the Multi-Radar Multi-Sensor (MRMS) system have been updated to incorporate variables from the recent dual-polarization upgrade to the Next Generation Radar (NEXRAD) network (Zhang et al. 2020).

  2. Additional products from the Flooded Locations and Simulated Hydrographs project (FLASH), a component of MRMS, have become operational in the National Weather Service (NWS) and include outputs from distributed hydrologic models (Gourley et al. 2017).

  3. While the study period, number of products, thresholds, and accumulations intervals (AIs) examined in HS18 exceed those studied by others, their study reveals some results that require further examination.

HS18 examined QPEs from three different sources including the rain gauge-adjusted MRMS radar product from January 2015 to June 2017. We consider here the radar-only MRMS product because it is used more ubiquitously in real-time given its low latency (generally less than 1 min) and is updated every 2 min, whereas the gauge-corrected product is generated at the top of the hour. Since the time the HS18 article was published, MRMS was upgraded to version 12, which includes a significant change to the radar-only algorithm. The new algorithm employs a combination of variables including an estimate of the specific attenuation A based on the total differential phase shift ΦDP along the rain path, specific differential phase KDP and measured reflectivity factor Z (Ryzhkov et al. 2014; Wang et al. 2019; Zhang et al. 2020). This change has an impact on the upper tail of the rainfall rate distribution. Zhang et al. (2020) evaluated the legacy Z-based algorithm and compared it to the new dual-polarization synthetic algorithm. They found that in heavy rainfall (with 24-h gauge accumulations greater than 2.66 in.) the mean bias ratio increased from 0.80 to 0.92. In heavy rain with 24-h gauge accumulations greater than 5.45 in., the mean bias ratio increased from 0.81 to 0.95. The new dual-polarization, radar-only MRMS QPE serves as the primary input to the FLASH system and is thus the source of QPE when comparing to fixed rainfall thresholds (FTs) and flash flood guidance (FFG) thresholds, and for deriving average recurrence intervals (ARIs).

This study complements HS18 by incorporating contemporary products from the FLASH system. Specifically, Gourley et al. (2017) described the use of the MRMS radar-only QPE product to force distributed hydrologic models. The three different hydrologic model cores produce forecasts of streamflow values normalized by upstream contributing basin area, i.e., unit streamflow Qu. Values greater than 2 m3 s−1 km−2 were associated to FFRs when examining data across the conterminous United States (CONUS) from 2005 to 2011 (Gourley and Clark 2018). These values agree with the findings based on subjective evaluations from NWS forecasters (Martinaitis et al. 2017). Training materials have been developed by NWS forecasters and trainers, which include an upper category for catastrophic flash flooding (http://training.weather.gov/wdtd/courses/ffawoc/index.php). Values of Qu greater than 10 m3 s−1 km−2 are monitored for possible issuance of high-end flash flood warnings and emergencies that trigger the NWS wireless emergency alerts. The hydrologic model forecasts have the potential to provide early alerting capabilities over urban areas through the consideration of impervious surfaces and for regions downstream of the heavy rainfall via channel routing. It would be informative to quantify the skill associated with the more sophisticated hydrologic model forecasts, identify the Qu thresholds, and compare results to algorithms based on QPE thresholds.

In the closing section of HS18, they make the following select points regarding the optimal use of QPE-based tools for monitoring flash flooding: 1) QPE exceedances corresponded more with FFRs when longer accumulation periods were considered (i.e., 24-h QPE exceedances had better skill than 1-h QPE exceedances); 2) the simplest method of computing a QPE exceedance (i.e., using a fixed rainfall threshold of 2.5 in. day−1) had better correspondence to FFRs than comparisons to more complex thresholds with spatial variability (i.e., ARIs) and to thresholds that varied in both space and time (i.e., FFG); and 3) the optimal ARI was between 1 and 5 years. The authors duly noted the limitations of their results and explored the large geographic dependence of the QPE algorithms and associated exceedance thresholds. Nevertheless, HS18 describe some of the points as “surprising” and “contrary to expectations.”

In general, we would expect better skill with the shorter-term QPE accumulation intervals (AIs) given the definition of flash flooding to be a phenomenon of hydrologic responses within minutes to several hours of the causative rainfall. Clark et al. (2014) quantified the benchmark skill of the QPE comparisons to FFG across the CONUS over a 4-yr period. They concluded that the best skill was associated with the 1- and 3-h accumulation products and the skill declined for the 6-h accumulations. This result was consistent when using FFRs as ground truth as well as using flood stage exceedances in basins instrumented with U.S. Geological Survey stream gauges. This finding is counter to the first HS18 summary point paraphrased above.

To our knowledge, HS18 is the first paper to quantify the relative skill of QPE exceedances over fixed rainfall thresholds and to dynamic ones (i.e., ARIs and FFG). The FFG method employs a hydrologic model to account for the initial soil moisture states and river stages, and the derived rainfall thresholds vary in space and time. NWS forecasters have refined the methods used to derive FFG so that they are optimized on a regional basis (Clark et al. 2014). We would expect these methods to perform better than a fixed rainfall threshold of 2.5 in. day−1. Lincoln and Thomason (2018) examined ARIs to characterize flash flooding for 24 events across the eastern United States. They found that the 2-yr ARI for a 3-h AI captured 90% of their FFRs while a 25-yr ARI was more closely linked to major flooding events. More recently, Stovern et al. (2020) describe an extreme precipitation forecast table developed by NWS forecasters that uses a 100-yr ARI as the default configuration, which is considered as the operationally accepted threshold for defining an extreme precipitation event. The HS18 study highlighted smaller ARIs of 1–5 years as being the optimal thresholds for flash flooding. There are some discrepancies in the guidance for ARI thresholds that correspond to flash flooding, requiring additional investigation.

This study examines flash flooding thresholds from the MRMS and FLASH version 12 product suite to guide operational decision making. Section 2 describes the datasets employed in this study and the methodology for matching gridded products to FFRs in space and time. Results describing the skill metrics of the MRMS and FLASH product suite are presented in section 3. Section 4 summarizes the findings from this study and discusses the similarities and differences with HS18.

2. Materials and methods

This study examines 1) MRMS dual-pol, radar-only QPEs; 2) ARIs of rainfall; 3) ratios of QPE to FFG; and 4) unit streamflow from the Sacramento Soil Moisture Accounting (SAC-SMA) model (Burnash 1995; Koren et al. 2004), the Coupled Routing and Excess Storage (CREST) model (Wang et al. 2011), and a variant of CREST that prohibits infiltration called the hydrophobic (HP) model. We examine AIs ranging from 2-min rainfall rates up to 24 h of accumulated rainfall. Grid cells from the gridded products are matched to collocated FFRs based on local storm reports from the NWS across the CONUS from 31 May 2018 to 1 June 2019.

The v12 MRMS QPEs are computed from the new synthetic algorithm based on dual-polarization radar variables. The ARI products use the MRMS QPEs as inputs and compare them to the gauge-based frequency analysis from NOAA Atlas 14, which now includes the latest Vol. 11 for the state of Texas (Perica et al. 2018). At the time of the study, the frequency analyses had not been completed for the following states: Washington, Oregon, Idaho, Montana, and Wyoming. In these locations, a machine-learning approach was trained to provide estimates of the gauge-based frequencies. These will continue to serve in place of the NOAA Atlas 14 frequencies in the operational version of FLASH for computing ARIs until the maps become available. The QPEs are compared to the static frequency values at each grid point for AIs of 0.5, 1, 3, 6, 12, and 24 h to compute the closest ARI.

The ratio of QPE to FFG (“Ratio” hereafter) is computed for AIs of 1, 3, and 6 h. The FFG product is produced by each NWS River Forecast Center (RFC), transmitted out to local NWS offices, and subsequently mosaicked by the Weather Prediction Center (WPC). The method for computing FFG varies by RFC and more details about its derivation are provided in Clark et al. (2014). The v12 MRMS QPEs are divided by the FFG values at each grid point to compute the Ratio product for the corresponding AI.

The same v12 MRMS QPEs are used to force the three distributed hydrologic models. In this case, the hydrologic models are forced with 2-min precipitation rates and provide forecasts of streamflow and unit streamflow up to 12 h in the future. The maximum unit streamflow within the 12-h time window is considered. There is no need to compute values for different AIs. All gridded, flash flood products are operational and produced on the same 1-km resolution grid across the CONUS.

Flash flood products evaluated herein follow the methodology employed in HS18. This verification method requires pairing the gridded products and observations in space and time, which poses challenges related to the inherent uncertainty in the reports and spatiotemporal displacement between the impacts of flash floods and their causative rainfall. To minimize these deleterious effects, both the gridded products and observations are mapped onto a common grid with 50-km spatial resolution as daily exceedances. This enables a pixel-to-pixel comparison for each day to compute the product-to-observation joint distributions of occurrences associated with different threshold values. This is done for each of the flash flood products for all available AIs. As in HS18, only a single daily exceedance count per pixel is considered regardless of how many exceedances occurred subdaily.

A difference in the methodology between this study and HS18 is in the use of NWS storm report polygons instead of point-based NWS FFRs. Following the approach outlined in Clark et al. (2014) for evaluation of Ratios, polygon centroids of the FFRs are computed and then mapped onto the 50-km resolution grid. Exceedances based on the flash flood gridded products, on the other hand, are first assessed at their original 1-km resolution and then mapped onto the common grid. Only a single exceedance is counted per 50-km pixel regardless of the number of 1-km pixels that have exceeded a particular threshold.

The flash flooding products are subjected to a threshold-based analysis and are evaluated as dichotomous (yes/no) events through skill metrics based on contingency table statistics. From the contingency tables, we compute the geometric mean equitable threat score (ETS) for fixed thresholds (FT) of rainfall, ARIs, Ratios, and unit streamflow variables across the AIs (where applicable). The ETS values are then segregated into seasons (i.e., DFJ, MAM, JJA, SON) and into the same geographic regions as HS18 (their Fig. 11) for various thresholds applied to the flash flooding products.

3. Results

The HS18 verification study analyzed 30 months of data from January 2015 to June 2017. The data sample in this study is limited to a more recent time period of 12 months over which the MRMS QPE product was upgraded to the dual-polarization synthetic algorithm. The two time periods are different in duration and have no overlap. Figure 1 shows a heat map of exceedances over the 1-yr ARI for the MRMS QPE AI of 3 h. The figure has been drafted using the same color scheme as in Fig. 5b of HS18 so the two can be compared to indicate the representativeness of the data sample. HS18 outlines the causes for the departures from uniform patterns of ARI exceedances. Regardless of the cause, the spatial patterns of ARI exceedances from Fig. 1 are similar to HS18 with maxima noted over mountainous areas in Arizona and New Mexico, as well as along the Front Range of the Rocky Mountains. In the east, the event counts in the more recent data sample are higher along the Appalachian Mountains and over the state of Florida. Higher counts overall would have been expected in the HS18 sample given that it spans a longer time period. However, they are similar in many regions, and in some cases higher in our study. Nevertheless, the sample sizes are large enough and similar in spatial pattern to justify comparing results from both studies.

Fig. 1.
Fig. 1.

Heat map of exceedances of MRMS QPEs over the 1-yr ARI for an AI of 3 h. The color table and symbology match that of Fig. 5b in HS18, with blue lines outlining River Forecast Center boundaries, black lines indicating NWS county warning area boundaries, and green circles representing radar locations.

Citation: Journal of Hydrometeorology 22, 3; 10.1175/JHM-D-20-0215.1

a. Fixed rainfall thresholds

MRMS QPE thresholds are compared with FFRs for AIs of 1, 3, 6, and 24 h for several FTs. The resulting mean ETS values are computed and presented in Fig. 2a for the entire data sample. The highest ETSs are associated to MRMS FTs of 4.0 in. (3 h)−1 and equivalently to 5.0 in. (6 h)−1. To examine the spatial and temporal variability of the optimal use of FTs, we segregated the sample into regions and seasons. The highest correspondence with FFRs during the cool season months (DJF) occurs with FTs of 2.5–3.0 in. (6 h)−1 or approximately half the rainfall amounts for the same AI during the warm season months (Fig. 2b). The same geographic regions examined in HS18 are evaluated here and two contrasting results are presented. Figure 2c shows that the ETSs greater than 0.086 occur with rainfall rates of 2.0 in. (1 h)−1, 2.5 in. (3 h)−1, and 2.5 in. (6 h)−1 in the Southwest region for all seasons combined. In the Southeast region, the greatest ETSs occur with longer-duration and higher rainfall rates of 5.0 in. (3 h)−1, 5.0 in. (6 h)−1, and 6.0 in. (24 h)−1, which is approximately double the amounts for the Southwest region. Analyses for the additional regions and seasons are provided in the online supplemental material.

Fig. 2.
Fig. 2.

Mean ETSs for fixed rainfall thresholds using FFRs. The rainfall source is MRMS QPE and is evaluated across AIs for (a) all regions and all seasons, (b) all regions and cool season, (c) Southwest region and all seasons, and (d) Southeast region and all seasons. Following HS18, the top number of each row label corresponds to the threshold for the 1-h QPE exceedances, the middle number applies to the 3- and 6-h accumulation comparisons, and the bottom number to the 24-h QPEs.

Citation: Journal of Hydrometeorology 22, 3; 10.1175/JHM-D-20-0215.1

b. Average recurrence intervals

The same analysis performed above is applied to ARIs. These can be considered as static FTs but having spatial variability. Figure 3a reveals that the highest ETSs (greater than 0.1) occur with ARIs of 75–100 years for 3-h AI and 50–75 years for 6-h AI when examining the entire dataset combined. When the dataset is segregated into the cool season months, the highest ETSs (greater than 0.04) occur with much lower ARIs of 1.0–1.5 years for 1-h AI, 1.0–2.5 years for 3-h AI, and 1.0–2.5 years for 6-h AI (Fig. 3b). The spatial dependence of these optimum thresholds is shown with two contrasting regions in Figs. 3c and 3d. When all seasons are combined, the highest ETS (greater than 0.74) in the Southwest region occurs with ARIs of 40–100 years for 3-h AI and 25–75 years for 6-h AI. The Southeast region differs from the Southwest by having ETSs greater than 0.12 for ARIs of 7.5–30 years for 12-h AIs and 15–30 years for 24-h AIs. Additional figures for the other regions and seasons are provided in the supplemental material.

Fig. 3.
Fig. 3.

Mean ETSs for ARIs using FFRs. The rainfall source is MRMS QPE and is evaluated across ARIs for (a) all regions and all seasons, (b) all regions and cool season, (c) Southwest region and all seasons, and (d) Southeast region and all seasons.

Citation: Journal of Hydrometeorology 22, 3; 10.1175/JHM-D-20-0215.1

c. Rainfall to flash flood guidance ratio

Next, the Ratio product is evaluated in the same manner. This product also falls into the rainfall threshold class but differs from ARIs in that the thresholds are updated dynamically in correspondence to changes in soil moisture and river stages. Figure 4a shows the highest ETSs (greater than 0.120) occur with Ratios of 150%–200% for 3 h and 200%–300% for 6 h. In terms of seasonal dependence, the highest ETS of 0.119 occurs with a Ratio of 150% for 3 h, quite similar to the values for the entire dataset (Fig. 4b). The regional dependence is more significant with this product with the highest ETS in the Southwest region occurring with a Ratio of 200% in 1 h (Fig. 4c). The optimum Ratios shift in the Southeast region to much larger values as high as 500% in 6 h (Fig. 4d). The additional analyses for other regions and seasons are provided in the supplemental material.

Fig. 4.
Fig. 4.

Mean ETSs for Ratios using FFRs. The rainfall source is MRMS QPE and is evaluated across Ratios for (a) all regions and all seasons, (b) all regions and cool season, (c) Southwest region and all seasons, and (d) Southeast region and all seasons.

Citation: Journal of Hydrometeorology 22, 3; 10.1175/JHM-D-20-0215.1

d. Unit streamflow from hydrologic models

The fourth group of flash flooding products to be evaluated with FFRs comes from the CREST, SAC-SMA, and HP distributed hydrologic models. Specifically, the unit streamflow variables from the three models are analyzed across seasons and regions. Figure 5a shows all regions and all seasons combined for the three models. The highest ETSs (greater than 0.110) for the CREST model occur with values from 7.0 to 9.5 m3 s−1 km−2, while the highest ETS (greater than 0.090) with SAC-SMA occur from 3.5 to 6.0 m3 s−1 km−2 and best ETSs (greater than 0.050) with HP higher than 11.0 m3 s−1 km−2. The seasonal variability of these guidance values is assessed by examining ETS values during the cool season months of DJF. Figure 5b shows that the optimal ETSs (greater than 0.045) with CREST have lowered to 3.0–5.0 m3 s−1 km−2, with ETSs greater than 0.040 with SAC-SMA occurring from 2.0 to 3.0 m3 s−1 km−2, and ETSs greater than 0.060 occurring with HP unit streamflow values higher than 9.0 m3 s−1 km−2. Regional dependence is assessed using the same contrasting regions of the Southwest and Southeast. CREST unit streamflow values from 3.5 to 5.5 m3 s−1 km−2 are optimal in the Southwest, while the best ranges associated with SAC-SMA and HP are 1.0–2.0 m3 s−1 km−2 and greater than 10.5 m3 s−1 km−2, respectively. The ETS values generally improve in the Southeast region compared to the Southwest and the optimum thresholds increase to 7.5–9.5 m3 s−1 km−2 for CREST and to 4.0–5.5 m3 s−1 km−2 for SAC-SMA. Results for additional regions and seasons are supplied in the supplemental material.

Fig. 5.
Fig. 5.

Mean ETSs for unit streamflow values from each distributed hydrologic model in FLASH using FFRs. The rainfall source is MRMS QPE and is evaluated across unit streamflow values for (a) all regions and all seasons, (b) all regions and cool season, (c) Southwest region and all seasons, and (d) Southeast region and all seasons.

Citation: Journal of Hydrometeorology 22, 3; 10.1175/JHM-D-20-0215.1

4. Discussion and summary

This study reassesses the guidance on threshold values of commonly used products for flash flood forecasting, referred to as “precipitation proxies” in HS18. First, we expect some deviations in our results from those presented in HS18 owing to differences in the QPE forcing, data sample sizes and time periods, and methodologies in matching the products’ grid cells to the FFRs in space and time. In fact, a primary motivation for conducting this study was the change in the MRMS QPE algorithm, which now utilizes radar variables from the dual-polarization upgrade to the NEXRAD network. According to Zhang et al. (2020), the change to the synthetic algorithm that uses an attenuation-based estimator in rain has yielded an increase in the mean bias ratio in heavy rain from 0.81 to 0.95. Furthermore, we wanted to complement these analyses by including contemporary products based on distributed hydrologic model forecasts that comprise the operational FLASH system. Last, this study examines additional aspects with the guidance thresholds including their seasonal and regional dependencies.

HS18 provided the first comprehensive evaluation of precipitation proxies for flash flood verification. There are some precursor studies on the evaluation of FFG across the CONUS, but prior to HS18 there has been little documented on how precipitation FTs and ARIs relate to FFRs. NWS forecasters have developed their own subjective guidance based on operational experience using the MRMS and FLASH product suite (Fig. 6). According to forecasters’ experience regarding 1-h FTs, the possible range for flooding is 2.0 in. (1 h)−1 for urban/hilly settings up to 4.0 in. (1 h)−1 in rural settings. These thresholds increase to 3.0 in. (1 h)−1 up to 6.0 in. (1 h)−1 for significant flooding. For hourly accumulations, HS18 found the best ETSs were with FTs of 1.5 in. (1 h)−1. In our study, the best ETS for hourly rainfall was associated to an FT of 3.0 in. (1 h)−1. This FT is approximately double that reported in HS18 and is closer in agreement with the forecaster-based guidance.

Fig. 6.
Fig. 6.

Subjective guidance developed based on experience from NWS forecasters on the use of the MRMS and FLASH products for operational flash flood warning. Note the unit streamflow values are presented above in units of ft3 s−1 mi2, whereas they are presented in m3 s−1 km−2 in the article.

Citation: Journal of Hydrometeorology 22, 3; 10.1175/JHM-D-20-0215.1

In HS18, the best threshold overall when considering multiple AIs was 2.5 in. (24 h)−1. While this threshold was not considered by forecasters in developing their quick reference guide, it is surprising to see the best overall threshold being associated to a daily rainfall accumulation rather than with a shorter-duration AI given that pluvial flash floods respond to the causative rainfall within minutes to a few hours. In our study, we found the best overall FT to be 4.0 in. (3 h)−1 and equivalently to 5.0 in. (6 h)−1. The optimal FTs decrease to 2.5–3.0 in. (6 h)−1 when we examined events during the cool season months of December–February. These thresholds are regionally dependent as well. The best ETS in the Southwest region was associated to FTs of 2.5 in. (3 h)−1 while the optimum values in the Southeast were 5.0 in. (3 h)−1 and 5.0 in. (6 h)−1. These results are more plausible given the short-duration, high-intensity convective thunderstorms in the Southwest that trigger flash floods, which contrasts with the longer duration and higher accumulations of rainfall with storms common to the Southeast.

When examining ARIs, HS18 found the best predictions to be associated with a 1-yr ARI for an AI of 24 h. The NWS subjective guidance for ARIs notes the most likely range to be 10–30 years for flooding and a possible range of 50–100 years for significant flooding. Our study identified the best performing ARIs to be 75–100 years for 3 h and 50–75 years for 6 h. These values are much closer to the NWS subjective guidance and are significantly higher than those found in HS18. The best thresholds during the cool season months dropped dramatically to 1.5 years for 3 h rainfall. When examining contrasting regions, we found the best ARIs to be with 100 years in 1 h in the Southwest and 15 years in 12 h or 15–25 years in 24 h in the Southeast. The sensitivity of ARIs to rainfall amount reflects the underlying distributions describing the nonlinear relation between rainfall amount and annual exceedance probability.

HS18 reported the best AI for exceeding FFG thresholds was 6 h. Clark et al. (2014) also examined FFG thresholds and found that the best predictors were with 1- and 3-h rainfall that exceeded FFG by ratios greater than 100%, i.e., 150%–200%. The NWS subjective guidance does not specify with AI they considered, yet they indicate the most likely range for flooding is with QPE-to-FFG exceedances of 150%–225%. Our study highlights 3-h QPE exceeding FFG by 200% as the best predictor of flash flooding, in agreement with the NWS subjective guidance and prior results from Clark et al. (2014). This guidance threshold is much less dependent on season as compared to FTs and ARIs; the wintertime optimum only drops to 150% exceedance for 3-h rainfall. There is a strong regional dependence though. The best threshold in the Southwest is a ratio of 200% for 1-h rainfall, which inflates to 500% for 6-h rainfall in the Southeast. The FFG thresholds were developed for exceeding bankfull conditions and do not necessarily indicate the magnitude of flash flooding.

Last, we provide guidance thresholds on the newly released unit streamflow variables coming from the CREST, SAC-SMA, and HP models in FLASH. The NWS forecaster quick reference guide indicates they consider flooding to be most likely for CREST unit streamflow values of 2.0–4.0 m3 s−1 km2 and 7.5–10.5 m3 s−1 km2 for significant flooding. In our study, the best threshold was 8.5 m3 s−1 km2 with the CREST model, 4.5 m3 s−1 km2 with SAC-SMA, and 12.0 m3 s−1 km2 with HP. Optimum thresholds during the cool season approximately halved for both CREST and SAC-SMA while they remained at 12.0 m3 s−1 km2 for the “worst case scenario” HP model. In the Southwest, the best thresholds were 4.0–4.5 m3 s−1 km2 for CREST and 1.5 m3 s−1 km2 for SAC-SMA. These optimum values increased to 9.0 and 4.5 m3 s−1 km2 for CREST and SAC-SMA, respectively, when examining events in the Southeast. Overall, the results presented in this study agree with the ranges experienced by operational forecasters.

When we consider the relative ETS values among the different classes of products for all seasons and all regions combined, we find that the ranking from worst to best is as follows: 1) FT of 4.0 in. (3 h)−1 or 5.0 in. (6 h)−1 with an ETS = 0.085, 2) ARI of 75 years for 3 h and 50–75 years for 6 h with an ETS = 0.101, 3) CREST unit streamflow of 8.5 m3 s−1 km−2 with an ETS = 0.114, and 4) FFG exceedance by 200% for 3-h rainfall with an ETS = 0.130. A sensitivity study (not shown) was conducted on the predictor-to-observation matching method and revealed that the absolute ETS values depended on the parameter settings of the matching method; focus herein is on the relative ETS differences. In general, we found that the relative ETS values increased with the sophistication of the flash flooding products. There was one exception to this finding and that was with the relatively lower skill of CREST compared to Ratios. We explored this outcome in more detail by segregating the analysis regions into percent imperviousness land cover classes given that CREST utilizes a parameter that is sensitive to impervious land cover in urban areas. It turns out CREST achieved an ETS of 0.126 when we segregated events into areas with 6%–50% imperviousness at a unit streamflow threshold of 7.5 m3 s−1 km−2. When we did the same for urban areas with imperviousness > 50%, the ETS jumped to 0.202 for a CREST unit streamflow threshold of 11.0 m3 s−1 km−2. This result, presented in the supplemental material, highlights the importance of considering urban land cover and routing in simulating pluvial flash flooding.

In closing, while we reiterate that differences are to be expected between this study and that of HS18 for the aforementioned reasons, we draw the following distinctions based on these results:

  • HS18 found QPE exceedances corresponded more with FFRs when longer accumulation periods were considered with an optimal FT of 2.5 in. (24 h)−1. Our study associated the best skill with an FT of 4.0 in. (3 h)−1 and equivalently to 5.0 in. (6 h)−1, or approximately double the HS18 values over much shorter durations.

  • HS18 stated the simplest method for yielding a precipitation proxy of flash flooding (i.e., placing a static threshold on rainfall rates) had better correspondence to FFRs than the more complex methods that consider spatiotemporal variability of the thresholds. Our study found the opposite. ETS values improved with the sophistication of the methods in the following order: FT, ARI, Ratio, and distributed hydrologic model forecasts.

  • HS18 found the optimal ARI to be 1–5 years. Our results indicated the best ETSs were with ARIs of 75–100 years for 3 h and 50–75 years for 6 h.

Studies, such as HS18 and this one, are needed to provide improved guidance to end-users of the flash flooding products while new tools are being developed and revised. Their error characteristics change following version updates, which was the case with the upgrade to dual-polarization with MRMS and FLASH. Furthermore, over time, precipitation intensities may change which will also require revisiting the thresholds.

Acknowledgments

Funding was provided by NOAA/Office of Oceanic and Atmospheric Research under NOAA-University of Oklahoma Cooperative Agreement NA11OAR4320072, U.S. Department of Commerce. We are grateful to the Multi-Radar Multi-Sensor team for providing the version 12 QPE products enabling us to conduct this study.

Data availability statement

Datasets can be accessed to evaluate the reproducibility of the results presented in this study at http://doi.org/10.5281/zenodo.4022675.

REFERENCES

  • Burnash, R. J. C., 1995: The National Weather Service River Forecast System—Catchment modeling. Computer Models of Watershed Hydrology, V. P. Singh, Ed., Water Resources Publications, 311–366.

  • Clark, R. A., J. J. Gourley, Z. L. Flamig, Y. Hong, and E. Clark, 2014: CONUS-wide evaluation of National Weather Service flash flood guidance products. Wea. Forecasting, 29, 377392, https://doi.org/10.1175/WAF-D-12-00124.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gourley, J. J., and R. Clark III, 2018: Real-time flash flood forecasting. Oxford Encyclopedia of Natural Hazard Science, Oxford University Press, https://doi.org/10.1093/acrefore/9780199389407.013.298.

    • Crossref
    • Export Citation
  • Gourley, J. J., and Coauthors, 2017: The FLASH project: Improving the tools for flash flood monitoring and prediction across the United States. Bull. Amer. Meteor. Soc., 98, 361372, https://doi.org/10.1175/BAMS-D-15-00247.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herman, G. R., and R. S. Schumacher, 2018: Flash flood verification: Pondering precipitation proxies. J. Hydrometeor., 19, 17531776, https://doi.org/10.1175/JHM-D-18-0092.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koren, V., S. Reed, M. Smith, Z. Zhang, and D.-J. Seo, 2004: Hydrology laboratory Research Modeling System (HL-RMS) of the US National Weather Service. J. Hydrol., 291, 297318, https://doi.org/10.1016/j.jhydrol.2003.12.039.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lincoln, W. S., and R. F. L. Thomason, 2018: A preliminary look at using rainfall average recurrence interval to characterize flash flood events for real-time warning forecasting. J. Oper. Meteor., 6, 1322, https://doi.org/10.15191/nwajom.2018.0602.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Martinaitis, S. M., and Coauthors, 2017: The HMT multi-radar multi-sensor hydro experiment. Bull. Amer. Meteor. Soc., 98, 347359, https://doi.org/10.1175/BAMS-D-15-00283.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Perica, S., S. Pavlovic, M. St. Laurent, C. Trypaluk, D. Unruh, and O. Wilhite, 2018: Version 2.0: Texas. Vol. 11, Precipitation-Frequency Atlas of the United States, NOAA Atlas 14, NOAA/National Weather Service, 283 pp., https://www.weather.gov/media/owp/oh/hdsc/docs/Atlas14_Volume11.pdf.

  • Ryzhkov, Z., M. Diederich, P. Zhang, and C. Simmer, 2014: Potential utilization of specific attenuation for rainfall estimation, mitigation of partial beam blockage, and radar networking. J. Atmos. Oceanic Technol., 31, 599619, https://doi.org/10.1175/JTECH-D-13-00038.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stovern, D. R., J. A. Nelson, S. Czyzyk, M. Klein, K. Landry-Guyton, K. Mattarochia, E. Nipper, and J. W. Zeitler, 2020: The extreme precipitation forecast table: Improving situational awareness when heavy rain is a threat. J. Operat. Meteor., 8, 93104, https://doi.org/10.15191/nwajom.2020.0807.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, J., and Coauthors, 2011: The Coupled Routing and Excess Storage (CREST) distributed hydrological model. Hydrol. Sci. J., 56, 8498, https://doi.org/10.1080/02626667.2010.543087.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, Y., S. Cocks, L. Tang, A. Ryzhkov, P. Zhang, J. Zhang, and K. Howard, 2019: A prototype quantitative precipitation estimation algorithm for operational S-band polarimetric radar utilizing specific attenuation and specific differential phase. Part I: Algorithm description. J. Hydrometeor., 20, 985997, https://doi.org/10.1175/JHM-D-18-0071.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, J., L. Tang, S. Cocks, P. Zhang, A. Ryzhkov, K. Howard, C. Langston, and B. Kaney, 2020: A dual-polarization radar synthetic QPE for operations. J. Hydrometeor., 21, 25072521, https://doi.org/10.1175/JHM-D-19-0194.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

Supplementary Materials

Save