The impacts of excessive rainfall need almost no introduction in the United States, which experienced numerous deadly and damaging rainfall-related floods from 2010 to 2021. A mesoscale convective system, stationary for more than 9 h, flooded Boulder, Colorado, and surrounding communities in 2013 (Gochis et al. 2015). Training thunderstorms produced 127–178 mm (5–7 in.) of rain in 2–3 h and torrents of water through Ellicott City, Maryland, twice in separate years (2016 and 2018; National Weather Service Baltimore/Washington Weather Forecast Office 2016; Viterbo et al. 2020). Stalled moist conveyor belts produced synoptic-scale heavy rain that inundated the Great Plains, Midwest, and Southeast almost annually from 2015 to 2019, with resulting flooding along the Ohio and Mississippi River basins lasting weeks (National Weather Service New Orleans/Baton Rouge Weather Forecast Office 2019; National Centers for Environmental Information 2019). And landfalling hurricanes, some of which were exceptionally slow moving, like Harvey (2017) and Florence (2018), shattered state and even continental United States records for rainfall directly associated with a single tropical cyclone (Blake and Zelinsky 2018; Stewart and Berg 2019; Garmon et al. 2020; Martinaitis et al. 2021). There have been 35 separate billion-dollar disasters from rainfall-induced flooding, including tropical cyclones in which flooding was a significant factor, in the contiguous United States (CONUS) since 2010 (NCEI 2022). Many locations in recent years have experienced repeated heavy rainfall events and have recorded their wettest year in at least the past five decades (Fig. 1).
Areas in which the annual precipitation in 2016–21, respectively, exceeded the 1969–2021 maximum by at least 25.4 mm (1 in.) in the Parameter-Elevation Regressions on Independent Slopes Model (PRISM) database from Oregon State University (Daly et al. 1994, 2008). The color shading and scale indicate the magnitude of exceedance in inches.
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
The Weather Prediction Center (WPC), one of the nine National Centers for Environmental Prediction (NCEP), specializes in rainfall forecasts (Novak et al. 2014). WPC services include value-added quantitative precipitation forecasts (QPFs), Mesoscale Precipitation Discussions (MPDs), and the Excessive Rainfall Outlook (ERO). The ERO is a probabilistic forecast of rainfall exceeding River Forecast Center (RFC)-provided flash flood guidance (FFG; Sweeney 1992; Schmidt et al. 2007; Clark et al. 2014), describing rainfall events that may lead to flash flooding (Fig. 2a). The ERO is valid for the CONUS at lead times of 1, 2, and 3 days, and now experimentally at lead times of 4 and 5 days. For the purposes of this product, flash flooding refers to events in which the rapid rise of water relates to causative rainfall occurring on time scales of a few minutes up to 6 h and is thus consistent with the widely used flash flood definition (AMS 2020). Flash flooding, as defined here, describes water events on the ground that occur primarily outside of main stem rivers (although main stem rivers have been known to experience flash floods) and are below the spatial and temporal scales of events that are traditionally modeled by RFCs at designated points along main stem rivers. The ERO has traditionally been driven primarily by meteorologists and predicted rainfall, but it is tied by definition to hydrologist-provided FFG and is increasingly benefiting from the input of hydrologists at the National Water Center (NWC) and RFCs. WPC has made several enhancements to the ERO, since 2013, by strengthening the science and presentation of the product and increasing outreach to stakeholders.
Example Excessive Rainfall Outlook (a) graphic and (b) typical discussion components.
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
In partnership with the National Hurricane Center (NHC) and Central Pacific Hurricane Center, WPC also performs tropical cyclone rainfall prediction from the Atlantic basin to Hawaii, including for landfalling systems in the CONUS. Since 2019, the forecast process at WPC has included exchange of hydrometeorological insights with the newly created NWC, and these centers have engaged in event-driven joint product creation (e.g., tropical cyclone key messages for flooding).
Given these roles and services, WPC helped lead the creation of a collaborative forecast process (CFP) for heavy rain and flash flooding across National Weather Service (NWS) field offices. This was an iterative process that began in 2014, when NWS national and regional leadership began prioritizing interoffice forecast consistency to complement forecast accuracy in delivering effective services to users (Uccellini and Ten Hoeve 2019). The aim of developing a heavy rainfall CFP was to make forecast products and messages consistent with one another regardless of the users’ scope of interest and method of receipt. By 2020, interoffice communication mechanisms and shared expectations and workflows had been developed and put into practice dozens of times for events in each respective NWS CONUS region. WPC products, beginning with the ERO, and collaborative conference calls serve as the upper portion of a conceptual funnel in which forecasts and partner interactions cascade toward increasingly detailed timing and impact information emanating from the NWC, RFCs, and local Weather Forecast Offices (WFOs).
The ERO and the heavy rainfall CFP regularly shape event messaging that reaches out through the weather community and its users and partners during the lead-up to high-impact hydrometeorological events. User groups may have an interest in, and may benefit from, developing a greater understanding of the ERO and the related heavy rainfall CFP. This paper will take the reader inside WPC operations to reveal what tools, thought processes, and collaborations are involved in ERO construction. Data are also presented on a striking percentage of all flood-related damages and fatalities that occurred on ERO High Risk days1 from 2010 to 2020. ERO utility on such days and the associated heavy rainfall CFP are illustrated via in-depth case studies of the 9 January 2018 Montecito, California, flash flood and debris flow and Hurricane Harvey (August 2017). Finally, challenges are highlighted, which could be overcome by research to further improve ERO utility.
Excessive Rainfall Outlook definition
Product definition.
The ERO graphically maps the probability of rainfall exceeding FFG2 during a 24-h period (updates to the Day 1 ERO are valid for less than 24 h) within 40 km of any point over the CONUS (National Weather Service 2019a). Marginal, Slight, Moderate, and High Risks denote at least a 5%, at least a 15%, at least a 40%, and at least a 70% chance of rainfall exceeding FFG, respectively (Fig. 2a). Sub-Marginal areas are not explicitly depicted, but any blank area on the map may be presumed to carry probabilities of excessive rainfall ranging from 0% to less than 5%.
Accompanying the graphic product is an associated Excessive Rainfall Discussion, which details the forecaster thought processes, worst-case scenarios, areas of uncertainty, and the expected magnitude of impacts (Fig. 2b). To get the full value of the ERO, the user should think of the graphic and discussion as inseparable. The ERO is delivered to NWS offices via the Advanced Weather Interactive Processing System (AWIPS-2) and is available publicly on the WPC website (www.wpc.ncep.noaa.gov/qpf/excessive_rainfall_outlook_ero.php) and through a number of NWS dissemination streams.
EROs are issued for days 1, 2, and 3, and experimental day 4 and 5 outlooks were added in 2022. EROs cover 24-h valid periods that have their end times permanently affixed to 1200 UTC. For updates to the Day 1 Outlook product, the valid period becomes progressively shortened, as some of the 24-h period has already passed. This approach to duration and update times (Table 1) is closely parallel with the Storm Prediction Center (SPC) Convective Outlooks (Hitchens and Brooks 2014).
ERO product issuance schedule.
The High Risk category as a predictor of damages and fatalities.
It is easier to identify meteorological environments that will support heavy short-term rain rates, and, by extension, flash flooding, than it is to predict the severity of flash flood impacts (Doswell et al. 1996). Atmospheric ingredients are generally better understood, while flash flood severity is dependent on the precise combination of a variety of factors, including the placement of the heavy rain rates, the underlying basin and land-use characteristics, and antecedent soil moisture (e.g., Barthold et al. 2015; Gourley et al. 2017; Erickson et al. 2019). The ERO is in large part calibrated to the frequency and spatial density of flash flood occurrences. Similar to SPC Convective Outlooks, forecasters attempt to reserve the Moderate and High Risk categories for events that are “a cut above,” representing a pronounced threat of severe impacts. This approach harkens to the desired user response, especially for the emergency management community. Many municipalities and other organizations invoke certain actions involving staffing and movement of resources when a Moderate or High Risk is in effect, so there is a desire to make this type of action the exception rather than the rule. Whereas SPC outlook categories incorporate probabilities of higher-magnitude impacts, using formal definitions of “significant hail,” “significant tornadoes,” and “hurricane force winds,” the science has not yet caught up with respect to flash flood impacts. There have been recent efforts to create a scale of flash flood intensity, but so far these have not been adopted operationally. This is an area for potential future study (e.g., Schroeder et al. 2016; Lincoln and Thomason 2018).
Still, recent multiyear verification supports that the ERO is skillful and well calibrated at all thresholds, including the critical Moderate and High Risks (Erickson et al. 2021). Events that produce a greater spatial density of reports, corresponding to Moderate and High Risk definitions, naturally affect greater numbers of stream basins, cityscapes, and people, and can also result in a greater number of high-end impacts to lives and property. WPC performed a study to measure the potential utility of a skillful, calibrated ERO High Risk category. All Storm Data event reports that were tagged as flooding, flash flooding, heavy rain, or debris flow, from 2010 to 2020, were identified and mapped relative to the ERO that was valid during each respective event. The study found that High Risk was issued on only 4% of all days. The number of weather systems that prompt High Risk occur even more seldom than “High Risk days” because such systems can result in multiday High Risk episodes (e.g., hurricanes Harvey in 2017 and Florence in 2018). Reports were counted as a “hit” if they occurred in a High Risk area or nearby Moderate and Slight Risk areas that surrounded a High Risk. The numbers reported are therefore a measure of the entire regional event that takes place on a High Risk day, giving credit to enhanced messaging even outside the technical bounds of the High Risk contour. This approach also accounts for the imprecise timing of impacts reported in Storm Data. Unlike (for example) a hail or wind gust report, which will have a discrete time of occurrence, flood reports are frequently logged for several hours or days, and the exact time a fatality or damage occurred can be unclear.
WPC found that 83% of all flood-related damage and 39% of all flood-related fatalities reported in Storm Data for the CONUS occurred in or near an ERO High Risk area (Fig. 3). High Risk days strongly discriminate between days with more common excessive rain impacts and those with high-end impacts. The High Risk as depicted on a given ERO represents the most likely region for impactful rainfall, as well as the placement of strong environmental cues for an extreme rainfall event. But the data show that the potential for especially severe instances of flash flooding also extends more broadly to nearby areas covered by lower ERO risk categories on High Risk days. WPC has begun branding the issuance of a High Risk as a “potentially deadly and damaging day” based on this information, to gain traction toward a greater response to the ERO from emergency managers, media, and other users.
Percent distribution of (a) flood-related fatalities and (b) damages by ERO risk category from 2010 to 2020. Flood events in this analysis include flood, flash flood, heavy rain, and debris flow events from Storm Data. High Risk was issued on approximately 4% of days, though the number of events reaching High Risk is less than the number of days reaching High Risk, as many such events span multiple days. Excludes Oso, WA, landslide, which occurred well after rainfall and on a sunny day. Damage estimate used for Montecito debris flow.
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
ERO construction and the heavy rainfall CFP
ERO construction at WPC.
The NWS has long employed a successful paradigm by which broad-scale, long-lead-time outlooks precede mesoscale discussions and watch products of greater detail, then giving way to local warnings, some of which may drill down to the subcounty level. This idealized forecast funnel is not possible for all events, but it does provide scaffolding for operational meteorologists to consider as they balance lead time, confidence, and relative specificity of information. Probabilistic outlook products like the ERO are well suited to play a vital role as this scaffolding transitions toward the continually updating, probabilistically framed hazard information envisioned by NOAA’s Forecasting a Continuum of Environmental Threats program (FACETs; Rothfusz et al. 2018).
WPC forecasters employ 1) ingredients-based conceptual models for heavy rainfall, 2) model QPFs and tools that present postprocessed output from deterministic and ensemble guidance, and 3) tools that characterize the current sensitivity of hydrologic basins to heavy rainfall in constructing the ERO. The ingredients-based approach to heavy rainfall forecasting considers factors such as forcing, moisture, instability, the duration of rainfall (related to inflow strength and convective cell motions), and pattern recognition. WPC forecasters analyze these ingredients across multiple model systems of varying scales, with respect to both absolute magnitudes and climatological anomalies.
WPC operations, supported by the Environmental Modeling Center, federal atmospheric research laboratories, the WPC Development and Training Branch, and the Hydrometeorological Testbed (HMT), has continual access to a nearly unabridged set of high-resolution weather models, including experimental versions and tools designed specifically to support the ERO. The High-Resolution Ensemble Forecast (HREF) system (Roberts et al. 2019; Harrison et al. 2022) is heavily utilized at shorter lead times. The HREF system provides both neighborhood and point probabilities of rainfall magnitudes, FFG exceedance, and average recurrence interval (ARI) exceedance. Simulated radar reflectivity and explicit 1-, 3-, and 6-h QPFs from individual HREF members also help forecasters characterize the evolution and intensity of forecast events and aid in revealing which conceptual models need to be considered. Global model deterministic QPFs and probabilistic ensemble forecasts are examined, especially at longer lead times (e.g., day 3 and beyond). Given the frequency of large shifts in operational guidance from run to run, especially at longer lead times, some percentage of continuity (the previous WPC forecast) may be retained in both WPC QPFs and the ERO.
Forecasters also have two tools that provide a first guess for ERO risk areas. A machine learning prediction model developed by Colorado State University (CSU-MLP; Herman and Schumacher 2018a,b; Schumacher et al. 2021) synthesizes forecast information such as model QPFs, convective ingredients, historical model biases, and the local precipitation climatology to generate a probabilistic product that can be used as a proxy for flash flood risk across the CONUS. Another first guess tool designed at WPC is derived from the WPC Probabilistic QPF (PQPF; Brill 2017; Novak 2019) product. WPC PQPF is designed so that the mode of the forecast distribution is set to the human-adjusted WPC QPF at each grid point across the CONUS. WPC PQPF is already weighted toward expectations of the WPC forecasters. The first guess tool then utilizes different PQPF thresholds exceeding FFG and the 5-yr ARI to generate forecasts for the different ERO risk categories.
Forecaster workflow often starts with a recognition of any large-scale threats that are clearly signaled via the synoptic pattern and/or quantitative tools such as QPF and the two first-guess models. The workflow then continues down toward 1) identifying smaller-scale threats and/or 2) identifying aspects of the large-scale threats that are relatively more or less certain. This approach enables forecasters to describe their synthesis of the weather pattern, model signals, and timing and evolution of upcoming rain events in the Excessive Rainfall Discussion. Once forecasters have a sense of their confidence in the forecast details, they evaluate the hydrologic conditions in the area of interest in collaboration with hydrologists at the NWC, considering factors such as land use, terrain, antecedent rainfall, streamflow anomalies, and soil moisture anomalies. These measures inform forecasters as to the likelihood of a greater than normal percentage of rainfall becoming rapid surface runoff. This task is becoming more quantitative with tools like the National Water Model (Cosgrove and Klemmer 2016).
The heavy rainfall CFP.
Even with increasingly numerous and increasingly accurate meteorological and hydrologic observations and model tools, outlook messaging requires human synthesis and distilling of information, often under rapidly changing conditions and often in coordination across spatial scales. Scale-interdependent consistent messaging is a key component of overcoming confirmation bias, especially ahead of significant severe weather (Mileti and Sorenson 1990; Uccellini and Ten Hoeve 2019). The heavy rainfall CFP provides the best, unified forecast by leveraging expertise across local, regional, and national levels within a common operating picture, in a collaborative manner, and with clear roles and responsibilities. The goal: “one event, one forecast.”
ERO construction follows a similar cadence to the traditional WFO forecast “packages,” with forecasts most strongly influenced by 0000 and 1200 UTC model cycles (see issuance times in Table 1). NWS operations are moving toward hourly updating automated starting points, however, for quantitative fields such as QPF. The National Blend of Models (NBM; Hamill et al. 2017) is one such automated, hourly updating dataset that has been used to center the heavy rainfall CFP for QPFs since successful testing in 2020 (National Weather Service Operations Proving Ground 2020).
WPC keeps in continual communication with WFOs via internal AWIPS-2 collaboration chat software. WPC will frequently post preliminary ERO graphics to an internal collaboration website intended for WFO and RFC use about 3–4 h prior to the product deadline. Both chat and preliminary graphics facilitate discussion and refinement of ideas between WPC and WFOs. The collaborative relationship between WPC and NWC since 2020 has included daily calls between the two centers to discuss forecast precipitation areas as they relate to signals in the NWM and use of a newly developed playbook for tropical cyclone rainfall and flood messaging.
Event-driven conference calls that have become a mainstay of the heavy rainfall CFP are initiated when WPC proposes an initial upgrade to the Moderate or High Risk category (Slight Risk threshold for the Western Region). WFOs, RFCs, and Regional Operations Centers may also request a formal call. WPC will host the call or co-host the call with the Regional Operations Centers. NWS Regional Operations Centers are not staffed 24 h, but their direct leadership of the process when feasible, and during extended hours for foreseeable high-impact events, speaks to the desire of the NWS to leverage a support structure and improve message consistency at all levels. Heavy rainfall collaboration calls often progress from refinement of the proposed ERO (or EROs if a multiday event) to a local-level proposition of watches. WFOs and RFCs gain a voice in WPC products that cover their local areas of responsibility and may communicate knowledge of antecedent wetness and streamflow to WPC. In turn, WPC forecasters voice the output of unique operational tools and their own well-informed (via daily CONUS-wide use of an unabridged set of model tools) interpretation of numerical QPF guidance and large-scale environmental clues. RFCs and the NWC share insights on hydrologic sensitivities that may influence messaging—or may even tip the scales in favor of an ERO risk category.
An idealized heavy rainfall CFP workflow and case example are presented in Fig. 4. In the example (Fig. 4b), from 31 May 2016, WPC QPFs accumulated to widespread heavy amounts over a span of 3 days of the forecast. Confidence in mesoscale heavy rain areas was insufficient, however, to prompt more than an ERO Slight Risk on any single day. While multiday QPF does not directly contain information on 0–6-h rain rates, Oklahoma and Texas are prone to intense short-term rain rates and heavy-rainfall impacts during the spring months. The Southern Region–Regional Operations Center (SR-ROC) sensed a mismatch between the heavy multiday QPF and an absence of WFO Flash Flood Watches. SR-ROC reached out to WPC to initiate a collaboration conference call with WFOs and RFCs in Oklahoma and Texas. During the call, WFOs lobbied WPC to upgrade to Moderate Risk on at least one of the days, noting that this would focus communication with their partners. WFOs then coordinated the issuance of seamless Flash Flood Watches from southern Texas to southern Oklahoma. Deadly flash flooding took place within the Moderate Risk and Flash Flood Watch area on 2 June 2016.
(a) The steps in an idealized heavy rainfall CFP workflow for the daytime forecast cycle (night cycle workflow is similar but may preclude regional office involvement at times). (b) Graphical depiction of a successful collaboration from 31 May 2016.
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
Case illustrations of the NWS CFP for flash flooding
Montecito, California, flash flood and debris flow (2018).
The forecast evolution leading up to the 9 January 2018 flash flood and debris flow at Montecito, CA (Kean et al. 2019), is typical of the lead up to many heavy rain events. The event spanned two ERO periods by virtue of the heavy rain straddling 1200 UTC 9 January. The forecast risk for the 24-h period ending 1200 UTC 9 January evolved quickly from Marginal on the Day 3 ERO to a small Moderate on the Day 2 ERO across Southern California (Figs. 5b,c). WPC coordinated the day 2 upgrade via chat messaging with WFO Oxnard. The ERO text specifically called out the Thompson burn scar and high-resolution model output predicting 1 in. h−1 rainfall. The Thompson fire and resulting burn scar had been in national news as then the largest in California history, thus raising awareness at WPC.
(a) Approximation of WFO Flash Flood Watches (in green) put in place by 8 Jan 2018, in California. Progression of EROs from (b) day 3 to (c) day 2 to (d) day 1, along with (e) verification of the Day 1 ERO High Risk and surrounding risk areas for the 8–9 Jan 2018 flash flood and debris flow event in Southern California.
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
The general vulnerability of burn scars is known, and even modest rain rates can trigger flash floods and debris flows (Tryhorn et al. 2008). Burn scars and associated debris flows from rainfall are specifically identified as Flood and Flash Flood Watch and Warning issuance criteria in NWS policy directives (Highland and Bobrowsky 2008; National Weather Service 2019b) and often result in a lowering of FFG or local office warning guidance, especially in areas of complex terrain. While WPC typically does not account for burn scars in the ERO because of the small geographic scale, larger burn scars or clusters of burn scars have influenced the outlook at times. These localized risk upgrades are always coordinated with a WFO, but that does have the potential to introduce inconsistencies across WFO areas of responsibility based on local preferences.
The WPC night shift on the morning of 8 January inherited the small Moderate Risk area over Southern California. WPC forecasters drafted a potential High Risk for the new day 1 period based on model trends and the severity of wording in WFO Oxnard official products. With other heavy rain potential extending up the length of the state, WPC initiated a collaboration conference call at 0730 UTC 8 January with all WFOs based in California and with WFO Reno, Nevada, which serves the northern Sierra. WFO Oxnard readily agreed to the High Risk upgrade, as the expected rain rates would easily overwhelm the Thompson burn scar landscape. WFO Oxnard also noted another large burn scar, the La Tuna, that was located only about 60 mi. (∼100 km) away, near Burbank, was of nearly equal concern. This local knowledge informed WPC to make the High Risk larger to encompass both burn scars. This action was consistent with the meteorology because rain rates forecast over the La Tuna scar were minimally less than for the Thompson scar. Both WFO Oxnard and WPC expressed high confidence in what were likely to be severe impacts during the event, leading to the following in the ERO discussion issued after the collaboration: “Significant slides of mud … rock … and debris … appear to be a near certainty given predicted hourly rain rates as great as 1.25 inches.” Having these predictions repeated in coordinated fashion through local and national messaging is one important reason for the heavy rainfall CFP.
On this same call, WPC and WFOs throughout northern California decided that local Flash Flood Watches (Fig. 5a) would determine the placement of multiple ERO Slight Risk areas embedded within the broader Marginal Risk (Fig. 5d). WPC was able to speak to the characteristic strength and weakness of the various convection-allowing models, as it became evident through conversation that some California forecasters on the call had seldom practiced using hourly rain rate predictions from these models. Thus, the call was beneficial in both directions and resulted in a consistent NWS product suite from national to local levels with one exception. WPC would have benefited from inviting WFO Las Vegas, Nevada, to the call, as they later issued and verified with one local storm report, a Flash Flood Watch for Death Valley National Park, where the ERO forecast was no risk.
A High Risk for the Southern California burn scars for day 2, the 24-h period ending 1200 UTC 10 January, was issued later that day on 8 January, foregoing the requisite conference call, as this was an extension in time of the ERO that had been well coordinated that morning. The High Risk messaging was consistent throughout the event. The update of the Day 1 ERO issued at 0100 UTC 9 January noted in the discussion portion of the ERO, “These well-coordinated risk areas over California were again changed very little.”
Hurricane Harvey (2017).
Hurricane Harvey operations serve as a benchmark for the heavy rainfall CFP and ERO development for events of large scale and long duration. NWS offices maximized the frequency of their formal collaborative interactions during Harvey, with conference call goals ranging from the medium-range forecast track to vetting the integrity of record-breaking measurements from automated rain gauges. Many of the tactics documented in this section could be scaled down to serve the collaborative needs of the more common, smaller-scale heavy rainfall events.
The relevant offices, WPC, NHC, Southern Region–Regional Operations Center (SR-ROC), WFO Houston, and West Gulf River Forecast Center (WGRFC), had been well prepared to work together through a combination of training exercises and operational events (the NWC had not yet taken on a larger role in the CFP in 2017). First, relationship building between WPC and Southern Region offices had been facilitated by their collaborating during frequent large-scale flash flood events over the prior 3–4 years—many of them impacting Texas. Second, in 2016 the Effective Hurricane Messaging course, co-hosted by NHC, WPC, Southern Region, and Eastern Region, with WFO and WGRFC attendees, performed a hands-on simulation based on a hypothetical landfalling storm in which heavy rainfall was the dominant threat. This exercise allowed operational staff at local, regional, and national levels to practice time-sensitive, joint communication of heavy rainfall information to each other and to users, including mock briefings to state officials. Then, early in 2017, several months prior to Harvey, WPC and the Cooperative Program for Operational Meteorology, Education, and Training (COMET) released a self-paced online training course designed to inform users how to interpret tropical cyclone–related QPF and excessive rainfall products from WPC.
The heavy rainfall CFP for Harvey began on 22 August 2017 (Fig. 6), 5 days before the escalation of flooding in Houston. High-level communication between NWS Headquarters (NWSHQ), the Regions, and NCEP prompted initiation of a nonroutine 1445 UTC call involving all relevant parties at the national, regional, and local levels. NHC and WPC co-led discussion of track and intensity forecasts and rainfall forecasts, yet this was outside the window for the ERO. NHC Advisories on Tropical Storm Harvey resumed by 23 August 2017.3 The cadence for NHC advisories included “hotline” coordination calls (1400, 2000, 0200, and 0800 UTC) that take place 1 h prior to advisory issuance. These are intended as a briefing and an opportunity for quick, tactical questions, including an opportunity for WFOs to voice approval or request tweaks of the rainfall statement written within the NHC public advisories. These rainfall statements are short, plain language descriptions of forecast rainfall amounts and possible impacts that originate from WPC (now WPC/NWC). WPC and NHC strive to keep language consistent between the rainfall statement and the ERO. The Day 3 ERO as of 23 August displayed Moderate Risk along a lengthy stretch of the Texas Gulf Coast and extending about 80 km inland (not shown).
Timeline of actions taken as part of the heavy rainfall CFP during a 7-day period encompassing the landfall of Hurricane Harvey and the subsequent record flood event.
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
The heavy rainfall CFP ramped up significantly on the morning of 24 August 2017, with landfall about 36 h away. One ever-present challenge is to connect higher-confidence rainfall forecasts directly to potential flood impacts. Spatial errors in warm-season convective rainfall forecasts have led RFCs east of the Rocky Mountains to a practice of including 36 h or fewer of forecast rainfall accumulation as forcing for river models (G. Waller 2017, personal communication). WPC was highly confident in the Harvey rainfall forecast on unusually large space and time scales, including beyond 36 h. WPC communicated this to WGRFC early on 24 August during a high-level meeting that regularly occurs outside of the heavy rainfall CFP. By the time another midmorning heavy rainfall CFP call was organized by SR-ROC, WGRFC spoke of “devastating” flood impacts based on the rainfall forecast. Through coordination with WPC, impacted offices, deployed personnel, and NWS Public Affairs, the WGRFC message was eventually changed to “catastrophic and life-threatening” flooding, and was shared consistently across talking-points documents, social media posts, and official forecast products at all levels. The meteorology and hydrology sides of the NWS had worked together to put forth a bold and confident forecast of high-impact flooding on a large scale before a drop of rain had fallen.
The SR-ROC and WPC also determined it would be sensible to follow each 6-hourly NHC hotline call with a heavy rainfall CFP call at 15 min after the top of the hour. While NHC holds scheduled hotline calls at expected times, the WPC methodology—tied also to the Regional Operations Centers and affected by workflows and time zones in different regions—had not yet established clear and consistent expectations. The decision on 24 August established such expectations for the remainder of the event. The SR-ROC facilitator called it “battle rhythm.”
The level of detail conveyed in forecast products increased on 25 August 2017, and the collaboration process and product stream were relatively routine up until about 2010 UTC. Immediately following the NHC hotline call, a heavy rainfall CFP call was set to begin within 5–7 min. This period of time, exceedingly short in operational terms, saw a remarkable collaborative undertaking. A WPC forecaster continued data interrogation and decision-making right up to the time of these calls. Through this continuous data analysis, the forecaster declared to the operations floor a high level of confidence to put forth a High Risk on the Day 3 ERO that was set to be publicly issued following the CFP call. NWS Directives in 2017 did not allow for a day 3 High Risk (this has since changed), owing to a lack of confidence and the state of the science prior to that time. With a large, slow-moving, major hurricane, strong model consensus, and a high-confidence manually adjusted rainfall forecast of “15 to 30 inches, isolated 40 inches,” the consensus of WPC forecasters and managers in the room was to push this idea forward. The WPC Science and Operations Officer confirmed that the necessary script changes to enable High Risk on day 3 graphics could be made easily. The WPC Director sought and received approval from NWSHQ to temporarily operate outside the directives. Moreover, WFO Houston voiced no objections on the conference call. This decision to break code and deliver an emphatic forecast message at 3-day lead time was coordinated from College Park to Silver Spring, Maryland, to Houston, Texas, within about 10 min.
When the same day 3 forecast became day 2 on 26 August, WFO Houston placed a direct call to the WPC Day 2 Desk to continue a conversation that had been occurring via chat software about expanding the High Risk to more wholly include the Houston metropolitan area. Thus, aspects of the forecast were coordinated through a variety of means over the multiday period.
These rains are currently producing catastrophic and life-threatening flooding, and flash flood emergencies are in effect for portions of southeastern Texas. DO NOT ATTEMPT TO TRAVEL IN THE AFFECTED AREA IF YOU ARE IN A SAFE PLACE. DO NOT DRIVE INTO FLOODED ROADWAYS. Please see warnings and other products issued by your local National Weather Service office for additional information on this dire and life-threatening situation.
WPC crafted a tweet that was intended to go viral and speak to the enormity of the unfolding situation (Fig. 7). Expert WPC forecasters could see that the eventual rain footprint from Harvey would be greater in combination of magnitude and coverage than any storm documented before and was occurring over a major city.
Tweet issued from the WPC Twitter account during the escalation of excessive rainfall impacts in Houston on 27 Aug 2017.
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
Heavy rainfall CFP calls continued as a means of keeping the lines of communication open for any issues that needed to bubble up at any given time. One important use of this open line was vetting of rain gauge sources, as the event total rainfall began to breach the continental U.S. record for rainfall from any single tropical cyclone. WFO Houston was able to verify that they knew and trusted the Cedar Bayou gauge,4 which as of the morning of 28 August had exceeded the previous CONUS rainfall record for a tropical cyclone with an observation of 1317 mm (51.88 in.). This confirmation allowed WPC to spread this news in messaging at the national level, including in interviews with national media outlets.
Harvey’s heavy rainfall story gives us many examples of well-practiced execution as well as impromptu invention. As the lead author, then with WPC, told a reporter from Time magazine on 28 August 2017, “In my 15-year career I have never seen the National Weather Service so well coordinated, displaying such cohesiveness, speaking with one voice at all levels, as I am seeing right now.”
Challenges in relating heavy rainfall to impacts
The case studies and the impact-based findings involving the High Risk category illustrate the utility of the ERO for high-end rainfall events. Other, more common events often present shortcomings of the available hydrometeorological tools and local variation in operational practices. These make it difficult to design the ERO to act in concert with all local messaging, but also represent opportunities for research.
Shortcomings of flash flood guidance.
The CONUS is covered by 12 NWS RFCs, and they provide FFG on a 4-km grid as an estimate of the 1-, 3-, and 6-h rain accumulations that would produce flooding on small streams (Sweeney 1992; Schmidt et al. 2007; Clark et al. 2014). FFG gives the ERO its tie-in to hydrology so that what is forecast is not merely a risk of heavy rainfall but of rainfall-related impacts, including flash flooding.
FFG provides a first level assessment of regional sensitivity to heavy rainfall, but it is also fraught with shortcomings that limit its utility. Figure 8 shows examples of national mosaics of 1- and 3-h FFG and denotes several commonly occurring artifacts. WPC forecasters must transition in and out of different thought processes as they evaluate flash flood potential on either side of the continental divide; FFG values over the West remain static, regardless of antecedent conditions. In the East, FFG varies according to antecedent dryness or wetness, but is updated on a 6-hourly cycle; WPC forecasters must mentally adjust FFG in areas where heavy rain falls in between issuances. FFG at respective RFCs has been tuned regionally, even employing different hydrologic models. The product is automated and is not designed to function as a continuous mosaic, so there may be notable discontinuities along RFC borders at times. Even in the West, where values are static, there is at least one noticeable discontinuity; the NWRFC sets significantly lower FFG than adjacent RFCs.
Flash flood guidance at (a) 1- and (b) 3-h increments created at 12 River Forecast Centers across the continental United States and mosaicked for use at WPC.
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
Perhaps the starkest feature in Fig. 8 is the absence of FFG west of the Cascades. In the area that includes the Portland, Oregon, and Seattle, Washington, WFOs the term flash flooding is either not used or may go many years without being seen in local NWS forecasts and warnings (D. McDonnal 2016, personal communication). There are several reasons that flash flooding west of the Cascades crest is rare, including infrequent convective enhancement to the rainfall and a landscape that efficiently absorbs rainfall (e.g., dense canopy, hundreds of small dams). The region also experiences frequent river flooding, and WFOs there report that attempts to resolve and separate flash flooding from river flooding have met with customer and partner confusion (D. Elson 2016, personal communication). Northwest RFC (NWRFC) chooses not to depict FFG west of the Cascades crest in western Washington and northwestern Oregon, leaving excessive rainfall there technically undefined. WPC works with the Portland and Seattle WFOs and the NWRFC to collaboratively design each ERO and related messaging as weather events dictate.
Recent FFG verification studies showed that FFG is only marginally skillful, although there is some variability depending on region and report source (Clark et al. 2014; Gourley et al. 2012; Herman and Schumacher 2018c; Gourley and Vergara 2020; Schumacher and Herman 2021). Whether or not FFG can be substantially improved in the future, the trend in hydrologic modeling is to find suitable replacements, such as output from the NWM and FLASH. WPC retains gridded FFG as a guidance product, and very low or very high FFG values may influence outlook upgrade or downgrade decisions. The definition of excessive rainfall as “rainfall exceeding FFG,” however, is not to be read literally. Due to the shortcomings described here, WPC considers the term “excessive rainfall” to apply more generally as rainfall that would be expected to cause impacts, including flash flooding, excessive stormwater drainage, and soil erosion. WPC produces ERO verification data tuned to the literal product definition, Stage IV precipitation exceedance of FFG, and separately, impact-based data that include FFG exceedance, flash flood reports, and the 5-yr rainfall ARI (Erickson et al. 2021). Stage IV rainfall analysis is generated at River Forecast Centers and combines radar precipitation estimates and rain gauges with some bias correction and manual quality control of data (Nelson et al. 2016).
The influence of reporting practices.
ERO risk areas are driven by a combination of formal definition, regionally variable relationships with the phenomenon, and operational messaging considerations. Relationships with the term “flash flooding” have evolved regionally and locally. A sudden flood wave sweeping vehicles away through sloped terrain is usually reported as flash flooding, and standing water at a four-way intersection in otherwise flat terrain can also be reported as flash flooding. The threat of damage and the risk to lives between the two flooding scenarios is not nearly equal. Many flash flood reports do not contain enough information to determine the relative risk to lives and property, such as reports of “water over the road” and “flash flooding occurred in ….” Some local NWS offices apply a lower threshold for issuing Flash Flood Warnings and verifying them for a number of risky low-water crossings under hilly terrain. Other offices take a minimalist approach, reserving the Flash Flood Warning product for exceptional rain rates relative to the local climate while making more frequent use of the Flood Advisory product.
Variations in warning strategies from 2014 to 2018 are shown in Fig. 9. The figure indicates WFOs use advisory and warning products to varying degrees. It also shows that a given office may report the observed impacts of heavy rainfall in ways that relate to the warning strategy of that office. The observed impacts and/or magnitude of heavy rain may be reported under the event heading of “flash flood,” “flood,” “heavy rain,” or “debris flow” in Storm Data. Color shading of the data points shows that offices displaying an Areal Flood Warning preference (those below the magenta line) tended to record more flood event types than flash flood event types (green data points), and vice versa (red data points). This influences the ERO, as events classified as flood may predominantly include those with causative rainfall that occurs over greater than 6 h and/or does not exceed FFG. Research to better understand the climatology of flash floods that are forced by 0–6-h causative rainfall and the frequency at which events straddle flood and flash flood definitions could improve the synergy of ERO and WFO/RFC products and related messaging.
Ternary plot of the percentages of flash flood, areal flood, and flood advisory products issued per WFO from 2014 to 2018. The marker shapes denote the NWS region. Dark dashed lines leading to the hollow circle show the flood forecast product percentages for an example WFO that issued Flash Flood Warnings ∼30%, Areal Flood Warnings ∼60%, and Flood Advisories ∼10% of the time. The magenta line separates the offices that issued more (below the line) or fewer (above) Areal Flood Warnings compared to Flash Flood Warnings. The color fill of the WFO markers identifies offices that issued more (green) or fewer (red) flood local storm reports than flash flood local storm reports and those which issued about the same number of each (gray).
Citation: Bulletin of the American Meteorological Society 104, 3; 10.1175/BAMS-D-21-0281.1
Summary and future work
WPC scientists and other collaborators are addressing the extreme rainfall challenge, making use of improved observational networks, high-resolution modeling, and a collaborative forecast process to enhance the information provided during the outlook phase of forecasting. These enhancements have accelerated since 2010, with investment into the presentation, dissemination, and underlying science of the ERO. For example, WPC has introduced neighborhood probabilities, ERO-derived graphics posted to the NHC website during landfalling tropical cyclones, and user engagement through new channels such as stakeholder meetings and operational ties to the Federal Emergency Management Agency. The High Risk category, in spite of its sparing use, has been shown to strongly correspond to the total flood and flash flood fatalities and damages recorded over the past decade.
The collaborative forecast process and the ERO will continue evolving as new tools and concepts mature. Experimental Day 4 and 5 EROs are already being produced. WPC is increasing partnership with the NWC to infuse more rigorous and quantitative hydrologic assessment into the ERO and with the NWS Western Region to improve messaging of impactful rainfall events driven by the unique landscape of the western United States. The NWC is developing a National Hydrologic Outlook and other services that will complement meteorological information with detailed hydrologic information.
Current variation in local warning and reporting practices presents a challenge in designing the national-scale ERO to act in concert with all local messaging. The NWS product landscape, however, is undergoing change through expanded use of probabilistic concepts via FACETs (Rothfusz et al. 2018) and through a Hazard Simplification project (Eastern Research Group, Inc. 2020), which has already begun to consolidate and reformat some flood-related products (National Weather Service 2021). Social science research to understand user receipt and understanding of these new products and services will ideally be woven into the research-to-operations process. These efforts, along with increasing synergy between meteorology and hydrology groups in both research and operations, have the potential to aid end-to-end forecasting of excessive rainfall.
The study included events that took place in a High Risk area or nearby Moderate and Slight Risk areas that surrounded a High Risk. The numbers reported are therefore a measure of the entire regional event that takes place on a High Risk day, giving credit to enhanced messaging even outside the technical bounds of the High Risk contour.
The ERO does not forecast flash flooding from causes other than heavy rainfall (ice jams, dam failures, etc.)
NHC advisories on Harvey were suspended during a period of ragged disorganization starting at 0300 UTC 20 August. When the tropical wave strengthened over the Bay of Campeche on 23 August advisories were resumed, and the wave retained the name Harvey.
WFO Houston later learned that this gauge had become submerged, and its measurements were in error. Another gauge near Nederland, TX, recorded 1539 mm (60.58 in.) and was credited with setting a new CONUS tropical cyclone rainfall record.
Acknowledgments.
The views expressed are those of the authors and do not necessarily represent a NOAA position. The authors would like to especially thank WPC managers, developers, and forecasters past and present, including a cadre who form the Rainfall Hazards Team. All these persons’ tireless dedication to their craft has made the ERO what it is today. We would also like to thank the dozens of participants, including all those from NWS field offices, who have contributed to the experimental evaluation of the ERO at the annual Flash Flood and Intense Rainfall Experiment. NCEP leadership has set a supportive environment, and interaction with NWS Regions, the National Water Center, NSSL, and WPC’s sister offices (especially the National Hurricane Center and Storm Prediction Center) have helped improve the scientific rigor of the ERO while raising awareness of the ERO program among end users. This paper also benefited tremendously from formal reviews (both from BAMS and Weather and Forecasting) and internal reviews from David Novak, Steven Martinaitis, and Pam Heinselman.
Data availability statement.
Quantitative findings were derived from NWS warning and advisory polygons from the Iowa Mesonet archive at https://mesonet.agron.iastate.edu/archive/, from Storm Data at www.ncdc.noaa.gov/stormevents/, and from archives of the Excessive Rainfall Outlook product housed at WPC. ERO heat maps are archived online at https://ftp.wpc.ncep.noaa.gov/ERO_verif/, and QPF and ERO graphical products at www.wpc.ncep.noaa.gov/archives/web_pages/ero/ero.shtml. One year of ERO archive in GIS format is available via ftp using the instructions at www.wpc.ncep.noaa.gov/html/about_gis.shtml. For a longer archive, please contact the authors.
References
Barthold, F. E., T. E. Workoff, B. A. Cosgrove, J. J. Gourley, D. R. Novak, and K. M. Mahoney, 2015: Improving flash flood forecasts: The HMT-WPC flash flood and intense rainfall experiment. Bull. Amer. Meteor. Soc., 96, 1859–1866, https://doi.org/10.1175/BAMS-D-14-00201.1.
Blake, E. S., and D. A. Zelinsky, 2018: Hurricane Harvey. NOAA/National Hurricane Center, accessed 19 February 2021, www.nhc.noaa.gov/data/tcr/AL092017_Harvey.pdf.
Brill, K., 2017: WPC 6-hour probabilistic precipitation guidance for days 1–3. NOAA/Weather Prediction Center, accessed 25 March 2021, www.wpc.ncep.noaa.gov/pqpf/conus_hpc_pqpf.php.
Clark, R. A., J. J. Gourley, Z. L. Flamig, Y. Hong, and E. Clark, 2014: CONUS-wide evaluation of National Weather Service flash flood guidance products. Wea. Forecasting, 29, 377–392, https://doi.org/10.1175/WAF-D-12-00124.1.
Cosgrove, B., and C. Klemmer, 2016: The National Water Model. NWS Office of Water Prediction, accessed 15 December 2020, https://water.noaa.gov/about/nwm.
Daly, C., R. P. Neilson, and D. L. Phillips, 1994: A statistical–topographic model for mapping climatological precipitation over mountainous terrain. J. Appl. Meteor., 33, 140–158, https://doi.org/10.1175/1520-0450(1994)033<0140:ASTMFM>2.0.CO;2.
Daly, C., M. Halbleib, J. Smith, W. Gibson, M. Doggett, G. Taylor, J. Curtis, and P. Pasteris, 2008: Physiographically sensitive mapping of climatological temperature and precipitation across the conterminous United States. Int. J. Climatol., 28, 2031–2064, https://doi.org/10.1002/joc.1688.
Doswell, C. A., H. E. Brooks, and R. A. Maddox, 1996: Flash flood forecasting: An ingredients-based methodology. Wea. Forecasting, 11, 560–581, https://doi.org/10.1175/1520-0434(1996)011<0560:FFFAIB>2.0.CO;2.
Eastern Research Group, Inc., 2020: Support for the Hazard Simplification Project Phase V: Findings and recommendations from the remote focus groups. NOAA/National Weather Service, accessed 21 March 2021, www.weather.gov/media/hazardsimplification/Final%20Report_Summary%20of%20Remote%20Focus%20Groups_9_29_20.pdf.
Erickson, M. J., J. S. Kastman, B. Albright, S. Perfater, J. A. Nelson, R. S. Schumacher, and G. R. Herman, 2019: Verification results from the 2017 HMT–WPC flash flood and intense rainfall experiment. J. Appl. Meteor. Climatol., 58, 2591–2604, https://doi.org/10.1175/JAMC-D-19-0097.1.
Erickson, M. J., B. Albright, and J. A. Nelson, 2021: Verifying and redefining the Weather Prediction Center’s Excessive Rainfall Outlook forecast product. Wea. Forecasting, 36, 325–340, https://doi.org/10.1175/WAF-D-20-0020.1.
Garmon, J., and Coauthors, 2020: September–October 2018 Hurricane Florence and Hurricane Michael. National Weather Service, Service Assessments, 164 pp., www.weather.gov/media/publications/assessments/Hurricanes_Florence_Michael4-20.pdf.
Gochis, D., and Coauthors, 2015: The great Colorado flood of September 2013. Bull. Amer. Meteor. Soc., 96, 1461–1487, https://doi.org/10.1175/BAMS-D-13-00241.1.
Gourley, J. J., and H. Vergara, 2020: Comments on “Flash flood verification: Pondering precipitation proxies”. J. Hydrometeor., 22, 739–747, https://doi.org/10.1175/JHM-D-20-0215.1.
Gourley, J. J., J. M. Erlingis, Y. Hong, and E. B. Wells, 2012: Evaluation of tools used for monitoring and forecasting flash floods in the United States. Wea. Forecasting, 27, 158–173, https://doi.org/10.1175/WAF-D-10-05043.1.
Gourley, J. J., and Coauthors, 2017: The FLASH Project: Improving the tools for flash flood monitoring and prediction across the United States. Bull. Amer. Meteor. Soc., 98, 361–372, https://doi.org/10.1175/BAMS-D-15-00247.1.
Hamill, T. M., E. Engle, D. Myrick, M. Peroutka, C. Finan, and M. Scheuerer, 2017: The U.S. national blend of models for statistical postprocessing of probability of precipitation and deterministic precipitation amount. Mon. Wea. Rev., 145, 3441–3463, https://doi.org/10.1175/MWR-D-16-0331.1.
Harrison, D. R., M. S. Elliott, I. L. Jirak, and P. T. Marsh, 2022: Utilizing the high-resolution ensemble forecast system to produce calibrated probabilistic thunderstorm guidance. Wea. Forecasting, 37, 1103–1115, https://doi.org/10.1175/WAF-D-22-0001.1.
Herman, G. R., and R. S. Schumacher, 2018a: “Dendrology” in numerical weather Prediction: What random forests and logistic regression tell us about forecasting extreme precipitation. Mon. Wea. Rev., 146, 1785–1812, https://doi.org/10.1175/MWR-D-17-0307.1.
Herman, G. R., and R. S. Schumacher, 2018b: Money doesn’t grow on trees, but forecasts do: Forecasting extreme precipitation with random forests. Mon. Wea. Rev., 146, 1571–1600, https://doi.org/10.1175/MWR-D-17-0250.1.
Herman, G. R., and R. S. Schumacher, 2018c: Flash flood verification: Pondering precipitation proxies. J. Hydrometeor., 19, 1753–1776, https://doi.org/10.1175/JHM-D-18-0092.1.
Highland, L. M., and P. Bobrowsky, 2008: The Landslide Handbook: A guide to understanding landslides. U.S. Department of the Interior and U.S. Geological Survey, 147 pp., https://pubs.usgs.gov/circ/1325/pdf/C1325_508.pdf.
Hitchens, N. M., and H. E. Brooks, 2014: Evaluation of the storm prediction center’s convective outlooks from day 3 through day 1. Wea. Forecasting, 29, 1134–1142, https://doi.org/10.1175/WAF-D-13-00132.1.
Kean, J. W., and Coauthors, 2019: Inundation, flow dynamics, and damage in the 9 January 2018 Montecito debris-flow event, California, USA: Opportunities and challenges for post-wildfire risk assessment. Geosphere, 15, 1140–1163, https://doi.org/10.1130/GES02048.1.
Lincoln, W. S., and R. F. L. Thomason, 2018: A preliminary look at using rainfall average recurrence interval to characterize flash flood events for real-time warning forecasting. J. Oper. Meteor., 6, 13–22, https://doi.org/10.15191/nwajom.2018.0602.
Martinaitis, S. M., S. B. Cocks, A. P. Osborne, M. J. Simpson, L. Tang, J. Zhang, and K. W. Howard, 2021: The historic rainfalls of Hurricanes Harvey and Florence: A perspective from the Multi-Radar Multi-Sensor system. J. Hydrometeor., 22, 721–738, https://doi.org/10.1175/JHM-D-20-0199.1.
Mileti, D. S., and J. H. Sorenson, 1990: Communication of emergency public warnings. Oak Ridge National Laboratory Rep. ORNL-6609, 162 pp., https://doi.org/10.2172/6137387.
National Centers for Environmental Information, 2019: National Climate Report – February 2019. Accessed 19 February 2021, www.ncdc.noaa.gov/sotc/national/201902.
National Centers for Environmental Information, 2022: U.S. billion-dollar weather and climate disasters. NOAA/NCEI, https://doi.org/10.25921/stkw-7w73.
National Weather Service, 2019a: National water resources products specification. National Weather Service Instruction 10-930, 13 pp., www.nws.noaa.gov/directives/sym/pd01009030curr.pdf.
National Weather Service, 2019b: Weather Forecast Office water resources products specification. National Weather Service Instruction 10-922, 92 pp., www.nws.noaa.gov/directives/sym/pd01009022curr.pdf.
National Weather Service, 2021: Service change notice 21-56. National Weather Service Headquarters, accessed 30 April 2022, www.weather.gov/media/notification/pdf2/scn21-56_consolidation_reformatting_flood_products_aab.pdf.
National Weather Service Baltimore/Washington Weather Forecast Office, 2016: Ellicott City historic rain and flash flooding of July 30, 2016. Accessed 19 February 2021, www.weather.gov/lwx/EllicottCityFlood2016#:∼:text=On%20Saturday%2C%20July%2030th%2C%202016,Two%20fatalities%20have%20been%20reported.
National Weather Service New Orleans/Baton Rouge Weather Forecast Office, 2019: Mississippi River flood history 1543–present. Accessed 19 February 2021, www.weather.gov/lix/ms_flood_history.
National Weather Service Operations Proving Ground, 2020: Operational Evaluation Report: The Collaborative Forecast Process for QPF. NWS Operations Proving Ground, 50 pp., https://vlab.noaa.gov/documents/214451/5286029/CFP+%28QPF%29+Virtual+Experiment+Report.pdf/0a42a6de-4254-c3df-ccf3-8cb9f7bd3583?t=1605800868969.
Nelson, B., O. Prat, D. Seo, and E. Habib, 2016: Assessment and implications of NCEP stage IV quantitative precipitation estimates for product comparisons. Wea. Forecasting, 31, 371–394, https://doi.org/10.1175/WAF-D-14-00112.1.
Novak, D., 2019: WPC probabilistic quantitative precipitation forecasts. NOAA/Weather Prediction Center, accessed 25 March 2021, www.wpc.ncep.noaa.gov/pqpf/PDD_WPC_PQPF.pdf.
Novak, D. R., C. Bailey, K. F. Brill, P. Burke, W. A. Hogsett, R. Rausch, and M. Schichtel, 2014: Precipitation and temperature forecast performance at the Weather Prediction Center. Wea. Forecasting, 29, 489–504, https://doi.org/10.1175/WAF-D-13-00066.1.
Roberts, B., B. T. Gallo, I. L. Jirak, and A. J. Clark, 2019: The High Resolution Ensemble Forecast (HREF) system: Applications and performance for forecasting convective storms. 2019 Fall Meeting, San Francisco, CA, Amer. Geophys. Union, Abstract A310-2797.
Rothfusz, L. P., R. Schneider, D. Novak, K. Klockow-McClain, A. E. Gerard, C. Karstens, G. J. Stumpf, and T. M. Smith, 2018: FACETs: A proposed next generation paradigm for high impact weather forecasting. Bull. Amer. Meteor. Soc., 99, 2025–2043, https://doi.org/10.1175/BAMS-D-16-0100.1.
Schmidt, J., A. Anderson, and J. Paul, 2007: Spatially-variable, physically-derived, flash flood guidance. 21st Conf. on Hydrology, San Antonio, TX, Amer. Meteor. Soc., 6B.2, https://ams.confex.com/ams/pdfpapers/120022.pdf.
Schroeder, A., and Coauthors, 2016: The development of a Flash Flood Severity Index. J. Hydrol., 541, 523–532, https://doi.org/10.1016/j.jhydrol.2016.04.005.
Schumacher, R. S., and G. R. Herman, 2021: Reply to “Comments on ‘Flash flood verification: Pondering precipitation proxies.’” J. Hydrometeor., 22, 749–752, https://doi.org/10.1175/JHM-D-20-0275.1.
Schumacher, R. S., A. J. Hill, M. Klein, J. A. Nelson, M. J. Erickson, S. M. Trojniak, and G. R. Herman, 2021: From random forests to flood forecasts: A research to operations success story. Bull. Amer. Meteor. Soc., 102, E1742–E1755, https://doi.org/10.1175/BAMS-D-20-0186.1.
Stewart, S. R., and R. Berg, 2019: Hurricane Florence. NOAA/National Hurricane Center, accessed 19 February 2021, www.nhc.noaa.gov/data/tcr/AL062018_Florence.pdf.
Sweeney, T. L., 1992: Modernized areal flash flood guidance. NOAA Tech. Memo NWS HYDRO 44, 37 pp., https://repository.library.noaa.gov/view/noaa/13498.
Tryhorn, L., A. Lynch, R. Abramson, and K. Parkyn, 2008: On the meteorological mechanisms driving postfire flash floods: A case study. Mon. Wea. Rev., 136, 1778–1791, https://doi.org/10.1175/2007MWR2218.1.
Uccellini, L. W., and J. E. Ten Hoeve, 2019: Evolving the National Weather Service to build a Weather-Ready Nation: Connecting observations, forecasts, and warnings to decision-makers through Impact-Based Decision Support Services. Bull. Amer. Meteor. Soc., 100, 1923–1942, https://doi.org/10.1175/BAMS-D-18-0159.1.
Viterbo, F., and Coauthors, 2020: A multiscale, hydrometeorological forecast evaluation of National Water Model forecasts of the May 2018 Ellicott City, Maryland, flood. J. Hydrometeor., 21, 475–499, https://doi.org/10.1175/JHM-D-19-0125.1.
