Search Results

You are looking at 1 - 6 of 6 items for

  • Author or Editor: Benjamin Albright x
  • All content x
Clear All Modify Search
Michael J. Erickson, Benjamin Albright, and James A. Nelson

Abstract

The Weather Prediction Center’s Excessive Rainfall Outlook (ERO) forecasts the probability of rainfall exceeding flash flood guidance within 40 km of a point. This study presents a comprehensive ERO verification between 2015 and 2019 using a combination of flooding observations and proxies. ERO spatial issuance frequency plots are developed to provide situational awareness for forecasters. Reliability of the ERO is assessed by computing fractional coverage of the verification within each probabilistic category. Probabilistic forecast skill is evaluated using the Brier skill score (BSS) and area under the relative operating characteristic (AUC). A “probabilistic observation” called practically perfect (PP) is developed and compared to the ERO as an additional measure of skill. The areal issuance frequency of the ERO varies spatially with the most abundant issuances spanning from the Gulf Coast to the Midwest and the Appalachians. ERO issuances occur most often in the summer and are associated with the Southwestern monsoon, mesoscale convective systems, and tropical cyclones. The ERO exhibits good reliability on average, although more recent trends suggest some ERO-defined probabilistic categories should be issued more frequently. AUC and BSS are useful bulk skill metrics, while verification against PP is useful in bulk and for shorter-term ERO evaluation. ERO forecasts are generally more skillful at shorter lead times in terms of AUC and BSS. There is no trend in ERO area size over 5 years, although ERO forecasts may be getting slightly more skillful in terms of critical success index when verified against the PP.

Restricted access
Benjamin T. Blake, Jacob R. Carley, Trevor I. Alcott, Isidora Jankov, Matthew E. Pyle, Sarah E. Perfater, and Benjamin Albright

Abstract

Traditional ensemble probabilities are computed based on the number of members that exceed a threshold at a given point divided by the total number of members. This approach has been employed for many years in coarse-resolution models. However, convection-permitting ensembles of less than ~20 members are generally underdispersive, and spatial displacement at the gridpoint scale is often large. These issues have motivated the development of spatial filtering and neighborhood postprocessing methods, such as fractional coverage and neighborhood maximum value, which address this spatial uncertainty. Two different fractional coverage approaches for the generation of gridpoint probabilities were evaluated. The first method expands the traditional point probability calculation to cover a 100-km radius around a given point. The second method applies the idea that a uniform radius is not appropriate when there is strong agreement between members. In such cases, the traditional fractional coverage approach can reduce the probabilities for these potentially well-handled events. Therefore, a variable radius approach has been developed based upon ensemble agreement scale similarity criteria. In this method, the radius size ranges from 10 km for member forecasts that are in good agreement (e.g., lake-effect snow, orographic precipitation, very short-term forecasts, etc.) to 100 km when the members are more dissimilar. Results from the application of this adaptive technique for the calculation of point probabilities for precipitation forecasts are presented based upon several months of objective verification and subjective feedback from the 2017 Flash Flood and Intense Rainfall Experiment.

Full access
Michael J. Erickson, Joshua S. Kastman, Benjamin Albright, Sarah Perfater, James A. Nelson, Russ S. Schumacher, and Gregory R. Herman

Abstract

The Flash Flood and Intense Rainfall (FFaIR) Experiment developed within the Hydrometeorology Testbed (HMT) of the Weather Prediction Center (WPC) is a pseudo-operational platform for participants from across the weather enterprise to test emerging flash flood forecasting tools and issue experimental forecast products. This study presents the objective verification portion of the 2017 edition of the experiment, which examines the performance from a variety of guidance tools (deterministic models, ensembles, and machine-learning techniques) and the participants’ forecasts, with occasional reference to the participants’ subjective ratings. The skill of the model guidance used in the FFaIR Experiment is evaluated using performance diagrams verified against the Stage IV analysis. The operational and FFaIR Experiment versions of the excessive rainfall outlook (ERO) are evaluated by assessing the frequency of issuances, probabilistic calibration, Brier skill score (BSS), and area under relative operating characteristic (AuROC). An ERO first-guess field called the Colorado State University Machine-Learning Probabilities method (CSU-MLP) is also evaluated in the FFaIR Experiment. Among convection-allowing models, the Met Office Unified Model generally performed optimally throughout the FFaIR Experiment when using performance diagrams (at the 0.5- and 1-in. thresholds; 1 in. = 25.4 mm), whereas the High-Resolution Rapid Refresh (HRRR), version 3, performed best subjectively. In terms of subjective and objective ensemble scores, the HRRR ensemble scored optimally. The CSU-MLP overpredicted lower risk categories and underpredicted higher risk categories, but it shows future promise as an ERO first-guess field. The EROs issued by the FFaIR Experiment forecasters had improved BSS and AuROC relative to the operational ERO, suggesting that the experimental guidance may have aided forecasters.

Free access
Nathan Snook, Fanyou Kong, Keith A. Brewster, Ming Xue, Kevin W. Thomas, Timothy A. Supinie, Sarah Perfater, and Benjamin Albright

Abstract

During the summers of 2016 and 2017, the Center for Analysis and Prediction of Storms (CAPS) ran real-time storm-scale ensemble forecasts (SSEFs) in support of the Hydrometeorology Testbed (HMT) Flash Flood and Intense Rainfall (FFaIR) experiment. These forecasts, using WRF-ARW and Nonhydrostatic Mesoscale Model on the B-grid (NMMB) in 2016, and WRF-ARW and GFDL Finite Volume Cubed-Sphere Dynamical Core (FV3) in 2017, covered the contiguous United States at 3-km horizontal grid spacing, and supported the generation and evaluation of precipitation forecast products, including ensemble probabilistic products. Forecasts of 3-h precipitation accumulation are evaluated. Overall, the SSEF produces skillful 3-h accumulated precipitation forecasts, with ARW members generally outperforming NMMB members and the single FV3 member run in 2017 outperforming ARW members; these differences are significant at some forecast hours. Statistically significant differences exist in the performance, in terms of bias and ETS, among subensembles of members sharing common microphysics and PBL schemes. Year-to-year consistency is higher for PBL subensembles than for microphysical subensembles. Probability-matched (PM) ensemble mean forecasts outperform individual members, while the simple ensemble mean exhibits substantial bias. A newly developed localized probability-matched (LPM) ensemble mean product was produced in 2017; compared to the simple ensemble mean and the conventional PM mean, the LPM mean exhibits improved retention of small-scale structures, evident in both 2D forecast fields and variance spectra. Probabilistic forecasts of precipitation exceeding flash flood guidance (FFG) or thresholds associated with recurrence intervals (RI) ranging from 10 to 100 years show utility in predicting regions of flooding threat, but generally overpredict the occurrence of such events; however, they may still be useful in subjective flash flood risk assessment.

Full access
Steven M. Martinaitis, Jonathan J. Gourley, Zachary L. Flamig, Elizabeth M. Argyle, Robert A. Clark III, Ami Arthur, Brandon R. Smith, Jessica M. Erlingis, Sarah Perfater, and Benjamin Albright

Abstract

There are numerous challenges with the forecasting and detection of flash floods, one of the deadliest weather phenomena in the United States. Statistical metrics of flash flood warnings over recent years depict a generally stagnant warning performance, while regional flash flood guidance utilized in warning operations was shown to have low skill scores. The Hydrometeorological Testbed—Hydrology (HMT-Hydro) experiment was created to allow operational forecasters to assess emerging products and techniques designed to improve the prediction and warning of flash flooding. Scientific goals of the HMT-Hydro experiment included the evaluation of gridded products from the Multi-Radar Multi-Sensor (MRMS) and Flooded Locations and Simulated Hydrographs (FLASH) product suites, including the experimental Coupled Routing and Excess Storage (CREST) model, the application of user-defined probabilistic forecasts in experimental flash flood watches and warnings, and the utility of the Hazard Services software interface with flash flood recommenders in real-time experimental warning operations. The HMT-Hydro experiment ran in collaboration with the Flash Flood and Intense Rainfall (FFaIR) experiment at the Weather Prediction Center to simulate the real-time workflow between a national center and a local forecast office, as well as to facilitate discussions on the challenges of short-term flash flood forecasting. Results from the HMT-Hydro experiment highlighted the utility of MRMS and FLASH products in identifying the spatial coverage and magnitude of flash flooding, while evaluating the perception and reliability of probabilistic forecasts in flash flood watches and warnings.

NSSL scientists and NWS forecasters evaluate new tools and techniques through real-time test bed operations for the improvement of flash flood detection and warning operations.

Full access
Steven M. Martinaitis, Benjamin Albright, Jonathan J. Gourley, Sarah Perfater, Tiffany Meyer, Zachary L. Flamig, Robert A. Clark, Humberto Vergara, and Mark Klein

Abstract

The flash flood event of 23 June 2016 devastated portions of West Virginia and west-central Virginia, resulting in 23 fatalities and 5 new record river crests. The flash flooding was part of a multiday event that was classified as a billion-dollar disaster. The 23 June 2016 event occurred during real-time operations by two Hydrometeorology Testbed (HMT) experiments. The Flash Flood and Intense Rainfall (FFaIR) experiment focused on the 6–24-h forecast through the utilization of experimental high-resolution deterministic and ensemble numerical weather prediction and hydrologic model guidance. The HMT Multi-Radar Multi-Sensor Hydro (HMT-Hydro) experiment concentrated on the 0–6-h time frame for the prediction and warning of flash floods primarily through the experimental Flooded Locations and Simulated Hydrographs product suite. This study describes the various model guidance, applications, and evaluations from both testbed experiments during the 23 June 2016 flash flood event. Various model outputs provided a significant precipitation signal that increased the confidence of FFaIR experiment participants to issue a high risk for flash flooding for the region between 1800 UTC 23 June and 0000 UTC 24 June. Experimental flash flood warnings issued during the HMT-Hydro experiment for this event improved the probability of detection and resulted in a 63.8% increase in lead time to 84.2 min. Isolated flash floods in Kentucky demonstrated the potential to reduce the warned area. Participants characterized how different model guidance and analysis products influenced the decision-making process and how the experimental products can help shape future national and local flash flood operations.

Restricted access