Verifying and Redefining the Weather Prediction Center’s Excessive Rainfall Outlook Forecast Product

View More View Less
  • 1 Cooperative Institute for Research in Environmental Sciences, University of Colorado, College Park, MD
  • 2 NOAA/NWS/NCEP/Weather Prediction Center, College Park, MD
  • 3 Systems Research Group, Inc., College Park, MD
© Get Permissions
Restricted access

Abstract

The Weather Prediction Center’s Excessive Rainfall Outlook (ERO) forecasts the probability of rainfall exceeding Flash Flood Guidance within 40 km of a point. This study presents a comprehensive ERO verification between 2015 and 2019 using a combination of flooding observations and proxies. ERO spatial issuance frequency plots are developed to provide situational awareness for forecasters. Reliability of the ERO is assessed by computing fractional coverage of the verification within each probabilistic category. Probabilistic forecast skill is evaluated using Brier Skill Score (BSS) and Area Under the Relative Operating Characteristic (AUC). A “probabilistic observation” called Practically Perfect (PP) is developed and compared to the ERO as an additional measure of skill.

The areal issuance frequency of the ERO varies spatially with the most abundant issuances spanning from the Gulf Coast to the Midwest and Appalachians. ERO issuances occur most often in the summer and are associated with the Southwestern Monsoon, mesoscale convective systems, and tropical cyclones. The ERO exhibits good reliability on average, although more recent trends suggest some ERO defined probabilistic categories should be issued more frequently.

AUC and BSS are useful bulk skill metrics, while verification against PP is useful in bulk and for shorter-term ERO evaluation. ERO forecasts are generally more skillful at shorter lead times in terms of AUC and BSS. There is no trend in ERO area size over five years, although ERO forecasts may be getting slightly more skillful in terms of critical success index when verified against the PP.

Corresponding author address: Michael Erickson, National Center for Weather and Climate Prediction, Weather Prediction Center, 5830 University Research Court, College Park, MD 20740. E-mail: mjaerickson@gmail.com

Abstract

The Weather Prediction Center’s Excessive Rainfall Outlook (ERO) forecasts the probability of rainfall exceeding Flash Flood Guidance within 40 km of a point. This study presents a comprehensive ERO verification between 2015 and 2019 using a combination of flooding observations and proxies. ERO spatial issuance frequency plots are developed to provide situational awareness for forecasters. Reliability of the ERO is assessed by computing fractional coverage of the verification within each probabilistic category. Probabilistic forecast skill is evaluated using Brier Skill Score (BSS) and Area Under the Relative Operating Characteristic (AUC). A “probabilistic observation” called Practically Perfect (PP) is developed and compared to the ERO as an additional measure of skill.

The areal issuance frequency of the ERO varies spatially with the most abundant issuances spanning from the Gulf Coast to the Midwest and Appalachians. ERO issuances occur most often in the summer and are associated with the Southwestern Monsoon, mesoscale convective systems, and tropical cyclones. The ERO exhibits good reliability on average, although more recent trends suggest some ERO defined probabilistic categories should be issued more frequently.

AUC and BSS are useful bulk skill metrics, while verification against PP is useful in bulk and for shorter-term ERO evaluation. ERO forecasts are generally more skillful at shorter lead times in terms of AUC and BSS. There is no trend in ERO area size over five years, although ERO forecasts may be getting slightly more skillful in terms of critical success index when verified against the PP.

Corresponding author address: Michael Erickson, National Center for Weather and Climate Prediction, Weather Prediction Center, 5830 University Research Court, College Park, MD 20740. E-mail: mjaerickson@gmail.com
Save