Exploring Convection-Allowing Model Evaluation Strategies for Severe Local Storms using the Finite-Volume Cubed-Sphere (FV3) Model Core

View More View Less
  • 1 Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, Norman, Oklahoma
  • 2 NOAA/NWS/NCEP Storm Prediction Center, Norman, Oklahoma
  • 3 Research Applications Laboratory, National Center for Atmospheric Research, and Developmental Testbed Center, Boulder, Colorado
  • 4 NOAA/OAR National Severe Storms Laboratory, Norman, Oklahoma
  • 5 Center for Analysis and Prediction of Storms, Norman, Oklahoma
  • 6 NOAA/NWS/NCEP Environmental Modeling Center, and I.M. Systems Group, College Park, Maryland
  • 7 NOAA/OAR Geophysical Fluid Dynamics Laboratory, Princeton, New Jersey
  • 8 NOAA/Earth System Research Laboratory/Global Systems Division, Boulder, Colorado
© Get Permissions
Restricted access

Abstract

Verification methods for convection-allowing models (CAMs) should consider the fine-scale spatial and temporal detail provided by CAMs, and including both neighborhood and object-based methods can account for displaced features that may still provide useful information. This work explores both contingency table-based verification techniques and object-based verification techniques as they relate to forecasts of severe convection. Two key fields in severe weather forecasting are investigated: updraft helicity (UH) and simulated composite reflectivity. UH is used to generate severe weather probabilities called surrogate severe fields, which have two tunable parameters: the UH threshold and the smoothing level. Probabilities computed using the UH threshold and smoothing level that give the best area under the receiver operating curve result in very high probabilities while optimizing the parameters based on the Brier Score reliability component results in much lower probabilities. Subjective ratings from participants in the 2018 NOAA Hazardous Weather Testbed Spring Forecasting Experiment (SFE) provide a complementary evaluation source.

This work compares the verification methodologies in the context of three CAMs using the Finite-Volume Cubed-Sphere Dynamical Core (FV3), which will be the foundation of the United States’ Unified Forecast System (UFS). Three agencies ran FV3-based CAMs during the five-week 2018 SFE. These FV3-based CAMs are verified alongside a current operational CAM, the High-Resolution Rapid Refresh version 3 (HRRRv3). The HRRR is planned to eventually use the FV3 dynamical core as part of the UFS; as such evaluations relative to current HRRR configurations are imperative to maintaining high forecast quality and informing future implementation decisions.

Corresponding author’s address: Burkely T. Gallo, National Weather Center, SPC 120 David L. Boren Blvd., Norman, OK, 73072. Email: burkely.twiest@noaa.gov, Phone: 405-325-2793

Abstract

Verification methods for convection-allowing models (CAMs) should consider the fine-scale spatial and temporal detail provided by CAMs, and including both neighborhood and object-based methods can account for displaced features that may still provide useful information. This work explores both contingency table-based verification techniques and object-based verification techniques as they relate to forecasts of severe convection. Two key fields in severe weather forecasting are investigated: updraft helicity (UH) and simulated composite reflectivity. UH is used to generate severe weather probabilities called surrogate severe fields, which have two tunable parameters: the UH threshold and the smoothing level. Probabilities computed using the UH threshold and smoothing level that give the best area under the receiver operating curve result in very high probabilities while optimizing the parameters based on the Brier Score reliability component results in much lower probabilities. Subjective ratings from participants in the 2018 NOAA Hazardous Weather Testbed Spring Forecasting Experiment (SFE) provide a complementary evaluation source.

This work compares the verification methodologies in the context of three CAMs using the Finite-Volume Cubed-Sphere Dynamical Core (FV3), which will be the foundation of the United States’ Unified Forecast System (UFS). Three agencies ran FV3-based CAMs during the five-week 2018 SFE. These FV3-based CAMs are verified alongside a current operational CAM, the High-Resolution Rapid Refresh version 3 (HRRRv3). The HRRR is planned to eventually use the FV3 dynamical core as part of the UFS; as such evaluations relative to current HRRR configurations are imperative to maintaining high forecast quality and informing future implementation decisions.

Corresponding author’s address: Burkely T. Gallo, National Weather Center, SPC 120 David L. Boren Blvd., Norman, OK, 73072. Email: burkely.twiest@noaa.gov, Phone: 405-325-2793
Save