Search Results

You are looking at 1 - 3 of 3 items for

  • Author or Editor: Jessica J. Choate x
  • All content x
Clear All Modify Search
Katie A. Wilson, Pamela L. Heinselman, Patrick S. Skinner, Jessica J. Choate, and Kim E. Klockow-McClain

Abstract

During the 2017 Spring Forecasting Experiment in NOAA’s Hazardous Weather Testbed, 62 meteorologists completed a survey designed to test their understanding of forecast uncertainty. Survey questions were based on probabilistic forecast guidance provided by the NSSL Experimental Warn-on-Forecast System for ensembles (NEWS-e). A mix of 20 multiple-choice and open-ended questions required participants to explain basic probability and percentile concepts, extract information using graphical representations of uncertainty, and determine what type of weather scenario the graphics depicted. Multiple-choice questions were analyzed using frequency counts, and open-ended questions were analyzed using thematic coding methods. Of the 18 questions that could be scored, 60%–96% of the participants’ responses aligned with the researchers’ intended response. Some of the most challenging questions proved to be those requiring qualitative explanations, such as to explain what the 70th-percentile value of accumulated rainfall represents in an ensemble-based probabilistic forecast. Additionally, participants providing answers not aligning with the intended response oftentimes appeared to consider the given information with a deterministic rather than probabilistic mindset. Applications of a deterministic mindset resulted in tendencies to focus on the worst-case scenario and to modify understanding of probabilistic concepts when presented with different variables. The findings from this survey support the need for improved basic and applied training for the development, interpretation, and use of probabilistic ensemble forecast guidance. Future work should collect data for a larger sample size to examine the knowledge gaps across specific user groups and to guide development of probabilistic forecast training tools.

Full access
Patrick S. Skinner, Dustan M. Wheatley, Kent H. Knopfmeier, Anthony E. Reinhart, Jessica J. Choate, Thomas A. Jones, Gerald J. Creager, David C. Dowell, Curtis R. Alexander, Therese T. Ladwig, Louis J. Wicker, Pamela L. Heinselman, Patrick Minnis, and Rabindra Palikonda

Abstract

An object-based verification methodology for the NSSL Experimental Warn-on-Forecast System for ensembles (NEWS-e) has been developed and applied to 32 cases between December 2015 and June 2017. NEWS-e forecast objects of composite reflectivity and 30-min updraft helicity swaths are matched to corresponding reflectivity and rotation track objects in Multi-Radar Multi-Sensor system data on space and time scales typical of a National Weather Service warning. Object matching allows contingency-table-based verification statistics to be used to establish baseline performance metrics for NEWS-e thunderstorm and mesocyclone forecasts. NEWS-e critical success index (CSI) scores of reflectivity (updraft helicity) forecasts decrease from approximately 0.7 (0.4) to 0.4 (0.2) over 3 h of forecast time. CSI scores decrease through the forecast period, indicating that errors do not saturate during the 3-h forecast. Lower verification scores for rotation track forecasts are primarily a result of a high-frequency bias. Comparison of different system configurations used in 2016 and 2017 shows an increase in skill for 2017 reflectivity forecasts, attributable mainly to improvements in the forecast initial conditions. A small decrease in skill in 2017 rotation track forecasts is likely a result of sample differences between 2016 and 2017. Although large case-to-case variation is present, evidence is found that NEWS-e forecast skill improves with increasing object size and intensity.

Full access
Adam J. Clark, Israel L. Jirak, Burkely T. Gallo, Brett Roberts, Kent. H. Knopfmeier, Robert A. Clark, Jake Vancil, Andrew R. Dean, Kimberly A. Hoogewind, Pamela L. Heinselman, Nathan A. Dahl, Makenzie J. Krocak, Jessica J. Choate, Katie A. Wilson, Patrick S. Skinner, Thomas A. Jones, Yunheng Wang, Gerald J. Creager, Larissa J. Reames, Louis J. Wicker, Scott R. Dembek, and Steven J. Weiss
Full access