Search Results

You are looking at 1 - 10 of 15 items for

  • Author or Editor: Katie A. Wilson x
  • Refine by Access: All Content x
Clear All Modify Search
Katie A. Wilson, Pamela L. Heinselman, and Charles M. Kuster

Abstract

Thirty National Weather Service forecasters worked with 1-, 2-, and 5-min phased-array radar (PAR) volumetric updates for a variety of weather events during the 2015 Phased Array Radar Innovative Sensing Experiment. Exposure to each of these temporal resolutions during simulated warning operations meant that these forecasters could provide valuable feedback on how rapidly updating PAR data impacted their warning decision processes. To capture this feedback, forecasters participated in one of six focus groups. A series of open-ended questions guided focus group discussions, and forecasters were encouraged to share their experiences and opinions from the experiment. Transcriptions of focus group discussions were thematically analyzed, and themes belonging to one of two groups were identified: 1) forecasters’ use of rapidly updating PAR data during the experiment and 2) how forecasters envision rapidly updating PAR data being integrated into warning operations. Findings from this thematic analysis are presented in this paper, and to illustrate these findings from the forecasters’ perspectives, dialogue that captures the essence of their discussions is shared. The identified themes provide motivation to integrate rapidly updating radar data into warning operations and highlight important factors that need to be addressed for the successful integration of these data.

Full access
Katie A. Wilson, Pamela L. Heinselman, and Ziho Kang

Abstract

An eye-tracking experiment was conducted to examine whether differences in forecasters’ eye movements provide further insight into how radar update speed impacts their warning decision process. In doing so, this study also demonstrates the applications of a new research method for observing how National Weather Service forecasters distribute their attention across a radar display and warning interface. In addition to observing forecasters’ eye movements during this experiment, video data and retrospective recalls were collected. These qualitative data were used to provide an explanation for differences observed in forecasters’ eye movements. Eye movement differences were analyzed with respect to fixation measures (i.e., count and duration) and scanpath dimensions (i.e., vector, direction, length, position, and duration). These analyses were completed for four stages of the warning decision process: the first 5 min of the case, 2 min prior to warning decisions, the warning issuance process, and warning updates. While radar update speed did not impact forecasters’ fixation measures during these four stages, comparisons of scanpath dimensions revealed differences in their eye movements. Video footage and retrospective recall data illustrated how forecasters’ interactions with the radar display and warning interface, encounters with technological challenges, and varying approaches to similar tasks resulted in statistically significantly (p value < 0.05) lower scanpath similarity scores. The findings of this study support the combined use of eye-tracking and qualitative research methods for detecting and understanding individual differences in forecasters’ eye movements. Future applications of these methods in operational meteorology research have potential to aid usability studies and improve human–computer interactions for forecasters.

Full access
Katie A. Wilson, Pamela L. Heinselman, and Ziho Kang

Abstract

Eye-tracking technology can observe where and how someone’s eye gaze is directed, and therefore provides information about one’s attention and related cognitive processes in real time. The use of eye-tracking methods is evident in a variety of research domains, and has been used on few occasions within the meteorology community. With the goals of Weather Ready Nation in mind, eye-tracking applications in meteorology have so far supported the need to address how people interpret meteorological information through televised forecasts and graphics. However, eye tracking has not yet been applied to learning about forecaster behavior and decision processes. In this article, we consider what current methods are being used to study forecasters and why we believe eye tracking is a method that should be incorporated into our efforts. We share our first data collection of an NWS forecaster’s eye gaze data, and explore the types of information that these data provide about the forecaster’s cognitive processes. We also discuss how eye-tracking methods could be applied to other aspects of operational meteorology research in the future, and provide motivation for further exploration on this topic.

Full access
Katie A. Wilson, Burkely T. Gallo, Patrick Skinner, Adam Clark, Pamela Heinselman, and Jessica J. Choate

Abstract

Convection-allowing model ensemble guidance, such as that provided by the Warn-on-Forecast System (WoFS), is designed to provide predictions of individual thunderstorm hazards within the next 0–6 h. The WoFS web viewer provides a large suite of storm and environmental attribute products, but the applicability of these products to the National Weather Service forecast process has not been objectively documented. Therefore, this study describes an experimental forecasting task designed to investigate what WoFS products forecasters accessed and how they accessed them for a total of 26 cases (comprising 13 weather events, each worked by two forecasters). Analysis of web access log data revealed that, in all 26 cases, product accesses were dominated in the reflectivity, rotation, hail, and surface wind categories. However, the number of different product types viewed and the number of transitions between products varied in each case. Therefore, the Levenshtein (edit distance) method was used to compute similarity scores across all 26 cases, which helped to identify what it meant for relatively similar versus dissimilar navigation of WoFS products. The Spearman’s rank correlation coefficient R results found that forecasters working the same weather event had higher similarity scores for events that produced more tornado reports and for events in which forecasters had higher performance scores. The findings from this study will influence subsequent efforts for further improving WoFS products and developing an efficient and effective user interface for operational applications.

Restricted access
Katie A. Wilson, Pamela L. Heinselman, Patrick S. Skinner, Jessica J. Choate, and Kim E. Klockow-McClain

Abstract

During the 2017 Spring Forecasting Experiment in NOAA’s Hazardous Weather Testbed, 62 meteorologists completed a survey designed to test their understanding of forecast uncertainty. Survey questions were based on probabilistic forecast guidance provided by the NSSL Experimental Warn-on-Forecast System for ensembles (NEWS-e). A mix of 20 multiple-choice and open-ended questions required participants to explain basic probability and percentile concepts, extract information using graphical representations of uncertainty, and determine what type of weather scenario the graphics depicted. Multiple-choice questions were analyzed using frequency counts, and open-ended questions were analyzed using thematic coding methods. Of the 18 questions that could be scored, 60%–96% of the participants’ responses aligned with the researchers’ intended response. Some of the most challenging questions proved to be those requiring qualitative explanations, such as to explain what the 70th-percentile value of accumulated rainfall represents in an ensemble-based probabilistic forecast. Additionally, participants providing answers not aligning with the intended response oftentimes appeared to consider the given information with a deterministic rather than probabilistic mindset. Applications of a deterministic mindset resulted in tendencies to focus on the worst-case scenario and to modify understanding of probabilistic concepts when presented with different variables. The findings from this survey support the need for improved basic and applied training for the development, interpretation, and use of probabilistic ensemble forecast guidance. Future work should collect data for a larger sample size to examine the knowledge gaps across specific user groups and to guide development of probabilistic forecast training tools.

Full access
Nusrat Yussouf, Katie A. Wilson, Steven M. Martinaitis, Humberto Vergara, Pamela L. Heinselman, and Jonathan J. Gourley

Abstract

The goal of the National Oceanic and Atmospheric Administration’s (NOAA) Warn-on-Forecast (WoF) program is to provide frequently updating, probabilistic model guidance that will enable National Weather Service (NWS) forecasters to produce more continuous communication of hazardous weather threats (e.g., heavy rainfall, flash floods, damaging wind, large hail, and tornadoes) between the watch and warning temporal and spatial scales. To evaluate the application of this WoF concept for probabilistic short-term flash flood prediction, the 0–3-h rainfall forecasts from NOAA National Severe Storms Laboratory’s (NSSL) experimental WoF System (WoFS) were integrated as the forcing to the NWS operational hydrologic modeling core within the Flooded Locations and Simulated Hydrographs (FLASH) system. Initial assessment of the potential impacts of probabilistic short-term flash flood forecasts from this coupled atmosphere–hydrology (WoFS-FLASH) modeling system were evaluated in the 2018 Hydrometeorology Testbed Multi-Radar Multi-Sensor Hydrology experiment held in Norman, Oklahoma. During the 3-week experiment period, a total of nine NWS forecasters analyzed three retrospective flash flood events in archive mode. This study will describe specifically what information participants extracted from the WoFS-FLASH products during these three archived events, and how this type of information is expected to impact operational decision-making processes. Overall feedback from the testbed participants’ evaluations show promise for the coupled NSSL WoFS-FLASH system probabilistic flash flood model guidance to enable earlier assessment and detection of flash flood threats and to advance the current warning lead time for these events.

Free access
Katie A. Wilson, Pamela L. Heinselman, Charles M. Kuster, Darrel M. Kingfield, and Ziho Kang

Abstract

Impacts of radar update time on forecasters’ warning decision processes were analyzed in the 2015 Phased Array Radar Innovative Sensing Experiment. Thirty National Weather Service forecasters worked nine archived phased-array radar (PAR) cases in simulated real time. These cases presented nonsevere, severe hail and/or wind, and tornadic events. Forecasters worked each type of event with approximately 5-min (quarter speed), 2-min (half speed), and 1-min (full speed) PAR updates. Warning performance was analyzed with respect to lead time and verification. Combining all cases, forecasters’ median warning lead times when using full-, half-, and quarter-speed PAR updates were 17, 14.5, and 13.6 min, respectively. The use of faster PAR updates also resulted in higher probability of detection and lower false alarm ratio scores. Radar update speed did not impact warning duration or size. Analysis of forecaster performance on a case-by-case basis showed that the impact of PAR update speed varied depending on the situation. This impact was most noticeable during the tornadic cases, where radar update speed positively impacted tornado warning lead time during two supercell events, but not for a short-lived tornado occurring within a bowing line segment. Forecasters’ improved ability to correctly discriminate the severe weather threat during a nontornadic supercell event with faster PAR updates was also demonstrated. Forecasters provided subjective assessments of their cognitive workload in all nine cases. On average, forecasters were not cognitively overloaded, but some participants did experience higher levels of cognitive workload at times. A qualitative explanation of these particular instances is provided.

Full access
Charles M. Kuster, Pamela L. Heinselman, Jeffrey C. Snyder, Katie A. Wilson, Douglas A. Speheger, and James E. Hocker

Abstract

Many public safety officials (e.g., emergency managers and first responders) use weather-radar data to inform many life-saving decisions, such as sounding outdoor warning sirens and directing storm spotters. Therefore, to include this important user group in ongoing radar applications research, a knowledge coproduction framework is used to interact with, learn from, and provide information to public safety officials. From these interactions, it became clear that radar-based products that estimate a tornado’s location, intensity, or both could be valuable to public safety officials. Therefore, a survey was conducted and a focus group formed to 1) collect feedback on several of these products currently under development, 2) identify potential decisions that could be made with these products, and 3) examine the impact of radar update time on product usefulness. An analysis of the survey and focus group responses revealed that public safety officials preferred simple interactive products provided to them using multiple communication methods. Once received, any product that could clearly communicate where a tornado may have occurred would likely help public safety officials focus search and rescue efforts in the immediate aftermath of a tornado. Additionally, public safety officials preferred products created using rapid-update data (1–2-min volumetric updates) over conventional-update data (4–5-min volumetric updates) because it provided them with more complete information.

Full access
Thomas J. Galarneau Jr., Louis J. Wicker, Kent H. Knopfmeier, William J. S. Miller, Patrick S. Skinner, and Katie A. Wilson

Abstract

A multiscale analysis of the significant nocturnal tornado outbreak in Tennessee on 2–3 March 2020 is presented. This outbreak included several significant tornadoes and resulted in the second most fatalities (25) and most injuries (309) of all nocturnal tornado events in Tennessee in 1950–2020. The two deadliest tornadoes struck Nashville (EF3 intensity) and Cookeville (EF4) resulting in 5 and 19 fatalities, respectively. The supercell responsible for the tornado outbreak initiated at 0330 UTC 3 March within a region of warm frontogenesis in western Tennessee. Throughout its life cycle, the supercell was located in a region of convective available potential energy near 1000 J kg−1 and 0–1 km storm-relative helicity over 350 m2s−2. Retrospective 3-h forecasts from the experimental Warn-on-Forecast System (WoFS) convection-allowing ensemble initialized after the parent supercell initiated indicated a high probability, high severity scenario for tornadoes across Tennessee and into Nashville through 0700 UTC. Earlier WoFS forecasts indicated a low probability, high severity scenario owing to uncertainty in the initiation of supercells. The presence of these supercells was sensitive to the upstream thermodynamic conditions and warm frontogenesis regions that were inherited from the lateral boundary conditions. In all, this study highlights the potential of the WoFS ensemble to contribute useful probabilistic severe weather information to the short-term forecast process during a nocturnal significant tornado outbreak.

Restricted access
Burkely T. Gallo, Katie A. Wilson, Jessica Choate, Kent Knopfmeier, Patrick Skinner, Brett Roberts, Pamela Heinselman, Israel Jirak, and Adam J. Clark

Abstract

During the 2019 Spring Forecasting Experiment in NOAA’s Hazardous Weather Testbed, two NWS forecasters issued experimental probabilistic forecasts of hail, tornadoes, and severe convective wind using NSSL’s Warn-on-Forecast System (WoFS). The aim was to explore forecast skill in the time frame between severe convective watches and severe convective warnings during the peak of the spring convective season. Hourly forecasts issued during 2100–0000 UTC, valid from 0100 to 0200 UTC demonstrate how forecasts change with decreasing lead time. Across all 13 cases in this study, the descriptive outlook statistics (e.g., mean outlook area, number of contours) change slightly and the measures of outlook skill (e.g., fractions skill score, reliability) improve incrementally with decreasing lead time. WoFS updraft helicity (UH) probabilities also improve slightly and less consistently with decreasing lead time, though both the WoFS and the forecasters generated skillful forecasts throughout. Larger skill differences with lead time emerge on a case-by-case basis, illustrating cases where forecasters consistently improved upon WoFS guidance, cases where the guidance and the forecasters recognized small-scale features as lead time decreased, and cases where the forecasters issued small areas of high probabilities using guidance and observations. While forecasts generally “honed in” on the reports with slightly smaller contours and higher probabilities, increased confidence could include higher certainty that severe weather would not occur (e.g., lower probabilities). Long-range (1–5 h) WoFS UH probabilities were skillful, and where the guidance erred, forecasters could adjust for those errors and increase their forecasts’ skill as lead time decreased.

Significance Statement

Forecasts are often assumed to improve as an event approaches and uncertainties resolve. This work examines the evolution of experimental forecasts valid over one hour with decreasing lead time issued using the Warn-on-Forecast System (WoFS). Because of its rapidly updating ensemble data assimilation, WoFS can help forecasters understand how thunderstorm hazards may evolve in the next 0–6 h. We found slight improvements in forecast and WoFS performance as a function of lead time over the full experiment; the first forecasts issued and the initial WoFS guidance performed well at long lead times, and good performance continued as the event approached. However, individual cases varied and forecasters frequently combined raw model output with observed mesoscale features to provide skillful small-scale forecasts.

Restricted access