Search Results

You are looking at 1 - 10 of 14 items for

  • Author or Editor: Makenzie J. Krocak x
  • Refine by Access: All Content x
Clear All Modify Search
Makenzie J. Krocak and Harold E. Brooks

Abstract

One of the challenges of providing probabilistic information on a multitude of spatiotemporal scales is ensuring that information is both accurate and useful to decision-makers. Focusing on larger spatiotemporal scales (i.e., from convective outlook to weather watch scales), historical severe weather reports are analyzed to begin to understand the spatiotemporal scales that hazardous weather events are contained within. Reports from the Storm Prediction Center’s report archive are placed onto grids of differing spatial scales and then split into 24-h convective outlook days (1200–1200 UTC). These grids are then analyzed temporally to assess over what fraction of the day a single location would generally experience severe weather events. Different combinations of temporal and spatial scales are tested to determine how the reference class (or the choice of what scales to use) alters the probabilities of severe weather events. Results indicate that at any given point in the United States on any given day, more than 95% of the daily reports within 40 km of the point occur in a 4-h period. Therefore, the SPC 24-h convective outlook probabilities can be interpreted as 4-h convective outlook probabilities without a significant change in meaning. Additionally, probabilities and threat periods are analyzed at each location and different times of year. These results indicate little variability in the duration of severe weather events, which allows for a consistent definition of an “event” for all locations in the continental United States.

Full access
Makenzie J. Krocak and Harold E. Brooks

Abstract

While there has been an abundance of research dedicated to the seasonal climatology of severe weather, very little has been done to study hazardous weather probabilities on smaller scales. To this end, local hourly climatological estimates of tornadic event probabilities were developed using storm reports from NOAA’s Storm Prediction Center. These estimates begin the process of analyzing tornado frequencies on a subdaily scale.

Characteristics of the local tornado climatology are investigated, including how the diurnal cycle varies in space and time. Hourly tornado probabilities are peaked for both the annual and diurnal cycles in the plains, whereas the southeast United States has a more variable pattern. Areas that have similar total tornado threats but differ in the distribution of that threat are highlighted. Additionally, areas that have most of the tornado threat concentrated in small time frames both annually and diurnally are compared to areas that have a low-level threat at all times. These differences create challenges related to staffing requirements and background understanding of the tornado threat unique to each region.

This work is part of a larger effort to provide background information for probabilistic forecasts of hazardous weather that are meaningful over broad time and space scales, with a focus on scales broader than the typical time and space scales of the events of interest (including current products on the “watch” scale). A large challenge remains to continue describing probabilities as the time and space scales of the forecast become comparable to the scale of the event.

Full access
Makenzie J. Krocak and Harold E. Brooks

Abstract

While many studies have looked at the quality of forecast products, few have attempted to understand the relationship between them. We begin to consider whether or not such an influence exists by analyzing storm-based tornado warning product metrics with respect to whether they occurred within a severe weather watch and, if so, what type of watch they occurred within. The probability of detection, false alarm ratio, and lead time all show a general improvement with increasing watch severity. In fact, the probability of detection increased more as a function of watch-type severity than the change in probability of detection during the time period of analysis. False alarm ratio decreased as watch type increased in severity, but with a much smaller magnitude than the difference in probability of detection. Lead time also improved with an increase in watch-type severity. Warnings outside of any watch had a mean lead time of 5.5 min, while those inside of a particularly dangerous situation tornado watch had a mean lead time of 15.1 min. These results indicate that the existence and type of severe weather watch may have an influence on the quality of tornado warnings. However, it is impossible to separate the influence of weather watches from possible differences in warning strategy or differences in environmental characteristics that make it more or less challenging to warn for tornadoes. Future studies should attempt to disentangle these numerous influences to assess how much influence intermediate products have on downstream products.

Restricted access
Makenzie J. Krocak and Harold E. Brooks

Abstract

One of the challenges of providing probabilistic information on a multitude of spatiotemporal scales is ensuring that information is both accurate and useful to decision-makers. Focusing on larger spatiotemporal scales (i.e., from convective outlook to weather watch scales), historical severe weather reports are analyzed to begin to understand the spatiotemporal scales that hazardous weather events are contained within. Reports from the Storm Prediction Center’s report archive are placed onto grids of differing spatial scales and then split into 24-h convective outlook days (1200–1200 UTC). These grids are then analyzed temporally to assess over what fraction of the day a single location would generally experience severe weather events. Different combinations of temporal and spatial scales are tested to determine how the reference class (or the choice of what scales to use) alters the probabilities of severe weather events. Results indicate that at any given point in the United States on any given day, more than 95% of the daily reports within 40 km of the point occur in a 4-h period. Therefore, the SPC 24-h convective outlook probabilities can be interpreted as 4-h convective outlook probabilities without a significant change in meaning. Additionally, probabilities and threat periods are analyzed at each location and different times of year. These results indicate little variability in the duration of severe weather events, which allows for a consistent definition of an “event” for all locations in the continental United States.

Free access
Makenzie J. Krocak, Matthew D. Flournoy, and Harold E. Brooks

Abstract

Increasing tornado warning skill in terms of the probability of detection and false-alarm ratio remains an important operational goal. Although many studies have examined tornado warning performance in a broad sense, less focus has been placed on warning performance within subdaily convective events. In this study, we use the NWS tornado verification database to examine tornado warning performance by order-of-tornado within each convective day. We combine this database with tornado reports to relate warning performance to environmental characteristics. On convective days with multiple tornadoes, the first tornado is warned significantly less often than the middle and last tornadoes. More favorable kinematic environmental characteristics, like increasing 0–1-km shear and storm-relative helicity, are associated with better warning performance related to the first tornado of the convective day. Thermodynamic and composite parameters are less correlated with warning performance. During tornadic events, over one-half of false alarms occur after the last tornado of the day decays, and false alarms are 2 times as likely to be issued during this time as before the first tornado forms. These results indicate that forecasters may be better “primed” (or more prepared) to issue warnings on middle and last tornadoes of the day and must overcome a higher threshold to warn on the first tornado of the day. To overcome this challenge, using kinematic environmental characteristics and intermediate products on the watch-to-warning scale may help.

Restricted access
Makenzie J. Krocak, Joseph T. Ripberger, Sean Ernst, Carol L. Silva, and Hank C. Jenkins-Smith

Abstract

While previous work has shown that the Storm Prediction Center (SPC) convective outlooks accurately capture meteorological outcomes, evidence suggests stakeholders and the public may misinterpret the categorical words currently used in the product. This work attempts to address this problem by investigating public reactions to alternative information formats that include the following numeric information: 1) numeric risk levels (i.e., “Level 2 of 5”) and 2) numeric probabilities (i.e., “a 5% chance”). In addition, it explores how different combinations of the categorical labels with numeric information may impact public reactions to the product. Survey data comes from the 2020 Severe Weather and Society Survey, a nationally representative survey of U.S. adults. Participants were shown varying combinations of the information formats of interest, and then rated their concern about the weather and the likelihood of changing plans in response to the given information. Results indicate that providing numeric information (in the form of levels or probabilities) increases the likelihood of participants correctly interpreting the convective outlook information relative to categorical labels alone. Including the categorical labels increases misinterpretation, regardless of whether numeric information was included alongside the labels. Finally, findings indicate participants’ numeracy (or their ability to understand and work with numbers) had an impact on correct interpretation of the order of the outlook labels. Although there are many challenges to correctly interpreting the SPC convective outlook, using only numeric labels instead of the current categorical labels may be a relatively straightforward change that could improve public interpretation of the product.

Significance Statement

The SPC convective outlook contains vital information that can help people prepare for a severe weather event. The categorical labels in this product are often ordered incorrectly by members of the public. This work shows using numeric levels or probabilities reduces the tendency for people to order the levels incorrectly.

Restricted access
Makenzie J. Krocak, Joseph T. Ripberger, Hank Jenkins-Smith, and Carol Silva

Abstract

As numerical modeling methods and forecasting technologies continue to improve, people may start to see more specific severe weather timing and location information hours before the event occurs. While studies have investigated response actions on the warning time scales, little work has been done to understand what types of actions residents will take given 4–8 h of advance notice for a possible tornado. This study uses data from the 2018 Severe Weather and Society Survey, an annual survey of U.S. adults, to begin analyzing response actions and how those responses differ with either 4 or 8 h of advance notice. Results show that response actions are largely the same between the two time periods. The small differences that do exist show that sheltering behaviors are more common with 4 h of advance notice whereas monitoring behaviors are more common with 8 h of notice. In addition, respondents claimed they would “wait and see” more often in the 8-h category, indicating they would seek additional information before deciding how to respond. Perhaps more important than the types of actions that respondents identify is the increase in those who are unsure of how to react or would choose to do nothing when given 8 h of notice. Respondents may be anchored to the current system and may not have considered all of the possible actions they can take given more time. Therefore, we emphasize the need for education campaigns as technology, forecasts, and desired responses continue to evolve.

Full access
Sean Ernst, Joe Ripberger, Makenzie J. Krocak, Hank Jenkins-Smith, and Carol Silva

Abstract

Although severe weather forecast products, such as the Storm Prediction Center (SPC) convective outlook, are much more accurate than climatology at day-to-week time scales, tornadoes and severe thunderstorms claim dozens of lives and cause billions of dollars in damage every year. While the accuracy of this outlook has been well documented, less work has been done to explore the comprehension of the product by nonexpert users like the general public. This study seeks to fill this key knowledge gap by collecting data from a representative survey of U.S. adults in the lower 48 states about their use and interpretation of the SPC convective outlook. Participants in this study were asked to rank the words and colors used in the outlook from least to greatest risk, and their answers were compared through visualizations and statistical tests across multiple demographics. Results show that the U.S. public ranks the outlook colors similarly to their ordering in the outlook but switches the positions of several of the outlook words as compared to the operational product. Logistic regression models also reveal that more numerate individuals more correctly rank the SPC outlook words and colors. These findings suggest that the words used in the convective outlook may confuse nonexpert users, and that future work should continue to use input from public surveys to test potential improvements in the choice of outlook words. Using more easily understood words may help to increase the outlook’s decision support value and potentially reduce the harm caused by severe weather events.

Restricted access
Makenzie J. Krocak, Jinan N. Allan, Joseph T. Ripberger, Carol L. Silva, and Hank C. Jenkins-Smith

Abstract

Nocturnal tornadoes are challenging to forecast and even more challenging to communicate. Numerous studies have evaluated the forecasting challenges, but fewer have investigated when and where these events pose the greatest communication challenges. This study seeks to evaluate variation in confidence among U.S. residents in receiving and responding to tornado warnings by hour of day. Survey experiment data come from the Severe Weather and Society Survey, an annual survey of U.S. adults. Results indicate that respondents are less confident about receiving warnings overnight, specifically in the early morning hours [from 12:00 AM to 4:00 AM local time (0000–0400 LT)]. We then use the survey results to inform an analysis of hourly tornado climatology data. We evaluate where nocturnal tornadoes are most likely to occur during the time frame when residents are least confident in their ability to receive tornado warnings. Results show that the Southeast experiences the highest number of nocturnal tornadoes during the time period of lowest confidence, as well as the largest proportion of tornadoes in that time frame. Finally, we estimate and assess two multiple linear regression models to identify individual characteristics that may influence a respondent’s confidence in receiving a tornado between 12:00 AM and 4:00 AM. These results indicate that age, race, weather awareness, weather sources, and the proportion of nocturnal tornadoes in the local area relate to warning reception confidence. The results of this study should help inform policymakers and practitioners about the populations at greatest risk for challenges associated with nocturnal tornadoes. Discussion focuses on developing more effective communication strategies, particularly for diverse and vulnerable populations.

Restricted access
Joseph T. Ripberger, Makenzie J. Krocak, Wesley W. Wehde, Jinan N. Allan, Carol Silva, and Hank Jenkins-Smith

Abstract

Social criteria are important to achieving the mission of the National Weather Service. Accordingly, researchers and administrators at the NWS increasingly recognize a need to supplement verification statistics with complementary data about society in performance management and evaluation. This will require significant development of new capacities to both conceptualize relevant criteria and measure them using consistent, transparent, replicable, and reliable measures that permit generalizable inference to populations of interest. In this study, we contribute to this development by suggesting three criteria that require measurement (forecast and warning reception, comprehension, and response) and demonstrating a methodology that allows us to measure these concepts in a single information domain—tornado warnings. The methodology we employ improves upon previous research in multiple ways. It provides a more generalizable approach to measurement using a temporally consistent set of survey questions that are applicable across the United States; it relies on a more robust set of psychometric tests to analytically demonstrate the reliability of the measures; and it is more transparent and replicable than previous research because the data and methods (source code) are publicly available. In addition to describing and assessing the reliability of the measures, we explore the sensitivity of the measures to geographic and demographic variation to identify significant differences that require attention in measurement. We close by discussing the implications of this study and the next steps toward development and use of social criteria in performance management and evaluation.

Full access