As numerical modeling methods and forecasting technologies continue to improve, people may start to see more specific severe weather timing and location information hours before the event occurs. While studies have investigated response actions on the warning time scales, little work has been done to understand what types of actions residents will take given 4–8 h of advance notice for a possible tornado. This study uses data from the 2018 Severe Weather and Society Survey, an annual survey of U.S. adults, to begin analyzing response actions and how those responses differ with either 4 or 8 h of advance notice. Results show that response actions are largely the same between the two time periods. The small differences that do exist show that sheltering behaviors are more common with 4 h of advance notice whereas monitoring behaviors are more common with 8 h of notice. In addition, respondents claimed they would “wait and see” more often in the 8-h category, indicating they would seek additional information before deciding how to respond. Perhaps more important than the types of actions that respondents identify is the increase in those who are unsure of how to react or would choose to do nothing when given 8 h of notice. Respondents may be anchored to the current system and may not have considered all of the possible actions they can take given more time. Therefore, we emphasize the need for education campaigns as technology, forecasts, and desired responses continue to evolve.
1. Introduction and background
As forecast technology continues to improve, we may start to see more specific forecast information (including timing information) earlier in the event timeline. This may mean that residents could know their specific threat time frame 4–8 h ahead of the actual event. A change like this would open up a new realm of potential response actions, many of which have not been studied. This work begins the process of understanding what types of response actions individuals may take given hours of notice to tornadic events, and how those actions differ from those currently taken given minutes of lead time for tornado warnings.
The National Weather Service (NWS) is the government entity tasked with issuing hazardous weather forecasts for the United States for the protection of life and property and enhancement of the national economy (NWS 2019). Their suite includes products covering all hazard types, from air quality alerts to winter weather products. Specific to severe weather events, there are generally three different levels of products that compose the public communication process. The first level includes convective outlooks, which are forecast by the Storm Prediction Center (SPC) up to 8 days in advance and are generally on a regional or multistate scale. The second level includes mesoscale discussions and severe thunderstorm/tornado watches, which are also issued by the SPC. These products are issued on the day of the event, generally 1–3 h before the event begins, and are usually on a multicounty or statewide scale. The third level includes warnings, which are issued by the local NWS office. They are generally valid from just prior to the event occurring and are usually the size of a one or two counties.
While the current system includes three distinct levels of products, a proposed system from the Forecasting a Continuum of Environmental Threats (FACETs) project aims to provide a continuous flow of hazardous weather information (Rothfusz et al. 2018). In theory, this system would provide each individual user with information specific to their situation and threat tolerance. For example, this future system may supplement current products with a continuous stream of probabilistic hazard information (see Ling et al. 2015) that users can view at any point in time or space based on their chosen alert-level settings. While potentially beneficial, some key partners (like emergency managers) often rely on specific products to make decisions or activate procedures (Cross et al. 2019). Likewise, tornado watches seem to improve the tornado warning process in local forecast offices (Hales 1990). From a public perspective, watches serve as the first line of defense to initiate protective action. Generally, the more severe the watch type, the more likely people are to stop their activities and start monitoring the situation (Gutter et al. 2018). These findings raise an important question: should certain products (or product levels) be maintained in the proposed FACETs system? As forecast technology continues to improve, the current products may start to evolve and serve a different purpose, but their existence may still be important to core partners and the public.
If some or all of the current product structure remains in place, the challenge for the FACETs paradigm then becomes developing a continuous flow of information while maintaining the current product structure of discrete forecasts. Currently, a multiple-hour information gap may exist between the convective outlook and the first (if any) mesoscale discussion or watch, depending on the day. Although many NWS forecast offices provide information between these two products (often in the form of online or phone briefings and social media posts), there is currently no formalized product information available between the convective outlook and a mesoscale discussion or watch. One of the proposed solutions to help remedy this information gap is to include the time frame of severe weather along with the probabilistic and categorical risk levels in the convective outlook. An analysis of historical severe weather reports shows that a majority (greater than 95%) of daily reports occurring at a single location will occur within a 4-h period (Krocak and Brooks 2017). Currently, the SPC forecasts the probability of severe reports occurring within 25 mi (~40 km) of a location over a 24-h period. Since the analysis above shows that a majority of reports within 25 mi of a location will occur in a smaller timeframe (4 h), the SPC could, in theory, provide information about that smaller timeframe (such as when it will occur) without changing the definition of their probability forecasts. For example, the SPC could provide the categorical risk (marginal, slight, enhanced, moderate, or high risk) and a 4-h time frame (1300–1700 LT, 1600–2000 LT, etc.) for each location.
While there has been little work conducted to understand response actions to hours of advance notice before an event, there has been some work related to warning-scale response actions. Most studies that ask participants about response actions consider immediate sheltering to be the most correct response (e.g., Jauernic and Van Den Broeke 2016). When evaluating the factors that change response actions, studies find that demographic characteristics such as education, age, and gender can impact sheltering behaviors. Responsiveness increases until about 65 years of age and then decreases with increasing age (Chaney et al. 2013), it increases with education (Balluz et al. 2000), and women generally seek shelter more often than men (Ripberger et al. 2015).
Other factors that impact response behaviors include the wording of the actual product. Impact-based warnings include language that discusses the potential consequences of the event, including damages and loss of life. Studies find that this type of language increases response actions, including plans to shelter (Casteel 2016, 2018; Ripberger et al. 2015). However, studies also find that even when residents plan to shelter, that often is not the first action they take because they will often confirm warning information from multiple sources (Jauernic and Van Den Broeke 2016).
Last, many studies attempt to measure what fraction of participants respond to tornado warnings. Some include interviews after actual tornado events and find that anywhere from 43% to 79% of residents take action, depending on the region where the event occurred (Balluz et al. 2000; Miran et al. 2018; Chaney et al. 2013). Studies that measure hypothetical situations find higher response rates, with anywhere from 75% to 90% of respondents claiming they will take action during a future event (Schultz et al. 2010; Lindell et al. 2016; Ripberger et al. 2015).
There is a fundamental difference between response actions for minutes of advance notice and those for hours of advance notice. Some research suggests there is a threshold of “too much” lead time on warning scales, or a point at which the threat no longer seems imminent and residents do not immediately head to shelter (e.g., Hoekstra et al. 2011; Doswell 1999; Ewald and Guyer 2002). In fact, one particular study finds that lead times of over 15 min increase the number of fatalities when compared with an unwarned tornado (Simmons and Sutter 2011). However, while sheltering may be one of the only reasonable actions given 15–30 min of notice, there is a myriad of other actions that become more reasonable given hours of notice (i.e., leaving the area, monitoring the situation, or preparing their home and family) that would in theory set other protective actions in motion (such as preparing the shelter or important documents). This study aims to identify the actions individuals believe they will take given hours of advance notice for a tornadic event, and if (how) those actions change given either 4 or 8 hours of notice.
2. Data and methods
a. Survey data
The University of Oklahoma Center for Risk and Crisis Management fields a national survey to analyze public reception, comprehension, and response to severe weather forecast products (Silva et al. 2017, 2018). This survey, called the Severe Weather and Society Survey, has been fielded in 2017 and 2018, with plans to field it again in 2019. It utilizes an online format with a sample of U.S. adults (age 18+) provided by Qualtrics, LLC, which maintains a large pool of participants that agree to take Internet surveys. There were 2000 respondents in 2017 and 3000 in 2018 (the survey used in this study). Respondents generally took around 25 min to complete the survey and were compensated for their time. Dynamic sampling was employed, meaning participants were asked demographic questions before taking the survey and were not asked to complete the survey if their demographic profile was already well represented by the current pool of respondents. This process was used to ensure that the sample population was as representative of the U.S. population as it could be. After the survey was fielded, responses were also weighted according to U.S. Census estimates, further ensuring the results were demographically representative of the population.
One of the many unique aspects of the Severe Weather and Society Survey (hereinafter WX17 and WX18 for the 2017 and 2018 rounds, respectively) is that there are two different types of questions employed. Some questions are recurring, in which researchers attempt to establish a baseline of severe weather knowledge and response actions. Other questions rotate in and out, depending on what experiments researchers are interested in conducting each year. Although there were nearly 100 questions total on WX18, this study uses data from just a few different rotating questions to establish how more advance notice for the event (on the order of hours and not minutes) impacts tornado preparation and response actions.
The specific questions used in this study were open ended, meaning respondents could enter whatever information they liked, and the responsibility to interpret their responses was placed on the researchers. Respondents were asked to describe what they would do given the knowledge that a large and dangerous tornado would impact their location in either 4 or 8 h. The amount of advance notice was assigned randomly to each participant, resulting in 1500 responses to 4 h of advance notice and 1500 responses to 8 h of advance notice. Time of day was held constant at 0900 LT to ensure that all respondents were anchoring to the same time of day and the activities that correspond to that time of day. After removing unusable responses (e.g., blank responses or random letters/characters), we were left with 1392 responses in the 4-h category and 1404 responses in the 8-h category for analysis. Differences in responses were compared to identify how the shift from 4 h of notice to 8 h of notice would impact response actions. The exact wording of the survey question is “Imagine that it is 9:00 a.m. tomorrow morning and you are somewhat confident that a large tornado will hit your location in the next [RANDOMIZE: 4|8 h]. What would you do? Please be as specific as possible.”
Consistent with previous studies (e.g., Schultz et al. 2010; Ripberger et al. 2015), we use intended response actions as a proxy for actual response actions in this analysis. While there is little work that analyzes the relationship between intended and actual response actions to tornado warnings, there has been extensive work relating intended and actual behavior in other fields (e.g., Armitage and Conner 2001). This work shows that there is a significant link between intended and actual behavior, even when an individual is in a high stress situation (Kang et al. 2007). While it may not be a perfect proxy, we believe that our results provide some insight into what response actions might be given hours of advance notice for a possible tornado.
b. Response treatment
After fielding the survey, responses were divided into the two time categories (4 and 8 h) for further analysis. We begin by comparing key word usage across the time categories. We do this by identifying the most common words that participants used and then compare the percentage of respondents that used each word across the time categories. For example, the percentage of responses that contain the word “shelter” in the 4-h category is calculated as
where nshelter,4hours is the number of responses in the 4-h category that contain the word shelter and n4hours is the total number of respondents who were given 4 hours of notice in their response prompt. After those percentages are calculated, the difference in word use between 4 and 8 h is calculated by subtracting the percentage of word use in the 4-h category from the percentage of word use in the 8-h category:
While the analysis of single words is a good starting place to understand basic response characteristics, the context of those words also plays an important role in understanding the actions people will take. To further investigate these response actions, all usable survey text responses are categorized into one or more categories. These six categories (shown in Table 1) were chosen after reading the responses to ensure that they encompass nearly all of the described actions. The categories are not mutually exclusive; in fact, many of the responses fit into multiple categories. Once categorized, the percentages for each category are calculated for both 4 and 8 h of advance notice. Similar to the word analysis, the difference in these percentages is calculated to understand how changing from 4 to 8 h would change respondents’ actions. A two-tailed difference in proportions z test is performed for all of the difference calculations to identify which, if any, of the word or category differences are statistically significant.
3. Results and discussion
a. Word analysis
The top words used in text responses are very similar for 4 and 8 h of advance notice for a possible tornado (Fig. 1). In fact, 22 of the 25 top words used are found in both the 4- and 8-h categories. The only difference is in the order of the most used words by number of times used. The three most popular words used with 4 h of advance notice are “shelter,” “go,” and “monitor,” in that order. These words are found in 20.0%, 17.8%, and 16.8% of responses, respectively. Those same three words are also the most used with 8 h of advance notice, except the order is “monitor,” “shelter,” and then “go.” This time, they are found in 17.5%, 17.3%, and 15.9% of responses.
The differences between the most commonly used words suggest a pattern of more sheltering preferences with 4 h of notice and more monitoring preferences with 8 h of notice (Fig. 2). Words that relate to sheltering behaviors (such as “shelter,” “go,” “take,” “get,” “safe,” “basement,” and “find”) are more prevalent in the 4-h category; words that relate to monitoring the situation or information gathering (such as “monitor,” “stay,” “weather,” “prepare,” and “keep”) are more popular in the 8-h category. Because of the relatively low number of responses that contain the most common words (shelter appears in only 279 responses in the 4-h category and monitor appears in only 246 responses in the 8-h category), many of the differences are not statistically significant. However, “take,” “get,” “shelter,” “find,” “drive,” “away,” and “house” are all more prevalent in 4-h responses and are statistically different from the 8-h responses at the p < 0.1 level. This result may suggest that there is a slight preference for sheltering behaviors within the 4-h category. On the other hand, “keep,” “family,” and “weather” are more prevalent in the 8-h responses and are statistically different from the 4-h responses at the p < 0.1 level, which may indicate a slight preference for monitoring and preparatory behaviors in the 8-h category.
Another common theme in both categories is respondents using the current system as an anchor for what they would do in a situation in which they have not been before. Many responses displayed confusion or disbelief that there could even be 8 h of advance notice for a tornado. Some responses reflect this disbelief and then proceed to talk about what they would do with less time. This may highlight the need for education if a new system were to be put in place. If people are given some idea of what could be done with many hours of advance notice, they may be more likely to take precautionary actions.
Although the percentages of individual words used are interesting and a good starting point, those words exist within the context of the individual respondents’ situations. That context is often important when understanding what specific actions they plan to take. For example, the word shelter may be used more often in the 4-h category, but it is important to understand how it is being used in both categories. With regard to 4 h of advance notice, “shelter” is used mostly in a traditional sense, in statements like “I would go to shelter.” In the 8-h category, it is often used to say that they would not head to shelter until necessary, in statements like “I’d monitor the weather and head to shelter when necessary.”
b. Categorical analysis
Given that the context of the response (and not just the most commonly used words) is important for understanding common behaviors, we place each response into categories based on the most common response actions. Some of the same themes seen in the most commonly used words are also represented in the categorical analysis, but the categories also reveal actions that single words cannot (Fig. 3). As an example, prepare and monitor are the two most common categories in the 4-h group (32% and 29% of responses, respectively), followed by sheltering (26% of responses). This is likely seen because the descriptions of preparing and monitoring do not necessarily need to include the words “prepare” or “monitor.” For example, many respondents said that they would gather important items (prepare), head to a safe place (shelter), and then watch for updates on TV or their phone (monitor). None of those actions would have been captured in a word analysis, but they become evident when comparing categories.
Respondents in the 4-h group often mention gathering the most important documents/items and then sheltering with family. It is somewhat unexpected to see so many responses indicating they would immediately go to shelter when they would likely be in shelter for hours before the event occurred. In theory, they could get other tasks done or even leave the area before heading to shelter, but many responded as if they had mere minutes instead of hours.
The 8-h group has similar response category percentages to the 4-h group with just a few adjustments. The monitor and prepare categories switched places, with monitor being the highest category in the 8-h group (32% and 31% of responses, respectively; Fig. 3). Many responses indicate that they would look for more information and act when the event was closer to occurring. The nothing category also increased to nearly 8% of responses, which may indicate that 8 h was too much advance notice to begin taking precautions (Fig. 3).
The difference in percentage of responses in each category reflects a shift from action to monitoring when shifting from 4 to 8 h of advance notice (Fig. 4). The monitor, nothing, and unsure categories were more prevalent in the 8-h group, although only the difference in the nothing category was statistically significant. Still, these changes may indicate that either 8 h is too much advance notice or that people are unaware of what actions they can or should be doing with an entire workday of time. The 4-h group is more focused on sheltering and preparing (with differences of 1.3% and 3.8%, respectively), although the prepare category is well represented in both groups. The differences in the percentage of responses within the shelter category and the do-nothing category were statistically significant, which may further indicate that respondents within the 4-h category are more focused on sheltering than are those in the 8-h category.
4. Summary and conclusions
Recent National Oceanic and Atmospheric Administration (NOAA) initiatives like the Warn-On-Forecast and the FACETs projects have begun to usher in a glimpse of what forecast information could look like in the future (Rothfusz et al. 2018). Given that most severe weather reports at any location are confined to subdaily time periods (Krocak and Brooks 2017), it is within the realm of possibility that forecasters may soon be able to give hours of notice for severe weather events. While some work has been done to begin understanding how the public will react to increased specificity in products with warning-scale lead times, little work has been done to show how hours of advance notice for these events will impact response actions. This is vitally important as any actions taken a few minutes before the event are dependent on the actions taken previously.
After fielding a national survey of 3000 U.S. adults, we analyze and categorize text responses based on their content. First and foremost, we find that response actions are largely the same, regardless of how much time people are given. Analysis of single words show that sheltering behaviors may be slightly more common with 4 h of advance notice and monitoring behaviors may be slightly more common with 8 h of advance notice. However, many nuances are lost when we just look at single words. Categorical analysis of the responses show preparation and monitoring were the most common behaviors, regardless of how much time respondents were given. Although small, the differences we do find focus on preparing the most valuable items and sheltering when given 4 h of notice, and on monitoring the weather and confirming information as well as preparing home items, pets, and family members when given 8 h of notice.
Perhaps more important is that we find more uncertainty about what to do with 8 h of advance notice than with 4 h, which may indicate that either 8 h is too much time before the event occurs or that many respondents do not have a well conceptualized list of the kinds of actions they could take to prepare for severe weather with more time. It is important to recognize that respondents in our survey were likely working with knowledge of the current system to help them visualize what they would do in a completely different system. While some people may know their routine when given 15 min of lead time, they may never have thought about all of the additional actions they may want to take given hours of advance notice. When the respondents are stratified by region, we do see a slightly higher proportion of those in less tornado-prone areas (the eastern and western NWS regions of the United States) stating that they were unsure of what they should do in both time categories. Within the 4-h category, the eastern and western regions show 2.4% of responses in the unsure category while the central and southern NWS regions show 1.6% in the same category (the same percentages in the 8-h category are 2.8% for the eastern and western regions vs 1.9% for the central and southern regions). We find a similar result when the data are stratified by education level. Those with less education (i.e., a high school degree or lower) said they were unsure of what to do more often than those with more education (3.2% vs 1.4% in the 4-h category and 3.0% vs 2.0% in the 8-h category, respectively). Given that education and prior experience may help residents to understand what actions to take to prepare for these events, we follow recent reports from NOAA and the National Academy of Sciences (NAS) in emphasizing the need for collaborative work between the physical and social sciences in the weather enterprise (NOAA 2015; NAS 2018). We believe that implementing changes in product structure must coincide with an education or information campaign that explains the nature of the change and how residents can utilize that change to enhance their safety and resilience. As related to this work, we believe an education campaign should include information on some of the kinds of actions that people can and should take multiple hours before a tornado occurs to make sure that they are safe if (when) the storm hits.
We also recognize the limitations of this work, which leaves room for future projects and research paths. First and foremost, we focus on anticipated actions to a hypothetical event, which may differ from actual responses to a real event. Studies of actual behavior after tornadoes are needed to understand if and how intended actions differ from actual responses. Second, we study intended responses to a single hazard (tornadoes). While there is likely some overlap in preparatory actions, many of the relevant response actions for other weather hazards would likely be different, meaning the results of this study are not likely to be generalizable to other categories of weather hazards. In addition, our survey data are collected using an online platform, meaning vulnerable populations (like the elderly or those living in poverty) are likely to be underrepresented. We therefore see a need to employ multiple collection methods, including interviews and focus groups that target these populations to ensure results are generalizable. Last, we again emphasize the need for accompanying education campaigns, which suggests a close relationship among researchers, forecasters, emergency responders, and communities will be needed if a new system is to be implemented. We hope that this work begins the process of understanding whether and how response actions may change with more notice for hazardous weather events and where we should be investing time and money in education campaigns as the forecast system continues to evolve.
The data for this project were collected with support from the University of Oklahoma. The analysis was conducted with support from NOAA’s Office of Weather and Air Quality through the U.S. Weather Research Program.