1. Introduction
The Saffir–Simpson hurricane scale (SSHS) was institutionalized in 1973 in the aftermath of Hurricane Camille as a way of communicating storm intensity prior to landfall. Therefore, it is technically a watch scale, and despite its shortcomings (Kantha 2006; Senkbeil and Sheridan 2006), it has been retained in operational practice, with a modification of dropping storm surge in 2009 after anomalous values in several storms, notably, Hurricane Ike. Highly destructive events cause us to reconsider the efficacy of risk communication. With numerous fatalities resulting from violent tornadoes in the last 4 years, is it time to evaluate the possibility of a preevent scale for tornado outbreak potential? No current scale is widely advertised for this explicit purpose for tornadoes, although the Storm Prediction Center (SPC) has been communicating tornado risk categorically in the form of slight, moderate, and high risk days and also in probabilistic format. SPC products are explained in more detail in the following section. The objective of this research is to incorporate the basic principles of risk levels for tornado outbreaks in a format similar to the SSHS and to create a platform by which risk is more effectively communicated to the public.
Based on a system of rating tornado shelter/refuge adequacy (Mason and Senkbeil 2014), the focus of this manuscript is on transforming a tornado refuge rubric (TRR) into a Tornado Watch Scale (TWS). The crux of the TWS is a hierarchical classification system to communicate expected tornado intensity to the public. Based on the commendable performance of the SPC in tornado forecasts, this manuscript will not attempt to develop an additional or alternative method of forecasting tornado potential. Instead, the objective of this research is to communicate the risk associated with tornado watches in an entirely new format. An explanation of the TWS is presented along with an analysis of its safety value and favorability among participants in interviews.
There is a dearth of published literature on tornado watch risk perception; however, literature on tornado warning behavior and risk perception is abundant. Warning communication (Balluz et al. 2000; Sorensen 2000; Hammer and Schmidlin 2002; Paul et al. 2003; Farley 2007; Brotzge and Erickson 2009; Coleman et al. 2011; League et al. 2010; Schumacher et al. 2010; Sherman-Morris 2010; Hoekstra et al. 2011), false alarm rates (Barnes et al. 2007a,b; Simmons and Sutter 2009; Brotzge et al. 2011), probability of detection (Simmons and Sutter 2011; Brotzge et al. 2013), warning lead time, and possible improvements of the current watch system (Erickson and Brooks 2006; Simmons and Sutter 2008; Hoekstra et al. 2011) have all been discussed in recent literature. The goal of the TWS is to provide residents with a simple forecast of expected severity and corresponding precautionary actions in ample time for those who do not have a storm shelter or safe room readily available to travel to substantial shelter based on the forecast recommendations. Two brief sections explaining comparable and existing products and development of the TWS follow the introduction before proceeding to methods, results and discussion, and conclusions.
2. Comparable and existing products
The forecasting of tornado likelihood is currently addressed by multiple products, one example being the convective outlooks issued by the Storm Prediction Center. Risk levels are outlined up to 8 days in advance of an event and can contain the following levels: general or nonsevere thunderstorms (TSTM), which defines an area with a probability of thunderstorms of 10% or greater; marginal risk (MRGL), which is used to show “an area of severe storms of either limited organization and longevity, or very low coverage and marginal intensity”; slight risk (SLGT), showing a more organized area of severe storms which is low in coverage and varying in intensity; enhanced risk (ENH), implying an area with greater severe storm coverage than a slight risk, but still with varying levels of intensity; moderate risk (MDT), which implies a greater concentration and/or magnitude than an enhanced risk, including “days with several supercells producing intense tornadoes and/or very large hail, or an intense squall line widespread damaging winds” (National Weather Service 2015). The highest risk level shown in convective outlooks is HIGH, implying a high degree of certainty in a major severe weather outbreak, “from either numerous intense and long-tracked tornadoes or a long-lived derecho-producing thunderstorm complex that produces hurricane-force wind gusts and widespread damage” (National Weather Service 2015).
There are several other SPC products with similar objectives. In addition to the categorical convective outlooks discussed above, the SPC also issues probabilistic convectional outlooks up to 3 days in advance of an event. These are more descriptive of individual severe weather hazards (hail, winds, and tornadoes) and are used to address expectations for each hazard while including the forecaster’s confidence in the event (www.spc.noaa.gov/products/outlook/probinfo.html). Also in use by the SPC are products known as public severe weather outlooks (PWOs). While the previously mentioned outlooks provide more detailed information up to several days in advance of an event, a PWO is typically issued 12–24 h in advance of an event. These outlooks are used when a high risk is issued for tornadoes or widespread significant wind damage, or when areas within a moderate risk category contain at least a 15% probability of tornadoes or 45% probability of damaging winds. In contrast to the convective outlooks and probabilistic convective outlooks, the PWOs incorporate a plain-language forecast for use by the general public (www.spc.noaa.gov/misc/about.html#PWO). Yet another source for information before an event is generated by the SPC when a tornado watch is issued and can be found on the tornado watch web page in a section addressing the likelihood of certain hazards (e.g., http://www.spc.noaa.gov/products/watch/2013/ww0033_prob.html). These include probabilities for multiple tornadoes and for tornadoes that are EF2 or stronger on the Enhanced Fujita (EF) scale, along with other associated severe weather hazards.
While the combined effect of these products is valuable to the weather community, particularly meteorological professionals, use of the TWS could simplify communication of only tornado risk to the general public. As such, this manuscript contains only a recommendation for the final product as delivered to consumers, leaving the actual conversion methods and forecasting decisions to be decided on by other parties.
Another product with similar objectives to the TWS is the experimental Impact Based Warning system, which was tested by the NWS in Kansas and Missouri in 2012 and expanded to the entire central United States during 2013 (www.crh.noaa.gov/crh/?n=2013_ibw_info). These warnings are meant to provide more information to the public in order to facilitate improved response to a tornado warning (Harrison et al. 2014). These warnings include explicit information on storm characteristics, such as presence of a tornado, potential tornado damage, and hail size in an attempt to elicit urgent protective actions from listeners. While the goals of this system mirror those of the TWS, the objective of the TWS is to redesign tornado watch statements to provide this information to a more generalized area in the hours before an event.
In addition to these communication products, NWS weather forecast offices (WFOs) will also use multimedia impact briefings, hazardous weather outlooks, and graphics illustrating relative threat levels for different (National Weather Service 2015) hazards (e.g., www.srh.noaa.gov/bmx/?n=HWO) to communicate forecasts of tornado risk. While these products provide weather hazard information in a simple format, easily disseminated on social media to increase the viewing audience, they do not address the issue of appropriate tornado refuge/safety locations in advance of an event, particularly those days when violent tornadoes are possible.
Communicating tornado risk has also been addressed by private weather entities, such as the tornado condition (TOR:CON) index used by The Weather Channel. These proprietary scales and indices may provide benefits by increasing awareness of a severe weather threat. It may also add an element of confusion if a person is presented with different systems and different levels of risk from different information providers. Because of this potential problem, the TWS was developed as a product for use across the entire weather communication enterprise, including the SPC and NWS, Internet weather information services, local and national broadcast meteorologists, highway information signs, billboards, and cell phone weather alerts as a uniform method of communicating risk levels and recommended actions to the public hours in advance of an event.
3. Development of the TWS
The first step in the transformation of the TRR (Mason and Senkbeil 2014) into the TWS was to develop a way of communicating these appropriate shelter/refuge locations to people in enough time to make decisions or alter their daily routines. This need is particularly important on days with an elevated risk of strong or violent tornadoes. On days with elevated risk, seeking refuge in an interior room on the lowest level of a sturdy building is not a guarantee of safety.
Because this system is designed to motivate the public to seek substantial shelter, such as a storm shelter or safe room, it is imperative that listeners or viewers are given sufficient time to process the information, discuss a shelter location, make a decision, and travel to the destination before a tornado arrives (Lindell and Perry 2012). Because of this possible increase in required travel time when compared to the availability of typical residential structures, the average tornado warning lead time of 13 min (Hoekstra et al. 2011) may be insufficient for some who must travel from their home, workplace, or school to a storm shelter or safe room.
Using the TRR as a basis, two levels were added to each of the existing tornado intensity classes (weak, strong, and violent) to form a level 0 to level 5 scale (Fig. 1). For example, a level 0 tornado watch would be issued for a low-end event with the possibility of weak tornadoes, such as a marginal squall line, mesoscale convective system, or quasi-linear convective system. A level 1 tornado watch would be issued for a slightly more intense event, perhaps tornadoes associated with deteriorating tropical cyclones in favorable environments (Schultz and Cecil 2009; Rhodes and Senkbeil 2014). This hierarchy continues with level 2 and level 3 tornado watches representing low-end and high-end events with the possibility of strong tornadoes. Level 4 and level 5 tornado watches would be reserved for days in which atmospheric conditions indicate the possibility of supercells with violent tornadoes. These highest-level watches are similar in form and function to the current particularly dangerous situation (PDS) tornado watch, albeit with a slightly different design. In short, the TWS modifies the existing tornado watch structure by defining and transforming the forecast levels from tornado watch and PDS tornado watch into six levels and providing a section for accompanying shelter/refuge locations for each level (see Fig. 1). In addition to text and audio tornado watch statements, the graphical representation of the TWS makes use of colors to symbolize the different hazard levels. Each of the three intensity categories is represented by its own color: weak is yellow, strong is orange, and violent is red. Prior research has shown the color red to elicit the highest feelings of severity, intensity, danger, or hazard, followed by orange and yellow (Ryan 1991; Braun et al. 1994; Chapanis 1994; Smith-Jackson and Wogalter 2000).
The TWS, revised from the tornado refuge rubric (Mason and Senkbeil 2014) to include tornado watch levels.
Citation: Weather, Climate, and Society 7, 2; 10.1175/WCAS-D-14-00035.1
4. Methods
a. Overview and procedures
An audio interview was adapted to iPad and used with a sample of 38 Alabama residents to gather both quantitative and qualitative data for analysis of the efficacy of the TWS, along with tornado hazard knowledge and understanding. Interviews were administered face-to-face, resulting in a 90% response rate. Each interview lasted approximately 20 min and consisted of both open and closed questions, including classification, behavioral, knowledge, and perception question types (Bird 2009). Some questions contained multiple parts, such as “Do you have children at home? If so, how many?” and “Do you have a tornado safety plan for your home? If so, what is the plan?”
The primary focus of this research was to collect data on how the TWS influenced understanding of the tornado hazard being communicated, tornado safety decisions, and preferences between the current NWS tornado watches and the proposed TWS watches. This was accomplished by presenting each participant with audio recordings of tornado watch statements. Each audio statement was followed by questions used to gather information on
what actions he/she would take after hearing the previous tornado watch statement;
what actions he/she would take if a tornado warning was issued for his/her home after hearing that particular watch statement on their television, radio, or NOAA weather radio the morning of the event; and
perceived danger to the participant associated with the tornado watch statement on a 1–10 scale, with 1 being the least dangerous and 10 being the most dangerous.
It is important to note the question asking participants about their actions after being presented with a tornado warning for their home was phrased as “What would you do…?” instead of “What should you do…?” The question was purposely constructed in this way to gather more accurate data on the likely behaviors of the participants, given their available resources, rather than simply assessing if they understood what actions would be best in the given situation. For this reason, the responses for safety actions following each hypothetical scenario likely represent a low-end estimate of statement efficacy in communicating risk because of some participants not having reasonable access to optimal shelter.
The texts from these statements were modified from SPC tornado watch statements issued for northern and central Alabama during the spring of 2011. Interviewer bias was minimized by using audio recordings for each hypothetical tornado watch statement, avoiding possible inconsistencies incurred by reading each statement. The recording mimicked the way participants would receive the information if heard over broadcast radio, NOAA weather radio, or television. An example of excerpts from audio statements is provided in Table 1 for the three TWS levels. Care was also taken not to identify the interviewer as being associated with development of the TWS throughout any portion of the interview. The authors made use of a spatial sampling technique, selecting participants congregated at parks, public buildings, and sporting events in central and northern Alabama (Patton 1990). The interview questions are included in online supplementary material along with full transcripts from the watch statement recordings.
Examples of excerpts from TWS level 1, level 3, and level 5 audio statements.
Closed interview questions were developed to gather demographic information from respondents for analysis of differences in tornado hazard knowledge attributable to age, race/ethnicity, gender, education level, marital status, type of home, number of children in the home, and size of the group in which they normally make tornado safety decisions. There were also questions aimed at collecting information on participants’ tornado hazard knowledge, including whether the participant has had any meteorological or storm-spotter training, along with open-ended definitional questions for common terms used in tornado hazard communication such as, “tornado watch,” “tornado warning,” “PDS tornado watch,” “tornado emergency,” and “warning polygon.” Also, in an attempt to gather information on tornado experience, participants were asked if they had ever been in a tornado watch, tornado warning, PDS tornado watch, or a tornado emergency.
To obtain data as a basis for comparison, participants were asked if they currently had a tornado safety plan for their home, and if applicable, where they shelter and how long it takes them to travel there. This information was compared against responses given for refuge/shelter location after hearing each tornado watch statement. Participants were also questioned about tornado safety plans for their workplace and vehicle, though they were not asked if these plans would change after hearing any of the hypothetical watch statements. The interview also included questions on what the participant considered adequate shelter during a tornado, what he/she would consider to be the ideal lead time for a tornado warning, if he/she owns a NOAA weather radio, and how he/she receives his/her weather warning information.
Participants were instructed to listen to audio recordings for each hypothetical event to create the most realistic scenario. The goal of the authors was to assess participant understanding, decision-making, and preferences when the respondents were given only minimal information without being able to reference a graphical illustration of the TWS (see Fig. 1). Following these recordings, the participants were asked for their preference between the current NWS tornado watch system and the proposed TWS system and to provide their reason for this preference.
After recording responses from each of the five scenarios and participant preference between the current tornado watch system and the TWS, participants were allowed to examine a graphical illustration of the TWS and asked what actions they would take if a tornado warning was issued for their home while under a level 1, level 3, or level 5 tornado watch.
Participants were also asked if they would be willing to participate in a postevent survey if a tornado warning was issued for their zip code. Of the 38 respondents in the preliminary study, 37 consented to being contacted for the follow-up survey by phone. This information was intended to be used to compare participants’ hypothetical decisions with those decisions made during an actual event. However, at the time of Institutional Review Board (IRB) expiration of the study there had been no tornado warnings affecting any of their zip codes.
b. Statistical analysis
Perhaps the most important part of this research was ascertaining how each participant’s safety decisions changed based on the information presented. If the TWS was effective, participants would increase their relative safety as risk increased. The response following one scenario was compared with the same participant’s response after hearing each of the other scenarios to determine if the participant’s actions would result in an increase in safety, no change in safety, or a decrease in safety from one scenario to another. As an example, if a participant indicated he/she would seek refuge in a lowest-level interior room after hearing a current PDS tornado watch, then in a storm shelter after hearing a level 5 tornado watch, this was classified as an increase in safety. Conversely, if a participant chose a storm shelter after hearing a current tornado watch, then a lowest-level interior room after hearing a level 3 tornado watch, this was classified as a decrease in safety. The order in which the scenarios were presented was as follows: current watch, level 1, PDS, level 5, and level 3. This order was chosen to allow for each participant to hear similar scenarios in pairs and counteract potential ordering impacts by presenting scenarios always increasing in severity.
To allow for a more direct comparison when classifying responses for the participant’s current home plan, or after hearing a current tornado watch or PDS tornado watch, these refuge/shelter locations were graded using the adequacy ratings from the Tornado Watch Scale corresponding to the tornado watch level being used for comparison. For example, if responses following a current tornado watch are being compared to those following a level 3 tornado watch, the adequacy of the locations cited for the current watch are graded using the adequacy ratings corresponding to a level 3 tornado watch in the TWS. An additional classification, “optimal,” was used to group participants who chose a storm shelter or safe room for both scenarios.
Cochran’s Q was used to determine if there were statistically significant differences in the number of participants choosing safer shelter for each scenario when compared to their existing home shelter plans. Following Cochran’s Q, McNemar’s test was used to determine if there were statistically significant increases in safer shelter selection using paired scenarios. For example, did the number of participants choosing safer shelter options significantly increase after hearing a level 5 tornado watch when compared to a PDS watch? McNemar’s test is appropriate to use for paired comparisons following a significant Cochran’s Q test (Rovai et al. 2014).
5. Results and discussion
a. Demographics
The demographic information for the interview participants reasonably approximates a representative sample of the state of Alabama according to the U.S. Census Bureau in most categories (U.S. Census Bureau 2014). State estimates represent U.S. Census Bureau percentages applied to our sample size of 38 respondents. For example, the U.S. Census Bureau estimates the population of Alabama to be 52% female, indicating a representative sample of 38 would include 20 female participants. Despite each person who declined to participate in the study being male, the distribution of gender for the sample is still representative of the state. Race/ethnicity is also representative of the state, with each sample category equaling the expected estimate. Because of the nature of the sampling technique, the youngest and oldest age groups are less representative than the middle-age categories. This is due to an increased number of participants 19–24 and decreased number of participants 55 and older at the sampling locations—typically parks, sporting events, and public buildings. Mobile home and apartment residents also appear to be slightly oversampled, which is likely related to the increased number of younger people in the sample. Also related to this age discrepancy is the increase in the number of participants indicating “some college” as their education.
In addition to the above demographic characteristics, participants were also asked to indicate their marital status and how many children they had living at home. Of the 38 participants, 19 indicated they were not married, 15 indicated they were married and 4 indicated they were divorced. Also, 21 of the respondents were without children, while the remaining 17 averaged 1.9 children per home. Care was also taken to ensure that no two people from the same household were interviewed.
b. Hazard awareness
The next section of the study focused on participant knowledge and experience with tornado hazards. This consisted of open and closed questions assessing 1) how the participants made safety decisions; 2) how they understood terms associated with tornado hazard communication; 3) if they had prior experience with tornado events; 4) if they had previous severe weather training; 5) if they had established current tornado safety plans for their home, vehicle, or workplace; 6) the preferred tornado warning lead time; and 7) what the participants considered “safe” shelter during a tornado.
Participants were asked to define, in their own words, a series of terms related to tornado hazard communication. The NWS definitions provided for the terms tornado watch, tornado warning, PDS tornado watch, tornado emergency, and warning polygon were then assembled into categories and graded by their accuracy into groups of correct or partially correct responses and incorrect responses. Credit was given for partially correct responses to allow for participants who could communicate the general idea of a term, even if part of the NWS definition was omitted.
The results of this classification are presented in Table 2, which shows 25 respondents were able to provide a correct or partially correct definition for a tornado watch, the majority of which were quoted as saying “conditions are favorable for tornadoes,” while eight participants provided a description of a tornado warning instead. The other five participants stated that they did not know what a tornado watch was, and their responses were grouped into the category “incorrect/too vague/unknown.” The next term with the highest count of correct or partially correct responses was tornado warning. Of the 38 responses, 22 included descriptions indicating that a tornado has been spotted, was on the ground, or was indicated by radar. The remaining 16 responses were incorrect, with four confusing a tornado warning with a tornado watch. Ten of the respondents correctly described the general idea of a warning polygon as the area where a tornado could go, while 28 responses were incorrect, with one of those participants stating a warning polygon was used to indicate that eight tornadoes were on the ground.
Classified responses to tornado hazard term definitions from interview participants.
To assess the preference on tornado warning lead time, each participant was asked the question, “What would you consider to be the ideal lead time for a tornado warning?” Though the exact number was not recorded, several respondents were not sure what this question was asking, requiring clarification along the lines of, “How many minutes would you like to have between a tornado warning being issued and the tornado arriving at your location?” Because one participant indicated she would like a week of lead time (10 080 min) for a tornado warning, the average ideal lead time for the entire sample set was 302 min. When excluding this outlier, the average for the rest of the sample (n = 37) was 37 min. This agrees with findings from Hoekstra et al. (2011), who reported a preferred tornado warning lead time of 34 min when surveying 320 visitors to the National Weather Center in Norman, Oklahoma, during the summer and fall of 2009. Excluding the outlier, the maximum preferred lead time was 180 min and the minimum lead time was 5 min, with 10 participants choosing 30 min.
Participants who provided a refuge or shelter location for their home were asked to estimate how long it would take to travel to their home refuge/shelter location after receiving a tornado warning. Of the 29 participants with tornado safety plans for their home, the average estimated time to reach that location was 2 min, with a minimum value of 1 min and maximum value of 15 min. Several people indicated their travel time would be “less than a minute,” but these responses were rounded up to one to facilitate analysis. Those participants living in apartments (n = 6) reported the least travel time, with an average of 1.0 min, while those in permanent homes (18) averaged 1.2 min. Interview participants who reported living in mobile homes (5) had a substantially higher average travel time of 6.2 min.
To develop an understanding of what the participants viewed as a safe structure during a tornado, each was asked, “What do you consider safe shelter during a tornado?” The most common response chosen, a lowest-level interior room, corresponds to the language used in typical tornado warning statements from the NWS. Almost half (n = 16) of the respondents indicated they perceived this as safe shelter during a tornado, followed by 12 who stated they only consider areas underground to be safe, while six chose a storm shelter or safe room and four chose other responses. These other responses included, “somewhere I don’t get killed,” “anything that’s not moveable,” “under something heavy,” and “out of the path of the tornado.”
c. TWS evaluation and preferences
The first statement presented was that of a typical NWS SPC tornado watch, the basis of which was taken from Tornado Watch Number 140 issued at 1235 UTC 15 April 2011 for portions of northern and central Alabama, eastern Louisiana, and central and southern Mississippi. The date and time of the statement were amended in the interview in an attempt to disassociate the statement from that particular event. The detailed location description was omitted from each statement to expedite the interview process. The full transcript of the audio statement is included in supplementary material.
When asked what actions they would take if a tornado warning was issued for their home after hearing this message earlier that morning, 20 participants said they would go to a lowest-level interior room for shelter (Table 3). These locations included bathrooms, closets, hallways, and under stairwells. This was followed by seven who would go to a storm shelter, four to a fully underground basement, and four to a partially underground basement. An institutional building, a second-floor bathroom, and no action were each chosen once. This change in behavior was noticeably different from simply asking the participants if they had a tornado safety plan for their home, nine of which did not.
Refuge/shelter locations reported by interview participants when asked if they had a tornado safety plan for their home (current home tornado safety plan) and what actions they would take if a tornado warning was issued for their home after hearing the preceding tornado watch statement (current tornado watch statement through level 5 tornado watch statement).
When asked the same question after hearing an audio statement describing a PDS tornado watch, modeled after Tornado Watch Number 235, issued at 1845 UTC 27 April 2011 for portions of Alabama, Georgia, Mississippi, and Tennessee, the number of participants choosing a lowest-level interior room for shelter dropped to 18, while the number using a storm shelter increased to eight (Table 3). One participant still selected an institutional building, while no participant chose a second-floor bathroom or no shelter after hearing the PDS tornado watch statement. The largest change was seen in the number of participants who reported that they would evacuate after hearing the tornado warning. This increased from zero after hearing a standard tornado watch to three after hearing a PDS tornado watch.
Participants were also presented with hypothetical scenarios for three levels of the Tornado Watch Scale. These statements were also modeled after the same statements from the NWS SPC, with the addition of a section describing the corresponding common refuge/shelter options recommended in the TWS. The full transcripts of these hypothetical statements are also available in the supplementary material, with excerpts shown in Table 1.
When presented with a hypothetical level 1 tornado watch statement and asked what actions they would take if a tornado warning was issued for their home later that day, 22 participants chose a lowest-level interior room, while six chose a storm shelter, four chose a fully underground basement, three chose a partially underground basement, and one participant each indicated he/she would take refuge in an institutional building, a second-floor interior room, or evacuate (Table 3). An important difference in evacuation as recommended by the Tornado Watch Scale and included in the audio recording of the hypothetical TWS statements is that evacuation is only recommended if adequate shelter is unavailable, if the person evacuates the tornado watch area before a tornado warning is issued, and if the person is familiar with the area and has access to storm information during evacuation.
After hearing a hypothetical level 3 tornado watch statement, the number of participants indicating they would go to a lowest-level interior room fell to 13, while the number who chose a storm shelter or safe room rose to 11 (Table 3). In this scenario, six participants indicated they would go to a fully underground basement, four to a partially underground basement, and one to an institutional building. Additionally, three indicated they would evacuate the area before a tornado warning was issued.
The greatest divergence from the current home tornado safety plan was seen when participants listened to the level 5 tornado watch statement and were then asked what actions they would take if a tornado warning was issued for their home later in the day. This set of responses saw the number of participants who chose a lowest-level interior room drop to 10, while the number indicating they would seek out a storm shelter or safe room rose to 15 (Table 3). Four participants chose a partially underground basement for this scenario, while three chose a fully underground basement and one chose an institutional building. Also, five participants indicated they would evacuate if a level 5 tornado watch was issued for their area.
Cochran’s Q was used to determine if there was a statistically significant difference in the number of participants choosing safer shelter options for each scenario when compared to that resident’s existing home plan. The sample size was 32 for this test because of six participants who indicated they would use a storm shelter or safe room for their existing home plan. The Cochran’s Q test statistic was 30.8 (p < 0.01) with 4 degrees of freedom. Of the 32 participants, a total of 9 chose safer shelter for a current watch, 10 for level 1, 12 for PDS, 20 for level 5, and 17 for level 3. Thus, only level 3 and level 5 appear to have persuaded more than half of the participants toward safer shelter options.
Following the Cochran’s Q test, McNemar’s test was used to determine the statistical significance of change in each pair of scenarios (Rovai et al. 2014; Table 4). There were numerous pairs with statistically significant increases in safety. A level 3 watch (p < 0.01) and a level 5 watch (p < 0.01) both resulted in statistically safer actions when compared to a current NWS tornado watch. A level 1 watch (p = 0.50) and an NWS PDS watch (p = 0.13) did not result in an increase in safety compared to an NWS tornado watch. This suggests that the current NWS PDS tornado watch may not be particularly useful in eliciting safer shelter actions, while a level 1 watch is essentially the same as a current NWS tornado watch. Furthermore, a PDS watch had the same result as a current NWS tornado watch (p = 0.50) when compared to a level 1 watch. All other comparisons resulted in statistically significant increases in safety with alpha at 1%, with the exception of a PDS watch compared to a level 3 TWS watch (p = 0.03). These results suggest that the TWS is especially effective at stimulating safer decision-making when compared to current tornado watch statements. Because these results only represent response findings after hearing audio recordings, they are likely a conservative estimate of efficacy. These results would likely be even stronger if respondents had been allowed to view the TWS graphic as they were listening to the audio statements, or if the question had been framed to assess participant understanding instead of change in intended actions.
Results of McNemar’s test for statistical significance of change in relative safety of stated tornado shelter locations by scenario. Significant results at the 0.05 (0.01) level are italicized (bold italicized).
d. Does viewing the TWS graphic result in further safety increase?
After completing all of the previously discussed questions following audio-only statements, participants were then presented with a graphical depiction of the Tornado Watch Scale (see Fig. 1). After allowing time for each participant to become familiar with the graphic, he/she was asked a series of three questions involving each of the three previously used levels. For example, “If you are at home under a level 1 tornado watch and a tornado warning is issued for your location, what actions do you take?” These responses were then compared against the participant’s response for the corresponding level following the audio statement.
For a level 1 tornado watch, 33 participants saw no change in their relative safety when using the graphical illustration instead of the audio statements, while four chose optimal shelter and one increased their relative safety after using the TWS graphic (Table 5). Of the 33 who saw no change in safety, two of these originally chose a storm shelter following the audio statement, but then chose a first-floor interior room when using the graphic. Because both of these are considered “adequate” shelter for a level 1 tornado watch, they were classified as “no change” in relative safety.
Comparison of relative safety when participants’ responses for their tornado safety actions following the audio tornado watch statements are compared to their tornado safety actions after viewing the graphical representation of the TWS.
Using a level 3 tornado watch, 23 participants saw no change in their safety, while 10 chose a storm shelter or safe room, four participants increased their safety, and one decreased after using the TWS graphic (Table 5). The decrease came from a participant who initially chose a storm shelter following the audio statement, which is considered adequate shelter using the TWS, then chose a partially underground basement using the graphic, which is considered “questionable” shelter by the TWS.
Last, for a level 5 tornado watch, 19 participants’ responses saw no change in relative safety between audio and graphical methods, while 15 chose optimal shelter and four increased their relative safety (Table 5). These increases were accounted for by participants who chose a storm shelter or safe room after viewing the TWS graphic. Previously, these four participants had chosen first-floor interior rooms, partially underground basements, or evacuation as their safety actions after hearing the level 5 TWS statement. Ultimately, after being presented with the TWS graphic, 19 participants indicated they would use a storm shelter or safe room as their shelter area if presented with a tornado warning while under a level 5 tornado watch, up from seven who chose a storm shelter or safe room after hearing the current tornado watch statement. As was the case with the audio statements, these results likely underestimate participants’ ability to use the TWS scale correctly because of the syntax of the question asking respondents what actions they would take, instead of what actions they should take.
e. Ranking each category on an ordinal scale
Following each audio statement, participants were also asked to rate the danger they felt associated with each watch statement on a 1–10 scale, with 10 being the most dangerous. After averaging all participant responses, the danger associated with each statement was ranked as expected, with a PDS tornado watch being rated as more dangerous than a standard tornado watch and the average danger rating for each level of the TWS scale presented to the participants being rated successively. These responses resulted in a PDS tornado watch receiving a rating of 7.68 out of 10, while a standard tornado watch received a 5.76. As expected, the results show participants’ perceived danger from the current tornado watch statement (5.76) as comparable to the level 1 tornado watch statement (6.55). Participants also felt similar levels of danger associated with a PDS tornado watch (7.68) and a level 3 tornado watch (7.45). Participants rated the level 5 tornado watch as a 9.25, with 26 of the 38 participants rating their perceived danger as a 10.
f. Comments
Likely the most important question related to the proposed TWS was asking participants directly, “Of the two communication methods you just heard, the current National Weather Service tornado watch system, which used a tornado watch and a PDS tornado watch, and the Tornado Watch Scale, which used level 1, level 3, and level 5 tornado watches, which method did you prefer, and why?” Of the 38 participants in this study, 37 said they preferred the Tornado Watch Scale communications, while 1 chose the current NWS tornado watch system (Table 6). When asked for the reasoning behind this preference, the one person who chose the current system said, “Because it’s more readily available.” It is unclear if this participant, a 74-yr-old male, fully understood the question being asked, as he asked for clarification three times. Of those who preferred the TWS, the most common reasons for this choice included more information/more detail, easier to understand, describes severity, more explanation, and the numerical rating scale.
Participant responses when asked for their preference between the current NWS tornado watch and PDS tornado watch communication statements vs the TWS statements.
After completion of the interview questions, participants were given a chance to add their own questions, comments, or suggestions. Three participants took advantage of this opportunity, the first providing insight into the need for better advertisement of public storm shelters, saying, “I would give more alternative places to go and tell where storm shelters are located. I’m seeing storm shelter, but I don’t know where they are for real.” Another participant, an elementary school principal, remarked, “As a school principal, I would find this system helpful in understanding the severity of an event.” Last, one participant’s response encapsulated this study’s primary objective, which is motivating the public to choose adequate shelter by better communication of enhanced tornado risk saying, “I’ve got one shelter for everything now, but that level 5 made me think about getting something more substantial.”
6. Limitations
The concept of the TWS is still evolving, and this research represents an initial presentation of the idea and discussion of its potential value based on strong results in northern and central Alabama. Our study area has recently been impacted by several violent tornadoes in the last 5 years, and the TWS may have resonated more in the minds of our small sample of participants when compared to a less active region. Furthermore, our spatial sampling technique was not random and we may have had participants with more interest in weather than the general population. In future research we hope to continue to evaluate the efficacy of the TWS on a broader scale with a large random sample.
Potential implementation of the TWS as a functional operational product is plagued by the same concerns that all warning and watch scales and weather forecasts share (Barnes et al. 2007b). On the conceptual model of warning accuracy from Barnes et al. (2007b), weather forecasts ideally need to always be correct, but that will never be true. Forecasts need to be reasonably accurate because error reduces credibility and perception of the accuracy of warning systems (Ripberger et al. 2014). What if there is a reduction in credibility after a series of level 3, level 4, and level 5 tornado watches are issued with consistent overforecasting? What if a level 1 watch is issued, then 12 h later unexpected atmospheric developments result in several tornadoes in the EF3 intensity range? Just as tornado watches are currently issued and some are upgraded to PDS status, a TWS watch could also be upgraded or downgraded depending on changing conditions. Similarly, hurricane intensity forecasts are updated every 6 h as a storm prepares to make landfall.
One possible solution to deterministic forecasting is using uncertainty. In current research, we are attempting to incorporate uncertainty into the TWS so that users would be given a confidence interval for potential intensity that can possibly span across more than one TWS level. We preliminarily piloted this idea using a private sample of 70 followers in Tuscaloosa for an October 2014 severe weather event. We forecasted a confidence interval of 0.7–1.7 on the TWS (Fig. 1) for that event, placing the entire forecast within the “weak” category. Despite what we considered narrow uncertainty, some of our followers new to the region thought the information was overwhelming and somewhat confusing preevent, but later indicated that they understood postevent. Long-term residents almost universally supported the incorporation of uncertainty. If our confidence interval had spanned the weak and strong categories, our preliminary feedback may have been less positive. We will continue to explore using the TWS with and without uncertainty.
7. Conclusions
The Tornado Watch Scale was developed as a conversion of current probabilistic SPC products to a categorical forecast product in the 24 h prior to an expected tornado event. It is intended to be used in the temporal transition period from convective outlooks to tornado warnings by providing individuals with additional information on expected tornado intensity in the forecast area.
Through our preliminary study, it is evident the TWS was adequately understood when participants were given only an audio tornado watch statement on which to base their tornado safety decisions. This response was further improved after participants were presented with a graphical illustration (Fig. 1), particularly for the most intense classification. Results likely would have been stronger if the study had focused on participants choosing the correct recommended action instead of examining their intended safety actions, because of the limited availability of storm shelters and safe rooms. Based on these results, the TWS was effective in accomplishing its objective of providing tornado hazard communication information meant to stimulate appropriate life-saving protective action through more specific and descriptive tornado watch statements. Since the participants almost unanimously indicated preference for the Tornado Watch Scale communication statements, it is hoped that this system will be considered and modified for use as an experimental forecast product to determine its efficacy on a larger scale.
Acknowledgments
We would like to thank two anonymous reviewers for their helpful comments and suggestions.
REFERENCES
Balluz, L., Schieve L. , Holmes T. , Kiezak S. , and Lalilay J. , 2000: Predictors for people’s response to a tornado warning: Arkansas, 1 March 1997. Disasters, 24, 71–77, doi:10.1111/1467-7717.00132.
Barnes, L. R., Gruntfest E. C. , Hayden M. H. , Schultz D. M. , and Benight C. , 2007a: Can close calls help define warning accuracy? Bull. Amer. Meteor. Soc., 88, 1529.
Barnes, L. R., Gruntfest E. C. , Hayden M. H. , Schultz D. M. , and Benight C. , 2007b: False alarms and close calls: A conceptual model of warning accuracy. Wea. Forecasting,22, 1140–1147, doi:10.1175/WAF1031.1; Corrigendum, 24, 1452–1454, doi: 10.1175/2009WAF2222300.1.
Bird, D. K., 2009: The use of questionnaires for acquiring information on public perception of natural hazards and risk mitigation—A review of current knowledge and practice. Nat. Hazards Earth Syst. Sci., 9,1307–1325, doi:10.5194/nhess-9-1307-2009.
Braun, C. C., Sansing L. , Kennedy R. S. , and Silver N. C. , 1994: Signal word and color specifications for product warnings: An isoperformance application. Proc. Hum. Factors Ergon. Soc. Annu. Meet.,38,1104–1108, doi:10.1177/154193129403801707.
Brotzge, J., and Erickson S. , 2009: NWS tornado warnings with zero or negative lead times. Wea. Forecasting, 24, 140–154, doi:10.1175/2008WAF2007076.1.
Brotzge, J., Erickson S. , and Brooks H. , 2011: A 5-yr climatology of tornado false alarms. Wea. Forecasting, 26, 534–544, doi:10.1175/WAF-D-10-05004.1.
Brotzge, J., Nelson S. E. , Thompson R. L. , and Smith B. T. , 2013: Tornado probability of detection and lead time as a function of convective mode and environmental parameters. Wea. Forecasting, 28, 1261–1276, doi:10.1175/WAF-D-12-00119.1.
Chapanis, A., 1994: Hazards associated with three signal words and four colours on warning signs. Ergonomics, 37, 265–275, doi:10.1080/00140139408963644.
Coleman, T. A., Knupp K. R. , Spann J. , Elliott J. B. , and Peters B. E. , 2011: The history (and future) of tornado warning dissemination in the United States. Bull. Amer. Meteor. Soc., 92, 567–582, doi:10.1175/2010BAMS3062.1.
Erickson, S. A., and Brooks H. , 2006: Lead time and time under tornado warnings: 1986–2004. Preprints, 23rd Conf. on Severe Local Storms, St. Louis, MO, Amer. Meteor. Soc., 11.5. [Available online at https://ams.confex.com/ams/23SLS/techprogram/paper_115194.htm.]
Farley, J. E., 2007: Call-to-action statements in tornado warnings: Do they reflect recent developments in tornado-safety research. Int. J. Mass Emerg. Disasters, 25 (1), 1–36. [Available online at www.ijmed.org/articles/48/download/.]
Hammer, B., and Schmidlin T. W. , 2002: Response to warnings during the 3 May 1999 Oklahoma City tornado: Reasons and relative injury rates. Wea. Forecasting, 17, 577, doi:10.1175/1520-0434(2002)017<0577:RTWDTM>2.0.CO;2.
Harrison, J., McCoy C. , Bunting-Howarth K. , Sorensen H. , Williams K. , and Ellis C. , 2014: Evaluation of the National Weather Service impact-based warning tool. WISCU-T-14-001, 37 pp. [Available online at http://www.iisgcp.org/glssn/IBW_finalreport.pdf.]
Hoekstra, S., Klockow K. , Riley R. , Brotzge J. , Brooks H. , and Erickson S. , 2011: A preliminary look at the social perspective of warn-on-forecast: Preferred tornado warning lead time and the general public’s perceptions of weather risks. Wea. Climate Soc., 3, 128–140, doi:10.1175/2011WCAS1076.1.
Kantha, L., 2006: Time to replace the Saffir-Simpson hurricane scale? Eos, Trans. Amer. Geophys. Union, 87, 3–6, doi:10.1029/2006EO010003.
League, C. E., Diaz W. , Philips B. , Bass E. J. , Kloesel K. , Gruntfest E. , and Gessner A. , 2010: Emergency manager decision-making and tornado warning communication. Meteor. Appl., 17, 163–172, doi:10.1002/met.201.
Lindell, M. K., and Perry R. W. , 2012: The Protective Action Decision Model: Theoretical modifications and additional evidence. Risk Anal., 32, 616–632, doi:10.1111/j.1539-6924.2011.01647.x.
Mason, J. B., and Senkbeil J. C. , 2014: Implications of the 2011 Tuscaloosa EF4 tornado for shelter and refuge decisions. Nat. Hazards, 74, 1021–1041, doi:10.1007/s11069-014-1230-4.
National Weather Service, cited 2015: SPC products. NOAA/NWS/NCEP/SPC. [Available online at http://www.spc.noaa.gov/misc/about.html.]
Patton, M. Q., 1990: Qualitative Evaluation and Research Methods. 2nd ed. Sage Publications, 532 pp.
Paul, B. K., Brock V. T. , Csiki S. , and Emerson L. , 2003: Public response to tornado warnings: A comparative study of the May 4, 2003, tornadoes in Kansas, Missouri, and Tennessee. Quick Response Rep. 165, Natural Hazards Research Applications and Information Center, Boulder, CO, 27 pp. [Available online at www.colorado.edu/hazards/research/qr/qr165/qr165.pdf.]
Rhodes, C. M., and Senkbeil J. C. , 2014: Factors contributing to tornadogenesis in landfalling Gulf of Mexico tropical cyclones. Meteor. Appl., 21, 940–947, doi:10.1002/met.1437.
Ripberger, J. T., Silva C. L. , Jenkins-Smith H. C. , Carlson D. E. , James M. , and Herron K. G. , 2014: False alarms and missed events: The impact and origins of perceived inaccuracy in tornado warning systems. Risk Anal., 35, 44–56, doi:10.1111/risa.12262.
Rovai, A. P., Baker J. D. , and Ponton M. K. , 2014: Social Science Research Design and Statistics. 2nd ed. Watertree Press, 630 pp.
Ryan, J. P., 1991: Design of Warning Labels and Instructions. Wiley, 201 pp.
Schultz, L. A., and Cecil D. J. , 2009: Tropical cyclone tornadoes 1950–2007. Mon. Wea. Rev., 137, 3471–3484, doi:10.1175/2009MWR2896.1.
Schumacher, R. S., Lindsey D. T. , Schumacher A. B. , Braun J. , Miller S. D. , and Demuth J. L. , 2010: Multidisciplinary analysis of an unusual tornado: Meteorology climatology and the communication and interpretation of warnings. Wea. Forecasting, 25, 1412–1429, doi:10.1175/2010WAF2222396.1.
Senkbeil, J. C., and Sheridan S. C. , 2006: A post landfall hurricane classification system for the United States. J. Coast. Res., 22, 1025–1034, doi:10.2112/05-0532.1.
Sherman-Morris, K., 2010: Tornado warning dissemination and response at a university campus. Nat. Hazards, 52, 623–638, doi:10.1007/s11069-009-9405-0.
Simmons, K. M., and Sutter D. , 2008: Tornado warnings, lead times, and tornado casualties: An empirical investigation. Wea. Forecasting, 23, 246–258, doi:10.1175/2007WAF2006027.1.
Simmons, K. M., and Sutter D. , 2009: False alarms, tornado warnings, and tornado casualties. Wea. Climate Soc., 1, 38–53, doi:10.1175/2009WCAS1005.1.
Simmons, K. M., and Sutter D. , 2011: Economic and Societal Impacts of Tornadoes. Amer. Meteor. Soc., 282 pp, doi:10.1007/978-1-935704-02-7.
Smith-Jackson, T. L., and Wogalter M. S. , 2000: Applying cultural ergonomics/human factors to safety information research. Proc. Hum. Factors Ergon. Soc. Annu. Meet.,44,6.150–6.153, doi:10.1177/154193120004403319.
Sorensen, J. H., 2000: Hazard warning systems: Review of 20 years of progress. Nat. Hazards Rev., 1, 119–125, doi:10.1061/(ASCE)1527-6988(2000)1:2(119).
U.S. Census Bureau, cited 2014: Alabama Quickfacts from the U.S. Census Bureau. [Available online at http://quickfacts.census.gov/qfd/states/01000.html.]
