Measured consequence-based messages can have an important and desirable effect on the responsiveness to severe weather warnings.
On 22 May 2011, one of the most devastating tornadoes in U.S. history killed 159 people and injured thousands of others in Joplin, Missouri. According to the National Weather Service (NWS), this storm was a warned disaster in that the residents of Joplin received advance notice and critical information about the tornado prior to its arrival. Fortunately, many Joplin residents responded to these warnings by immediately going to the nearest shelter. Unfortunately, some residents received similar information (i.e., heard one of the siren alerts that were sounded), but for various reasons decided against protective action (NOAA 2011a; Kuligowski et al. 2014). In some instances this decision may have proved fatal.
In the aftermath of this and another set of historic tornadoes that swept through a number of southeastern states in April 2011, the NWS commissioned a set of service assessments (NOAA 2011a,b). These assessments provide an important and sobering reminder that scientific and technological advancements in our understanding of severe weather can only go so far—tornadoes occur in a social environment wherein people, not technology, are responsible for making decisions about when and how to protect themselves. In such an environment, effective communication between the NWS, broadcast meteorologists, emergency managers, and the public plays a critical role in mitigating the societal impact of tornadoes.
In response to this reminder, the Central Region Headquarters (CRH) of the NWS initiated an experimental product that supplements traditional tornado and severe thunderstorm warning products with information about the potential impact of warned storms. According to the CRH, impact-based warnings (IBWs) are designed to “motivate proper response to warnings by distinguishing situational urgency” (Central Region Headquarters 2014). They accomplish this by explicitly and concisely highlighting the potential impact of particularly dangerous storms. The logic associated with their design is rather simple—when faced with convective conditions, individuals assess (among other things) the probability that a tornado will touch down in their area and the consequences that it might have if it were to touch down. If the probability and/or consequences are judged to be high, then individuals will engage in some sort of protective action. If the probability and/or consequences are judged to be low, then people will be less likely to take protective action because it is costly in terms of time, effort, and loss of productivity/leisure (Sutter and Erickson 2010; Simmons and Sutter 2011).
EXAMPLE OF AN IMPACT-BASED WARNING
This figure provides an example of an IBW product that was issued by the NWS office in Wichita, Kansas, on 14 April 2012. The red portions of the figure highlight impact information, which is provided in two forms: an impact statement and a damage threat tag. Impact statements provide a brief narrative that lists the possible consequences of specific storms. They can and do vary across warned storms. Damage threat tags, by comparison, represent single words intended to provide a standardized description of the potential consequences associated with warned storms. As of 2013, IBW products that include tornado damage threat tags use one of two words—considerable or catastrophic—to describe the potential consequences of warned storms. The word considerable is used when “credible evidence that a tornado, capable of producing considerable damage, is imminent or ongoing” (NWS; Central Region Headquarters 2013). Catastrophic is used when a “severe threat to human life and catastrophic damage from a tornado is occurring, and will only be used when reliable sources confirm a violent tornado” (NWS; Central Region Headquarters 2013). For more information on the IBW experiment, see online (at www.crh.noaa.gov/crh/?n=2013_ibw_info or www.weather.gov/impacts/#.U1VfD17308N).
Traditionally, NWS warning products focused on the probability side of this equation while neglecting the consequences side. As a result, people were unable to differentiate between high- and low-consequence events, which could lead to suboptimal responses. This began to change in the 1970s when the NWS started to include situation-specific call-to-action (CTA) statements in the warnings they issued. Among other things, CTA statements provide forecasters an opportunity to indicate the degree of danger posed by the particular event and tell users what can be done to prevent, avoid, or minimize that danger (McLuckie 1974; NWS 2010). Though useful, the content of CTA statements varies rather broadly from weather forecast office (WFO) to WFO. Some WFOs use them to provide “generic safety rules,” whereas other WFOs use them to provide detailed information about the potential impact of an approaching storm (Troutman Smith and Rose 2001). IBWs address this inconsistency by explicitly adding information about the potential impact of particularly dangerous storms to all of the warnings they issue. With this information, the NWS and their partner organizations can provide the public with more complete and consistent messages about the potential consequences of a storm, which should save lives by increasing the likelihood that people will take protective action in response to potentially catastrophic tornadoes like the one that hit Joplin on 22 May 2011.
EMPIRICAL RESEARCH.
Despite the simplicity of this logic, we know relatively little about the influence of consequence-based messages on public responses to tornado warnings. There is, however, some research on the relationship between projected impact and hurricane evacuation. Peacock and Gladwin (1993), for example, surveyed 1,300 residents of Dade County, Florida, three months after Hurricane Andrew made landfall in August 1992. One of the many questions they asked prompted respondents to indicate how they would respond if a future hurricane were approaching their residence. Almost half (47%) of their respondents said that their response would depend on the intensity of the storm, suggesting that individuals do, in fact, consider the potential consequences of an event when making decisions about how to respond. Dow and Cutter (1998) replicated this study in the aftermath of Hurricanes Bertha (1996) and Fran (1996) and came to a similar conclusion—a large number of people use information about the severity of a storm when contemplating evacuation. They confirmed this in a later study (Dow and Cutter 2000), wherein they interviewed a diverse group of South Carolina residents who were affected by Hurricane Floyd in September 1999. Among other things, a large fraction of the people they interviewed said that the projected severity of the storm was the most important factor they considered when deciding whether or not to evacuate.
These findings were corroborated in interviews conducted by Morss and Hayden (2010) in Galveston and Kemah, Texas, after Hurricane Ike hit the region in September 2008. When asked about future storms, less than half of the people they interviewed indicated that they would evacuate in advance of a category 1 or 2 hurricane, whereas the vast majority said that they would evacuate for category 3, 4, or 5 storms. That said, individual reactions to specific consequence-based messages that were issued immediately before Ike made landfall, like “persons not heeding evacuation orders in single family one or two story homes will [may] face certain death,” were mixed—some residents indicated that they were “blunt” but effective, whereas others said that they were “harsh” and “overblown.” In the end, however, Morss and Hayden (2010) note that the statements did motivate at least a few respondents to leave and thus “may have saved lives” (p. 186).
In a separate but complementary line of research, a number of studies have used experimental techniques to isolate the causal effect of consequence-based messages on public responses to hurricanes. For example, Baker (1995) conducted a survey of roughly 400 residents of Pinellas County, Florida. One of the questions on Baker’s survey asked respondents to read and indicate how they would respond to a hypothetical scenario involving a hurricane. The text describing the scenario, however, was not the same for every respondent. Rather, one group was randomly assigned to a category 1 hurricane involving 85 mile per hour (mph; 37.9984 m s–1) sustained winds. The other group was randomly assigned to a category 4 hurricane scenario with 150 mph sustained winds. He then compared responses across groups and found that the probability of evacuation was significantly higher among the groups who were assigned to the higher-impact scenario. Whitehead et al. (2000) used a similar procedure to survey approximately 900 residents of North Carolina who were affected by Hurricane Bonnie in August 1998. In so doing, they arrived at the same conclusion. Respondents who were presented with relatively intense hypothetical storms were more likely to opt for evacuation than respondents who were presented with less intense scenarios.
Though useful, it is important to reiterate that these studies were designed to answer questions about protective action in response hurricane warnings. Hurricane warnings provide more lead time and cover much broader regions than most tornado warnings. As a result, the protective action calculations associated with responding to tornado warnings are very different than the calculation associated with hurricane warnings. In a select few instances, for example, evacuation may be an appropriate response to a tornado warning; in most other instances, evacuation may increase rather than decrease the risks associated with tornadoes (Simmons and Sutter 2012).1 The inverse is true of hurricane warnings. Thus, it is necessary to test the generalizability of the abovementioned findings to short-fuse tornado warnings intended to elicit different types of protective action. In this study, we accomplish this by addressing two interrelated questions—do consequence-based messages influence public responsiveness to tornado warnings? If so, how?
DATA.
To answer these questions, we designed and administered a survey experiment to residents of the most tornado-prone regions of the continental United States. Including the experiment, the survey contains 144 questions that gauge perceptions about weather, tornadoes, and warnings, as well as a variety of sociodemographic characteristics, including geographic location, residential situation, income, race, ethnicity, and education.2 We administered the survey twice, in 2012 and 2013. The 2012 survey was fielded in eight weekly waves between 12 September and 1 November with each wave consisting of approximately 500 randomly selected members of the SurveySpot Internet survey panel. Similarly, the 2013 survey was fielded in eight weekly waves between 8 May and 27 June with each wave consisting of approximately 500 randomly selected members of the same panel.
Our survey panel was recruited and maintained by Survey Sampling International (SSI).3 Because we are interested in individual perceptions about and responses to tornadoes, we geographically conditioned our selection of both samples such that the people asked to take our survey reside in a tornado-prone region of the United States. Members of the panel qualified as living in a tornado-prone region if the address they registered with SSI is located in one of the high-vulnerability regions listed by Ashley (2007) in his climatological study of significant and fatal tornadoes between 1880 and 2005. We also oversampled members of the panel that reside in rural settings so as to maximize geographic coverage and combat the urban bias typically associated with Internet access and participation in web-based surveys (Couper 2000).
This process yielded data from 3,989 respondents in 2012 and 3,975 respondents in 2013 that reside at the locations depicted in Fig. 1. Demographic characteristics of these samples are presented in Table ES1 in the supplementary material (online at http://dx.doi.org/10.1175/10.1175/BAMS-D-13-00213.2).
Approximate location of survey respondents (by zip code centroid).
Citation: Bulletin of the American Meteorological Society 96, 4; 10.1175/BAMS-D-13-00213.1
RESEARCH DESIGN.
Survey experiment.
To isolate the causal effect of consequence-based messages on public responses to tornado warnings, we followed the lead of Baker (1995) and Whitehead et al. (2000) by presenting subjects in both surveys with the following experimentally manipulated scenario and then asking them to indicate how they would respond if they were to receive such a warning in the future.
Q88: While you are at home during daylight hours, if you were to learn that the National Weather Service has issued a tornado warning for (randomize: light; moderate; significant; severe; devastating; and incredible) tornadoes in your local area, which of the following most accurately describes what you would do?
0—Nothing; continue on as before the warning was received.
1—Move to the most sheltered part of your residence, but do not leave your residence.
2—Move to a specially constructed storm shelter on your property.
3—Move to a nearby location or building that you consider to provide better shelter.
4—Leave your residence and drive away from the tornado warning area.
5—Something else (please specify).
Table provided to survey respondents that describes and illustrates the difference between light, moderate, significant, severe, devastating, and incredible tornadoes.
Citation: Bulletin of the American Meteorological Society 96, 4; 10.1175/BAMS-D-13-00213.1
Warnings response.
Consistent with previous research (i.e., Schultz et al. 2010), we use intended responses to this hypothetical warning as a rough proxy for actual behavior in response to a real tornado warning. Specifically, we created two variables that capture intended responses to the scenario. First, we created a binary “protective action” variable, where “do nothing” responses were coded as 0 and responses 1, 2, 3, and 4 were coded as 1 (“take action”). When measured in this way, 91% of the 2012 respondents said that they would take some sort of protective action in response to the hypothetical warning, whereas 9% indicated that they would do nothing. In 2013, 89% of the respondents said that they would take protective action, and 11% indicated that they would do nothing.4
Second, we created a three-level—do nothing, shelter in place, or leave residence—categorical response variable based on the option that respondents selected. Respondents were assigned to the do nothing category if they selected option 0 in response to Q88, to the “shelter in place” category if they selected option 1 or 2, and the “leave residence” category if they selected option 3 or 4 in response to the hypothetical warning. This five-option choice set was collapsed into three similar categories because preliminary analysis (Hausman–McFadden tests) indicated that options 1 and 2—both of which involve sheltering in place—and 3 and 4—which involve leaving the residence/property for an alternative location—were not independent and irrelevant alternatives (IIA), which is an assumption of analytical strategy that we employ in the analysis that follows (multinomial logistic regression). Collapsing the responses into this set of three choices satisfies this assumption. When measured in this way, 9% of the 2012 respondents indicated that they would do nothing, 67% said that they would shelter in place, and 24% said that they would leave their residences for what they perceive to be a safer location. In 2013, the distribution of responses was 11%, 66%, and 23%, respectively.5
Note that both variables exclude the respondents who indicated that they would do “something else (please specify)” in response to the hypothetical warning. In 2012, 3.0% (n = 121) of respondents selected this option; in 2013, 2.8% (n = 112) selected this option. These responses were excluded because the number of respondents who specified any one action after selecting this option (i.e., gather more information) was too small to permit reliable quantitative analysis.
Analytical strategy and hypotheses.
To assess the relationship between consequence-based messages and warning response, we estimate a set of three models—one validation and two test models—using the data collected in 2012. Then, we replicate these models using the data collected in 2013 to ensure that our estimates and findings are reliable. In total, this procedure yields six models—two validation and four test models.
The validation models leverage existing research to assess the criterion validity of our dependent variable by exploring the relationship between intended response and a set of variables shown to influence actual response in previous research [see Brotzge and Donner (2013) for an excellent review of this research]. For example, previous research has shown that heightened risk perceptions (Kalkstein and Sheridan 2007), knowledge of the hazard (Sorensen 2000), and having a plan (Balluz et al. 2000; Nagele and Trainor 2012) increase the probability that an individual will take protective action when a warning is issued; being male (Comstock and Mallonee 2005; Bateman and Edwards 2002) and possessing fatalistic values (Sims and Baumann 1972) by comparison decrease the probability. If our measure of intended response provides a valid proxy for the actual response, then we will observe a similar pattern in the way that our subjects responded to the scenario. Such a finding would corroborate the criterion validity of our measure and, subsequently, the validity of our test models.
To estimate the validation models, we use logistic regression to predict the warning response based on the abovementioned variables. Specifically, we regress our binary indicator of protective action on perceived risk from tornadoes, knowledge about tornadoes, preparation for tornadoes, gender, and fatalism, which are measured using the following items:
Perceived risk, Q29: Considering both the likelihood and potential consequences of tornadoes throughout the year and using a scale from 0 to 10, where 0 means no risk and 10 means extreme risk, how do you rate the overall risk to you and your family from tornadoes?
Knowledge (additive scale based on the number of correct responses): Please indicate to the best of your knowledge whether each of the following statements about tornadoes is true or false.6
Q38: Tall buildings protect large cities from tornadoes (correct answer: false).
Q39: Mountains, rivers, and lakes do not protect nearby areas from tornadoes (correct answer: true).
Q41: When driving, you should not take shelter from tornadoes under a bridge or overpass (correct answer: true).
Q42: When sheltering in a house from a tornado, you should open all the windows to equalize the pressure inside and outside to prevent the house from exploding (correct answer: false).
Preparation: Which of the following do you currently have available at your residence?
Q71: A disaster response plan for you and your family (yes = 1).
Gender, Q13: Are you male or female (male = 1)?
Fatalism: Please rate the degree to which each of the following four groups of statements describes your outlook on life, using a scale from 0 to 10, where 0 means not at all and 10 means completely.
Q17: Life is unpredictable and I have little control. I have to live by lots of rules, but I don’t get to make them. My fate in life is determined mostly by chance. I cannot become a member of the groups that make most of the important decisions affecting me. Getting along in life is largely a matter of doing the best I can with what comes my way, so I focus on taking care of myself and the people closest to me.
After assessing the validation models, we estimate a set of test models that assess the relationship between consequence-based messages and warning response. To conduct the first test, we add our categorical indicator of projected impact to the validity models described above.7 If consequence-based messages influence the warning response, then we will observe a positive and statistically significant relationship between the impact level that respondents were assigned and protective action behavior. Respondents that were assigned to higher-impact categories will be more likely to choose protective action than respondents who were assigned to lower-impact categories.
To conduct the second test, we use multinomial logistic regression to account for the possibility that consequence-based messages provoke different types of protective action behavior.8 Respondents that were assigned to higher-impact categories, for example, may be more inclined to leave their residences in search of better shelter than respondents who were assigned to lower-impact categories who may be comfortable sheltering in place. To allow for this possibility, we reestimate the first test models using multinomial logistic regression to predict our respondents’ decision to do nothing, shelter in place, or leave their residences for a safer location.
FINDINGS.
The first two columns of Table 1 present the estimates derived from our validation models. Consistent with previous research, the coefficients in both samples indicate that respondents who perceive that tornadoes pose a high (10) risk to them or their family were a bit more likely to opt for protective action than respondents who have a low (0) risk perception (Δp = 0.029). The same is true with respect to knowledge and prestorm preparation. All else equal, respondents who know more about tornadoes were slightly more likely to opt for protective action than respondents who are less knowledgeable (Δp = 0.02). Similarly, respondents who indicated that they have a response plan were more likely to select a protective action than respondents who indicated that they do not have such a plan (Δp = 0.03). The opposite is true for male and fatalistic respondents. Male respondents were less likely to choose a protective action than female respondents (Δp = −0.03). Likewise, respondents who scored high (10) on our indicator of fatalism were less likely to select a protective action option than respondents who scored low (0) on the same indicator (Δp = −0.05). These findings are consistent with previous research on the factors that influence actual behaviors in response to tornado warnings. We are therefore cautiously optimistic that our measure of intended response provides a valid proxy for the actual response and that our tests of the relationship between consequence-based messages and warning response will provide some insight into future behavior.
Binomial logistic regression predicting binary protective action behavior. Note that standard errors are in parentheses.
The third and fourth columns in Table 1 present the results derived from our first set of tests. Figure 3 illustrates these results. Both models indicate that respondents who were assigned to higher-impact categories were, on average, more likely to opt for protective action than respondents who were assigned to lower-impact categories. In the 2012 model, the predicted probability of protective action among respondents who were told to expect light tornadoes is 0.80 when the other parameters in the model are set to their mean value. This probability increases to roughly 0.90 among respondents who were assigned to the moderate category and 0.95 among respondents who were warned of significant tornadoes. It is interesting to note, however, that the difference in predicted probability of protective action between the groups is nonlinear. The difference between respondents who were assigned to the light, moderate, and significant groups is rather substantial, whereas the difference between the respondents who were assigned to the significant, severe, devastating, and incredible groups is negligible.
Predicted influence of projected impact on binary protective action behavior; the predictions were derived from the (a) 2012 and (b) 2013 test models summarized in Table 1.
Citation: Bulletin of the American Meteorological Society 96, 4; 10.1175/BAMS-D-13-00213.1
The 2013 replication model reveals a strikingly similar pattern, which suggests that these findings are robust and reliable.10 The predicted probability of protective action among respondents who were told to expect light tornadoes is 0.75 when the other parameters in the model are held at mean value. This probability increases to almost 0.90 when the warning is moderate and 0.95 when the warning mentions the possibility of significant tornadoes. Again, however, the probability of protective action plateaus among respondents assigned to the severe, devastating, and incredible categories.
As a pair, these models suggest that when tornado warnings include information about the potential consequences of a storm, increasing the projected impact of a warned storm increases the probability that people will take some sort of protective action. At some point, however, there appears to be a threshold beyond which escalating the projected impact of a storm no longer increases the probability of protective action. Thus, it may be that escalations beyond this threshold are inconsequential and have no influence on warning response. Alternatively, it may be that these escalations are significant, but that their influence is masked by indiscriminately measuring protective action behavior. Perhaps higher-impact categories prompt different response behaviors. To account for this possibility, the next set of models discriminate between two types of protective action—sheltering in place and leaving the residence in search of a location.
Table 2 presents the results derived from our second set of test models. Do nothing was the reference category in both models, so the parameter estimates listed in Table 2 represent the shifts in the log odds of doing nothing versus sheltering in place and doing nothing versus leaving the residence associated with one unit increases in the predictor variables. According to the 2012 model, for example, a shift from light to moderate in the projected impact of the hypothetical storm prompted a 0.799 factor increase in the odds that our respondents selected shelter in place versus do nothing and a 1.312 factor increase in the odds that they chose to leave the residence versus do nothing. In the 2013 replication model, the pattern was similar; the same shift in projected impact produced a 0.909 factor increase in the odds of sheltering in place versus do nothing and a 1.577 factor increase in the odds of leaving the residence versus do nothing.11
Multinomial logistic regression predicting type of protective action behavior. Note that standard errors are in parentheses.
We used the estimates derived from these models to calculate the predicted probability of each type of response conditional on the impact groups to which our respondents were assigned. Figure 4 illustrates these results and, in so doing, reveals an interesting pattern.
Predicted influence of projected impact on type of protective action behavior; the predictions were derived from the (a) 2012 and (b) 2013 models summarized in Table 2.
Citation: Bulletin of the American Meteorological Society 96, 4; 10.1175/BAMS-D-13-00213.1
At lower levels of projected impact, the relationship between impact and intended response is positive and monotonic. Increasing the projected impact of the storm—from light to moderate and moderate to significant—simultaneously increased the probability that respondents in both samples selected a shelter in place or leave residence option. At higher levels of projected impact, this relationship changes. Increasing the projected impact of the storm—from significant to severe, severe to devastating, and devastating to incredible—systematically decreased the probability that respondents said that they would shelter in place and significantly increased the probability that respondents would leave their residences for what they perceive to be a safer location.
DISCUSSION AND IMPLICATIONS.
Discussion of findings.
Returning now to the question that motivated this study: do consequence-based messages influence public responsiveness to tornado warnings? Yes, when warnings include information about the potential consequences of a storm, increasing the projected impact of a warned storm increased the probability that our respondents selected some sort of protective action. At some point, however, there appears to be a threshold beyond which escalating the projected impact of a storm no longer significantly increases the net probability of protective action. To explain this phenomenon, we showed that the relationship between consequence-based messaging and protective action depends (in part) on the type of action being considered. At lower levels of projected impact, increases in the projected consequences of the storm simultaneously increased the probability that respondents selected a shelter in place or leave residence option. At higher levels of projected impact, this relationship changes—increases in the projected consequences of the storm prompt decreases in the probability that respondents would shelter in place and increases in the probability that respondents would leave their residences for a safer location. These findings suggest that high-consequence language embedded in severe storm messages may cause individuals to question the level of safety that their residences can provide and, as a result, induce them to flee in search of what they perceive to be a safer location. In some situations, this may increase rather than decrease the risks associated with convective storms (Glass et al. 1980).
This explanation is consistent with the situation depicted in Fig. 5, which shows a radar image of a convective storm approaching the Oklahoma City metropolitan area and one of many traffic jams that occurred near the path of the storm. If our findings are correct, then repeated messages from local broadcasters about the potentially catastrophic consequences of the approaching storm likely prompted some residents to flee their homes to escape the anticipated danger associated with the inbound storm. Widespread decisions to flee, in turn, generated traffic jams throughout the city that exposed stranded motorists to greater risks than they would have faced had they chosen to shelter in place. The implications are particularly stark for those people who left permanent structures only to get stuck in a stationary vehicle as the storm passed. Though estimates differ, most researchers agree that the risk of death or injury from a tornado is significantly higher in stationary vehicles than permanent structures, like single-family homes (Hammer and Schmidlin 2000; Schmidlin et al. 2002; Wurman et al. 2007; Brooks et al. 2008).
Radar image from RadarScope depicting the “velocity couplet” associated with a tornado that occurred in the Oklahoma City, OK, metro on 31 May 2013, and a photo of a traffic jam that occurred in the city at approximately the same time. The red colors in the radar image indicate motion toward the radar the green colors indicate motion away from the radar; the circular symbols on the radar image indicate the geographic location of local storm spotters. [Source: Star Tribune, “On Weather with Paul Douglas” (www.startribune.com/blogs/209882771.html).]
Citation: Bulletin of the American Meteorological Society 96, 4; 10.1175/BAMS-D-13-00213.1
Implications for the IBW experiment.
The IBW experiment is predicated on the logic that using high-consequence language in tornado warnings will save lives by increasing the probability that people will take protective action in response to particularly dangerous storms. Our research provides empirical evidence that, up to a point, this language can have an important and desirable effect. Vulnerable residents that are told to expect a high-consequence event may be more likely to take protective action than respondents who are told to expect a lower-consequence event. Beyond this point, however, extreme warning language may be counterproductive; efforts to induce exposed residents to respond can lead to actions that increase rather than decrease the risk of death or injury.
General implications notwithstanding, it is important to reiterate that this study was designed to assess the theoretical effect of consequence-based messages on public responsiveness to tornado warnings. It was not designed as a direct evaluation of the IBW experiment. Among other things, the consequence-based messages we employ in this study are different from the messages used in IBWs. For instance, we use the words light and moderate to characterize the impact tornadoes that are expected to cause relatively little damage, whereas IBWs refrain from using threat tags in these situations. At the higher end of the spectrum, we use the words significant, severe, devastating, and incredible to describe the consequences of tornadoes that are expected to produce substantial damage, whereas IBWs use one of two threat tags, “considerable” or “catastrophic,” to demarcate the elevated risk associated with high-consequence events. Thus, a direct evaluation of the IBW experiment will require different experimental conditions than the ones employed in this study. At minimum, it will require a baseline “control” condition, where respondents are given no information about the potential impact of the storm; an elevated condition, where respondents are told to expect considerable damage; and an extreme condition, where respondents are told to expect catastrophic damage.
Directions for future research.
In addition to a direct evaluation of the IBW experiment, more research is required to fully understand the threshold beyond which consequence-based messages become counterproductive. Likewise, more research is needed to determine the social and cognitive mechanisms that mediate or moderate the relationship between consequence-based messages and action and the type of people who are more (or less) likely to respond to messages of this sort. As research moves in this direction, extant research on risk perception and risk communication may prove useful (i.e., Kasperson et al. 1988; Slovic 2000). This research may provide some insight into the mechanisms that amplify or attenuate public perceptions of risk from tornadoes, and consequently, the effect that these perceptions have on protective action behavior.
As scholars begin to identify these mechanisms, we encourage explicit analysis of positive and negative feedback processes generated by tornado warnings that include information about potential consequences. How, for example, do people assess the relative accuracy or inaccuracy of warnings that contain this sort of information? What if a low-consequence tornado occurs when a high-consequence tornado is predicted? Is this perceived to be a false alarm? What if a high-consequence tornado occurs when a low-consequence tornado is forecast? Is this perceived to be a missed event? More importantly, how might these assessments influence public trust in the NWS and, consequently, the likelihood that people will respond when future warnings are issued? Researchers that are interested in answering these questions might look to other research on the influence of warning system accuracy on warning response for inspiration (i.e., Breznitz 1984; Barnes et al. 2007; Simmons and Sutter 2009, 2011; Ripberger et al. 2015).
Last, but not least, it is important to remember that our findings are based on intended responses to a hypothetical warning. Though our validation model increases our confidence in the generalizability of our results, we recognize the need for future research on the extent to which consequence-based messages influence actual responses to real tornado warnings. Again, we encourage scholars interested in this relationship to begin their pursuit by looking at research in other disciplines, like the work that psychologists have done on the theory of reasoned action and the theory of planned behavior (i.e., Fishbein and Ajzen 1975; Ajzen 1985, 1991). In addition to empirically modeling the relationship between behavioral intentions and actions, this research attempts to specify the barriers that prevent people from translating their intentions into actions, which may provide additional insight into the contextual and cognitive factors that influence protective action behavior in response to real tornado warnings that contain information about the potential consequences of the storm.
Better understood and fielded, we believe that consequence-based messages offer significant promise in saving lives—but first we must understand how and when to use them in ways that avoid compounding the risks posed by severe weather events.
ACKNOWLEDGMENTS
Partial funding for this project was provided by NOAA/Office of Oceanic and Atmospheric Research under NOAA-OU Cooperative Agreement NA11OAR4320072, U.S. Department of Commerce, and the OU Office of the Vice President for Research. The statements, findings, conclusions, and recommendations are those of the authors and do not necessarily reflect the views of NOAA, the U.S. Department of Commerce, or the University of Oklahoma.
REFERENCES
Ajzen, I., 1985: From intentions to actions: A theory of planned behavior. Action Control: From Cognition to Behavior, J. Kuhl and J. Beckmann, Eds., Springer Series in Social Psychology, Vol. 12, Springer-Verlag, 11–39.
Ajzen, I., 1991: The theory of planned behavior. Organ. Behav. Hum. Decis. Processes, 50, 179–211, doi:10.1016/0749-5978(91)90020-T.
Armor, D. A., and A. M. Sackett, 2006: Accuracy, error, and bias in predictions for real versus hypothetical events. J. Pers. Soc. Psychol., 91, 583–600, doi:10.1037/0022-3514.91.4.583.
Ashley, W. S., 2007: Spatial and temporal analysis of tornado fatalities in the United States: 1880–2005. Wea. Forecasting, 22, 1214–1228, doi:10.1175/2007WAF2007004.1.
Baker, E. J., 1995: Public response to hurricane probability forecasts. Prof. Geogr., 47, 137–147, doi:10.1111/j.0033-0124.1995.00137.x.
Balluz, L., L. Schieve, T. Holmes, S. Kiezak, and J. Malilay, 2000: Predictors for people’s response to a tornado warning: Arkansas, 1 March 1997. Disasters, 24, 71–77, doi:10.1111/1467-7717.00132.
Barnes, L. R., E. C. Gruntfest, M. H. Hayden, D. M. Schultz, and C. Benight, 2007: False alarms and close calls: A conceptual model of warning accuracy. Wea. Forecasting, 22, 1140–1147, doi:10.1175/WAF1031.1.
Bateman, J. M., and B. Edwards, 2002: Gender and evacuation: A closer look at why women are more likely to evacuate for hurricanes. Nat. Hazards Rev., 3, 107–117, doi:10.1061/(ASCE)1527-6988(2002)3:3(107).
Breznitz, S., 1984: Cry Wolf: The Psychology of False Alarms. Lawrence Erlbaum Associates, 265 pp.
Brooks, H. E., C. A. Doswell III, and D. Sutter, 2008: Low-level winds in tornadoes and potential catastrophic tornado impacts in urban areas. Bull. Amer. Meteor. Soc., 89, 87–90, doi:10.1175/BAMS-89-1-87.
Brotzge, J., and W. Donner, 2013: The tornado warning process: A review of current research, challenges, and opportunities. Bull. Amer. Meteor. Soc., 94, 1715–1733, doi:10.1175/BAMS-D-12-00147.1.
Central Region Headquarters, cited 2014: Impact based warnings. National Weather Service Regional Office Central Region Headquarters. [Available online at http://www.crh.noaa.gov/crh/?Fn=2013_ibw_info.]
Comstock, R. D., and S. Mallonee, 2005: Comparing reactions to two severe tornadoes in one Oklahoma community. Disasters, 29, 277–287, doi:10.1111/j.0361-3666.2005.00291.x.
Couper, M. P., 2000: Web surveys: A review of issues and approaches. Public Opin. Quart., 64, 464–494, doi:10.1086/318641.
Dow, K., and S. L. Cutter, 1998: Crying wolf: Repeat responses to hurricane evacuation orders. Coastal Manage., 26, 237–252, doi:10.1080/08920759809362356.
Dow, K., and S. L. Cutter, 2000: Public orders and personal opinions: Household strategies for hurricane risk assessment. Global Environ. Change, 2B, 143–155, doi:10.1016/S1464-2867(01)00014-6.
Fishbein, M., and I. Ajzen, 1975: Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research. Addison-Wesley, 578 pp.
Glass, R. I., R. B. Craven, D. J. Bregman, B. J. Stoll, N. Horowitz, P. Kerndt, and J. Winkle, 1980: Injuries from the Wichita Falls tornado: Implications for prevention. Science, 207, 734–738, doi:10.1126/science.207.4432.734.
Goebbert, K., H. Jenkins-Smith, K. Klockow, M. Nowlin, and C. Silva, 2012: Weather, climate and worldviews: The sources and consequences of public perceptions of changes in local weather patterns. Wea. Climate Soc., 4, 132–144, doi:10.1175/WCAS-D-11-00044.1.
Hammer, B. O., and T. W. Schmidlin, 2000: Vehicle-occupant deaths caused by tornadoes in the United States, 1900–1998. Global Environ. Change, 2B, 105–118, doi:10.1016/S1464-2867(01)00005-5.
Kalkstein, A. J., and S. C. Sheridan, 2007: The social impacts of the heat–health watch/warning system in Phoenix, Arizona: Assessing the perceived risk and response of the public. Int. J. Biometeor., 52, 43–55, doi:10.1007/s00484-006-0073-4.
Kasperson, R. E., O. Renn, P. Slovic, H. S. Brown, J. Emel, R. Goble, J. X. Kasperson, and S. Ratick, 1988: The social amplification of risk: A conceptual framework. Risk Anal., 8, 177–187, doi:10.1111/j.1539-6924.1988.tb01168.x.
Kuligowski, E. D., F. T. Lombardo, L. T. Phan, M. L. Levitan, and D. P. Jorgensen, 2014: Draft final report, National Institute of Standards and Technology (NIST) technical investigation of the May 22, 2011, tornado in Joplin, Missouri. Tech Rep. NIST NCSTAR 3, 428 pp. [Available online at http://www.nist.gov/manuscript-publication-search.cfm?pub_id=914787.]
McLuckie, B. F., 1974: Warning—A call to action: Warning and disaster response, a sociological background. DOC/NOAA/NWS Southern Region Headquarters Rep., 94 pp. [Available online at www.ofcm.gov/wg-ssr/noaa-nws-sr-a-call-to-action.pdf.]
Morss, R. E., and M. H. Hayden, 2010: Storm surge and “certain death”: Interviews with Texas coastal residents following Hurricane Ike. Wea. Climate Soc., 2, 174–189, doi:10.1175/2010WCAS1041.1.
Nagele, D. E., and J. E. Trainor, 2012: Geographic specificity, tornadoes, and protective action. Wea. Climate Soc., 4, 145–155, doi:10.1175/WCAS-D-11-00047.1.
NOAA, 2011a: NWS central region service assessment: Joplin, Missouri, tornado, May 22, 2011. DOC/NOAA/NWS Rep., 41 pp. [Available online at www.nws.noaa.gov/om/assessments/pdfs/Joplin_tornado.pdf.]
NOAA, 2011b: The historic tornadoes of April 2011. DOC/NOAA/NWS Rep., 76 pp. [Available online at www.nws.noaa.gov/om/assessments/pdfs/historic_tornadoes.pdf.]
NWS, 2010: National Weather Service instruction 10-1701: Text product formats and codes. Operations and Services, Dissemination Services NWSPD 10-17, 61 pp. [Available online at www.nws.noaa.gov/directives/sym/pd01017001curr.pdf.]
Peacock, W. G., and H. Gladwin, 1993: Assessing the likelihood of evacuation next time: Some preliminary findings from the FIU Hurricane Andrew survey. Prelim. Rep. 3, Department of Sociology and Anthropology, Florida International University, Miami, Florida, 16 pp.
Ripberger, J. T., C. L. Silva, H. C. Jenkins-Smith, D. E. Carlson, M. James, and K. G. Herron, 2015: False alarms and missed events: The impact and origins of perceived inaccuracy in tornado warning systems. Risk Anal., 35, 44–56, doi:10.1111/risa.12262.
Schmidlin, T., B. Hammer, P. King, Y. Ono, L. S. Miller, and G. Thumann, 2002: Unsafe at any (wind) speed?: Testing the stability of motor vehicles in severe winds. Bull. Amer. Meteor. Soc., 83, 1821–1830, doi:10.1175/BAMS-83-12-1821.
Schultz, D. M., E. C. Gruntfest, M. H. Hayden, C. C. Benight, S. Drobot, and L. R. Barnes, 2010: Decision making by Austin, Texas, residents in hypothetical tornado scenarios. Wea. Climate Soc., 2, 249–254, doi:10.1175/2010WCAS1067.1.
Simmons, K. M., and D. Sutter, 2009: False alarms, tornado warnings, and tornado casualties. Wea. Climate Soc., 1, 38–53, doi:10.1175/2009WCAS1005.1.
Simmons, K. M., and D. Sutter, 2011: Economic and Societal Impacts of Tornadoes. American Meteorological Society, 282 pp.
Simmons, K. M., and D. Sutter, 2012: The 2011 tornadoes and the future of tornado research. Bull. Amer. Meteor. Soc., 93, 959–961, doi:10.1175/BAMS-D-11-00126.1.
Sims, J. H., and D. D. Baumann, 1972: The tornado threat: Coping styles of the north and south. Science, 176, 1386–1392, doi:10.1126/science.176.4042.1386.
Slovic, P., 2000: The Perception of Risk. Earthscan, 473 pp.
Sorensen, J. H., 2000: Hazard warning systems: Review of 20 years of progress. Nat. Hazards Rev., 1, 119–125, doi:10.1061/(ASCE)1527-6988(2000)1:2(119).
Stensrud, D. J., and Coauthors, 2009: Convective-scale warn-on-forecast system. Bull. Amer. Meteor. Soc., 90, 1487–1499, doi:10.1175/2009BAMS2795.1.
Sutter, D., and S. Erickson, 2010: The time cost of tornado warnings and the savings with storm-based warnings. Wea. Climate Soc., 2, 103–112, doi:10.1175/2009WCAS1011.1.
Troutman, T. W., R. Smith, and M. A. Rose, 2001: Situation specific call-to-action statements. NOAA/NWS Rep. [Available online at www.srh.noaa.gov/ssd/techmemo/sr215.htm.]
Whitehead, J. C., B. Edwards, M. Van Willigen, J. R. Maiolo, K. Wilson, and K. T. Smith, 2000: Heading for higher ground: Factors affecting real and hypothetical hurricane evacuation behavior. Global Environ. Change, 2B, 133–142, doi:10.1016/S1464-2867(01)00013-4.
Wurman, J., P. Robinson, C. Alexander, and Y. Richardson, 2007: Low-level winds in tornadoes and potential catastrophic tornado impacts in urban areas. Bull. Amer. Meteor. Soc., 88, 31–46, doi:10.1175/BAMS-88-1-31.
As noted by Simmons and Sutter (2012), these calculations may change if the convective-scale warn-on-forecast system is adopted by the NWS (Stensrud et al. 2009).
For more in this line of research, see Goebbert et al. (2012).
At the time of our sample the SurveySpot panel consisted of approximately two million households with about five million household members. In addition to this large panel, SSI maintains a subpanel of approximately 400,000 members whose demographics (e.g., race, gender, and education) are roughly proportionate to national census characteristics. Our sample was randomly drawn from the 400,000 census-balanced subpanel.
Though the differences were quite small, two sample tests of proportion indicate that the differences between the 2012 and 2013 samples are statistically significant (p < 0.05).
Two sample tests of proportion indicate that the differences between the do nothing category in the 2012 and 2013 (9% vs 11%) are statistically significant (p < 0.05); differences between the shelter in place and leave residence categories are not statistically significant.
This set of “knowledge” questions was derived from a NWS list of the “Most Common Tornado Myths” (available online at www.crh.noaa.gov/mkx/?n=taw-part2-tornado_myths.)
Dummy regression models with interaction terms were estimated to explore the possibility that the effect of consequence-based messages on protective action behavior varies according to parental status (children vs no children), housing type (mobile home vs nonmobile home), gender, region, age, previous hazard experience, and/or the availability of an on-site shelter or safe room. There were no statistically significant differences between these groups that were consistent across the two samples (2012 and 2013), suggesting that the influence of consequence-based messages on protective action behavior is somewhat insensitive to group differences.
Dummy regression models with interaction terms were estimated to explore the possibility that the relationship between projected impact and type of protective action behavior varies according to parental status, housing type, gender, region, age, previous hazard experience, and/or the availability of an on-site shelter or safe room. There were no statistically significant differences between these groups that were consistent across the two samples (2012 and 2013), suggesting that the influence of consequence-based messages on the type of protective action that people engage in is somewhat insensitive to group differences.
The term Δp denotes the average difference in the predicted probability of protective action between the two groups while holding the other parameters in the models at their mean values.
Dummy regression models with interactions confirm that the coefficients associated with projected impact are statistically indistinguishable across the two samples.
Dummy regression models with interactions confirm that the coefficients associated with projected impact are statistically indistinguishable across the two samples.
