Evacuation before severe coastal storms is a critical tool for keeping coastal residents safe. Effective messaging of evacuations could help save lives, but there is little evidence-based guidance on the advantages or disadvantages of specific messaging. Ideally, evacuation messages would convince those most at risk to evacuate and those who do not need to evacuate to stay in their homes. Using an online survey of 1716 coastal residents in Connecticut, New Jersey, and New York, this study randomly assigned respondents to message conditions in each of two hypothetical storm scenarios. Results from the first scenario indicate that those who saw mandatory evacuation messages had higher evacuation intentions than those who saw advisory messages, and both of those messages resulted in slightly higher evacuation intentions than voluntary evacuation messages. However, voluntary messages resulted in lower evacuation intentions for those that did not live in evacuation zones compared to those who did live in evacuation zones, which may help reduce shadow evacuation. In the second scenario, identifying an evacuation area by the municipality name or the individual’s street name resulted in similar evacuation intentions across all participants. Messages identifying an evacuation area by “flood zone” or “flood-prone area” resulted in equally high evacuation intentions for those who believe they live in a flood zone, but these messages suppressed evacuation intentions for those who do not believe they live in a flood zone. This indicates that such messages could also be an effective approach for reducing shadow evacuation. Implications for risk communicators and emergency managers are discussed.
Evacuation is a critical emergency management tool that can reduce the loss of life in a hazardous weather event (Brunkard et al. 2008). However, it is often difficult to get people to evacuate their homes in advance of an adverse weather event such as a hurricane or coastal storm. For example, two recent studies of New Jersey (NJ) residents who reported living in mandatory evacuation areas in the days prior to Hurricane Sandy found that less than half, and perhaps as few as 28%, evacuated prior to the storm (Abramson et al. 2015; Monmouth University Polling Institute 2013).
While it can be extremely difficult to get some people who are at high risk to evacuate for a hazardous weather threat, risk managers also face the parallel problem of overresponse or “shadow evacuations” (Dash and Gladwin 2007; Zhang et al. 2007; Lamb et al. 2012; Huang et al. 2012; Weinisch and Brueckner 2015). Shadow evacuations refer to when people at relatively low risk (i.e., because they live outside the evacuation zones) decide to evacuate. This can cause additional traffic and stress on shelters, sometimes making it more difficult for those who are at high risk to leave their homes. By choosing to leave their homes, “shadow evacuators” can also place themselves at higher risk.
Decisions to evacuate as a hurricane approaches are complex and influenced by many factors (Hasan et al. 2011; Lazo et al. 2015; Lindell and Perry 2012; Morss et al. 2015; Whitehead et al. 2000). One influential factor is the information that people receive related to the approaching hurricane, including evacuation notices and other risk messages (Gladwin et al. 2001; Burnside et al. 2007; Dow and Cutter 1998; Morss and Hayden 2010). In addition, unlike many other factors that influence evacuation decisions, those issuing risk messages often have control over which messages are developed, how they are worded, and to which residents they are disseminated.
Evacuation messages are often provided in inconsistent ways across municipalities. A recent winter coastal storm forecasted to affect NJ shore towns provides a clear example of how neighboring municipalities can send very different messages in advance of a storm (Kopp et al. 2016). Figure 1 illustrates that although geographically proximate municipalities faced a similar storm threat, the information provided in the evacuation notices varied in terms of content and wording. One town in Ocean County issued a “mandatory” evacuation notice, while others in the same county issued “voluntary” evacuation notices, and other towns “encouraged” residents to evacuate. In addition to variation in the evacuation notice level, there are a number of different of ways of describing the geographic locations to be evacuated, ranging from the entire municipality (e.g., city or township), low-lying areas, and flood zones, as well as identifying the evacuation area in terms of border streets and listing the names of specific streets to be evacuated.
The variation in these evacuation notices leads directly to our research questions, which investigate the effectiveness of different evacuation messages with the goal of encouraging the highest possible levels of evacuation from high-risk areas while maintaining relatively low evacuation rates in lower-risk areas. Using data from an online survey in which 1716 coastal residents of Connecticut (CT), NJ, and New York (NY) were presented with messages about hypothetical storm scenarios, we examine the following research questions (RQs):
Research question 1: How does the evacuation notice level (voluntary evacuation, mandatory evacuation, evacuation advisory, evacuation advised, evacuation strongly recommended) affect evacuation intentions?
Research question 2: How does specific geographic or location-based information about evacuation areas affect evacuation intentions?
Research question 3: How do these types of messages affect shadow evacuation?
Each of these research questions, and related scientific literature, is discussed in turn, followed by the study methodology and results.
2. Background literature and research hypotheses
a. Effects of evacuation notice levels: Mandatory, voluntary, advisory, and recommended (research question 1)
Studies of hurricane evacuation decision making demonstrate that for many people one of the primary factors influencing prelandfall evacuation is the perception that they are under an evacuation order (Baker 1991; Whitehead et al. 2000; Gladwin et al. 2001; Dash and Gladwin 2007). For example, recent research indicates that during Hurricane Sandy, those people who believed that they were under a mandatory evacuation order were 24 times more likely to evacuate than those who believed they were under a voluntary evacuation (Daziano 2015). Although the effect was large in the Daziano study, the investigation involved cross-sectional research that relied on participants’ memories of what type of evacuation notice they had received. Therefore, further study is needed using other research designs to better understand the effects of different types of evacuation notices. Building on related work, we hypothesize that recipients of messages mentioning mandatory evacuations would have higher evacuation intentions than those receiving nonmandatory evacuation messages.
Despite its potential effectiveness, many local officials do not make evacuation notices mandatory. Often it is because they do not have the desire and/or resources to forcibly remove people from their homes (Dow and Cutter 2000; Fairchild et al. 2006). If there is not a state-level mandatory evacuation order in effect, that municipality will be fully responsible for the costs of a mandatory evacuation (Fairchild et al. 2006). However, as in the example of the recent January 2016 storm (Fig. 1), municipalities do sometimes issue mandatory evacuations in the absence of a federal- or state-level order. Another reason that a mandatory evacuation order may not be issued is that there are evacuation situations in which a storm is not expected to be so severe that it would be appropriate for all residents in an evacuation zone to leave; instead, a nonmandatory evacuation would indicate that an evacuation notice is for those residents who would be at particular risk should they stay, such as the medically frail (Wolshon et al. 2005). Finally, there is also a fear of evacuating people unnecessarily, which may reduce people’s trust in subsequent evacuation messages. Given the uncertainty of storm predictions, emergency managers are concerned about the “crying wolf” phenomenon (Dow and Cutter 1998; Barnes et al. 2007; Morss et al. 2015). So while it is important to investigate the effectiveness of these various evacuation notices, it is also important to recognize that a mandatory evacuation is not appropriate for every evacuation scenario.
b. Effects of location-based messages: Flood zones, street names, and townwide evacuation notices (research question 2)
Variation in how evacuation notices describe the at-risk geographic area (as in Fig. 1) can also affect evacuation behavior. Figure 1 provides examples of notices indicating evacuation for (a) flood-prone areas, (b) residents of a flood zone, (c) specific streets, and (d) the municipality as a whole. Research suggests that many people cannot correctly identify whether they live in a flood or hurricane evacuation zone. For example, a recent study found that 70% of Connecticut evacuation zone residents say that they either do not live or do not know if they live in an evacuation zone (Marlon et al. 2015). Similarly, Zhang et al. (2004) found that one-third of participants living along the coast of Texas incorrectly identified their own flooding risk area. This may be in part because residents do not encounter this information, but there is evidence that even when that information is presented, it can be difficult to interpret. For example, in another study Arlikatti et al. (2006) report that only a third (36%) of Texan coastal residents were able to correctly identify their residence’s risk area using hurricane risk area maps, and an additional 28% were off by one risk category area (e.g., if they were in risk category/zone 1, they believed they were in risk category/zone 2).
Given that residents may not know whether their home is in a flood zone or at-risk area, we hypothesize that more geographically specific messages, such as those that identify that a resident’s own street is included in an evacuation, will increase the likelihood of evacuation. This may be simply because all coastal residents know their street name, but also because localized risk messages can help receivers personalize the risk and make clear that the message applies to them (Mileti and O’Brien 1992). However, for messages using terms such as flood zone and flood-prone area, we hypothesize that the effectiveness of the message will depend on an individual’s belief about whether or not they live in the affected area.
The manner of communicating geographic locations to evacuate is particularly salient as there have been significant developments in both the communications technologies used by emergency managers and those commonly used by residents that permit communications to be more geographically targeted than in the past. In fact, emergency text messaging systems allow emergency managers and other government officials to send tailored evacuation messages to their residents, including those tailored based on location (Cuite et al. 2015; Strawderman et al. 2012). These include reverse notification systems, such as Reverse 911, which rely on databases of the residences and phone numbers of local residents, as well as systems such as Nixle, which has the ability to target some subscribers based on their current locations. The rise of cell phones and smart phone technology means that many residents can receive these alerts, text messages, emails, and telephone calls leading up to and throughout many storms. Emergency managers report that these newer, more technological approaches are often used in conjunction with more traditional approaches, like door-to-door notices (Cuite et al. 2015).
c. Shadow evacuation (research question 3)
In addition to evaluating whether messages increase the likelihood of evacuation for those who are at highest risk and should evacuate, it is also important to examine whether certain messages may be more likely to increase (or suppress) “shadow” evacuations, among those who are at lower risk and have not been asked to evacuate. An example that highlights the significance of this is that during Hurricane Rita, 47% of those who did not reside in an evacuation zone reported evacuating (Stein et al. 2010). Such shadow evacuations can be a significant problem when limited resources, including roadways and shelters, are used by those who do not need them (Dash and Gladwin 2007; Peacock et al. 1997).
We explore the influence of location-based messages on shadow evacuation, and whether at least some shadow evacuations may be caused by residents simply not knowing if an evacuation notice applies to them. Specifically, in the case of an evacuation of a flood zone or a flood-prone area, correctly deciding whether an evacuation notice applies to oneself requires knowing if one lives in a flood zone or flood-prone area. People who incorrectly think they live in a flood zone but do not may be much more likely to evacuate than those residents who do not believe they live in a flood zone or do not know if they do, regardless of the actual location of their home (Lamb et al. 2012). Thus, we hypothesize that actual residence in a flood zone is less important than the perception of living in a flood zone or flood-prone area in determining whether or not someone evacuates when they receive an evacuation notice framed in terms of these types of areas.
Similarly, we explore the role of the evacuation notice levels in reducing shadow evacuation. We hypothesize that voluntary evacuation notices and evacuation advisories or recommendations may reduce unnecessary evacuation among those who believe they do not live in a flood zone. Again, we predict that it is the perception of where one lives rather than the actual location that will influence evacuation intentions.
With both message types examined here (i.e., notice level and location-based), a critical question is whether it is possible to reduce shadow evacuation intentions for those who should not evacuate, while simultaneously not lowering the evacuation intentions of those who should. For both message types, we therefore examine the interaction effects of the message variables with perceived residence in a flood zone as well as with actual residence in a flood zone. This allows us to look for differential effects of the messages on evacuation intentions, both among those who should evacuate (or at least believe they live in an area that should) and those who should not (or at least believe they live in an area that should not).
Data were collected via an online survey over the course of four weeks in April and May 2015. Participants took a median of 15 min to complete the survey. The protocol was approved by the Institutional Review Board of Rutgers University.
The sampling and online survey administration were conducted by GfK Custom Research. The sample of adult coastal residents in CT, NY, and NJ was recruited in two ways. The sample included 203 participants from the randomly selected KnowledgePanel (66.2% cooperation rate1), a large national panel that is recruited through random digit dialing of landline and cell phone numbers, with a sampling frame that covers approximately 97% of the population and includes those without computers and Internet (prior to joining the study). Because KnowledgePanel did not have sufficient sample in the targeted geographic areas, the majority of the sample, 1513 participants, was recruited from a nonprobability “opt-in” panel (8.8% cooperation rate). These respondents were recruited online, and were contacted via their patronage of GfK’s corporate partners. Respondents in both samples received financial incentives for their participation. Up to five e-mail reminders to complete the survey were sent to the KnowledgePanel participants, and up to eight were sent to the opt-in participants.
For both samples, participants were selected based on the ZIP code of their primary residence, the smallest geographic sampling unit available. ZIP codes were chosen based on criteria designed to maximize the number of potential participants who live in areas at risk for storm surge flooding.
In NJ and NY, respondents were sampled from ZIP codes where 40% or more of the landmass within the ZIP code area is located in the National Weather Service’s Sea, Lake and Overland Surges from Hurricanes (SLOSH) zone 2 (ArcGIS desktop: release 10.2), which indicates that residents are at risk for storm surge flooding in the event of a moderate to severe coastal storm (National Hurricane Center 2015; New York State 2016). However, because of the different topography in CT, relatively few ZIP codes meet this criterion (University of Connecticut 2008); therefore, the approach was modified to include any ZIP code in CT where over 1% of the landmass is located in a SLOSH zone 2. Using these criteria resulted in the inclusion of potential participants from 116 ZIP codes in NJ, 70 in NY, and 54 in CT, and a total sample of 567 (33%) participants in NJ, 698 (41%) in NY, and 451 (27%) in CT. The ZIP codes from which participants were selected are shown in Fig. 2.
After the survey was completed, we used GIS software to identify where participants actually live in terms of SLOSH inundation zones. Using their home addresses, we found that 40.6% of the participants reside in SLOSH zones 1 or 2, meaning that they could experience storm surge flooding as a result of moderate coastal storms, and 21% reside in SLOSH zones 3 or 4, meaning that they are likely to be affected by surge flooding only by stronger storms. The remaining 38.4% of the participants do not live in SLOSH zones, indicating that they are at least risk for experiencing surge flooding as the result of a storm and thus more likely to be potential shadow evacuators. Taken together, this distribution indicates that our SLOSH zone–based sampling strategy successfully resulted in the inclusion of participants with a range of storm surge flooding risk levels, some for whom evacuation notices would be relevant, and others much less so.
b. Experimental design
The data presented here come from an online survey that included four between-group experiments. Each respondent received all four experiments, presented in randomized order. Each experiment included a hypothetical scenario about a coastal storm; within each scenario, respondents were randomly assigned to different experimental conditions. In this article, results are presented from only two of the four experiments (i.e., from the experiments on evacuation notice level and on location-based messages). Results from the other two experiments will be examined in future work.
A series of questions about demographics, home address, and whether participants believed their current residence was located in “an officially designated flood zone, such as a 100-yr floodplain” were asked before the experimental scenarios. This was done to screen appropriate participants based on the sampling approach, to ensure that we had an accurate street address and municipality to use in the experimental manipulations, and to measure important covariates for the analyses.
The respondents were then presented with each of the four experiments, one at a time. Within each experimental scenario, respondents were randomly assigned to receive a message about an approaching coastal storm (discussed further below), followed by a series of questions measuring dependent variables, including evacuation and other behavioral response intentions, evacuation importance, likelihood of recommending evacuation to others, likelihood of harm from the storm, perceptions of storm severity and message relevance, emotional response, and message comprehension.
The order of the dependent-variable questions was randomized within each of the four scenarios. This paper examines message effects on only one dependent variable: participants’ evacuation intentions, specifically how likely they would be to evacuate their home after hearing the message, measured on a Likert-type scale with 1 “not at all likely” to 7 “extremely likely.” We focus on this measure because it is the most closely related to actual evacuation, the ultimate goal of many of these messages.
The evacuation notice and location-based experiments are described below. Although each experiment included multiple experimental manipulations embedded within a coastal storm message, this article focuses on the aspects of each experiment that address the three research questions discussed above.
1) Experiment on evacuation notice level
For the experiment described in this section, we focus on the effects of manipulating the level of evacuation notice provided to respondents. This factor had five conditions: “mandatory evacuation order” (n = 334), “voluntary evacuation order” (n = 324), “evacuation advisory” (n = 351), or that emergency managers “strongly recommend” (n = 356) or “advise” (n = 351) that residents evacuate. Table 1 presents the wording of the stimuli used in this experiment.
Two additional factors were manipulated in this experiment but are not discussed at length here. We manipulated the channel through which the message was delivered; participants were asked to imagine that they received a message through one of five channels2 (emergency text, n = 271; social media, n = 259; phone call, n = 373; flyer, n = 390; or in person, n = 423). The presentation of the text was exactly the same in each of these conditions; respondents were simply asked to imagine the different delivery modes. These factors were fully crossed with an additional factor, storm type, which varied whether the storm was called a “hurricane” (n = 869) or a “nor’easter” (n = 847). All of the data presented here are collapsed across channel and storm type. There was no consistent effect of channel across experimental scenarios. The effects of storm type will be presented elsewhere.
2) Experiment on location-based messages
As in the prior experiment, this experiment included several message manipulations, but we focus on only one here—the location-based message received. This factor had four levels: municipality name (n = 424), street name (n = 428), flood zone (n = 436), and flood-prone area (n = 428). Stimuli wording is presented in Table 1.
Once again, participants were asked to imagine that they were receiving a message through one of five specific communication channels (emergency text, n = 276; social media, n = 247; phone call, n = 419; flyer, n = 418; or in person, n = 356). The third message variable was the inclusion (or not) of a description of the effects of storm surge (description, n = 862; no description, n = 854). The results reported here are aggregated across the channels and the surge description manipulation.
Each of the three research questions is considered separately. We conducted an ANOVA for research question 1 to examine the effects of level of evacuation notice on evacuation intention. For research question 2, an ANOVA was conducted to explore the effect of including location-based descriptions of the storm-affected area in evacuation notices. Because they have previously been found to be important predictors of evacuation, demographic variables (gender, age, education level, income, and ethnicity) were included as controls within both models (e.g., Whitehead et al. 2000; Smith and McCarty 2009; Huang et al. 2015; Lazo et al. 2015).
For research question 3, we consider the role of perceived and actual residence in a flood zone, and how these factors interact with the two message variables being studied. To that end, we conducted separate ANOVAs that examine the role of perceived and actual residence in a flood zone and how those factors interact with the message factors, while controlling for demographics.
For all of the analyses, post hoc comparisons for differences among groups with main and interaction effects were conducted using the least significant differences (LSD) test. Analyses were conducted using SPSS (IBM SPSS Statistics for Windows, version 23.0). A Bonferroni correction was used to reduce the family-wise error rate (Field 2013); given the three research questions we use α = 0.0167.
The order of the scenarios did not have a significant effect on evacuation intentions in either experiment, and so will not be considered here.
This section begins with an overview of sample demographics, and then a comparison of respondents’ actual and perceived residence in a flood zone. Following that, we address research question 1 through an analysis of the effect of evacuation notice level on evacuation intentions. Next, we examine research question 2 through an analysis of location-based messages on evacuation intentions. At the end of each of these discussions, we briefly examine the influence of the demographic variables on evacuation intentions in each of the analyses. Finally, we test the hypotheses for research question 3 with a series of ANOVAs examining how perceived residence in a flood zone and actual residence in the FEMA 100-yr flood zone interact with the message variables to affect evacuation intentions.
Women comprised 56.6% of the sample, and the mean age was 54.80 years (SD = 13.86), with a range from 18 to 86 years. Over half (60.4%) of the sample had earned a bachelor’s degree or higher, and participants were primarily white non-Hispanic (81.6%), with the remaining sample being black non-Hispanic (5.7%), Hispanic (5.5%), and multiracial or other (7.2%). The opt-in panel had more non-Hispanic white participants [X2 (4, N = 1716) = 45.510, p < .05], had higher levels of education [X2 (3, N = 1716) = 16.846; p < .001], and was slightly younger [t (1714) = 1.58, p < .05] than the KnowledgePanel participants; otherwise the panels had no significant differences on other demographic variables.
b. Actual and perceived flood zone residence
We used GIS to analyze whether respondents live in the FEMA 100-yr floodplain, using the National Flood Hazard Layer from FEMA (FEMA 2015). This analysis indicates that 19.5% of the participants live in a 100-yr floodplain (see Table 2), while the remaining 80.6% do not. Because SLOSH zones are not generally communicated to the public, and emergency managers sometimes use “flood zone” to describe evacuation areas (see Fig. 1), we used both actual residence in the FEMA 100-yr flood zone and perceptions of whether one lives in the FEMA 100-yr flood zone in all subsequent analyses.
The relationship between participants’ beliefs about where they live and where their residence is actually located relative to FEMA 100-yr flood zones is examined in Table 2. Consistent with existing research described above, the results suggest considerable confusion on the part of the participants as to whether they do or do not live in a flood zone. A quarter (23.4%) of the 332 participants who live in flood zones say they do not or they are not sure if they do. In contrast, 50% of the 510 participants who reported that they lived in a flood zone actually do not.
As described above, other research has found that some people inaccurately perceive whether they reside in a flood zone. We include both perceived flood zone and actual flood zone residence in the analyses below, but we consider these variables in separate models. This is largely because participants are not equally distributed into the different cells in the Table 1, which results in unbalanced ANOVAs when perceived and actual flood zone are considered in the same model, with very small cells (e.g., n = 10) for some experimental conditions.
c. Research question 1: How does evacuation notice level (voluntary, mandatory, advisory, and recommended) affect evacuation intentions?
To investigate our first research question regarding the influence of different types of evacuation notices, we tested the effects of “mandatory” and “voluntary” evacuation notices, as well as messages where evacuation is recommended or advised, on evacuation intentions using a one-way, between-subjects ANOVA (Table 3). As shown in Table 3, the adjusted R2 value is 0.072, indicating that the predictor variables, including the message variable, explain 7.2% of the variance in evacuation intentions.
Regarding RQ1, the analysis shows that there was a significant main effect of notice level on evacuation intentions. The evacuation notice level variable had a (partial eta squared) value of 0.052, indicating that it explained 5.2% of the variance in evacuation intentions. Post hoc tests indicate that evacuation intentions were significantly higher when an evacuation was described as mandatory than in all other conditions, and significantly lower than in all other conditions when it was described as voluntary (see Fig. 3). There were no significant differences among the three remaining three notice levels (i.e., among “strongly recommends,” “evacuation advisory,” and “advises”).
Examining the demographics included as covariates, we found that age, education, and income had no effect on evacuation likelihood. Women were significantly more likely than men to say they would evacuate (M = 4.56 and 4.00, respectively). The combined race/ethnicity variable had a significant effect on evacuation intentions, and black non-Hispanic respondents had significantly higher evacuation intentions than white non-Hispanic or other ethnicity respondents (M = 4.79, 4.29, and 3.89, respectively).
d. Research question 2: How does specific geographic or location-based information about evacuation areas affect evacuation intentions?
RQ2 addresses whether different types of geographic location-based descriptions of evacuation areas (municipality-wide, street name, flood zone, and flood-prone areas) have different effects on evacuation intentions. As for RQ1, the results are examined using a one-way, between-subjects ANOVA, reported in Table 4. The model tested includes the message variable, as well as demographic variables and perceived flood zone residence. The R2 value indicates that the overall model explained 5.9% of variance in evacuation intentions.
The location-based message variable has a small but significant main effect on evacuation intentions, and the value indicates that it explains 2.8% of the variance in evacuation intentions. Post hoc tests indicate that, across all participants, messages about evacuation of a participant’s municipality or their specific street resulted in significantly higher evacuation intentions than messages about evacuating flood-prone areas or flood zones in the participant’s municipality (Fig. 4).
The demographic variables that affect evacuation intentions are the same as in the previous section. We see a significant effect of gender, with women reporting higher evacuation intentions than men (M = 4.81 and 4.14, respectively). Race/ethnicity was again a significant predictor of evacuation intentions, with black non-Hispanic and Hispanic participants having significantly higher evacuation intentions (M = 5.01 and 5.03, respectively) compared to white non-Hispanic and participants who are from all other races and are non-Hispanic (M = 4.47 and 3.97, respectively). In addition, participants who reported being two or more races and non-Hispanic (M = 5.12) had significantly higher evacuation intentions than other races non-Hispanic participants (M = 3.97). As in the previous model, education, age, and income were not related to evacuation intentions.
e. Research question 3: How can these types of messages help reduce shadow evacuation?
To investigate whether the evacuation notice level (mandatory, voluntary, etc.) might have an effect on shadow evacuation, we examined the interaction between the evacuation notice level and perceived flood zone, while also controlling for demographics and the main effects of those variables (model R2 = 0.089). We found a main effect of the belief that one lives in a flood zone on evacuation intentions [F(1, 1675) = 22.79, p < 0.001; = 0.013], such that across all experimental evacuation notice messages, individuals who believe they live in a flood zone report being more likely to evacuate than those who do not believe they live in a flood zone or are not sure (M = 4.62 and 4.18, respectively). We found no statistically significant interaction between the perception that one lives in flood zone and the evacuation level message variable (p = .028).
We then ran a similar analysis using actual flood zone residence instead of perceived flood zone residence, to test the interaction of actual residence in a flood zone with the evacuation notice message level along with the main effects of actual flood zone, the message variable, and demographics (model R2 = 0.085). In this model we see a significant effect of actually living in a flood zone [F(1, 1681) = 10.22, p < 0.001; = 0.006]; those who live in a 100-yr flood zone have higher evacuation intentions than those who do not (M = 4. 59 and 4.25, respectively). In addition, we find a significant interaction between actual residence in a flood zone and the evacuation notice message [F(4, 1681) = 4.32, p < .002; = 0.010]. As depicted in Fig. 5, there is no significant difference in evacuation intentions between residents and nonresidents of flood zones for any of the message conditions, except with voluntary message. For participants who received “voluntary evacuation” messages, those who do not live in flood zones had significantly lower evacuation intentions than those who do live in evacuation zones. This indicates that voluntary evacuation notices may be one way to reduce shadow evacuation.
While we investigate the influence that different evacuation notice levels might have on shadow evacuation, we were particularly interested in location-based messages. Specifically, we hypothesized that messages about evacuations in flood zones and flood-prone areas might reduce shadow evacuation compared to municipality-wide or street-level evacuations, so that those who believe that they do not live in flood zones would have lower evacuation intentions. As above, we first investigated whether the location-based messages might have an effect on shadow evacuation by looking at the interaction between the location-based message variable and perceived flood zone residence, while also controlling for demographics and the main effects of those variables (model R2 = 0.090). Again, we found a main effect of the perception of living in a flood zone [F = (1, 1681) = 40.139, p < 0.001; = 0.023], such that those who believe they live in a flood zone have significantly higher evacuation intentions than those who do not believe they live in a flood zone or are not sure (M = 4.94 vs M = 4.34).
We also found a statistically significant interaction between location-based message and perceived residence in a flood zone [F(3, 1600) = 5.85, p < 0.001; = 0.010]. As shown in Fig. 6, the flood zone and flood-prone area messages result in higher evacuation intentions among participants who believe they live in the flood zone compared to those who do not believe they live in a flood zone or do not know. This supports our hypothesis that those who do not believe they live in an evacuation zone would be less likely to evacuate if given the flood zone and flood-prone area messages. In contrast, participants who saw the municipality or street-level message did not have significantly different evacuation intentions, whether they believe they are residents of flood zones or not.
Finally, we tested a similar model, but with actual flood zone residence substituted for perceived flood zone residence (model R2 = 0.077). As above, there was a main effect of actual flood zone residence on evacuation intentions [F(1, 1687) = 33.07, p < 0.001; = 0.019], with participants residing in flood zones having higher evacuation intentions than those not in flood zones (M = 5.06 and 4.38, respectively). However, there was no significant interaction between actual flood zone residence and the location-based message variable.
The results of this study highlight that evacuation messaging is a complicated issue, and the types of messages that work best depend on the goals of the communicator and the characteristics and perceptions of the message receiver. The study found that message wording had small but significant effects on evacuation intentions, which is consistent with previous research (Dow and Cutter 2002; Stein et al. 2010). In addition, we found that some message variables interacted with actual residence in a flood zone or perceived flood zone residence to influence evacuation intentions, indicating that where people live and where they think they live can affect how they respond to different messages. These findings have several implications for emergency managers and risk communicators.
This paper examines both main effects of message variables and interaction effects. To understand how the different messages might affect shadow evacuation (research question 3), we need to examine how the messages interact with where people live—either inside or outside of the areas being asked to evacuate—and where they think they live. Both actual flood zone residence and the perception that one resides in a flood zone are important to consider, since they are highly related but do not have 100% concordance (see Table 2); for example, almost a quarter of those who live in a flood zone believe they do not or are not sure.
While the interactions are key to understanding how actual and perceived flood zone residence combines with messages to affect evacuation intentions, risk communicators usually do not know whether or not someone believes that they live in a flood zone, and thus cannot tailor messages accordingly. In addition, risk communicators cannot limit their message to one geographical area, as messages are often carried on traditional media, social media, and shared from person to person (Palen 2008; Jin Liu and Austin 2014; Gladwin et al. 2007; Zhang et al. 2007). Since even specially targeted messages may reach all residents, it is important to understand the overall effects of any given evacuation notice, which we find in the main effects of the messages.
Regarding evacuation notice level, our analysis found that regardless of respondent characteristics, “mandatory” evacuation messages resulted in higher evacuation intentions when compared to any of the other messages. “Advisory” type messages had higher evacuation intentions than “voluntary” evacuation messages. While this finding suggests that using the word mandatory will increase evacuation, risk communicators cannot simply make every evacuation mandatory for a number of reasons. Officials have the valid concern that repeated mandatory evacuations would result in public perceptions of officials “crying wolf.” In addition, officials are resistant to enforcing mandatory evacuations by forcibly removing people from their homes, and not every storm warrants asking all residents to evacuate. Although the effect size is small, explaining about 5.2% of the variance in evacuation intentions, our findings do indicate that if the storm severity warrants it, a mandatory evacuation message is likely to result in the highest evacuation rate across all residents who hear the message. In situations where a mandatory evacuation would not be appropriate, we find that urging evacuation with phrases like “emergency managers strongly encourage” or “advise evacuation” is likely to result in greater evacuation levels across all participants than calling an evacuation “voluntary.” In short, this study, along with Daziano’s (2015) findings, suggest that emergency managers facing storms in which they wish to encourage evacuation of all area residents should not use the word “voluntary.”
Our interaction results, however, suggest that voluntary evacuation notices have a potential advantage, in that they may decrease shadow evacuations, without decreasing evacuation in those that do live in flood zones. In this study, participants who live in a flood zone had similarly high evacuation intentions in response to the voluntary evacuation message and the advisory and mandatory messages. In contrast, those who do not live in a flood zone had significantly lower evacuation intentions in response to the voluntary evacuation compared to the mandatory message. This indicates that if public officials are concerned about shadow evacuations, using voluntary evacuation notices may be more effective at stopping people not in the flood zone from evacuating compared to advisory and mandatory evacuation notices. Additional research is needed, however, to further test this finding.
Using municipality and street name evacuation messages resulted in higher evacuation intentions than messages about evacuations in flood zones and flood-prone areas across all respondents. It was surprising that specifying that the respondent’s street name was being evacuated did not result in higher evacuation intentions than specifying the respondent’s municipality, as this could be seen as likely to increase perceived personal relevance of the message, which has been shown to increase evacuation (Mileti and O’Brien 1992). This finding suggests that it may not be worth the resources for emergency managers to create targeted street-level messages, as we did not find evidence that they would yield additional evacuations. However, additional study of targeted street-level messages is needed before concluding that they do not increase evacuation in a real world context. In addition, while we did find main effects for these location-based messages, overall these messages had only a small effect (explaining 2.8% of variance in evacuation intentions).
Location-based messages also offer some promise in lowering shadow evacuation. We found that messages evacuating “flood zones” or “flood-prone areas” reduce evacuation among those who do not believe they live in a flood zone, without significantly suppressing evacuation intentions of those who do live in the flood zone. In this case, the location-based message variable did not interact with actual flood zone residence, but only perceived flood zone residence (which is different than the evacuation level message interaction described above). Given that there are residents who live in flood zones and do not realize it (4.6% of the total sample), it is important to consider that these types of messages may also suppress evacuation among those who are at risk for coastal flooding but do not know it. This is a group that would be most likely to benefit from educational outreach about flood zone residence.
This study has several limitations. It was conducted online, using a hypothetical scenario, which lowers the ecological validity of the study. For example, we found that the communication channel (phone, text, in-person, etc.) did not result in any differences in evacuation intentions. However, respondents were simply asked to imagine each channel; for example, in the in-person communication channel they were told that “an official comes to your home and tells you the following,” and for the text message condition they were told that they received “an emergency text message that says the following.” Thus, one cannot conclude from this study that in the real world, someone knocking on a door telling residents to evacuate would have the same effect as a text. However, for those effects of message variables that our study did find, the use of hypothetical scenarios likely attenuated rather than altering the direction of effects.
External validity is another potential concern. We sampled only from a limited geographical area. And while the sample did include some members of a randomly selected panel, the majority of the panel was made up of an opt-in panel of nonrandomly selected survey respondents. Although this mixed sampling approach could potentially limit the generalizability of the findings, we did not find differences between the two samples in terms of evacuation likelihood in either scenario, and there were only limited differences in demographics. Because respondents were randomly assigned to conditions, the sampling strategy is unlikely to pose a threat to the internal validity of the study, which is most important for understanding the different effects of the messages in the experimental conditions.
Finally, it is important to note that the data presented here represent only some of what was collected in the survey. We conducted a series of four experiments, and present data here from only two. This was done largely as a result of space constraints. There were significant main effects of the main message variables we tested in each of the four experiments. Again, here we present only a subset of the results due to space constraints; we anticipate publishing those additional findings in the future. And, although we collected data on multiple dependent variables, we present in this article only data on one dependent variable, evacuation intentions. We chose to focus these analyses on evacuation intentions because evacuation is generally the ultimate goal of evacuation communications. Most of the other dependent variables, such as perceived storm severity and emotional response to the messages, were included in the survey so that they could be investigated as potential mediating or moderating variables in additional work (Demuth et al. 2016). Finally, to decrease the likelihood of type I error, we used a Bonferroni correction in all analyses.
b. Future research
This study raises important questions for future research. Sometimes residents are given more than the individual pieces of information used in our experimental scenarios. For example, one New Jersey town’s evacuation notice for the 23 January 2016, storm included a map, street names, and a mention of the tier 1 flood zone being evacuated. In reality, when residents make evacuation decisions, they use different combinations of information provided in various formats from a range of sources. In future research, it will be important to understand how different message components work together to determine what pieces are the most important and can help to motivate individuals at high risk to evacuate. It will also be important to identify the conditions under which multiple pieces of information might reduce shadow evacuation.
Many evacuation communications use references to flood zones, as we did in the current study. These are defined by FEMA and used for the National Flood Insurance Program, and they are a designation that is consistent across states. Whether one’s home is in a FEMA-designated flood zone is only sometimes communicated to the residents, most often to those who have mortgages. Across all messages, we found that individuals who believe they live in a flood zone report being more likely to evacuate than those who do not believe they live in a flood zone or are not sure. This indicates that ongoing conversations and educational campaigns about whether people live in a flood zone could potentially play a role in increasing evacuation rates. However, the effectiveness of such educational campaigns needs to be evaluated.
In addition, municipalities are increasingly moving toward creating coastal evacuation zones and educating their residents about them, although almost none of the municipalities we studied currently have clearly identified evacuation zones (New York City being the notable exception; New York City Office of Emergency Management 2016). As clearly identified and communicated evacuation zones become more common, it will be important to repeat experiments such as those performed here using evacuation zones, to test if the findings from the current study generalize from flood zones to evacuation zones.
We found that the way in which an evacuation message is constructed can result in small but important differences in evacuation intentions. In our study, mandatory evacuation messages resulted in the highest evacuation intentions across the entire population compared to the other evacuation levels. However, if meteorological, political, financial, or other factors preclude using mandatory evacuations, saying that evacuation is encouraged, that there is an evacuation advisory, or that residents are encouraged to evacuate may result in significantly higher evacuation levels across the entire population. In addition, municipality wide or street-level evacuation messages resulted in higher evacuation levels across the entire recipient population when compared to messages about evacuations in flood zones or flood-prone areas.
The findings outlined in the previous paragraph are less important if reducing shadow evacuation is paramount. We found that compared to the other location-based messages tested, saying that only flood zones or flood-prone areas are being evacuated resulted in lower evacuation intentions among those who do not believe they live in flood zones, without reducing evacuation intentions among those who believe they do live in flood zones. Similarly, compared to the other evacuation notice levels, voluntary evacuation messages reduced evacuation intentions among those who do not live in flood zones, while not reducing evacuation intentions for those that do live in a flood zone. Unfortunately, some messages that reduce shadow evacuation risk leaving behind the small percentage of people who live in flood zones but do not know it.
In sum, the types of messages that will be most effective in a given situation depend on many factors. The findings presented here can help emergency managers and risk communicators to understand the effects that their messages can have on different segments of the population, and when different types of messages are most appropriate.
This research was supported by the NJ Sea Grant Coastal Storm Awareness Program, Grant R/CSAP-1-NJ. We wish to acknowledge the assistance of Lucas Marxen and the Rutgers Office of Research Analytics for their assistance with GIS analysis, and our collaborators Steven Decker, Christopher Obropta, Karen O’Neil, and David Robinson.
Because participants were being recruited from an existing panel, we use cooperation rate rather than response rate. This indicates the percentage of panel members who were invited to participate that actually completed the survey.
Participants were asked if they had cell phones that can receive texts, as well as social media accounts. To increase ecological validity, only those who did were eligible to be randomized into the text and social media conditions.