• Baker, E. , 1991: Hurricane evacuation behavior. Int. J. Mass Emerg. Disasters, 9, 287310.

  • Baker, E. , 1995: Public response to hurricane probability forecasts. Prof. Geogr., 47, 137147, https://doi.org/10.1111/j.0033-0124.1995.00137.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blome, R. , 2005: Staying? Better write your SS# on your body… NBC News, 22 September, www.nbcnews.com/id/9439537/ns/us_news-katrina_the_long_road_back/t/staying-better-write-your-ss-your-body/#.X1_QE5NKiYU.

  • Broad, K. , A. Leiserowitz, J. Weinkle, and M. Steketee , 2007: Misinterpretations of the “cone of uncertainty” in Florida during the 2004 hurricane season. Bull. Amer. Meteor. Soc., 88, 651668, https://doi.org/10.1175/BAMS-88-5-651.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cameron, A. C. , and P. K. Trivedi , 2010: Microeconometrics Using Stata. Stata Press, 706 pp.

  • Christensen, L. , and C. E. Ruch , 1980: The effect of social influence on response to hurricane warnings. Disasters, 4, 205210, https://doi.org/10.1111/j.1467-7717.1980.tb00273.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cokely, E. , M. Galesic, E. Schulz, S. Ghazal, and R. Garcia-Retamero , 2012: Measuring risk literacy: The Berlin numeracy test. Judgment Decis. Making, 7, 2547, https://doi.org/10.1037/t45862-000.

    • Search Google Scholar
    • Export Citation
  • Cokely, E. , A. Feltz, S. Ghazal, J. N. Allan, D. Petrova, and R. Garcia-Retamero , 2018: Skilled decision theory: From intelligence to numeracy and expertise. The Cambridge Handbook of Expertise and Expert Performance, K. A. Ericsson et al., Eds., Cambridge University Press, 476505.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cox, D. R. , 1972: Regression models and life-tables. J. Roy. Stat. Soc., B34, 187202, https://doi.org/10.1111/j.2517-6161.1972.tb00899.x.

    • Search Google Scholar
    • Export Citation
  • Cox, D. R. , and D. Oakes , 1984: Analysis of Survival Data. Chapman and Hall, 201 pp.

  • Dash, N. , and H. Gladwin , 2007: Evacuation decision making and behavioral responses: Individual and household. Nat. Hazards Rev., 8, 6977, https://doi.org/10.1061/(ASCE)1527-6988(2007)8:3(69).

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dawson, C. , E. J. Kubatko, J. J. Westerink, C. Trahan, C. Mirabito, C. Michoski, and N. Panda , 2011: Discontinuous Galerkin methods for modeling hurricane storm surge. Adv. Water Resour., 34, 11651176, https://doi.org/10.1016/j.advwatres.2010.11.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dow, K. , and S. L. Cutter , 2002: Emerging hurricane evacuation issues: Hurricane Floyd and South Carolina. Nat. Hazards Rev., 3, 1218, https://doi.org/10.1061/(ASCE)1527-6988(2002)3:1(12).

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, L. , 2012: Scientific prerequisites to comprehension of the tropical cyclone forecast: Intensity, track, and size. Wea. Forecasting, 27, 462472, https://doi.org/10.1175/WAF-D-11-00041.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fu, H. , C. G. Wilmot, H. Zhang, and E. J. Baker , 2007: Modeling the hurricane evacuation response curve. Transp. Res. Rec., 2022, 94102, https://doi.org/10.3141/2022-11.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gudishala, R. , and C. Wilmot , 2017: Modeling emergency managers’ hurricane evacuation decisions. Transp. Res. Rec., 2604, 8287, https://doi.org/10.3141/2604-10.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoss, F. , and P. Fischbeck , 2016: Increasing the value of uncertain weather and river forecasts for emergency managers. Bull. Amer. Meteor. Soc., 97, 8597, https://doi.org/10.1175/BAMS-D-13-00275.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huang, S.-K. , M. K. Lindell, and C. S. Prater , 2016: Who leaves and who stays? A review and statistical meta-analysis of hurricane evacuation studies. Environ. Behav., 48, 9911029, https://doi.org/10.1177/0013916515578485.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kahneman, D. , and A. Tversky , 1979: Prospect theory: An analysis of decision under risk. Econometrica, 47, 263, https://doi.org/10.2307/1914185.

  • Keneally, M. , 2017: Mayor tells those not evacuating for hurricane to put Social Security numbers on arms. ABC News, 25 August, https://abcnews.go.com/US/mayor-tells-evacuating-hurricane-harvey-put-social-security/story?id=49417573.

    • Search Google Scholar
    • Export Citation
  • Knabb, R. , J. Rhome, and D. Brown , 2011: Tropical Cyclone Report: Hurricane Katrina 23–30 August 2005. National Hurricane Center, 43 pp., www.nhc.noaa.gov/data/tcr/AL122005_Katrina.pdf.

    • Search Google Scholar
    • Export Citation
  • Kubatko, E. J. , J. J. Westerink, and C. Dawson , 2006: hp discontinuous Galerkin methods for advection dominated problems in shallow water flow. Comput. Methods Appl. Mech. Eng., 196, 437451, https://doi.org/10.1016/j.cma.2006.05.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Loomis, J. , 2011: What’s to know about hypothetical bias in stated preference valuation studies?: Hypothetical bias in stated preference valuation studies. J. Econ. Surv., 25, 363370, https://doi.org/10.1111/j.1467-6419.2010.00675.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Losee, J. E. , K. Z. Naufel, L. Locker, and G. D. Webster , 2017: Weather warning uncertainty: High severity influences judgment bias. Wea .Climate Soc., 9, 441454, https://doi.org/10.1175/WCAS-D-16-0071.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mele, C. , 2016: How to get people to evacuate? Try fear. The New York Times, 8 September 2017, www.nytimes.com/2016/10/07/us/how-to-get-people-to-evacuate-try-fear.html.

    • Search Google Scholar
    • Export Citation
  • Meyer, R. , K. Broad, B. Orlove, and N. Petrovic , 2013: Dynamic simulation as an approach to understanding hurricane risk response: Insights from the Stormview lab. Risk Anal., 33, 15321552, https://doi.org/10.1111/j.1539-6924.2012.01935.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morrow, B. H. , J. K. Lazo, J. Rhome, and J. Feyen , 2015: Improving storm surge risk communication: Stakeholder perspectives. Bull. Amer. Meteor. Soc., 96, 3548, https://doi.org/10.1175/BAMS-D-13-00197.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nicholson, W. C. , 2007: Emergency planning and potential liabilities for state and local governments. State Local Gov. Rev., 39, 4456, https://doi.org/10.1177/0160323X0703900105.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NOAA, 2005: Hurricane Rita images. NOAA National Geodetic Survey, https://geodesy.noaa.gov/storm_archive/storms/rita/index.html.

  • Peerbolte, S. L. , and M. L. Collins , 2013: Disaster management and the critical thinking skills of local emergency managers: Correlations with age, gender, education, and years in occupation. Disasters, 37, 4860, https://doi.org/10.1111/j.1467-7717.2012.01291.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peters, E. , D. Västfjäll, P. Slovic, C. K. Mertz, K. Mazzocco, and S. Dickert , 2006: Numeracy and decision making. Psychol. Sci., 17, 407413, https://doi.org/10.1111/j.1467-9280.2006.01720.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pham, E. O. , C. T. Emrich, Z. Li, J. Mitchem, and S. L. Cutter , 2020: Evacuation departure timing during Hurricane Matthew. Wea. Climate Soc., 12, 235248, https://doi.org/10.1175/WCAS-D-19-0030.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Regnier, E. , 2008: Public evacuation decisions and hurricane track uncertainty. Manage. Sci., 54, 1628, https://doi.org/10.1287/mnsc.1070.0764.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sherman-Morris, K. , and K. B. Antonelli , 2018: Hurricane knowledge and interpretation of forecasted error cone and wind potential graphics. J. Emerg. Manage., 16, 137148, https://doi.org/10.5055/jem.2018.0363.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, V. , 1976: Experimental economics: Induced value theory. Amer. Econ. Rev., 66, 274279.

  • Thompson, R. R. , D. R. Garfin, and R. C. Silver , 2017: Evacuation from natural disasters: A systematic review of the literature. Risk Anal., 37, 812839, https://doi.org/10.1111/risa.12654.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tobin, J. , 1958: Estimation of relationships for limited dependent variables. Econometrica, 26, 2436, https://doi.org/10.2307/1907382.

  • Tversky, A. , and D. Kahneman , 1981: The framing of decisions and the psychology of choice. Science, 211, 453458, https://doi.org/10.1126/science.7455683.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • USACE, 2008: Flood insurance study: Southwestern Parishes, Louisiana. Intermediate Submission 2, USACE, 697 pp.

  • USACE, 2011: Flood insurance study: Coastal counties, Texas. Intermediate Submission 2, USACE, 150 pp.

  • Wallace, J. W. , C. Poole, and J. A. Horney , 2016: The association between actual and perceived flood risk and evacuation from Hurricane Irene, Beaufort County, North Carolina. J. Flood Risk Manag., 9, 125135, https://doi.org/10.1111/jfr3.12115.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wernstedt, K. , P. S. Roberts, J. Arvai, and K. Redmond , 2019: How emergency managers (mis?)interpret forecasts. Disasters, 43, 88109, https://doi.org/10.1111/disa.12293.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whitehead, J. C. , B. Edwards, M. Van Willigen, J. R. Maiolo, K. Wilson, and K. T. Smith , 2001: Heading for higher ground: Factors affecting real and hypothetical hurricane evacuation behavior. Environ. Hazards, 2, 133142, https://doi.org/10.3763/ehaz.2000.0219.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilson, L. R. , and R. McCreight , 2012: Public emergency laws & regulations: Understanding constraints & opportunities. J. Homeland Secur. Emerg. Manage., 9, 7, https://doi.org/10.1515/1547-7355.2034.

    • Search Google Scholar
    • Export Citation
  • Windham, G. , E. Posey, and B. Spencer , 1977: Reactions to storm threat during Hurricane Eloise. Tech. Rep. 51, Mississippi State University, 74 pp.

    • Search Google Scholar
    • Export Citation
  • Wu, H.-C. , M. K. Lindell, C. S. Prater, and C. D. Samuelson , 2014: Effects of track and threat information on judgments of hurricane strike probability. Risk Anal., 34, 10251039, https://doi.org/10.1111/risa.12128.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wu, H.-C. , M. K. Lindell, and C. S. Prater , 2015a: Process tracing analysis of hurricane information displays. Risk Anal., 35, 22022220, https://doi.org/10.1111/risa.12423.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wu, H.-C. , M. K. Lindell, and C. S. Prater , 2015b: Strike probability judgments and protective action recommendations in a dynamic hurricane tracking task. Nat. Hazards, 79, 355380, https://doi.org/10.1007/s11069-015-1846-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    Informational conditions by treatment.

  • View in gallery
    Fig. 2.

    DG-SWEM inundation projections provided to subjects.

  • View in gallery
    Fig. 3.

    Spatial density of evacuation decisions. Choropleth maps identify spatial density of mandatory evacuation decisions made by subjects. Zonal division aligns with the Harris County Office of Emergency Management’s established hurricane evacuation planning zones. Numerate/innumerate, which refers to more and less numerate, indicates subject was successful/unsuccessful on the single-item Berlin Numeracy Test. “Human exposure” indicates subject self-identified in post-experiment survey as knowingly overevacuating zone(s) even though they thought the likelihood of inundation to be low.

All Time Past Year Past 30 Days
Abstract Views 21 21 0
Full Text Views 932 929 26
PDF Downloads 458 454 14

Informational Determinants of Large-Area Hurricane Evacuations

Noah DormadyJohn Glenn College of Public Affairs, The Ohio State University, Columbus, Ohio;

Search for other papers by Noah Dormady in
Current site
Google Scholar
PubMed
Close
,
Anthony FasanoDepartment of Physics, The Ohio State University, Columbus, Ohio;

Search for other papers by Anthony Fasano in
Current site
Google Scholar
PubMed
Close
,
Alfredo Roa-HenriquezCollege of Business, North Dakota State University, Fargo, North Dakota;

Search for other papers by Alfredo Roa-Henriquez in
Current site
Google Scholar
PubMed
Close
,
Drew FlanaganJohn Glenn College of Public Affairs, The Ohio State University, Columbus, Ohio;

Search for other papers by Drew Flanagan in
Current site
Google Scholar
PubMed
Close
,
William WelchJohn Glenn College of Public Affairs, The Ohio State University, Columbus, Ohio;

Search for other papers by William Welch in
Current site
Google Scholar
PubMed
Close
, and
Dylan WoodCivil and Environmental Engineering, University of Notre Dame, Notre Dame, Indiana

Search for other papers by Dylan Wood in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

This study reports on two experiments to investigate the informational determinants of hurricane evacuation decisions (temporal and spatial). Whereas most observational and experimental studies in this domain address the public’s response to forecast information, this study addresses emergency management decisions. Using a subject sample of emergency managers and other public safety leaders, contrasted with a more typical university subject pool, this study presents an experimental design that overcomes the counterfactual problem present in all prior published experiments, by relying on an actual storm (Hurricane Rita) with a known outcome. Several methodological advancements are presented, including the use of an established numeracy instrument, integration of advanced hydrodynamic forecasts, and use of a loss aversion frame to improve generalizability. Results indicate that the availability of additional forecast information (e.g., wind speed, forecast tracks) significantly increases the probability and improves the timing of early voluntary evacuation. However, we observe that more numerate subjects are less likely to avoid relying upon forecast information that is characterized by probability (e.g., the uncertainty in the forecast track, sometimes referred to as the “cone of uncertainty”). Consequently, more numerate emergency managers are almost twice as likely as less numerate ones to provide additional evacuation time to their coastal communities, and they do so by longer than a typical workday (8.8 h). Results also indicate that subjects knowingly overevacuate large populations when making spatial mandatory evacuation orders. However, results indicate that numeracy mitigates this effect by more than half in terms of the population subject to mandatory evacuation.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Noah Dormady, dormady.1@osu.edu

Abstract

This study reports on two experiments to investigate the informational determinants of hurricane evacuation decisions (temporal and spatial). Whereas most observational and experimental studies in this domain address the public’s response to forecast information, this study addresses emergency management decisions. Using a subject sample of emergency managers and other public safety leaders, contrasted with a more typical university subject pool, this study presents an experimental design that overcomes the counterfactual problem present in all prior published experiments, by relying on an actual storm (Hurricane Rita) with a known outcome. Several methodological advancements are presented, including the use of an established numeracy instrument, integration of advanced hydrodynamic forecasts, and use of a loss aversion frame to improve generalizability. Results indicate that the availability of additional forecast information (e.g., wind speed, forecast tracks) significantly increases the probability and improves the timing of early voluntary evacuation. However, we observe that more numerate subjects are less likely to avoid relying upon forecast information that is characterized by probability (e.g., the uncertainty in the forecast track, sometimes referred to as the “cone of uncertainty”). Consequently, more numerate emergency managers are almost twice as likely as less numerate ones to provide additional evacuation time to their coastal communities, and they do so by longer than a typical workday (8.8 h). Results also indicate that subjects knowingly overevacuate large populations when making spatial mandatory evacuation orders. However, results indicate that numeracy mitigates this effect by more than half in terms of the population subject to mandatory evacuation.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Noah Dormady, dormady.1@osu.edu

Research evaluating household evacuation decisions in response to hurricane evacuation orders is extensive (Baker 1991; Dash and Gladwin 2007; Thompson et al. 2017). However, very little is known about how those evacuation orders are made by emergency managers (EMs) and other public safety professionals. When a hurricane is approaching, what explains the timing of voluntary evacuation orders? When mandatory evacuation orders are issued, why are some communities evacuated and others not? We know from ex post observational data that compliance with these orders varies considerably (Baker 1995; Dow and Cutter 2002; Fu et al. 2007; Huang et al. 2016; Pham et al. 2020; Wallace et al. 2016), and since at least 2005, some officials have resorted to scare tactics to enforce compliance, even urging noncompliant residents to write their social security numbers on their arms and abdomens with markers to facilitate body identification (Blome 2005; Keneally 2017; Mele 2016). Given the commonplace evacuation of nonexposed communities, there may be adverse societal consequences if residents come to expect some degree of hedging by EMs who knowingly overevacuate.

From the limited EM decision-making research, we have learned that when provided multiple types of storm forecast information (e.g., maximum wind speeds, storm path, probable areas for landfall), EMs are subject to many of the same cognitive errors and decision biases as nonprofessionals (Drake 2012; Wernstedt et al. 2019) with some notable exceptions (Hoss and Fischbeck 2016). Just like nonprofessionals, research finds that EMs tend to overly focus on the forecasted path of the storm, known to forecasters as the “center track,” rather than the so-called “cone of uncertainty,” a tool which forecasters use to visually represent the probabilistic future track of a tropical storm, either to the left or right of its forecasted center track (Broad et al. 2007; Meyer et al. 2013; Sherman-Morris and Antonelli 2018). We have also learned that there can be anchoring bias, wherein some decision-makers may “anchor” onto information with higher perceived severity (Losee et al. 2017). And, ex post observational studies suggest that physical properties of the local terrain appropriately factor into decisions (Gudishala and Wilmot 2017).

Some of these informational parameters of the storm forecast presented to EMs are implicitly probabilistic in nature (e.g., the cone of uncertainty). Because these parameters are probabilistic in nature, a decision-maker’s ability to effectively utilize them may be tied to their ability to understand probability (i.e., statistical numeracy) (Cokely et al. 2012; Peters et al. 2006). Consequently, a few studies have considered the numeracy (Wernstedt et al. 2019) and critical thinking capacity (Peerbolte and Collins 2013) of EMs. To date, no one has evaluated whether numeracy influences which aspects of the storm forecast information that EMs rely upon, and whether, in the absence of numeracy, decision-makers avoid relying on information that is perceived to be probabilistic in nature. Moreover, no one has evaluated whether this relationship impacts social welfare by shaping earlier versus later or over- versus underevacuation orders.

Methodologically, hypothetical surveys feature prominently in this domain (Baker 1995; Drake 2012; Wernstedt et al. 2019), but are subject to hypothetical bias (Loomis 2011). And unlike experiments, observational studies (Gudishala and Wilmot 2017; Regnier 2008) are limited by the absence of controlled counterfactuals. The experiments that do exist, again, predominantly focus on household decision-making (Christensen and Ruch 1980; Losee et al. 2017; Meyer et al. 2013; Sherman-Morris and Antonelli 2018; Wu et al. 2014) and very few focus on the decisions of emergency managers (Wernstedt et al. 2019; Wu et al. 2015a,b). And, none of those evaluate the role of numeracy in spatial or temporal decision-making.

The experiment presented here is unique in several important ways. Prior experimental designs were limited to cross-treatment comparisons without a counterfactual. In other words, they were able to analyze subject behavior across treatments, but unable to compare decisions to the storm’s ultimate outcome because the scenarios were fictitious. This experiment overcomes this limitation by replicating and obfuscating an actual historic hurricane (Hurricane Rita in the area of Houston and Galveston, Texas, in 2005) so an assessment can be made relative to how the disaster actually transpired. Unlike experiments relying solely on student subjects (Losee et al. 2017; Wu et al. 2015a,b), this experiment also incorporates a sample of professional subjects including county EMs from select hurricane-observing states (excluding Texas) and public safety leaders.

In this two-stage experiment, we study both temporal and spatial dimensions. The stage 1 experiment evaluates the timing of voluntary evacuation order recommendations for coastal and low-lying communities in the context of experimentally controlled forecast information. The stage 2 experiment evaluates subjects’ spatial decisions on which of the Houston area’s established evacuation zones are to be mandatorily evacuated. Because this experiment overcame the absence of a counterfactual, subject decisions could be scored. This enables the design to be the first of its kind to integrate induced value theory (Smith 1976) with embedded decision scoring. In other words, while prior evacuation experiments were both hypothetical and had payment unconnected to decision performance, this experiment is not so limited, and incentivizes subjects to perform as they would in the field. The scoring functions are novel in design, accounting for both over- and underevacuation.

Beyond this, the scoring functions utilized in this experiment further incentivize externally valid decisions by building upon a loss (rather than a gain) frame (Tversky and Kahneman 1981). To induce the endowment effect, subjects were given the single-item battery of the Berlin Numeracy Test (Cokely et al. 2012). Thus, this experiment is the only experiment in this domain that can simultaneously incentivize externally valid and real-case decisions while controlling for subject numeracy using an established risk literacy instrument. Additional controls are integrated through a post-experiment survey. Statistical analyses are presented to evaluate both stages.

Experimental methods

Experiments are, by their very nature, abstractions of reality. They are designed for the purpose of testing theories regarding mechanisms or behaviors and are not meant to be inclusive of every facet of a physical or topographical area of study, which would otherwise be impractical and confuse subjects. Experiments can play an important role in evaluating risk behaviors because they can control for counterfactuals in a way that observational data cannot. In this domain, there have been a handful of experiments focusing on the evacuation decisions of households (Christensen and Ruch 1980; Losee et al. 2017; Meyer et al. 2013; Sherman-Morris and Antonelli 2018; Wu et al. 2014), but even fewer have focused on the decisions of emergency managers (Wernstedt et al. 2019; Wu et al. 2015a,b). Prior experimental designs were limited to cross-treatment comparisons without a counterfactual as they used fictional scenarios to cross-analyze subject behavior under various treatments, but had no measure of the “correctness” of the subjects’ decision-making with respect to an actually observed event. This experiment overcomes this limitation by replicating an actual historic hurricane (Hurricane Rita in 2005),1 so an assessment of each decision can be made relative to how the disaster actually transpired.

Subjects took two experiments, stage 1 and stage 2. In the first stage, subjects were randomly assigned to one of four treatment groups in a between-subjects design online experiment.2 Subjects assumed a role as Senior Advisor at the Texas Office of Emergency Management to advise the Texas governor on making large-area evacuation decisions for a storm approaching from the Gulf. The experiment included two distinct stages. In stage 1, subjects made a voluntary evacuation recommendation for the coastal and low-lying areas in the Houston metro area. In stage 2, subjects made a spatial mandatory evacuation recommendation consistent with Harris County’s four established hurricane evacuation zones. Stage 2 used a within-subjects design. Across the two stages, subjects made a temporal voluntary decision followed by a spatial mandatory one.

Stage 1 experiment.

Subject entry into the experiment coincided with the time of the first forecast advisory by the National Hurricane Center (NHC) at which the storm made the transition to a category 1 on the Saffir–Simpson index [1100 local time (LT) Tuesday, 20 September]. At that time, the storm was located just south of Miami, Florida, and headed west. Subjects were presented with actual hurricane forecast information from Rita, obfuscated by name to “Rebecca” to avoid recall identification. Forecast information from the successive NHC advisories was progressively added across treatments. Advisories were presented in a series of nine decision periods, or rounds, that mapped directly onto the nine advisories issued for Rita before the NHC issued its critical “Hurricane Warning” for the area (1000 LT Thursday, 22 September). Subjects were given the opportunity to make the voluntary order recommendation each round. Once the recommendation was made, subjects exited the stage and were informed that a Hurricane Warning had been issued and hurricane-force winds were expected within 24–36 h. Subjects not making a voluntary evacuation recommendation proceeded through all nine advisories, or rounds, and were then notified that the Hurricane Warning had been issued. Advisories were renumbered to exclude early storm formation for decision simplicity.

Treatment conditions selectively presented subjects with increased probabilistic forecast information. T1 represented a baseline control condition in which only historic and current storm information was provided: historic center track, current center location, and current max sustained wind speed. T2 added forecast center track and forecasted watch and warning areas. T3 added cone of uncertainty. T4 added forecasted max wind Speed. See Fig. 1.

Fig. 1.
Fig. 1.

Informational conditions by treatment.

Citation: Bulletin of the American Meteorological Society 103, 3; 10.1175/BAMS-D-21-0008.1

Stage 2 experiment.

Stage 2 used a within-subjects design in which subjects were asked to make a spatial mandatory evacuation order recommendation. From an interactive zone map of the region, they were given one of seven possible mandatory evacuation configurations (see section 4.2.11 in the supplemental material). Subjects identified one of the seven possible mandatory evacuation configurations that correspond to established Harris County evacuation areas.

Because today’s advanced hydrological models are often presented to decision-makers in the form of best versus worst-case inundation scenarios, subjects were presented with three possible inundation maps, or maps of the maximum extent of flooding induced by the hurricane storm surge. The maps were produced by finite element analysis in the hydrodynamic model, DG-SWEM (Discontinuous Galerkin–Shallow Water Equations Model) (Dawson et al. 2011; Kubatko et al. 2006). The hydrodynamic model analysis was executed on a computational mesh grid known as TX2008, a grid of more than 2.8 million nodes developed for FEMA Flood Insurance Studies (FIS) on the Texas coastline by the U.S. Army Corp of Engineers. The grid represents the bathymetry, i.e., the surface of Earth that lies underwater, in the Gulf of Mexico and western North Atlantic Ocean as well as topography, the surface of Earth above water, in coastal Texas with nodal spacings varying significantly throughout, where maximum grid resolution is roughly 19–24 km in the deep Atlantic Ocean and minimum grid resolution is roughly 30 m in Texas (USACE 2011). Hydrodynamic forces in the model were based on tidal constituents enforced at the model boundaries as well as a quadratic drag law for wind stress, where observational wind speeds and air pressures (10 m above sea level) for the hurricane were provided from data assimilation analysis by Oceanweather Inc., also developed for FEMA FIS studies in both Louisiana and Texas (USACE 2011, 2008). Hurricane Rita was simulated up until the time of NHC advisory number 20 for the storm, the first NHC advisory for which a hurricane warning was issued in the study area (Houston–Galveston, Texas) and consistent with the decision timing provided to subjects.

From this point, three different scenarios were modeled: a “center track” scenario, where the storm proceeded directly along the center track forecasted in NHC advisory number 20, and “veer-left” and “veer-right” scenarios, where the storm track deviated from the forecasted center track in either the left or right direction, based on the NHC specifications on the extent of the cone of uncertainty for its forecasted hurricane tracks. The corresponding inundation maps are generated by considering the maximum water surface elevations at each nodal point in the computational grid over the entirety of the simulation for each storm scenario modeled, and by drawing color where the depth of the water on land (i.e., initially dry areas) exceeds 0.15 m (roughly half a foot). See Fig. 2.

Fig. 2.
Fig. 2.

DG-SWEM inundation projections provided to subjects.

Citation: Bulletin of the American Meteorological Society 103, 3; 10.1175/BAMS-D-21-0008.1

Scoring functions and numeracy test.

Advancements from the field of experimental economics can meaningfully inform disaster evacuation experiments. To date, all prior evacuation experiments have omitted the integration of induced value theory (see the work of Nobel Laureate Vernon Smith; Smith 1976) into their designs. This typically occurs when subject payment is tied to decision-making performance to induce subjects to make decisions that more closely approximate decisions that they would make in the external context being evaluated. Because prior evacuation experiments omitted a counterfactual from their designs, they were not able to score the appropriateness of decisions, and, therefore, they were unable to induce value. Additionally, because they relied upon hypothetical scenarios, hypothetical bias (Loomis 2011) may have also been a concern. In other words, in previous experiments, subjects got paid either way and had no direct inducements to behave as they would in a real disaster.

This experiment not only overcomes the issues but takes the added step of building a scoring function (i.e., the scoring mechanism) that is tied to a loss frame (Kahneman and Tversky 1979; Tversky and Kahneman 1981). To induce the endowment effect, subjects were given the single-item battery of the Berlin Numeracy Test (Cokely et al. 2012)3 at the outset of the experiment. The test “specifically measures the range of statistical numeracy skill that is important for accurately interpreting and acting on information about risk—i.e., risk literacy” (Cokely et al. 2012, p. 37). Subjects were informed that by completing the assessment, they would be earning points that they would either retain, or lose, based on their decision performance in the experiment. They were informed that at the end of the experiment points would be converted to dollars for their subject payment. Subjects were not informed of their performance on the numeracy test and were given 200 points each for their completion of it. In this way, we were able to statistically account for numeracy using the same assessment that we used to generate the subject’s endowment.

Subject payment ratios differed by subject type. Professional subjects’ conversion rate was 15 cents per point remaining at the end of the experiment. Student subjects’ rate was 6 cents. Rates were set to approximate a rate of $60 per hour. based on the mean duration recorded during pilot experiments with graduate student subjects. Subjects also earned an additional $5 (student subjects $3) for completing a post-experiment survey that provided additional explanatory variables. Mean total payouts were $24.47 and $11.62 for professional and student subjects, respectively.

Scoring functions were structured to coincide with best-possible outcomes given the realized impacts of Rita. Stage 1 decision scoring was straightforward, given observed landfall just north of the Houston Metro area. Voluntary evacuation orders for the coastal and low-lying areas (e.g., Galveston) were an appropriate decision, and an early voluntary evacuation recommendation was appropriate for those communities. Stage 2 decision scoring required the creation of an inundation map. While no official map was created or publicly released, our student team developed one from the postdisaster aerial imagery provided by NOAA’s Geodetic Survey (NOAA 2005) and the NHC’s Tropical Cyclone Report for Rita (Knabb et al. 2011). The report provides geographic inundation details based on geographic indicators, including flood insurance claims and high-water marks. Inundation was observed only for the coastal evacuation zone including Galveston, with minor wind-induced inland flooding.

The scoring function for each stage (text and mathematical derivation provided in SI-6) made use of 100 points, or half of the endowment. The stage 1 function accounted for each possible decision outcome and was designed to account for improved early evacuation as well as the adverse effects of false positives (Regnier 2008). The stage 2 function was developed to account for the at-risk populations expected to remain in each of the county’s four evacuation zones.4 Estimated populations remaining in each zone were presented to subjects and were visible to subjects separately for each decision selection and visible through an interactive color-adaptive spatial map. The scoring function was designed to simultaneously account for both under- and overevacuation. While no metric could accurately capture the relative value of each, we presented subjects with a value function that scored underevacuation twice as adversely as overevacuation.

Video instructions.

Subject instructions were delivered as videos that took the form of a professional briefing; one preceding each stage. Instructions were presented by Darryl Anderson, former Interoperability Coordinator for the U.S. Dept. of Homeland Security Office of Emergency Communications, and Commandant Ret. of the Ohio Highway Patrol Academy. Videos were professionally recorded and edited. Separate video tracks were developed for each treatment group. Videos also contain treatment-specific instructions for reading advisories taken from a different storm, Ophelia (2005). The experiment interface was coded to prevent fast-forwarding or skipping of the videos. Closed captioning was manually revised to 100% accuracy and autoenabled.

Subject sample.

The subject pool included both professional subjects as well as a more typical student population. Professional subjects (N = 81) consisted predominantly of emergency managers or assistant emergency managers drawn from coastal hurricane-observing states,5 excluding Texas to avoid memory bias. All valid publicly listed emergency manager email addresses in those states received an invitation, yielding a net completion rate of 8.9%. Beyond EMs, a small group of subjects were invited from the Ohio State University Public Safety Leadership Academy, which provides leadership training to senior public safety officials (e.g., chiefs and sheriffs). Student subjects (N = 227) were randomly drawn from the Ohio State University experimental economics subject pool. Oversampling was conducted for upper-division and graduate-level students from disciplines more common to the emergency management field (e.g., public affairs, business, ROTC) to improve representativeness. Subjects were 81% and 44% male for professional and student subjects, respectively (<1% reported transgender or nonbinary). They were 18.5% and 3.5% veteran, respectively, and 67% of the professional subjects reported a college degree or higher.

Results

Stage 1 experiment: Temporal voluntary evacuation decision.

Hurricane forecast information increased voluntary evacuation recommendation times by between 16.6 and 22.8 h, much-needed time for coastal communities to prepare for and implement evacuation measures. By evaluating treatment effects relative to the absence of forecast information (Treatment 1, or T1), a clean estimate of the individual contribution of each key informational parameter can be assessed, that does not preclude intertreatment comparison. At the most basic level and while holding numeracy constant, the social value of the forecast center track (T2) represented an expected value of 16.6 additional hours (p < 0.05) to coastal communities. Adding a cone of uncertainty (T3) extended this time to 19.1 h (p < 0.05), and adding forecasted max wind speed (T4) extended this time to 22.8 h (p < 0.01). The addition of the forecast center track (T2) increased the likelihood of early voluntary evacuation by a magnitude of 2.7 times (n.s.). Addition of the cone (T3) increased the magnitude to 3.1 times (p < 0.10), and addition of forecasted max wind speed (T4) increased this magnitude to 4.4 times (p < 0.05).

These estimates were derived by regression analysis of the experiment data, presented in Table 1. Model 1 presents Tobit and Model 2 presents survival regression estimates using the Cox proportional hazard (PH) model. Tobit models are specifically used to account for censoring effects in the data in which observations are “censored” or cut off at the initial and concluding rounds/advisories (Cameron and Trivedi 2010; Tobin 1958). Survival models (also called “Accelerated Failure Time” or AFT models) are a type of regression model designed for evaluating dependent variables involving time until a binary event (in this case, time until evacuation recommendation). They allow us to treat evacuation as a “failure” event (a statistical term not an evaluative one) where the modeled outcome is the hazard ratio, which is an estimate of the increased likelihood of this event occurring at any given advisory relative to the control. Cox models (Cox 1972; Cox and Oakes 1984) are an advanced class of these models. Results are separated by subject type; similar results are obtained for student subjects. The dependent variable in Model 1 is hours remaining prior to the issuance of the hurricane warning by the NHC (i.e., when early evacuation ends), which is left- and right-censored at start and end advisories. The AFT variable (or dependent variable) for Model 2 is the advisory in which the subject recommended voluntary evacuation. Robustness checks and alternative model specifications are provided in SI-3 at Tables SI-3.5–3.6. Demographics (e.g., sex, age, education, veteran status) were not robust explanatory factors.

Table 1.

Regression estimates for temporal voluntary evacuation. The p values are denoted as follows: one asterisk (*) = p < 0.1, two asterisks (**) = p < 0.05, and three asterisks (***) = p < 0.01. Model 1 reports Tobit regression coefficients with standard errors in parentheses. Dependent variable is hours remaining prior to issuance of hurricane warning by NHC. 32 left-censored observations and 1 right-censored observation were observed in the professional subject model. These values are 72 and 20, respectively, for the student subject model. Model 2 reports Cox PH ratios with Breslow method for tied failures. Accelerated failure time variable is advisory in which subject recommended voluntary evacuation. Both Cox models meet the PH assumption using Schoenfeld’s residuals (p > 0.10).

Table 1.

Importantly, estimates in both models control for subject numeracy. Among professional subjects, numeracy improved hurricane voluntary evacuation recommendation times by an average of 8.8 h (p < 0.10) and increased the hazard ratio by 1.7 (p < 0.10). Numeracy is only statistically significant for professional subjects, which is influenced by the fact that 25% of professionals successfully completed the numeracy assessment compared to 52% of students. We note that this also provides a highly robust validity check of the experimental results—Cokely et al. (2012), who introduced the Berlin Numeracy Test, obtain the exact same percentage accurate in their student sample. We find that numeracy is uncorrelated with education (ρ = 0.01 for college degree, ρ = 0.08 for postgraduate degree) and time taken (Cokely et al. 2018) (see SI-3 Table 3.10). Extrapolating from these results, we observe that more numerate emergency managers are almost twice as likely as less numerate ones to provide additional voluntary evacuation time to their coastal communities regardless of forecast information, and they do so by longer than a typical workday.

Student subjects generally outperformed professional subjects, on average recommending voluntary evacuation approximately 1.3 advisories earlier. Students evacuated, on average, after 6.6 advisories. Professionals did so after 7.9. Students were 8% more likely to make advance recommendation. 31.7% of students and 39.5% of professionals did not recommend voluntary evacuation prior to the NHC warning. Additional evacuation rate details by advisory and treatment are provided in SI-3 (see Table SI-3.9 and Figs. SI-3.1–3.6).

Further refinement of results can be obtained by evaluating the post-experiment survey. Subjects were asked to identify the three informational attributes they relied upon most and then rank-order them. By self-identifying those informational criteria, subjects provided valuable information on the forecast elements that most influenced their decisions. Detailed summary statistics are provided in SI-3, along with a full suite of statistical tests of treatment equality (see Table SI-3.8). Subjects relied most heavily on the current center location in the absence of forecast information.

Of critical importance is the unwavering reliance by subjects on the forecast center track in all forecast treatments. Even with the addition of the cone of uncertainty, the relative weight, or importance of the forecast center track remained the most ascribed informational attribute, and consistently so across subject type (all tests safely fail to reject the null). This is important and comports with Regnier (2008), Wernstedt et al. (2019), Wu et al. (2014), and several others whose findings suggest that both the general public and emergency managers make significant judgment errors by overrelying on the center track in evaluating forecast information. Consequently, after 2006, the NHC began providing separate advisory graphics both including and excluding the center track, allowing site visitors to self-select advisory graphics (Morrow et al. 2015). However, our results indicate the critical importance of one additional caveat.

That is, results indicate that numeracy plays an important role in influencing which informational attributes subjects rely upon. In a comparison of T2 and T3 mean values between subject types, we clearly observe that when provided a cone, professionals place half as much weight as students on it. Recalling that observed student numeracy rates are twice that of professionals, we evaluated the relationship between numeracy and informational attribute importance. Bivariate logistic regression on subject data from forecast treatments reveals a positive and statistically significant relationship (β = 1.121, p < 0.10 for professionals and β = 0.717, p < 0.05 for students) between numeracy and the importance ranking of the cone of uncertainty. Predicted margins from these regressions indicate that subjects who received the cone are 27.2% more likely to rely upon the cone if they are more numerate (34.3% vs 61.5%). For students, this value is 17.5% (46.8% vs 64.3%).

Taken together, these results provide at least some evidence that less numerate decision-makers, as measured by a well-established risk literacy instrument, avoid the cone of uncertainty as an informational determinant. This indicates a predisposition toward relying upon informational attributes that they perceive to be less probabilistic in nature and paying greater attention to attributes such as the forecast center track, which are not directly presented as a function of statistical error.

Stage 2 experiment: Spatial mandatory evacuation decision.

A mandatory evacuation order is an implicit spatial decision that involves two consequences of potential judgment error: over- and underevacuated communities. At the mean, professional subjects evacuated a remaining population (i.e., those remaining after NHC warning) of just over 320,000 persons; students evacuated just over 285,000 persons (difference of 34,000 persons, p < 0.16). While 3% of student subjects chose to evacuate zero communities, all professional subjects evacuated at least one zone: 13.6% and 18.1% of professionals and students, respectively, evacuated only the coastal zone (i.e., the Galveston area). 50.6% and 46.7% of professionals and students, respectively, evacuated both the Coastal and Zone A—the nearest inland zone. Choropleth spatial density maps of evacuation decisions are presented in Fig. 3.

Fig. 3.
Fig. 3.

Spatial density of evacuation decisions. Choropleth maps identify spatial density of mandatory evacuation decisions made by subjects. Zonal division aligns with the Harris County Office of Emergency Management’s established hurricane evacuation planning zones. Numerate/innumerate, which refers to more and less numerate, indicates subject was successful/unsuccessful on the single-item Berlin Numeracy Test. “Human exposure” indicates subject self-identified in post-experiment survey as knowingly overevacuating zone(s) even though they thought the likelihood of inundation to be low.

Citation: Bulletin of the American Meteorological Society 103, 3; 10.1175/BAMS-D-21-0008.1

Stage 2 used a within-subjects design (i.e., all exposed to the same treatment) in which subjects were asked to make a spatial mandatory evacuation order recommendation. From an interactive zone map of the region, subjects selected one of seven possible mandatory evacuation configurations that correspond to established Harris County evacuation areas (see SI 4.2).

All subjects were provided three stormwater inundation projections that coincide with three potential storm track scenarios (i.e., veer-left, center track, veer-right). Post-experiment survey instruments provide further explanatory power. Subjects were asked to identify which of the three projection maps they relied upon most. Only 5% report relying on the optimistic (veer-right) projection—the most accurate relative to actual inundation; 59% report reliance on the center projection.

Subjects were also asked a post-experiment decision-making rationale question that features prominently in statistical robustness tests—if they knowingly chose to evacuate any zones even though they thought the likelihood of human exposure to flooding was low (a variable we refer to as “human exposure”). While 56.5% of subjects reported in the affirmative, the difference between professional subjects and student subjects is not statistically significant on the whole. However, among the 35% of subjects who reported reliance on the veer-left (worst-case) projection, professional and student subjects differ significantly in self-admitting overevacuation (p < 0.02 using a Wilcoxon test). Of these subjects relying on the worst-case projection, 42.8% of professionals and 70% of student subjects self-identified as overevacuating.

Moreover, perceptions were not consistent with evacuated populations. Students evacuated an average of 124,000 persons when self-reporting overevacuating, but this number is 166,000 among professionals. Of those subjects reporting reliance on the veer-left projection, the additional evacuated population for professional subjects associated with self-reported overevacuating is over 200,000 persons, nearly twice the difference for student subjects (119,000). Put simply, when subjects relied upon the worst-case projection, even though professional subjects generally evacuated larger populations than students, they were less likely to admit to overevacuating. However, when doing so, professionals evacuated populations nearly twice the size when relying on the worst-case inundation projection.

Regression analyses provide further explanation. Two main dependent variables were evaluated: 1) a scoring function (detailed in SI-6) that accounts for both over- and underevacuation and 2) the total remaining population mandatorily evacuated (Table 2). While we can confirm the absence of statistical collinearity in the models (meaning there is no pairwise relationship between reliance on one inundation projection over another and other explanatory variables), we also confirm that these inundation projections lack statistical robustness in all models, indicating that reported reliance on one inundation projection over another was less of an influencing factor for evacuation decisions than self-admitted overevacuation (i.e., the human exposure variable). Across the board, this variable is statistically associated with overevacuating populations (negative coefficients in the scoring function models and positive in the population models, indicating over- rather than underevacuation). This generally confirms that subjects knew that they were overevacuating.

Table 2.

Regression estimates for spatial mandatory evacuation. The p values are denoted as follows: one asterisk (*) = p < 0.1, two asterisks (**) = p < 0.05, and three asterisks (***) = p < 0.01. Models 3 and 4 report Tobit regression coefficients with standard errors in parentheses. The dependent variable is the scoring function value described in the methods section that accounts for both over- and underevacuation rates. 3 (1) left-censored observations and 41 (11) right-centered observations in the student (professional) subjects’ models. Models 5 and 6 report ordinary least squares (OLS) estimates in which the dependent variable is the total population evacuated. In each model, the reference category excluded for comparison is subjects reporting greatest reliance on the center inundation map.

Table 2.

We similarly confirm that numeracy reduces overevacuation but is only statistically robust among the student population. While numeracy is not statistically correlated with self-admitted overevacuation, the favorable effects of numeracy offset more than half of the overevacuation effects (67.3% population difference at p < 0.01). Put simply, the adverse social consequences of overevacuation decisions among our sample of student subjects is significantly mitigated by numeracy. This is not the case, however, for professional subjects who knowingly overevacuated larger populations regardless of numeracy.

Discussion of results

Noncompliance with hurricane evacuation orders is a social problem. But therein lies a paradox—observational and experimental studies find that residents who had previously lived through a hurricane are often more likely to be noncompliant (Baker 1991; Meyer et al. 2013). Nearly five decades ago, Windham et al. (1977) referred to this as the “experience–adjustment paradox” and Meyer et al. (2013) refer to this as “false experience effects.” These experimental results might move the proverbial needle in explaining this paradox from the standpoint of informational parameters. Here, results suggest that less numerate decision-makers avoid what they perceive to be probabilistic forecast information, and the consequence of this is less advance warning to communities—by more than a typical workday. Results also indicate that numeracy can play a role in significantly reducing the predilection to knowingly overevacuate large areas of remaining populations. If residential populations come to expect that their public leaders’ risk considerations are asymmetric from their own, the important public trust that is necessary to avoid loss of life in major disasters can be called into question. Escalation effects can promulgate if public safety leaders feel the need to scare increasingly distrustful populations into compliance.

These experimental results also serve to address a long-established conundrum in presenting tropical cyclone forecasts—the overreliance on the forecast center track by both residential populations and public safety officials (Broad et al. 2007; Meyer et al. 2013; Sherman-Morris and Antonelli 2018). Our results highlight the fact that it is not necessarily that decision-makers are overreliant on the track line, but instead, the less numerate disregard what is presented as a function of statistical error (i.e., the cone). This result calls into question the judgment of media and others who, for more than a decade now, have begun to selectively omit the forecast center track and present only the cone. What remains is what is otherwise disregarded by many less numerate decision-makers. Given the absence of consensus regarding the presentation of scientific uncertainty to public safety practitioners, further research is needed to improve the presentation of probabilistic information.

This experiment has made several methodological advancements beyond existing experimental research in this domain. These include integration of endowment generation and scoring functions, overcoming the counterfactual challenge that exists in hypothetical experiments, use of a well-established numeracy metric, and integration of a professional subject population. One implication regarding the subject population deserves to be highlighted. Whereas existing experiments have consistently relied upon student subject pools to extrapolate public safety decisions, by including both student subjects and professional decision-makers, side by side, our results paint in stark relief the potential validation challenge that is present in existing experimental works relying solely on student subjects.

Further research like ours is needed in this domain to extend these findings. This can include integration of additional physical and hydrodynamic forecast properties, including storm surge and infrastructure (e.g., levee) fragility modeling and probabilistic wind conditions. This can include simultaneous interactive experimental designs that integrate residential and public safety decisions to gauge escalation effects. This can also include eye-tracking studies to observe forecast parameters receiving the most visual attention. And, while this experiment was designed to provide greater analytical depth of a single storm, future research can extend this work to other trajectory classifications and perils.

While we caution against monolithic policy guidance on the basis of a single study, the results of this research motivate pragmatic policy questions deserving greater attention. These include the degree to which merit-based public safety decisions are moderated by, or influenced by additional social or contextual factors such as legal risk aversion (Nicholson 2007; Wilson and McCreight 2012). Public safety professionals enjoy a degree of insulation from legal liability that may shape their decisions involving risk. As we identify a predilection toward overevacuation in the absence of risk literacy, further research may be warranted to strike the appropriate balance between insulating those decision-makers and building accountability for moral hazard.

Additionally, this debate involving the presentation of scientific information is taxing for scientists who bear a disproportionate responsibility in this domain. One may argue that they are unequally yoked, responsible not only for developing important forecasts for public safety, but also responsible for ensuring that those forecasts are interpreted correctly in the absence of requisite scientific literacy, such as numeracy and ultimately risk literacy. An important public debate needs to occur to identify the relative responsibilities of both the scientific community and public sector leaders informed by that community.

1

Utilization of Hurricane Rita followed a consultative criteria-driven selection process that included review of numerous alternatives, consultation with engineering experts in storm surge and hydrodynamic modeling, and an informal interview with the Emergency Manager of Harris County, Texas. Our criteria required the storm to be of sufficient age and relatively low salience to prevent recall identification. They required the storm to have a relatively common track to further obfuscate recall identification. They required the storm to be a category 5 that made landfall. For ease of presentation to subjects, they required the storm to be nonmultijurisdictional (i.e., at-risk populations limited to a single state). Whereas prior evacuation experiments presented subjects with only straight-line forecasts (Wu et al. 2014), we identified a strong preference for a curvilinear track to improve external validity. Rita met each of these characteristics.

2

Replication link and instructions are provided in SI-2. A description of the random assignment algorithm is provided in SI-5.

3

Cokely et al.’s (2012) assessment is the following: Out of 1,000 people in a small town 500 are members of a choir. Out of these 500 members in the choir 100 are men. Out of the 500 inhabitants that are not in the choir 300 are men. What is the probability that a randomly drawn man is a member of the choir?

4

Post-voluntary evacuation population remaining percentages were estimated in Dow and Cutter (2002), Fu et al. (2007), Pham et al. (2020), and Whitehead et al. (2001), with them collectively finding approximately 15%–45% voluntary evacuation compliance rates. See SI-7 for detailed methodology for estimating remaining at-risk population.

5

These included AL, DE, FL, GA, LA, MD, NC, NJ, NY, SC, and VA. Emergency managers represent 74% of the professional subject population.

Acknowledgments.

The authors are grateful for research support and assistance from Darryl Anderson, Antonio Gil De Rubio-Cruz, Mehrzad Rahimi, Abdollah Shafieezadeh, Ethan Kubatko, Coral Wonderly, Kelly Lash, Tim Bailey, and Sam Stelnicki. This work was supported by National Science Foundation Grant 1563372.

Data availability statement.

All data and code necessary for replication will be published to NSF DesignSafe and available from the authors.

References

  • Baker, E. , 1991: Hurricane evacuation behavior. Int. J. Mass Emerg. Disasters, 9, 287310.

  • Baker, E. , 1995: Public response to hurricane probability forecasts. Prof. Geogr., 47, 137147, https://doi.org/10.1111/j.0033-0124.1995.00137.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blome, R. , 2005: Staying? Better write your SS# on your body… NBC News, 22 September, www.nbcnews.com/id/9439537/ns/us_news-katrina_the_long_road_back/t/staying-better-write-your-ss-your-body/#.X1_QE5NKiYU.

  • Broad, K. , A. Leiserowitz, J. Weinkle, and M. Steketee , 2007: Misinterpretations of the “cone of uncertainty” in Florida during the 2004 hurricane season. Bull. Amer. Meteor. Soc., 88, 651668, https://doi.org/10.1175/BAMS-88-5-651.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cameron, A. C. , and P. K. Trivedi , 2010: Microeconometrics Using Stata. Stata Press, 706 pp.

  • Christensen, L. , and C. E. Ruch , 1980: The effect of social influence on response to hurricane warnings. Disasters, 4, 205210, https://doi.org/10.1111/j.1467-7717.1980.tb00273.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cokely, E. , M. Galesic, E. Schulz, S. Ghazal, and R. Garcia-Retamero , 2012: Measuring risk literacy: The Berlin numeracy test. Judgment Decis. Making, 7, 2547, https://doi.org/10.1037/t45862-000.

    • Search Google Scholar
    • Export Citation
  • Cokely, E. , A. Feltz, S. Ghazal, J. N. Allan, D. Petrova, and R. Garcia-Retamero , 2018: Skilled decision theory: From intelligence to numeracy and expertise. The Cambridge Handbook of Expertise and Expert Performance, K. A. Ericsson et al., Eds., Cambridge University Press, 476505.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cox, D. R. , 1972: Regression models and life-tables. J. Roy. Stat. Soc., B34, 187202, https://doi.org/10.1111/j.2517-6161.1972.tb00899.x.

    • Search Google Scholar
    • Export Citation
  • Cox, D. R. , and D. Oakes , 1984: Analysis of Survival Data. Chapman and Hall, 201 pp.

  • Dash, N. , and H. Gladwin , 2007: Evacuation decision making and behavioral responses: Individual and household. Nat. Hazards Rev., 8, 6977, https://doi.org/10.1061/(ASCE)1527-6988(2007)8:3(69).

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dawson, C. , E. J. Kubatko, J. J. Westerink, C. Trahan, C. Mirabito, C. Michoski, and N. Panda , 2011: Discontinuous Galerkin methods for modeling hurricane storm surge. Adv. Water Resour., 34, 11651176, https://doi.org/10.1016/j.advwatres.2010.11.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dow, K. , and S. L. Cutter , 2002: Emerging hurricane evacuation issues: Hurricane Floyd and South Carolina. Nat. Hazards Rev., 3, 1218, https://doi.org/10.1061/(ASCE)1527-6988(2002)3:1(12).

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Drake, L. , 2012: Scientific prerequisites to comprehension of the tropical cyclone forecast: Intensity, track, and size. Wea. Forecasting, 27, 462472, https://doi.org/10.1175/WAF-D-11-00041.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fu, H. , C. G. Wilmot, H. Zhang, and E. J. Baker , 2007: Modeling the hurricane evacuation response curve. Transp. Res. Rec., 2022, 94102, https://doi.org/10.3141/2022-11.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gudishala, R. , and C. Wilmot , 2017: Modeling emergency managers’ hurricane evacuation decisions. Transp. Res. Rec., 2604, 8287, https://doi.org/10.3141/2604-10.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoss, F. , and P. Fischbeck , 2016: Increasing the value of uncertain weather and river forecasts for emergency managers. Bull. Amer. Meteor. Soc., 97, 8597, https://doi.org/10.1175/BAMS-D-13-00275.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huang, S.-K. , M. K. Lindell, and C. S. Prater , 2016: Who leaves and who stays? A review and statistical meta-analysis of hurricane evacuation studies. Environ. Behav., 48, 9911029, https://doi.org/10.1177/0013916515578485.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kahneman, D. , and A. Tversky , 1979: Prospect theory: An analysis of decision under risk. Econometrica, 47, 263, https://doi.org/10.2307/1914185.

  • Keneally, M. , 2017: Mayor tells those not evacuating for hurricane to put Social Security numbers on arms. ABC News, 25 August, https://abcnews.go.com/US/mayor-tells-evacuating-hurricane-harvey-put-social-security/story?id=49417573.

    • Search Google Scholar
    • Export Citation
  • Knabb, R. , J. Rhome, and D. Brown , 2011: Tropical Cyclone Report: Hurricane Katrina 23–30 August 2005. National Hurricane Center, 43 pp., www.nhc.noaa.gov/data/tcr/AL122005_Katrina.pdf.

    • Search Google Scholar
    • Export Citation
  • Kubatko, E. J. , J. J. Westerink, and C. Dawson , 2006: hp discontinuous Galerkin methods for advection dominated problems in shallow water flow. Comput. Methods Appl. Mech. Eng., 196, 437451, https://doi.org/10.1016/j.cma.2006.05.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Loomis, J. , 2011: What’s to know about hypothetical bias in stated preference valuation studies?: Hypothetical bias in stated preference valuation studies. J. Econ. Surv., 25, 363370, https://doi.org/10.1111/j.1467-6419.2010.00675.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Losee, J. E. , K. Z. Naufel, L. Locker, and G. D. Webster , 2017: Weather warning uncertainty: High severity influences judgment bias. Wea .Climate Soc., 9, 441454, https://doi.org/10.1175/WCAS-D-16-0071.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mele, C. , 2016: How to get people to evacuate? Try fear. The New York Times, 8 September 2017, www.nytimes.com/2016/10/07/us/how-to-get-people-to-evacuate-try-fear.html.

    • Search Google Scholar
    • Export Citation
  • Meyer, R. , K. Broad, B. Orlove, and N. Petrovic , 2013: Dynamic simulation as an approach to understanding hurricane risk response: Insights from the Stormview lab. Risk Anal., 33, 15321552, https://doi.org/10.1111/j.1539-6924.2012.01935.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Morrow, B. H. , J. K. Lazo, J. Rhome, and J. Feyen , 2015: Improving storm surge risk communication: Stakeholder perspectives. Bull. Amer. Meteor. Soc., 96, 3548, https://doi.org/10.1175/BAMS-D-13-00197.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nicholson, W. C. , 2007: Emergency planning and potential liabilities for state and local governments. State Local Gov. Rev., 39, 4456, https://doi.org/10.1177/0160323X0703900105.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • NOAA, 2005: Hurricane Rita images. NOAA National Geodetic Survey, https://geodesy.noaa.gov/storm_archive/storms/rita/index.html.

  • Peerbolte, S. L. , and M. L. Collins , 2013: Disaster management and the critical thinking skills of local emergency managers: Correlations with age, gender, education, and years in occupation. Disasters, 37, 4860, https://doi.org/10.1111/j.1467-7717.2012.01291.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peters, E. , D. Västfjäll, P. Slovic, C. K. Mertz, K. Mazzocco, and S. Dickert , 2006: Numeracy and decision making. Psychol. Sci., 17, 407413, https://doi.org/10.1111/j.1467-9280.2006.01720.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pham, E. O. , C. T. Emrich, Z. Li, J. Mitchem, and S. L. Cutter , 2020: Evacuation departure timing during Hurricane Matthew. Wea. Climate Soc., 12, 235248, https://doi.org/10.1175/WCAS-D-19-0030.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Regnier, E. , 2008: Public evacuation decisions and hurricane track uncertainty. Manage. Sci., 54, 1628, https://doi.org/10.1287/mnsc.1070.0764.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sherman-Morris, K. , and K. B. Antonelli , 2018: Hurricane knowledge and interpretation of forecasted error cone and wind potential graphics. J. Emerg. Manage., 16, 137148, https://doi.org/10.5055/jem.2018.0363.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, V. , 1976: Experimental economics: Induced value theory. Amer. Econ. Rev., 66, 274279.

  • Thompson, R. R. , D. R. Garfin, and R. C. Silver , 2017: Evacuation from natural disasters: A systematic review of the literature. Risk Anal., 37, 812839, https://doi.org/10.1111/risa.12654.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tobin, J. , 1958: Estimation of relationships for limited dependent variables. Econometrica, 26, 2436, https://doi.org/10.2307/1907382.

  • Tversky, A. , and D. Kahneman , 1981: The framing of decisions and the psychology of choice. Science, 211, 453458, https://doi.org/10.1126/science.7455683.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • USACE, 2008: Flood insurance study: Southwestern Parishes, Louisiana. Intermediate Submission 2, USACE, 697 pp.

  • USACE, 2011: Flood insurance study: Coastal counties, Texas. Intermediate Submission 2, USACE, 150 pp.

  • Wallace, J. W. , C. Poole, and J. A. Horney , 2016: The association between actual and perceived flood risk and evacuation from Hurricane Irene, Beaufort County, North Carolina. J. Flood Risk Manag., 9, 125135, https://doi.org/10.1111/jfr3.12115.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wernstedt, K. , P. S. Roberts, J. Arvai, and K. Redmond , 2019: How emergency managers (mis?)interpret forecasts. Disasters, 43, 88109, https://doi.org/10.1111/disa.12293.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whitehead, J. C. , B. Edwards, M. Van Willigen, J. R. Maiolo, K. Wilson, and K. T. Smith , 2001: Heading for higher ground: Factors affecting real and hypothetical hurricane evacuation behavior. Environ. Hazards, 2, 133142, https://doi.org/10.3763/ehaz.2000.0219.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilson, L. R. , and R. McCreight , 2012: Public emergency laws & regulations: Understanding constraints & opportunities. J. Homeland Secur. Emerg. Manage., 9, 7, https://doi.org/10.1515/1547-7355.2034.

    • Search Google Scholar
    • Export Citation
  • Windham, G. , E. Posey, and B. Spencer , 1977: Reactions to storm threat during Hurricane Eloise. Tech. Rep. 51, Mississippi State University, 74 pp.

    • Search Google Scholar
    • Export Citation
  • Wu, H.-C. , M. K. Lindell, C. S. Prater, and C. D. Samuelson , 2014: Effects of track and threat information on judgments of hurricane strike probability. Risk Anal., 34, 10251039, https://doi.org/10.1111/risa.12128.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wu, H.-C. , M. K. Lindell, and C. S. Prater , 2015a: Process tracing analysis of hurricane information displays. Risk Anal., 35, 22022220, https://doi.org/10.1111/risa.12423.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wu, H.-C. , M. K. Lindell, and C. S. Prater , 2015b: Strike probability judgments and protective action recommendations in a dynamic hurricane tracking task. Nat. Hazards, 79, 355380, https://doi.org/10.1007/s11069-015-1846-z.

    • Crossref
    • Search Google Scholar
    • Export Citation

Supplementary Materials

Save