1. Introduction
Over the past century, annual U.S. tornado deaths have declined because of advancements in the monitoring and forecasting of meteorological phenomena, as well as the ability to disseminate warning information via multiple media platforms (Ashley 2007; Simmons and Sutter 2005, 2008). Yet, research from several recent tornado events suggests that even though warning messages are widely broadcast and received, the ways in which people interpret and act upon these warnings may be inconsistent with communicators’ intended meanings and recommended responses (Donner et al. 2012; Klockow 2011; Montz 2012; Nagele and Trainor 2012). Thus, there is much room for improvement in the communication of tornado warnings, and further research that contributes to this goal is needed [Lazo 2012; Lindell and Brooks 2013; (National Research Council) NRC 2006].
The U.S. National Weather Service (NWS) began issuing storm-based warnings (SBWs) in 2007 (Coleman et al. 2011). Previously, the NWS issued severe thunderstorm and tornado warnings by county, meaning that even if a tornado was expected to impact only a very small section of a county, the entire county would be warned nevertheless (Waters et al. 2005). SBWs are more geographically specific and are depicted visually as trapezoid or polygon shapes encompassing the areas that could experience a tornado over the duration of the warning period (usually 30–45 min). An SBW is drawn in tandem with accompanying warning text; however, this text is not always received together with a visual representation of the warning. SBWs are deterministic in that they convey a uniform, elevated probability of occurrence within the polygon. Despite inherent uncertainty in the warning in terms of the presence of a tornado, its location, and its future movement, this uncertainty is not explicitly represented visually.
The primary advantage of the storm-based format is geographic specificity, reducing the size of the area and number of people being warned. Sutter and Erickson (2010) argue that SBWs reduce the amount of time spent sheltering unnecessarily, and that the time-cost savings using SBWs compared to county warnings could amount to hundreds of millions of dollars. SBWs are also advantageous in that they may be used to target the delivery of warning messages to individuals within a warned area (Jacks and Ferree 2007). However, Nagele and Trainor (2012) report that being located within a SBW may not significantly increase protective action behaviors. Clearly, the effectiveness of SBWs in targeting warning messages and spurring protective action is not fully proven. Nevertheless, SBWs are routinely disseminated via the Internet and mobile devices (Casteel and Downing 2013; Mass 2012; Sherman-Morris 2010). Because SBWs are increasingly available in visual formats without a meteorologist to aid interpretation, it is imperative to understand how nonexperts interpret SBWs.
As the public is becoming more familiar with SBWs, meteorologists are already researching ways to enhance SBWs to include probabilistic information. These probabilistic warnings anticipate a shift from warnings based on radar and spotter detection to warnings based on forecast model outputs (Hoekstra et al. 2011; Stensrud et al. 2013). It is not known whether tornado warning effectiveness will be enhanced or diminished by including probabilistic information. When no uncertainty information is given in temperature and precipitation forecasts, users form mental representations of uncertainty according to their own experiences or perceptions (Morss et al. 2008, 2010). Similarly, nonmeteorologists’ mental schemas of uncertainty for visual representations of tornado warnings may be very different from those of meteorologists. Thus, it is prudent to investigate how the general public interprets the current deterministic-style SBW visual format and begin to understand how a change from a deterministic-style to a probabilistic-style visual format might alter reactions and behavioral responses of tornado warnings.
This research tests how three different tornado warning visual displays are perceived using a series of hypothetical tornado warning situations. The experimental probabilistic-style SBW visual formats present probability gradients using two different color schemes, while the current deterministic-style SBW visual format is characterized as a uniform area where there is elevated probability of a tornado. The study poses the following research questions: 1) Do fear or protective action responses vary across different warning designs? 2) Where do high fear and protective action responses cluster within each warning design? 3) Is there a decreasing trend in responses from the zone of highest probability toward the edges of the warning for each design? 4) Are there differences between responses inside versus outside the warned area? 5) Are there differences between responses just inside the edge of the warning and those outside?
2. Relevant literature
a. Risk communication, risk perception, and protective action
A tornado warning is a form of risk communication that relies upon imminent danger and fear of injury as the primary motivations for protective action during a crisis situation (Lundgren and McMakin 2009). While risk is conceptualized in various ways in the physical and social sciences, its definition in risk and hazards research often involves at least two basic components: the probability that a hazardous event will occur and the probability of consequences or impacts stemming from that event (Bostrom et al. 2008; Haimes 2009). Studies fully addressing the perception of any risk or threat and subsequent behavior, following from the above definition, should therefore investigate the perception of the probability of a hazardous event and the perception of potential consequences. The study presented here focuses only on the perception of the probability of a specific hazard or threat—a tornado—communicated solely through visual means. The assumption is that, given a higher perceived probability of a tornado, participants will indicate that they would be more afraid, and thus, more likely to take protective action.
The process of receiving and understanding risk communication messages and making a decision to take protective action is quite complex. Some researchers highlight the importance of the sources and components of warning messages (Lindell and Perry 2012; Mileti and Peek 2000; Sorensen 2000). Many behavioral risk models include individual characteristics such as personality, prior experience, gender, and self-efficacy as important factors that influence a decision-making process (Fishbein 2008; Lindell and Perry 2012; Maloney et al. 2011). Between the time a warning is received and any subsequent behavioral response is taken, analytical reasoning and emotions together play an important intervening role in forming beliefs or perceptions about the hazard and its potential consequences (Fishbein 2008; Lindell and Perry 2012; Sjöberg 2007).
A useful conceptual framework was recently proposed by Severtson (2013) that takes into account the important factors discussed above, and does so in the context of how environmental maps can influence risk beliefs and behavior. Severtson’s Integrated Representational and Behavioral Framework (IRBF) posits that behaviors follow from risk beliefs formed through both cognitive and emotional representations in response to a visual representation of risk. An individual’s comprehension of a visual representation is mediated by visual features on the map or graphic itself, by the individual’s personal characteristics, and by the steps that person undertakes to consciously and subconsciously process the visual representation. Cognitive or analytical representation influences the comprehension of information and the forming of perceptions or beliefs about the implications of the information. This entails thinking about specific aspects of the hazard itself, one’s ability to mitigate the consequences if the potential hazard event occurs, and what other people are doing or saying about the situation (Fishbein 2008; Lindell and Perry 2012; Maloney et al. 2011).
Emotional or affective representation of risk is complementary to cognitive representation (Sjöberg 2007). It involves the development of feelings, both conscious and subconscious, as well as attitudes toward the hazard and toward actions that may be taken to mitigate consequences associated with the hazard (Loewenstein et al. 2001; Slovic et al. 2004).
b. Intentions and behavior
Linking risk representations to subsequent actual behaviors is not always feasible in real time or even in postevent analyses, particularly when the research context is a dangerous and rapidly changing phenomenon such as a tornado. There is evidence from human behavioral studies to support the claim that intended behavior can reasonably be assumed to predict actual behavior (Fishbein 2008; Sheeran 2002; Whitehead 2005; Wogalter et al. 2002). However, Webb and Sheeran (2006) note that a moderate-to-large change in intentions only leads to a small-to-moderate change in actual behavior. The strength of the intention–behavior relationship is reduced further when unexpected impediments arise that can stymie a predetermined plan (Lindell and Perry 2012; Webb and Sheeran 2006). It is unknown the extent to which protective action intentions stated for a hypothetical tornado warning might translate to a real warning situation. This is an avenue for further investigation and improvement in research design for weather risk communication and behavioral studies (Meyer et al. 2013).
c. Visual representation of risk
Environmental hazards of all types can be represented visually through graphs, chart diagrams, and maps. Several factors facilitate a person’s ability to comprehend what is visually depicted. Severtson (2013) considers several of these factors including four concepts from Pinker’s work on graph comprehension, a number of preattentive properties of features, and underlying semiotic properties of these features.
1) Pinker’s factors
Pinker (1990) outlines four concepts that influence the ability to understand a graphical display: units of perception and their location, Gestalt principles, representation of magnitude, and the coordinate system. Distinguishable units of perception and their configuration in space influence the ability to compare and contrast values assigned to the units. Gestalt principles suggest that similarity, proximity, common fate (i.e., equivalent attribute or value), and continuity of visual features influence how visual representations are interpreted as a whole. Magnitude, which assists organization of features into a hierarchy, may be represented as absolute or relative depending upon the data and purpose of the graphic. Coordinate systems also aid in visual organization, and they may be based on relative or absolute reference frames.
These concepts can be useful in explaining how people understand risk in maps or graphical displays. For instance, map readers draw different conclusions about water contamination levels when using a hazard map with points as the units of perception rather than administrative units such as townships (Severtson and Vatovec 2012). Also, when contours are used in environmental hazard maps to depict the magnitude of contamination, they elicit the strongest risk beliefs in areas where the magnitude is high (Severtson and Myers 2013). Pinker’s factors are also likely to be useful in understanding comprehension of tornado warning visual representations.
2) Preattentive properties
Preattentive properties influence subconscious processing of visual features that happens prior to cognitive processing where the primary meaning of the representation is assessed (Cleveland and McGill 1984). These properties include length or proximity, direction, shape or curvature, area, shading or color saturation, and position on a scale or map, among others. Visual salience, the quality of a representation that aids to quickly focus attention on important features, can be enhanced through the use of preattentive properties such as high-contrast shading or color schemes, delineation of regions with sharp bordering, or by suggesting directionality using a particular shape (Severtson and Vatovec 2012; Sherman-Morris 2005; Wogalter et al. 2002).
Unfortunately, preattentive properties that enhance visual salience also hold the potential to distort the intended message of a visual representation. For example, in their study of graphical water contamination reports, Severtson and Henriques (2009) demonstrate that a position on a graphical scale just below a water contamination threshold level tended to be perceived as much less dangerous than a position the same distance above the threshold. In a meteorological context, the inclusion of a centerline in a hurricane track forecast graphic results in a narrow focus on the likely path of the eye of the storm (Broad et al. 2007). Such a narrow focus detracts from the cognition of other possible storm paths and from the fact that severe impacts may extend far from the eye of a hurricane.
Visual salience problems may also arise when there is a lack of features with preattentive properties. Two recent instances relating to visual representations of SBWs during the April 2011 tornado outbreak exemplify this issue. Alabama residents did not fully comprehend whether they were in danger because static television maps did not help viewers infer thunderstorm directions of movement and did not include recognizable features to help viewers pinpoint their own locations (Klockow 2011). In another study, Mississippi residents indicated that the geographic center of a tornado warning should be more likely to experience a tornado than near or outside the perimeter (Sherman-Morris and Brown 2012). Seemingly, the deterministic-style polygon did not assist them to infer that over time the direction of movement would increase the likelihood of a tornado near the warning edge.
3) Semiotic properties
Semiotics refers to properties of features that imbue certain meanings derived from sociocultural contexts (MacEachren 2004). For instance, color has an important role in influencing human perception and behavior; however, this influence is specific to the cultural and psychological contexts in which certain colors are used (Elliot and Maier 2007; Meier et al. 2012). Many authors agree that red has semiotic value for communicating danger—particularly in North American and European cultural contexts (Bostrom et al. 2008; Griffith and Leonard 1997; Leonard 1999; Wogalter et al. 2002). Yellow frequently connotes caution, whereas green and blue are associated more often with safety (Elliot and Maier 2007; Severtson and Vatovec 2012). Given the danger context of a tornado warning, the semiotic properties of color are an important consideration for designing future probabilistic-style formats.
3. Methods
a. Warning design
To address the five research questions, three different warning designs are constructed and tested. The first warning design—referred to as the original design—is based on the current NWS warning format. It is deterministic in nature, as it visually represents a simple elevated probability of occurrence of a tornado within the outlined geographic area. Similar to current tornado warning polygons, which may be shaded solid red, hatched red, or presented as a red-outlined area, the original design in this study uses a single, red outline (Fig. 1a).
Three warning designs tested in this study.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00021.1
Two other warning designs anticipate future communication of tornado warnings in probabilistic terms. Each of these probabilistic-style designs visually divides the warned area into several zones of decreasing tornado probability. One depicts 10 probability zones using a spectral color scheme (Fig. 1b), while the other depicts 5 zones using a red color gradient (Fig. 1c). The motivation for testing a spectral color scheme is twofold. First, warning visuals produced by the National Oceanic and Atmospheric Administration’s (NOAA’s) Experimental Warning Program (Kuhlman et al. 2009) include spectral designs. Second, several extant studies suggest that using a familiar color scheme reliant on differences in hue may increase overall visual salience, decrease the time needed for cognitive processing of the visual, and shorten the time needed for completion of attendant actions in response to the visual information (Brewer 1997; Hoffman 1991; Hoffman et al. 1993). Inclusion of a red gradient design in the study is supported by cartographic convention arguing for the use of several shades of one hue to denote differences in quantity for a single phenomenon, with darker shades representing larger quantities (Monmonier 1999; Slocum et al. 2005). The semiotic value of red as associated with danger is the rationale for this color choice in both the original and red gradient designs.
The three warning designs have several elements in common that either facilitate comparison of results between them or serve as necessary simplifications due to the hypothetical nature of the experiment. The first of these elements is the warning shape. The narrowness of the warning at one end is meant to convey the current location of a possible tornado, while the gradually widening shape is intended to inherently communicate the uncertainty of the tornado’s future location. The rounded warning shape is a generalization anticipating future reliance on a smoothed tornado path probability model. These design elements mimic those of hurricane track-based cones of uncertainty (Broad et al. 2007), though in our warning designs the centerline is omitted to keep consistent with current SBW designs. Because these warning designs are not based on an actual or modeled tornado event, a simple symmetrical warning design was deemed the most defensible as a starting point. It must be acknowledged, however, that variations in supercell structure, direction of movement, and mesoscale environmental conditions may lead to tornado probability models with asymmetrical shapes. The warning designs represent a southwest-to-northeast direction of movement due to the high frequency of this track orientation in the United States; this is another necessary simplification as individual tornado motions are highly variable. Each of the warning designs also features a compass rose and scale bar as basic spatial references to orient viewers (Dymon and Winter 1993).
Probability zones are also drawn to compare responses across the designs. Differing magnitudes of probability are shown with changes in color or shade (Cleveland and McGill 1984; Pinker 1990). The location of the two highest probability zones on the spectral design coincides with the highest probability zone on the red design, and so forth. Though it would have enabled a one-to-one comparison, creating a red gradient design with 10 probability zones was deemed a detriment to visual salience (Slocum et al. 2005), hence, the 5 probability zones and a two-to-one comparison. Collocating probability zones facilitates evaluation of research questions (RQs) 3, 4, and 5 regarding the decreasing trend in viewer responses and differences in responses around the edges of the designs. For these research questions, the original design that does not include probability zones acts as an experimental control.
b. Experimental design
This study relied upon a large-sample survey of University of South Carolina students (n = 501). The independent variable being tested is a series of distinct tornado warning scenarios. Each scenario includes the combination of one warning design with a hypothetical position (i.e., a location inside or just outside the warning design) (Fig. 2). When presented with these two pieces of information, participants used a five-point Likert scale (1 to 5) to answer the following questions: 1) if you were located at the dot, how afraid would you be for your life and property? 2) If you were located at the dot, how likely would you be to take protective action (e.g., go to a basement or interior room; leave a mobile home or vehicle for sturdy shelter; lie in a ditch)? A Likert score of 5 represented the highest fear or the greatest likelihood of protective action. Thus, fear and intention to protect oneself are the two dependent variables assessed.
Example hypothetical warning scenario and corresponding survey questions. Actual images in questionnaires were in color (Fig. 1).
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00021.1
The study is designed to compare viewer’s behavioral intentions based on visual representations of a tornado warning scenario. Warning designs and positions tested vary visual features such as proximity, magnitude, color, and others contained in Severtson’s (2013) IRBF. The question about fear is intended to target the interaction of cognitive and emotional representations in the hypothetical scenario. The question about behavioral intention is an approximation of actual behavior in the absence of an actual tornado warning event. It should be noted, however, that the authors do not consider fear and protective action responses to be sufficient to fully represent risk perception as defined earlier in the paper. To do so would require participants to more comprehensively express cognitive and emotional representations of both the probability and consequences of a tornado event. In this study, risk or threat perception is simplified to probability perception as implied by the viewer’s answers to fear and protective action questions.
c. Data collection
Study participants completed a 5-page, 24-item questionnaire within 15 min, after which educational handouts on tornado watches, warnings, and common myths were distributed. Students were recruited from geography and mass communications courses in the fall semester 2011, and their participation was on a voluntary basis. Of the 501 surveys collected, 480 surveys were complete; however, not one of the returned surveys was entirely unusable, and therefore responses from all 501 surveys are included in the analysis. Participants were 54% female and 46% male with a median age of 19 years. Eighty-three percent of participants identified as white, 10% as black, 5% as Asian, and 3% as Hispanic. In total, 94% of participants were from east of the Rocky Mountains, and 63% were from South Carolina.
The students viewed 11 hypothetical tornado warning scenarios, each placing the participant at a different position in or near one of the three warning designs. The two questions on fear and intent to take protective action were posed for each of the 11 scenarios. The warning design was held constant for each individual survey, such that participants who rated scenarios for a spectral warning design did not rate any scenarios for the original or red gradient designs. This was done to ensure that viewer responses to one design would not be influenced by viewing another.
The 11 positions on each survey were drawn from a grid of 352 positions (Fig. 3), spread over and around each warning graphic at an interval of 2 km. The points extended beyond the visual boundaries of the warning area by 3 km, which was necessary to address RQs 4 and 5 about responses inside and outside the warning design. Through random sampling of positions, a total of 32 unique sets of 11 hypothetical positions per warning design were generated. The large sample size meant that at least four participants saw each of the 352 positions for each of the three warning designs.
Grid of 352 hypothetical positions at which fear and protective action responses are measured.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00021.1
A Kendall’s Tau test (α = 0.05) is used to determine if the ordering of the warning scenarios influenced responses. Test results suggest respondents gave slightly lower ratings toward the end of the questionnaire. However, the effect sizes were small for spectral (τ = −0.11 for fear and τ = −0.127 for protective action) and red gradient designs (τ = −0.11, τ = −0.109), and not significant for the original design (τ = 0.02, τ = −0.02).
d. Hypotheses and statistical analyses
To compare responses between the three warning designs (RQ 1), fear and protective action responses are analyzed using Mann–Whitney U (MWU) tests in the statistical package SPSS. The MWU test is the nonparametric equivalent of the independent samples t test, and the test can be applied to ordinal level data such as the Likert responses given by participants in this study (Burt et al. 2009). While the MWU test may be said to test differences in medians between independent samples, mean values are reported in text and tables here because they are more easily interpreted by readers. It is anticipated that there will be significant differences in the mean responses to each of the graphics tested. The red gradient design is expected to result in higher fear and protective action responses because of its explicit visual representation of probability levels as well as the semiotic power of the color red.
The Getis–Ord Gi* statistic (Getis and Ord 1992; Ord and Getis 1995) is a standardized statistic that iteratively compares values of a variable within a specified distance of each spatial observation unit to values of that variable at all units across the entire study area. Here the statistic is calculated using ArcGIS 10.0 to identify spatial clusters of the strongest and weakest fear and protective action responses (RQ 2). It is also widely used in crime and disease mapping to identify areas where higher or lower incidence rates are clustered together spatially (e.g., Frazier et al. 2013; Winters et al. 2010). In this study, the mean response at each of the 352 positions (Fig. 3) is first calculated. A threshold distance of 2.82 km is then specified in order to capture the point of interest along with all eight cardinal and diagonal neighboring points. After the Gi* statistic is calculated, statistical significance at each point is evaluated as with z scores, wherein positive values above 1.96 indicate clusters of high responses and values less than −1.96 indicate clusters of low responses. It is anticipated that the probabilistic-style designs will show stronger spatial clustering of higher fear and protective action responses following the visual patterns of warmer hues and darker shades along the implied track centerline. It is hypothesized that responses to the original design will show little to no clustering because this design does not visually represent a tornado probability gradient, but only a binary between warned area and nonwarned area.
Five probability zones (Fig. 4) delineate groups of positions whose responses can be compared to determine, for each warning design, if responses decrease from where the probability of a hypothetical tornado would be highest (zone 5) to where it would be lowest (zone 1) (RQ 3). Mean responses in each of the five zones are first computed to determine whether the fear and protective action response values have some relationship to the probability zones. A linear-by-linear association chi-square test is appropriate to determine whether statistical power is gained by looking for a linear trend in the relationship. Since both the zones and the Likert responses are ordinal, this test provides more statistical power than a Pearson’s chi-square test (Agresti 2002; Howell 2013). It is anticipated that all warning designs will display a decreasing linear trend in response values from zone 5 to 1, though stronger decreasing trends are expected for probabilistic-style designs than for the original design.
Five probability zones depict tornado probability levels with respect to the hypothetical tornado. Zone 5 represents the highest probability, while zone 1 represents the lowest probability.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00021.1
The mean response to positions outside the warning design is compared to the mean response inside in two ways. First, to ascertain whether the response to a position anywhere inside the warning differs from anywhere outside, MWU tests (Burt et al. 2009) compare the distributions (RQ 4). Second, MWU tests compare responses for inside positions only in zone 1 (the zone nearest the warning’s outer edge) with the responses for positions outside the warning design. In this way, responses may be tested to understand how the warning design’s outer boundary, itself a graphically represented probability threshold, affects viewers’ behavioral intentions (RQ 5). Significant differences are anticipated between responses inside the warning and responses from outside across all warning designs. In comparing responses in zone 1 to those outside, significant differences are expected only for the original graphic because the single red outline provides the only visual probability threshold for viewers to interpret.
4. Results
a. RQ 1: Differences between warning designs
Overall, protective action responses to all three designs average slightly higher than fear responses. The original warning design elicits the highest responses for fear (mean = 3.21) and protective action (mean = 3.33), followed by the red design (2.87, 3.04), and then the spectral design (2.75, 2.83). MWU tests (α = 0.05) show highly significant differences (p < 0.01) in fear and protective action responses between each pair of designs.
b. RQ 2: Spatial clustering of responses
Results from the Getis–Ord Gi* spatial cluster analyses for fear and protective action responses, respectively, are contained in Fig. 5. The size of each dot represents the statistical significance of clustering within the specified neighborhood (i.e., within 2.82 km). The smallest dots do not display statistically significant clustering, whereas the two larger-sized dots denote positions with z scores beyond ±1.96. The largest dots therefore represent the most statistically significant clusters with z scores beyond ±2.58. Black dots depict high responses to fear and protective action questions, while white dots represent low responses relative to neighboring positions.
Getis–Ord Gi* spatial cluster analyses for fear and protective action responses in the three warning designs. Medium dots represent z scores that exceed ±1.96, while large dots represent z scores that exceed ±2.58.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00021.1
In the original warning design, the highest fear and protective action responses are located near the warning’s geographic centroid (Figs. 5a,d). Respondents gave higher responses near the narrow portions (bottom left) of the spectral and red warning designs, however, for both fear (Figs. 5b,c) and protective action questions (Figs. 5e,f). When comparing responses between spectral and red designs, there were also more significant clusters of high responses in the red gradient design.
The spatial clustering of low responses differs between designs as well. In the original design, all significant low clusters occur on or beyond the edge of the warned area (Figs. 5a,d). These responses are also scattered about all sides of the warning. By contrast, both fear and protective action responses to the spectral design (Figs. 5b,e) and fear responses to the red design (Fig. 5c) show significant clustering only around the northeastern edge of each design (i.e., farthest from the area of highest probability). Low clusters for protective action responses to the red design (Fig. 5f) are located mainly around the eastern periphery (i.e., farthest downstream and to the right of the implied track). Highly significant low clusters on the red design tend to lie beyond the edge of the graphic for the most part, while significant low clusters on the spectral design lie both inside and outside the edge of the warned area.
c. RQ 3: Association between responses and probability zones
When mean responses are aggregated across designs, fear and protective action responses decrease sequentially from zone 5 to zone 1; however, a different pattern results when mean responses for each zone are examined within each warning design (Table 1). While both probabilistic designs show the highest responses in zone 5 with decreasing responses outward, the highest mean response for the original design is found in zone 4. In fact, mean responses in zone 5 are lower than any other zone in the original design. Examining the range of mean responses among the probability zones within each design, the red warning design shows the largest overall range in mean response values (i.e., highest minus lowest) for both fear (1.52) and protective action questions (1.35). The spectral warning design showed slightly smaller ranges (1.35 for fear, 1.25 for protective action), while the original design showed considerably smaller ranges (0.47, 0.4).
Mean responses by probability zones for all warning designs together and for each warning design separately.
When considering all responses within each probability zone (Fig. 4) for each of the three warning designs, the proportion of high fear and protective action responses is found to display a decreasing linear trend from zone 5 outward to zone 1. Figure 6 suggests qualitatively that this trend is stronger for both probabilistic designs than for the original design. Linear-by-linear chi-square association tests (df = 1) verify these relationships. These tests show that red (χ2 = 125.41, χ2 = 94.64 for fear and protective action; p < 0.001 for both) and spectral (χ2 = 109.81, χ2 = 87.94; p < 0.001 for both) designs result in much stronger decreasing linear trends than the original design (χ2 = 4.22, χ2 = 3.85; p = 0.04, p = 0.05), for which results narrowly fall within the nominal significance level (α = 0.05). In general, the stated levels of fear and protective action seem to reflect viewers’ abilities to discern their proximity to the threat.
Percentage of fear (F) and protective action (P) Likert responses in each probability zone by warning design. Dark gray solid lines depicted on the diagram qualitatively visualize the linear trend found in response patterns to each design using linear-by-linear association chi-square tests.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00021.1
d. RQ 4: Differences in responses inside versus outside the warning
When results are aggregated across designs, MWU tests (α = 0.05) show significantly higher (p < 0.001) mean fear responses inside the warnings (mean = 3.09) than outside (mean = 2.56). Respondents are also more likely to take protective action when they are located inside the warnings (mean = 3.22) rather than outside (mean = 2.68). Comparisons of inside versus outside positions within each of the three warning designs show identical results (p < 0.001) for all tests. Comparisons for all inside versus outside combinations between designs reveal significant differences in all but one combination (Table 2). Responses inside the spectral warning are not significantly different from responses outside the original warning. The result is the same for both fear and protective action questions. Interestingly, this suggests that respondents’ perceptions and intentions outside a deterministic warning design may be roughly equivalent to what they would be inside certain probabilistic designs.
Mean responses inside vs outside for all warning designs together and for each warning design separately. Significant differences between individual designs (MWU, α = 0.05), where p < 0.001 and p < 0.01 are also noted. NS indicates instances where responses were not significantly different, and empty cells indicate comparisons that were not calculated.
e. RQ 5: Differences in responses near the warning edge
When aggregating across designs, MWU tests (α = 0.05) reveal significant differences (p < 0.001) between responses in probability zone 1 and responses outside the warned area for both fear and protective action (Table 3). When considering differences between zone 1 and outside responses within designs, the most significant differences are observed for fear and protective action responses to the original design (p < 0.001 for both). In the red design, fear responses are significantly different, while protective action responses are not (p = 0.028, p = 0.053). No such differences are observed for the spectral design (p = 0.059, p = 0.094). These findings suggest that, in general, respondents perceive a threshold at the edge of a warning design; however, they show less of a tendency to recognize the edge of a probabilistic warning design as a critical threshold than they do the edge of a deterministic warning design.
Mean responses in probability zone 1 vs outside for all warning designs together and for each warning design separately. Significant differences between individual designs (MWU, α = 0.05), where p < 0.001, p < 0.05, and p < 0.01 are also noted. NS indicates instances where responses were not significantly different, and empty cells indicate comparisons that were not calculated.
5. Discussion
a. Trade-offs
Despite the common goal of warning for a tornado, results reveal that each of the three warning designs tested in this study communicates the nature of the tornado threat differently. This is evidenced by differences in respondents’ self-reported fear and anticipated protective responses when the warning design and geographic position are varied. Each warning design exhibits what could be considered strengths and weaknesses; however, no design is superior overall. In trying to disseminate improved tornado warning graphics, meteorologists, emergency managers, and broadcasters may choose a warning design intending to emphasize the likelihood of a tornado in one geographic area, but find that the design inadvertently deemphasizes the likelihood in other areas. Deciding which type of warning design should be used for public dissemination requires making trade-offs in how the probability of a tornado is visually communicated and how it is interpreted by warning recipients. These trade-offs are not necessarily dichotomous; they are not simply good or bad, correct or incorrect. Rather, they represent conceptual spectra along which viewer interpretations and responses may vary. Both the communicator’s intended message and the desired viewer interpretation and response should be considered when making design choices. The results of this study point to four particular trade-offs that stem from design choice (Fig. 7).
Trade-offs in viewer interpretation when choosing a visual tornado warning design.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00021.1
The first of these is a trade-off between consistently high or variable responses across the tornado warning area. Fear and protective action responses are high and vary little for a deterministic warning using a red outline, as evidenced by a greater overall mean (RQ 1) and a small range of mean response values across the five probability zones in the original design (RQ 3). Conversely, the probabilistic warnings garner higher responses, which are concentrated in zone 5 (where tornado probability is highest). The red gradient design produces the most variation in warning responses between those in the highest probability zone (zone 5) and those in the lowest probability zone (zone 1) but still within the warned area (RQ 3). These findings mirror those of Severtson and Myers (2013), who find that delineating geographic zones elicits stronger responses in areas with relatively higher probabilities of occurrence or magnitude of impacts.
A second trade-off is revealed in the spatial cluster analyses (RQ 2). In the original design, the region of significantly clustered high responses is small; however, this area is located near the geographic centroid of the warning polygon and away from the region of presumed highest probability. On the other hand, high responses in both probabilistic designs span a larger area, but they are located near and just ahead of locations with the highest tornado probability. Of the two probabilistic designs, the red gradient design elicits the most widespread clustering of high responses. This overall pattern suggests that the color gradation of the probabilistic designs allows the study participants to infer a higher probability of a tornado in the darker shaded portions of the warned area; however, in the absence of any guidance (as with the original design), viewers assume the higher probabilities exist near the center of the shape. This finding is concordant with those of Morss and colleagues (2008, 2010), demonstrating that deterministic forecasts spur nonexpert warning recipients to create their own mental representations of uncertainty. As yet, it is unclear how the addition of a radar image or animation to each of the warning designs would modify a viewer’s ability to infer where the likelihood of a tornado is greatest. This remains an avenue for future study.
A third trade-off pertains to interpreting the future directional path of a moving threat and the concomitant ranking of responses based on this path. Although the shape alone of the original design is meant to communicate general movement of a tornado threat and uncertainty in its future path, the mean responses in each of the successive probability zones are not sequential (RQ 3). Furthermore, the fact that the presumed highest threat area in the original design (zone 5) displays the lowest mean responses suggests that viewers are not interpreting the sequencing of probability levels as indicated by the shape. This is verified by chi-square tests, which show a rather weak linear trend (RQ 3). By contrast, the color patterns of the probabilistic designs seem to communicate directional movement and variability in threat levels more clearly. This is evidenced not only by the sequentially decreasing means in successive probability zones, but also by the chi-square tests, which show a strong linear trend in the spectral design and especially in the red design (RQ 3).
A fourth and final trade-off emerges when comparing responses inside versus outside the warning designs. In this case, the trade-off is one where more detailed information on the probability of a tornado—as in a probabilistic design with multiple units of perception—is exchanged for a more definite boundary between areas inside and outside the danger area—as in a deterministic design. Statistically significant differences between zone 1 responses and outside responses are observed for the original design, but not for the spectral design (RQ 5). Results also suggest that warning recipients located inside the spectral design are perhaps no more likely to take shelter than those located outside the original warning design (RQ 4). Getis–Ord Gi* analyses show significant low clusters both inside and outside the edge of the spectral warning, even though low clusters only appear outside the edge of the original warning (RQ 2). Taken in sum, these findings suggest that design choices have the potential to alter the size of the area perceived to be under threat by either emphasizing or deemphasizing the outer edge of the warning. Although their study did not investigate geographic boundaries, Severtson and Henriques’ (2009) identified a similar trade-off in perceived risk based on the visual emphasis of graphical danger/safety thresholds.
The sources of these trade-offs may be related to several of Pinker’s (1990) factors that affect how people interpret visuals. First, the visual depiction of multiple levels of magnitude in the form of probability zones induces, to a large degree, the response variability, the location focus offset from center, and the perception of directionally ordered probability zones observed when viewers rate probabilistic designs. By contrast, because there is only one binary classification of warned or unwarned in the original design, Gestalt laws of continuity and common fate seem to explain the response consistency trade-off and the well-defined edge/threshold trade-off observed in relation to this design. Of course, this binary classification is based on spatial location, which is implicit in Pinker’s unit of perception factor. In the case of both the edge definition and focus location trade-offs, the fact that probabilistic designs had multiple units of perception while the deterministic original design had but one unit of perception helps clarify why attention may have been shifted from the boundary of the warning to an area offset from the graphical centroid where tornado probability was hypothetically highest. In all trade-offs, color functions as an essential preattentive and semiotic device. It simultaneously acts to differentiate position on a scale with properties of hue or shade, to mark proximity to a location of focus by demarcating contiguous regions of differing magnitude, and to imbue each warning design with a distinct semiotic message based on the palette chosen.
Severtson’s IRBF (Severtson 2013) is useful for exploring how some visual features promote clarity of interpretation and influence behavior. However, the visual factors contained in the framework are not precisely defined and are not mutually exclusive. For example, proximity is classified as a preattentive feature, but it also functions as one of the Gestalt grouping principles in Pinker’s factors. Additionally, IRBF poorly represents key concepts such as shape and perimeter, which do not easily fall into the classification schema, and orientation, which may not always be defined in terms of a latitude–longitude coordinate system. A more precise conceptual framework for salience in visual weather risk communication is needed to systematically build effective visuals for the gamut of meteorological hazards.
b. Study limitations
Applicability of this study to all tornado warning contexts is limited by a variety of factors. First, the majority of participants were university students from South Carolina. It is unclear how the results would change if this study were implemented with the general public on a national scale. Second, survey participants were asked about their levels of fear and intent to take protective action when a real tornado event was not in progress. Although behavioral intentions are often linked to subsequent actual behaviors in many human behavioral studies (Webb and Sheeran 2006), the controlled experimental setting does not account for situational factors that could be present during an actual tornado warning and may alter one’s intended course of action (Lindell and Perry 2012).
A third limitation of the study is inherent in the research design itself. The surveys lacked numerical probability values, base maps, geographical names, real-world landmarks, and radar imagery, which can accompany actual tornado warnings received through TV or mobile devices. The inclusion of these features may alter how the designs are interpreted. Future studies that continue this line of research will need to consider these factors.
6. Conclusions
The findings communicated in this study are valuable to the meteorological community because they demonstrate that messages intended for warning recipients are not always the same messages as those received and potentially acted upon. Even though each of the tornado warning designs presented intends to communicate that a tornado may occur within the outlined area, interactions between each recipient’s position and the warning design result in different patterns of fear and intended protective action for each of the three designs.
On average, the original design garners the strongest fear and protective action responses; however, the red gradient design incites both the strongest responses in any of the five probability zones and the largest range of responses between the zones. The strongest responses in the original design cluster tightly near the geographic centroid of the warning, while the strongest responses in both probabilistic designs display more expansive clusters in the narrow portion of the warning, nearest the maximum tornado threat. Probabilistic designs are superior to the deterministic design in assisting recipients to appraise their proximity to the tornado and respond with appropriately high ratings of fear and protective action. Responses inside each warning design are higher than responses outside that design, though these findings may not hold true when comparing between different warning designs. Finally, responses near the edge of the original design reflect a sharp decline in fear and intended action, whereas responses near the edge of the spectral design decline gradually.
It is clear that no single design tested in this study is superior to the others in all respects. Instead, the choice of visual warning design presupposes a number of trade-offs (Fig. 7) in the received warning message that is ultimately acted upon. These include trade-offs between consistency and variability of warning responses, an offset or central location focus, higher or lower perceived ordering of threat levels along the future tornado track, and a definite or indefinite outer warning edge. Consideration of these and other trade-offs, in conjunction with the intended message that meteorologists and risk communicators seek to convey, will be important as probabilistic information becomes a mainstay in weather warning communication.
Acknowledging that probabilistic warning designs are still under development, this study does not propose that either of the probabilistic designs tested be placed “as is” into operational use. Instead, the intention of this study was to test basic visual features that may be used in the future construction of probabilistic warnings. The present study provides a baseline for future studies that manipulate deterministic and probabilistic warning designs, testing them in increasingly sophisticated ways. Response patterns may differ if this study is replicated in a geographical region with higher tornado frequencies or with a population other than university students. Similarly, a modification in warning orientation (i.e., not southwest to northeast), shape (i.e., asymmetric warnings), or the inclusion of a centerline might alter responses.
The consideration of geographic position in concert with visual features is a notable strength of the research design that can inform future studies on visual weather communication. Future research will test for changes in interpretation with additional cognitive aids such as local base maps with locations and/or landmarks, legends, an accompanying text message, or a radar animation. Future work should also include collection of qualitative responses to the warning designs, particularly with vulnerable and highly exposed populations such as the elderly and mobile home residents.
This research is a first step toward improved tornado warning visual communication with probabilistic information. Meteorologists and social scientists working in cooperation hold the power of choice when designing these warnings. Visual tornado warnings can be crafted in such a way as to better elicit appropriate affective responses and protective behavioral strategies. One final question remains: What warning message do we want to send?
Acknowledgments
This research was supported by a grant from the Hazards and Vulnerability Research Institute at the University of South Carolina. The authors wish to thank Susan Cutter and John Besley for their suggestions on earlier drafts of this work. Gratitude is extended to Andrea Tanner and graduate students in both the Department of Geography and the School of Journalism and Mass Communications for allowing the survey to be conducted in their classes. Thanks also go to the three anonymous reviewers whose detailed and pointed comments improved the clarity of the manuscript.
REFERENCES
Agresti, A., 2002: Categorical Data Analysis. John Wiley & Sons, 710 pp.
Ashley, W. S., 2007: Spatial and temporal analysis of tornado fatalities in the United States: 1880–2005. Wea. Forecasting, 22, 1214–1228.
Bostrom, A., Anselin L. , and Farris J. , 2008: Visualizing seismic risk and uncertainty. Ann. N. Y. Acad. Sci., 1128, 29–40.
Brewer, C. A., 1997: Spectral schemes: Controversial color use on maps. Cartogr. Geogr. Info. Syst., 24, 203–220.
Broad, K., Leiserowitz A. , Weinkle J. , and Steketee M. , 2007: Misinterpretations of the “cone of uncertainty” in Florida during the 2004 hurricane season. Bull. Amer. Meteor. Soc., 88, 651–667.
Burt, J. E., Barber G. M. , and Rigby D. L. , 2009: Elementary Statistics for Geographers. Guilford Press, 653 pp.
Casteel, M. A., and Downing J. R. , 2013: How individuals process NWS weather warning messages on their cell phones. Wea. Climate Soc., 5, 254–265.
Cleveland, W. S., and McGill R. , 1984: Graphical perception: Theory, experimentation, and application to the development of graphical methods. J. Amer. Stat. Assoc., 79, 531–554.
Coleman, T. A., Knupp K. R. , Spann J. , Elliott J. B. , and Peters B. E. , 2011: The history (and future) of tornado warning dissemination in the United States. Bull. Amer. Meteor. Soc., 92, 567–582.
Donner, W. R., Rodriguez H. , and Diaz W. , 2012: Tornado warnings in three southern states: A qualitative analysis of public response patterns. J. Homeland Secur. Emerg. Manage., 9, 1547–7355, doi:10.1515/1547-7355.1955.
Dymon, U. J., and Winter N. L. , 1993: Evacuation mapping: The utility of guidelines. Disasters, 17, 12–24.
Elliot, A. J., and Maier M. A. , 2007: Color and psychological functioning. Curr. Dir. Psychol. Sci., 16, 250–254.
Fishbein, M., 2008: A reasoned action approach to health promotion. Med. Decis. Making, 28, 834–844.
Frazier, A. E., Bagchi-Sen S. , and Knight J. , 2013: The spatio-temporal impacts of demolition land use policy and crime in a shrinking city. Appl. Geogr., 41, 55–64.
Getis, A., and Ord J. K. , 1992: The analysis of spatial association by use of distance statistics. Geogr. Anal., 24, 189–206.
Griffith, L. J., and Leonard S. D. , 1997: Association of colors with warning signals. Int. J. Ind. Ergon., 20, 317–325.
Haimes, Y. Y., 2009: On the complex definition of risk: A systems-based approach. Risk Anal., 29, 1647–1654.
Hoekstra, S., Klockow K. , Riley R. , Brotzge J. , Brooks H. , and Erickson S. , 2011: A preliminary look at the social perspective of Warn-On-Forecast: Preferred tornado warning lead time and the general public’s perceptions of weather risks. Wea. Climate Soc., 3, 128–140.
Hoffman, R. R., 1991: Human factors psychology in the support of forecasting: The design of advanced meteorological workstations. Wea. Forecasting, 6, 98–110.
Hoffman, R. R., Detweiler M. , Conway J. A. , and Lipton K. , 1993: Some considerations in using color in meteorological displays. Wea. Forecasting, 8, 505–518.
Howell, D. C., 2013: Statistical Methods for Psychology. Wadsworth, 773 pp.
Jacks, E., and Ferree J. , 2007: Socio-economic impacts of storm-based warnings. Preprints, Second Symp. in Policy and Socio-Economic Impacts, San Antonio, TX, Amer. Meteor. Soc., 2.2. [Available online at http://ams.confex/ams/87ANNUAL/techprogram/paper_120849.pdf.]
Klockow, K., 2011: Investigation of individual spatial awareness relating to response during the April 27, 2011 tornado outbreak. Quick Response Rep. 224, Natural Hazards Center, University of Colorado at Boulder, Boulder, CO, 15 pp.
Kuhlman, K. M., Gruntfest E. , Stumpf G. J. , and Scharfenberg K. , 2009: Beyond storm-based warnings: An advanced WAS*IS workshop to study communication of probabilistic hazardous weather information. Preprints, Fourth Symp. on Policy Socio-Economic Research, Phoenix, AZ, Amer. Meteor. Soc., 3.5. [Available online at https://ams.confex.com/ams/pdfpapers/150887.pdf.]
Lazo, J. K., 2012: One economist’s entreaty for increased research on weather risk communication. Wea. Climate Soc., 4, 233–235.
Leonard, S. D., 1999: Does color of warnings affect risk perception? Int. J. Ind. Ergon., 23, 499–504.
Lindell, M. K., and Perry R. W. , 2012: The protective action decision model: Theoretical modifications and additional evidence. Risk Anal., 32, 616–632.
Lindell, M. K., and Brooks H. E. , 2013: Workshop on weather ready nation: Science imperatives for severe thunderstorm research. Bull. Amer. Meteor. Soc., in press.
Loewenstein, G. F., Weber E. U. , Hsee C. K. , and Welch N. , 2001: Risk as feelings. Psychol. Bull., 127, 267–286.
Lundgren, R. E., and McMakin A. H. , 2009: Risk Communication: A Handbook for Communicating Environmental, Safety, and Health Risks. IEEE Press, 362 pp.
MacEachren, A. M., 2004: How Maps Work: Representation, Visualization, and Design. Guilford Press, 515 pp.
Maloney, E. K., Lapinski M. K. , and Witte K. , 2011: Fear appeals and persuasion: A review and update of the extended parallel process model. Soc. Pers. Psychol. Compass, 5, 206–219.
Mass, C. F., 2012: Nowcasting: The promise of new technologies of communication, modeling, and observation. Bull. Amer. Meteor. Soc., 93, 797–809.
Meier, B. P., D’Agostino P. R. , Elliot A. J. , Maier M. A. , and Wilkowski B. M. , 2012: Color in context: Psychological context moderates the influence of red on approach- and avoidance-motivated behavior. PLoS ONE, 7, e40333, doi:10.1371/journal.pone.0040333.
Meyer, R., Broad K. , Orlove B. , and Petrovic N. , 2013: Dynamic simulation as an approach to understanding hurricane risk response: Insights from the Stormview Lab. Risk Anal., 33, 1532–1552, doi:10.1111/j.1539-6924.2012.01935.x.
Mileti, D. S., and Peek L. , 2000: The social psychology of public response to warnings of a nuclear power plant accident. J. Hazard. Mater., 75, 181–194.
Monmonier, M., Ed., 1999: Managed myopia. Air Apparent: How Meteorologists Learned to Map, Predict, and Dramatize Weather, University of Chicago Press, 213–232.
Montz, B. E., 2012: Assessing responses to National Weather Service warnings: The case of a tornado. Studies in Applied Geography and Spatial Analysis: Addressing Real World Issues, R. Stimson, and K. E. Haynes, Eds., Edward Elgar Publishing, 311–324.
Morss, R. E., Demuth J. L. , and Lazo J. K. , 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974–991.
Morss, R. E., Lazo J. K. , and Demuth J. L. , 2010: Examining the use of weather forecasts in decision scenarios: Results from a U.S. survey with implications for uncertainty communication. Meteor. Appl., 17, 149–162.
Nagele, D. E., and Trainor J. E. , 2012: Geographic specificity, tornadoes, and protective action. Wea. Climate Soc., 4, 145–155.
NRC, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. The National Academies Press, 124 pp.
Ord, J. K., and Getis A. , 1995: Local spatial autocorrelation statistics: Distributional issues and an application. Geogr. Anal., 27, 286–306.
Pinker, S., 1990: A theory of graph comprehension. Artificial Intelligence and the Future of Testing, R. Freedle, Ed., Lawrence Erlbaum Associates, 73–126.
Severtson, D. J., 2013: The influence of environmental hazard maps on risk beliefs, emotion, and health-related behavioral intentions. Res. Nurs. Health, 36, 330–348.
Severtson, D. J., and Henriques J. B. , 2009: The effect of graphics on environmental health risk beliefs, emotions, behavioral intentions, and recall. Risk Anal., 29, 1549–1565.
Severtson, D. J., and Vatovec C. , 2012: The theory-based influence of map features on risk beliefs: Self-reports of what is seen and understood for maps depicting an environmental hazard. J. Health Commun., 17, 836–856.
Severtson, D. J., and Myers J. D. , 2013: The influence of uncertain map features on risk beliefs and perceived ambiguity for maps of modeled cancer risk from air pollution. Risk Anal., 33, 818–837.
Sheeran, P., 2002: Intention-behavior relations: A conceptual and empirical review. Eur. Rev. Soc. Psychol., 12, 1–36.
Sherman-Morris, K., 2005: Enhancing threat: Using cartographic principles to explain differences in hurricane threat perception. Fla. Geogr., 36, 61–83.
Sherman-Morris, K., 2010: Tornado warning dissemination and response at a university campus. Nat. Hazards, 52, 623–638.
Sherman-Morris, K., and Brown M. E. , 2012: Experiences of Smithville, Mississippi residents with the 27 April 2011 tornado. Natl. Wea. Dig., 36, 93–101.
Simmons, K. M., and Sutter D. , 2005: WSR-88D radar, tornado warnings, and tornado casualties. Wea. Forecasting, 20, 301–310.
Simmons, K. M., and Sutter D. , 2008: Tornado warnings, lead times, and tornado casualties: An empirical investigation. Wea. Forecasting, 23, 246–258.
Sjöberg, L., 2007: Emotions and risk perception. Risk Manage., 9, 223–237.
Slocum, T. A., McMaster R. B. , Kessler F. C. , and Howard H. H. , 2005: Thematic Cartogrography and Geographic Visualization. 2nd ed. Pearson/Prentice Hall, 528 pp.
Slovic, P., Finucane M. , Peters E. , and MacGregor D. G. , 2004: Risk as analysis and risk as feelings: Some thoughts about affect, reason, risk, and rationality. Risk Anal., 24, 1–12.
Sorensen, J. H., 2000: Hazard warning systems: Review of 20 years of progress. Nat. Hazards Rev., 1, 119–125.
Stensrud, D. J., and Coauthors, 2013: Progress and challenges with Warn-on-Forecast. Atmos. Res., 123, 2–16.
Sutter, D., and Erickson S. , 2010: The time cost of tornado warnings and the savings with storm-based warnings. Wea. Climate Soc., 2, 103–112.
Waters, K. R., and Coauthors, 2005: Polygon weather warnings—A new approach for the National Weather Service. Preprints, 21st Int. Conf. on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, San Diego, CA, Amer. Meteor. Soc., P14.1. [Available online at http://ams.confex.com/ams/pdfpapers/86326.pdf.]
Webb, T. L., and Sheeran P. , 2006: Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychol. Bull., 132, 249–268.
Whitehead, J. C., 2005: Environmental risk and averting behavior: Predictive validity of jointly estimated revealed and stated behavior data. Environ. Resour. Econ., 32, 301–316.
Winters, A. M., and Coauthors, 2010: Spatial risk assessments based on vector-borne disease epidemiologic data: Importance of scale for West Nile virus disease in Colorado. Amer. J. Trop. Med. Hyg., 82, 945–953, doi:10.4269/ajtmh.2010.09-0648.
Wogalter, M. S., Conzola V. C. , and Smith-Jackson T. L. , 2002: Research-based guidelines for warning design and evaluation. Appl. Ergon., 33, 219–230.
