1. Introduction

Bad weather and its downstream effects kill people, destroy property, and impoverish communities. This happens despite the best efforts of responsible national meteorological and hydrological services (NMHSs) to provide accurate and timely disseminated warning information (WMO 2015a). In 2016, the number of people who reported to be affected by natural disasters worldwide (564.4 million) was the highest since 2006—amounting to 1.5 times the 2006–15 annual average (224 million)—and almost 70 000 die each year (Guha-Sapir et al. 2017). In Europe, reported economic losses from extreme weather- and climate-related events over the 1980–2016 period amounted to EUR 436 billion (European Environment Agency 2017a). Losses may be even greater under future climate change, as nearly all extreme weather- and climate-related events are projected to change in their frequency, intensity, spatial extent, duration, and timing (IPCC 2012). Along with climate change, socioeconomic developments such as growing population and economic wealth, developments in hazard-prone areas, and degradation of natural ecosystems will influence the exposure and vulnerability of many regions, across the world and in Europe (European Environment Agency 2017b).

In response, NMHSs have improved their capacity to forecast high-impact weather events, with growing precision in terms of both the hazard and the impacts. More precisely, the second half of the twentieth century witnessed improvement in the quality, quantity, and availability of weather and climate information, including forecasts, warnings, and advisory services (WMO 2015b). Enhanced technology, making use of real-time data collection and the capability of modeling and dissemination of information, has given rise to warning systems that are becoming increasingly sophisticated (Aitsi-Selmi et al. 2016). Thus, forecasts are being made with greater accuracy, geographic precision, and lead time, which allows people at risk to take appropriate protective action. Simultaneously, advances in computer simulation and modeling have made it possible to also include information about the underlying hazards, as well as about the exposure of the population, so that warning information can truly inform response (Aitsi-Selmi et al. 2016).

However, there is a growing list of documented cases where people do not respond to these warnings. For example, in 2013, Tropical Cyclone Haiyan struck the Philippines as a category 5 storm, causing many casualties on top of severe damage to infrastructures and agriculture. A post-typhoon expert mission report by the World Meteorological Organization (WMO) highlighted that accurate warnings were issued on time by the meteorological agency for heavy rains and wind, and the government deployed preparedness plans (WMO 2015a). However, in the communities at risk, the low level of knowledge, especially about the storm surge, hindered or delayed evacuations from areas hit by the cyclone. In developed countries as well, there are documented cases of people failing to take appropriate protective action with severe consequences. In 2012, Superstorm Sandy in the United States was preceded by accurate forecasts and timely disseminated warnings, but many people failed to take appropriate action that experts were hoping to see (National Academies of Sciences 2017). During deadly tornado outbreaks in Oklahoma in 2013, thousands of people fled their homes in cars despite years of messaging about the dangers of encountering storms in a vehicle (National Academies of Sciences 2017). Thus, including more information about impacts and possible behavioral recommendations could help people to take more informed decisions.

Some research has examined how responses could be improved by making it easier or more desirable for people to engage in self-protective behaviors. For example, Sharma and Patt (2012) examined in coastal India the decision to evacuate to cyclone shelters in response to warnings; they found that prior experience with the comfort and quality of the shelter, as well as with loss of property due to crime in the villages being evacuated, were major determinants of self-protective behavior. Other research has focused on the psychological factors that determine whether people are motivated to protect themselves at all, and hence how warnings could address these factors (e.g., Lindell and Perry 2012; Slovic et al. 2007). In studies of Germans’ decisions to protect themselves from river flooding events, researchers found people’s subjective self-confidence in their ability to protect themselves to make a larger difference than objective indicators, such as wealth or access to information (Grothmann and Patt 2005; Grothmann and Reusswig 2006). As individual characteristics of the population receiving the warning affect warning response, targeting warning messages on the basis of these social or knowledge characteristics is another possibility (Mileti and Sorensen 1990). But perhaps the simplest change would be to alter the content of the warnings themselves, providing different information about the hazard that is predicted, the effects of that hazard, and ways for people to protect themselves. Mileti and Sorensen (1990) proposed that warning messages ought to contain five information elements—hazard, location, guidance, time, and source—in order to be more helpful. Mileti and Peek (2000) reiterated that effective warning messages should include clear, specific, accurate, certain, and consistent information. At the same time, however, there has been little empirical study examining the relative importance of these different factors, particularly in the case of short-term forecasts of high-impact weather events.

In this paper we focus on the separate and combined effects of including behavioral recommendations (BRs) and weather impacts in the information reaching the general public. In our study, all warnings that contain impact messaging are named impact-based warnings (IBWs) and also include all other elements to a message (such as source, time, or hazard severity). BRs are a set of actions that experts think are helpful and reflect what they recommend people to do (or not to do). However, these recommendations are generally but not universally recognized to be appropriate when facing severe thunderstorms. The “appropriate” behavior in response to warnings may not always be the same behavior, but there are better things to do than others. IBWs are designed to express the expected impacts as a result of the expected weather and provide people with more information about the hazard and particularly its impacts (WMO 2015a; Casteel 2016). Including this information comes at some potential cost, both to the agency issuing the warning (which needs to analyze these factors) and to the audience (which needs to take the time to listen to them); neither is currently common practice. To justify the added costs of either or both measures, there is a need to empirically test whether BRs and IBWs, both separately and in combination, affect the perception of and behavioral response to warning information. We hypothesize that both IBWs and BRs, separately or in combination, could increase the likelihood with which people take action. We fill important empirical gaps in the literature and show a large potential for improvements in warning design to lead to greater appreciation of danger and taking of protective actions.

2. Background

The standard practice in high-impact weather warnings is to describe only the hazard. Current standard warnings (SWs) focus on the weather phenomenon itself, rather than the potential effects of the event. Even though some NMHSs (and also the WMO) have started to put greater emphasis on the so-called IBWs (WMO 2015a), there are many reasons we can think of for the continued dominant position of SWs in weather communication. In the following we list some of the authors’ own assumptions. First, SWs require less effort and resources to prepare, for instance in the coordination between meteorological offices and agencies that are entrusted with protecting public safety, such as police and fire departments, which have greater knowledge of local circumstances, such as specific vulnerabilities (e.g., to flash floods) or evacuation routes. In fact, the differences in mandates, roles, and responsibilities between NMHSs and responding agencies (e.g., the Office of Civil Protection) in itself are already a major challenge that makes the generation of IBWs only more complex. For example, in Switzerland, an NMHS is the responsible agency to warn about severe weather, but it is the local or regional authorities that have to provide behavioral recommendations for very severe events. So, the responding agencies have to interpret the warnings (and take account of the related uncertainties), consult further information (e.g., about the current situation on the ground), and ultimately reach a decision (e.g., to evacuate an area), all under significant time pressure. Second, by only focusing on describing the hazard itself (i.e., type of hazard, severity, timing, and location), NMHSs avoid information overload. Third, SWs guarantee less risk of giving specific information that is inappropriate for the user. Finally, without giving clear impact information and BRs, NMHSs avoid criticism of being paternalistic.

At the same time, however, there is an extensive literature documenting the dangers of NMHSs assuming users to have the ability to translate hazard forecasts into appropriate actions, and suggesting that it is important to have a “fit” between the information and users’ actual knowledge (Orlove and Tosteson 1999). Based on the growing list of documented cases where people do not respond to the standard phenomenon-based warnings, the WMO has speculated that some of the human losses are due to people’s inadequate understanding of the warnings; their background knowledge is less than what would be needed to fully understand the SWs that NMHSs are communicating (WMO 2015a). Thus, people lack knowledge that is needed to translate the information on the weather phenomenon into an appraisal of risk. As Mileti and Peek (2000) have noted, “It cannot be assumed that the public will know what would constitute an appropriate protective action” (p. 185). In a study on public response to an earthquake warning, Mileti and Darlington (1997) found that the more informative the guidance people received in a warning message, the more likely they were to respond to it. Likewise, it cannot be assumed that the public will know what impacts severe weather can have as they may not be familiar with some types of hazards or their respective severities. Previous research underlines that in order to be effective, warnings should describe the exact nature of the threat (Mileti and Sorensen 1990), provide a source of confirmation (Lindell and Perry 2012), and be personally relevant (Mileti 1999). IBWs used in our study take all this into account.

Moreover, participants’ sociodemographic or knowledge characteristics affect warning response (Mileti and Sorensen 1990). Previous work suggests that sociodemographic characteristics, including gender, age, education, residence area and ownership, risk perception, and knowledge characteristics, such as experience with and knowledge of weather and warnings, may all be associated with differences in protective responses. For example, in research related to hurricane protective responses, scholars found that hurricane evacuation likelihood was higher for female respondents than males (Lazo et al. 2015; Morss et al. 2016). Regarding age, some studies have shown that older residents are more likely to evacuate (Morss et al. 2016; Peacock et al. 2011), but others have found contradictory results (Lazo et al. 2015). In a few studies, higher educational level has also been found to correspond to a higher likelihood of evacuation in case of hurricane threats (Peacock et al. 2011). With regard to residence area, people in rural areas showed more adequate coping behaviors in a crisis situation in the past, compared to the urban residents, who also tend to be younger. Likewise, Scolobig et al. (2012) highlight that risk awareness was significantly higher among those residents who were living in isolated mountain communities, compared to urban communities. With respect to risk perception, Lazo et al. (2015) highlight that several studies focusing on storms have found that perceived risk is a primary factor in hurricane evacuation decision making, even though some did not (Lazo et al. 2015; Morss et al. 2016). Previous studies in various natural hazard contexts have shown that people’s previous experiences with a hazard (such as flooding) can influence risk perception and protective action (Knocke and Kolivras 2007; Paton et al. 2000; Sharma and Patt 2012; Siegrist and Gutscher 2006; Wagner 2007). Finally, previous research has shown that heightened knowledge of the hazard increases the probability that an individual will take protective action when a warning is being disseminated (Ripberger et al. 2015b; Sorensen 2000). We consider these previously studied sociodemographic characteristics as potential explanatory variables in this study.

There has been a growing suspicion within the expert community that providing more specific information would be beneficial in the specific case of high-impact weather warnings, but there is very little empirical testing of this. Based on qualitative interviews with forecasters, emergency managers, and broadcast meteorologists, Losego et al. (2013) and Harrison et al. (2014) identified IBWs to create an added value as compared to standard warning information. There are four studies that provide quantitative evidence, but all have their shortcomings.

The first study was by Perreault et al. (2014). They primarily investigated the effects of using fear-inducing language to describe tornados, although that language also included a brief mention of impacts and BRs. Thus, they investigated the additive effect of IBWs. In more detail, they tested the effectiveness by showing participants four different experimental stimuli—regular TV warning messages, new scary TV warning messages, regular radio warning messages, and new scary radio warning messages—and observed that behavioral intentions were not affected by any of the experimental stimuli. Moreover, they found that regular warning messages were perceived as more credible than the impact-based messages. In sum, they found no effect of IBWs on credibility and no effects on intended behavioral response. However, they left open the effects of IBWs on other perception attributes, as well as the effects of BRs on intended behavioral response.

The second study was by Ripberger et al. (2015b). They designed and administered a survey experiment to residents in tornado-prone regions in the United States to answer the question of whether consequence-based messages influence public responsiveness to tornado warnings. They found that participants receiving hypothetical tornado warnings were more likely to take some sort of protective action as tornado impact descriptions increased in severity. However, they also identified a threshold beyond which increasing the projected impact of a storm no longer significantly increases the probability of taking protective action. Thus, at lower levels of projected impact, increases in the projected consequences of the storm increased the probability to take protective action, whereas at higher levels of projected impact this relationship reverses. Moreover, their results indicate that the respondents who know more about tornadoes were a bit more likely to opt for protective action than respondents who are less knowledgeable. However, they left unresolved the effects of BRs, and also of IBWs on perception and understanding of warnings.

The third study was by Casteel (2016). He too used a randomized control survey design, focusing on tornado warnings as a case, with respondents in the hypothetical role of a factory operator having to decide whether to order workers to take shelter, in response to two types of warnings, IBWs and SWs. He included the BRs to seek shelter in all warnings and investigated the additive effect of IBWs. He found that decisions did not significantly differ between the IBWs and the SWs at decision point 1 (occurring when the two versions of the warning presented the same information). However, at points 2 (when hazard, source, and impact information were presented in the IBWs, but absent in the SWs) and 3 (at the end of the warnings), IBWs produced higher likelihoods of sheltering in place compared to SWs. Thus, he found the additive effect of IBWs and BRs to have an influence on the intended behavioral response. He also tried to account for knowledge differences but found that the pattern of intended response did not differ between the two groups that were studied (undergraduate students in psychology or communication and graduate students in emergency management), suggesting that the enhanced text used in the IBWs is clear and understandable to a large audience. His study left open the effects of BRs, and also of IBWs on perception. As participants were only students, the generalization of results is also questionable.

The fourth study has been by Potter et al. (2018). They conducted an online survey in New Zealand to investigate the public’s perceptions and intended protective actions in a hypothetical severe weather event involving strong winds. They examined the effects of adding impact information to warnings lacking BRs and examined the effects on both perception and intended behavioral response. They found an effect of IBWs on perception. The authors reported that IBWs make it easier to understand possible effects of the weather event, make the event more threatening, and make recipients more concerned about the event, in comparison to those who received SWs. However, they reported that there was no significant difference in perceptions of credibility of the message. Regarding intended behavioral response, they found an effect of IBWs on information seeking behavior. Interestingly, participants who received the IBWs were significantly more likely to state that they would check other information sources for confirmation or advice than those who received SWs. They also found an effect on the likelihood to take nonprotective behavior. People who received SWs were more likely to “do nothing differently” (i.e., to engage in a dangerous behavior) than those people who received IBWs. However, they did not find an effect of IBWs on increasing actual protective behavior. They leave open the effects of BRs. Moreover, the positive intended protective actions they investigated, for which they found no significant effect of IBWs, were primarily geared toward mitigating damage to property (such as driving more carefully or securing loose items) and also avoiding possible high-risk activities (such as avoiding outdoor work or activities), rather than actively seeking safety for oneself. This leaves open the question whether such behavior would also not be affected by IBWs.

Following the four studies on IBWs, then, two questions remain ambiguous or unexamined. First, do both BRs and IBWs have effects, and what are their relative magnitudes? Second, are effects to be found on both perception and understanding, as well as intended behavioral response? Like Potter et al. (2018), we make use of self-reported intended responses to a hypothetical and imagined situation. As we describe in more detail in the next section, we answer these questions with a multimethod study, which we have conducted in our home country of Switzerland, focusing on people’s responses to warnings of severe thunderstorms. Thunderstorms are an important high-impact event in Switzerland (BABS 2015). These storms develop rapidly and can be destructive and deadly. In Switzerland, each year people get harmed or killed by lightning strikes (BABS 2015, 2013). Thunderstorms also often evolve with substantial variability and uncertainty in impacts. Beside hail and lightning strikes, flash flooding can be a significant source of danger, especially in small mountain creeks or heavily urbanized and sealed surfaces, and cause significant damage (MeteoSwiss 2018). In spite of the most up-to-date tools and methods, it is currently not possible to issue several hours’ warnings of the strength, timing, and location of severe thunderstorms (MeteoSwiss 2018). Thus, timely warning and protective decision-making are critical in case of a thunderstorm threat and, in particular, effective communication of actionable information about the hazard.

3. Methods

Our primary method was a large sample randomized control survey, where we tested for the separate and combined effects of IBWs and BRs on perception and intended behavioral response. We also supplemented this with qualitative interviews with experts in the same country, to verify that there is a perception in that expert community of the importance of IBWs and/or BRs, and also to see if our survey results in any way contradicted expert belief.

For the qualitative piece, we conducted 16 semistructured interviews with stakeholders involved in the natural hazard warning chain in Switzerland. To recruit the interviewees, we used the snowball sampling technique (Flick 2002; Mayring 2002; Lamnek 2005). We identified MeteoSwiss as the key contact as they are the official warning authority for severe weather events in Switzerland. They provided us with numerous contacts (e.g., weather forecasters, cantonal officials), which then again helped to recruit more interviewees. Moreover, we also actively contacted people/organizations that we found could provide relevant input and that were not identified by the initial interviewees (such as private sector meteorology or insurance companies). Thus, in the end we had (to our estimate) a fair representation of different stakeholders with different viewpoints and who represented the entire weather information value chain; from the weather (monitoring, observation, modeling, and forecasting) to the dissemination of information, the perception, the interpretation, and, ultimately, the information use and decision-making. The interviewees in the different organizations were selected on the basis of their experience and knowledge of the warning system. All interviewed people had a leading or coordinating role within their organization and were linked directly or indirectly to the weather warning chain. In more detail, they included officials in MeteoSwiss (crisis management, forecasting unit, communication management) and in other organizations dealing with risk and emergency management at cantonal and federal level (e.g., federal civil protection agency, cantonal and federal environmental agencies), as well as local decision-makers (e.g., Office of Waste, Water, Energy and Air in Zürich), together with university professors, research institutes involved in the warning process (e.g., the Swiss Seismological Service or the Institute for Snow and Avalanche Research), insurance companies providing warning information, and private sector meteorology. Participants were first contacted via e-mail, in which they were informed about the purpose and nature of our research and why they were selected for participation. The interviews followed a protocol, developed according to the state of the art in the literature (Flick 2002; Mayring 2002; Schnell et al. 2011; Lamnek 2005). The protocol was used as guidance and included questions about the warning system in general, the interviewees’ expertise on high-impact weather events, their roles in the weather value chain and warning process, their perspectives about problems and solutions, and the economic framework for information processing and the evaluation of information, among others. All interviews were conducted in person at the office of the respondent. The length of interviews ranged from 46 to 120 min (on average 75 min). The interview length had no effect on the content of the interviews (as all topics were addressed in each interview). Notes were taken during and after each interview and all interviews were recorded with the permission of the participants. The interview transcripts were categorized into different topics and key challenges that depicted the strengths and weaknesses of the warning system in Switzerland. No specific software was used for data analysis.

For the quantitative element, we conducted a survey containing 22 questions based on a decision scenario. The thunderstorm scenario was placed at the beginning of the survey. Respondents were asked to imagine that it is 2 p.m. on a Sunday and that they would be walking in the countryside in the foothills of the Jura Mountains, when they get a thunderstorm warning message on their smartphone. While we considered and pretested many options for the hypothetical scenario (moving an outdoor picnic indoors, taking a bicycle to work, going for a walk in the woods, etc.), we used the “hiking” scenario because it (i) was simple, straightforward, and realistic, (ii) forced people to face a real threat taking them out of their comfort zone, and (iii) required people to make the decisions on their own (unlike in a scenario at work, where the employer could intervene).

The 1219 respondents completed the survey from 23 November to 2 December 2017. Respondents were recruited through the access panel provider Respondi to participate in an online survey. Prior to the survey and the interviews, participants were informed about the ongoing research, that the data was evaluated anonymously (names were not collected), and that their participation was voluntary. Participants received a financial incentive to complete the survey, which took them on average about 9 min and 2 s. Because of unrealistically short answering times, we excluded answers of 98 respondents from analysis, leading to 1121 respondents. We conducted the survey (in German) with a sample of the Swiss population in the German-speaking cantons (see Fig. 1). Respondents ranged in age from 15 to 86 years [mean (M) = 48.54, standard deviation (SD) = 16.40; n = 591 (52.7%) were female]. The majority of respondents completed vocational school (44.4%, n = 498), followed by college or university education (17.8%, n = 200), with 7% indicating at least some compulsory education. Compared to the average Swiss population, the sample was slightly older than the average (M = 43.14 yr), similar for the gender ratio of female (50.5%) and male (49.5%) (FSO 2017a) and slightly more educated than the Swiss population (more respondents with higher vocational training or university degree and fewer respondents who only completed compulsory school) (FSO 2017b). However, as the survey was conducted online, based on the people registered in the database of the online panel, it did not reach people who do not have Internet access or are not in the data pool. Therefore, the results of the research should not be taken as being representative of the Swiss German population. Nevertheless, they can be an indication of the opinions of the users of online weather warnings.

View Full Size

Map of Switzerland with the four language regions.

Citation: Weather, Climate, and Society 10, 4; 10.1175/WCAS-D-18-0038.1

• Download Figure
• Download figure as PowerPoint slide

We randomly assigned respondents to one of four warnings types described in Table 1 (SW; SW with BR; IBW; IBW with BR), achieving roughly even subgroups out of the total sample size of 1219. Table 2 describes the specific messages for the separate groups. These were based on current warning information of MeteoSwiss (element A in Table 2) and adapted with publicly available information (elements B and C) that is provided on the MeteoSwiss website (http://www.meteoswiss.admin.ch) and the Natural Hazards Portal (http://www.natural-hazards.ch), which is run by a consortium of federal agencies. The BRs are representative for the kind of information that experts agree one could give in these situations. Experts (forecasters and other staff of MeteoSwiss) had reviewed the warning messages for plausibility. Thus, the messages reflect reality and could be disseminated in a similar form and content (given the same hazard severity) in the near future.

Table 1.

Four warning types.

View Table

Table 2.

Four different thunderstorm warning messages: SW, element A; SW + BR, elements A and C; IBW, elements A and B; IBW + BR, elements A, B, and C.

View Table

We asked questions on two sets of dependent variables: perception and understanding (five attributes) as well as intended behavioral response (five attributes). Perceptions measured were credibility (“I believe the message to be credible”) and concern (“The warning makes me concerned for my safety”). We also collected data on three types of understanding: the warning (“The warning is clear and easy to understand”), the threats to safety (“Based on the warning I understand the threats to my safety”), and how to respond (“Based on the warning it is clear to me how I should modify my behavior, if necessary”). The five intended behavioral responses included a strong risk minimizing behavior (“I immediately interrupt the hike and take care of my safety”), a weak risk minimizing behavior (“I continue with the hike, but avoid potential dangers and make sure that a suitable shelter can always be reached quickly”), and a nonprotective action (“I would not alter my plans because of the information”), as well as two information-seeking behaviors (“I check out other information sources via my smartphone for confirmation or advice” and “I pay close attention to the sky, wind, and sounds of possible thunder”). Participants indicated how much they agreed or disagreed on a five-point Likert scale from “totally disagree” to “totally agree” with each of these positively framed statements. We also gathered data on personal factors including risk perception, experience, knowledge, and other social and demographic factors. To collect the data about risk perception, participants had to indicate how strongly they agreed or disagreed with four statements that thunderstorms could pose a threat to 1) their personal safety, 2) the safety of their friends and families, 3) their property, and 4) their daily life. This scale was similar to the one used by Prior (2010) and yielded good internal consistency (Cronbach’s alpha = 0.87, N = 4). A thunderstorm risk perception variable was computed based on the four items (by calculating the average). The full questionnaire is available in the online supplemental material.

We performed the data analysis using IBM SPSS software, version 22, used for statistical analysis in social science. We conducted one-way analyses of variances (ANOVAs) to study the effects of IBWs and BRs on intended behavioral response and respondents’ perception of warning information. An average sample of n = 280 respondents per experimental condition satisfied requirements for ANOVAs. Moreover, we conducted one-way ANOVAs to include risk perception, knowledge, experience with warnings, and with thunderstorms, as well as relevant demographic variables (age, gender, education, urban/rural) as a covariant. In a further step, we realized a series of two-way ANOVAs to study possible interaction effects of warning type and participants’ characteristics (knowledge, experience, and other sociodemographic variables) on the dependent variables mentioned above. In addition, we conducted a series of multiple regression analyses to investigate the effects of perception and understanding of the warning information, along with risk perception, hazard and warning experience, thunderstorm knowledge, whether people have already reacted to warnings in the past, gender, age, level of education, and living area, on the intended action “interrupt the hike.”

4. Results

Qualitative interviews with experts involved in the natural hazard warning chain underlined the suspicion within the expert community that providing more specific impact information would be beneficial in the case of high-impact weather warnings. The majority of the stakeholders thought that IBWs and BRs may be especially helpful for the public and increase the overall understanding and right interpretation of warnings. They thought that in order to be helpful in taking appropriate actions, warnings should focus on “impacts and recommendations, instead of warning categories” (staff from MeteoSwiss). In addition, they highlighted the need for IBWs and BRs in order to “help people understand the difference between different warning levels, e.g. a significant and severe hazard” (weather forecaster from the private sector meteorology). Many experts also depicted the problems with current warnings and the potential benefits of IBWs and BRs in describing a frequent wrong behavior during thunderstorms: most people believe that one has to protect the window from the hail that can accompany thunderstorms and therefore let the shutter down, although the opposite reaction would be the right one [“metal shutters, which can dent, are more often damaged by hail than are glass windows” (staff from insurance company)]. Thus, information should focus on IBWs and BRs, instead of on the weather phenomenon itself and the respective warning category. However, some experts (such as weather forecasters), but especially those from the cantonal (i.e., regional and local) administrations, raised two main concerns. First, a message that presents too general information regarding impacts and behavioral recommendations could have negative impacts on the credibility of the warning message and thus of the weather provider. For example, one expert mentioned that too general recommendations could lead people to “ignore the warning” (weather forecaster from private sector meteorology). Second, too much information might overwhelm the recipients and people may not understand it. One practitioner said, for instance, that “we have to be careful with providing additional information on impacts, which may be too complicated or not adhere to the local circumstances” and highlighted that one has to “find the right balance between too much information and too little information” (cantonal official). To summarize, we identified a perception in the expert community of the importance of IBWs and/or BRs, as well as some concerns. In the following we will see if our survey results in any way contradicted expert beliefs (i.e., the concerns about understanding and perception of warning information).

Moving to the quantitative results from the survey instrument, we start by examining the relationship between warning type and the perception variables. Figure 2 shows the effects of IBWs and BRs on all five perception attributes, and these are very similar: warning perception is lowest among those who received SWs and highest among those who received IBWs with BRs, whereas SWs with BRs perform better in terms of perception than IBWs without BRs. In general, the effect of warning type was highly significant on all five variables: clarity of information [F (3, 1105) = 7.32, p < 0.001, r = 0.14]; credibility of information [F (3, 1103) = 3.96, p = 0.008, r = 0.1]; level of concern [F (3, 1117) = 7.65, p < 0.001, r = 0.14]; understanding of threats to participants’ safety [F (3, 1112) = 15.41, p < 0.001, r = 0.20]; and understanding how to respond to the information [F (3, 1109) = 19.16, p < 0.001, r = 0.22] (using the Holm–Bonferroni method to control for the familywise error rate for multiple hypothesis tests). Likewise, we also observed highly significant linear trends.

View Full Size

Mean perception of warning information for all four warning types. Bars indicate how much participants agreed or disagreed on a five-point Likert scale from “totally disagree” to “totally agree” with each of the five perception attributes. Error bars indicate the 95% confidence interval.

Citation: Weather, Climate, and Society 10, 4; 10.1175/WCAS-D-18-0038.1

• Download Figure
• Download figure as PowerPoint slide

As we document in more detail in Table 3, for all the five variables, planned contrasts revealed that those participants who received SWs reported the information to be less clear and easy to understand, and less credible; furthermore, recipients were less concerned about their safety and less understood the threats to their safety and how to respond to the information compared to those who received a warning with information about behavioral recommendations or impacts or both. In addition to the contrasts provided in Table 3, we did an extra test comparing SWs with SWs + BRs. Bonferroni post hoc testing showed that respondents who received the SWs reported finding the warning less clear (M = 4.07, SD = 0.73) and less understood the threats to their safety (M = 3.92, SD = 0.76) and how to respond to the warning (M = 3.83, SD = 0.84) compared to those who received the SWs + BRs (M = 4.25, SD = 0.61), p = 0.010, respectively (M = 4.21, SD = 0.63), p < 0.001, and (M = 4.17, SD = 0.66), p < 0.001. Thus, there is an individual effect of BRs on these three perception attributes. We also found an additive effect of BRs and IBWs on improving perception/understanding across all five attributes as shown in Table 4.

Table 3.

Effects of IBWs and BRs on perception and understanding of warning information. Note: one asterisk refers to p < 0.05, two asterisks refer to p < 0.01, and three asterisks refer to p < 0.001; M indicates the mean, SD the standard deviation, and df the degrees of freedom

View Table

Table 4.

Additive effect of BRs and IBWs on perception and understanding of warning information.

View Table

Similarly, for the second contrast in Table 3, participants who received IBWs reported the information as less clear and less easy to understand, and they less understood the threats to their safety and how to respond to the information, compared to those who received a warning with information about BRs. Again, beside the planned comparisons, some additional Bonferroni post hoc testing showed that respondents who received the IBWs were less concerned (M = 3.50, SD = 0.95) and less understood the threats to their safety (M = 4.05, SD = 0.73) and how to respond to the warning (M = 3.95, SD = 0.74) compared to those who received the IBWs + BRs, p = 0.037, respectively, p < 0.001, and p < 0.001 (M and SD in Table 4). Moreover, we find that those who received the SWs + BRs better understood the threats to their safety (M = 4.21, SD = 0.63) and how to respond than those who received IBWs (M = 4.05, SD = 0.73), p = 0.030, respectively (M = 3.95, SD = 0.74), p = 0.003.

The third planned contrast, comparing SWs + BRs to IBWs + BRs, was nonsignificant for four of the five variables but not for the concern perception [SWs + BRs (M = 3.54, SD = 0.98) vs IBWs + BRs (M = 3.74, SD = 0.96), p = 0.014, Cohen’s d = 0.21]. Thus, this indicates that there is an effect of IBWs on feelings of concern evoked by the warnings. Bonferroni post hoc testing for the five perception and understanding attributes was not significant. So, there is no difference in the influence of SWs on perception and understanding of warning in comparison to IBWs. Including thunderstorm knowledge and warning and thunderstorm experiences, as well as sociodemographic variables, as covariants did not alter the significance of the differences in warning type. However, some covariables had a significant influence on warning response alone. As results were very similar for all the five perception and understanding attributes, we summarized the influence of participants’ characteristics below. People with positive past warning experience (i.e., they received correct warnings), females, and older people perceived and understood the warnings, on average, better than those with negative warning experience (i.e., warnings were incorrect) [t(1114) = 3.29, p = 0.001, r = 0.10], males [t(1114) = 3.59, p < 0.001, r = 0.11], and younger people [t(1114) = 6.71, p < 0.001, r = 0.20].

The next set of relationships we examined was between perception and intended action. Three of the perception attributes correlate with the action “interrupting the hike.” Table 5 shows that irrespective of warning type received, perceptions of credibility and concern, as well as understanding the threats to their safety, influence taking the intended risk-minimizing behavior, as do risk perception and whether people have already taken action in response to warnings in the past. First, the higher the risk perception, the higher the likelihood to interrupt the hike (a unit increase in risk perception predicted a 0.058 unit increase in interrupting the hike). Second, when people already reacted to warnings in the past, they are more likely to interrupt the hike (by 0.298 compared to people who have not reacted). Moreover, two perceptions and three variables related to the understanding of information showed significant effects: believing the message to be credible (β = 0.119), being concerned about one’s safety (β = 0.343), and understanding the threats (β = 0.146).

Table 5.

Multiple linear regression with the risk-minimizing behavior “interrupt the hike” as the dependent variable. Note: R² = 0.33 (p < 0.001); two asterisks refers to p < 0.01, three asterisks refers to p < 0.001. We controlled for the familywise error rate for multiple hypothesis tests by using the Holm–Bonferroni method. Significant results are in boldface; B indicates the unstandardized coefficients, SE the standard error, and β the standardized coefficients.

View Table

Finally, we examined the relationships between warning type and intended action. The upper line in Fig. 3 represents the strong risk minimizing behavior “interrupt the hike” and shows a linear trend [F (1, 1117) = 18.94, p < 0.001, r = 0.13], indicating that as the warnings provide more information, the likelihood to stop the hike increased proportionately, whereas IBWs alone have a greater effect than BRs alone on interrupting the hike. The effect of warning type was significant [F (3, 1108) = 6.80, p < 0.001, r = 0.13]. Planned contrasts revealed that those participants who received a SW (M = 3.86, SD = 1.05) reported being less likely to interrupt the hike than those who received a warning with information about behavioral recommendations or impacts or both [M = 4.06, SD = 0.93, t (1108) = 2.93, p = 0.004, Cohen’s d = 1.31]. As an additional test, Bonferroni post hoc testing showed that respondents who received the SWs (M = 3.86, SD = 1.05) had a significantly lower likelihood to interrupt the hike compared to those receiving IBWs + BRs (M = 4.22, SD = 0.85), with p < 0.001. Thus, there is an additive effect of BRs and IBWs on taking the intended action of interrupting the hike. The second planned contrast showed that those people who received the SWs + BRs (M = 3.95, SD = 1.00) were less likely to interrupt the hike than those who received either one of the IBWs (M = 4.11, SD = 0.88), t (1108) = 2.39, p = 0.017, Cohen’s d = 0.18. According to Bonferroni post hoc testing, respondents who received the SWs + BRs (M = 4.06, SD = 0.93) had a significantly lower likelihood to interrupt the hike compared to those that received the IBWs + BRs (M = 4.22, SD = 0.85), p = 0.005. Thus, there is an individual effect of IBWs on performing the strong risk minimizing behavior. Finally, in the last contrast, we observed IBWs with BRs (M = 4.22, SD = 0.85) to result in an even higher likelihood to interrupt the hike compared to IBWs without BRs (M = 4.01, SD = 0.91), t (1108) = 2.53, p = 0.012, Cohen’s d = 0.23. Thus, there is an effect of BRs on performing the strong risk minimizing behavior.

View Full Size

Mean likelihood to engage in different behaviors for all four warning types. Lines indicate how much participants agreed or disagreed on a five-point Likert scale from “totally disagree” to “totally agree” with each of the five behavioral responses. Solid lines represent the more or less risk minimizing behaviors, dashed lines represent information seeking actions. Error bars indicate the 95% confidence interval.

Citation: Weather, Climate, and Society 10, 4; 10.1175/WCAS-D-18-0038.1

• Download Figure
• Download figure as PowerPoint slide

The nonprotective behavior “not alter plans” and the more risk maximizing behavior “continue the hike but avoid potential dangers” (i.e., to engage in dangerous behaviors) are represented by the two bottom lines in Fig. 3. Results show that the people who received SWs are more likely to engage in a dangerous behavior than those people receiving SWs + BRs, IBWs, or IBWs + BRs, and that those who receive IBWs + BRs are least likely to not change plans. While the effect of warning type was significant for the nonprotective behavior “do not alter plans,” F (3, 1093) = 3.95, p = 0.032, r = 0.10 [with a significant linear trend, F (1, 1093) = 10.38, p = 0.001, r = 0.10], planned contrasts showed that compared to SWs (M = 2.00, SD = 1.04) having any extra information in the warnings (so either BRs or IBWs or both) (M = 1.80, SD = 0.87) significantly decreased the likelihood to not alter plans, t (1093) = −3.11, p = 0.002, Cohen’s d = 0.21. As an additional test, Bonferroni post hoc testing revealed that respondents who received the SWs (M = 2.00, SD = 1.04) had a significantly higher likelihood to not alter plans compared to those who received IBWs + BRs (M = 1.74, SD = 0.90), p = 0.004. Thus, there is an additive effect of BRs and IBWs on avoiding nonprotective behavior. When controlling for the familywise error rate using the Holm–Bonferroni method, the effect of warning type on the behavior “continue the hike but avoid potential dangers” became insignificant, F (3, 1117) = 2.87, p = 0.105, r = 0.09.

Third, the dashed lines in Fig. 3 show the changes in the two information-seeking behaviors “check out other information sources” and “pay close attention to the environment”. IBWs and BRs had no effect on whether participants were more or less likely to seek information, with respective values of F (3, 1117) = 1.49, p = 0.215 and F (3, 1117) = 1.27, p = 0.283.

Including thunderstorm knowledge and warning and thunderstorm experiences, as well as sociodemographic variables, as a covariant did not alter the significance of the differences in warning type. However, some covariables had a significant influence on warning response alone. First, people who are less knowledgeable about thunderstorms, males, and people who have not reacted to prior warnings are more likely to not take action compared to those with more knowledge [t(1094) = −2.15, p = 0.032, r = 0.07], females [t(1094) = −3.45, p = 0.001, r = 0.10], and people with warning experience [t(1094) = −4.48, p < 0.001, r = 0.13]. Second, males are also more likely to continue the hike while adapting to the circumstances compared to females [t(1118) = −3.33, p = 0.001, r = 0.10].

Third, people who have higher education levels, females, older people, and people who have reacted to prior warnings are more likely to interrupt the hike compared to those with lower education levels [t(1109) = −3.00, p = 0.003, r = 0.09], males [t(1109) = 4.70, p < 0.001, r = 0.14], younger people [t(1109) = 2.41, p = 0.016, r = 0.07], and people with warning experience [t(1109) = 3.40, p = 0.001, r = 0.10]. Fourth, females, younger people, and people who have reacted to prior warnings are more likely to seek information compared to males [t(1118) = 3.22, p = 0.001, r = 0.10], older people [t(1118) = −3.63, p < 0.001, r = 0.11], and people with warning experience [t(1118) = 2.30, p = 0.021, r = 0.07]. Moreover, two-way ANOVAs showed no significant interaction effects between warning type and these individual characteristics of participants.

5. Discussion

Impact-based warning and behavioral recommendations both increase warning perception and improve intended behavioral response, with effects that are additive. The ordering between IBWs and BRs differed according to perception or intended behavioral response. BRs alone have a greater effect than IBWs alone in increasing perception, whereas IBWs alone have a greater effect than BRs in promoting intended behavioral response.

Our study of warning perception and understanding included five attributes (perceptions of credibility and concern, and understanding of the warning, the threats to safety, and how to respond), three of which correlate with the risk minimizing action “interrupt the hike.” Irrespective of warning type received, perceptions of credibility and concern, as well as understanding of threat, influence interrupting the hike, as do risk perception and whether people have already taken action in response to warnings in the past. These findings confirm other research results. Credibility of official warnings has shown to affect warning response (Ripberger et al. 2015a). High levels of fear have been found to be an influencing factor in prompting behavioral response (Sutton et al. 2018). As Lazo et al. (2015) highlight, several studies focusing on storms have found that perceived risk is also a primary factor in taking decisions to evacuate an area. Recommendations for practitioners that can be derived from these findings are that they have to make sure that the warnings are clear and easy to understand, and that they evoke feelings of concern and make people understand the threats to their safety, which can be stimulated by including additional impact and/or behavioral information into the warnings.

The combined effects of IBWs and BRs are on all five perception attributes. When only one piece of information is given, BRs have a significant effect, but IBWs do not. First, we found an individual but overall nonsignificant effect of IBWs (except for perceptions of concern). This finding does not support the findings of Potter et al. (2018), as they identified a significant effect of IBWs alone on perception. Second, we found an individual significant effect of BRs. Recipients of warnings that included information about BRs reported finding the warning clearer and easier to understand than recipients of warnings without these recommendations. They were also more concerned about their safety and better understood the threats to their safety and behaviors to engage in. Finally, IBWs and BRs together had the greatest effect on improving perception and understanding (on all five attributes). However, message length could also be an influencing factor, resulting in higher perception and understanding levels for IBWs + BRs (i.e., the longest message). For example, Sutton et al. (2018) have found that longer messages including information about the location of impact, threat-associated risks, and recommended protective actions were associated with better message understanding and quicker intended response.

Intended behavioral responses included more or less risk-minimizing behaviors, continuing with a nonprotective action, and also seeking information actions. There were individual and additive effects of both IBWs and BRs on the positive intended action, a combined effect of IBWs and BRs on negative intended actions, and no effect on seeking intended information actions. First, we observed that IBWs, with or without BRs, resulted in a higher likelihood of interrupting the hike compared to SWs both with and without BRs. Our findings add to the body of literature on this topic; they complement the results of Casteel (2016), but not of Perreault et al. (2014) and Potter et al. (2018). At the same time, we also observed IBWs with BRs to result in an even higher likelihood of taking intended positive action compared to IBWs without BRs. Second, we found that recipients of SWs were more likely to not alter plans compared to the other three warnings with enhanced information and especially those with IBWs + BRs, which confirms earlier findings of Potter et al. (2018). Third, unlike Potter et al. (2018), we found no significant effect of IBWs or BRs on seeking more (or less) information. This may be due to the fact that BRs, which add more information, were absent in the warning messages used in the study of Potter et al. (2018). In sum, if practitioners expect recipients of warning information to engage in risk-minimizing behaviors (i.e., to evacuate an area), then they should focus on disseminating information that contains both IBWs and BRs (e.g., in very severe weather warnings). However, we have to bear in mind that many NMHSs are responsible to warn only for severe weather events. Giving recommendations is often the responsibility of the national or regional civil protection agencies. For example, in most cantons in Switzerland it is either the firemen or the policemen who have to provide the population with recommendations as they are the ones with the best local knowledge, and thus are expected to know the potential impact. Integrating BRs into warning messages will therefore need to ensure that weather forecast and impact related uncertainty is taken into account and communicated appropriately, and to also guarantee that time delays due to consultation are kept as small as possible.

Differences between individuals (personal experience with weather hazards, level of knowledge about weather hazards, age, education, gender, urban/rural) had no significant interaction effects on either perception or intended behavioral response. This complements the results of Casteel (2016), who found that knowledge had no effect on decision-making in response to warnings. However, some of these personal characteristics had an influence on warning response, warning perception, and understanding. Similar to previous research on hurricane evacuation likelihood (Lazo et al. 2015; Morss et al. 2016), we found that females are more likely to interrupt the hike (and less likely to take no action) than males. Females also perceive the warnings to be more credible, are more concerned for their safety, and understand the warning, the threats to safety, and how respond to them better than males do. Potter et al. (2018) also found an influence of gender on credibility, but do not for the concern perception. In terms of age, we found that the older people are more likely to interrupt the hike, which confirms prior research (Morss et al. 2016; Peacock et al. 2011). Like Potter et al. (2018), we also found that perceptions of credibility and concern are higher among older people. Moreover, warning experience significantly influenced waning response. People with positive warning experience also perceived the warnings more positively and people who have already reacted to prior warnings were more likely to interrupt the hike and less likely to not change behavior. This confirms general research that experience positively influences taking protective action (e.g., Sharma and Patt 2012). Education levels, knowledge, and residence area had only a very weak or no influence on warning response.

The consequences of our results for practitioners are twofold. First, they are an encouraging sign as they suggest that IBWs and BRs are clear and understandable, and evoke feelings of concern and make people understand the threats to their safety regardless of the target audiences’ sociodemographic characteristics or their knowledge and experience with weather and warnings. Second, it may well be that targeting warning messages on the basis of some individual characteristics of the recipients, as literature and best practices suggest (e.g., NOAA 2016; Lindell and Perry 2012), is not as important as providing clear IBWs and BRs. Whereas experts were concerned about information overload, it appears that in all cases providing more information only helps. First, people understood the warnings: those warnings with most information (namely IBWs + BRs) were perceived as the clearest and easiest to understand among the four warning messages and also people better understood the threats to their safety and how to respond to the warning. Second, there is no evidence that the quite general information reduces trust in the warnings as IBWs + BRs were believed to be most credible and people were most concerned about their safety.

Our results extend our knowledge compared to previous studies in terms of removing ambiguity about whether it is IBWs or BRs that matter, and whether the effects are on perception or intended behavioral response: we show benefits in all cases. We show a large potential for improvements in warning design to lead to greater appreciation of danger and taking intended protective actions. The research provides important validation and support for a wider implementation of IBWs and BRs, especially when considered in combination, into future warnings in order to improve individual decision-making and, ultimately, save lives.

The one shortcoming of our study, which we share with previous studies, is to make use of self-reported intended responses to a hypothetical and imagined situation, rather than a field observation of actual behavior in response to actual danger, even though some argue that intentions are a good proxy for actual behaviors (Ripberger et al. 2015b). In addition, an imagined situation suffers from a lack of real consequences for decisions and people might be less risk-seeking in real-life decisions, as was pointed out also by Kox and Thieken (2017). Furthermore, the sampling technique and the different lengths of the warning messages used in this experiment limit the generalizability of our findings. Finally, evaluating different warning messages with impact and/or BRs is somewhat subjective. As Casteel (2016) highlighted, a legitimate question to ask is what constitutes an effective response to warnings. Thus, it is important to remember that given the inherent uncertainty in severe weather events as well as the local context and conditions that can influence the weather, the most effective response may not always be the same action. Also, given the predictive nature of the entire warning process, any suggested impacts or actions is only the forecaster’s best guess at one specific point in time (Casteel 2016).

A future research direction could be to test the effectiveness of different warning types during a real event, rather than using a simulated event [e.g., as McCaughey et al. (2017) did]. While such a study would offer results of high validity, it would also pose complicated methodological and potential ethical challenges. The methodological challenge can be addressed by using an existing app (or create a new one), to disseminate two types of warnings, IBWs and SWs, to the subscribers for a real event and retrieve warning response. As additional information in the warnings influences how the warning information is perceived but does not necessarily translate into a lower likelihood to engage in a dangerous behavior or any change in seeking more information, future research should also address the link between perceptions and less risk minimizing actions.