1. Introduction
The National Weather Service (NWS) is tasked with providing the public with information about marine and atmospheric conditions throughout the United States and its territories. One specific role of the NWS is the forecasting of severe weather and the administration of warnings in “the United States, its territories, adjacent waters and ocean areas, for the protection of life and property” (from the official NWS Mission Statement; see http://www.nws.noaa.gov/mission.php). Generating a weather warning draws on a wide range of expertise involving science, technology, social interaction, public policy, and related domains. Although significant effort has gone into developing issuance criteria used to determine when a severe weather warning is warranted (http://www.nws.noaa.gov/mission.php), how the public utilizes warning information in the face of a hazardous event is unclear (Lazo et al. 2009). Researchers have also acknowledged the importance of prior experiences and knowledge in decision making during weather warnings, although their specific influences are not as well investigated (Dillon et al. 2011; Leiserowitz 2006). Current research involving other types of natural hazards have also identified the relationship between knowledge/experience and decision making and believe it to be a reasonable approach when investigating behaviors associated with natural hazard risk (Morss et al. 2008; Whitmarsh 2008).
This study specifically addresses the influence of prior experiences, termed “episodic memories” here, and knowledge, termed “semantic memories” here, during an artificial weather warning. Episodic memories represent actual life experiences, and likely have both knowledge and affective components (Tulving 1972). Semantic memories are information about the world that has been learned through reading, media, schooling, and other indirect experiences (Tulving 1972). Specifically, we designed a classroom experiment that probed both their semantic and episodic memories and gave participants a sense of immediacy, as if a real tornado warning were being issued. This experiment addressed the central research question: “What influences do varying levels of semantic and episodic memories have in influencing decision making about tornado warnings?” We hypothesized that those individuals with personal memories of a tornado would be more inclined to take precautionary measures than those without such experience; however, results did not support that outcome.
When an individual is faced with a risk, both types of memory play an active role during analytical and experiential processing of associated decisions, since decisions draw upon past memories (Marx et al. 2007). Constraints on time, knowledge, and cognitive load are vital when determining a course of action in risk situations. These impact an individual’s potential options when responding to warnings by limiting the opportunity to integrate past experiences and information into decision making.
2. Theoretical foundations
A number of studies have suggested a link between experiences, knowledge, and decision making under risk exists. In fact, people have a tendency to make decisions based on small amounts of previous experience (Hertwig and Pleskac 2010). Navigating risk situations requires a complete understanding and assessment of the risk and needs to take into consideration previous experiences and other sources of information impacting the context of the situation (Kasperson and Kasperson 1996). When exposed to risk, individuals call upon a wide range of tools in order to assess potential danger and determine the appropriate behavior. Often the level of an individual’s knowledge of the risk, as well as the fear associated with the actual event itself, will be considered before taking action (Morrow 2009). People are exposed to a variety of risk situations as part of their daily routine. These situations vary greatly, ranging from business decisions to medical predicaments, all of which are impacted by experiences and knowledge.
Studies of business managers have shown that decisions often follow intuitive paths, or are highly context dependent. Management decisions can also be based on an individual’s fear of being wrong, lack of a common decision-making structure across groups of individuals, or insufficient knowledge (Riabacke 2006). Medical decision making is another area that can require individuals to make decisions under risk. Pregnancy, for example, is a situation in which individuals are presented with information typically categorized as a high or low risk. Prior experience with pregnancy alleviates some of the unfamiliarity associated with decision making around risks, although differences in pregnancies require an individual to evaluate risk in the context of the current medical condition (Lyerly et al. 2007). Many of these same factors, such as knowledge, previous experience, context, and the perceived level of risk, influence decisions encountered during exposure to natural hazards.
Decision making under natural hazard risk is similar to risk encountered in other situations. Uncertainty arises because of lack of knowledge, in this case of natural hazards, as well as the unpredictability associated with the systems and processes that influence natural hazards. The way in which these hazards impact society and an individual’s choices related to the hazard is often difficult to ascertain because of the information and outcomes of past events that are available, many of which may represent an imperfect record of past events (UKCIP 2003).
Both previous experience, or episodic memory, and knowledge, or semantic memory, lie at the core of decisions made during risk situations. As a result, this study will take a closer look at how each impacts decisions made during exposure to risk associated with tornado warnings.
For the purposes of this study, we will focus on specific memories, both episodic and semantic, and attempt to determine the role they play when an individual is exposed to tornado warnings. During decision-making tasks, people may draw on semantic knowledge and/or episodic experience as they make decisions based on their available memories. When attempting to determine the behavior associated with these memories, it is difficult to predict the outcome since decidedly different scenarios may occur despite similar past experience or knowledge. In this study, we hypothesized that someone who has experienced a tornado will act upon the warning by heeding the message to take precautionary measures. Conversely, those individuals who have not previously experienced a tornado and have little knowledge of the actual threat may choose to ignore the warning. These hypotheses will also take into account the impact of an individual’s knowledge of severe weather and tornadoes, since it has been shown to influence decisions (Johnson 1992), and the likelihood that an inverse relationship between the risk and benefits associated with taking precautionary measures during a tornado warning may exist (Slovic 2000).
3. Methods
A purposeful sample of students participated in this study, completing four questionnaires and participating in a brief stimulus. The first was the Semantic Severe Weather Questionnaire (SQ1), which determined the student’s level of knowledge about severe weather. Next was the Episodic Severe Weather Questionnaire (EQ1), which determined the student’s personal experience with severe weather, in general and specifically tornadoes. Following the completion of SQ1 and EQ1, participants completed a 4-point Likert scale Severe Weather Decision Questionnaire (D1), followed by a slideshow stimulus, then completion of the final Severe Weather Decision Questionnaire (D2). Questionnaire data provided 1) semantic knowledge of severe weather (SQ1); 2) personal experience with severe events and tornadoes (EQ1); and 3) potential responses to tornado warnings, collected both before (D1) and after (D2) viewing of a slideshow stimulus consisting of tornadoes and related damage. The intent of the stimulus was to determine the impact of a recent experience on decision making, by either enabling prior recollection or creating a recent experience.
a. Participants
Forty-nine undergraduate students at a large Midwestern institution participated in a 1-h, cognitively based experiment focusing on decision making during a regularly scheduled classroom time and location. The experimental population averaged 22 years in age and was composed of male (n = 21) and female (n = 28) participants. The population consisted of varying levels of academic completion (freshman, sophomore, junior, and senior), all of whom were nonscience majors.
b. Data collection
Data were collected in four successive steps during class (Fig. 1). Participants were introduced to the cognitive definitions of semantic and episodic memory, given a brief explanation of the experiment, and informed that participation was voluntary. A 2-page semantic questionnaire (SQ1) was then administered. The questionnaire consisted of 5 demographic, 3 informational, and 10 weather knowledge questions mainly centered on severe thunderstorms and tornadoes. The episodic questionnaire (EQ1) followed and consisted of 7 questions focusing on personal experience with severe weather and tornadoes. After completing the semantic and episodic questionnaires, the students were asked to complete a decision-making questionnaire (D1). The decision-making questionnaire prompted students to consider their actions during a tornado warning, and listed 10 potential scenarios of varying degrees of precautionary measures for students to choose from, and posed 2 open-ended questions. D1 was intended to determine actions participants would most likely take during a tornado warning. Upon completion of these questionnaires, the participants viewed a 5-min slideshow stimulus containing a series of images of actual tornadoes and associated property damage. Participants were asked to immerse themselves in the stimulus as if they and their families were actually experiencing the tornadoes both physically and emotionally. The slideshow stimulus was intended to invoke a personal experience, or act as a priming event in the case of those with previous tornado experiences. After viewing the slideshow, the students responded anew to the decision-making questionnaire (D2). The intent of D2 was to determine the impact of the stimulus on the decision-making process. The session ended with a deeper explanation of the experiment and a question-and-answer period with the participants.
Experiment progression and timing of the questionnaires and slideshow stimulus.
Citation: Weather, Climate, and Society 5, 1; 10.1175/WCAS-D-11-00042.1
We investigated the factorability of the 10 items contained within the decision-making instrument (D1) through exploratory factor analysis (Table 1). Criteria demonstrating factorability were met, including a Kaiser–Meyer–Olkin measure of sampling adequacy of 0.733, above the 0.6 value recommended for factor analysis. Bartlett’s Test of sphericity was significant [χ2 (45) = 257.4, p < 0.001]. Communalities were at or above 0.5, except for one just below that value, indicating shared variance with other items. Given these data, exploratory factor analysis was performed on all 10 items. A 1-factor solution was indicated by scree plot analysis, while eigenvalue analysis suggests a 3-factor solution. Ultimately, after considering the strength of factor loadings, number of items per factor, and internal consistency of factors, we retained one scale, the “Ignore Warning Scale,” which contains four items (5, 6, 7, 9; Table 1) and which explains 80.1% of the variance in these four items. This scale has high Cronbach’s alpha of 0.90. A high score on this scale indicates that a respondent is likely to ignore a tornado warning, while a low score indicates a less likely probability that they would ignore the warning. Confirmatory factor analysis on the same items after the stimulus (D2) confirms the stability of this scale, with equivalent factor loadings and alpha values at both administrations.
Factor loadings and communalities for the Ignore Warning Scale. The strength of the factor loadings and number of items per factor suggest the retention of one scale, the “Ignore Warning Scale.”
c. Data analysis
Participant questionnaire responses were collected and analyzed quantitatively. First, we describe the results of the semantic and episodic questionnaire using simple descriptive statistics. Second, a Related Samples Wilcoxon Signed Rank Test was performed on the decision results both before and after the stimulus. The test is useful when investigating any change in scores from one time point to another when individuals are subjected to more than one condition, which in this case was the stimulus. We considered results for four different populations: the high semantic, high episodic group, the high semantic, low episodic group, the low semantic, high episodic group, and the low semantic, low episodic group. Based on these results, a multivariate analysis of variance (MANOVA) was performed to determine the significance of the main and interaction effects suggested by the Related Samples Wilcoxon Signed Rank Test results.
d. Validity and reliability
Quantitative research enables a researcher to ask specific questions, generate hypotheses, collect data in a controlled setting, and subsequently test hypotheses (Golafshani 2003). Validity in quantitative research is dependent on the ability to measure and observe the intended variables and reliability is implied when the experiment and resulting data can be replicated. Criteria for validity and reliability of our survey instruments and study design have been established. We explain each of these criteria in detail; see Clark and Libarkin (2011) and references therein for more information. Note that overall, the findings of the current study are limited to college-aged populations in the Midwest.
Content validity is the extent to which questions are actually measuring the construct intended. Often, experts are asked to comment on item validity. In this case, materials generated by two expert organizations, the National Oceanic and Atmospheric Administration (NOAA) and NWS, were used in developing all three survey instruments. The D1 and D2 utilized information from the NOAA and NWS tornado preparedness guidelines (NOAA 1995). The EQ1 incorporated items modified from the Enhanced Fujita Scale (http://www.spc.noaa.gov/efscale/). The SQ1 included items generated by NOAA and NWS (NOAA 1995; http://www.srh.noaa.gov/jetstream/), as well as misconceptions identified in the research literature (NOAA 1995). All three surveys were revised based on comments from two geoscientists from the Geocognition Research Laboratory.
Conclusion validity is the ability to determine the relationship between variables being researched. Experts exposed to the findings of this research through formal presentations at professional conferences found the results noteworthy and deserving of further investigation. In general, experts agreed that our finding of a relationship between knowledge/experience and decision making was enlightening; this concurs with current research on other types of natural hazards (Morss et al. 2008; Whitmarsh 2008).
Construct validity refers to the ability of instruments to correlate with the underlying theory of the study. Current understanding of the importance of knowledge in decision making suggests that higher knowledge should lead to more effective decision making (Johnson 1992). In general, participants with higher SQ1 scores displayed more effective tornado-warning decisions relative to those with lower SQ1 scores. This effect was mitigated by the impact of personal experience, the focus of this study.
Reliability of the instruments was evaluated through demonstrated consistency of tests results with similar populations utilizing comparable instruments. The small sample size limits internal reliability considerations, although ongoing work will allow for comparison of findings across populations.
4. Results
Results of SQ1 indicate that participants were on average “somewhat likely” to consult a weather forecast as part of their daily routine. Participants checked weather forecasts an average of 4.9 times each week (approximately once daily) and referenced a wide range of sources for their information about current weather (Fig. 2). Results also indicate that 63% of participants understood the difference between the NWS severe weather statements. The two most prominent statements in this study, tornado watch and tornado warning, should prompt different levels of action; a watch suggesting conditions exist for tornado development, and a warning indicating a tornado exists. While the majority of participants understood that a “warning” was the most urgent NWS statement and required immediate personal action, a substantial number of participants (37%), were oblivious to the difference and would not realize the need for taking precautionary measures during a tornado warning. Based on the results of SQ1, participants were divided into two groups determined by their SQ1 score. Those participants who averaged below the overall average, which happened to be 50%, were classified as low semantic (LS), and those who scored above 50% were classified as high semantic (HS).
Participants primary source of weather information. The Internet was the primary source of weather information, followed by TV with friends and family (F/F) ranking third.
Citation: Weather, Climate, and Society 5, 1; 10.1175/WCAS-D-11-00042.1
Results of EQ1 indicate that students had a significant amount of experience with severe weather and tornadoes. Responses to two questions, “Have you personally experienced a severe weather event” (EpiQ3) and “Have you personally experienced a tornado” (EpiQ5), indicated that 88% of the participants had personal experience with severe weather warnings and events, while slightly fewer than 50% had experienced a tornado (Table 2). These questions were used as indicators of overall severe weather and tornado episodic experience. Participants who indicated no experience with tornadoes were classified as low episodic (LE), and those participants who indicated experience with at least one tornado were classified as high episodic (HE). Since this study concerns itself specifically with tornado experience, the statistical analysis incorporated only responses to EpiQ5, which specifically asks the participant to state their tornado experience.
Participants (n = 49) and their personal lifetime warning and event experiences. Results indicate a significant experience with severe weather warnings and a fair amount of experience with tornadoes.
The main and interaction effects of knowledge of severe weather impacts and varying levels of personal experience with tornadoes on decision making before and after a simulated warning (stimulus) were considered. Essentially, the interaction allowed for insight into the impact of knowledge and experience when students were faced with a decision that involved ignoring a tornado warning. The impact of the interaction emphasized the greater likelihood of participants in the high semantic group toward taking precautionary measures during tornado warnings (Fig. 3).
D1 and D2 results showing the interaction of experience and knowledge when faced with the decision of ignoring a tornado warning.
Citation: Weather, Climate, and Society 5, 1; 10.1175/WCAS-D-11-00042.1
For the Wilcoxon Signed Rank Test, we tested the hypothesis that no difference exists between the mean of D1 and D2. The test is often used when comparing two related samples, which in our case are D1 and D2. The test results indicated that those possessing low episodic and low semantic memories and low episodic and high semantic memories both experienced significant impacts due to the stimulus. Those possessing high episodic and low semantic memories and high episodic and high semantic memories experienced only moderate change after the stimulus (Table 3).
Results of the Related Samples Wilcoxon Signed Rank Test indicating that the stimulus had significant impact on the low episodic/low semantic and low episodic/high semantic groups.
A MANOVA was completed in order to assess the statistical significance of the independent variables “episodic” and “semantic” and the interaction term “episodic*semantic” on the dependent variables D1 and D2. This was chosen since the MANOVA allows us to investigate whether changes in the independent variables impact multiple dependent variables, and offer insight into the interactions between dependent and independent variables.
MANOVA results suggested that the main effects, semantic and episodic memory were not significant with semantic D1 = 0.953, D2 = 0.638, and episodic D1 = 0.654, D2 = 0.521. However, the interaction of semantic and episodic, D1 = 0.128, D2 = 0.174, memory may be considered somewhat significant given the relatively small number of participants and the overall difference in significance indicated by the test (Table 4).
Results of the MANOVA analysis suggesting the main effects were not significant but the interaction of the two were somewhat significant given the small number of participants.
In all, participant responses to SQ1 and EQ1, question EpiQ5, indicate interesting interactions between knowledge and prior tornado experience and anticipated responses to tornado warnings. These responses were compared using the previously identified “choose to ignore” warning scale. Participant decision making prestimulus (D1) indicates that the HE group with a high level of knowledge (HS) was more likely to take precautionary measures during a tornado warning than the HE group with a low level of knowledge (LS), while the LE group with a low level of knowledge (LS) was more likely to take precautionary measures than the LE group with a high level of knowledge (HS). The interaction between semantic and episodic memory was significant after the stimulus. Participant decision making poststimulus (D2) resulted in interesting movements among the groups. The HE group with a high level of knowledge (HS) was found to be more likely to ignore a tornado warning than the HE group with a low level of knowledge (LS), while the LE group with a high level of knowledge was less likely to ignore a tornado warning than the LE group with a low level of knowledge (LS). It appears that tornado experience may provide impetus for participants to take precautionary measures, especially when combined with a high level of knowledge of severe weather/tornadoes (Table 5). It is not known, however, how long the impact of recent memories, or priming of existing memories, of tornado experiences provides this effect.
Pre- and poststimulus results illustrating the participants likelihood of ignoring a warning for both D1 and D2.
5. Discussion and conclusions
Data analyzed from the study point to interesting interactions between knowledge and experience. Varying levels of severe weather knowledge coupled with instances, or lack of, personal tornado experience allow for a wide range of potential outcomes when an individual is faced with a tornado warning. Although the study was limited by availability of prior research in the topic area as well as sample size (n = 49), and the relatively narrow scope of the student demographics, the results are still considered attributable to cognitive and demographic factors that substantiate the need for further research using refined methods and more diverse participants.
In general, this study suggests that participants with a higher level of semantic knowledge (HS) will be more likely to take precautionary measures during a tornado warning, than those who scored lower on the semantic questionnaire (LS). In addition, students who have previously experienced a tornado (HE) are the least likely to react favorably to a warning if they possess less knowledge of severe weather. This may be due to the possibility of the students discounting the apparent danger and relying on the greater availability of relatively harmless past experiences. After the stimulus, the episodic group increased their likeliness to react to the warning, but only slightly. The results tend to support the likelihood of semantic memories, or knowledge, providing greater impetus for action during a severe weather event, when compared to only episodic memories, or experiences of tornadoes.
Both semantic and episodic memories play an important role in an individual’s decision making during risk situations. Though decision making is often assumed to rely more on the recollection of prior experiences, consideration should be given to the impacts of an individual’s severe weather knowledge. In this study, participants were faced with a decision-making task both before and after viewing a 5-min slideshow stimulus of tornadoes and related storm damage. The results indicate a strong correlation between the level of severe weather knowledge and the likelihood to heed the warning associated with the event. Overall, previous tornado experience (HE) indicated a greater tendency to ignore a tornado warning than those who had not experienced a tornado event.
The stimulus itself provides tantalizing suggestions about the structure that more effective tornado warnings could take. In this study, the stimulus consisted of tornado footage and related storm damage that may have impacted the warning response among participants. While it has been shown that imagery plays a part in decision making (Leiserowitz 2006), it is not the intent of this paper to determine the impact of the portrayal of tornado images upon the students as an affective component of memory. Generally both D1 and D2 results illustrate that individuals with greater knowledge of severe weather events were less likely to ignore the tornado warning, while those possessing previous tornado experiences were more likely to ignore the warning, especially if they also had a low level of severe weather knowledge. While it is possible that a crossover effect due to the short lapse of time between D1 and D2 may have impacted the likelihood of participants to ignore a tornado warning, it is beyond the scope of this study to determine how separate treatments may have altered the results; that is better determined by further study. Recency of prior tornado warnings may impact the likelihood of action since recent events, even those without an actual tornado event, are more salient in an individual’s memory than a memory of an actual tornado experience in the past. If prior tornado or other severe weather events were relatively minor in intensity, individuals may use this information by anchoring their judgments on information/experiences that are not relevant to their current situation (Newell and Pitman 2010).
Greater knowledge of severe weather provides participants with additional means to determine when precautionary measures should be taken during severe weather. The conclusions reported in this study reinforce the importance of memories on decision-making behavior. Specifically, knowledge is a critical factor when one is considering taking precautionary measures during a tornado warning. Modifying an individual’s knowledge, or semantic memory, would appear a logical step in influencing precautionary behavior since this could be accomplished through a variety of educational or media avenues. Attempting to modify an individual’s personal experience, or episodic memory, would be much more difficult if not impossible since this would require direct experience with a tornadic event. Since knowledge appears to be a significant factor based on the results of this study, future research will address the limitations of the current study by providing an educational intervention to control for differences in severe weather knowledge among participants.
This information, coupled with an understanding of the impacts of additional variables utilized during risk decisions, plays a significant part in determining future directions in the development of tornado-warning practices.
Acknowledgments
This work was completed while in residence at the Geocognition Research Laboratory at Michigan State University. I thank Emily Geraghty Ward and Julie Libarkin for assistance with this paper as well as all students who graciously participated in this research.
APPENDIX A
Semantic Questionnaire (SQ1)
Citation: Weather, Climate, and Society 5, 1; 10.1175/WCAS-D-11-00042.1
APPENDIX B
Episodic Questionnaire (EQ1)
Citation: Weather, Climate, and Society 5, 1; 10.1175/WCAS-D-11-00042.1
APPENDIX C
Decision-Making Questionnaire (D1 and D2)
Citation: Weather, Climate, and Society 5, 1; 10.1175/WCAS-D-11-00042.1
REFERENCES
Clark, S. K., and Libarkin J. C. , 2011: Designing a mixed-methods research instrument and scoring rubric to investigate individuals’ conceptions of plate tectonics. Spec. Pap. Geol. Soc. Amer., 474, 81–96, doi:10.1130/2011.2474(07).
Dillon, R., Tinsley C. , and Cronin M. , 2011: Why near-miss events can decrease an individual’s protective response to hurricanes. Risk Anal., 31, 440–449.
Golafshani, N., 2003: Understanding reliability and validity in qualitative research. Qual. Rep., 8 (4), 597–607.
Hertwig, R., and Pleskac T. , 2010: Decisions from experience: Why small samples? Cognition, 115, 225–237.
Johnson, B., 1992: Advancing understanding of knowledge’s role in lay risk perception. Risk, Issues Health Saf., 189, 189–212.
Kasperson, R., and Kasperson J. , 1996: The social amplification and attenuation of risk. Ann. Amer. Acad. Polit. Soc. Sci., 545, 95–105.
Lazo, J. K., Morss R. E. , and deMuth J. L. , 2009: 300 billion served: Sources, perceptions, uses, and values of weather forecasts. Bull. Amer. Meteor. Soc., 90, 785–798.
Leiserowitz, A., 2006: Climate change risk perception and policy preferences: The role of affect, imagery, and values. Climatic Change, 77, 45–72, doi:10.1007/s10584-006-9059-9.
Lyerly, A. D., Mitchell L. M. , Armstrong E. M. , Harris L. H. , Kukla R. , Kuppermann M. , and Little M. O. , 2007: Risks, values, and decision making surrounding pregnancy. Obstet. Gynecol., 109, 979–984.
Marx, S. M., Weber E. U. , Orlove B. S. , Leiserowitz A. , Krantz D. H. , Roncoli C. , and Phillips J. , 2007: Communication and mental processes: Experiential and analytic processing of uncertain climate information. Global Environ. Change, 17, 47–58.
Morrow, B., 2009: Risk behavior and risk communication: Synthesis and expert interviews final report for the NOAA Coastal Services Center. NOAA Tech. Rep., 53 pp.
Morss, R. E., Demuth J. L. , and Lazo J. K. , 2008: Communicating uncertainty in weather forecasts: A survey of the U.S. public. Wea. Forecasting, 23, 974–991.
Newell, B. R., and Pitman A. J. , 2010: The psychology of global warming. Bull. Amer. Meteor. Soc., 91,1003–1014.
NOAA, 1995: Tornadoes, nature’s most violent storms: A preparedness guide, including safety information for schools. NOAA, 11 pp. [Available online at http://www.nws.noaa.gov/os/brochures/tornado.shtml.]
Riabacke, A., 2006: Managerial decision making under risk and uncertainty. IAENG Int. J. Comput. Sci., 32 (4), 1–7.
Slovic, P., 2000: The Perception of Risk. Earthscan Publications Ltd., 518 pp.
Tulving, E., 1972: Episodic and semantic memory. Organization of Memory, E. Tulving and W. Donaldson, Eds., Academic Press, 381–403.
UKCIP, 2003: Climate adaptation: Risk, uncertainty and decision-making. UKCIP Tech. Rep., 154 pp.
Whitmarsh, L., 2008: Are flood victims more concerned about climate change than other people? The role of direct experience in risk perception and behavioral response. J. Risk Res., 11 (3), 351–374.
