To Warn or Not to Warn: Factors Influencing National Weather Service Warning Meteorologists’ Tornado Warning Decisions

Jiyoun Kim aDepartment of Communication, University of Maryland, College Park, College Park, Maryland

Search for other papers by Jiyoun Kim in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-9946-6972
,
Anita Atwell Seate aDepartment of Communication, University of Maryland, College Park, College Park, Maryland

Search for other papers by Anita Atwell Seate in
Current site
Google Scholar
PubMed
Close
,
Brooke F. Liu aDepartment of Communication, University of Maryland, College Park, College Park, Maryland

Search for other papers by Brooke F. Liu in
Current site
Google Scholar
PubMed
Close
,
Daniel Hawblitzel bNational Weather Service Nashville, Nashville, Tennessee

Search for other papers by Daniel Hawblitzel in
Current site
Google Scholar
PubMed
Close
, and
Theodore Funk cNational Weather Service Indianapolis, Indianapolis, Indiana

Search for other papers by Theodore Funk in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

Weather warnings are critical risk communication messages because they have the potential to save lives and property during emergencies. However, making warning decisions is challenging. While there have been significant advances in technological weather forecasting, recent research suggests that social factors, including communication, influence warning meteorologists’ decisions to warn. We examine the roles of both scientific and social factors in predicting warning meteorologists’ decisions to warn on tornadoes. To do so, we conducted a cross-sectional survey of National Weather Service forecasters and members of management in the southern and the central regions of the United States, as well as conducted a retrospective data analysis of cross-sectional survey data from the central region Tornado Warning Improvement Project. Results reveal that dependency on radar velocity couplet and a variety of social factors predicted decisions to warn.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Hawblitzel’s current affiliation: National Weather Service Twin Cities, Chanhassen, Minnesota.

Corresponding author: Jiyoun Kim, jkimcomm@umd.edu

Abstract

Weather warnings are critical risk communication messages because they have the potential to save lives and property during emergencies. However, making warning decisions is challenging. While there have been significant advances in technological weather forecasting, recent research suggests that social factors, including communication, influence warning meteorologists’ decisions to warn. We examine the roles of both scientific and social factors in predicting warning meteorologists’ decisions to warn on tornadoes. To do so, we conducted a cross-sectional survey of National Weather Service forecasters and members of management in the southern and the central regions of the United States, as well as conducted a retrospective data analysis of cross-sectional survey data from the central region Tornado Warning Improvement Project. Results reveal that dependency on radar velocity couplet and a variety of social factors predicted decisions to warn.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Hawblitzel’s current affiliation: National Weather Service Twin Cities, Chanhassen, Minnesota.

Corresponding author: Jiyoun Kim, jkimcomm@umd.edu

1. Introduction

Adequately and effectively warning the public about severe weather is a constant challenge for the National Weather Service (NWS; Hurley and Hawblitzel 2021). Warning meteorologists play an important role in both the tornado forecast and the warning processes. When a potential threat is detected, meteorologists must determine whether a tornado warning is needed and issue warnings to at-risk publics. To make these decisions, warning meteorologists collect and analyze meteorological data from a variety of sources to prepare and issue forecast and warning products. Despite the critical role warning meteorologists play in alerting the public about severe weather threats, only a handful of prior studies have examined meteorologists’ severe weather communication (e.g., Anthony et al. 2014; Compton 2018).

Deciding whether to issue a severe weather warning is not a simple decision. The decision-making process is infused with warning meteorologists’ concerns for saving lives (Bostrom et al. 2016; Brooks and Doswell 2002) and avoiding unnecessary alarm and financial costs to their communities (Doswell 2004). Previous research suggests that warning meteorologist-specific characteristics, such as prior experience with a similar weather event and expertise, are important to consider in understanding warning decision-making processes (Hawkins et al. 2017; Hoffman et al. 2017; Pliske et al. 2004). Other research suggests that social factors, particularly communication, are important to consider as well (Daipha 2015; Nagele 2015; Roeder et al. 2021). Indeed, the lion’s share of prior research is qualitative in nature (e.g., Bostrom et al. 2016; Roeder et al. 2021), which provides inductively derived foundational knowledge for subsequent survey research, as we conducted in this study. Moreover, this study builds on the prior literature by testing a single model with meteorological scientific factors, warning meteorologist characteristics, and social factors. By doing so, we examine how, if at all, scientific factors work with social factors in predicting meteorologists’ decisions to warn on tornadoes.

To address these research needs, we conducted a cross-sectional survey of NWS forecasters and members of management operating in NWS offices located in the southern and the central regions of the United States (N = 119). We also conducted a retrospective data analysis of cross-sectional survey data from the central region Tornado Warning Improvement Project (TWIP; N = 326) (see Fig. 1). The survey inquired about meteorologists’ decisions to warn about tornadoes, as well as asking for a second opinion in the tornado warning process. We also measured characteristics illuminated by the qualitative literature, such as perceived information overload (e.g., Daipha 2015; Karstens et al. 2018). By gaining insights about decision-making processes, we can provide avenues for improvements to the warning system and insights for continued theory development.

Fig. 1.
Fig. 1.

Map of regions that are included in the study.

Citation: Weather, Climate, and Society 14, 3; 10.1175/WCAS-D-20-0115.1

2. Literature review

a. Warning meteorologists’ decision to warn

A warning meteorologist’s decision to warn on severe weather events has been described as a mix of art and science wherein weather forecasters assemble data provided by technology (Daipha 2015). Existing technology constitutes a significant component in meteorologists’ weather forecasting process. With current weather technologies, meteorologists decide whether to issue a tornado warning primarily based on reliable data from their real-time observations, such as radar and storm-spotter reports (Durage et al. 2013). Prior research has emphasized the benefit of accurate (i.e., at least high-quality) weather information in warning decision-making processes (Mehrkanoon 2019).

However, work by Daipha (2015) and Morss et al. (2015) underscores the importance of social factors in the decision to warn. Specifically, Daipha (2015) conceptualized weather forecasters’ decision-making as “a fundamentally practical activity that relies on available heuristics, techniques, and resources—as determined by both the object at hand and the evolving material and symbolic context of action—to fashion a provisionally coherent solution to routine and non-routine challenges” (p. 20). It is important to note that this definition not only accounts for the specific scientific informational needs required to make a decision (e.g., near-storm environment data), but also underscores the importance of social influences (e.g., communication between warning meteorologists) in the decision-making process.

b. Warning meteorologists’ opinion seeking and warning decision-making

Weather forecasters are expert decision-makers, but expertise is rarely formed in isolation (Daipha 2015; Hoffman et al. 2017). Despite their professional capacity, decision-makers still sometimes engage in communicative processes (e.g., soliciting others’ opinions) to build the informational basis of a collective choice (Fine 2009; Yaniv and Kleinberger 2000). For instance, in an investigation of hurricane-related warning decisions, Bostrom et al. (2016) found that, prior to issuing any warnings, National Hurricane Center (NHC) meteorologists were likely to consult with their superiors and with relevant weather forecasting offices (WFOs) before making decisions. Similarly, meteorologists consulted with local advisors to assess the level of wind impacts before defining a proper warning level (Hemingway and Robbins 2020). Work has yet to replicate these findings in the tornado context, which has very different spatial and temporal scales than the hurricane context. Furthermore, the level of warning authority is also different in the tornado context. For instance, severe thunderstorms and tornadoes are warned at the local level, while hurricanes are warned at the national level. However, it is possible that some warning meteorologists seek opinions from others to make the best decision about tornado warnings, as suggested by prior research (Alexiev et al. 2019).

The extent to which meteorologists seek second opinions is an open empirical question. Some meteorologists may be more convinced of their own decision based on available evidence and make their own decision without input from others. In other situations, some meteorologists may want to hear others’ opinions before making a decision. Also, meteorologists sometimes may need others’ opinions, but other times may not need input from others, particularly when the event is clear-cut. The decision-making literature proposes several concepts that could help in predicting warning meteorologists’ opinion seeking, including task difficulty, perceived information overload, personal experience, and decision-making confidence. We discuss each of these concepts in turn.

1) Task difficulty

The decision-making literature has shown that people often ask for advice to make a quality decision when they find tasks challenging (e.g., Brooks et al. 2015; Gino and Moore 2007). Even the most advanced computer systems cannot perfectly predict the weather (Hoffman et al. 2017), making warning decisions challenging. Still, meteorologists report that some storms are harder to warn on than others, leading to variability in the decision-making process based on the task difficulty (e.g., Hoffman et al. 2017). Furthermore, there simply may not be time during a major severe weather outbreak for meteorologists to seek a second opinion even when the decision to warn is a difficult task.

2) Perceived information overload

Research suggests that technological advancements have produced data overload for weather forecasters (Daipha 2015; Hoffman et al. 2017). Informational overload occurs when meteorologists have more data than they can use and interpret to make forecasting decisions (Hoffman et al. 2017). Studies have shown that having more technology-generated data does not necessarily result in warning decision improvement (e.g., Stewart et al. 1992; Webber and Zheng 2020) because data overload may decrease the ability to accurately and promptly process weather information (Brotzge and Donner 2013). With an abundance of data, meteorologists express concern about not knowing which data and modeling tools are the most useful to consult, given the multitude of available options (Doswell 2004; Morss et al. 2015). In some cases, data from live camera video, dual-polarization, trained storm spotters, and other sources present clear evidence to issue a tornado warning without meteorologists experiencing information overload (Golden and Adams 2000; Liu et al. 2020; National Severe Storms Laboratory 2022).

3) Personal experience

Warning meteorologists report relying on their professional and personal experience when making warning-related decisions (Bostrom et al. 2016; Heinselman et al. 2012; Hemingway and Robbins 2020; Hoffman et al. 2017), although years of experience does not always translate into more accurate predictions (Boustead and Mayes 2014). Researchers find that inexperienced meteorologists (e.g., newer meteorologists) are more likely to issue tornado warnings than are experienced meteorologists (Boustead and Mayes 2014; Ellis et al. 2020).

4) Decision-making confidence

Scholars have suggested that people with high confidence are less likely to seek further information for decisions than those who are less confident (Phillips et al. 2014). In addition, feeling confident decreases advice-taking intention (Gino and Moore 2007; See et al. 2011; Soll and Larrick 2009), although confidence in a judgment does not indicate the objective accuracy of that judgment (Oskamp 1965).

Given the literature, we offer the first set of hypotheses (H1): Perceived task difficulty (H1a), perceived information overload (H1b), personal experience (H1c), and confidence in warning decision-making (H1d) predict warning meteorologists’ decisions to ask for a second opinion prior to issuing a tornado warning.

c. Warning meteorologists’ decisions to issue a tornado warning

As noted above, research suggests that warning meteorologists’ severe weather decision-making process is a combination of art and science. According to Daipha (2015), in order to make a severe weather warning decision, meteorologists must perform screenwork (i.e., using computer screens to process scientific information). Screenwork is important because meteorologists must be able to see the weather to be able to think and talk about the weather (Daipha 2015; Hoffman et al. 2017). Daipha (2015) noted that screenwork integrates scientific factors (i.e., meteorological data) with social factors (e.g., discussions with colleagues about the data at hand). The current study advances knowledge by integrating both scientific and social factors into one model to provide insight into how these factors may work together. Moreover, previous research has noted that personal characteristics, such as experience, influence the decision to warn (Doswell 2004). One of the primary goals of the current study is to examine how these factors work together in predicting meteorologists’ tornado warning decisions. In the following sections, we review what factors affect warning meteorologists’ decisions to issue a tornado warning. We first review literature on evidence-based decision-making, then we review literature on meteorologists’ characteristics and social factors. We conclude with our hypotheses.

1) Evidenced-based warning decisions

Although warning decisions are made by individual meteorologists’ subjective judgments, decisions should be based on the level of situational awareness, taking into account various scientific considerations including the meteorological dataset and conceptual models (Alley et al. 2019; Andra et al. 2002; Hoffman et al. 2017; Scher 2018). Forecasting and warning decisions are primarily led by established meteorological parameters. Improved observing capabilities have led to a better understanding of storm behavior and structure (Brooks et al. 2019), which in turn have helped forecasters use this information to improve forecast accuracy (Alley et al. 2019; Wilson et al. 2019).

To accomplish the NWS mission of saving lives and property through accurate and timely weather predictions, meteorologists rely on a variety of datasets and techniques to warn about severe weather (Anthony et al. 2014). For instance, NWS forecasters use WSR-88D radar—a specialized radar that uses the Doppler effect to produce velocity data of objects at a distance (Coleman et al. 2011; Yu and Li 2017) to assess real-time tornado potential. This velocity data may include the inference of storm-scale rotation via a feature commonly known as a velocity couplet. Additionally, data on meteorological parameters are collected via technology such as satellites and radars and fed into statistical modeling software. Results of such modeling may then be used to influence warning decision-making, for example, by helping to assess the meteorological near-storm environment (Bostrom et al. 2016; Daipha 2015; Hoffman et al. 2017; Morss et al. 2015).

In addition to radar signals that indicate tornado potential, meteorologists can also make tornado warning decisions based on submitted credible reports of a tornado hazard (Durage et al. 2013; Golden and Adams 2000). Liu et al. (2020) also noted that, in some cases, trained spotters provide clear evidence that helps to issue tornado warnings.

Given the literature reviewed above, we hypothesize the following (H2): Dependency on radar velocity couplet (H2a) and spotter reports (H2b) influences warning meteorologists’ decisions to issue a tornado warning.

2) Decisions to warn: Personal-level and communal-level characteristics

Forecasting and warning decisions are holistically made at the individual and communal levels (Bostrom et al. 2016; Daipha 2015; Demuth et al. 2012). In terms of individual level predictors, our previous work (Atwell Seate et al. 2019) suggested that warning meteorologists have a general attitude toward the warning decision-making process, wherein some meteorologists are more conservative (i.e., less likely to warn) and some meteorologists are more liberal (i.e., more likely to warn) in their decision to warn. We have termed this general attitude tornado warning philosophy. While little work has explored the role of tornado warning philosophy in predicting decisions to warn, several strands of research suggest the possibility. For instance, Daipha’s (2015) ethnography of NWS forecasters’ decision-making processes showed that forecasters believe people approach forecasting differently.

When deciding to issue a tornado warning, with the ultimate goal of success at predicting all tornadoes, warning meteorologists try to strike a balance among three tornado-related verification metrics: false-alarm rate (FAR; the number of times that a warning was issued but not observed divided by the total number of warnings issued), probability of detection (POD; the hit rate for a categorical forecast), and lead time (i.e., the average difference in time between the warning issuance and the event; Brotzge and Donner 2013; Ellis et al. 2020). While most warning meteorologists comprehensively consider tornado-related verification metrics, some meteorologists put more importance on POD than on FAR (Ellis et al. 2020). By contrast, in the realm of tornado forecasts there is a fear that overwarning at-risk publics (i.e., false alarms) may diminish people’s willingness to take appropriate protective actions (i.e., termed “crying wolf” in the literature; Breznitz 1984; Ripberger et al. 2015). While the evidence for the crying-wolf effect is mixed (e.g., Simmons and Sutter 2009; Trainor et al. 2015), research showed no false-alarm effect on the public for tornado warnings in the Southeast (Lim et al. 2019). Although recent data suggest that false-alarm perceptions for warning meteorologists may be overblown (Lim et al. 2019; Walters et al. 2020), it is an open empirical question whether these perceptions influence meteorologists’ decisions to warn, as some recent empirical work suggests.

Although it is conjecture, it seems that these patterns reported in the literature underscore a general attitude toward issuing a tornado warning. We empirically examine this possibility in the current study. Given the literature, we predict the following (H3): Tornado warning philosophy (H3a) and thinking about two verification metrics—POD/FAR—(H3b) influence warning meteorologists’ decisions to issue a tornado warning.

Daipha’s (2015) ethnographic work also points to the importance of communal factors in warning meteorologists’ decisions to warn on severe weather. According to this approach, forecasting decisions are rarely made in isolation, and meteorological decisions are often linked to social factors, including communication. Organizationally, factors affecting weather decision-making include organizational constraints and policies, official guidelines for issuing warning products, and office-level established wisdom (Demuth et al. 2012). According to Daipha (2015), meteorologists may not go against organizational wisdom in issuing severe weather warnings because following institutionally approved courses of action may prevent forecasters from considering alternatives.

The communication among meteorologists is an important office culture dynamic that influences decision-making (Daipha 2015). We examine both asking for a second opinion (e.g., Daipha 2015; Hoffman et al. 2017) and inclusive communicative culture (Nembhard and Edmondson 2006) in predicting tornado warning decisions. As previously noted, some meteorologists routinely ask for guidance in their decisions to warn (Bostrom et al. 2016; Hemingway and Robbins 2020). Moreover, teams working in inclusive office environments (e.g., working with a democratic and supportive leadership) are likely to feel greater psychological safety (Nembhard and Edmondson 2006), which in turn may predict the decision to warn on tornadoes. Therefore, we propose the following hypotheses (H4): Warning meteorologists’ asking for a second opinion (H4a) and inclusive communication culture in the workplace (H4b) influence warning meteorologists’ decisions to warn about tornadoes.

Given that there can be multiway conversations about meteorological data that occur during warning operations, one of the goals of the current study is to examine how scientific and social factors work together in predicting warning meteorologists’ decisions to warn on tornadoes. Therefore, we ask the following research questions (RQ1): How, if at all, do meteorological evidence-driven decision-making factors interact with subjective interpretation (i.e., personal-level) factors in predicting decisions to warn on tornadoes (RQ1a)? And, how, if at all, do data-driven decision-making factors interact with communal-level factors in predicting decisions to warn (RQ1b) on tornadoes?

3. Study 1

a. Method

The current study employed a participatory action research approach (Ivankova 2015). Participatory action research stresses experts (i.e., NWS forecasters and management members) participation and action in all stages of the research process from study design, data collection, and interpreting results. Specifically, we partnered with three WFOs in the southern region. We worked closely with each WFO’s science and operations officer (SOO) to develop the survey materials (described below). These data are from a larger project examining NWS forecasters’ tornado warning decisions, their relationships with key partners, and how WFOs communicate tornado risk.

Although many forecasters feel great pressure and heavy responsibility when making decisions to put a specific area under a tornado watch or warning (NWS 2016), scarce research empirically demonstrates how warning meteorologists make decisions about tornado warnings. Hence, prior to conducting our survey research, we conducted preliminary research to identify and validate critical features about the decision-making and warning process. All procedures described below were approved by our university’s institutional review board (IRB).

1) The current study

We conducted a 45-min online survey, which was hosted by Qualtrics—an online survey platform—from March 2019 through July 2019. Items about decision-making and warning communication processes were distributed across the survey instrument.

(i) Participants

We surveyed NWS forecasters and management team members in the central and southern regions; 144 participants agreed to participate in the survey. To enhance data quality, 25 responses that completed fewer than half of the survey items were eliminated from the final dataset. In total, there were 119 cases for data analysis. See Table 1 for the participants’ demographic information.

Table 1

Participants’ demographic information.

Table 1

(ii) Procedure

NWS southern region headquarters distributed the recruitment script and survey link to meteorologist-in-charge (MIC), SOO, and warning coordination meteorologist (WCM) listservs in the southern region on our behalf. The recruitment script instructed management members to share the recruitment materials with their operational forecasters. Our initial participant recruitment began on 19 March 2019. A reminder email was sent about a month later. The survey was included in weekly notes sent across the southern region and was announced in relevant regionwide meetings (e.g., monthly SOO meetings). To increase the sample size for our statistical analyses, we began recruiting from the central region using the same procedure as in the southern region (e.g., having central region headquarters recruit on our behalf) in June 2019.

(iii) Measures

Self-reported survey items were constructed based on prior academic literature (e.g., Daipha 2015; Hoffman et al. 2017; Jahn and Black 2017) and the central region Tornado Warning Improvement Project (Funk 2018). TWIP is a working group of forecasters with the goal of improving tornado warnings in the central region. We used items from TWIP based on the recommendation of our SOO partners. Specifically, given our participatory action approach, one member of the research team met with our partners to discuss the academic literature and survey item development. During this meeting, one of our partners indicated that the survey items from TWIP would be appropriate for the current study. However, some concepts (e.g., management inclusive communication) that we identified from the literature were not measured by TWIP, and we used the academic literature to develop those items (e.g., Jahn and Black 2017; Nembhard and Edmondson 2006). See Table 2 for measurement information, including variable names, survey items, descriptive statistics, and reliability estimates.

Table 2

Psychometric properties for variables and items (study 1). Each variable was measured on a 7-point scale. Higher numbers reflect being “more” of the concept under consideration.

Table 2

(iv) Analytical technique

To test the hypotheses and research questions, we employed hierarchical ordinary least squares (OLS) regression models. To find the predictors’ relative explanatory variance for the final model, we entered the independent variables into blocks (see, Woltman et al. 2012). The order of the blocks was determined based on the fact that meteorologists interpret the data when objective scientific data are detected, discuss it with others, and make a warning decision. We discussed the analytic plan with our SOO partners prior to analysis. We included interaction terms in these regression models. Each of the interaction terms was constructed by multiplying the standardized value of its components (i.e., the specific predictors) to reduce multicollinearity between an interaction term and its components (Little 2013).

b. Results

1) Factors influencing forecasters’ decisions to ask for others’ opinions

We proposed that perceived task difficulty (H1a), perceived information overload (H1b), personal experience (H1c), and decision-making confidence (H1d) predict warning meteorologists’ decision to ask for a second opinion prior to issuing a tornado warning. As shown in Table 3, our analysis showed statistical evidence that perceived task difficulty has significant effects on warning meteorologists’ decisions to ask for a second opinion. In other words, people are more willing to ask for other’s opinions if they perceive more difficulty in making a warning decision. We failed to show significant effects of perceived information overload, personal experience, and decision-making confidence on seeking a second opinion. Hence, H1 was partially supported.

Table 3

Regression model predicting ask for a second opinion. Here one asterisk indicates p < 0.05, two asterisks mean p < 0.01, and CI is confidence interval.

Table 3

2) How do warning meteorologists decide to issue tornado warnings?

We explored to what extent warning meteorologists rely on meteorological data to make tornado warning decisions (H2). As shown in Table 4, the result showed that there was a significant effect of radar velocity couplet dependency (H2a) on the decision of whether to issue a tornado warning. However, there was no significant effect of storm reports on warning decisions (H2b). Thus, H2 was partially supported.

Table 4

Regression model predicting tornado warning decision. One asterisk indicates p < 0.05, and two asterisks mean p < 0.01. Cell entries are before entry standardized regression coefficient (beta) for block 4, and final standardized regression coefficient (beta) for block 1, 2, and 3.

Table 4

Next, we examined whether personal factors influence warning meteorologists’ decisions to issue a tornado warning (H3). There was no empirical evidence indicating that tornado warning philosophy (H3a) predicts warning decisions, whereas there was a positive linear association between thinking about verification metrics (H3b) and meteorologists’ decisions to issue a tornado warning. That is, warning meteorologists thinking about verification metrics seems to play a role in the warning decision-making process. So, H3 was partially supported.

We also posited the roles of communal factors in predicting tornado warning decisions (H4). Contrary to our expectations, there was no empirical evidence indicating that asking for a second opinion (H4a) and perceived inclusive communication culture in the workplace (H4b) predicted warning decisions. Hence, H4 was rejected.

3) How, if at all, does the scientific and the social work together in predicting decisions to warn?

One of the current study goals is to examine how, if at all, scientific factors work with social factors in predicting meteorologists’ decisions to warn. This study provides no evidence that there are interaction effects on warning decisions (see, Table 4).

4. Study 2: Replication analysis

We also conducted a retrospective review of cross-sectional survey data collected by the central region TWIP (N = 326) to replicate the statistical analyses from study 1.

a. Data

In November 2019, the TWIP team provided us access to a de-identified dataset they collected using a voluntary cross-sectional survey of National Weather Service forecasters and management members. The TWIP team collected these data in Fall 2016. The goal of the TWIP survey closely mirrors study 1—to provide insight into the scientific and social factors influencing warning meteorologists’ decisions to warn on tornadoes. Indeed, given the overlap in research scope between study 1 and the TWIP survey, one of our SOO partners suggested that we include survey items from the TWIP survey in the study 1 survey instrument. Hence, study 1 and the TWIP survey contain many of the same variables, allowing us to replicate our study 1 analyses with the TWIP data. However, the data provided by TWIP did not include demographic information. See Table 5 for the descriptive statistics and reliability estimates for the TWIP data.

Table 5

Psychometric properties for variables and items (study 2). Each variable was measured on a 5-point scale. Higher numbers reflect being “more” toward the measures.

Table 5

b. Results

In predicting meteorologists’ decisions to ask for a second opinion before issuing a tornado warning, the TWIP data revealed significant effects of perceived task difficulty and perceived information overload. However, no statistical evidence indicated significant effects of personal experience and decision-making confidence on seeking a second opinion (Table 3).

Also, as with the findings from study 1, the results showed a significant main effect of dependency on radar velocity couplet in predicting meteorologists’ decisions to issue a tornado warning. Unlike the results from study 1, no statistical evidence indicated that thinking about verification metrics predicted warning decisions. Instead, there were significant main effects of storm-reports consideration, tornado warning philosophy, and asking for a second opinion on meteorologists’ decisions to warn. That is, the TWIP data reveal that the impact of storm reports has a more significant effect on tornado warning decisions for respondents who give more consideration to storm reports from spotters than for those who give less consideration to storm reports. Also, tornado warning philosophy has a greater influence on tornado warning decisions for respondents with a liberal warning philosophy than for respondents with a more conservative warning philosophy. Asking for a second opinion influences tornado warning decisions for respondents who seek more opinions from others than for those who less frequently ask for a second opinion (see Table 4).

Additionally, there is a significant interaction of storm-reports consideration and thinking about two verification metrics in predicting meteorologists’ decisions to warn. We used the PROCESS macro to interpret these interactions (Hayes 2013, model 1). Results reveal a positive linear relationship between storm-reports consideration and tornado warning decisions among warning meteorologists who are less likely to think about verification metrics (POD/FAR) [i.e., 1 standard deviation (SD) below the mean; b = 0.31, t = 3.04, and p < 0.01] and the average level of thinking about verification metrics (i.e., at the mean; b = 0.16, t = 2.03, and p < 0.05). However, there was no association between storm-reports consideration and decisions to warn among meteorologists who are more likely to think about verification metrics (i.e., 1 SD above the mean; b = −0.00, t = −0.03, and p > 0.10). That is to say, our data show that storm-reports consideration has more impact on tornado warning decisions for respondents who think less about verification metrics than for respondents who think more about verification metrics (see Fig. 2). This finding answers RQ1.

Fig. 2.
Fig. 2.

Interaction effect between storm-reports dependency and verification-metrics consideration on warning decision.

Citation: Weather, Climate, and Society 14, 3; 10.1175/WCAS-D-20-0115.1

5. Discussion

Warnings are critical risk communication messages because they have the possibility to save lives and property. While there is a large body of scholarship on how to craft warning messages (e.g., Ash et al. 2014; Casteel and Downing 2016) and how members of the public respond to warnings (e.g., Bean et al. 2016; Heath et al. 2019; Sutton et al. 2015), there are only a few studies on how warning meteorologists decide to issue warnings, including for tornadoes. Moreover, this prior research on tornado warning decisions has primarily focused on improved technological forecasting methods (e.g., Hemingway and Robbins 2020; Karstens et al. 2018; Scher 2018). Little research has focused on how meteorologists use technical information to make warning decisions (Doswell 2004) and how social factors affect warning decisions (Daipha 2015). To fill these research gaps, we provide further insight into what factors affect meteorologists’ warning decisions.

a. Main findings

We assume that the tornado warning decision is a human decision informed by both objective and subjective inputs. The main findings from the two studies are as follows. First, meteorologists are more willing to ask for other’s opinions if they perceive difficulty in making a warning decision. Second, meteorologists’ tornado warning decisions are influenced by relying on radar velocity couplet in the warning decision-making process. Results from study 2 additionally show that meteorologists are influenced by second opinion seeking, considering spotters’ storm reports, and having a liberal tornado warning philosophy. Last, our findings reveal that social and scientific factors sometimes work together in predicting tornado warning decisions. Specifically, results from study 2 showed that storm-reports consideration worked with thinking about two verification metrics (POD/FAR) in predicting tornado warning decisions.

The findings of this research should be interpreted with caution. While the findings from the two studies converge, particularly with regard to the role of task difficulty in predicting second-opinion seeking, there were meaningful differences between study 1 and study 2 in predicting decisions to warn. There are several factors to consider in this regard. First, the data were collected several years apart, and a continued organizational shift toward impact-based and decision-support services during this time could have influenced results (Lazo et al. 2020; Liu and Atwell Seate 2021). Second, study 1 contained respondents from both the central and southern regions, whereas study 2 only had respondents from the central region. Cultural differences and the frequency and types of tornado events between the two regions could have influenced the results. Last, in comparison with study 2, study 1 has a relatively small sample size. Although power analysis results revealed we had the minimum sample size necessary for the analysis (required sample size N = 90 for a 95% confidence interval), a comparatively small number of elements used in the analysis likely reduced the power of the study 1 and potentially missed scientifically meaningful findings. Therefore, we expect the future research can unpack whether the different results are due to changes in the forecasting environment, organizational change, NWS region, or related to study procedures.

In processing our findings, we suggest that scholars and practitioners pay attention to the presented effect sizes. First, our effect sizes were small to medium in magnitude. Due to the magnitude of these effect sizes, caution should be exercised in their application to understanding the actual warning issuing process until they have been replicated by future work. Also, we found that meteorologists not only make evidence-driven decisions based on meteorological factors but are also influenced by subjective judgments based on personal factors. However, according to the effect size shown in our results, meteorologists appear to be reliant relatively more on the observation of a rotation couplet on Doppler radar when making warning decisions. Future research should examine how NWS procedures and trainings emphasize issuing of a warning when rotation was observed, as previous research indicates that forecasters may not go against organizational wisdom in their decision-making (Daipha 2015).

b. Implications of this study

This study contributes to a growing body of scholarship examining how scientific and human factors influence decisions to warn. By performing regression models, this study examines which factors are predicted across meteorologists, and how these factors influence each other. In turn, the factors identified in this study could be tested in other hazardous weather contexts (e.g., hurricanes and floods) to develop a robust model of high-impact weather warning decision-making. Future research should provide evidence whether the predictors examined in this research work across a variety of storm types.

Furthermore, our results show that technology and science development are not the only considerations when issuing forecasts. Although advanced technologies enable a better understanding of the climate and weather (Brooks et al. 2019; Wilson et al. 2019) and have helped warning meteorologists make more effective warning decisions (Karstens et al. 2018), our results confirm previous research that weather forecasting is a human decision (e.g., Daipha 2015). Our findings indicate that thinking about two verification metrics (i.e., POD/FAR) is associated with tornado warning decisions. Therefore, this study provides a solid foundation for future research on the role of human judgment in a data-driven weather forecasting world.

We should also acknowledge that this is the first attempt to empirically explore the concept of “ask for a second opinion” in the weather warning context. The idea of “ask for a second opinion” is important but complex, especially in the forecast operations space, because there can be much multiway conversation during warning operations. To provide an initial exploration of this concept, we operationalized the definition as “seeking opinions of others to make the best decision on issuing a tornado warning” and measured it with a straightforward single statement: “I frequently ask for a second opinion before issuing a TOR [tornado] warning.” Despite the contribution of this research to this topic, we admit that measuring the complex concept in such a simple way is an avenue for future research improvement. For instance, the current study shows that soliciting others’ opinions influences warning decisions but cannot show how opinion seeking affects the warning performance. However, we note the findings of existing research that effective team communication helps make rapid and improved forecast decisions that affect public safety with maximized expertise (Roeder et al. 2021). Hence, we look forward to future research to provide deeper insights into this topic.

c. Limitations of the study

While this study has meaningful implications for the weather warning decision-making process, as with any research, there are several limitations to be acknowledged. First, most variables relied on a single item. For instance, as previously mentioned, asking for a second opinion was measured on a single item, which falls short of capturing the complexity of this topic. Like prior research (e.g., Ash et al. 2014; Palm et al. 2020), we did not evaluate the single-item measures used in this study for content, convergent, and discriminant validity. We suggest future research replicate our results with more sophisticated, multiitem measures.

Second, the verification measure is double-barreled, that is, it asks about both POD and FAR in a single item. Although some meteorologists consider a specific verification metric more than others (e.g., some warning meteorologist considered POD more than FAR in Ellis et al. 2020), with the current measure, our study is unable to present how each metric may play a different role in warning decisions, which future research can investigate.

Third, we refer extensively to radar data contributions as objective technical input to the warning process, even though the objective scientific contributions come from both observed and model data. As such, the results may be limited.

Fourth, our sample size for study 1 is relatively small. Despite this limitation, our sample is unique because only a handful of prior studies have examined forecasters’ severe weather communication (e.g., Anthony et al. 2014; Compton 2018). In order not to miss meaningful findings due to the modest sample size, we provided 95% confidence intervals on each beta value to show the range of the estimates. Moreover, we were able to attempt replication of these findings using a larger dataset collected by TWIP.

A fifth limitation is that the study only examined southern and central regions of U.S. warning meteorologists’ tornado warning decisions, due to the nature of the funding for this project. Future research should test our model in different hazard contexts as well as outside of the central and southern United States. Also, this study measured factors that impact decisions to warn, but not whether these factors improve warning metrics, which future research can investigate. Finally, this study did not measure participants’ recent experiences with tornado warnings, which likely influences decisions to warn. Future research would be well served by doing so.

d. Conclusions

Advanced technology is powering more accurate extreme weather forecasts, but it is still difficult for warning meteorologists to make tornado warning decisions. Notably, this research documents the roles of meteorological evidence and human judgment on meteorologists’ warning decisions with survey data. Technological advances help meteorologists make better predictions, but we must remember that humans are the ones who make the crucial decision of whether to warn.

Acknowledgments.

This research was supported by the National Oceanic and Atmospheric Administration (NOAA) VORTEX-SE program through Award NA17OAR4590194. The views and conclusions contained in the article are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of NOAA. We thank our partners Kevin Laws and John DeBlock from Birmingham, Alabama; Krissy Hurley and Larry Vannozzi from Nashville, Tennessee; David Nadler, Steven Nelson, and Keith Stellman from Peachtree City, Georgia; John J. Brost and Gregory Patrick from NWS southern region headquarters; Randall Graham from NWS central region headquarters; and Regional SOO/DOH Program Manager Jeff Waldstreicher for the help of survey recruitment. We thank Matt Bunkers, Rod Donavon, Fred Glass, Aaron Johnson, Jason Schaumann, John Stoppkotte, and Ray Wolf from the Central Tornado Warning Improvement Project for providing the data for our replication analysis.

REFERENCES

  • Alexiev, A., H. Volberda, J. Jansen, and F. Van Den Bosch, 2019: Contextualizing senior executive advice seeking: The role of decision process comprehensiveness and empowerment climate. Organ. Stud., 41, 471497, https://doi.org/10.1177/0170840619830128.

    • Search Google Scholar
    • Export Citation
  • Alley, R. B., K. A. Emanuel, and F. Zhang, 2019: Advances in weather prediction. Science, 363, 342344, https://doi.org/10.1126/science.aav7274.

    • Search Google Scholar
    • Export Citation
  • Andra, D. L., Jr., E. M. Quoetone, and W. F. Bunting, 2002: Warning decision making: The relative roles of conceptual models, technology, strategy, and forecaster expertise on 3 May 1999. Wea. Forecasting, 17, 559566, https://doi.org/10.1175/1520-0434(2002)017<0559:WDMTRR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Anthony, K. E., K. R. Cowden-Hodgson, D. H. O’Hair, R. L. Heath, and G. M. Eosco, 2014: Complexities in communication and collaboration in the hurricane warning system. Commun. Stud., 65, 468483, https://doi.org/10.1080/10510974.2014.957785.

    • Search Google Scholar
    • Export Citation
  • Ash, K. D., R. L. Schumann, and G. C. Bowser, 2014: Tornado warning trade-offs: Evaluating choices for visually communicating risk. Wea. Climate Soc., 6, 104118, https://doi.org/10.1175/WCAS-D-13-00021.1.

    • Search Google Scholar
    • Export Citation
  • Atwell Seate, A., B. F. Liu, D. Hawblitzel, I. Iles, and E. Hervoic, 2019: To warn or not to warn? Understanding National Weather Service forecasters’ tornado warning philosophies. National Weather Association Meeting, Huntsville, AL, NWA, https://nwas.org/annual-meeting-events/annual-meeting/meeting-agenda/#n-weatheranalysisandforecasting.

    • Search Google Scholar
    • Export Citation
  • Bean, H., B. F. Liu, S. Madden, J. Sutton, M. Wood, and D. Mileti, 2016: Disaster warnings in your pocket: How audiences interpret mobile alerts for an unfamiliar hazard. J. Contingencies Crisis Manage., 24, 136147, https://doi.org/10.1111/1468-5973.12108.

    • Search Google Scholar
    • Export Citation
  • Bostrom, A., R. Morss, J. Lazo, J. Demuth, H. Lazrus, and R. Hudson, 2016: A mental models study of hurricane forecast and warning production, communication, and decision-making. Wea. Climate Soc., 8, 111129, https://doi.org/10.1175/WCAS-D-15-0033.1.

    • Search Google Scholar
    • Export Citation
  • Boustead, J. M., and B. E. Mayes, 2014: The role of the human in issuing severe weather warnings. 27th Conf. on Severe Local Storms, Madison, WI, Amer. Meteor. Soc., 4B.2, https://ams.confex.com/ams/27SLS/webprogram/Paper254547.html.

    • Search Google Scholar
    • Export Citation
  • Breznitz, S., 1984: Cry Wolf: The Psychology of False Alarms. Psychology Press, 280 pp.

  • Brooks, A. W., F. Gino, and M. E. Schweitzer, 2015: Smart people ask for (my) advice: Seeking advice boosts perceptions of competence. Manage. Sci., 61, 14211435, https://doi.org/10.1287/mnsc.2014.2054.

    • Search Google Scholar
    • Export Citation
  • Brooks, H. E., and C. A. Doswell III, 2002: Deaths in the 3 May 1999 Oklahoma City tornado from a historical perspective. Wea. Forecasting, 17, 354361, https://doi.org/10.1175/1520-0434(2002)017<0354:DITMOC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Brooks, H. E., and Coauthors, 2019: A century of progress in severe convective storm research and forecasting. A Century of Progress in Atmospheric and Related Sciences: Celebrating the American Meteorological Society Centennial, Meteor. Monogr., No. 59, Amer. Meteor. Soc., https://doi.org/10.1175/AMSMONOGRAPHS-D-18-0026.1.

    • Search Google Scholar
    • Export Citation
  • Brotzge, J., and W. Donner, 2013: The tornado warning process: A review of current research challenges, and opportunities. Bull. Amer. Meteor. Soc., 94, 17151733, https://doi.org/10.1175/BAMS-D-12-00147.1.

    • Search Google Scholar
    • Export Citation
  • Casteel, M. A., and J. R. Downing, 2016: Assessing risk following a wireless emergency alert: Are 90 characters enough? J. Homeland Secur. Emerg. Manage., 13, 95112, https://doi.org/10.1515/jhsem-2015-0024.

    • Search Google Scholar
    • Export Citation
  • Coleman, T. A., K. R. Knupp, J. Spann, J. B. Elliott, and B. E. Peters, 2011: The history (and future) of tornado warning dissemination in the United States. Bull. Amer. Meteor. Soc., 92, 567582, https://doi.org/10.1175/2010BAMS3062.1.

    • Search Google Scholar
    • Export Citation
  • Compton, J., 2018: When weather forecasters are wrong: Image repair and public rhetoric after severe weather. Sci. Commun., 40, 778788, https://doi.org/10.1177/1075547018799101.

    • Search Google Scholar
    • Export Citation
  • Daipha, P., 2015: Masters of Uncertainty: Weather Forecasters and the Quest for Ground Truth. University of Chicago Press, 280 pp.

  • Demuth, J. L., R. E. Morss, B. H. Murrow, and J. K. Lazo, 2012: Creation and communication of hurricane risk information. Bull. Amer. Meteor. Soc., 93, 11331145, https://doi.org/10.1175/BAMS-D-11-00150.1.

    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., III, 2004: Weather forecasting by humans—Heuristics and decision making. Wea. Forecasting, 19, 11151126, https://doi.org/10.1175/WAF-821.1.

    • Search Google Scholar
    • Export Citation
  • Durage, S. W., S. C. Wirasinghe, and J. Ruwanpura, 2013: Comparison of the Canadian and US tornado detection and warning systems. Nat. Hazards, 66, 117137, https://doi.org/10.1007/s11069-012-0168-7.

    • Search Google Scholar
    • Export Citation
  • Ellis, K. N., D. Burow, K. N. Gassert, L. R. Mason, and M. S. Porter, 2020: Forecaster perceptions and climatological analysis of the influence of convective mode on tornado climatology and warning success. Ann. Amer. Assoc. Geogr., 110, 10751094, https://doi.org/10.1080/24694452.2019.1670042.

    • Search Google Scholar
    • Export Citation
  • Fine, G. A., 2009: Authors of the Storm: Meteorologists and the Culture of Prediction. University of Chicago Press, 280 pp.

  • Funk, T., 2018: The effect of office culture and human bias in the tornado warning decision process in the NWS central region. 43 Annual Meeting of the National Weather Association Meeting, Saint Louis, MO, NWA, https://objects-us-east-1.dream.io/nwafiles/Annual%20Meetings/2018/Abstracts/The%20Effect%20of%20Office%20Culture%20and%20Human%20Bias%20in%20the%20Tornado%20Warning%20Decision%20Process%20in%20the%20NWS%20Central%20Region.htm.

    • Search Google Scholar
    • Export Citation
  • Gino, F., and D. A. Moore, 2007: Effects of task difficulty on use of advice. J. Behav. Decis. Making, 20, 2135, https://doi.org/10.1002/bdm.539.

    • Search Google Scholar
    • Export Citation
  • Golden, J. H., and C. R. Adams, 2000: The tornado problem: Forecast, warning, and response. Nat. Hazards Rev., 1, 107118, https://doi.org/10.1061/(ASCE)1527-6988(2000)1:2(107).

    • Search Google Scholar
    • Export Citation
  • Hawkins, M., V. Brown, and J. Ferrell, 2017: Assessment of NOAA National Weather Service methods to warn for extreme heat events. Wea. Climate Soc., 9, 513, https://doi.org/10.1175/WCAS-D-15-0037.1.

    • Search Google Scholar
    • Export Citation
  • Hayes, A. F., 2013: Introduction to Mediation, Moderation, and Conditional Process Analysis. The Guilford Press, 507 pp.

  • Heath, R. L., J. Lee, and L. L. Lemon, 2019: Narratives of risk communication: Nudging community residents to shelter-in-place. Public Relat. Rev., 45, 128137, https://doi.org/10.1016/j.pubrev.2018.12.004.

    • Search Google Scholar
    • Export Citation
  • Heinselman, P. L., D. S. Ladue, and H. Lazrus, 2012: Exploring impacts of rapid-scan radar data on NWS warning decisions. Wea. Forecasting, 27, 10311044, https://doi.org/10.1175/WAF-D-11-00145.1.

    • Search Google Scholar
    • Export Citation
  • Hemingway, R., and J. Robbins, 2020: Developing a hazard‐impact model to support impact‐based forecasts and warnings: The Vehicle OverTurning (VOT) model. Meteor. Appl., 27, e1819, https://doi.org/10.1002/met.1819.

    • Search Google Scholar
    • Export Citation
  • Hoffman, R. R., D. S. LaDue, M. Mogil, P. J. Roebber, and J. G. Trafton, 2017: Minding the Weather: How Expert Forecasters Think. MIT Press, 470 pp.

    • Search Google Scholar
    • Export Citation
  • Hurley, K., and D. Hawblitzel, 2021: Communicating low-probability/high-impact severe thunderstorm events: Moving forward from the March 2020 Tennessee tornado outbreak. Sixth Symp. on Societal Applications: Policy, Research and Practice, Online, Amer. Meteor. Soc., 10.5, https://ams.confex.com/ams/101ANNUAL/meetingapp.cgi/Paper/380000.

    • Search Google Scholar
    • Export Citation
  • Ivankova, N. V., 2015: Mixed Methods Applications in Action Research. Sage, 472 pp.

  • Jahn, J. L. S., and A. E. Black, 2017: A model of communicative hierarchical foundations of high reliability organizing in wildland firefighting teams. Manage. Commun. Quart., 31, 356379, https://doi.org/10.1177/0893318917691358.

    • Search Google Scholar
    • Export Citation
  • Karstens, C. D., and Coauthors, 2018: Development of a human–machine mix for forecasting severe convective events. Wea. Forecasting, 33, 715737, https://doi.org/10.1175/WAF-D-17-0188.1.

    • Search Google Scholar
    • Export Citation
  • Lazo, J. K., H. R. Hosterman, J. M. Sprague-Hilderbrand, and J. E. Adkins, 2020: Impact-based decision support services and the socioeconomic impacts of winter storms. Bull. Amer. Meteor. Soc., 101, E626E639, https://doi.org/10.1175/BAMS-D-18-0153.1.

    • Search Google Scholar
    • Export Citation
  • Lim, J. R., B. F. Liu, and M. Egnoto, 2019: Cry wolf effect? Evaluating the impact of false alarms on public responses to tornado alerts in the southeastern United States. Wea. Climate Soc., 11, 549563, https://doi.org/10.1175/WCAS-D-18-0080.1.

    • Search Google Scholar
    • Export Citation
  • Little, T. D., 2013: Statistical Analysis. Vol. 2, The Oxford Handbook of Quantitative Methods in Psychology, Oxford University Press, 784 pp.

    • Search Google Scholar
    • Export Citation
  • Liu, B. F., and A. Atwell Seate, 2021: The evolving weather service: Forecasters’ perceptions of their relationships with core partners. Wea. Climate Soc., 13, 437448, https://doi.org/10.1175/WCAS-D-20-0097.1.

    • Search Google Scholar
    • Export Citation
  • Liu, B. F., A. Atwell Seate, I. Iles, and E. Herovic, 2020: Eyes of the storm: How citizen scientists contribute to government forecasting and risk communication. Wea. Climate Soc., 12, 263277, https://doi.org/10.1175/WCAS-D-19-0131.1.

    • Search Google Scholar
    • Export Citation
  • Mehrkanoon, S., 2019: Deep shared representation learning for weather elements forecasting. Knowl.-Based Syst., 179, 120128, https://doi.org/10.1016/j.knosys.2019.05.009.

    • Search Google Scholar
    • Export Citation
  • Morss, R., J. Demuth, A. Bostrom, J. Lazo, and H. Lazrus, 2015: Flash flood risks and warning decisions: A mental models study of forecasters, public officials, and media broadcasters in Boulder, Colorado. Risk Anal., 35, 20092028, https://doi.org/10.1111/risa.12403.

    • Search Google Scholar
    • Export Citation
  • Nagele, D., 2015: Evolution of culture among warning system organizations. Ph.D. dissertation, University of Delaware, 262 pp., http://udspace.udel.edu/handle/19716/17180.

    • Search Google Scholar
    • Export Citation
  • National Severe Storms Laboratory, 2022: Research tools: Dual polarized radar. NOAA, https://www.nssl.noaa.gov/tools/radar/dualpol/.

  • Nembhard, I. M., and A. C. Edmondson, 2006: Making it safe: The effects of leader inclusiveness and professional status on psychological safety and improvement efforts in health care teams. J. Organ. Behav., 27, 941966, https://doi.org/10.1002/job.413.

    • Search Google Scholar
    • Export Citation
  • NWS, 2016: Forecasting severe weather: A day in the life of the severe weather forecaster. NOAA, https://www.weather.gov/wrn/spring2016-article-severe-weather.

    • Search Google Scholar
    • Export Citation
  • Oskamp, S., 1965: Overconfidence in case-study judgments. J. Consult. Psychol., 29, 261265, https://doi.org/10.1037/h0022125.

  • Palm, R., T. Bolsen, and J. T. Kingsland, 2020: “Don’t tell me what to do”: Resistance to climate change messages suggesting behavior changes. Wea. Climate Soc., 12, 827835, https://doi.org/10.1175/WCAS-D-19-0141.1.

    • Search Google Scholar
    • Export Citation
  • Phillips, B. K., V. R. Prybutok, and D. R. Peak, 2014: Decision confidence, information usefulness, and in-formation seeking intention in the presence of disconfirming information. Informing Sci., 17, 124, https://doi.org/10.28945/1932.

    • Search Google Scholar
    • Export Citation
  • Pliske, R., B. Candall, and G. Klein, 2004: Competence in weather forecasting. Psychological Investigations of Competence in Decision Making, K. Smith, J. Shanteau, and P. Johnson, Eds., Cambridge University Press, 4068.

    • Search Google Scholar
    • Export Citation
  • Ripberger, J. T., C. L. Silva, H. C. Jenkins‐Smith, D. E. Carlson, M. James, and K. G. Herron, 2015: False alarms and missed events: The impact and origins of perceived inaccuracy in tornado warning systems. Risk Anal., 35, 4456, https://doi.org/10.1111/risa.12262.

    • Search Google Scholar
    • Export Citation
  • Roeder, A. C., R. S. Bisel, and W. T. Howe, 2021: High-reliability organizing and communication during naturalistic decision making: U.S. National Weather Service (NWS) forecasting teams’ use of ‘floating’. J. Appl. Commun. Res., 49, 441459, https://doi.org/10.1080/00909882.2021.1907855.

    • Search Google Scholar
    • Export Citation
  • Scher, S., 2018: Toward data‐driven weather and climate forecasting: Approximating a simple general circulation model with deep learning. Geophys. Res. Lett., 45, 12 61612 622, https://doi.org/10.1029/2018GL080704.

    • Search Google Scholar
    • Export Citation
  • See, K. E., E. W. Morrison, N. B. Rothman, and J. B. Soll, 2011: The detrimental effects of power on confidence, advice taking, and accuracy. Organ. Behav. Hum. Decis. Process., 116, 272285, https://doi.org/10.1016/j.obhdp.2011.07.006.

    • Search Google Scholar
    • Export Citation
  • Simmons, K. M., and D. Sutter, 2009: False alarms, tornado warnings, and tornado casualties. Wea. Climate Soc., 1, 3853, https://doi.org/10.1175/2009WCAS1005.1.

    • Search Google Scholar
    • Export Citation
  • Soll, J. B., and R. P. Larrick, 2009: Strategies for revising judgment: How (and how well) people use other’s opinions. J. Exp. Psychol. Learn. Mem. Cogn., 35, 780805, https://doi.org/10.1037/a0015145.

    • Search Google Scholar
    • Export Citation
  • Stewart, T. R., K. F. Heideman, W. R. Moninger, and P. Reagan-Cirincione, 1992: Effects of improved information on the components of skill in weather forecasting. Organ. Behav. Hum. Decis. Process., 53, 107134, https://doi.org/10.1016/0749-5978(92)90058-F.

    • Search Google Scholar
    • Export Citation
  • Sutton, J., C. League, T. L. Sellnow, and D. Sellnow, 2015: Terse messaging and public health in the midst of natural disasters: The case of the Boulder floods. Health Commun., 30, 135143, https://doi.org/10.1080/10410236.2014.974124.

    • Search Google Scholar
    • Export Citation
  • Trainor, J., D. Nagele, B. Philips, and B. Scott, 2015: Tornadoes, social science, and the false alarm effect. Wea. Climate Soc., 7, 333352, https://doi.org/10.1175/WCAS-D-14-00052.1.

    • Search Google Scholar
    • Export Citation
  • Walters, J. E., L. R. Mason, K. Ellis, and B. Winchester, 2020: Staying safe in a tornado: A qualitative inquiry into public knowledge, access, and response to tornado warnings. Wea. Forecasting, 35, 6781, https://doi.org/10.1175/WAF-D-19-0090.1.

    • Search Google Scholar
    • Export Citation
  • Webber, K. L., and H. Zheng, 2020: Data analytics and the imperatives for data-informed decision-making in higher education. Big Data on Campus: Data Analytics and Decision Making in Higher Education, The Johns Hopkins University Press, 329.

    • Search Google Scholar
    • Export Citation
  • Wilson, K. A., P. L. Heinselman, P. S. Skinner, J. J. Choate, and K. E. Klockow-McClain, 2019: Meteorologists’ interpretations of storm-scale ensemble-based forecast guidance. Wea. Climate Soc., 11, 337354, https://doi.org/10.1175/WCAS-D-18-0084.1.

    • Search Google Scholar
    • Export Citation
  • Woltman, H., A. Feldstain, J. C. MacKay, and M. Rocchi, 2012: An introduction to hierarchical linear modeling. Tutorials Quant. Methods Psychol., 8, 5269, https://doi.org/10.20982/tqmp.08.1.p052.

    • Search Google Scholar
    • Export Citation
  • Yaniv, I., and E. Kleinberger, 2000: Advice taking in decision making: Egocentric discounting and reputation formation. Organ. Behav. Hum. Decis. Process., 83, 260281, https://doi.org/10.1006/obhd.2000.2909.

    • Search Google Scholar
    • Export Citation
  • Yu, Y., and Y. Li, 2017: An improved clutter suppression method for weather radars using multiple pulse repetition time technique. Adv. Meteor., 2017, 8173643, https://doi.org/10.1155/2017/8173643.

    • Search Google Scholar
    • Export Citation
Save
  • Alexiev, A., H. Volberda, J. Jansen, and F. Van Den Bosch, 2019: Contextualizing senior executive advice seeking: The role of decision process comprehensiveness and empowerment climate. Organ. Stud., 41, 471497, https://doi.org/10.1177/0170840619830128.

    • Search Google Scholar
    • Export Citation
  • Alley, R. B., K. A. Emanuel, and F. Zhang, 2019: Advances in weather prediction. Science, 363, 342344, https://doi.org/10.1126/science.aav7274.

    • Search Google Scholar
    • Export Citation
  • Andra, D. L., Jr., E. M. Quoetone, and W. F. Bunting, 2002: Warning decision making: The relative roles of conceptual models, technology, strategy, and forecaster expertise on 3 May 1999. Wea. Forecasting, 17, 559566, https://doi.org/10.1175/1520-0434(2002)017<0559:WDMTRR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Anthony, K. E., K. R. Cowden-Hodgson, D. H. O’Hair, R. L. Heath, and G. M. Eosco, 2014: Complexities in communication and collaboration in the hurricane warning system. Commun. Stud., 65, 468483, https://doi.org/10.1080/10510974.2014.957785.

    • Search Google Scholar
    • Export Citation
  • Ash, K. D., R. L. Schumann, and G. C. Bowser, 2014: Tornado warning trade-offs: Evaluating choices for visually communicating risk. Wea. Climate Soc., 6, 104118, https://doi.org/10.1175/WCAS-D-13-00021.1.

    • Search Google Scholar
    • Export Citation
  • Atwell Seate, A., B. F. Liu, D. Hawblitzel, I. Iles, and E. Hervoic, 2019: To warn or not to warn? Understanding National Weather Service forecasters’ tornado warning philosophies. National Weather Association Meeting, Huntsville, AL, NWA, https://nwas.org/annual-meeting-events/annual-meeting/meeting-agenda/#n-weatheranalysisandforecasting.

    • Search Google Scholar
    • Export Citation
  • Bean, H., B. F. Liu, S. Madden, J. Sutton, M. Wood, and D. Mileti, 2016: Disaster warnings in your pocket: How audiences interpret mobile alerts for an unfamiliar hazard. J. Contingencies Crisis Manage., 24, 136147, https://doi.org/10.1111/1468-5973.12108.

    • Search Google Scholar
    • Export Citation
  • Bostrom, A., R. Morss, J. Lazo, J. Demuth, H. Lazrus, and R. Hudson, 2016: A mental models study of hurricane forecast and warning production, communication, and decision-making. Wea. Climate Soc., 8, 111129, https://doi.org/10.1175/WCAS-D-15-0033.1.

    • Search Google Scholar
    • Export Citation
  • Boustead, J. M., and B. E. Mayes, 2014: The role of the human in issuing severe weather warnings. 27th Conf. on Severe Local Storms, Madison, WI, Amer. Meteor. Soc., 4B.2, https://ams.confex.com/ams/27SLS/webprogram/Paper254547.html.

    • Search Google Scholar
    • Export Citation
  • Breznitz, S., 1984: Cry Wolf: The Psychology of False Alarms. Psychology Press, 280 pp.

  • Brooks, A. W., F. Gino, and M. E. Schweitzer, 2015: Smart people ask for (my) advice: Seeking advice boosts perceptions of competence. Manage. Sci., 61, 14211435, https://doi.org/10.1287/mnsc.2014.2054.

    • Search Google Scholar
    • Export Citation
  • Brooks, H. E., and C. A. Doswell III, 2002: Deaths in the 3 May 1999 Oklahoma City tornado from a historical perspective. Wea. Forecasting, 17, 354361, https://doi.org/10.1175/1520-0434(2002)017<0354:DITMOC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Brooks, H. E., and Coauthors, 2019: A century of progress in severe convective storm research and forecasting. A Century of Progress in Atmospheric and Related Sciences: Celebrating the American Meteorological Society Centennial, Meteor. Monogr., No. 59, Amer. Meteor. Soc., https://doi.org/10.1175/AMSMONOGRAPHS-D-18-0026.1.

    • Search Google Scholar
    • Export Citation
  • Brotzge, J., and W. Donner, 2013: The tornado warning process: A review of current research challenges, and opportunities. Bull. Amer. Meteor. Soc., 94, 17151733, https://doi.org/10.1175/BAMS-D-12-00147.1.

    • Search Google Scholar
    • Export Citation
  • Casteel, M. A., and J. R. Downing, 2016: Assessing risk following a wireless emergency alert: Are 90 characters enough? J. Homeland Secur. Emerg. Manage., 13, 95112, https://doi.org/10.1515/jhsem-2015-0024.

    • Search Google Scholar
    • Export Citation
  • Coleman, T. A., K. R. Knupp, J. Spann, J. B. Elliott, and B. E. Peters, 2011: The history (and future) of tornado warning dissemination in the United States. Bull. Amer. Meteor. Soc., 92, 567582, https://doi.org/10.1175/2010BAMS3062.1.

    • Search Google Scholar
    • Export Citation
  • Compton, J., 2018: When weather forecasters are wrong: Image repair and public rhetoric after severe weather. Sci. Commun., 40, 778788, https://doi.org/10.1177/1075547018799101.

    • Search Google Scholar
    • Export Citation
  • Daipha, P., 2015: Masters of Uncertainty: Weather Forecasters and the Quest for Ground Truth. University of Chicago Press, 280 pp.

  • Demuth, J. L., R. E. Morss, B. H. Murrow, and J. K. Lazo, 2012: Creation and communication of hurricane risk information. Bull. Amer. Meteor. Soc., 93, 11331145, https://doi.org/10.1175/BAMS-D-11-00150.1.

    • Search Google Scholar
    • Export Citation
  • Doswell, C. A., III, 2004: Weather forecasting by humans—Heuristics and decision making. Wea. Forecasting, 19, 11151126, https://doi.org/10.1175/WAF-821.1.

    • Search Google Scholar
    • Export Citation
  • Durage, S. W., S. C. Wirasinghe, and J. Ruwanpura, 2013: Comparison of the Canadian and US tornado detection and warning systems. Nat. Hazards, 66, 117137, https://doi.org/10.1007/s11069-012-0168-7.

    • Search Google Scholar
    • Export Citation
  • Ellis, K. N., D. Burow, K. N. Gassert, L. R. Mason, and M. S. Porter, 2020: Forecaster perceptions and climatological analysis of the influence of convective mode on tornado climatology and warning success. Ann. Amer. Assoc. Geogr., 110, 10751094, https://doi.org/10.1080/24694452.2019.1670042.

    • Search Google Scholar
    • Export Citation
  • Fine, G. A., 2009: Authors of the Storm: Meteorologists and the Culture of Prediction. University of Chicago Press, 280 pp.

  • Funk, T., 2018: The effect of office culture and human bias in the tornado warning decision process in the NWS central region. 43 Annual Meeting of the National Weather Association Meeting, Saint Louis, MO, NWA, https://objects-us-east-1.dream.io/nwafiles/Annual%20Meetings/2018/Abstracts/The%20Effect%20of%20Office%20Culture%20and%20Human%20Bias%20in%20the%20Tornado%20Warning%20Decision%20Process%20in%20the%20NWS%20Central%20Region.htm.

    • Search Google Scholar
    • Export Citation
  • Gino, F., and D. A. Moore, 2007: Effects of task difficulty on use of advice. J. Behav. Decis. Making, 20, 2135, https://doi.org/10.1002/bdm.539.

    • Search Google Scholar
    • Export Citation
  • Golden, J. H., and C. R. Adams, 2000: The tornado problem: Forecast, warning, and response. Nat. Hazards Rev., 1, 107118, https://doi.org/10.1061/(ASCE)1527-6988(2000)1:2(107).

    • Search Google Scholar
    • Export Citation
  • Hawkins, M., V. Brown, and J. Ferrell, 2017: Assessment of NOAA National Weather Service methods to warn for extreme heat events. Wea. Climate Soc., 9, 513, https://doi.org/10.1175/WCAS-D-15-0037.1.

    • Search Google Scholar
    • Export Citation
  • Hayes, A. F., 2013: Introduction to Mediation, Moderation, and Conditional Process Analysis. The Guilford Press, 507 pp.

  • Heath, R. L., J. Lee, and L. L. Lemon, 2019: Narratives of risk communication: Nudging community residents to shelter-in-place. Public Relat. Rev., 45, 128137, https://doi.org/10.1016/j.pubrev.2018.12.004.

    • Search Google Scholar
    • Export Citation
  • Heinselman, P. L., D. S. Ladue, and H. Lazrus, 2012: Exploring impacts of rapid-scan radar data on NWS warning decisions. Wea. Forecasting, 27, 10311044, https://doi.org/10.1175/WAF-D-11-00145.1.

    • Search Google Scholar
    • Export Citation
  • Hemingway, R., and J. Robbins, 2020: Developing a hazard‐impact model to support impact‐based forecasts and warnings: The Vehicle OverTurning (VOT) model. Meteor. Appl., 27, e1819, https://doi.org/10.1002/met.1819.

    • Search Google Scholar
    • Export Citation
  • Hoffman, R. R., D. S. LaDue, M. Mogil, P. J. Roebber, and J. G. Trafton, 2017: Minding the Weather: How Expert Forecasters Think. MIT Press, 470 pp.

    • Search Google Scholar
    • Export Citation
  • Hurley, K., and D. Hawblitzel, 2021: Communicating low-probability/high-impact severe thunderstorm events: Moving forward from the March 2020 Tennessee tornado outbreak. Sixth Symp. on Societal Applications: Policy, Research and Practice, Online, Amer. Meteor. Soc., 10.5, https://ams.confex.com/ams/101ANNUAL/meetingapp.cgi/Paper/380000.

    • Search Google Scholar
    • Export Citation
  • Ivankova, N. V., 2015: Mixed Methods Applications in Action Research. Sage, 472 pp.

  • Jahn, J. L. S., and A. E. Black, 2017: A model of communicative hierarchical foundations of high reliability organizing in wildland firefighting teams. Manage. Commun. Quart., 31, 356379, https://doi.org/10.1177/0893318917691358.

    • Search Google Scholar
    • Export Citation
  • Karstens, C. D., and Coauthors, 2018: Development of a human–machine mix for forecasting severe convective events. Wea. Forecasting, 33, 715737, https://doi.org/10.1175/WAF-D-17-0188.1.

    • Search Google Scholar
    • Export Citation
  • Lazo, J. K., H. R. Hosterman, J. M. Sprague-Hilderbrand, and J. E. Adkins, 2020: Impact-based decision support services and the socioeconomic impacts of winter storms. Bull. Amer. Meteor. Soc., 101, E626E639, https://doi.org/10.1175/BAMS-D-18-0153.1.

    • Search Google Scholar
    • Export Citation
  • Lim, J. R., B. F. Liu, and M. Egnoto, 2019: Cry wolf effect? Evaluating the impact of false alarms on public responses to tornado alerts in the southeastern United States. Wea. Climate Soc., 11, 549563, https://doi.org/10.1175/WCAS-D-18-0080.1.

    • Search Google Scholar
    • Export Citation
  • Little, T. D., 2013: Statistical Analysis. Vol. 2, The Oxford Handbook of Quantitative Methods in Psychology, Oxford University Press, 784 pp.

    • Search Google Scholar
    • Export Citation
  • Liu, B. F., and A. Atwell Seate, 2021: The evolving weather service: Forecasters’ perceptions of their relationships with core partners. Wea. Climate Soc., 13, 437448, https://doi.org/10.1175/WCAS-D-20-0097.1.

    • Search Google Scholar
    • Export Citation
  • Liu, B. F., A. Atwell Seate, I. Iles, and E. Herovic, 2020: Eyes of the storm: How citizen scientists contribute to government forecasting and risk communication. Wea. Climate Soc., 12, 263277, https://doi.org/10.1175/WCAS-D-19-0131.1.

    • Search Google Scholar
    • Export Citation
  • Mehrkanoon, S., 2019: Deep shared representation learning for weather elements forecasting. Knowl.-Based Syst., 179, 120128, https://doi.org/10.1016/j.knosys.2019.05.009.

    • Search Google Scholar
    • Export Citation
  • Morss, R., J. Demuth, A. Bostrom, J. Lazo, and H. Lazrus, 2015: Flash flood risks and warning decisions: A mental models study of forecasters, public officials, and media broadcasters in Boulder, Colorado. Risk Anal., 35, 20092028, https://doi.org/10.1111/risa.12403.

    • Search Google Scholar
    • Export Citation
  • Nagele, D., 2015: Evolution of culture among warning system organizations. Ph.D. dissertation, University of Delaware, 262 pp., http://udspace.udel.edu/handle/19716/17180.

    • Search Google Scholar
    • Export Citation
  • National Severe Storms Laboratory, 2022: Research tools: Dual polarized radar. NOAA, https://www.nssl.noaa.gov/tools/radar/dualpol/.

  • Nembhard, I. M., and A. C. Edmondson, 2006: Making it safe: The effects of leader inclusiveness and professional status on psychological safety and improvement efforts in health care teams. J. Organ. Behav., 27, 941966, https://doi.org/10.1002/job.413.

    • Search Google Scholar
    • Export Citation
  • NWS, 2016: Forecasting severe weather: A day in the life of the severe weather forecaster. NOAA, https://www.weather.gov/wrn/spring2016-article-severe-weather.

    • Search Google Scholar
    • Export Citation
  • Oskamp, S., 1965: Overconfidence in case-study judgments. J. Consult. Psychol., 29, 261265, https://doi.org/10.1037/h0022125.

  • Palm, R., T. Bolsen, and J. T. Kingsland, 2020: “Don’t tell me what to do”: Resistance to climate change messages suggesting behavior changes. Wea. Climate Soc., 12, 827835, https://doi.org/10.1175/WCAS-D-19-0141.1.

    • Search Google Scholar
    • Export Citation
  • Phillips, B. K., V. R. Prybutok, and D. R. Peak, 2014: Decision confidence, information usefulness, and in-formation seeking intention in the presence of disconfirming information. Informing Sci., 17, 124, https://doi.org/10.28945/1932.

    • Search Google Scholar
    • Export Citation
  • Pliske, R., B. Candall, and G. Klein, 2004: Competence in weather forecasting. Psychological Investigations of Competence in Decision Making, K. Smith, J. Shanteau, and P. Johnson, Eds., Cambridge University Press, 4068.

    • Search Google Scholar
    • Export Citation
  • Ripberger, J. T., C. L. Silva, H. C. Jenkins‐Smith, D. E. Carlson, M. James, and K. G. Herron, 2015: False alarms and missed events: The impact and origins of perceived inaccuracy in tornado warning systems. Risk Anal., 35, 4456, https://doi.org/10.1111/risa.12262.

    • Search Google Scholar
    • Export Citation
  • Roeder, A. C., R. S. Bisel, and W. T. Howe, 2021: High-reliability organizing and communication during naturalistic decision making: U.S. National Weather Service (NWS) forecasting teams’ use of ‘floating’. J. Appl. Commun. Res., 49, 441459, https://doi.org/10.1080/00909882.2021.1907855.

    • Search Google Scholar
    • Export Citation
  • Scher, S., 2018: Toward data‐driven weather and climate forecasting: Approximating a simple general circulation model with deep learning. Geophys. Res. Lett., 45, 12 61612 622, https://doi.org/10.1029/2018GL080704.

    • Search Google Scholar
    • Export Citation
  • See, K. E., E. W. Morrison, N. B. Rothman, and J. B. Soll, 2011: The detrimental effects of power on confidence, advice taking, and accuracy. Organ. Behav. Hum. Decis. Process., 116, 272285, https://doi.org/10.1016/j.obhdp.2011.07.006.

    • Search Google Scholar
    • Export Citation
  • Simmons, K. M., and D. Sutter, 2009: False alarms, tornado warnings, and tornado casualties. Wea. Climate Soc., 1, 3853, https://doi.org/10.1175/2009WCAS1005.1.

    • Search Google Scholar
    • Export Citation
  • Soll, J. B., and R. P. Larrick, 2009: Strategies for revising judgment: How (and how well) people use other’s opinions. J. Exp. Psychol. Learn. Mem. Cogn., 35, 780805, https://doi.org/10.1037/a0015145.

    • Search Google Scholar
    • Export Citation
  • Stewart, T. R., K. F. Heideman, W. R. Moninger, and P. Reagan-Cirincione, 1992: Effects of improved information on the components of skill in weather forecasting. Organ. Behav. Hum. Decis. Process., 53, 107134, https://doi.org/10.1016/0749-5978(92)90058-F.

    • Search Google Scholar
    • Export Citation
  • Sutton, J., C. League, T. L. Sellnow, and D. Sellnow, 2015: Terse messaging and public health in the midst of natural disasters: The case of the Boulder floods. Health Commun., 30, 135143, https://doi.org/10.1080/10410236.2014.974124.

    • Search Google Scholar
    • Export Citation
  • Trainor, J., D. Nagele, B. Philips, and B. Scott, 2015: Tornadoes, social science, and the false alarm effect. Wea. Climate Soc., 7, 333352, https://doi.org/10.1175/WCAS-D-14-00052.1.

    • Search Google Scholar
    • Export Citation
  • Walters, J. E., L. R. Mason, K. Ellis, and B. Winchester, 2020: Staying safe in a tornado: A qualitative inquiry into public knowledge, access, and response to tornado warnings. Wea. Forecasting, 35, 6781, https://doi.org/10.1175/WAF-D-19-0090.1.

    • Search Google Scholar
    • Export Citation
  • Webber, K. L., and H. Zheng, 2020: Data analytics and the imperatives for data-informed decision-making in higher education. Big Data on Campus: Data Analytics and Decision Making in Higher Education, The Johns Hopkins University Press, 329.

    • Search Google Scholar
    • Export Citation
  • Wilson, K. A., P. L. Heinselman, P. S. Skinner, J. J. Choate, and K. E. Klockow-McClain, 2019: Meteorologists’ interpretations of storm-scale ensemble-based forecast guidance. Wea. Climate Soc., 11, 337354, https://doi.org/10.1175/WCAS-D-18-0084.1.

    • Search Google Scholar
    • Export Citation
  • Woltman, H., A. Feldstain, J. C. MacKay, and M. Rocchi, 2012: An introduction to hierarchical linear modeling. Tutorials Quant. Methods Psychol., 8, 5269, https://doi.org/10.20982/tqmp.08.1.p052.

    • Search Google Scholar
    • Export Citation
  • Yaniv, I., and E. Kleinberger, 2000: Advice taking in decision making: Egocentric discounting and reputation formation. Organ. Behav. Hum. Decis. Process., 83, 260281, https://doi.org/10.1006/obhd.2000.2909.

    • Search Google Scholar
    • Export Citation
  • Yu, Y., and Y. Li, 2017: An improved clutter suppression method for weather radars using multiple pulse repetition time technique. Adv. Meteor., 2017, 8173643, https://doi.org/10.1155/2017/8173643.

    • Search Google Scholar
    • Export Citation
  • Fig. 1.

    Map of regions that are included in the study.

  • Fig. 2.

    Interaction effect between storm-reports dependency and verification-metrics consideration on warning decision.

All Time Past Year Past 30 Days
Abstract Views 596 88 0
Full Text Views 340 212 24
PDF Downloads 320 176 27