1. Introduction
Flooding is one of the most hazardous and costly weather phenomena in the United States. Since 2015, the south-central region of the United States has experienced eight $1 billion disasters associated with extreme rainfall (NOAA NCEI 2021) and more than 250 people have lost their lives to flooding (NWS 2021a). Extreme rainfall events are projected to increase in frequency and intensity with climate change (e.g., Mullens et al. 2013; Hayhoe et al. 2018; Trenberth et al. 2003). Increases have already been identified in many parts of the country, including in the frequency of rainfall events at or above the 95th percentile (Mallakpour and Villarini 2017) and in the magnitude of 90th-percentile hourly events (Brown et al. 2020).
As extreme rainfall events are projected to increase in frequency and intensity (Easterling et al. 2017) so will the importance of effective communication of these events by meteorologists. Effective communication of forecast information to the public and partners is key to the National Weather Service (NWS) mission of protecting life and property (NWS 2019). However, how NWS forecasters process and then communicate weather events, including extreme rainfall events, has not been well studied (Morss et al. 2015). Because extreme rainfall events are, by definition, rare, they are especially difficult to communicate due to forecasters’ lack of experience with them. This study aims to add to the knowledge base of how NWS forecasters in the south-central United States translate technical meteorological material into usable information for their audiences. Specifically, this study examines the following three research questions (RQs) within the context of extreme rainfall events: 1) How do forecasters internally process model outliers?; 2) How do forecasters externally communicate model outliers and outlier events?; and 3) Do forecasters consider climate change when forecasting extreme rainfall events, and if so, does it impact how they process and communicate these events?
Thematic analysis identified elements of sensemaking (Weick et al. 2005; Doswell 2004), judgment and decision-making (Millet et al. 2020), and principles of forecasting (Armstrong 2001a) involved in the forecast process. Altogether, this forms what we propose to be the extreme event forecast communication process model and conceptually answers RQ1 and RQ2. We also present practical findings regarding RQ1 and RQ2, as well as RQ3. The resulting model and the practical findings present a high-level conceptualization of the forecast and communication processes for extreme rainfall events in the south-central United States.
2. Literature review
Meteorologists have a plethora of information available to them when making forecasts. They cannot analyze all of it within their time constraints so they must choose what they think is the most reliable (Daipha 2015; Doswell 2004). Weather models, including ensemble guidance, are one source that adds value to extreme precipitation forecasts (Schumacher 2017). However, interpretation can be subjective (Evans et al. 2014) and forecasters sometimes struggle to interpret probabilistic guidance from ensembles (Wilson et al. 2019). Once forecasters have interpreted the appropriate information and used it to create the forecast, they must then communicate that forecast to the public and their partners. Over the past decade or so, there has been increasing interest and progress in understanding how best to communicate meteorological information to lay audiences. Effective forecast communication has also become an increasingly important part of NWS operations.
a. Forecast communication
Weather Forecast Offices (WFOs) must work within themselves (Daipha 2015) and with neighboring offices, national centers, and partners to ensure clear, consistent messaging (Childs and Schumacher 2018; Sherman-Morris et al. 2018). Research on how end users interpret and use forecast information continues to grow (e.g., Gigerenzer et al. 2005; Joslyn and Savelli 2010; Burgeno and Joslyn 2020; Morss and Hayden 2010; Ripberger et al. 2022; Perreault et al. 2014). However, less is known about how forecasters use their meteorological knowledge to translate complex atmospheric data into understandable, actionable information to end users (Morss et al. 2015).
It is important that audiences can understand and use weather forecasts. Using language that is too technical or subjective can make the product difficult to correctly interpret (Sivle and Aamodt 2019). Carr et al. (2016) found that their participants thought many NWS flood products were difficult to understand or visually unappealing. Bostrom et al. (2016) found that several NWS products are not used at all by partners. The NWS is increasing its use of social media as a communication method with public audiences and partners (Hubbard 2018). Social media posts should include actionable information that can help the end user protect themselves rather than just informing them that a threat is coming (Eachus and Keim 2019). NWS offices struggle, however, to include actionable information while the threat is ongoing (Olson et al. 2019).
WFOs think they work well with partners, especially emergency managers (EMs), to understand their needs (Sherman-Morris et al. 2018). These interactions educate forecasters on the real-world value of their forecasts (Hoffman et al. 2017), thus improving forecast communication systems. When WFOs collaborate with end users to better understand their needs, it builds trust within that relationship (Kuonen et al. 2019). To be successful, these interactions should be consistently maintained (Demuth et al. 2012; Liu and Seate 2021) as it is insufficient to only interact when an event is imminent (Senkbeil et al. 2020).
b. Conceptual frameworks applied to meteorological contexts
Although research into forecast communication continues to grow, there remains a lack of understanding of how best to communicate extreme rainfall events. Social science disciplines have developed conceptual frameworks and mental models that can help explain how people process and communicate information. This literature can be drawn upon to investigate meteorological contexts. Meteorologists have used conceptual models to visualize their knowledge for decades (Hoffman et al. 2017). Bostrom et al. (2016) and Morss et al. (2015) both used mental models to understand how forecasters and stakeholders processed hurricane and flash flood risks, respectively. Lejano et al. (2016) established a model that focuses on how organizational processes impact the risk communication process (see Fig. 3 in Lejano et al. 2016). In one step past the Lejano et al. (2016) model, Hoffman et al. (2017) presented the base model of expertise (see Fig. 10.1 in Hoffman et al. 2017). This model details the intraorganizational processes that result in a certain action—in the case of NWS operations, the creation of forecast products. The analysis of the research data, which will be described in methodology, revealed that elements of some of the abovementioned models and frameworks help explain how forecasters internally process and externally communicate extreme rainfall events. The frameworks include sensemaking, principles of forecasting, and judgment and decision-making.
1) Sensemaking
Sensemaking is described as the critical processes people use to understand unfamiliar situations so they can take the appropriate actions in those circumstances (Weick et al. 2005; Stigliani and Ravasi 2012; Tisch and Galbreath 2018). Butterworth (2010) applied the eight properties of sensemaking (Weick et al. 2005) to broadcast meteorology. This work built upon that and applied the properties to the context of NWS forecasters. These properties are organizing flux, noticing and bracketing, labeling, retrospect, presumption, social and systemic, action, and organizing through communication. Doswell (2004) also introduced representativeness, a concept referred to as pattern recognition in the forecasting community, as a sensemaking property. These sensemaking processes are described in Table 1.
Descriptions of the properties of sensemaking that were identified in this study. Examples of direct quotations from the interviewees accompany these descriptions and are further explained in the results section.
2) Principles of forecasting
The book Principles of Forecasting (Armstrong 2001b) compiled a series of papers that discussed 139 principles of forecasting that can be applied across various fields of study, such as economics, finance, psychology, and meteorology. The principles that can be applied to this study have been sorted into three sections: forecast purpose, forecast presentation, and forecast uncertainty. They are defined in Table 2.
Descriptions of the principles of forecasting that were identified in this study. Examples of direct quotations from the interviewees accompany these descriptions and are further referenced in the results section.
3) Judgment and decision-making
Studies on judgment and decision-making (JDM) identify heuristics and biases that influence the choices people make when facing uncertainty (Millet et al. 2020). Heuristics are described as “mental shortcuts” people take when making decisions (Milch et al. 2018) that could lead to biases that impact decisions (Millet et al. 2020). The Doswell (2004) study was one of the first to apply heuristics to weather forecasting and emphasized the need for further work on the subject. Millet et al. (2020) suggested using JDM principles to adjust how forecast information is presented, which would, in turn, improve user decision-making. The paper provides an extensive table of the various heuristics and biases they identified in their literature review (see Table 1 in Millet et al. 2020). These heuristics—affect, anchoring and adjustment, availability bias, confirmation bias, finite pool of worry, gambler’s fallacy, loss aversion, and temporal/spatial myopia (Table 3)—were found to be applicable to the forecasting and communication processes of extreme rainfall events, so this study builds on their application within meteorology.
Descriptions of the properties of judgment and decision-making that were identified in this study. Examples of direct quotations from the interviewees accompany these descriptions and are further referenced in the results section.
3. Methods
a. Event selection and participant recruitment
This research focused on extreme rainfall events in Oklahoma, Texas, Arkansas, and Louisiana that the NWS Hydrometeorological Design Studies Center (HDSC) found to have an annual exceedance probability (AEP) of less than 1/500 (HDSC 2021). Figure 1 provides an example of such an analysis. Nine such events occurred in the south-central region between 2015 and 2019. Table 4 summarizes the meteorological details of these events.
Annual exceedance probability (AEP) analysis by the NWS’s Hydrometeorological Design Studies Center for Hurricane Harvey (source: HDSC 2017).
Citation: Weather, Climate, and Society 15, 3; 10.1175/WCAS-D-22-0141.1
Events included in this study. Events in this study occurred between 2015 and 2019 in the south-central United States and were designated by the NWS HDSC to have an AEP of less than 1/500 (HDSC 2021). Maximum precipitation values were identified using xmACIS based on the dates listed in the HDSC analyses. For some events, unofficial rain gauges measured values higher than what is shown in the table but are not accounted for by the official sources included within xmACIS (Eggleston 2021).
The WFOs associated with these events were identified and then contacted following study approval by the University of Oklahoma Institutional Review Board (study 11608) and NWS headquarters. A recruitment email was then sent to the meteorologist in charge at each WFO to solicit participation from forecasters. In some cases, a participant identified another forecaster who could potentially participate (i.e., snowball sampling). Because of the sampling method, it is difficult to know exactly how many forecasters received the recruitment email. The researchers were in direct contact with 28 potential participants, and 21 NWS forecasters throughout the south-central United States participated in semistructured phone interviews. All participants had at least 10 years of experience working in the weather enterprise, and the group averaged 22 years of experience with the NWS. While these forecasters represented various roles and offices, results may not be generalizable across the region or country. Instead, this study opens the door to a better understanding of the forecast and communication processes in NWS offices during extreme rainfall events.
b. Interview protocol
The interview protocol included open-ended questions that sought to address the three RQs. For instance, for RQ1 (How do forecasters internally process model outliers?), forecasters were asked about how past experiences influenced their trust in model output and how model output influenced their confidence leading up to an event. For RQ2 (How do forecasters externally communicate model outliers and outlier events?), forecasters responded to questions about working with partners, communicating forecast uncertainty, and what they thought was important for audiences to understand about extreme rainfall events. RQ3 (Do forecasters consider climate change when forecasting and communicating extreme rainfall events?), was addressed when the forecaster brought up climate change in response to a question about whether extreme rainfall events were increasing in frequency and/or intensity.
Several interview questions were tailored to the extreme rainfall event (Table 4) experienced by the participant (i.e., worked), but many participants also spoke about heavy and extreme rainfall events more generally. Social media posts about the events in question that had been published by the relevant WFO were used as conversational starting points. During the original protocol development, the climate change question was intended to be a central focus. However, the NWS approved the protocol subject to climate change only being addressed if brought up by the participant. Because of these constraints, RQ3 was not addressed as thoroughly as desired and the subsequent findings related to the topic are necessarily partial. The interview protocol is available in its entirety from the authors by request.
c. Data collection and analysis
All but two interviews were conducted over the phone. One forecaster opted to send the responses via email, and one WFO participated with three forecasters via video conference. The interviews were recorded and lasted about 1 h each (M = 56 min). By the final interviews, answers to several questions were no longer unique, indicating a saturated sample.
All interviews were transcribed by the lead author and responses were organized by interview question to allow for comparison across interviews. Forecaster identifiers were removed to maintain participant anonymity and will be referred to by their interviewee numbers (e.g., I1, I2, I3). Initially, the analysis was approached using inductive reasoning, not aiming to identify themes from previous research (Braun and Clarke 2006). However, the thematic analysis revealed concepts that closely aligned with existing frameworks. The sensemaking, judgement and decision-making, and principles of forecasting frameworks alone did not entirely explain the forecasting process from forecast purpose to forecast presentation. So, these frameworks and principles were combined to create the extreme event forecast communication process model.
4. Results
Elements of existing social science frameworks were identified in the data (Braun and Clarke 2006), including sensemaking, the principles of forecasting, and JDM. Taken altogether, an extreme events forecast communication process model (Fig. 2; “extreme event model,” henceforth) was developed to describe how forecasters internally process and externally communicate extreme rainfall events. The extreme event model builds upon and extends the work of Lejano et al. (2016) and Hoffman et al. (2017).
The extreme event forecast communication process model applies sensemaking, judgment and decision-making, and principles of forecasting to the extreme rainfall forecast and communication process. The sensemaking and JDM boxes on the left represent the processes that forecasters experience. The sensemaking and JDM boxes on the right represent the processes that forecasters consider their audiences experiencing.
Citation: Weather, Climate, and Society 15, 3; 10.1175/WCAS-D-22-0141.1
This model illustrates the movement of information from the source (data) to the organization, in this case, the NWS, where it is translated (intraorganizational processes) by the forecasters into information that can be distributed to the receiver (forecast presentation). The Hoffman et al. (2017) base model of expertise starts the process with identifying the “problem of the day” (forecast purpose), then examining data and going through sensemaking processes to make judgments and taking a course of action (forecast presentation). Sensemaking and JDM processes not only apply to forecasting an event (the sensemaking and decision-making boxes on the left in Fig. 2) but forecasters also consider the sensemaking and JDM processes of their audiences (the sensemaking and decision-making boxes on the right in Fig. 2), which influence how forecast information is presented. Forecast uncertainty will always be present (Hoffman et al. 2017), is identified in the intraorganizational processes, and impacts how the forecast is presented. Below, how the forecasters’ responses align with these models and frameworks is described.
a. Forecast purpose
At the beginning of the forecast process, forecasters think about what decisions will be made on the basis of that forecast and what is needed to make those decisions (Armstrong 2001b). Fourteen forecasters talked about the importance of knowing their EM partners’ thresholds and information needs when forecasting an event. Less detailed information is often sufficient for media and other public audiences, as they do not have to “make these large-scale early decisions for a lot of people” (I18). Understanding the forecast purpose (Table 2) enables forecasters to begin to process the data in front of them.
b. Intraorganizational processes: Sensemaking
Sensemaking processes (Table 1) help NWS meteorologists interpret the data available to them. Twenty forecasters (95%) said that when they see multiple models over multiple runs pointing to a high-impact rain event over a large area, they know that outcome is likely not wrong. This is known as organizing flux. The forecaster can then start to determine potential impacts and analyze specific products to verify those impacts. For example, I4 looked to parameters such as precipitable water, I7 determined what forcing mechanisms played a role, and I15 analyzed satellite images. By organizing flux, forecasters can avoid being overwhelmed with information and can focus their attention.
Forecasters also use noticing and bracketing when they encounter an unfamiliar situation. By noting differences, forecasters can begin making sense of the situation. For example, when a model produces a precipitation value that is greater than any of the others, the forecasters will take note. To some forecasters (n = 9; 43%), the outlier can be considered the extreme maximum amount possible. For other forecasters (n = 5; 24%), outliers did not carry a lot of weight. However, most forecasters (n = 15; 71%) agreed that the model outlier should be monitored to see how other models and model runs behave. Forecasters also use noticing and bracketing sensemaking processes in the long term. Most forecasters (n = 18; 86%) stated they noticed increasing frequency and/or intensity in extreme rainfall events. I13 noted that multiple, high-end, localized rainfall events per year are occurring within the county warning area. I8 observed that rainfall amounts are higher than they once were. However, when asked if that knowledge impacted how they internally process and externally communicate major rainfall events, all forecasters said that is not something they consider leading up to or during the event.
Another sensemaking property used by forecasters is retrospect. Experience with extreme rainfall events makes forecasters less likely to ignore model outliers. Seeing extreme rainfall events occur opens forecasters’ minds to precipitation amounts that once seemed impossible. I12 discussed that forecasters try to learn from each event and adjust accordingly so that they are better able to understand the next event.
Presumption is a basic sensemaking property in the forecast process, as forecasters are tasked with predicting how the situation will unfold and identifying potential associated impacts. For example, I2 recalled deciding to increase the precipitation forecast to an extreme value because of the rainfall amounts they were seeing and how they thought the storm was going to evolve. Seventeen forecasters (81%) discussed connections between certain meteorological features and extreme rainfall. Forecasters were also aware of factors that will worsen impacts, such as antecedent soil conditions.
Forecasters also use presumption when considering extreme rainfall events in the context of a changing climatology. Forecasters connect the predicted increase in frequency and intensity of these events as well as their impacts to what they know to be true. Six forecasters talked about conducting research to prove to themselves that these events are increasing. There were five who linked increased impacts to increasing human development in vulnerable areas. Whether forecasters are linking climatological or human (or both) causes to the increased impacts of these events, they are still connecting that trend to what they see as concrete evidence, which aligns with the presumption sensemaking property.
Sensemaking can also be impacted by organizational constraints (social and systemic) and other social factors (Weick et al. 2005). Of the nine forecasters in the study who were asked if they thought about the potential impacts of climate change during an event, eight expressed that they did not or tried to keep the climate aspect separate because it was outside the time and space scale for which they were responsible.
NWS forecasters work with others both in their office and in other offices to determine what is going on during an event, also known as organizing through communication. I20 described working with a coworker as they both realized that the event was something they had never seen before. This realization led them to the decision that the first flood warning in 20 years for a major river in their area was necessary. WFOs participate in multi-office calls during which they come to an agreement on rainfall amounts, especially near county warning area borders. Through communication with others, both within their office and with other offices, forecasters can make better sense of an event.
The sensemaking property of representativeness (Doswell 2004) is what NWS forecasters refer to as pattern recognition (I9). When forecasters see a meteorological setup that is familiar to them, pattern recognition helps them figure out how the event might unfold. This could help them catch an event that the models may not be picking up. Twelve forecasters talked about recognizing specific characteristics of a storm, such as slow movement or a lot of moisture, that could cause significant impacts. By recognizing patterns that tend to cause these issues, forecasters are better able to make sense of an impending event.
The goal of sensemaking is to determine which, if any, actions are necessary (Weick et al. 2005). For example, forecasters are tasked with analyzing model output and knowing when to leave it as is or to adjust (I3). Forecasters may use any of the above properties to distinguish between model output that needs improvement and that which does not.
c. Intraorganizational processes: Judgment and decision-making
When working an event, forecasters are often influenced by unconscious heuristics and biases that impact their decision-making (Millet et al. 2020). These include affect, anchoring and adjustment, availability bias, confirmation bias, and temporal or spatial myopia (Table 3).
As forecasters face high-impact events, even as scientific experts, emotions come into play. Affect (Millet et al. 2020) can impact decision-making when forecasters see events unfold where they live (Smith 2020). For example, the August 2016 Louisiana event flooded thousands of homes. In 2019, Hurricane Barry looked like it was going to have similar impacts. The WFO issued products based on rain that had not yet fallen, an atypical practice (I13). It is entirely possible that the office responsible would not have issued those forecasts if the 2016 event had not had such a significant societal impact on the region, including for families of forecasters.
The forecasting process also includes anchoring and adjustment. Based on the initial model output, forecasters may incrementally change that output in the forecast based on their meteorological knowledge and continue to do so as the event gets closer and information evolves. For example, I8 described how I8’s office initially underforecast the event but then adjusted as the event developed.
Availability bias also impacts forecasters’ decisions. For example, based on experience in the area, I6 felt confident going into a rain event because they knew locally intense rain was possible at that time of year. While NWS forecasters periodically come across significant events, they may struggle to grasp the magnitude of an extreme or unprecedented event. Similarly, forecasters also have confirmation biases. I14 described that some forecasters may feel that such an extreme event is impossible. They emphasized the importance of being open to such an event and believing the evidence being presented. If forecasters are hesitant to issue a forecast that pushes them out of their comfort zone, that could result in under-forecasting the amounts and associated impacts.
Temporal or spatial myopia can also impact forecaster decisions. For example, I8 described a high-confidence, high-impact event when they had to urge another forecaster to issue a watch earlier than normal to maximize lead time. While the other forecaster was following NWS guidelines (NWS 2021b), issuing the product early likely helped decision-makers act sooner than they otherwise would have.
d. Forecast uncertainty
In the intraorganizational processes of sensemaking and decision-making, forecasters encounter something that is inherent in weather forecasting and complicates forecast presentation: forecast uncertainty (Table 2). Forecasters must decide how to account for and if they should communicate this uncertainty. One factor in forecast uncertainty is model consistency. When models consistently signal an extreme event, uncertainty decreases, and forecasters are “able to use stronger wording” (I21). Two-thirds of the forecasters (n = 14; 67%) mentioned that increased confidence, which stems from decreased uncertainty, allows them to talk about high amounts and significant impacts earlier on.
Forecasters find it difficult to express uncertainty, but they do not want to imply precision they do not have. Some NWS graphics express rainfall forecasts to the hundredth of an inch, which implies precision. When asked about such a graphic, I18 said that while expressing forecasts in that way is not ideal, it is a limitation of the computer system that creates the products.
Forecasters in this study disagreed on how uncertainty should be communicated. Seven forecasters stated they prefer to use ranges of possible rainfall amounts to quantify uncertainty. Nine (43%) discussed adding verbiage like “locally higher amounts” to their ranges and even specifying what those amounts could be. Nine (43%) suggested describing confidence levels by using words such as “likely” or “expected.” I10 discussed how confidence levels can be subjective and that each forecaster would describe how certain they were in an event differently. Three forecasters (14%) said they would prefer to refrain from expressing uncertainty altogether, as the lack of confidence causes public audiences to lose trust in the forecast. Some forecasters thought it was more effective to message the spectrum of possible impacts. Two forecasters (9.5%) discussed using probabilities of exceedance of a certain return interval, while others (n = 4; 19%) pointed out that return intervals are not always accurate.
Some forecasters noted that how uncertainty is expressed depends on the audience receiving the information. Five forecasters (24%) said they prefer to not express uncertainty in external products. However, forecasters can provide more complicated information to their sophisticated partners. Increasingly, it is common for forecasters to brief sophisticated partners on the most likely forecast but also give a reasonable worst-case scenario.
There was no consensus on the best way to express uncertainty for external audiences, but three forecasters thought educational campaigns to educate audiences on uncertainty information would be helpful. Most forecasters were willing to work with researchers to establish best practices.
e. Forecast creation: Sensemaking
Once forecasters have processed the information themselves and accounted for uncertainty, they then think about how to message the event. Forecasters consider the sensemaking processes their audiences will go through when they receive this information. These processes include labeling, retrospect, presumption, action, and organizing through communication (Table 1 and the sensemaking box on the right side of Fig. 2).
Forecasters will use labeling to help their audiences understand an event. For example, assigning an event to a certain category (e.g., hurricane category number) affects how the event is perceived by audiences. I16 hypothesized that Tropical Storm Imelda was named because people pay more attention to storms with names. Forecasters will also use specific words to catch the attention of their audiences. I1 discussed making the decision to use the word “catastrophic” and how it caught the attention of and mobilized the media. I17 discussed that uncertainty quantifiers could be added to the labels. However, forecasters also noted that to maintain trust with the audience, it is important to avoid false alarms with such strong language.
Forecasters think about how their audiences will apply retrospect to make sense of a situation. I9 said that sophisticated partners of the office are more alert to extreme rainfall threats if they have experienced an event before. In addition to being more alert to a flooding threat, past events enable partners to identify flood-prone areas and give them special attention. Forecasters then know to include those problem spots in their briefings (I10).
Presumption is another sensemaking property that forecasters anticipate their audiences using as they process an impending event. Like forecasters, partners will connect the current environmental situation to potential impacts. I5 recalled that partners of the office knew more rain would cause significant impacts based on the heavy rains they had recently seen. When partners are aware of such a possibility, it raises a sense of urgency and attention to what the NWS forecasters are telling them.
Forecasters will organize through communication to ensure audiences understand the situation that is unfolding. If a partner does not understand a situation, forecasters can communicate further with that partner, as I18 recalled. Forecasters also share information with their sophisticated partners that may not yet be released to the public (I8). This way, partners can be prepared for the decisions they may have to make before forecasters have the confidence to brief it to a broader audience.
Message consistency is achieved by coordinating with partners across the weather enterprise, which is not always easy. I6 recounted the frustration of trying to get the media to shift focus from a tornado threat to a flood threat, then resolving the issue through coordination in NWSChat.1 Nine forecasters (43%) said that to communicate openly with partners and ensure message consistency, trust must be established during quiet times, before an event occurs. I14 described conducting weekly briefings with partners so the partners are familiar with the forecasters and not dealing with strangers when the stakes are high. Forecasters also rely on communication from their audiences to understand a situation. Social media has increasingly become an avenue for storm report submission. Keeping track of what the media is covering also helps with situational awareness when partners such as emergency managers are too busy to send in storm reports (I10).
Once the audience has made sense of the impending event, they should be moved to action. I8 talked about including actionable information when communicating to audiences as they prepare for the event, such as having enough supplies to last through a multiday event. Forecasters provide such information in the hope that their audiences take the threat seriously and do not put themselves in harm’s way during the event.
f. Forecast creation: Judgment and decision-making
Similar to sensemaking, forecasters also consider the decision-making elements that their audiences will use (see JDM box on the right side of Fig. 2). Like forecasters, the audience’s decision-making can be influenced by affect. I5 stated that people in I5’s area are very sensitive about rainfall after the May 2015 flooding event. I10 shared a similar experience, saying that people who live in a flood-prone region will become concerned and start asking questions once a potential heavy rain event is mentioned. These emotions toward extreme rainfall events could mean that audiences are more likely to decide to take appropriate action.
Audience decision-making is also impacted by availability bias. This means that forecasters must be able to communicate events in a way that allows the audience to picture the event. Forecasters felt that audiences have trouble understanding numbers, and I17 described an interaction with a partner who did not understand the potential impacts of a 40-in. (1 in. = 2.54 cm) rainfall forecast. I3 also expressed frustration in the public’s interest in the numbers rather than the impacts of the event. By focusing on impacts, forecasters may provide a clearer image for their audiences. However, I3 also noted that sometimes it can be difficult to do that in a way to which the public can relate. I2 discussed referencing past events to help end users understand the situation, but that strategy is not useful for everyone. I16 noted that population growth in I16’s area means that although extreme rainfall events occur relatively frequently, new residents may not be familiar with extreme rainfall amounts. I12 expressed the need to message that an event may flood differently than past events so those that have not flooded previously do not have a false sense of security. Sometimes no comparison is possible, and I13 stated that one of the biggest challenges is messaging unprecedented events, because I13 is aware that people can only comprehend what they have seen before.
Like forecasters, audiences also exhibit confirmation bias. I8 thought that members of the public focus on a deterministic number and if that forecast number does not “verify,” they may lose trust in NWS forecasts. The same goes for a missed location. If the event occurred but someone did not personally experience it, they may perceive it as a busted forecast. Similarly, confirmation bias is why forecasters are hesitant to give the public the worst-case scenario, as they worry that people might fixate on that worst-case scenario.
Forecasters also consider that audiences have a finite pool of worry. For example, during Hurricane Harvey, the media focused on the appearance of the storm on satellite rather than the actual threats it was posing. I2 discussed partners focusing on the wind and landfall of the storm and that forecasters had to emphasize the heavy rainfall and flooding impacts. It is up to the forecasters to make sure that their audiences are focused on the most dangerous threat or multiple threats if relevant.
While extreme rainfall events are rare, that does not necessarily mean that they cannot occur in short succession. NWS forecasters must be able to convince their audiences of this to combat gambler’s fallacy. I12 discussed avoiding the use of return intervals to communicate event probabilities, as the public may believe that a 100-yr event can only happen once every 100 years.
People are also loss averse. NWS forecasters can use this to their advantage when relaying safety information to their audiences. They want their audiences to understand just how dangerous flooding can be, but often the public does not grasp that danger. For example, I21 discussed the issue of people trying to cross flooded roadways overnight and that the message should be to stay at home and not risk losing their lives.
g. Forecast presentation and communication
The culmination of the forecast purpose, intraorganizational processes, forecast uncertainty, and the sensemaking and decision-making frameworks lead to the presentation of the forecast (Table 2). Not every meteorological variable is presented in the same way. I1 discussed being able to use numbers when presenting hazards such as heat index or snowfall to audiences but that they use qualitative descriptors for rainfall impacts. I4 recalled using numerical rainfall forecasts for an extended event but instead of messaging the storm total, rainfall amounts were divided into 12–24-h forecasts to help audiences understand the event as it happened. Forecasters thought their audiences may not understand what quantitative forecasts were saying without context. Sometimes, both numbers and safety information will be presented (I8). However, the forecasters seemed to agree that, when forecasting extreme rainfall events, overall impacts were more meaningful than numbers.
Most forecasters (n = 14; 67%) said that how the forecast is presented also depends on the audience that is being addressed. I7 expressed that they had little faith in what public audiences could understand. I19 described providing a straightforward, most likely forecast, while still being aware of confirmation bias. Overall, the forecasters seemed to agree that forecast information that is communicated to broad audiences should be kept qualitative, whereas sophisticated partners are able to interpret quantitative information appropriately.
Forecasters rely heavily on in-person, conference call, and/or email briefings to communicate with sophisticated partners such as emergency managers. I15 described consistent communication between I15’s office and emergency managers up to 3–5 days leading up to an event. During briefings, the forecasters focus on messages such as possible threats, level of confidence, timing, and impacts. Forecasters can also provide these sophisticated partners with various possible scenarios. This way, forecasters can communicate model outliers as reasonable worst-case scenarios. Forecasters hesitate to share this information with the public because of the public’s confirmation bias. As far as sharing the worst-case scenario with the media, forecasters’ opinions were divided. I15 felt that the media understands they are only possible situations. Others (I8, I19) were worried about the media sensationalizing the worst-case scenario.
The relationships between forecasters and their audiences strengthen the presentation of the forecasts. When providing briefings in person, or via video or phone, body language and tone of voice help convey the severity of a situation. I20 recalled partners telling him they could gauge his concern by his tone and word choice. I18 recalled a time an EM asked if they should prepare for evacuations and that they did not need an answer when they saw the look on the forecaster’s face. I20 talked about how EMs in I20’s area are comfortable sending a text any time to ask for the forecaster’s opinion. A few forecasters expressed concern that such connections with partners could be lost or weakened when they were limited to virtual interactions during the COVID-19 pandemic.
Forecast information is communicated to partners in a variety of ways. In addition to briefings and one-on-one interactions, offices distribute email briefings to partners. I5 estimated that about 1000 people received the briefing and could forward the briefings to others. Some offices may even post the briefing on social media. Forecasters will also use NWSChat to distribute information, especially to the media that may not be involved in briefings.
The NWS has become more public facing in recent years. Prior to the development of social media, the public could only reach WFOs via email or phone, which made it difficult to provide quick responses. Seventeen forecasters (81%) discussed how social media has increased interactions with public audiences. I13 stated that they prefer social media over NOAA Weather Radio because it is faster and reaches more people. Social media enables forecasters to discuss the development of the situation, possible rainfall amounts, how things could change, potential impacts, and even uncertainty. However, forecasters are aware that social media is not without flaws. I19 expressed that it is difficult to form relationships on social media. A few forecasters also expressed concern that the graphics they post could be manipulated by people trying to imitate an official source. Although social media are not perfect, forecasters thought that they have become an incredibly useful tool.
While many tools exist for presenting forecast information, forecasters expressed that a perfect forecast is useless if it is not communicated in an understandable way. I14 said, “If we can’t communicate in language that the public understands, then it doesn’t matter how confident we are.” Forecasters want each person to know potential impacts and to be able to understand their personal risk and what they need to do to avoid danger.
5. Discussion and conclusions
This study offers an extreme event forecast communication process model (Fig. 2) to visualize the extreme rainfall forecasting and communication process and provide theoretical answers to RQs 1 and 2. The model is simpler than the actual forecast process but provides a high-level schematic. This model combines two preexisting models, the textual processing model of risk communication (Lejano et al. 2016) and the base model of expertise (Hoffman et al. 2017), as well as forecasting principles (Armstrong 2001b) and the conceptual frameworks of sensemaking (Weick et al. 2005; Butterworth 2010; Doswell 2004) and JDM (Millet et al. 2020).
Forecasters use the sensemaking properties of retrospect, presumption, representativeness, and organizing through communication and the decision-making properties of availability bias and affect, calling on their expertise and experience as they work to comprehend the evolving weather situation. How forecasters process information is also impacted by organizational guidelines to which the social and systemic sensemaking property and the anchoring and adjustment decision-making property can be applied.
When forecasters are processing model output, the sensemaking properties of action and organizing flux play a role. Forecasters must decide if the model output is valid or if it needs adjustment. Forecasters must process a lot of data (Daipha 2015; Doswell 2004), and consistent model output highlights on which information they should focus. Model consistency increases forecasters’ confidence in the event. Forecasters stated that when they see a model outlier, they monitor it to see how it and other models evolve. That outlier may also be communicated as a possible worst-case scenario, especially to sophisticated users.
When communicating these events, forecasters must produce consistent messages. As found in previous research (Daipha 2015; Childs and Schumacher 2018; Sherman-Morris et al. 2018), forecasters ensure consistency by discussing the message within their offices, with other NWS offices, and with their decision-making and media partners (organizing through communication).
How the forecast is presented also depends on the meteorological variable, and forecasters in this study thought that extreme rainfall events are best communicated through impacts rather than numbers. Impacts can help audiences better picture what an event could look like, reducing availability bias. How the event is communicated also depends on lead time. Farther out in time from the event, uncertainty may be greater and fewer details known. Forecasters begin messaging a heavy rainfall event, then narrow down locations, amounts, and impacts as the event gets closer, consistent with Bostrom et al. (2016).
Forecasters in this study said they tailor their messaging to different audiences, which aligns with prior literature (e.g., Rouleau 2016; Sanders et al. 2020). For public audiences, forecasters keep information relatively simple. More complex information is provided to sophisticated partners. Forecasters try to be aware of thresholds that partners have for certain decisions (forecast purpose) and prioritize that information in briefings.
Forecasters also consider the sensemaking and decision-making processes that their users experience when creating these forecast products. These processes include the sensemaking properties of retrospect, labeling, and presumption and the decision-making properties of availability bias, confirmation bias, gambler’s fallacy, affect, finite pool of worry, and loss aversion. Forecasters want to improve communication so their audiences can better protect themselves when they face an extreme rainfall event. They all agreed that the best way to do so is by providing impact-based information, which is currently a significant focus of the NWS (NWS 2019).
Forecasters struggle to both process and communicate inherent forecast uncertainty (Schumacher 2017). Uncertainty can be subjective, and forecasters do not agree on how it should be expressed. Some forecasters avoid communicating uncertainty to public audiences, thinking they are not capable of understanding probabilistic information (Daipha 2015; Stewart et al. 2015; Demeritt et al. 2010) or that uncertainty would cause them to lose trust in the forecast (Rouleau 2016). However, a few forecasters and Fundel et al. (2019) and Roulston and Kaplan (2009) note that when probabilities are properly presented, the public can understand them (Ripberger et al. 2022).
The extreme event forecast communication process model provides insight into forecast and communication processes before and during extreme rainfall events. It also provides theoretical answers to RQs 1 and 2. The data from this study indicate there is substantial variability in terms of how experts interpret information and subsequently communicate. The model could be the starting point to providing a framework for training forecasters on how such concepts impact the forecasting and communication process. However, more research will be needed to ensure that this model is applicable in other regions of the United States and to other weather hazards before any training becomes operationalized. We also present practical findings to the three RQs. First, forecasters internally process model outliers by mentally acknowledging them and monitoring subsequent models to see how that outlier evolves. They may even treat that outlier as a possible worst-case scenario. Second, forecasters sometimes communicate outliers as possible worst-case scenarios to sophisticated partners but not to all public audiences. Forecasters also emphasize the impacts of extreme rainfall events so that their users can make decisions to protect themselves and others. For the third research question, the forecasters who brought up climate change acknowledged that it is playing a role in extreme rainfall events in a general sense, but linking climate change to individual events, especially while an event is ongoing, is beyond the scope of their position.
This study included a few limitations. First, the discovery that forecasters do not consider climate change during an event should be considered a partial finding. This is a reflection of the approved interview protocol. Future research may be able to study this topic more thoroughly. Next, forecaster recollection of events may have been incomplete. Some forecasters were interviewed about events that had occurred 4 years prior. While forecasters did say these events were significant enough that they remembered them vividly, memories are not perfect. Another limitation is that the study was conducted in a geographic area that sees significant rainfall events relatively frequently, especially in recent years. Therefore, forecaster responses in this study may differ from those of forecasters who work in regions where such events do not happen as frequently or with such great magnitudes. Furthermore, it is possible that bias was introduced during the coding process, which was performed by the lead author. Using two or more coders is methodologically ideal but did not occur due to funding constraints. The lead author made every effort to be as unbiased as possible, but adding another coder may have resulted in slight analytical differences.
In addition to addressing forecaster perceptions on climate change and its impact on the forecasting and communication processes, future work could build on the answers to the research questions and test the applicability of the extreme event forecast communication process model to other weather hazards and/or geographic regions. If further testing finds the model viable and applicable to other regions and hazards, it could be used in forecaster training.
Future projects could also investigate what forecasters look for in the environmental data when they see a model outlier. Do they look to ensemble forecasting tools such as the extreme forecast index (NWS 2023)? Do forecasters use or see themselves using such tools when communicating extreme events? What technological developments do forecasters want to be better able to communicate such events to their audiences? Projects could also investigate how situational factors like event scale (isolated flash flooding versus an atmospheric river type of event) or forecaster experience impact communication practices. This research opens the door to answering these questions and others.
Acknowledgments.
Funding was provided by the National Oceanic and Atmospheric Administration (NOAA) Climate Program Office Grant NA18OAR4310337 and NOAA/Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA16OAR4320115, U.S. Department of Commerce. The authors thank Drs. Kodi Berry and Harold Brooks for their participation on the lead author’s master’s degree committee. The authors also thank the NWS forecasters who participated and helped to recruit participants. We also thank the four anonymous peer reviewers whose comments helped to improve the paper.
Data availability statement.
Interview questions and de-identified data are available from the author upon reasonable request. Those data are not publicly available because of the institutional review board regulations.
REFERENCES
Armstrong, J. S., Ed., 2001a: Standards and practices for forecasting. Principles of Forecasting: A Handbook for Researchers and Practitioners, J. S. Armstrong, Ed., International Series in Operations Research and Management Science, Vol. 30, Springer Science & Business Media, 679–732.
Armstrong, J. S., Ed., 2001b: Principles of Forecasting: A Handbook for Researchers and Practitioners. International Series in Operations Research and Management Science, Vol. 30, Springer Science & Business Media, 850 pp.
Bostrom, A., R. E. Morss, J. K. Lazo, J. L. Demuth, H. Lazrus, and R. Hudson, 2016: A mental models study of hurricane forecast and warning production, communication, and decision-making. Wea. Climate Soc., 8, 111–129, https://doi.org/10.1175/WCAS-D-15-0033.1.
Braun, V., and V. Clarke, 2006: Using thematic analysis in psychology. Qual. Res. Psychol., 3, 77–101, https://doi.org/10.1191/1478088706qp063oa.
Brown, V. M., B. D. Keim, and A. W. Black, 2020: Trend analysis of multiple extreme hourly precipitation time series in the southeastern United States. J. Appl. Meteor. Climatol., 59, 427–442, https://doi.org/10.1175/JAMC-D-19-0119.1.
Burgeno, J. N., and S. L. Joslyn, 2020: The impact of weather forecast inconsistency on user trust. Wea. Climate Soc., 12, 679–694, https://doi.org/10.1175/WCAS-D-19-0074.1.
Butterworth, R. E., 2010: The diffusion of new media and radar technology and the role of broadcasters’ prior experience and public perceptions in television severe weather coverage. M.S. thesis, The Graduate College, University of Oklahoma, 150 pp., https://hdl.handle.net/11244/325577.
Carr, R. H., B. Montz, K. Maxfield, S. Hoekstra, K. Semmens, and E. Goldman, 2016: Effectively communicating risk and uncertainty to the public: Assessing the National Weather Service’s flood forecast and warning tools. Bull. Amer. Meteor. Soc., 97, 1649–1665, https://doi.org/10.1175/BAMS-D-14-00248.1.
Childs, S. J., and R. S. Schumacher, 2018: Cold-season tornado risk communication: Case studies from November 2016 to February 2017. Wea. Climate Soc., 10, 419–433, https://doi.org/10.1175/WCAS-D-17-0073.1.
Daipha, P., 2015: Masters of Uncertainty: Weather Forecasters and the Quest for Ground Truth. University of Chicago Press, 271 pp.
Demeritt, D., S. Nobert, H. Cloke, and F. Pappenberger, 2010: Challenges in communicating and using ensembles in operational flood forecasting. Meteor. Appl., 17, 209–222, https://doi.org/10.1002/met.194.
Demuth, J. L., R. E. Morss, B. H. Morrow, and J. K. Lazo, 2012: Creation and communication of hurricane risk information. Bull. Amer. Meteor. Soc., 93, 1133–1145, https://doi.org/10.1175/BAMS-D-11-00150.1.
Doswell, C. A., III, 2004: Weather forecasting by humans–heuristics and decision making. Wea. Forecasting, 19, 1115–1126, https://doi.org/10.1175/WAF-821.1.
Eachus, J. D., and B. D. Keim, 2019: A survey for weather communicators: Twitter and information channel preferences. Wea. Climate Soc., 11, 595–607, https://doi.org/10.1175/WCAS-D-18-0091.1.
Easterling, D. R., and Coauthors, 2017: Precipitation change in the United States. Climate Science Special Report: Fourth National Climate Assessment, D. J. Wuebbles et al., Eds., Vol. I, U.S. Global Change Research Program, 207–230.
Eggleston, K. L., 2021: xmACIS Version 1.0.66c. NOAA Northeast Regional Climate Center, accessed 19 July 2021, http://xmacis.rcc-acis.org/#.
Evans, C., D. F. Van Dyke, and T. Lericos, 2014: Forecaster’s forum: How do forecasters utilize output from a convection-permitting ensemble forecast system? Case study of a high-impact precipitation event. Wea. Forecasting, 29, 466–486, https://doi.org/10.1175/WAF-D-13-00064.1.
Fundel, V. J., N. Fleischhut, S. M. Herzog, M. Göber, and R. Hagedorn, 2019: Promoting the use of probabilistic weather forecasts through a dialogue between scientists, developers, and end-users. Quart. J. Roy. Meteor. Soc., 145, 210–231, https://doi.org/10.1002/qj.3482.
Gigerenzer, G., R. Hertwig, E. van den Broek, B. Fasolo, and K. V. Katsikopoulos, 2005: “A 30% chance of rain tomorrow”: How does the public understand probabilistic weather forecasts? Risk Anal., 25, 623–629, https://doi.org/10.1111/j.1539-6924.2005.00608.x.
Hayhoe, K., and Coauthors, 2018: Our changing climate. Impacts, Risks, and Adaptation in the United States: Fourth National Climate Assessment, Volume II, D. R. Reidmiller et al., Eds., U.S. Global Change Research Program Rep., 72–144, https://doi.org/10.7930/NCA4.2018.
HDSC, 2017: Hurricane Harvey, 25–31 August 2017 annual exceedance probabilities (AEPs) for the worst case 4-day rainfall. NOAA NWS, accessed 21 May 2021, https://hdsc.nws.noaa.gov/pub/hdsc/data/aep/201708_Harvey/AEP_HurricaneHarvey_August2017.pdf.
HDSC, 2021: HDSC annual exceedance probability: Exceedance probability analysis for selected storm events. NWS, accessed 22 February 2021, https://www.weather.gov/owp/hdsc_aep.
Hoffman, R. R., D. S. LaDue, H. M. Mogil, P. J. Roebber, and J. G. Trafton, 2017: Minding the Weather: How Expert Forecasters Think. Massachusetts Institute of Technology, 470 pp.
Hubbard, J. N., 2018: Creating guidelines for effective weather communication: A qualitative analysis of message strategies through the lens of meteorologists. M.S. thesis, Dept. of Communication, George Mason University, 169 pp., http://jbox.gmu.edu/handle/1920/11092.
Joslyn, S., and S. Savelli, 2010: Communicating forecast uncertainty: Public perception of weather forecast uncertainty. Meteor. Appl., 17, 180–195, https://doi.org/10.1002/met.190.
Kahneman, D., and A. Tversky, 1979: Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–292, https://doi.org/10.2307/1914185.
Kuonen, J., F. Conway, and T. Strub, 2019: Navigating mental models of risk and uncertainty within the ocean forecast system: An Oregon case study. Wea. Climate Soc., 11, 431–447, https://doi.org/10.1175/WCAS-D-18-0057.1.
Lejano, R. P., J. M. Ran, and A. M. W. Wilson, 2016: A textual processing model of risk communication: Lessons from Typhoon Haiyan. Wea. Climate Soc., 8, 447–463, https://doi.org/10.1175/WCAS-D-16-0023.1.
Linville, P. W., and G. W. Fischer, 1991: Preferences for separating or combining events. J. Pers. Soc. Psychol., 60, 5–23, https://doi.org/10.1037/0022-3514.60.1.5.
Liu, B. F., and A. A. Seate, 2021: The evolving weather service: Forecasters’ perceptions of their relationships with core partners. Wea. Climate Soc., 13, 437–448, https://doi.org/10.1175/WCAS-D-20-0097.1.
Losee, J. E., N. Z. Naufel, L. Locker Jr., and G. D. Webster, 2017: Weather warning uncertainty: High severity influences judgement bias. Wea. Climate Soc., 9, 441–454, https://doi.org/10.1175/WCAS-D-16-0071.1.
Mallakpour, I., and G. Villarini, 2017: Analysis of changes in the magnitude, frequency, and seasonality of heavy precipitation over the contiguous USA. Theor. Appl. Climatol., 130, 345–363, https://doi.org/10.1007/s00704-016-1881-z.
Meyer, R., and H. Kunreuther, 2017: The myopia bias. The Ostrich Paradox: Why We Underprepare for Disasters, R. Meyer and H. Kunreuther, Eds., Wharton School Press, 13–20.
Milch, K., K. Broad, B. Orlove, and R. Meyer, 2018: Decisions science perspectives on hurricane vulnerability: Evidence from the 2010–2012 Atlantic hurricane seasons. Atmosphere, 9, 32, https://doi.org/10.3390/atmos9010032.
Millet, B., A. P. Carter, K. Broad, A. Cairo, S. D. Evans, and S. J. Majumdar, 2020: Hurricane risks communication: Visualization and behavioral science concepts. Wea. Climate Soc., 12, 193–211, https://doi.org/10.1175/WCAS-D-19-0011.1.
Morss, R. E., and M. H. Hayden, 2010: Storm surge and “certain death”: Interviews with Texas coastal residents following Hurricane Ike. Wea. Climate Soc., 2, 174–189, https://doi.org/10.1175/2010WCAS1041.1.
Morss, R. E., J. L. Demuth, A. Bostrom, J. K. Lazo, and H. Lazrus, 2015: Flash flood risks and warning decisions: A mental models study of forecasters, public officials, and media broadcasters in Boulder, Colorado. Risk Anal., 35, 2009–2028, https://doi.org/10.1111/risa.12403.
Mullens, E. D., M. Shafer, and J. Hocker, 2013: Trends in heavy precipitation in the southern USA. Weather, 68, 311–316, https://doi.org/10.1002/wea.2113.
NWS, 2019: National Weather Service 2019–2022 strategic plan. Tech. Rep., 28 pp., https://www.weather.gov/media/wrn/NWS_Weather-Ready-Nation_Strategic_Plan_2019-2022.pdf.
NWS, 2021a: NWS preliminary US flood fatality statistics. NWS, accessed 12 April 2021, https://www.weather.gov/arx/usflood.
NWS, 2021b: National Weather Service Instruction 10-922, Weather Forecast Office Water Resources Products Specification. NWS Tech. Rep. NWSPD 10-9, 97 pp., https://www.nws.noaa.gov/directives/sym/pd01009022curr.pdf.
NWS, 2022: NWSChat. Accessed 4 June 2022, https://nwschat.weather.gov.
NWS, 2023: Ensemble situational awareness table: ECMWF extreme forecast index. Accessed 3 February 2023, https://satable.ncep.noaa.gov/efi/.
Nickerson, R. S., 1998: Confirmation bias: A ubiquitous phenomenon in many guises. Rev. Gen. Psychol., 2, 175–220, https://doi.org/10.1037/1089-2680.2.2.175.
NOAA NCEI, 2021: U.S. billion-dollar weather and climate disasters. https://www.ncdc.noaa.gov/billions/.
Olson, M. K., J. Sutton, S. C. Vos, R. Prestley, S. L. Renshaw, and C. T. Butts, 2019: Build community before the storm: The National Weather Service’s social media engagement. J. Contingencies Crisis Manage., 27, 359–373, https://doi.org/10.1111/1468-5973.12267.
Perreault, M. F., J. B. Houston, and L. Wilkins, 2014: Does scary matter?: Testing the effectiveness of new National Weather Service tornado warning messages. Commun. Stud., 65, 484–499, https://doi.org/10.1080/10510974.2014.956942.
Ripberger, J., A. Bell, A. Fox, A. Forney, W. Livingston, C. Gaddie, C. Silva, and H. Jenkins-Smith, 2022: Communicating probability information in weather forecasts: Findings and recommendations from a living systematic review of the research literature. Wea. Climate Soc., 14, 481–498, https://doi.org/10.1175/WCAS-D-21-0034.1.
Rouleau, T. L., 2016: Risk communication and behavior: Best practices and research findings. NOAA Social Science Committee Rep., 66 pp., https://repository.library.noaa.gov/view/noaa/29484.
Roulston, M. S., and T. R. Kaplan, 2009: A laboratory-based study of understanding of uncertainty in 5-day site-specific temperature forecasts. Meteor. Appl., 16, 237–244, https://doi.org/10.1002/met.113.
Sanders, S., T. Adams, and E. Joseph, 2020: Severe weather forecasts and public perceptions: An analysis of the 2011 super outbreak in Tuscaloosa, Alabama. Wea. Climate Soc., 12, 473–485, https://doi.org/10.1175/WCAS-D-18-0090.1.
Schumacher, R. S., 2017: Heavy rainfall and flash flooding. Natural Hazard Science, Oxford Research Encyclopedias, Oxford University Press, https://oxfordre.com/naturalhazardscience/view/10.1093/acrefore/9780199389407.001.0001/acrefore-9780199389407-e-132.
Senkbeil, J. C., L. Myers, S. Jasko, J. R. Reed, and R. Mueller, 2020: Communication and hazard perception lessons from category five Hurricane Michael. Atmosphere, 11, 804–823, https://doi.org/10.3390/atmos11080804.
Sherman-Morris, K., H. Lussenden, A. Kent, and C. MacDonald, 2018: Perceptions about social science among NWS warning coordination meteorologists. Wea. Climate Soc., 10, 597–612, https://doi.org/10.1175/WCAS-D-17-0079.1.
Sivle, A. D., and T. Aamodt, 2019: A dialogue-based weather forecast: Adapting language to end-users to improve communication. Weather, 74, 436–441, https://doi.org/10.1002/wea.3439.
Slovic, P. M., M. L. Finucane, E. Peters, and D. G. MacGregor, 2007: The affect heuristic. Eur. J. Oper. Res., 177, 1333–1352, https://doi.org/10.1016/j.ejor.2005.04.006.
Smith, R., 2020: Put your oxygen mask on first: Managing mental health in meteorology. 19th Annual Student Conf., Boston, MA, Amer. Meteor. Soc., https://ams.confex.com/ams/2020Annual/meetingapp.cgi/Paper/372323.
Stewart, A. E., C. A. Williams, M. D. Phan, A. L. Horst, E. D. Knox, and J. A. Knox, 2015: Through the eyes of experts: Meteorologists’ perceptions of the probability of precipitation. Wea. Forecasting, 31, 5–17, https://doi.org/10.1175/WAF-D-15-0058.1.
Stigliani, I., and D. Ravasi, 2012: Organizing thoughts and connecting brains: Material practices and the transition from individual to group-level prospective sensemaking. Acad. Manage. J., 55, 1232–1259, https://doi.org/10.5465/amj.2010.0890.
Tisch, D., and J. Galbreath, 2018: Building organizational resilience through sensemaking: The case of climate change and extreme weather events. Bus. Strategy Environ., 27, 1197–1208, https://doi.org/10.1002/bse.2062.
Trenberth, K. E., A. Dai, R. M. Rasmussen, and D. B. Parsons, 2003: The changing character of precipitation. Bull. Amer. Meteor. Soc., 84, 1205–1218, https://doi.org/10.1175/BAMS-84-9-1205.
Tversky, A., and D. Kahneman, 1971: Belief in the law of small numbers. Psychol. Bull., 76, 105–110, https://doi.org/10.1037/h0031322.
Tversky, A., and D. Kahneman, 1974: Judgement under uncertainty: Heuristics and biases. Science, 185, 1124–1131, https://doi.org/10.1126/science.185.4157.1124.
Weick, K. E., K. M. Sutcliffe, and D. Obstfeld, 2005: Organizing and the process of sensemaking. Organ. Sci., 16, 406–421, https://doi.org/10.1287/orsc.1050.0133.
Wilson, K. A., P. L. Heinselman, P. S. Skinner, J. J. Choate, and K. E. Klockow-McClain, 2019: Meteorologists’ interpretations of storm-scale ensemble-based forecast guidance. Wea. Climate Soc., 11, 337–354, https://doi.org/10.1175/WCAS-D-18-0084.1.
