Spot weather forecasts (SWFs) are issued by Weather Service offices throughout the United States and are primarily for use by wildfire and prescribed fire practitioners for monitoring local-scale weather conditions. This paper focuses on use of SWFs by prescribed fire practitioners. Based on qualitative, in-depth interviews with fire practitioners and National Weather Service forecasters, this paper examines factors that influence perceptions of accuracy and utilization of SWFs. Results indicate that, while several well-understood climatological, topographical, and data-driven factors influence forecast accuracy, social factors likely have the greater impact on perceptions of accuracy, quantitative accuracy, and utilization. These include challenges with building and maintaining relationships between forecasters and fire managers, communication issues around updating SWFs, and communicating forecast confidence and uncertainty. Operationally, improved quantitative skill in a forecast is always desirable, but key opportunities for improving accuracy and utilization of these forecasts lie in 1) enhancing the processes and mechanisms for communication between a Weather Forecast Office and fire practitioners—before, during, and after an SWFs is issued—and 2) working with the wildland fire community to experiment with forecast uncertainty and confidence information in SWFs and evaluate impacts of these approaches.
In this paper, we report—using results from qualitative, in-depth interviews with fire practitioners and forecasters—exploration of use of spot weather forecasts (SWFs) by fire practitioners (e.g., burn bosses, incident commanders, and fire management officers) for prescribed burns (utilized as vegetation management treatments). In the United States, prescribed fires on state and federal lands follow a fire prescription plan, which details parameters and implementation procedures that must be considered to proceed with a planned burn. These plans require a forecast of weather conditions, such as relative humidity, wind speed and direction, temperature, and often other elements related to smoke and air quality (Nauslar et al. 2015). These parameters are determined in advance of a burn, depending on the management objective (e.g., hazardous fuel reduction or riparian restoration). To meet the operating conditions set forth in the fire prescription plan, a fire practitioner usually requests an SWF the night before or morning of the scheduled burn to evaluate the weather conditions at the burn location to determine if they are “in prescription.”
This study did not consider use of SWFs for wildfire operations. While there is overlap between uses of SWFs in prescribed fire and wildfire events, SWFs have a different planning structure, staffing levels, and applications in decision-making in ways that are unique to each type of event. This study is also part of a larger project that focused on developing tools to evaluate the quantitative accuracy of SWFs (Lammers and Horel 2014; Nauslar et al. 2015). In an effort to understand perceptions of SWF accuracy and utilization by forecasters and fire practitioners, the research described in this paper used qualitative, in-depth interviews with fire practitioners and forecasters to explore these factors. This exploratory study asked participants open-ended questions about SWF accuracy, with no intention of addressing all the possible components that influence these factors. Nonetheless, the responses provide insight into how SWFs as a decision-support tool by fire practitioners may be enhanced. As Morss and Ralph (2007, p. 533) suggest, this qualitative approach, “while relatively rare in the weather forecasting community, can generate rich, detailed knowledge about the processes examined.” The research questions that guided the study were as follows:
What are the perceptions of accuracy (or uncertainty) of SWFs in the prescribed fire community?
Is there a perceived bias of any of the SWF elements (e.g., wind speed too high or too low)?
What role do SWFs have in the execution of prescribed fire programs?
In the following subsection, we provide an overview of SWFs, their most common uses, and a brief review of previous research on the general public and more specialized forecast users’ perceptions of forecast uncertainty in deterministic forecasts.
The value of weather forecasts in relation to wildland fire has been recognized since the early 1900s (Beals 1914). Today, there are numerous fire weather forecast products available to inform fire management and the public. SWFs (Fig. 1) are issued by National Weather Service (NWS) forecasters at Weather Forecast Offices (WFOs) and are requested by users through an online system (Fig. 2). Between April 2009 and November 2013, approximately 103 370 SWFs were issued nationwide, with the vast majority of these related to either wildfires or prescribed burns. These are local-scale forecasts requested by fire or emergency personnel and are often related to fire, emergencies, search and rescue, or hazardous material situations. Nationwide, SWFs are issued twice as often for prescribed burns as they are for wildfires (Lammers and Horel 2014).
For most uses, SWFs provide a narrow range for each forecast element, expressed as specific magnitudes for a particular place and time (although it is not unusual for time to be more generalized, such as “afternoon”). Forecast uncertainty for an SWF is usually addressed through provision of a range of forecast values. Fire practitioners tend to receive little information regarding the uncertainty inherent in an SWF save the forecast ranges (Brown and Murphy 1987; NOAA 2008; Office of the Federal Coordinator for Meterological Services and Supporting Research 2007).
Similar to other forecasting products, NWS forecasters have a highly structured set of tools and outputs available for SWFs available to them for providing forecast services that can limit the ability of both the forecaster and end user to express forecast uncertainty or confidence (Demuth et al. 2012). Fire practitioners also may infer the degree of uncertainty in an SWF, much as the general public and other sophisticated user groups do, and the biases associated with uncertainty may influence perceptions of forecast accuracy (Demuth et al. 2012; Joslyn and LeClerc 2012; Morss and Ralph 2007; Morss et al. 2008, 2010; NOAA 2008; Office of the Federal Coordinator for Meterological Services and Supporting Research 2007; Savelli and Joslyn 2012; Sivle et al. 2014).
Uncertainty information in forecasts is important in decision-making, as this information avoids imparting a sense of false confidence in the forecast (AMS 2008; Murphy 1998; NRC 2003, 2006). This may be especially true for fire practitioners, who must make decisions based on aggregating information from several different sources, including staffing levels (such as available fire crews), other available resources (such as fire engines), burn proximity to sensitive resources (such as air sheds with dense populations), and forecast weather values. Uncertainty or low confidence in the forecast that is not communicated to an end user may lead to a perception of inaccuracy, lowering the credibility of SWFs and forecasters. Perceptions of uncertainty or low confidence in a forecast, or how accurate a forecast is perceived to be, however, may be expressed very differently under circumstances when a fire practitioner is using an SWF as a decision tool. For example, uncertainty that is expressed in an SWF as a wider range of wind speed values may support a fire practitioner’s ability to maintain flexibility in optimizing burn windows by providing greater latitude for existing conditions to fall within the burn prescription plan. Conversely, a fire practitioner’s reluctance to have confidence in a forecast may lead to greater levels of risk aversion and a reduction in flexibility, depending on the contextual factors mentioned above, particularly if the possible impacts from a negative outcome are high (Savelli and Joslyn 2012). As well, fire practitioners, similar to other sophisticated or “high stakes” forecast users, may view uncertainty very differently than forecasters, who, as trained scientists, primarily attempt to reduce or quantify uncertainty (Demuth et al. 2012; Morss et al. 2005; Savelli and Joslyn 2012). Additionally, effectively communicating risk associated with the uncertainty and confidence in the forecast is equally challenging. Research in risk communication suggests that only communicating the “numbers” associated with an event is inadequate to communicate risk (Fischhoff 1995) and requires considering risk attributes, such as risk feelings, or the emotion or affect of risk (Slovic et al. 2004).
While discussions about including uncertainty and confidence information in forecasts are not a new trend (NRC 2006), recent research has provided additional support for including more probabilistic information. Survey research, for example, suggests that NWS forecasters would like to know more about what type of uncertainty information their users need and that there is a high demand by users for uncertainty information, as reported by 74% of forecasters (Novak et al. 2008). These requests often were made by “sophisticated users,” including fire weather officials (Novak et al. 2008). The general public also have expressed a positive orientation toward availability of more uncertainty information in forecasts (Joslyn and Savelli 2010) and, in some experimental research, preferred uncertainty formats over deterministic forecast formats (Morss et al. 2008). Additionally, some experimental studies (using students) have found that probabilistic forecast information improves decision-making (Joslyn and Nichols 2009; Roulston and Kaplan 2009). Others have noted a possible expansion of the forecaster’s role in interpreting and communicating uncertainty (AMS 2008; Mass 2003; Baars and Mass 2005; NRC 2006) as well as forecaster needs for training in how to assess and communicate uncertainty (Novak et al. 2008). The 2006 “Completing the Forecast” report from the National Research Council (NRC) also suggests that incorporating user needs and effectively communicating the value of uncertainty information is an important task for the weather and climate community (NRC 2006).
Unlike more general forecast products, however, fire practitioners must often use SWFs as a deterministic decision tool, and SWFs play a critical role in deciding if a burn commences as planned. While the consequences of uncertainty or lack of confidence in a forecast can have major implications for a prescribed burn, many fire practitioners do not have the flexibility to integrate uncertainty information into go/no-go decisions—either the conditions are within prescription, or they are not. If an SWF’s values fall outside of a prescribed burn prescription, proceeding with a burn can be risky, with possible outcomes such as losing control of the fire or having negative impacts on achieving the original management objectives of the burn. Although only an extremely small percentage of prescribed burns escape, public outcry can be extremely negative in these circumstances, creating an amplification of the perceived risk for going ahead with a prescribed fire (Kasperson et al. 1988). This potentially creates an affect heuristic, where the risk of a decision is influenced not only by rational or analytic modes of thinking, but also how a decision maker feels about the decision and the possible risk (Slovic et al. 2005). In summary, the use of uncertainty and confidence information in SWFs is a complex issue dependent on a number of different factors and contexts, including how the risks associated with uncertainty and confidence are communicated. Additionally, the ways in which fire practitioners respond to forecast uncertainty and how this impacts overall perceptions of SWF accuracy and utilization are not well understood. The exploratory research reported here presents perspectives and experiences of NWS forecasters and fire practitioners in producing and utilizing SWFs and provides insight into how perceptions of accuracy, uncertainty, and confidence impact SWF utilization.
Section 2 presents the methodology used in our study, including selection of study participants and analysis design. In section 3, we present our findings from the in-depth interviews with focus groups and individuals from the NWS forecast and fire practitioner communities. Last, section 4 explores implications of this research and next steps for consideration in enhancing the utilization and accuracy of SWFs.
This research project used qualitative, in-depth, semistructured interviews with both focus groups and individuals. Focus groups are usually facilitator-led, discussion-based groups that are generally fewer than one dozen people. All institutional review board requirements were met previous to data collection. There were nine focus group interviews with approximately 90 students attending an S-490 course (“Advanced Fire Behavior Calculations”) in three geographical areas of the United States in 2014.1 The S-490 course is attended by firefighters who are selected based on their service record and desire to be qualified as a fire behavior analyst, long-term fire analyst, or prescribed fire burn boss or who are deemed to benefit from completion of the course. Fire behavior and long-term fire analysts collect weather and fuel information to forecast fire behavior and progression that are utilized in tactical and strategic firefighting decisions (National Wildfire Coordinating Group 2013). Prescribed fire burn bosses oversee and conduct prescribed fire planning and operations.
In recruiting for study participants, we coordinated with each S-490 lead instructor in advance to identify the best time during the course to conduct the focus groups interviews. The screening mechanism for the study participants was the requirement to attend or teach the S-490 course; therefore, any participant or course instructor attending the S-490 course was eligible to participate in the study. For each S-490 course, the lead instructor introduced us in the morning, and we briefly described the project and extended an invitation to participate, clearly stating that all participation was voluntary and all responses were confidential. The lead instructor then announced when during the day the opportunity to join a focus group would occur. In each location, there was a minimum of two focus groups, generally with 5–10 participants, but some were much larger, including one with more than 20 participants. Participation ranged from approximately 50%–90% of the S-490 students and sometimes the course instructors (depending on their availability given their teaching responsibilities).
After data collection at the S-490 courses and initial analysis, we held a focus group discussion with NWS forecasters and fire weather/behavior experts. This included an informal presentation describing the themes that had emerged from the initial data coding and a facilitated discussion (see below for a description of analysis methods). This focus group was held at the “Large Wildland Fire Conference” in Missoula, Montana, in 2014. The purpose was to elicit their impressions of the data and identify any gaps in data collection thus far based on their expertise. While this group felt that the data gathered from the fire practitioner community were sufficient, they suggested that an additional set of interviews with the NWS forecast community would be beneficial to understand the data from the fire practitioners attending the S-490 courses. The expert focus group felt this would provide additional context and insight from the operational side of issuing SWFs. Based on their suggestions, we scheduled and conducted a smaller and more focused second round of phone interviews in 2015 with seven NWS forecasters. The geographic distribution of these interviews matched that from the first round of interviews. The forecasters were selected from within their geographic region using the snowball sampling method (Marshall and Rossman 2015).
We prepared a semistructured interview guide for both the fire practitioners and forecasters. Focus groups were generally 30–45 min, and the interviews were generally 45–60 min in length. All of the interviews were recorded, transcribed, and coded using Dedoose (dedoose.com), an online qualitative analysis tool. A basic list of codes was developed after comparing the transcripts from across all the focus groups (and, after the second round of data collection, applied to the interviews with the NWS forecasters and enlarged), using an inductive approach and ethnographic content analysis (Altheide 1987) emphasizing identification of emerging themes from the data, rather than applying a previously developed list of codes (Charmaz 2006; Corbin and Strauss 2014). Ethnographic content analysis is characterized by a highly reflexive and interactive approach to data collection, coding, analysis, and interpretation. Use of this approach contributed to our decision to collaborate with the expert focus group and return to data collection in the project. Unlike other qualitative content analysis methods, ethnographic content analysis is not used to verify hypothesized relationships. While research questions were used to guide this research, we deliberately framed this as exploratory research and did not use hypothesized relationships that might skew our interpretation of emerging themes. There were more than 60 initial codes, or descriptor terms, developed to begin parsing the transcribed interviews and focus groups to identify emerging themes. This list was then refined to 39 codes, including 13 primary codes with 26 subcodes. Emergent themes were used to synthesize data in a second round of coding in thematic groups, which resulted in the following seven thematic areas: physical factors that influence accuracy; turnover/mobility in forecast and fire practitioner communities; communication before, during, and after SWFs are issued; updating SWFs; and confidence and uncertainty in SWFs. Our final analysis focused on developing and expanding relationships between the thematic groups. Throughout, we compared transcripts, coded excerpts, and labels to evaluate the strength of and relationships between different thematic groups (Charmaz 2006; Corbin and Strauss 2014). In the results section below, each individual quoted is given a unique, random numeric code to distinguish between study participants. Quotes from fire practitioners are noted as FP and forecasters as FC, with the numeric code following.
3. Results and discussion
Forecasters and fire practitioners identified many factors that influence the accuracy or perception of accuracy of an SWF. Some of these factors are influenced by the limitations of technology (i.e., models) and data available to forecasters. Others are caused by physical characteristics of the landscape that influence weather and climate at local scales. While this section will briefly cover some of these factors, our focus here is primarily on factors that are rooted in the “social components” of using SWFs. While increasing quantitative forecast skill through model improvements or technical methods is always desired in an operational setting, results from our research suggest that addressing social factors may represent the best opportunity to create substantial improvements in perceptions, accuracy, and successful utilization of SWFs by the fire management community. This section will focus on (section 3a) the challenges with communicating forecast confidence and uncertainty, (section 3b) building and maintaining relationships between forecasters and fire managers, (section 3c) the role of updating SWFs, and (section 3d) geographical limitations of SWFs.
a. Influence of uncertainty and confidence in perceptions of accuracy of spot weather forecasts
In SWFs, similar to all deterministic forecasts, a forecaster’s unexpressed confidence or uncertainty in a forecast can result in users perceiving a forecast as “wrong.” Previous research has found that both the general public and more sophisticated forecast users assume uncertainty in a deterministic forecast (Demuth et al. 2012; Joslyn and Savelli 2010; Morss et al. 2008; Savelli and Joslyn 2012). A major theme that emerged from our research was concern among fire practitioners and forecasters regarding how unexpressed forecast uncertainty and forecaster confidence influence fire practitioners’ perceptions of SWFs as inaccurate. In forecasts, uncertainty refers to lack of precision in the quantitative values of the forecast. Probabilistic forecasts inherently account for uncertainty; but, for SWFs, ranges of values (e.g., high temperature 80°–85°F, minimum relative humidity 28%–33%) represent an indication of uncertainty. Confidence here refers to how assured the forecaster is of the forecast or, for fire practitioners, the extent to which they trust the forecast. Forecaster confidence is based on various factors, including experience, local environmental knowledge, and model guidance performance. Fire practitioners often struggled with understanding how much confidence to have in a forecast, and forecasters struggled with how to communicate uncertainty and their confidence. A challenge with SWFs is that several factors push for a deterministic forecast, with absolute numbers. As one forecaster noted, however, the “NWS is better with trends than exact numbers.” A senior Incident Meteorologist (IMET) forecaster noted the following:
They [fire practitioner] might feel that would be an inaccurate forecast because they’re looking for the hard numbers, [and] they say, “Well this number didn’t occur,” and the forecaster in his mind is like, “Yeah, but I wasn’t quite 100% certain that number would occur.” So the uncertainty, if it’s not conveyed, might come across as a bad forecast when it’s just really more uncertainty than inaccuracy. (FC06)
In the context of understanding fire practitioners’ perceptions of SWF inaccuracy, the limited ability to convey confidence and uncertainty in SWFs (by both forecasters and fire practitioners) likely plays an important role in successful utilization of SWFs. For example, situations with difficult-to-predict conditions, such as terrain-induced winds or dry lightning, likely have contributed to perceptions of SWFs as being inaccurate, or, as one fire practitioner stated, “Ironically . . . the general forecast is usually better than the spot.”(FP05). Conversely, from a forecaster’s perspective, an SWF should be one of the most accurate products from the NWS, as it is one of the few products that is vetted and reviewed by individual forecasters.
Additionally, how an SWF is used as a decision-support tool adds to the challenges in communicating forecast uncertainty or confidence between the forecaster and fire practitioner. As noted previously, many of the fire practitioners we interviewed categorically stated that an SWF was used as a yes–no decision tool: if an SWF were outside the fire prescription parameters, then the decision was made to not burn. Others noted that they used the SWF more as a planning tool. The current utilization of SWFs as a yes–no decision tool necessitates SWFs retaining deterministic forecast information, but this does not strictly preclude including uncertainty information. For example, assigning a probability or some indicator of confidence for the parameter forecasted range could achieve this, or perhaps it could be more nuanced by varying the uncertainty or confidence for individual values or data bins across the forecasted range. However, if SWF protocols and formats were to begin including probabilities, scenarios, or confidence intervals, fire personnel would need the flexibility to include this type of information in their decision process. This would require the fire management agencies more broadly to adopt this change as policy and incorporate it into training.
b. Importance of relationships between forecasters and fire practitioners before, during, and after a spot weather forecast
A prominent theme in interviews with both forecasters and fire practitioners was the need to build direct relationships between the two parties that facilitate communication before, during, and after an SWF is issued. Especially for larger, more complex fire projects, multiday fire projects, or project burns adjacent to populated areas or in sensitive air sheds, this level of communication was widely seen as necessary by more experienced fire practitioners and forecasters in the study. Specifically, the credibility of forecasters (and their products) often appeared to be built on personal relationships between forecasters and fire practitioners. Also, a fire practitioner’s sense of the level of experience and understanding a forecaster had about fire behavior influenced perceptions of credibility:
I think it has a lot to do with who you’re talking to. There may be five meteorologists there, but only one of them may be an IMET or have some kind of fire background. And what they’re giving you, the product of what they’re giving you is good for a picnic or you know for a day on the beach, but they don’t understand the burning part of it. And so if you’re just haphazardly calling, you may get a very good, sound meteorologist but he doesn’t know anything about fire behavior and smoke from a fire. So it’s not necessarily he’s giving you the wrong information; he just doesn’t understand how you’re utilizing that in your day job. (FP10)
The importance of relationships to build trust between the producers of information and the end users and the impact on their perceptions of the credibility, legitimacy, and saliency of science-based outputs is supported by previous research (Brugger et al. 2015; Buizer et al. 2012; Carbone and Dow 2005; Cash et al. 2006; Dilling and Lemos 2011; Jacobs 2002; Jacobs et al. 2005; Kloprogge and Sluijs 2006; Moser 2009; Reed et al. 2014). It was clear from the interviews and focus groups that developing and supporting these relationships through various communication efforts was important. Several fire practitioners emphasized that they made efforts to directly contact forecasters with questions and concerns and to request an SWF several days before a proposed burn to help forecasters become aware of variability in local conditions at the burn site. With few exceptions, the fire practitioners commented that the forecast community frequently asked them for feedback on SWFs. Some fire practitioners demonstrated a good understanding of a forecaster’s need to become familiar with an area and provided observations from a site before the day of the burn. Forecasters also often tried to directly contact fire practitioners when an SWF was issued, if their workload permitted, to better understand needs for the forecast:
When we get a spot request, we call the requesting person pretty much right away to ask, “Is there anything specific you want in the forecast? Is there something that’s more critical than other parameters?” so that we’ve opened up a dialogue. And then we tell them, “Hey, you see anything going wrong, call us.” So we’ve done a lot of outreach here in the state of Florida with our users to get communication. So I think that’s helped out a lot, where 10 years ago we did not have nearly as much communication with the people requesting the spot forecast, and we have much more communication now. (FC01)
This effort at outreach appeared to be a new focus for the NWS WFOs, with forecasters commenting on a shift from producing forecasts to becoming an organization focused on meeting specific customer needs and outreach. Many noted that they now spend time doing outreach with the fire management community in their area. While forecasters agreed that there is likely office-to-office variation on outreach efforts, it was clear from the forecasters and fire practitioners interviewed that both groups felt there was a high level of value in promoting interactions through mutual training opportunities. For forecasters, this meant making the effort to get out on burn days, despite the challenges with scheduling that often result with forecaster shift work. Fire practitioners often spoke of their appreciation of early season talks as well as the willingness of forecasters to attend fire trainings and teach weather components.
As several people noted, however, there are multiple barriers to developing these relationships and achieving a high level of communication before, during, and after an SWF is issued. One forecaster found it difficult to both reach out to new fire practitioners in his area and maintain those relationships:
You get so many people coming [and going], and, to them, we’re kind of a mystery; we’re this mystical black box, and they don’t know how we do things; they don’t know what we need. It might even be intimidating to a certain extent. And I think they think that they’re bothering us. They just don’t know if they’re being a nuisance, and we try to emphasize over and over and over that that’s not the case. We gladly take phone calls; we want that information to do a better job. It’s just hard to bridge that gap. (FC03)
This comment points to a deeper cultural issue of the difference between the forecasting community (highly technical and removed from on-the-ground activities) and the fire practitioner community (highly trained in very specific fire behavior science and enmeshed in the on-the-ground reality of fire in front of them). From these interviews, it was clear that at the community level there was enormous respect for the difficulty of the jobs each does. Understanding the differences and the overlap between these communities, however—how they communicate and the mechanisms that encourage/discourage communication—may be an important factor in increasing successful use of SWFs in prescribed fire programs. Another forecaster noted the challenges with a highly mobile fire community that moves around the United States, as needed, during wildfire season, that may bring new personnel into a forecaster’s region and how this can influence credibility:
The challenge we have is that our customer base changes throughout the course of the summer with people and resources moving around the country. You might have established a good relationship with your local folks, but you’re going to get a bunch of other resources to come in during your peak season, and they don’t know who you are. And they’re going to perceive their relationship with you is a little bit less than what their relationship is with their back-home forecaster that they know because they trust him . . . if they have a good relationship, they’re probably going to give some credibility to you because they think good about the weather service, but you’re still not him. You probably have a little less credibility in their mind. (FC02)
Forecasters also noted that feedback during and after a project was vital for improving SWF accuracy. From the forecasters’ perspectives, however, they infrequently receive feedback on the accuracy of SWFs issued without deliberate follow up on their part. One of the central themes articulated by forecasters was the need for feedback on all SWFs, regardless of their accuracy. Forecasters also noted that while it was nice to get a “good job,” what they really wanted to know was the missteps and how that affected a fire practitioner’s ability to meet their burn project objectives. Forecasters commented that they felt fire practitioners were afraid of criticizing them but noted that they need both negative and positive feedback:
If we never get feedback, we’re thinking we just did a perfect job on them [SWFs], and we know that’s not the case. So especially for ongoing units, ongoing fires, we need that feedback to try to improve . . . . What I’d love to know is how did the forecast—if [it] went awry—how did that negatively impact the project? I think that might scare the forecasters a little bit, but that’s what the forecasters need to know. (FC04)
In summary, a salient, credible SWF starts long before the day it is issued. Efforts at the agency and individual levels to develop lines of communication, relationships between forecasters and fire practitioners, mutual training opportunities, and field trips all contribute to developing these trust relationships. While developing and maintaining these relationships and communication networks is challenging, both fire practitioners and forecasters perceive them as valuable. Future challenges include how to develop and maintain robust communication networks and improve the flow of information after an SWF is issued during a prescribed fire event.
c. Updating SWFs during a prescribed fire event
For those working on the ground at a proposed burn, knowledge of potential changes in forecasted conditions can mean the difference between a successful burn (or the decision to not move ahead) and “smoking out” sensitive, highly populated air sheds up to the loss of property and placing people in unsafe conditions. The mechanism for updating SWFs, however, is often tenuous at best and relies heavily on individuals both at the WFO and on location at a prescribed fire. At the WFO, workload can play a major role in the ability of forecasters to update an SWF, particularly on high-volume days. One forecaster expressed his concern about a WFO’s ability to accurately monitor upward of 20 + SWFs during a single day:
And that’s a huge point . . . while fire weather is probably our biggest program here . . . and a lot of the West, we have a lot of other aspects of weather that we deal with. So we have to load balance all of that. (FC05)
In other situations, when a forecaster receives an SWF update through the online system, it is not necessarily clear why the update is being requested, which is a situation that can cause miscommunication between the forecaster and fire practitioners. One fire practitioner noted that when he asked for an update, sometimes he received the same SWF he had received that morning and was frustrated in not always getting updated weather information:
If we have an ongoing prescribed fire largely and we’re requesting spot forecasts on a daily basis and then it gets back to us that they don’t really want new requests every day and so then they’re just updating the one that was submitted. And so that’s been a conversation locally for us to try and figure out the process they want that works for them, and again it’s not a direct conversation. (FP08)
From the forecaster perspective, however, a request for an update through the online system with no comments can be equally frustrating:
Sometimes, in fact, they will ask for an update and we don’t know why . . . . I try to always get a hold of them and say, “Well, what’s wrong with the first forecast? Are you seeing things evolve differently? I’ll gladly update it if you are, but otherwise I don’t know what to change.” (FC03)
Given data limitations and variations in local-scale conditions, it can be difficult to bridge the perceptual gap between forecasters, often working from more than 100 miles away from a project burn, with the perspective of the fire practitioner, who is dealing with conditions and fire behavior immediately before their eyes. The consequences of not bridging this gap can result in situations ranging from inconvenience to those that threaten property and lives, as one fire practitioner noted:
I’ve been running down the road with a torch, and we have a 180 degree wind shift and it’s running toward the houses. [I’m] trying to light this thing off to catch it, talking to the meteorologist at the same time saying, “What is this?” [forecaster replies] “Oh, that’s an anomaly. It’s not a problem. It should be over in about an hour or two.” (FP11)
In other situations, when forecasters make an attempt to update field personnel on a burn, they have found difficulties in reaching people directly. Just as fire personnel can feel like they are “bothering” a forecaster, forecasters can feel like they are “stating the obvious” to fire personnel on the ground. Additionally, in some regions, it is more common for dispatchers to enter SWF requests, rather than fire practitioners. This adds an additional layer of communication, as well as increases the possibility that error can be introduced into the communication chain:
It would be nice to have a better way to update people who are way out in the field. And we try to do that. We’ll call a dispatch, try to communicate, ask them to call the fire because sometimes we don’t have a direct line of communications with the people out on the incident, but you never know. That dispatcher could get really busy with some other emerging issues as soon as I hang up the phone and that message may possibly not be delivered. So communication, especially when we go through a dispatch office, is a problem. (FC04)
There also is evident frustration in both communities on who is responsible for providing updates to the SWF. From a fire practitioner’s perspective, there was both an acknowledgment and frustration with the updating process and who was responsible for updating an SWF as conditions changed:
There are some guys [forecasters] that will call you and be like, “Look, here’s what I’ve got for you.” But for the most part, these guys are expecting you to call if there’s a problem. And I think this does need to be more of a two-way . . . the responsibility falls on us for this burn. But we have to base everything that we’re putting on ourselves on what they’re giving us. (FP13)
Similarly, one forecaster noted that, while they do their best to keep on top of changing conditions that could affect fire behavior on a project burn, they also expect that fire practitioners and personnel on the ground will keep a “weather eye” out on the conditions and note what is going on. This suggests that there may be implied roles, or a presumed limitation to roles and responsibilities assumed by each community. Similar to what Demuth et al. (2012) termed the hurricane warning system, the NWS WFOs and the fire practitioner community interact within the context of prescribed fires and wildfires. There is overlap between the agency objectives (safety and protecting the general public), and there are also sets of broader organizational constraints and priorities that differ significantly from each other that can hinder communication. Limitations to communication, as one forecaster commented, and the role an SWF can and cannot play in communicating weather information and updates on the day of a burn are key issues to be resolved:
I think communication is the biggest limitation. The spot forecast cannot be seen as the end-all communication tool between the forecasters and the burners—it’s a tool for them to use, but if they want to communicate, they’ve got to reach out. We try too. (FC03)
It seems clear that, while both communities recognize a responsibility for monitoring changes, both are constrained in their ability to do so. Facilitating communication between forecasters and fire practitioners on the ground would likely have a positive impact on improving the timeliness and accuracy of SWF updates during a project burn. Additionally, improving the ability of forecasters and fire practitioners to monitor conditions also could have a strong impact on accuracy. Both forecasters and fire practitioners indicated that improving fire personnel’s ability to make accurate, timely, onsite observations to feed back into SWF updates would further improve accuracy.
d. Geographical limitations of SWFs on accuracy
Inaccuracy in SWFs due to geographical limitations was also a strong trend identified in the interview and focus groups. For example, forecasters and fire practitioners often noted the difficulty in forecasting local-scale conditions for large areas as well as challenges in understanding local-scale wind patterns and behavior. Also, forecasters often were frustrated by the lack of Remote Automatic Weather Stations (RAWS) data availability plus onsite observational data. Like fire practitioners, the forecasters seem to recognize this as a limitation that is not likely to be improved anytime soon. Issues of RAWS siting came up repeatedly throughout the interviews for this project, especially in the western United States, where RAWS stations often “cover” terrain that has widely different elevations, orientations, and/or fuel types. The challenges for selecting a representative RAWS were noted by both forecasters and fire practitioners. This is also documented in a recent National Wildfire Coordinating Group report (Brown et al. 2011). Several commented on use of portable RAWS units as a way to augment the RAWS network and improve SWF accuracy. After several near misses with undesirable smoke effects, one southeastern fire practitioner noted that his district had purchased several incident RAWS (IRAWS) units to deploy on burn days in sensitive air sheds.
The challenge for the fire community, however, often seems to lie in the narrow window of parameters that controls the decision to begin a prescribed fire. When asked what parameters were most likely to be inaccurate in an SWF, a fire practitioner replied as follows:
Wind direction, wind speed. And, like I said, it’s not very dramatic, but often these prescription parameters are so tight, four points2 make a difference to us. (FP20)
The quantitative component of our research supports this observation. Lammers and Horel (2014) showed higher quantitative forecast skill associated with SWFs than the National Digital Forecast Database (NDFD) output. In SWFs, forecasts of maximum temperature represented the largest improvement in forecasted conditions compared to observed conditions; maximum wind speed exhibited the least improvement. Nauslar et al. (2015) showed that SWFs of transport winds and Haines index exhibit relatively small mean and median absolute errors, but mixing-height SWFs exhibited large errors. In response to these types of forecast inaccuracies, fire practitioners make their own adjustments to preserve flexibility in prescription parameters. For example, one fire practitioner commented that they have found the forecast for 20-ft winds (above the ground) to be more accurate than eye-level winds; in consequence, they now use this forecast element in the prescribed fire plan, ask for it in their SWFs, and wait to calculate the eye-level wind on site the day of the prescribed fire. This alteration allows them to reduce their dependence on a potentially inaccurate element of the forecast and maintain flexibility in their window to proceed with a prescribed fire. This type of user adjustment is similar to findings in how other high-stakes forecast users adjusted for perceived biases in a deterministic forecast (Savelli and Joslyn 2012).
4. Summary and recommendations
Results of our research support the notion that issues surrounding communication between forecasters and fire practitioners and updating SWFs are equally important in both perceptions of forecast accuracy and quantitative accuracy. When an SWF was issued, the communication that took place before the forecast request between the WFO, forecasters, and fire practitioner community was often an essential precursor to the user’s perception of the forecast as credible and salient. Firefighters often work seasonally, changing crews and moving to new areas frequently, and move around during the fire season, depending on resource needs. While individual relationships are undoubtedly important, developing and maintaining a formal structure for mutual training and outreach could greatly enhance the ability for these relationships to form each fire season between new individuals while, at the same time, maintaining previous relationships.
During a prescribed fire, the primary challenges are centered around barriers to successfully updating an SWF: recognizing the need for an update, communicating that need successfully, and receiving the update in a timely way. Specifically, these exploratory research results suggest that the process of monitoring weather conditions and communicating changes, as two-way communication between the WFO and field personnel, is limited without well-defined protocols and mechanisms. In addition, there are workload issues within the WFOs’ as well as forecasters’ abilities to effectively monitor SWFs on high-volume days.
After a prescribed fire, or on multiday burns, providing feedback on SWF accuracy faces several barriers as well. Fire practitioners—already burdened with postburn obligations, such as monitoring, releasing resources, equipment cleaning, and paperwork—are often not incentivized to provide feedback to forecasters. This may be especially true if the forecast was accurate, but as several forecasters noted, they need feedback for both inaccurate and accurate SWFs to effectively improve accuracy. Fire practitioners also may be reluctant to “criticize” forecasters for what seem like small errors but, cumulatively, may have a perceptible influence on quantitative accuracy.
These communication barriers also may influence how SWF uncertainty and confidence can be expressed. As noted by previous researchers, “effectively providing and using forecast uncertainty information is not a simple problem with a single solution” (Demuth et al. 2012, p. 1617). Particularly with SWFs, there is the need for deterministic information in how they are often currently utilized. SWFs are just one of the factors in the prescribed burn checklist but can often solely influence a yes–no burn decision. Including uncertainty and confidence information in SWFs, however, could potentially increase confidence in SWFs, improving perceptions of accuracy and allowing fire practitioners to more effectively utilize SWFs in the array of information they consider in making decisions. Additionally, there is a paradigm shift within the NWS and weather and fire communities to embrace and utilize probabilistic forecasts as the foundation for weather and fire potential forecasting moving forward (Karstens et al. 2015; Rothfusz et al. 2014). One of the challenges will be understanding how the depth and quantity of information produced by probabilistic forecasts can be effectively communicated to forecast users, requiring greater inquiry into how the information in SWFs is interpreted by fire practitioners (Demeritt et al. 2010; Morrow et al. 2015) and also how risk can be more effectively communicated (Lazo 2012).
If the effort to incorporate forecast uncertainty and confidence information into SWFs is piloted, we suggest that these efforts should be systematically evaluated, using both qualitative and quantitative methods to collect data during and after forecast creation and use. A systems-based evaluation approach (Hargreaves 2010) would allow forecasters to adjust how they communicate uncertainty and confidence while also meeting user needs for deterministic forecasts to meet fire plan prescription parameters. For example, the Joint Fire Science Program (JFSP) has recently funded projects to examine weather data in the context of decision-making that could be applicable in developing a systematic evaluation process for SWFs.
We gratefully thank the fire practitioners and NWS forecasters who graciously offered their time and thoughts about SWFs and the group of fire behavior experts who offered their comments throughout the project. The Joint Fire Science Program under Grant 12-1-05-3 provided funding for this project, and the National Oceanic and Atmospheric Administration’s Climate Program Office supported the research through Grant NA11OAR4310150, along with the California Nevada Applications Program at the Desert Research Institute.
Because of the limited number of S-490 courses held each year, to ensure confidentiality, the geographic areas selected for this study are not disclosed.
“Points” in this quote refers to miles per hour (i.e., an SWF with a range of 12–16-mph winds).