Wildland fire managers in the United States currently utilize the gridded forecasts from the National Digital Forecast Database (NDFD) to make fire behavior predictions across complex landscapes during large wildfires. However, little is known about the NDFDs performance in remote locations with complex topography for weather variables important for fire behavior prediction, including air temperature, relative humidity, and wind speed. In this study NDFD forecasts for calendar year 2015 were evaluated in fire-prone locations across the conterminous United States during periods with the potential for active fire spread using the model performance statistics of root-mean-square error (RMSE), mean fractional bias (MFB), and mean bias error (MBE). Results indicated that NDFD forecasts of air temperature and relative humidity performed well with RMSEs of about 2°C and 10%–11%, respectively. However, wind speed was increasingly underpredicted when observed wind speeds exceeded about 4 m s−1, with MFB and MBE values of approximately −15% and −0.5 m s−1, respectively. The importance of accurate wind speed forecasts in terms of fire behavior prediction was confirmed, and the forecast accuracies needed to achieve “good” surface head fire rate-of-spread predictions were estimated as ±20%–30% of the observed wind speed. Weather station location, the specific forecast office, and terrain complexity had the largest impacts on wind speed forecast error, although the relatively low variance explained by the model (~37%) suggests that other variables are likely to be important. Based on these results it is suggested that wildland fire managers should use caution when utilizing the NDFD wind speed forecasts if high wind speed events are anticipated.
The National Digital Forecast Database (NDFD) is a seamless mosaic of gridded forecasts produced by the National Weather Service (NWS) for public use and national preparedness (Glahn and Ruth 2003). Local forecasts of sensible weather variables are compiled at each Weather Forecast Office (WFO) based on numerical weather prediction (NWP) models, observations, and forecaster experience; the forecasts are then stitched together at the national level to produce the NDFD. Wildland fire managers rely on the NDFD to provide accurate and timely forecasts for weather variables that are known to affect both short- and long-term fire behavior and fire danger including air temperature, relative humidity, and wind speed and direction (Jolly 2009; Burgan et al. 1997). In particular, decision-makers on large wildland fires utilize the NDFD forecasts to aid in fire spread projections that guide strategic decisions and inform tactical operations, which ultimately affect private and public resources (Calkin et al. 2011).
Wildland fire behavior is primarily influenced by the local fire environment, which includes the fuel, weather, and topography in the area adjacent to the fire (Countryman 1966). Fuel and topography are usually considered static on the time scales relevant for fire behavior prediction but weather is both highly variable and dynamic (Barrows 1951). Several weather variables at small and large scales can affect the dynamics of fire behavior, but near-surface air temperature and relative humidity, through its effects on dead-fuel moisture, and wind speed and direction typically have the largest impacts on fire rate of spread and intensity (Cheney et al. 1993; Rothermel 1972). For example, fuel moisture tends to dampen fire spread as a result of the high specific heat of water (Anderson 1969; Byram et al. 1952), and the local wind field (Liu et al. 2015a; Sanderlin and Sunderson 1975) enhances forward fire spread and intensity by increasing both the rate of combustion and by directing hot combustion products toward unburned fuels (Catchpole et al. 1998; Albini 1982).
Near-surface wind speed and direction are affected by terrain (Wakes et al. 2010) and vegetation (Linn et al. 2013) through mechanisms such as channeling or sheltering. Local and large-scale variations in terrain shape, orientation, and complexity can result in wind flows through valleys that can override and/or enhance synoptic winds (Weber and Kaufmann 1998). Likewise, vegetation type and size can alter the magnitude of the wind flow near the surface as a result of the effects of bulk drag from crown foliage (Albini and Baughman 1979). Wildland fires often occur in rugged terrain across a variety of vegetation types, where near-surface wind, temperature, and moisture vary substantially over space and time. Wagenbrenner et al. (2016) reported that operational NWP models with horizontal grid sizes larger than 1 km are too coarse to predict the variability in near-surface winds exhibited in complex terrain. The finest NDFD grid currently available has a horizontal resolution of 2.5 km and, thus, would not be expected to completely capture the range of variability in near-surface weather in complex terrain. Higher-resolution modeling in complex terrain may improve predictions in some cases, but traditional NWP modeling (e.g., with WRF), upon which the NDFD is at least partially based, is limited to about 1-km horizontal grid resolution because of limitations with discretization schemes over steep slopes (e.g., Lundquist et al. 2010) and the planetary boundary layer schemes used for turbulence closure (Wyngaard 2004).
Previous verification studies of the NDFD have shown that it generally produces accurate forecasts of air temperature and precipitation for a variety of lead times (Huntemann et al. 2015; Myrick and Horel 2006; Dallavalle and Dagostaro 2004). However, the NDFD’s suitability for wildland fire applications has not been directly assessed. The routine verification procedures currently in use for the NDFD (available online at http://www.mdl.nws.noaa.gov/~verification/ndfd/) tend to be weighted toward observations near urban centers (e.g., Ruth et al. 2009), as opposed to remote regions of complex terrain where wildfires often occur. Additionally, the averaging times used to compute model error statistics (monthly) are large compared to time scales important for wildland fire [from minutes to hours; e.g., Dagostaro et al. (2004)]. Large averaging periods effectively smooth out errors during rare, extreme events that, arguably, are the most critical in terms of wildland fire behavior.
Despite the lack of NDFD evaluation in fire-prone regions, it is used to support wildland fire decision-making. The Wildland Fire Decision Support System (WFDSS; Noonan-Wright et al. 2011) is the suggested decision support tool for large wildland fires in the United States (NWCG 2009). WFDSS utilizes NDFD grids of air temperature, relative humidity (RH), precipitation, cloud cover, wind speed, and wind direction to aid in short- and near-term fire behavior predictions (i.e., 1–7 days). The forecast grids are used in conjunction with a number of fire behavior models to estimate rate of spread, spread direction, and intensity across diverse assemblages of topography and fuels. NWS spot weather forecasts are also requested by fire managers when time allows. These spot weather forecasts are based, at least in part, on the NDFD, but are typically expected to be more accurate than raw NDFD predictions since forecaster expert knowledge of the local and regional terrain and meteorological conditions can be taken into account. However, the accuracy of spot weather forecasts is not well known and while requests for spot weather forecasts are encouraged, time constraints do not always allow for them.
This work provides an evaluation of the NDFD for regions in the conterminous United States (CONUS) susceptible to wildland fire and quantifies the associated uncertainty in predicted fire behavior. The specific objectives were to 1) quantify the error in forecasted weather variables (air temperature, relative humidity, and wind speed) used for predicting surface head fire rate of spread, 2) estimate and rank the importance of NDFD forecast error for individual weather elements on surface head fire rate-of-spread error, 3) estimate the wind speed forecast accuracy needed to achieve “good” surface head fire rate-of-spread predictions, and 4) determine the geographic, meteorological, and human factors that have the greatest influence on NDFD wind speed forecast error.
2. Data and methods
Observed weather from surface weather stations across the CONUS and hourly NDFD forecasts from each observation station location for calendar year 2015 were compiled and merged with the relevant fire environment variables required to make fire behavior predictions (Table 1). These data were used as inputs into a semiempirical fire behavior model developed by Rothermel (1972) and known as the Rothermel model, which is applied throughout the United States for wildland fire management (Andrews 2014). Potential fire behavior was predicted using both the observed and forecasted data and analyzed to determine fire behavior model sensitivity to forecast error (forecast − observed) for the relevant weather variables. Based on those results, the forecast accuracies needed to achieve good rate-of-spread predictions were estimated. Additionally, the impact of several geographic, meteorological, and human factors on NDFD forecast error were assessed for the variable that had the largest impact on predicted fire behavior.
b. Observed data
Hourly weather observations from across the CONUS were obtained from available Remote Automated Weather Stations (RAWS; available online at http://www.raws.dri.edu) and Automated Surface Observing System (ASOS) weather stations (available online at https://mesonet.agron.iastate.edu/request/download.phtml) for calendar year 2015. The weather station variables analyzed were air temperature, RH, 6-m (20 ft) wind speed, precipitation, cloud cover, and solar radiation. Note that in keeping with standard fire weather terminology we will refer to the 6-m wind speed as the 20-ft wind speed throughout the rest of the paper. Station transmit times (UTC) were rounded to the nearest hour and units were converted to SI units. Wind speeds from ASOS stations (10-m height) were converted to 20-ft height assuming a logarithmic wind profile, neutral atmospheric stability, and roughness lengths of 0.01, 0.43, and 1.0 m for grass, brush, and timber fuel types, respectively (Campbell and Norman 1998). Fuel type was assigned based on each station’s fuel model (see section 2d) following Scott and Burgan (2005). Solar radiation values for ASOS stations were estimated using the Solar Position and Intensity (SOLPOS) algorithm (National Renewable Energy Laboratory 2000) with corrections for cloud cover following the procedures used to estimate the state of the weather in the Weather Information Management System (NWCG 2003).
Partial or incomplete hourly weather observations were removed, and quality control following CEFA (2007) was undertaken with one exception; precipitation was allowed to exceed 51 mm in any single hour but was constrained to the all-time record for a 24-h period for the state in which the station was located. Additionally, observations were removed if the hourly change in air temperature exceeded 30°C. The climate extremes used to identify individual state thresholds for data removal for air temperature, RH, wind speed, and precipitation were obtained online [National Centers for Environmental Information (NCEI); available online at https://www.ncdc.noaa.gov/extremes/scec/records]. The quality control procedures resulted in the removal of approximately 5% and 20% of the ASOS and RAWS data, respectively.
c. Forecast data
The archived NDFD forecasts that were issued every hour during calendar year 2015 were obtained online (NCEI; available online at https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/national-digital-forecast-database-ndfd). NDFD forecasts are available out to 168 h and can be updated at the discretion of the WFO. Since there can be multiple forecasts valid for a given hour, in this work we only considered forecasts with a 1-h lead time (i.e., the first time step in the forecast). Since forecast skill typically decreases with forecast lead time, the 1-h lead time was chosen to provide a best-case assessment of the NDFD. The forecasted air temperature, RH, cloud cover, precipitation, and wind speed were extracted from the grid point nearest to each station’s location and converted to SI units. The 10-m wind speed was transformed to a 20-ft wind speed assuming a logarithmic profile, neutral atmospheric stability, and a roughness length corresponding to the fuel type at the station location. Solar radiation was calculated for each hour using SOLPOS with corrections for cloud cover obtained from the NDFD forecast grids.
The 6-h quantitative precipitation forecast was converted to an hourly precipitation forecast by dividing the total forecast amount into equal proportions for each hour within the forecast period. Although it is unlikely that precipitation would be evenly distributed over the 6-h time frame, this simplification is expected to have minimal effect on the results on average. Precipitation events that occur later in the 6-h window will have little impact as the time available for drying after the event is the same, but precipitation events that occur early in the 6-h window may result in higher than expected dead-fuel moisture levels, and thus lower fire behavior, as there would be less time available for drying and dead-fuel moisture to decrease after the event. As there is no basis to expect precipitation events to happen early or later in the 6-h window, the net effect is likely negligible.
d. Fuel and topography
Additional fuel and topographic information needed to estimate dead-fuel moisture and run Rothermel’s model were obtained from the LANDFIRE project (LF 1.3.0; Rollins 2009). Specifically, the aspect, slope, elevation, fire behavior fuel model (Scott and Burgan 2005), canopy height, and canopy cover were extracted from the grid cell in which each station was located. Those stations that were located on a nonburnable fuel model were removed from the analysis. Additionally, the fuel models extracted at each station location were aggregated into two fuel type categories, timber (timber understory, timber litter, and slash blowdown) and nontimber (grass, grass–shrub, and shrub) to assess the sensitivity of predicted fire behavior in horizontal versus vertically oriented fuel beds to weather forecast error.
Dead-fuel moistures for the 1-, 10-, and 100-h fuel size classes (Fosberg 1970) were estimated using an adaptation of the Nelson (2000) dead-fuel moisture model, which computes a fuel moisture content (% oven-dry weight) given inputs of air temperature, RH, solar radiation, and cumulative rainfall. The dead-fuel moisture model is a bookkeeping-type model (Viney 1991) that estimates values with partial dependence on previous predictions. To accommodate this, the relevant weather data were organized by station, sorted by date and time, and processed with an initial starting fuel temperature of 20°C and 5% moisture content. The times required for dead-fuel moisture contents to stabilize to the environmental conditions vary by size class but are on the order of from hours to days for the smallest dead-fuel size classes that drive fire behavior (Rothermel 1983). Thus, the impact of the starting moisture contents on predicted fire behavior is expected to be minimal, particularly for those stations located in the northern tier of the United States because we excluded observations prior to 1 April (see section 2g). Live herbaceous and woody fuel moisture content levels for all observations were set to 30% and 60%, respectively, as these values represent typical fire season conditions when herbaceous fuels have cured and live woody material is entering dormancy (Rothermel 1983; Scott and Burgan 2005).
e. Fire behavior
The fuel and topographic information, combined with an estimate of midflame wind speed, were used to calculate the potential fire behavior for each set of hourly observations (observed and forecast). Midflame wind speed was estimated from the observed and forecasted 20-ft wind speed following Finney (2004) and Andrews (2012), with canopy cover less than 5% classified as unsheltered. The surface fire rate of spread in the heading direction only (i.e., parallel to slope and wind) was extracted from the fire behavior outputs and used to conduct the analysis. The surface fire rate of spread describes the speed of the flame front in surface fuels that are typically within about 1.8 m of the ground (Rothermel 1983). This includes grasses, shrubs, and timber litter but not the crown fuels associated with a timbered overstory. Potential crown fire behavior was excluded from the current analysis in order to avoid confounding the influence of the weather variables on two different fire behavior models. Additionally, the rate of spread was the primary focus of the current evaluation because it is the fire behavior characteristic that is most directly correlated with near-surface weather and is an important variable for wildland fire managers.
f. Explanatory variables
The effect of several explanatory variables (including the weather, fuel, and topographic variables already described) on forecast error was investigated (Table 2). A 30-m digital elevation model (DEM) for the CONUS was used to determine the landform (10 levels) for each station based on the topographic position index (Jenness 2006). Terrain complexity was characterized using the elevation range and the standard deviation of the elevation calculated within a 2.5 km × 2.5 km window centered on each station location (Santos-Alamillos et al. 2013). Aspect was transformed to a linear range between 0 and 2 following Beers et al. (1966), and distances to the coast and nearest NDFD grid point were calculated within a geographical information system. The distance between each weather station and the nearest city with a population greater than 50 000 was also determined based on the 2010 U.S. Census (available online at https://www.census.gov/geo/maps-data/data/tiger.html). The difference in elevation between the nearest NDFD grid point and the weather station was also included as a potential explanatory variable.
Additional explanatory variables were station type (ASOS vs RAWS), month (1–12), and hour (0–23), where hour was transformed from UTC to local time based on station location. The Weather Forecast Office (116 levels) and the Geographic Area Coordination Center (GACC; 9 levels) that each station was located within were included to account for differences among forecast methods and regional differences in fire weather. To incorporate the effect of high wind speed events on forecast error, the observed wind speed at each hour was classified into four categories (m s−1); low (0–2), moderate (2–6), high (6–12), and very high (>12).
All variables were merged into a database consisting of observed and forecasted weather, topography, fuel, and resulting fire behavior for each date and time present in the observed dataset, (i.e., forecast data were removed where observed values were missing), which was used to calculate various error statistics. Forecast error for individual hourly observations was assessed using raw error (predicted − observed) and for all observations using root-mean-square error (RMSE) and mean bias error (MBE) following Willmott (1982),
as well as mean fractional bias (MFB) following Boylan and Russell (2006),
where is the predicted value for the ith observation, is the observed value for the ith observation, and n is the total number of observations. Additionally, hourly rate of spread was calculated as the percent error and as a modified z score (raw error/standard deviation of error by station) to account for differences in fuel-type sensitivity to wind speed. Wind speed error was also calculated on a percent error basis to aid in the determination of the wind speed accuracies needed for good rate-of-spread predictions. To incorporate the potential influence of RH error on head fire rate of spread, at each time step seven separate variables were calculated; the raw error was recorded for each of the previous 5 h, including the current hour and the sum of the error over the same time period.
The rate-of-spread predictions were also classified as good or bad following the recommendations of Cruz and Alexander (2013). Specifically, based on an analysis of several fire spread model evaluation datasets for seven fuel-type groups, they determined that ±35% error of head fire rate of spread was a reasonable standard for fire behavior model adequacy (Cruz and Alexander 2013). Rate-of-spread predictions that were within ±35% error were classified as good while all others were classified as bad.
Random forests (Breiman 2001) as implemented in the randomForestSRC package (Ishwaran and Kogalur 2016) in the R statistical package, version 3.3.2 (R Core Team 2015), were used for both classification and regression. Random forests are a nonparametric ensemble learning approach to data analysis that does not have distributional assumptions, can utilize different types of data (e.g., continuous, ordinal) simultaneously, is robust to outliers, can incorporate complex interactions in high-dimensional data, and performs well with spatial data (Evans et al. 2011). The dependent variables were either regressed or classified against the explanatory variables and outputs of variable importance and partial dependence were evaluated. Specifically, to address objective 2, rate-of-spread error (modified z score) was regressed against forecast error in the weather station variables to assess the relative importance of individual sensible weather variables on rate-of-spread error. To address objective 3, the classified rate-of-spread predictions were regressed against forecast error in the sensible weather variables to evaluate the dependence of good and bad rate-of-spread predictions on wind speed forecast error. Objective 4 was addressed by regressing the wind speed forecast error, for all the data and just the cases where the observed wind speed was ≥ 4 m s−1, against the full set of explanatory variables listed in Table 2.
The analysis focused on periods of active fire spread; therefore, only those hours where head fire rate of spread based on the observed data was ≥0.0056 m s−1 [i.e., 1 ch h−1; a chain (ch) is a standard unit of length used in wildland fire; 1 ch = 66 ft] were analyzed. This spread-rate threshold allows for the evaluation to focus on conditions under which wildfires typically occur, that is, conditions that promote active fire behavior that requires fire suppression. Additionally, the hourly observations were limited to periods considered to be during the potential fire season to minimize bias due to snow cover. Specifically, those stations located in the northern GACCs (Fig. 1) (i.e., Northwest, North Ops, Northern Rockies, Great Basin, Rocky Mountain, and Eastern) were limited to observations from 1 April to 1 November, while those in the southern tier of the CONUS (i.e., South Ops, Southwest, and Southern) were not constrained by time of year. Although snow cover can exist between 1 April and 1 November in the northern tier of the CONUS and during the winter in the southern tier of the CONUS, it was preferable to keep the time horizon wide so as to include a larger number of potential fire spread events. This assumption likely resulted in the inclusion of data where fire spread would not be possible (i.e., snow is covering the surface fuel); however, this should have little or no impact on the final results as the underlying dependence of the fire behavior model on NDFD forecast error remains unchanged. These constraints resulted in approximately 5.3 million hourly observations from 1217 RAWS and 807 ASOS stations (Fig. 1). The data were further constrained to analyze NDFD performance when the observed 20-ft wind speed was ≥4 m s−1 (approximately 1.5 million observations).
For each random forests analysis approximately 130 000 hourly observations were randomly selected (with replacement) from the full dataset to construct the training and testing datasets. Initial sensitivity testing indicated that results were unchanged with larger sample sizes. The number of “trees grown per forest” was set to 100 for all analyses as the out-of-bag prediction error did not substantially decrease with additional trees (Oshiro et al. 2012). Additionally, variable importance was calculated using the Breiman–Cutler permutation (Breiman 2001), and the number of variables randomly selected at each node split was set to p/3 for regression and sqrt(p) for classification, where p is the number of variables. Model validation was completed by assessing the variance explained and the overall error rate for regression and classification, respectively, using both the out-of-bag samples and the test dataset.
a. General NDFD forecast skill
NDFD forecasts for air temperature and RH from both datasets (all and wind ≥4 m s−1) produced RMSEs of about 2°C and 10%–11%, respectively (Table 3). Overall, the forecast bias during hours with significant fire spread was toward slightly lower temperatures and higher RH values than observed, but no apparent trend in bias was evident when viewed over the entire range of values in the dataset (Fig. 2). This skill in forecasting air temperature and RH resulted in dead-fuel moisture contents with RMSEs between 7% and 17%, with the greatest error associated with the smallest dead-fuel size class. Overall, the bias in dead-fuel moisture content was toward wetter fuels (i.e., higher fuel moistures).
The NDFD generally overpredicted wind speed when considering all the data, with an RMSE and MBE of 1.9 and 0.5 m s−1; however, the data indicated a significant and increasing underprediction bias when the observed wind speed was ≥4 m s−1, with an RMSE and an MBE of 2.1 and −0.5 m s−1, respectively (Table 3). Generally, as the observed wind speed increased, the underprediction bias in predicted wind speed also increased (Fig. 3).
b. Relative importance of forecast error on head fire rate of spread
Surface head fire rate-of-spread error was most strongly affected by the error in forecasted wind speed (Fig. 4). The random forests model explained 42% of the variance in the modified surface rate-of-spread z score with the wind speed error having more importance than the next most important variable, the sum of the error in RH during the previous 5 h (Table 4). The error in forecasted RH was slightly more important in the immediately preceding hours (1–4) than in either the current hour or 5 h previous.
c. Accuracies for good surface head fire rate-of-spread predictions
Classification of good and bad surface head fire rate-of-spread predictions using the Cruz and Alexander (2013) threshold of ±35% error resulted in an overall classification error rate of 28% (Table 4). The 20-ft wind speed forecast accuracy needed to achieve a good surface head fire rate-of-spread prediction varied according to desired prediction accuracy (Fig. 5). At the 50% probability threshold, the Cruz and Alexander (2013) method required wind speed accuracies of approximately ±20%–30% of the observed value.
d. Wind speed forecast error
Regression of the 20-ft wind speed error against the full set of topographic, fuel, and miscellaneous variables produced a model that explained 37% of the total variance (Table 4). The most important variable affecting NDFD wind speed forecast accuracy was wind type; that is, as the observed wind speed increased, the 20-ft wind speed error also increased (Fig. 6). The WFO and GACC were also important variables with the most pronounced differences apparent under wind speeds that exceeded 12 m s−1 (Fig. 7). For example, some WFOs and GACCs tended to have poorer high wind speed forecasts than other WFOs or GACCs (Fig. 8). When only wind speed events where the observed wind was ≥4 m s−1 were considered, there was a decrease in the total variance explained (27%) (Table 4), with the WFO and GACC variables becoming most important. However, the importance of the terrain complexity variables increased. That is, as terrain complexity increased (elevation standard deviation or elevation range), the underprediction bias in wind speed forecast error also increased (Fig. 9). The other explanatory variables had a relatively minor influence on the wind speed forecast error and are not discussed further.
a. NDFD forecasts and fire behavior
The 1-h lead-time forecasts generally produced accurate air temperature and RH values during 2015. Similar to Myrick and Horel (2006), who focused on the western United States during the 2003/04 winter season, we found that the average forecast error for air temperature was about 2°C. Within the context of the current study, the effect of accurate air temperature and RH forecasts on fire behavior is through higher precision in the modeled dead-fuel moisture content. Dead fuels, particularly fine dead fuels (particle diameter < 6 mm), readily exchange moisture with the atmosphere and reach equilibrium with the environment over time scales dependent upon particle size (Fosberg and Deeming 1971). The time required to reach equilibrium is generally on the order of hours, even for the smallest dead-fuel particles, which delays the impact of changing environmental conditions on dead-fuel moisture content (Britton et al. 1973; Fosberg and Deeming 1971). The importance of this delayed impact was confirmed in our analysis that suggested it was the cumulative effect of RH forecast error that was more important to the rate-of-spread predictions than the forecast error in any single hour. This dependency on a relatively long time horizon makes it difficult to define guidelines for acceptable forecast accuracy as the relative importance of time on RH forecast error is not clear (i.e., the forecast error in some preceding hours is more important than others).
Across the CONUS the NDFD in 2015 tended to overpredict the 20-ft wind speed; although, approximately 70% of the data occurred when the observed wind was <4 m s−1. When the dataset was subset to periods when the observed wind speed was ≥4 m s−1, the wind speed forecasts became increasingly biased toward underprediction. This is similar to previous work by Zhu and Pi (2014), who also found that the NWS underpredicted wind speeds when observed winds were greater than about 8.9 m s−1, based on a historical analysis of weather forecasts for 60 metropolitan areas across the CONUS. In terms of the NDFD and its current use in the wildland fire community, an overprediction of wind speed is more acceptable than an underprediction as fire-spread projections are more likely to be conservative (i.e., underestimating time of arrival). This suggests that the NDFD wind speed forecasts are suitable for making surface fire rate-of-spread predictions under most conditions. However, because of the underprediction bias during high wind speeds, caution should be used when weather events that are associated with high wind speeds are anticipated, such as during thunderstorms and cold-front passage, as this will result in fire spread predictions that overestimate the time of arrival. This point is particularly important to fire managers because wildfire case studies and fatality investigation reports have frequently recognized the links between high wind speed events and large fire growth (Butler and Reynolds 1997; Graham et al. 2011) and firefighter fatalities (Alexander et al. 2015).
The strong link between rate of spread and wind speed found in the present study was anticipated as wind has long been known to be an important factor in wildland fire spread. Previous sensitivity analyses of the Rothermel model using Monte Carlo–based methods (Jimenez et al. 2008; Liu et al. 2015b) or global sensitivity analyses (Liu et al. 2015a) have demonstrated that the predicted rate of spread is highly sensitive to wind speed, which varies by fuel model, particularly for fuel models associated with horizontally oriented fuel beds such as timber litter (Rothermel 1972; Catchpole et al. 1993). Fire behavior model sensitivity to wind is desirable as wind is known to play a dominant role in convective heat transfer (Cheney et al. 1998; Frankman et al. 2013) and tends to be significantly correlated with the rate of spread obtained from field measurements (Cheney et al. 1993; Cruz et al. 2013).
Based on the present analysis, we propose that wind speed forecasts should strive for accuracies within ±20%–30% of the observed value, as this is the window when the predicted rate of spread is most likely to be within ±35% error. This desired forecast accuracy is specific to the Rothermel model and is considered most applicable at the national scale as the analysis was focused on the average effect of wind speed error across multiple fuel types and fuel models. It is expected that a wider wind speed accuracy window would be acceptable for predicting rate of spread in fuel models associated with horizontally oriented fuel beds or for nonhead fire spread directions. An artifact of the presentation of the desired forecast accuracy as a percentage of the observed wind speed is that it implies that increasing wind speed forecast accuracy, in terms of the actual value, is required at lower observed wind speeds. The reality of wildland fire behavior prediction is that the accuracy of rate-of-spread predictions under low wind conditions is less important than at high wind speeds. Thus, emphasis should be placed on achieving the forecast accuracy of ±20%–30% during higher wind speed conditions.
b. Wind speed forecast error
Analysis of the underlying factors controlling wind speed forecast error within the NDFD indicated that several variables could be important. The location of the forecast, in terms of the WFO and GACC where it originated, was shown to be one of the more important factors affecting the wind speed forecasts. This was similar to the findings of Zhu and Pi (2014), who also concluded that forecast accuracies for areas across the CONUS were sensitive to geographic location. Assessment of the differences in wind speed forecast error between individual WFOs and GACCs is beyond the scope of the current study. However, there are potentially a number of factors that could be related to these differences, including small-scale weather station location/placement issues and unusual weather activity for a particular region in 2015. Additionally, individual forecaster intervention or WFO-specific methodologies for interpreting and interpolating NWP model output could help explain why some WFOs produced more accurate high wind speed forecasts than others.
As demonstrated in previous studies (i.e., Wagenbrenner et al. 2016), terrain complexity was identified as a significant contributor to wind speed forecast error. Unresolved terrain complexity hinders the ability of models to capture and incorporate important terrain influences on wind speed (Butler et al. 2015), although promising techniques do exist that can be utilized to help downscale coarse-grid wind predictions to incorporate important terrain effects (Wagenbrenner et al. 2016; Forthofer et al. 2014). Despite the factors described above, it should be noted that the current analysis of wind speed forecast error explained a relatively minor proportion of the total variability. Additional variables such as the presence of short-lived atmospheric boundaries and/or a more detailed analysis with a smaller subset of weather stations could likely significantly improve our understanding of the factors controlling wind speed forecast error.
The NDFD is an important tool for wildland fire managers in the United States as it allows them to predict fire behavior and subsequently assess its impacts to firefighters and affected communities. The evaluation during periods of active fire spread and in fire-prone locations across the CONUS during 2015 revealed that the NDFD is capable of producing accurate air temperature and RH forecasts, which is manifested in increased precision in estimates of dead-fuel moisture. Additionally, on the whole the NDFD produces wind speed forecasts that are conservative in nature (i.e., overpredict wind speed). However, when wind speed exceeds approximately 4 m s−1, the NDFD forecasts display an increasing underprediction bias. This underprediction during high wind speeds is critical as those are the times known to coincide with rapid fire rates of spread and large fire growth. The underlying causes of the wind speed forecast error remain largely unknown, but appear to be related to spatial location, in terms of specific WFO and geographic area, and terrain complexity. Engagement of the wildland fire community by NWS forecasters on the effective use of the NDFD wind speed forecasts for fire behavior prediction, including applying bias corrections or working within an envelope of expected outcomes, will likely help facilitate better predictions and address potential concerns. Future evaluation of the NDFD should focus on determining the underlying source of the underprediction bias in the high wind speed forecasts. In the meantime, wildland fire managers should be aware of the current limitations of the NDFD and work toward utilizing additional tools (e.g., spot weather forecasts) during critical fire weather conditions.
The authors wish to thank the anonymous reviewers for their helpful comments as well as the National Wildfire Coordinating Group, Fire Behavior Subcommittee, and the USDA Forest Service Office of the Deputy for Research for funding this research.