a. Climatological and map-type lightning predictors

Climatological and pattern-type lightning frequencies were developed and used as candidate predictors to capture local enhancements due to interactions between the low-level wind, thermal circulations, and coastline topography (e.g., Pielke 1974; Arritt 1993; Laird et al. 1995; Lericos et al. 2002). These predictors have the potential to add detailed information about local effects that may not be well resolved by NWP models (Reap 1994a).

We used a simple correlation technique described in Lund (1963) and Reap (1994a) to develop the map-type predictors. Based on Reap (1994a), the correlation technique was applied to 3-hourly observed sea level pressure (SLP) fields from RUC analyses spanning the 1998–2005 warm seasons (∼1224 days). SLP patterns imply both the direction and speed of the low-level flow. Although this procedure also was applied to the 950-hPa height fields, SLP produced the most distinct pattern types and the greatest number of maps that could be classified into a type.

The pattern classification was performed over the geographical area shown in Fig. 1. To capture only the regional-scale patterns (i.e., the prevailing wind) and to smooth small-scale variations, the RUC SLP values were interpolated to a coarser grid (100 km) (Fig. 1). Each smoothed SLP map then was correlated with every other map in the sample (9613 available maps). The pressure pattern with the most maps correlated with it at a threshold of 0.70 or greater (e.g., Reap 1994a) was denoted type A and removed from the sample along with all other type A maps. The procedure was repeated to determine subsequent map types until the residual sample contained less than 3% of the maps that were correlated at 0.70 or greater (Lund 1963; Reap 1994a).

Table 2 shows results of the map-type classification. Five map types (A–E) were developed using the 0.70 correlation threshold. Two types (A and B) make up ∼44% of the total sample, while types C–E make up ∼34%. The remaining ∼22% of the sample could not be classified at the 0.70 threshold. This threshold usually is considered the smallest acceptable for pattern classification (Lund 1963; Reap 1994a). Increasing the threshold produces more map types and more detailed map patterns, but at the expense of producing more unclassified cases.

Using the binary indicators for one or more flashes (section 2), relative lightning frequencies (MTFREQ) were calculated for each map type and 3-h period. Similarly, 3-h flash totals were used to calculate an unconditional mean number of flashes for each map type (MTMEAN). Only time periods when the SLP map could be classified at 0.70 or greater were used in preparing the map-type predictors. However, when developing the lightning guidance equations, the unclassified maps were assigned to the type with which they were most correlated. Climatological relative frequencies and unconditional means also were calculated using all warm-season days during 1995–2005. The climatological and map-type lightning frequencies and means were submitted as candidate predictors for the regression analysis described later.

Composite SLP patterns associated with each map type are shown in the left panels of Fig. 3, while the right panels show spatial distributions of the mean number of flashes (MTMEAN) for the 1800–2059 UTC period. The five map types represent distinctly different flow patterns, and are similar to those from previous studies (e.g., Reap 1994a; Lericos et al. 2002). The predominant pattern, type A (Figs. 3a,b), is characterized by high pressure northeast of Florida that produces prevailing easterly and southeasterly flow across the state. As a result, most of the lightning is confined to the west coast, with maxima near Tampa Bay, Fort Myers, and east of Lake Okeechobee. Map type B (Figs. 3c,d) contains a surface ridge over south Florida that results in southwesterly flow across the state. This focuses the lightning along the east coast of the peninsula, with coastline interactions evident near the big bend of the panhandle (Camp et al. 1998). Map type C (Figs. 3e,f) represents a transition between types A and B, in which the east–west surface ridge is located over central Florida. This pattern produces southeasterly flow over south Florida and south-southwesterly flow over the northern peninsula. Thus, the lightning patterns are a combination of types A and B, with maxima along both coasts. Map type D (Figs. 3g,h) is characterized by high pressure north of Florida and lower pressure to the southeast, which is most common during May and September after a cold frontal passage. The dry northeasterly flow confines most of the lightning to south Florida. Finally, map type E (Figs. 3i,j) is a variation of type B, exhibiting a lobe of high pressure over the Gulf of Mexico and lower pressure to the northeast. West-northwesterly flow confines most lightning to the east coast and Big Bend, with generally less coverage than observed with type B.

b. Model-analyzed candidate predictors

A large number of RUC-analyzed predictors were investigated for possible inclusion in the candidate predictor pool, many of which have been found useful in previous studies. The parameters investigated, their abbreviations, and a short description of each are listed in Table 3. The parameters were calculated from the RUC-analyzed temperature, dewpoint, wind, height, and surface pressure fields valid every 3 h (0000, 0300 UTC, . . . , etc.). The fields were interpolated to the 10 km × 10 km grid (Fig. 1) and transformed into the format of a vertical sounding at each grid point (Bothwell 2002). RUC cloud hydrometeor profiles also were investigated; however, a documented error in the RUC cloud analysis procedure through June 2006 (Forecast Systems Laboratory 2006) rendered these fields unusable.

An important assumption is that the model analyses provide the best estimate of the state of the atmosphere at the analysis time, and thus can be treated as “observations” for purposes of developing the PP equations. Although it is important to investigate as many relevant predictors as possible, we focused on those that are well handled by today’s NWP models. Nonetheless, some parameters that can be difficult to forecast on small scales (e.g., MFLXC, LCAPE, CCTHGT) were investigated, with the expectation that as the spatial resolution and physics of mesoscale models continue to improve, forecasts of these parameters also will improve.

c. Generalized linear models

MLR has been used in the majority of previous statistical lightning studies (e.g., Neumann and Nicholson 1972; Reap and Foster 1979; Reap and MacGorman 1989; Reap 1994a; Hughes 2001). However, unless the assumptions of constant variance and Gaussian residuals are met (which is rarely the case with count data), these methods can lead to undesirable and sometimes nonsensical results. Thus, we considered alternative regression methods; namely, the family of generalized linear models (GLMs).

When the predictand is either “yes” or “no,” one such method is BLR. Logistic regressions are fit to binary predictands according to the nonlinear equation

View Expanded

or

View Expanded

where p_i is the predicted probability resulting from the ith set of predictors (x₁, x₂, . . . , x_K). The quantity on the left of (1) is the logit link function, which relates the log of the odds ratio (p/1 − p) to a linear combination of predictors. In BLR, the regression parameters (b₀, b₁, . . . , b_K) are estimated by maximizing a log-likelihood function using iterative methods [Wilks (2006) gives a thorough description of these methods]. Unlike MLR, (2) guarantees that the probabilities are bounded within the interval (0, 1). BLR does not assume a direct linear relationship between the predictors and the response and accommodates the non-Gaussian (Bernoulli) distributions of the regression residuals (Lehmiller et al. 1997).

We used BLR to develop equations giving the probability of one or more flashes (PROB ≥ 1) within a 10-km radius of each grid point (Fig. 1) to produce spatial probability forecasts for each 3-h period. BLR has been used successfully in previous lightning forecasting studies (e.g., Bothwell 2002; Mazany et al. 2002; Lambert et al. 2005; Shafer and Fuelberg 2006). The procedure used to develop the equations is described later in section 3d.

Our second objective was to develop equations for the amount of lightning during each 3-h period, conditional on one or more flashes occurring. For reasons previously stated, the most appropriate model for count data is the Poisson family of GLMs (Elsner and Schmertmann 1993; Gardner et al. 1995; Elsner and Jagger 2004). As in BLR, this approach employs a log link function to linearize the expected value (μ) of the dependent variable (y):

View Expanded

View Expanded

where μ[x_i] is the mean response resulting from the ith set of predictors (x₁, x₂, . . . , x_K). If one assumes that events occur randomly and at a constant average rate (μ) with Var(y) = μ, then the events are said to be generated by a Poisson process with the probability model

View Expanded

A histogram of the conditional count distribution for our most active lightning period (1800–2059 UTC) is shown in Fig. 4. It is clear that the counts are strongly skewed, with the majority of cases having 10 or fewer flashes and few cases having 100 or more. The count distributions for the other time periods (not shown) are even more strongly skewed, with more cases in the lower count bins and fewer cases in the higher count bins. Since the variance of the distribution is very large, ∼80 times greater than the mean (μ ∼ 23 flashes), the data do not fit the Poisson assumption that Var(y) = μ. The most likely explanation is that the counts were generated by an inhomogeneous Poisson process (also known as a Cox process), whereby the number of storms over a given region and the number of flashes produced per storm are both approximately Poisson. This “mixed” Poisson process results in the lightning counts having much more dispersion than is accounted for by a homogeneous Poisson model (T. Jagger, The Florida State University, 2006, personal communication).

An alternative probability model is the negative binomial (NB). As in Poisson regression, the mean response (μ) is modeled by (4); however, the variance Var(y), now is a quadratic function of μ:

View Expanded

where θ⁻¹ is the shape parameter (estimated by maximum likelihood). The resulting probability model for the number of flashes, y, as a function of μ and θ is given by

View Expanded

where Γ is the gamma function (Crawley 2002).

Figure 5 shows the probability distributions implied by the Poisson [(5)] and NB [(7)] models with only the intercept term (b₀) included for μ = 23.15 and θ = 0.342 (estimated from the observed data using the S-PLUS software). Also shown is the observed frequency distribution. It is clear that the Poisson model is a poor choice for representing the count distribution since too little probability is assigned to the smallest lightning counts while too much is assigned to counts near the mean. The NB model is a much better fit to the data, capturing the large number of cases with 10 or fewer flashes and more closely representing the tail of the observed distribution.

Since the NB provides a much better fit to the observed frequency distribution (Fig. 5), it was our method of choice. The NB has been used in previous studies to model thunderstorm activity at Kennedy Space Center (KSC) (e.g., Falls et al. 1971; Williford et al. 1974) as well as thunderstorm and hail days probabilities in Nevada (Sakamoto 1973). However, to the best of our knowledge no prior study has used the NB as the probability model for lightning counts. Since the count distribution (Fig. 5) is left-truncated at one flash, the distribution is not strictly NB since (7) includes y = 0. However, if instead we treat y − 1 as having a NB distribution, then (7) can be used to estimate the probability for each y − 1. Since (7) is a probability density function, the individual probabilities (y = 1, ∞) must sum to 1. Thus, the probability of meeting or exceeding any count threshold, T, can be obtained from

View Expanded

d. Equation development

We determined whether relationships between observed predictors and lightning were generally the same for the entire study area or if they varied significantly from one portion of the state to another. We first subdivided the domain into nine areas (Fig. 6). Then, separate sets of equations were developed for each area (i.e., a regionalized operator approach), with the results compared to those obtained using a model developed for all grid points (i.e., a generalized operator approach). A comparison of verification scores revealed that it would be sufficient to consolidate the nine areas into four larger regions: East Coast, West Coast, Panhandle, and Alabama–Georgia (Fig. 6). To minimize spatial discontinuities at the boundaries, the regions were permitted to overlap, and the probabilities for grid points within the overlapping regions were averaged.

Parameters calculated from the RUC analyses (Table 3), as well as the map type and climatological frequencies for each 3-h period (section 3a), composed the initial set of candidate predictors. However, it is clear (Table 3) that many of the predictors contain redundant information. Including predictors with strong mutual correlation in a prediction equation can lead to poor estimates of the regression parameters (Wilks 2006). This problem was addressed by performing a principal component analysis (PCA) to examine intercorrelations among the predictors (Table 3) and to aid in choosing a smaller subset to retain for the regression analysis. This procedure is described in detail in Shafer and Fuelberg (2006). The principal component groupings, in conjunction with Spearman correlation coefficients between the predictors and the lightning predictands (section 2), were used as an objective method to select a subset of the most physically relevant predictors containing less mutual correlation.

The final list of candidate predictors is given in Table 4. To allow the possibility that one predictor may be more important than another in conjunction with others already in the model, several parameters with mutual correlation were retained for possible selection (KI and PRECPW, several LCAPEs, etc.). Also shown in Table 4 are Spearman rank correlation coefficients between each predictor and the binary indicator for one or more flashes during the 1800–2059 UTC period for the East Coast (EC) region. The correlations are low, meaning that no single observed predictor is a good indicator of lightning (Bothwell 2002; Shafer and Fuelberg 2006). Correlations for the amount of lightning (not shown) also were found to be low. To account for possible nonlinear and interaction effects, power terms up to the fourth degree and two-way cross products were calculated for each parameter selected thus far (Table 4) and then included in the final predictor pool. Finally, the 3-h change in each parameter also was calculated and submitted as candidate predictors.

As mentioned in section 2, the fixed-radius approach requires that the counting regions overlap. The main drawback of this approach is that different locations in the domain are not covered by the same number of counting regions. With a counting radius of 10 km and centers on a 10-km grid, a particular location could be covered by as few as two or as many as four regions. As a result, a lightning strike could be used as few as two or as many as four times in the data analysis. While this would not be a problem if each region were modeled separately, the assumption of independence may be violated if data for all grid points compose the developmental sample (as done in this study). As a precaution, only data for nonoverlapping (disjoint) regions were used for each screening regression sample (illustrated in Fig. 7). Each sample of nonoverlapping data contains ∼¼ of the available data points, resulting in four separate data samples for use in the regression analysis. It should be noted, however, that the variables selected for inclusion in each model and their respective coefficients generally were found to be similar even if data for all grid points composed the regression sample.

A combination of forward stepwise selection and cross validation was used to develop the BLR equations for each region (Fig. 6) and 3-h period using the SPSS software. This procedure is similar to that described in Shafer and Fuelberg (2006). Each database of nonoverlapping data first was subdivided into two samples of ∼300 days, one containing even years (2002 and 2004) and the other odd years (2003 and 2005). Data for even years were used as a “learning” sample for screening the variables for selection, while the odd years were used as an “evaluation” sample to test the model each time a variable was added or removed during the stepwise selection process. Thorough discussions of stepwise selection procedures are given in Hosmer and Lemeshow (1989) and Wilks (2006). The predictors comprising the model at the step with the highest percentage of correctly classified events for the evaluation sample were noted. Only parameters for which the sign of the coefficient made physical sense were retained in the model in any screening sample. The stepwise selection procedure was repeated for each sample of nonoverlapping data, and the predictors chosen for each sample were noted. This procedure identified the combination of predictors that is most likely to generalize to independent data and not overfit the dependent sample. The set of “best” predictors from this process then was re-entered using data for all grid points and all years to determine the final coefficients for each model.

The NB models for PROB ≥ T were developed using S-PLUS. We found that the overnight and early morning periods did not contain a sufficient number of events in the upper percentiles to allow stable, reliable models to be developed. Thus, NB models were developed only for the four most active periods (1500–1759, 1800–2059, 2100–2359, and 0000–0259 UTC). The same sampling procedure (i.e., even and odd years) described above was used to develop the models. However, since the S-PLUS software does not permit stepwise selection for NB regression, the predictors were entered simultaneously into the model. Those predictors contributing the greatest reduction of variance (usually 8 or 10) then were reentered, and the resulting model was tested on the evaluation sample (odd years). Predictors were removed from the model one at a time until the optimal set of predictors that produced the greatest reduction of variance for the evaluation sample was identified. As with the BLR models, the set of “best” predictors then was reentered using all data to determine the final coefficients for each region and 3-h period. The predictors comprising the models and their physical relationships to lightning are described in section 4a.

We used a model containing only climatology and persistence (denoted L-CLIPER) as a benchmark for assessing forecast skill. Climatology consisted of the lightning frequencies and unconditional means for each 3-h period (section 3a), as well as the sine of the day number. Persistence consisted of a binary indicator for whether one or more flashes occurred during the same 3-h period the previous day, as well as the previous-day flash count. Separate L-CLIPER models were developed for each region and 3-h period.

Bothwell (2002, 2005) appears to be the first to use the PP method to develop probabilistic guidance for CG lightning (over the western United States). However, our use of pattern-type predictors differs from Bothwell who included climatological predictors for different pentads. In addition, our NB approach for forecasting the amount of lightning requires only one model to calculate PROB ≥ T for any count threshold, whereas Bothwell used separate BLR models for each threshold. Finally, our guidance is produced on a higher resolution grid (10 km × 10 km versus Bothwell’s 40 km × 40 km). Other than the 12 km × 12 km grid used by Reap (1994a) for 12-h forecasts, our scheme appears to provide the highest-resolution guidance currently available for lightning. It is specifically designed for use with high-resolution models.

a. Discussion of model parameters

This section describes the parameters selected for the BLR and NB models as well as their relationships to lightning occurrence. Since the equations for each 3-h period are variations on a similar theme, the physical reasoning presented here can be extended to all other times. Therefore, this discussion focuses on the most active lightning period (1800–2059 UTC).

The BLR models giving PROB ≥ 1 and the NB models for PROB ≥ T during the 1800–2059 UTC period are shown in Tables 5 and 6, respectively, for the four study regions (Fig. 6). The predictors and standardized coefficients are indicated. A series of diagrams displaying the frequency of one or more flashes (FREQ ≥ 1) and the unconditional mean number of flashes (MEANNF) as a function of several important predictors is shown in Figs. 8 –11. The acronyms used to describe the predictors are defined in Tables 3 and 4.

PRECPW is the most important predictor for one or more flashes (Table 5; Fig. 8a), while KI, a measure of 850–700-hPa moisture as well as stability, was selected in 3 out of the 4 NB models estimating the amount of lightning (Table 6; Fig. 8b). This finding agrees with numerous studies indicating that deep-layer moisture provides the most favorable large-scale environment for warm-season thunderstorms over Florida (e.g., López et al. 1984; Reap and MacGorman 1989; Watson et al. 1995; Mazany et al. 2002). The inclusion of a second-order term (Table 5) implies that this relationship is nonlinear (Fig. 8a), with a peak in FREQ ≥ 1 for PRECPW ∼ 5.5 cm, followed by a decline for even greater values. Largest values of PRECPW usually are associated with widespread shallow convection from tropical systems, which tends to lack the vigorous updrafts and ice processes necessary for lightning formation (Price and Rind 1992; Zipser 1994).

BESTLI was selected as the second most important parameter in the BLR models (Table 5; Fig. 9a) and also is important for predicting the amount of lightning (Table 6; Fig. 9b). The negative coefficients and the relationships depicted in Fig. 9 imply that FREQ ≥ 1 and MEANNF increase with increasing instability (i.e., as BESTLI becomes more negative). Many studies have shown that sufficient instability leading to a persistent and strong updraft is necessary for charge generation (e.g., Price and Rind 1992; Solomon and Baker 1994; Zipser 1994; Petersen and Rutledge 1998). Other stability parameters such as LCAPE and SSI were selected for other time periods.

Coincident areas of abundant moisture (PRECPW) and instability (BESTLI) are expected to be regions of high thunderstorm probability; however, storms will not develop without a source of lift. The selection of MFLXC2 in the BLR and NB models (Tables 5 and 6) indicates that boundary layer forcing is important for lightning formation (e.g., Watson et al. 1987; Reap and MacGorman 1989; Watson et al. 1991). Large MFLXC2 usually is associated with low-level convergence due to the sea breeze and other boundaries (e.g., lake/river breezes, outflows, etc.). The relationships in Figs. 10a,b show that FREQ ≥ 1 and MEANNF generally increase with greater MFLXC2. However, a nonlinear effect is evident for large negative values, possibly due to lightning occurring in the divergent stratiform regions of decaying storms.

First- and second-order terms of 1000–700-hPa mean wind (MEANU3, MEANV3) were selected in several equations (Tables 5 and 6). This relationship is nonlinear for the EC region (Figs. 11a,b), with peak lightning for offshore speeds between 2 and 4 m s⁻¹, and a decline for increasing MEANU3. Weak offshore flow produces a better developed sea breeze and greater convergence, while strong offshore flow may prevent the sea breeze from penetrating inland (McPherson 1970; Pielke 1974; Arritt 1993). Interaction terms involving MEANU3 and the distance from the coast (DISTEC, DISTWC) also were selected, implying that this relationship is modulated by proximity to the coast.

Finally, the pattern-type predictors (MTFREQ and MTMEAN) enter all of the equations (Tables 5 and 6). Although MTFREQ does not rank highly in the BLR models during the 1800–2059 UTC period, it usually is among the first selected for other time periods. Conversely, MTMEAN consistently is the most important predictor in the NB models, implying that the prevailing wind greatly influences locations where storms are most likely to persist over an area and produce large lightning counts (López and Holle 1987; Lericos et al. 2002).

Forecast maps of PROB ≥ 1 are shown in Fig. 12 for 4–5 June 2004, that is, two days of the dependent dataset. The left panels indicate the 3-hourly PROB ≥ 1 based on the RUC analyses valid at the beginning of each 3-h period, while the right panels plot the lightning strikes that occurred during each period. This example, which begins at 1200 UTC on 4 June and ends at 0300 UTC on 5 June, was a very active day with over 36 000 flashes observed. The sequence of probability maps shows the expected diurnal trend in lightning, peaking during the afternoon and then diminishing. More importantly, the agreement between the forecasts and the verification is good, with most of the observed lightning contained within the higher probability contours. A forecast example also is presented in section 5 for the 2006 independent test period.

b. Reliability

Reliability is a measure of the quality of probabilistic forecasts, indicating how well the probabilities correspond with the observed frequency of the predictand (Wilks 2006). Figure 13 plots FREQ ≥ 1 as a function of PROB ≥ 1 for all regions combined during the 1800–2059 UTC period. Similarly, Figs. 14a,d show reliability plots for forecasting the unconditional probability of exceeding the 50th, 75th, 90th, and 95th percentiles during the 1800–2059 UTC period (Table 1). The unconditional probabilities for each threshold, Pr(y ≥ T), were calculated by

View Expanded

where Pr(y ≥ T|y ≥ 1) is PROB ≥ T conditional on one or more flashes occurring (7), and Pr(y ≥ 1) is PROB ≥ 1 obtained from (2). Figures 13 and 14 show that the forecasts exhibit good reliability and are well calibrated, meaning that the event-relative frequencies are nearly identical to the forecast probabilities. Reliability for other time periods (not shown) also is very good. Probabilistic verification and skill scores relative to climatology and persistence are presented next for the 2006 independent test period.

a. Description of model data

We applied the lightning guidance equations (section 4) to forecast output from two mesoscale models run by NCEP during the 2006 warm season and to output from local high-resolution runs of the Weather Research and Forecasting (WRF) Model for a domain over south Florida. The two NCEP models were the 1500 UTC run of the 13-km RUC (RUC13) and the 1200 UTC run of the 12-km North American Mesoscale (NAM12). The high-resolution WRF runs were initialized at 1500 UTC with NCEP 1/12° sea surface temperatures (SST) and data from the Local Analysis and Prediction System (LAPS).

A description of the RUC model was given in section 2. More detailed descriptions of RUC’s model physics and data assimilation methods can be found in Benjamin et al. (2002, 2004). We also used two versions of the WRF model to evaluate the lightning guidance equations—the 1200 UTC run of the 12-km NCEP operational NAM–WRF and a 4-km LAPS-initialized WRF (WRF–LAPS) that is run locally at the National Weather Service (NWS) Weather Forecast Office (WFO) in Miami, Florida. WRF is a state-of-the-art, regional atmospheric model for operational numerical weather prediction and atmospheric research (additional information is available online at http://www.wrf-model.org/index.php). It has two dynamical cores, the WRF–Nonhydrostatic Mesoscale Model (NMM) version developed by the Environmental Modeling Center (EMC) at NCEP and the Advanced Research WRF (ARW) developed by NCAR. Both contain a variety of options for physics packages, including cloud microphysics, boundary layer and surface processes, convective parameterizations, and shortwave and longwave radiation. These options provide sufficient sophistication so that WRF can be used for a broad range of research and operational applications (http://wrf-model.org/index.php). Janjic et al. (2004) and Skamarock et al. (2005) describe WRF’s model physics and parameterization options. A list of the various dynamics and physics options used in the 12-km NCEP NAM–WRF and the 4-km WRF–LAPS run at WFO Miami is shown in Table 7.

WRF–LAPS is part of the WRF Environmental Modeling System (WRF–EMS) that is distributed by R. Rozumalski at the University Corporation for Atmospheric Research (UCAR). WRF–EMS is a complete, full-physics, numerical weather prediction package that incorporates the dynamical cores of both the NCAR ARW and the NCEP NMM into a single end-to-end forecasting system (http://strc.comet.ucar.edu). Unlike the NCEP operational NAM–WRF, the version run at WFO Miami uses high-resolution LAPS data for model initialization (Table 7). LAPS is a diagnostic component of the Advanced Weather Information Processing System (AWIPS). LAPS produces a high-resolution three-dimensional analysis of the atmosphere by combining a background field (obtained from the 1-h forecast of the AWIPS 40-km RUC) with local data from a variety of observing systems. LAPS input data include surface observing systems, Doppler radars, satellites, wind and temperature profilers, and data from aircraft (Hiemstra et al. 2006). The LAPS analysis produced at Miami has a horizontal resolution of 5 km, with 39 vertical levels at 25-hPa intervals from 1000 to 50 hPa. It uses satellite data and level 3 reflectivity data from the Miami and Key West NWS radars to create three-dimensional diabatic analysis grids to initialize the WRF model (“hot start” initialization). The inclusion of data from local mesonetworks enhances the analysis of inland and coastal gradients, and better depicts the effects of Lake Okeechobee on surface fields (Etherton and Santos 2008).

WRF–LAPS forecasts (initialized at 1500 UTC) for the period 19–30 September 2006 were provided by P. Santos (NWS Miami), while runs for the 1 August–18 September 2006 period were produced locally at The Florida State University (FSU) using the WRF–EMS package. The LAPS, 1/12° SST, and NAM12 lateral boundary condition files that were required to produce the FSU runs were provided by P. Santos. The WRF–LAPS domain is centered on the Miami WFO county warning area (Fig. 1). The model configuration used to create the FSU runs for 1 August–18 September was identical to that used at Miami (Table 7). We did not compare results using different physics options or cumulus schemes. An examination of the sensitivity of the lightning forecasts to different model configurations is beyond the scope of this study.

Forecasts from RUC13 encompass the entire 2006 warm season (1 May–30 September), while forecasts from the NCEP operational NAM–WRF span 21 June–30 September (the NAM–WRF became operational on 20 June). We used forecasts valid every 3 h out to 12 h (i.e., the 0-, 3-, 6-, 9-, and 12-h projections). Forecast parameters needed for the lightning guidance equations were calculated from the model forecast temperature, dewpoint, wind, height, and surface pressure fields and interpolated to the 10 km × 10 km forecast grid (Fig. 1).

One should note that the 1200 UTC cycle time of the NAM–WRF hinders the forecasting of afternoon and evening lightning compared to the 1500 UTC runs of the RUC13 and WRF–LAPS. The time periods for which forecasts were available and the verification regions also differ among the three models. Thus, our intent is not to scrutinize differences in model performance. Rather, our objective simply is to describe the results that would have been achieved operationally if the most recently available run of each model had been used to generate the lightning forecasts valid at noon of each day.

b. Forecast verification

The most commonly used measure of accuracy for probabilistic forecasts is the Brier score (Brier 1950) given by

View Expanded

where N is the number of forecast–observation pairs (i.e., the number of grid points), f_i is the forecast probability, and o_i is the observation (set to 1 if the event occurred, or 0 if the event did not occur). The Brier score essentially is the mean of the squared differences between the forecast probabilities and the binary (0 or 1) observations. Perfect forecasts exhibit BS = 0, while less accurate forecasts have 0 < BS ≤ 1. We also calculated the Brier skill score, given by

View Expanded

where BS_MODEL is the Brier score for the model and BS_REF is the Brier score for a reference forecast. As discussed in section 3d, we developed models based on climatology and persistence (L-CLIPER) as well as persistence alone for the reference forecasts. It is clear from (11) that forecasts with a lower Brier score than the reference will have BSS > 0 (or positive skill), while forecasts with higher Brier scores than the reference forecasts have BSS < 0 (or negative skill).

Brier scores for forecasting the probability of one or more CG flashes during each 3-h period are shown in Table 8 for the 1200 UTC NCEP NAM12 (top), the 1500 UTC NCEP RUC13 (middle), and the 1500 UTC WRF–LAPS (bottom). The rightmost column shows the Brier skill score with respect to L-CLIPER (left) and persistence alone (right). Results show that Brier scores are smallest for the less active time periods, and greatest for the more active periods (i.e., 1800–2359 UTC). This occurs because Brier scores tend to be lower when the variance of the forecasts and observations is small (i.e., little lightning and small probabilities), and vice versa when the variance of forecasts and observations is large (Burrows et al. 2005). More importantly, Brier scores for all three models generally are an improvement over L-CLIPER and an even greater improvement over persistence alone through the 2100–2359 UTC period. The reader is reminded that results for each model should not be compared since they are from different initialization times (for the NAM12) and represent different time periods and different regions of the state.

Forecasting the amount of lightning is much more difficult than forecasting whether or not at least one flash will occur. Table 9 shows Brier scores and Brier skill scores for forecasting events in the 75th percentile of flashes or greater (Table 1), conditional on the occurrence of one or more flashes. It is evident that Brier scores generally are higher, and skill scores relative to L-CLIPER and persistence generally are lower compared to those for merely forecasting one or more flashes (Table 8). Nonetheless, skill scores generally are positive through the 2100–2359 UTC period. For events in the 90th percentile or greater (not shown), Brier skill scores are slightly positive or near zero at most time periods, with only the RUC13 producing positive skill through the 2100–2359 UTC period. This finding suggests that events in the 90th or greater percentiles are near the threshold of predictability, at least for forecasts longer than 3–6 h. This is likely due to an inherent weakness in the PP method, namely, the tendency to forecast extreme events unreliably when the accuracy of the predictors (i.e., the NWP model forecasts) decreases to the point where it becomes inadvisable to attempt to predict the extremes of the distribution (Glahn et al. 1991).

The Brier score computations (Tables 8 and 9) require that lightning events occur within a 10-km radius of a grid point. This is a very strict verification criterion that does not consider forecasts at neighboring grid points. A more relaxed approach is to use the maximum probability or the average probability within a certain radius of each grid point (e.g., Burrows et al. 2005). The maximum or average probability then is used in the Brier score computations instead of the gridpoint specific values. We determined that maximum probabilities within 20 km of each grid point generally give the best improvement relative to L-CLIPER and persistence. The results using this new verification approach are shown in Tables 10 and 11 for forecasting one or more flashes and forecasting events in the 75th percentile or greater, respectively. In most cases, magnitudes of the Brier scores remain relatively unchanged; however, Brier skill scores generally are higher when using the more relaxed verification criteria, especially during the first 6 h of the forecast. For reasons that are not clear, the relaxed criteria most benefit L-CLIPER and persistence during the later forecast periods (as evident by the lower skill scores for these periods in Tables 10 and 11).

As one would expect, skill scores deteriorate beyond the 6–9-h projections (Tables 8 –11) as errors in the position and magnitude of predicted convection and synoptic-scale features increase with time in the driving NWP models. Nonetheless, the results shown in Tables 8 –11 are encouraging, especially considering that L-CLIPER alone generally produces very good forecasts during Florida’s warm season. In fact, L-CLIPER is the most difficult standard of reference to beat since it represents an optimal linear combination of both climatology and previous-day persistence. L-CLIPER becomes particularly difficult to beat in situations when the synoptic pattern on a particular day is similar to that of the previous day, which often is the case during Florida’s warm season.

Figures 15 and 16 contain reliability diagrams for forecasting the probability of one or more flashes and the probability of ≥75th percentile events, respectively, for the most active lightning period (1800–2059 UTC). Results for the three mesoscale models are plotted on the same graph. Forecasts from the RUC13 lie reasonably close to the 1:1 line in both plots. However, forecasts for the NAM12 and WRF–LAPS show an underforecasting bias in the lower half of the probability range, and then bend back toward the 1:1 line for higher forecast probabilities. The reason for this behavior is not entirely clear, but most likely is due to inherent biases in the model moisture, temperature, and/or wind field forecasts. Reliability plots for other forecast projections (not shown) indicate similar biases. Reliability when forecasting the 90th percentile or greater (not shown) is similar to that for the 75th percentile (Fig. 16), but tend to deviate a bit more from the 1:1 line for higher forecast probabilities.

c. Forecast example

An example lightning probability forecast for 16–17 August 2006 using the 1500 UTC WRF–LAPS model (Fig. 17) is described next. A map containing county names and labeled geographical features is shown in Fig. 18. Results using the RUC13 and NAM–WRF on this day (not shown) compare favorably with those from WRF–LAPS (Fig. 17). The figure panels show the probability of one or more flashes (left panels), the unconditional probability of ≥90th percentile events (center), and the CG strike verification (right panels) for four 3-h time periods. The flow pattern on this day is type A (section 3), with prevailing southeasterly low-level flow and no synoptic or tropical influences. Between 1500 and 1759 UTC, the greatest probability of one or more flashes (between 30% and 40%) is forecast over eastern Broward and northern Miami–Dade (MD) counties, with probabilities of 10% or greater for areas south of Lake Okeechobee (LOK) (Fig. 17a). Forecast probabilities beginning at 1800 UTC (Figs. 17d,e) are considerably greater than those at 1500 UTC across south Florida (Figs. 17a,b), with the greatest values concentrated along the west coast as well as eastern Palm Beach (PB), Broward, and MD counties. The verification (Fig. 17f) reveals a significant increase in activity (over 7000 flashes) along the west coast and over Broward and MD counties. With the exception of the activity south of LOK, this verification agrees well with the forecast probabilities (Figs. 17d,e). Forecast probabilities for the 2100–2359 UTC period (Figs. 17g,h) have increased south of LOK, and lightning occurs just east of this area over central and western PB and Broward counties (Fig. 17i). Although the area of enhanced probabilities north of LOK does not verify during this period (Fig. 17i), lightning does occur there only one hour earlier, that is, between 2000 and 2030 UTC (Fig. 17f). Finally, forecast probabilities for the 0000–0259 UTC period (Figs. 17j,k) show a diminishing lightning threat, and indeed little activity occurs during this period (Fig. 17l).

The example in Fig. 17 is typical of many others during the 1 August–30 September 2006 period. That is, the sequence of probability maps shows the expected diurnal trend in lightning that peaks during the afternoon and then diminishes. The lightning forecasts generally show good agreement with the verification, with most of the observed lightning occurring within the higher forecast probability contours. However, as observed on 16–17 August, the timing and placement of lightning is not perfect. Nonetheless, the forecasts do capture the general spatial and temporal trends in observed lightning at a level of detail that, to our knowledge, has not been reported previously.