Mesoscale convective systems (MCSs) are a dominant climatological feature of the central United States and are responsible for a substantial fraction of warm-season rainfall. Yet very little is known about the predictability of MCSs. To help address this situation, a previous paper by the authors examined a series of ensemble MCS simulations using a two-dimensional version of a storm-scale (Δx = 1 km) model. Ensemble member perturbations in the preconvective environment, namely, wind speed, relative humidity, and convective instability, are based on current 24-h forecast errors from the North American Model (NAM). That work is now extended using a full three-dimensional model.
Results from the three-dimensional simulations of the present study resemble those found in two dimensions. The model successfully produces an MCS within 100 km of the location of the control run in around 70% of the ensemble runs using perturbations to the preconvective environment consistent with 24-h forecast errors, while reducing the preconvective environment uncertainty to the level of current analysis errors improves the success rate to nearly 85%. This magnitude of improvement in forecasts of environmental conditions would represent a radical advance in numerical weather prediction. The maximum updraft and surface wind forecast uncertainties are of similar magnitude to their two-dimensional counterparts. However, unlike the two-dimensional simulations, in three dimensions, the improvement in the forecast uncertainty of storm features requires the reduction of preconvective environmental uncertainty for all perturbed variables. The MCSs in many of the runs resemble bow echoes, but surface winds associated with these solutions, and the perturbation profiles that produce them, are nearly indistinguishable from the nonbowing solutions, making any conclusions about the bowlike systems difficult.
Mesoscale convective systems (MCSs) are a prominent meteorological feature over the eastern two-thirds of the contiguous United States and are responsible for a majority of the warm-season rainfall over most of this region (Fritsch et al. 1986; Schumacher and Johnson 2006). Individual MCSs can carry the risk of flash floods, damaging hail, and even tornadoes (e.g., Weisman and Trapp 2003), while MCS activity over a monthly to seasonal time scale is associated with flooding and droughts (Fritsch et al. 1986; Junker et al. 1995).
Despite this prominence, little is known about the predictability of these systems. Historically, predictability studies have relied on simplified statistical models (e.g., Robinson 1967; Lorenz 1969) or low-resolution global models (e.g., Simmons et al. 1995), where the focus is on fields such as 500-hPa heights. Extrapolations from these global models suggest mesoscale predictability to be on the order of hours. In contrast, initial experiments with mesoscale models lead to claims of enhanced predictability on these scales (e.g., Anthes 1986), but they largely ignored sensible weather. Subsequent work (e.g., Errico and Baumhefner 1987; Vukicevic and Errico 1990) called into question the enhanced predictability conclusion, noting several shortcomings in the experimental design. Recent storm-scale experiments have concentrated on the predictability of precipitation and the role that deep, moist convection plays in mesoscale predictability (Zhang et al. 2003; Hohenegger and Schär 2007). Conclusions from these studies tend to favor the relatively short predictability limits indicated by the early experiments based on statistical turbulence models, though fixed forcing mechanisms, such as orography, have been linked to extended predictability limits (Walser et al. 2004). Radar-based observational studies (Carbone et al. 2002; Germann et al. 2006) also suggest the possibility for enhanced predictability on the system scale, if not for individual elements within a convective system.
Some indirect investigations of MCS predictability consist of attempts to develop forecast aids for existing systems, whether simple extrapolations of the storm movement (Boucher and Wexler 1961) or prediction of system maintenance or decay based on environmental conditions (Coniglio et al. 2007). Early forecast experiments with high-resolution numerical models included both squall-line and supercell events (Brooks et al. 1993; Wicker et al. 1997) and achieved some qualitative predictive success, along with some definite failures.
In a more direct MCS predictability study, Stensrud and Wicker (2004) explicitly recognize that MCS development and evolution is sensitive to the preconvective environment in which the MCS forms (Rotunno et al. 1988; Coniglio et al. 2004; Cohen et al. 2007). They use a two-dimensional (2D) storm-scale model to produce a control run of a long-lived MCS using a given preconvective environment and a convection-resolving grid. They then examine MCS predictability by producing an ensemble of runs, each using a different preconvective environment perturbation added to the control environment, and forcing convective initiation to occur. Thus, instead of using infinitesimal perturbations to the environment, as is more commonly used in model-based predictability studies, Stensrud and Wicker (2004) take a very practical approach to examining predictability. They define the preconvective environmental perturbations based upon the typical errors found in operational numerical weather prediction models for selected forecast lead times. This allows them to explore how much confidence one can place in the forecast of a MCS for a given magnitude of environmental uncertainty. This is akin to taking the perspective of a forecaster looking at convection-resolving numerical model guidance for issuing a day 1 or day 2 prediction. If the forecaster sees the model developing an MCS in his forecast area 12 or 24 h into the run, what level of confidence can he place in the occurrence of that event or its subsequent evolution? Perturbing the preconvective environment with magnitudes consistent with 24-h forecast errors, only 60% of the ensemble members produced an MCS located within 200 km of the control run. Stensrud and Wicker (2004) note that raising that success rate to 90% would require improving the forecast error to magnitudes at or below even what is currently possible in model analysis systems.
It needs to be emphasized that the approach of Stensrud and Wicker (2004) does not follow those from previous predictability experiments. The object of study is not a continuous field such as winds or 500-hPa heights, but rather a specific intermittent phenomenon, the MCS. Furthermore, the simulated MCS exists within an initially homogeneous domain so that the MCS itself is the only feature capable of producing significant error growth in the simulations. As a result, traditional measures of error growth, such as the domain-averaged root-mean-squared error of a given field, are uninformative. Instead, predictability is focused on the various characteristics of the MCS of interest to forecasters, such as maximum surface wind fields, or even the (continued) existence of the MCS itself. In addition, as noted, Stensrud and Wicker (2004) examine predictability from the practical perspective of a forecaster making a day 2 forecast. The initial environmental perturbations for the convection-resolving simulations, then, are not on the level of noise but rather are consistent with the errors typically found in a contemporaneous operational forecast model at the day 2 time frame (i.e., 24 h into the forecast).
A companion piece by the authors (Wandishin et al. 2008, hereafter WSMW) updates Stensrud and Wicker (2004) by replacing a warm-rain-only microphysical scheme with one including ice, as well as making changes to the perturbation strategy such that more realistic environmental moisture profiles are achieved. However, the main findings of Stensrud and Wicker (2004) are reproduced: based on current 24-h forecast errors for relative humidity, wind speed and convective available potential energy (CAPE), around 70% of ensemble members successfully produce an MCS within 200 km of the control simulation, with the success rate rising to over 90% when the environmental uncertainty is reduced to the level of current analysis uncertainty. This improvement in forecast accuracy would yield a 10%–20% reduction in the uncertainty for maximum updraft strength and a 25%–35% reduction in the uncertainty for maximum surface wind. Improvements to the ensemble simulations resulting from reductions in forecast errors are not uniform, however. The uncertainty in the success rate, maximum updraft strength, and maximum surface wind are all more sensitive to changes in the relative humidity perturbations than to changes in the wind speed perturbations, while the opposite is true for the MCS size. For example, halving the wind speed errors increases the success rate for ensembles based on smaller CAPE errors, but it only has a minor effect on ensemble based on larger CAPE errors. The CAPE perturbations themselves have the biggest impact for the maximum surface wind, but have no impact on uncertainty in MCS size and only moderate impact on the success rate.
This study retains the perspective of a forecaster looking at a day 2 forecast of a developing MCS from an operational convection-resolving model, but extends WSMW by replacing the 2D model with a fully three-dimensional (3D) version. In doing so we hope to address the question of how much confidence a forecaster could have in convection-resolving model forecasts of a MCS in the day 2 time frame. For example, given the uncertainty in a forecast of surface wind speed, how much above the severe threshold (26 m s−1) must a forecast be in order to have a strong confidence that severe weather will occur? The 3D model is described in section 2 along with the ensemble perturbation strategy. Results are presented in section 3 focusing on a few different storm characteristics of interest to forecasters along with a brief comparison of bowlike versus more linear simulations followed by some discussion.
2. The numerical model and perturbation methodology
a. Description of the numerical model
All simulations are performed with the National Severe Storms Laboratory (NSSL) Collaborative Model for Mesoscale Atmospheric Simulation (NCOMMAS; Wicker and Wilhelmson 1995). The model was developed to study supercell dynamics and tornado genesis, but has been employed successfully to study a wide range of tasks, including a tornado outbreak associated with a landfalling hurricane (Romine and Wilhelmson 2002), convective initiation (Houston and Niyogi 2007), dryline morphology (Peckham and Wicker 2000), the evolution of convective cells within an incipient squall line (Jewett and Wilhelmson 2006), and even severe-weather forecasting (Wicker et al. 1997). Recently it has been used to simulate MCS evolution by Coniglio et al. (2006), who provide an in depth description of the model as it has been updated in the intervening years (see their appendix).
For the present study, an open boundary condition is used in the x direction, the boundaries are periodic in the y direction, and a Rayleigh damping layer is used near the lid. Microphysical processes are modeled by the three-class ice parameterization of Gilmore et al. (2004), a variant of the well-known Lin et al. (1983) scheme, with particle density and intercept values for the mixing ratio distributions of rain, snow, and graupel of (1000, 8 × 106), (100, 3 × 106), and (400, 4 × 108), respectively. As is appropriate for MCS simulations small graupel is favored over large hail (Gilmore et al. 2004). Severe hail reports associated with MCSs, when they occur, are most common in the earliest stages of development or with isolated cells near the leading convective line (Houze et al. 1990).
For modeling convective events, Bryan et al. (2003) propose that grid spacing on the order of 100 m should be used to reproduce small-scale turbulent eddies. They also find, however, that resolving the bulk properties of a convective cloud may be possible with 1 km grid spacing. Similarly, Weisman et al. (1997) find that 4 km may be sufficient to represent the system-scale properties of midlatitude squall-line-type convection. To achieve a balance between ensemble size and the fidelity of the simulated storms, the model grid spacing is Δx = Δy = 1 km and Δz = 500 m (the vertical grid spacing is 250 m in the lowest 1250 m and then stretches to 700 m near the top of the model domain, giving an average grid spacing of about 500 m), with a 10-s time step, covering a domain 700 km in length, 200 km in width, and 20 km in height. The grid spacing and domain size are consistent with other recent studies of MCS evolution (Coniglio et al. 2006) and mesoscale predictability (Hohenegger and Schär 2007).
Convection is initialized with a warm 3-K thermal line extending the entire width of the domain. The line is located at x = 50 km and includes random perturbations with a maximum amplitude of 0.1 K along the line to encourage 3D structures. The strength of the thermal line is sufficient to guarantee convective initiation, although convective initiation is at least as sensitive to perturbations in the environment of a potential storm as is storm maintenance (Crook 1996). Since the focus of the present study is limited to the latter, we explore the predictability of MCS forecasts assuming convective initiation occurs. As such, the results presented herein form an upper bound on MCS predictability overall.
Initial testing (not shown) shows that MCS development in the 3D runs is more robust than in 2D, as the 3D convective cells no longer evolve under a strong two-dimensional constraint (Rotunno et al. 1988). This difference in convective cell behavior between the 3D and 2D runs leads to different MCS success rates, with the 3D runs having higher success rates, when using the identical base-state environmental profile and perturbation strategy. This outcome hampers direct comparisons between the 3D and 2D results to changes in perturbation magnitude. Since WSMW show that changes in the maximum base-state RH above the boundary layer influence the success rate, but have little effect on updraft strength and surface winds, we have chosen to reduce the maximum base-state RH value to where the 3D runs match the success rate of the control ensemble (r823) in the 2D runs. Thus, the maximum base-state RH profile in the 3D runs is set to 75% as compared with 85% in the 2D runs. This base state allows for a greater influence of positive perturbations in the moisture field. In addition, the environments of many MCSs have midlevels RH value below 85% (e.g., Coniglio et al. 2004, 2006).
The simulations are each run out to produce an 8-h forecast. The control run, as shown in section 3, produces a strong and long-lived MCS. The control sounding is a relatively moist sounding with approximately 2600 J kg−1 of surface-based CAPE (Fig. 1). The sounding has little convective inhibition (7 J kg−1), so deep lifting likely is not necessary to initiate new cells in the simulations. The relative humidity has been capped at 90% in the boundary layer, and 75% above the boundary layer, to avoid spurious wave activity in saturated unstable layers. The wind profile for the control run increases linearly from 0.0 to 17.5 m s−1 at 2.5 km above the surface and no shear above that height (Fig. 1); this is the same wind profile used by Rotunno et al. (1988) to study MCS strength and longevity. The CAPE of the control sounding is comparable to the mean CAPE observed for developing MCSs (2742 J kg−1); the shear is a little strong at 2.5 km and a little weak at 5 km (observed values of 14.1 and 20.4 m s−1, respectively); and the RH is relatively strong, particularly in the lowest 2–3 km (cf. Fig. 13a in Coniglio et al. 2004).
b. Perturbation methodology
The perturbation methodology follows exactly that used for the 2D simulations of WSMW. Perturbations of temperature, RH, and u-component wind speeds are added onto the control sounding and applied across the horizontally homogeneous model domain. Perturbation sizes are based on 24-h 0000 UTC forecast errors from the 12-km North American Model (NAM, formerly the Eta; Black 1994) of the National Centers for Environmental Prediction (NCEP) during May and June of 2006. The 20-km Rapid Update Cycle (RUC; Benjamin et al. 2004a) model analyses are used as truth. The analyses from the NAM are bilinearly interpolated onto the RUC grid. The forecast errors are calculated over the continental United States east of the Rocky Mountains, within the region between 25°–49°N and 105°–70°W. Only errors from forecasts valid at 0000 UTC are considered to focus on those errors one would expect to find around the peak hours of convective development. The spread of the 24-h NAM forecast errors thus defines the environmental uncertainty for the developing MCS in the day 2 forecast period (Table 1).
Further comparisons indicate that the increase in forecast uncertainty between 12 and 24 h is minimal (Table 1). For example, the standard deviation of the wind speed errors only increases from 2.9 m s−1 for a 12-h forecast to roughly 3.1 m s−1 for a 24-h forecast. While the distribution of the 24-h forecast errors for wind speed and relative humidity are slightly peaked and right skewed (not shown), these deviations from Gaussianity are sufficiently small that the standard deviation of the data is a reasonable measure of the spread of the forecast errors. Similarly, the forecast uncertainty for relative humidity (RH) increases meagerly from 20% at 12 h to 21% at 24 h.1 To avoid having the CAPE errors dominated by zero CAPE values in stable regions, only locations with nonzero CAPE are evaluated, giving an error standard deviation of ∼800 J kg−1. The CAPE errors also show the largest change from 0 to 12 h, increasing from 300–500 at 0 h to 750 J kg−1 at 12 h, with little change between 12 and 24 h. The peak in the CAPE distribution (not shown) is a little larger than for wind speed and RH, and the NAM displays a high bias for CAPE, but again the distribution is sufficiently close to Gaussian to permit the use of the standard deviation as a measure of uncertainty.
Perturbations to the preconvective environment consistent with the 24-h error magnitudes are used to construct the ensemble members. The ensembles are named according to the perturbation magnitudes employed to generate its members, using the leading digit of the mean perturbation magnitude (e.g., for the r823 ensemble, the average perturbation sizes for CAPE, RH, and wind speed are 800 J kg−1, 20%, and 3.1 m s−1, respectively; Table 2). An additional set of preconvective environment perturbations is constructed using half the size of the 24-h errors instead of those consistent with the 12-h errors because of the relatively small differences between the 12- and 24-h forecast error magnitudes. Halving the 24-h forecast errors reduces the perturbations to a level roughly equal to typical analysis error (Table 1; Thompson et al. 2003; Benjamin et al. 2004b). It should be noted that the errors in the RUC jump markedly between the analysis and the 1-h forecast, with little further change through 12 h (Benjamin et al. 2004b). Thus, the halving of the perturbations does not represent the result of gradual forecast improvement (e.g., tomorrow’s 24-h forecast being as good as today’s 12-h forecasts), but instead represents the results from a radical advance in numerical modeling. That is, in order to reduce MCS forecast uncertainty by the levels shown in the next section, future 24-h forecast errors must be as small as current analysis errors.
The temperature profile for the control run is determined by the temperature at the surface and at the tropopause as well as an exponential defining the shape of the profile. Therefore, the desired variance in CAPE is achieved by randomly perturbing the tropopause temperature (e.g., a 8-K change in tropopause temperature yields a 800 J kg−1 change in CAPE); the CAPE perturbations are determined by these temperature perturbations alone. For the wind and RH perturbation profiles, random perturbations are drawn from a Gaussian distribution with zero mean and a standard deviation matching the errors described in Table 1 and assigned to the vertical model levels every 2500 m starting at the surface. A cubic spline is used to obtain values for intermediate levels. This method ensures that the wind and relative humidity profiles are vertically coherent, so that, for example, realistic dry layers can be introduced into the initial soundings. While both positive and negative perturbations are allowed for the moisture profile, the negative perturbations dominate as the control sounding is already moist and positive perturbation sizes must be limited to prevent saturation. No changes are made to the moisture profile within the boundary layer so that the CAPE is affected by the temperature perturbations alone. Sample environmental perturbation profiles for temperature and wind speed, along with the relative humidity profiles show that the perturbation method produces realistic vertical structures (Fig. 2). For each perturbation field, a single set of 50 perturbations is used and the different perturbation magnitudes are achieved by applying a multiplicative factor. Therefore, any differences between the different ensembles result from the perturbation magnitudes and not from a different sample of perturbations (e.g., more negative and fewer positive perturbations).
As stated earlier, the configuration of the 3D simulations is identical to that for the 2D runs with the exceptions that the domain is extended 200 km in the y direction (with periodic boundaries), the domain is shortened in the x direction to 700 km (from 800 km) to reduce computational and storage requirements, and the warm bubble is replaced by a thermal line extending the full width of the domain, with small random perturbations added to facilitate 3D structure. Because of the computational demands of an additional spatial dimension, the size of the ensembles is reduced from 100 to 50 members. The ensemble results are used to explore how much confidence one can place in the forecast of a MCS for a given magnitude of environmental uncertainty. All error bars shown denote the central 50% credible interval (i.e., the 25th–75th percentiles) for the plotted values. The interval is taken from the empirical distribution as determined by the bootstrapping technique in which the data are resampled with replacement 1000 times, with the statistic calculated for each resampled dataset.
The 3D control run (Fig. 3) is very similar to the 2D control run (cf. WSMW’s Fig. 3), except that the 3D storm moves somewhat slower (cf. ∼17 and ∼20 m s−1), and the trailing stratiform region is almost nonexistent in 3D. A shift in the perspective to a horizontal slice (Fig. 4), taken at z = 4 km, reveals a more complex scenario despite the horizontally homogeneous initial state. The maximum width of the system 8 h into the simulation is nearly double that indicated in the line-averaged cross section (Fig. 3). The fibrous nature of the MCS, particularly apparent in the final frame, is largely an artifact of the grid spacing and the chosen low-level shear. Reducing the grid spacing leads to smaller and weaker updrafts, with correspondingly smaller gaps between cells, while increasing the strength of the low-level shear produces a more filled-in line (G. Bryan 2007, personal communication).
As with the 2D ensembles, introducing the r823 perturbations equal to the 24-h forecast errors yields a wide variety of solutions (Fig. 5), ranging from strong, narrow lines (member 1) to more broad, diffuse systems (member 16) to more isolated cells (member 42) to solutions completely unable to maintain convection (member 7). Some of the solutions have the appearance of a bowing MCS (member 13), while others are distinctly linear (member 4). Average system movement ranges from ∼13 to nearly 24 m s−1. Reducing the uncertainty to represent analysis error still allows for a range of structures (Fig. 6), but the members are noticeably more similar for this ensemble, with much smaller differences in the speed of the systems as well. Additionally, all members of this r511 ensemble are able to maintain some convective activity.
a. Success rate
An MCS was defined to exist in the 2D runs if the region of total precipitation mixing ratio exceeding 0.2 g kg−1 extends for at least 20 km in length anywhere in the z = 3–5-km layer. The along-line variability in the MCS structure in the 3D runs necessitates an additional constraint for defining a successful MCS simulation, namely the 20-km width requirement must be met over at least 90% of the line and no gap in the line can exceed 10 km. These constraints are somewhat arbitrary, but, since the MCSs are initiated with a thermal line extending across the domain, the inability to maintain a solid convective line is fairly termed a failure. Moreover, the success rate is only weakly sensitive to the specifics of the criteria, as most failures are easily identifiable (cf. Figs. 5 and 6). The success rate is simply the percentage of ensemble runs that have an MCS within ±100 km of the location of the leading edge of the MCS in the control run. Whereas the success rate in the 2D runs rises quickly to above 50% by 75 min, MCS development occurs in only a third to half as many members in the 3D simulations by this time (Fig. 7). Successful MCS development typically requires an additional 2–3 h in the 3D runs, with the success rate increasing to around 70% for typical 24-h forecast errors (Fig. 7b). Recall that the success rate for the r823 ensemble is the same as for the 2D runs by design in order to facilitate comparisons between the 2D and 3D results. The asymptotic success rate achieved by reducing the initial uncertainty for all three fields (r511) is 80%–85% (Fig. 7a), or about 5–10 points lower than its 2D counterpart, suggesting that reducing the forecast uncertainty may be more difficult in 3D.
The 2D runs in WSMW possess a strong sensitivity to the environmental moisture profile. The 2D success rate drops by about 1% for every 1% decrease in the maximum cloud-layer base-state RH, such that the ensemble based on a control sounding with a maximum RH of 50% yields only a 30% success rate. For the 3D runs, the success rate decreases from nearly 100% to 70% as the RH in the control sounding is reduced from 85% to 75%. This suggests that the 3D simulations are at least as sensitive to the initial moisture profile as are the 2D simulations.
One interesting feature of the 3D success rate curves is the lack of organization according to the initial environmental perturbations. For the 2D runs, the success rate steadily improves as the initial CAPE errors are reduced and the curves tend to group according to the degree of initial relative humidity uncertainty (cf. WSMW’s Fig. 6). For the 3D runs, there is no such grouping nor is there any consistent dependence on the initial CAPE uncertainty (Fig. 7). Instead, all ensembles except r511 bunch together with many counterintuitive relationships. For example, reducing only the initial CAPE uncertainty from the 24-h forecast error levels to analysis levels (r823 versus r523) actually decreases the success rate. Similarly, reducing the initial wind speed uncertainty also decreases the success rate by about 5% between r523 and r521, and by about 10% between r823 and r821 during the period t = [4, 7] h. No discernible sensitivity to the RH uncertainty exists.
b. Maximum updraft strength
The overall uncertainty in maximum updraft strength is comparable between the 2D and 3D configurations, although temporal differences exist (Fig. 8, cf. WSMW’s Fig. 8). The peak uncertainty occurs almost an hour earlier in the 3D simulations and the spread of the ensembles throughout the runs is 20%–30% larger. As with the success rate curves, there is no concrete grouping among the 800 J kg−1 ensembles based upon which variables have larger perturbation magnitudes (Fig. 8b). The separation between the two curves is minimal but the spread of the r823 ensemble is consistently lower than that of r813 between t = 2 h and t = 6 h, suggesting that reducing the initial RH uncertainty can lead to decreased confidence in the forecast. The peak in the standard deviation from the 500 J kg−1 ensembles is somewhat lower than for the 800 J kg−1 ensembles, but by 3–4 h into the simulations the uncertainty in the maximum updraft strength is the same for both ensemble groups. Most ensemble groups lack sensitivity to the initial perturbation magnitude; only reducing all fields to the level of analysis uncertainty (e.g., r511) yields substantial improvement in forecast confidence (Fig. 8a).
c. Maximum surface wind
The forecast uncertainty from the 3D runs for the maximum surface wind (actually lowest model level) increases by as much as 50% compared to the 2D runs (Fig. 9, cf. WSMW’s Fig. 9). There are few discernible differences between the r813, r821, and r811 ensemble groups, though decreasing the initial uncertainty in only one or two variables can degrade forecast confidence (cf. r823 and r811). The patterns that emerge among the 500 J kg−1 CAPE uncertainty groups in the success rate (Fig. 7a) and maximum updraft (Fig. 8a) plots are more pronounced for the maximum surface wind (Fig. 9a). Reducing either the wind speed or RH errors by half yields large improvements in the forecast uncertainty, while reducing both the wind speed and RH initial errors leads to even greater improvements. It is not clear why the maximum surface winds speed should be more sensitive to the initial uncertainty than the updraft strength or success rate, but these results are consistent with those from the 2D runs.
One of the most important impacts of MCSs is hydrological, as they are responsible for much of the warm-season rainfall over the eastern two-thirds of the United States (Fritsch et al. 1986) as well as producing floods and flash floods (Junker et al. 1995; Schumacher and Johnson 2006). Two different rainfall measures, the maximum instantaneous rain rate and the 1-h rainfall accumulations, are examined, but first the rainfall totals integrated over the entire simulation are shown.
The integrated total rainfall among each ensemble group ranges from 6.72 × 105 mm for the r821 ensemble to 9.96 × 105 mm for the r511 ensemble (Table 3). Curiously, all the ensembles have similar means (6.7–7.5 × 105 mm), except the r823 ensemble (8.68 × 105 mm) and r511 (9.96 × 105 mm) ensemble, the ensembles representing current 24-h forecast errors and analysis error uncertainty, respectively. Despite having the highest mean rainfall total, the r511 ensemble also has, by far, the least spread (25%–40% lower). There is no simple relationship between the perturbations and either the mean or the standard deviation of the total rainfall, not even for RH as one might expect for a rainfall forecast. In fact, as with the means, there is not a great deal of variability in the spread of the different ensembles with the exception of the r511 ensemble, as noted above, and the somewhat smaller spread of the r811 and r513 ensembles. Normalizing the standard deviation by the mean yields a somewhat different picture, but one that is still unclear. Only the analysis-error ensemble (r511) has substantially lower spread than the 24-h forecast error ensemble (r823). Reducing the wind speed uncertainty has no effect on the r813 or the r523 ensemble (cf. r811 and r521, respectively) but results in a large reduction between the r513 and r511 ensemble. In general, reducing the CAPE or RH uncertainty reduces the normalized spread, but in each case there are exceptions (r823 versus r523 and r823 versus r813, respectively).
The maximum 1-h accumulations, for the 800 J kg−1 simulations suggest a clustering according to the wind speed perturbations up to about t = 4 h, at which point the curves converge (Fig. 10b). There is a hint that the curves then cluster according to the relative humidity perturbations over the last few hours. In contrast, for the groups with smaller initial CAPE uncertainty (Fig. 10a), the ensembles are similar up to about t = 4 h, then the spread of the r511 ensemble declines dramatically. Recall that for this ensemble, while not all members meet the MCS criterion, convection is present and robust in all members at t = 6 h (Fig. 6). However, for some members (e.g., members 21 and 25, among others) the convection is not very extensive. Locations in these runs receive rain for a shorter time period, since translation speeds are similar for this ensemble, leading to lower accumulations for these members and greater spread for the ensemble. Two hours later (not shown), most of these members have continued to develop and now cover an area similar in scope to the other members, reducing the spread.
Continuing to even smaller temporal scales, the standard deviation of the maximum instantaneous rain rate (Fig. 11) behaves much like that seen for the maximum updraft and maximum surface wind. The curves are indistinguishable among the 24-h CAPE uncertainty groups (Fig. 11b). The curves for the 500 J kg−1 CAPE uncertainty ensembles are distinct. The r521 ensemble follows the other ensembles for the first 3–4 h and then declines by more than half over the rest of the run, finally matching the low spread of the r511 ensemble—after showing the largest spread of any ensemble near t = 3 h. The r511 ensemble has the lowest spread overall.
The correspondence ratio (Stensrud and Wandishin 2000) provides another perspective on the spread of the rain forecasts. The correspondence ratio (CR) is a numerical representation of the information contained in a Venn diagram, defined as
where U is the area of the union of all specified field values and I is the intersection of these same specified field values. For example, U and I can be defined with respect to the grid points where precipitation exceeds a given threshold. However, I also can be defined for a subset of the full data, such as the region over which at least half of the members of an ensemble exceed a given threshold. The correspondence ratio thus defined is identified as CRf , where f is the number of members used to define the intersection. The CR is bounded by 0 and 1. Low values of the CR may be the result of either too few members exceeding the threshold value or phase errors in time or space. The CRf for 1-h accumulations exceeding 0.2 mm (basically a forecast of whether rain occurs) is shown in Fig. 12. (The transient nature of the instantaneous rain rate is ill suited for this measure. Instantaneous rain rate is closely tied to individual, short-lived cells, and so the correspondence at any given time may not be indicative of the overall level of agreement between forecasts.) As expected, the CRf declines substantially as f increases. The CRf shows a strong dependence upon the magnitude of the initial wind speed uncertainty. Reducing this uncertainty results in an improvement of as much as 0.6 around t = 4 h for CR10, with the curves converging as the runs progress. The greater certainty associated with the r511 ensemble is not present for CR10, but does manifest itself as more members are required for the intersection. The initial RH and CAPE uncertainty has little impact on the CRf values.
Maps of the various CRf intersections show the greater agreement within the r511 ensemble for the larger f values (Fig. 13). The growth of the area of union as times increases also is seen, both because of the growth in the MCS size and the increasing spread of the MCS locations. The large aerial coverage of the unions (light gray) at later times is largely the result of a number of relatively weak, broadly diffused systems (e.g., member 05 or 16; Fig. 5) and the larger differences in translation speed of the systems for the ensembles based on 24-h wind speed forecast uncertainty (cf. Figs. 5 –6). Since the union forms the denominator of the CR, this growth contributes to the gradual decline in scores after t = 3 h. Indeed, the size of the union appears to explain a great deal of the difference between the ensembles with the 24-h wind speed errors (r823, r813, r523, and r513) and those with the reduced wind errors (r821, r811, r521, and r511). The former group displays a tendency toward somewhat larger systems and a greater range of translation speed (not shown) leading to substantially larger unions and thus smaller CRs, despite similar-sized areas of intersection for the lower f thresholds.
Increasing the rainfall threshold to moderate levels, 2 (not shown) and 5 mm (Fig. 14), results in only a slight decline in CRf for f = 10 and 20 members, but a substantial reduction for the more restrictive f = 30 and 40 member thresholds. Furthermore, the separation of the curves based on the wind speed uncertainty becomes less pronounced. As the rainfall threshold is increased from 2 to 5 mm, the time at which the CR peaks shifts from t = 2–3 h to around t = 5 h. This later time also marks the peak in both the success rate curves (Fig. 7) and the curves marking the spread of the 1-h rain accumulations (Fig. 11). Thus, the storms must reach maturity to produce agreement for the rain higher thresholds.
The peak CR10 values for which at least 10 members of the ensembles agree on 1-h rainfall amounts exceeding 10 mm (not shown) are still reasonably large, showing only a modest decline from the 5-mm curves, but the CRs quickly drop away from that peak because of a combination of a smaller area of intersection and a larger union (not shown). Agreement for larger f thresholds is scant. Of course, one would not expect to find strong agreement on the timing and location of the stronger convective cores that are generally required to produce higher rainfall amounts. [The initial environment used here does not lead to the asymmetrical, training systems that can generate extremely large rainfall totals, as seen in Parker and Johnson (2000).]
A signal unique to the CR plots is the dependence on the initial wind speed uncertainty. Larger initial wind speed uncertainty produces a greater range of system translation speeds, thereby creating a larger area for the union of all members and leading to smaller CR values. The success rate is not as sensitive to spatial displacements (the degree of sensitivity is a function of the size of the search window used) and so shows no sensitivity to the wind speed uncertainty. The other measures focus on extreme values and so are likewise insensitive to location errors. This comparison highlights the importance of the choice of evaluation tool, as different tools will often focus on different facets of forecast behavior. It is necessary, therefore, to select an evaluation tool based on the forecast characteristics of interest.
e. Bowing segments
Among the variety of precipitation structures that develop in the runs are several that resemble bow echoes, a feature frequently associated with damaging winds (Fujita 1978; Weisman 2001). To examine these features more closely, the evolution of three apparent bowing MCSs and three linear MCSs from the r523 ensemble are examined (Figs. 15 and 16). The maximum surface wind is similar for both groups; the bows are no more likely to exceed the threshold for severe winds (∼26 m s−1) than are the lines. Similarly, there appears to be no connection between either the strength of the cold pool (not shown) or the location of the maximum near surface temperature perturbation (marked by a “C”). The perturbation pressure field (black contour = +3 mb) seems to be more extensive for the bowing members and is often coincident with the bowing segment, but given the presence of the +3-hPa perturbation pressure field among some of the linear members, it would be difficult to devise a bow echo forecast rule from this field.
Cross sections taken through the middle of the domain for the linear members and through the apex of the bow for the bowing members also fail to show any discrimination between the bowing and linear MCSs. There also is no clear signal in the maximum downdraft fields; the bowing group possesses a slight tendency toward stronger downdrafts, but there is a substantial overlap in the values. It does appear, however, that the descending rear-to-front flow, though not exceedingly strong, is more pronounced for the bowing members, while the front-to-rear flow is stronger in the linear members. That the members would develop different kinematics is not clear, however, from looking at the initial perturbations for these members (Fig. 17). So, whereas there are suggestions that the bowlike solutions are dynamically different than the more linear solutions with stronger descending rear-to-front flow leading to stronger surface pressure perturbations, the two sets of solutions are indistinguishable in terms of the strength of the downdrafts and surface winds. More importantly, from a forecast perspective, the two sets of solutions also are indistinguishable in terms of the perturbations from which they are initialized.
This study examines the predictability of a single intermittent mesoscale phenomenon, the MCS, with respect to the uncertainty in the preconvective environment in which the MCS develops. A convection-resolving 3D control run that contains a long-lived MCS is produced. Several sets of 50 perturbations to the preconvective environment are added to the control run sounding and model runs produced in order to explore MCS predictability. Each set of 50 environmental perturbations are specified by calculating 0–24-h forecast errors from present operational models. MCS predictability is examined from several perspectives, including the development and maintenance of the MCS as a whole, maximum updraft strength, maximum surface wind, and precipitation. Predictability is thus investigated in terms of point values, consistent with the 2D runs, and areal measures. The experimental setup can be viewed from the perspective of a forecaster looking at a day 2 operational mesoscale model forecast. If the forecaster sees the convection-resolving model developing an MCS in the forecast area, what level of confidence can be placed in both the occurrence of that event, and the features of the modeled system? By using 0–24-h forecast errors of relative humidity, wind speed, and CAPE to define the initial environmental perturbations, the large, storm-scale ensemble provides an estimate of the predictability of MCSs for various magnitudes of environmental uncertainty.
A cursory look at the results indicates that predictability estimates from the 3D runs are in many ways similar to those of the 2D runs from WSMW. (Recall that the base-state RH was decreased to make the uncertainty for the 3D control ensemble comparable to that for the 2D control ensemble.) Using environmental perturbations consistent with 24-h forecast errors, MCSs are produced in only 70% of the model runs. If the environmental perturbation magnitudes are reduced to those consistent with current analysis error, then the MCSs are produced in 85% of the runs. The MCS success rate, along with the spread of the maximum updraft and maximum surface wind speed, are comparable between the 3D and 2D experiments. Closer inspection of the 3D results, however, uncovers some intriguing and not easily explainable results. For example, in the 2D results there are definite sensitivities to the size of the perturbations in the three perturbed fields, such as a steady increase in agreement among ensemble members as the CAPE perturbation magnitude is reduced. In contrast, no such patterns emerge in the 3D runs. Instead all ensembles tend to group together, with the notable exception of the r511 ensemble, for which the initial uncertainty is reduced for all fields. Raising the MCS success rate from 70% to 90%, or substantially reducing uncertainty in forecasts of updraft strength or surface winds, is dependent on reducing the initial environmental perturbations to levels commensurate with current analysis errors. An improvement in the accuracy of mesoscale model environmental forecasts of this magnitude would represent a radical advance in forecasting.
Results show that the various 3D ensemble runs are not very sensitive to the initial magnitude of the environmental perturbations unless the perturbations are reduced in all fields. Previous studies of the sensitivity of convection to initial perturbations often involve simulations of supercells and not MCSs, nevertheless, one might expect the results to be qualitatively applicable to MCSs, as well. Moisture perturbations are found to have a large impact on accumulated rainfall (Park 1999) and updraft strength (Gilmore and Wicker 1998; Park and Droegemeier 2000). Updraft strength can also be sensitive to perturbations in the wind field by changes in the mixing associated with vertical wind (Gilmore and Wicker 1998). Furthermore, maintenance of an existing MCS is sensitive to both the CAPE and wind fields (Coniglio et al. 2007; Cohen et al. 2007). However, none of these individual variable sensitivities are observed in the runs presented here, despite the use of substantially larger initial perturbations. The lone exception is with the correspondence ratios of the precipitation fields. This measure is sensitive to location errors and thus the results show some groupings according to the size of the wind field perturbations.
Several of the 3D runs produce structures that resemble bow echoes, while others resemble linear MCSs. The bowlike solutions appear to have a somewhat more pronounced descending rear-to-front flow and consequent positive pressure perturbation field, similar to a surface mesohigh. Whereas observed bow echoes are often associated with damaging winds, the surface winds in the bowlike solutions are not any stronger than in the nonbowing solutions. Moreover, there is no clear signal in the initial perturbation profiles that could aid a forecaster in predicting the existence of bow echoes directly from environmental conditions.
Direct comparison of the predictability results presented herein with previous studies is difficult. The traditional predictability study approach has been to obtain estimates of predictability limits that define the longest lead time for which a skillful forecast is possible. This study instead inverts that approach to determine what level of forecast confidence is possible given an estimate of environmental uncertainty. Even so, the rapid increase in (and mostly sustained) uncertainty in the instantaneous fields, as well as the low CRs for higher rainfall amounts, is consistent with the short predictability limits found in recent high-resolution modeling studies (e.g., Zhang et al. 2007; Hohenegger and Schär 2007). At the same time, the relatively high CRs for low thresholds imply somewhat greater predictability for system-wide properties.
There are several limitations to this study mirroring those listed in WSMW. The two biggest limitations are likely that the use of a homogeneous environment and the near guarantee of convective initiation. Rarely will an MCS travel for 8 h without encountering a substantial change in environment, be it a change in convective inhibition or in the wind speed or direction. Related to the latter, while the domain is three dimensional, the perturbations are still only one dimensional (they vary only in the z direction). Adding this complexity is not likely to increase predictability. Convective initiation is a major forecast problem that has been mostly circumvented in this experimental design. Once again, its inclusion is expected to reduce predictability further. Additionally, since the truth is a single MCS event and initiation is forced, the important forecast problem of false alarms is also not considered. In addition to including the complexities just listed, the results suggest other future work. In particular, in depth analysis of the simulations and the structure of the initial perturbations may reveal why the forecast spread can increase when the initial uncertainty is reduced or why the inclusion of the third dimension seems to affect the behavior of the details of the simulations more than their general characteristics.
Finally, from a practical perspective, we return to the question of how much confidence should a forecaster have in numerical model forecast of a MCS in the day 2 time frame. With a 70% success rate, fully 30% of MCS events would be missed. Added to that are false alarms, and while this research cannot assess false alarms directly, the 2D experiments of WSMW provide some indication. When the maximum cloud layer relative humidity is reduced to 50%, the 2D control run is unable to produce an MCS, but roughly half of the ensemble members develop a MCS, suggesting a large false alarm rate. Even assuming that a MCS is correctly forecast, there remains the additional question of severe potential. The National Weather Service severe criterion for surface winds is 26 m s−1, meaning that the maximum surface wind uncertainty found in the 3D runs is nearly one-third of the severe threshold. The modeled wind field must be well over the severe threshold to have strong confidence that severe winds will occur, as a model forecast surface wind as strong as 35 m s−1 would still have nearly a 1-in-6 likelihood of being a bust. In summary, high-resolution model forecasts are appealing for their ability to produce realistic-looking fields, but the results herein suggest that there is a substantial gap between the capability to produce realistic MCS structures and skillful forecasts of MCSs.
We thank Ted Mansell for help with the N-COMMAS model and two anonymous reviewers whose comments have improved this paper. This research was supported in part by the National Science Foundation under Grant ATM-0432232.
Corresponding author address: Matthew S. Wandishin, NSSL, National Weather Center, 120 David L. Boren Blvd., Norman, OK 73072. Email: email@example.com
For the wind speed, errors increase slightly from 1000 up to about 800 hPa, remain nearly constant up to 500 hPa, then increase to a peak value near 250 hPa. RH errors increase from 1000 to 800 hPa (by about 8%) and decrease above 400 hPa. In between, there is a (roughly 1%) peak just below 500 hPa with nearly equal values above and below this peak. For both variables, the values shown in Table 1 represent the nearly constant midlevel values.