1. Introduction
As both the speed and availability of computational resources increases for atmospheric scientists to employ in numerical weather prediction (NWP), modelers have a tendency to decrease the horizontal grid spacing of computational domains to more precisely predict mesoscale features. This is especially true for studies dealing with processes that require convection to be adequately resolved, such as precipitation (Aligo et al. 2009; Chen et al. 2010; Dyer 2011; Lee et al. 2011; Liu et al. 2010; Lowrey and Yang 2008; Hong and Lee 2009; Zupanski et al. 2011), severe weather (Liu and Xue 2008; Xue and Martin 2006), or localized environmental prediction (Bernier and Bélair 2012; Bonnardot and Cautenet 2009; Grasso 2000; Hart et al. 2005). In such cases, horizontal grid spacing within the simulation framework often approaches 1 km, although a grid spacing where model accuracy is maximized has yet to be determined (Bryan et al. 2003; Petch 2006). A general threshold of 4-km horizontal grid spacing (a.k.a., minimum convection-allowing resolution) has been found at which convection-based weather features are permitted or allowed to be resolved by the model (Weisman et al. 1997; Kain et al. 2008; Clark et al. 2007; Weisman et al. 2008; Done et al. 2004).
Current NWP models are generally limited with respect to the availability of high-spatial-density surface and upper-level observations for data assimilation purposes and, therefore, possess an incomplete description of the underlying physical subgrid-scale processes during model initialization. At small horizontal grid spacing the sensitivity of model output to physical and statistical parameterizations, which act to resolve subgrid-scale features and processes, becomes a critical factor in model precision. As a result, a decrease in horizontal grid spacing without a corresponding improvement in the related parameterizations or quality/amount of initial observations can actually act to decrease the accuracy and precision of the model output (Li et al. 2014; Mass et al. 2002; Rife and Davis 2005).
To quantify variations in model precision resulting from initial conditions and physical parameterizations, modelers generally use ensemble prediction to define the uncertainty within a given model framework (Hagedorn et al. 2008; Hamill et al. 2008; Leutbecher and Palmer 2008; Palmer 2000; Sanyal et al. 2010). In such an approach, the model uncertainty is quantified by generating a series of simulations to estimate a probability density function (PDF) from which a single probabilistic forecast is obtained (Hamill and Colucci 1997; Eckel and Mass 2005; Clark et al. 2008). In a basic sense, an ensemble can be likened to a function such that each input (i.e., a single model run) can be linked to an associated output. The function itself can be thought of as being stochastic, and then any deterministic model run can be applied to that distribution for analysis. In other words, the parameterizations and initial conditions change the function, but from an analysis point of view the output remains a distribution over the associated inputs. By augmenting a deterministic forecast with uncertainty information derived from the ensemble-based PDF, large-scale features and processes can be better simulated since uncertain aspects of the simulation are filtered out (Leith 1974).
There are several sources of real-time global ensemble simulations, including the National Centers for Environmental Prediction (NCEP) Global Ensemble Forecast System (GEFS; Toth and Kalnay 1993) and the European Centre for Medium-Range Weather Forecasts’ Ensemble Prediction System (ECMWF-EPS; Molteni et al. 1996). These systems are already an invaluable forecasting tool for atmospheric processes at a variety of scales, although for mesoscale applications focused on convective processes, the fact that these ensemble simulations rely on convective parameterizations is sometimes a limiting factor. NWP simulations generated using explicit convection may produce superior results since the statistical properties of convection are more precisely represented (Fritsch and Carbone 2004; Kong et al. 2006, 2007); however, the associated increase in computational requirements at convection-allowing resolutions (≤4 km) makes subsequent near-real-time ensemble generation difficult above the regional scale. To this end, Clark et al. (2009) compared a five-member ensemble at 4 km with a 15-member ensemble at 20 km to test the efficacy of using a lower number of ensemble members (albeit at convection-allowing resolutions) to define model spread while improving the representation of mesoscale, convective-based weather features. Results indicated that the higher-resolution convection-allowing model framework showed equal, and in many cases superior, performance with respect to precipitation forecasts despite its reduced number of members. This shows that horizontal grid spacing does play a role in ensemble uncertainty, and that smaller grid spacing (especially in the realm of convection-allowing horizontal grid spacing) may provide additional benefits while enabling a reduction in computational requirements resulting from a decrease in the ensemble size.
Although real-time model simulations over a variety of scales and horizontal grid spacings are available to meteorologists, custom regional models are often required for specific purposes. In such cases, especially those where the timeliness of the output is a consideration, the primary approach is to develop a deterministic model with a relatively large horizontal grid spacing using a verified set of parameterization schemes (Roebber et al. 2004). This allows for the production of model output in a reasonable time frame for forecast guidance. To take advantage of the benefit of smaller horizontal grid spacing on ensemble output, however, a compromise must be made between model runtime and performance (Kober et al. 2012). One such method includes the coordinated development of a model framework with both a large and a small horizontal grid spacing over the same domain. The large horizontal grid spacing framework can be used to develop an ensemble for quantification of model uncertainty, while the framework with small horizontal grid spacing can be used to generate deterministic model output for direct analysis. By varying the grid spacing, modelers have more control over the total runtime and therefore more control over the timing and availability of the model output.
Although becoming more common for operational ensemble prediction systems (Hagedorn et al. 2008; Hamill et al. 2008), such an approach assumes that the model configurations are consistent enough that the PDF from a simulation with a larger grid spacing can be used to approximate the PDF from a simulation with a smaller grid spacing. In this line of thinking, then, a change in horizontal grid spacing would not alter the mean structure of the forecast, such that smaller horizontal grid spacing would only act to add detail to the model output and the associated underlying uncertainty field. On the other hand, if the various ensemble methods produce different distributions (i.e., independent functions), then the problem becomes more complex and horizontal grid spacing will influence the underlying structure of the ensemble uncertainty.
As such, a critical question in developing the associated simulation framework is how the horizontal grid spacing of the model influences the precision of the model output (within this context, precision refers to the repeatability of the model results). With the increase in local and regional operational modeling efforts using gridpoint models such as the Weather Research and Forecasting (WRF) Model, it is important to address this issue to 1) gain an understanding of how local-scale effects can influence model precision at various horizontal grid spacings and 2) provide an overview of how uncertainty can be tested in relation to grid spacing for future model development.
The objective of this paper is to quantify the influence of horizontal grid spacing on the statistical spread of WRF model ensembles over a regional domain centered on Southern California. The goal is not to determine the grid spacing that produces the best statistical spread or the lowest error relative to a ground-truth dataset; instead, the purpose is to quantify the difference in uncertainty (including magnitude and spatial patterns) among the various ensembles to define how an ensemble generated at a specific horizontal grid spacing compares to a deterministic or ensemble prediction at another grid spacing.
2. Data and methods
a. WRF modeling framework
The ensemble NWP simulations used in this project are generated using version 3.6.1 of WRF with the Advanced Research WRF (ARW) core. The 0.5° × 0.5° output from the Global Forecast System (GFS) is used for initial and lateral boundary conditions, with a 3-h boundary condition update frequency. A total of 57 vertical levels are used in the computation domain at all spatial horizontal grid spacings, utilizing a logarithmic profile that concentrates levels primarily near the surface but also at the top of the domain (although to a lesser extent).
The WRF modeling framework used in this study is based on the Army Research Laboratory (ARL) Weather Running Estimate–Nowcast (WRE–N) strategy (Dumais et al. 2004, 2013), which is focused upon development of a convection-allowing short-range “nowcasting” application (0–6-h forecast period) of WRF-ARW (Skamarock et al. 2008). The WRE–N is envisioned to be a forward-deployable rapid-update cycling application of WRF-ARW with four-dimensional data assimilation (FDDA; Liu et al. 2005; Deng et al. 2009), and optimally could refresh itself at up to hourly intervals dependent upon the local observation network (Dumais et al. 2012). To augment the model simulations with uncertainty metrics for forecast evaluation, an ensemble approach is needed; however, such an approach is extremely time critical with respect to model runtime. As such, the inclusion of an ensemble to accompany the deterministic model run must involve a change in horizontal grid spacing and/or the number of nests to produce uncertainty information in a timely manner.
For this project, the WRF model is run in a nonnested configuration with a domain size equal to that of the mother domain in the WRE–N configuration (Fig. 1). Since the WRE–N modeling framework is inherently a triple-nested configuration, future work will involve the inclusion of nests; however, a nonnested strategy was employed here to focus the analysis on the issue of horizontal grid spacing and model uncertainty over the largest spatial extent. The model is run with a base time of 1200 coordinated universal time (UTC) with output generated each hour from 1200 to 1200 UTC of the following day for a total of 25 simulation hours on each of five days in February and March of 2012 (see Table 1 for details of study events). The model top is set at 10 hPa, with a computational time step (in s) equal to 5 times the horizontal grid spacing (in km). To investigate the influence of horizontal grid spacing on model uncertainty, five different choices of spatial grid spacing are used based on the initial 9-km grid-spacing mother domain from the WRE–N framework. The grid spacing was chosen such that the model’s domain area is constant for all simulations. The grid spacing and associated grid extent are as follow: 9 km (175 × 175), 15 km (105 × 105), 25 km (63 × 63), and 35 km (45 × 45). Depending on the parameterization schemes used within each set of simulations, there was approximately a 12 times improvement in model runtime between the 35- and 9-km grid spacings, which in an operational setting (where simulation time is critical) shows the utility of producing an ensemble with larger horizontal grid spacing for uncertainty quantification.
The WRE–N simulation domain, which was used for the WRF model framework within this study.
Citation: Weather and Forecasting 31, 6; 10.1175/WAF-D-16-0030.1
General meteorological conditions for the case study days considered. All case studies are 24 h in length, running from 1200 to 1200 UTC on the days listed.
b. Ensemble generation strategy
There are various ways of producing NWP ensembles, each with its own advantages and disadvantages in terms of model spread and representation of model uncertainty. Utilizing a single model, the most common approaches are to perturb the initial/boundary conditions through different assimilation systems or stochastic variations (Toth and Kalnay 1997), to vary the physical parameterizations used within the modeling framework [a.k.a., multiphysics approach; Stensrud et al. (2000); Du et al. (2004)], or to employ a combination of both methods (Berner et al. 2011; Clark et al. 2008; Du et al. 2006). The use of time-lagged or time-expanding sampling approaches is viable as an ensemble method for rapid-update cycling nowcast models, although as with ensembles using perturbed initial/boundary conditions or multiple physical parameterizations, these approaches may prove to be underdispersive (Zhao et al. 2015; Du et al. 2015). In addition, one could develop a multimodel ensemble by using different external models or different lateral boundary conditions (Nutter et al. 2004). A critical issue with a multimodel or multiphysics approach is that each ensemble member has a different invariant distribution and climatology; therefore, the resulting ensemble is more a sample of deterministic model output from various sources and not a true distribution (Berner et al. 2011). However, the advantages with such an approach include truly independent bias within each member, as well as the fact that different models and/or physical parameterizations will perform differently for a given atmospheric state. As such, the bias within each member may be more or less representative of a specific event, leading to a good quantification of the uncertainty associated with a particular simulation.
For a stochastic ensemble, such as the one produced when using the stochastic kinetic-energy backscatter scheme (SKEBS; Berner et al. 2011) within WRF, all ensemble members are based on the same model framework and baseline initial conditions. As such, all members are considered to have the same underlying distribution and therefore a consistent bias between members. This is a disadvantage in that model bias (as it relates to variations in model output due to changes in model settings such as parameterizations, grid spacing, etc.) cannot be specifically quantified within the ensemble framework, and it is an advantage since an improvement in the core model means an improvement in the entire ensemble. Unfortunately, the specific contribution of each method in describing the ensemble spread is not clearly defined. Clark et al. (2008) showed that ensemble spread resulting from perturbations of initial and lateral boundary conditions was higher than with multiphysics parameterization schemes, although Stensrud et al. (2000) indicated the opposite was true during the first 24 h of a forecast. This discrepancy is attributed by Clark et al. (2008) to higher biases in the mixed parameterization ensemble, which leads to an increase in the ensemble variance. Berner et al. (2011) showed that ensembles generated using the SKEBS scheme outperformed those from a multiphysics scheme, although a combination of both methods produces a better representation of model uncertainty. Regardless of the method used, the concept of an ensemble and the utility of the subsequent output may change depending on the method chosen and the expected final outcome.
Regarding the use of these methods in operational models, perturbation of initial conditions is a common method used for most ensemble configurations (e.g., GEFS and ECMWF-EPS), in addition to other approaches such as multimodel or multiphysics methods. The perturbation of the initial conditions approach, which utilizes the “perfect model” assumption, should logically underestimate forecast error (Houtekamer et al. 1996), and in fact this approach is known to produce poor spread estimates at shorter forecast lead times (Buizza 1997; Hamill and Colucci 1997, 1998; Stensrud et al. 2000). For regional models, the use of a mixed-physics ensemble that focuses on known model deficiencies through parameterizations can produce more spread than the use of initial condition ensembles at short lead times (Stensrud et al. 2000; Houtekamer et al. 1996). Depending on the geographic extent of the model area, however, the influence of lateral boundary conditions can act to limit the increase in ensemble spread (Errico and Baumhefner 1987; Warner et al. 1997), although the amount of time before this effect becomes apparent is based on the advective time scale of flow across the domain [which is a function of domain size and synoptic-scale flow velocity; Vukicevic and Paegle (1989)]. Since mid- and upper-tropospheric short waves may have faster propagation speeds across the model domain than advective winds, these effects can possibly occur at even shorter forecast lead times.
To focus the efforts of this research on the influence of horizontal grid spacing on ensemble uncertainty, the ensemble generation strategy relies on simulations using independent stochastic and parameterization ensembles with a single set of initial and boundary conditions for each test case. It is common to use variable initial and lateral boundary condition data for regional ensemble generation, and the choice of data does play a role in the ensemble uncertainty (Weidle et al. 2016); however, for this research the model error is the only source of uncertainty that is of interest within the ensemble framework. In a real-time implementation of the WRE–N system, variable initial/lateral boundary conditions may be used; however, the effort may not be advantageous if the lateral boundary conditions are produced by global models with large horizontal grid spacing relative to the regional framework (Nutter et al. 2004).
For the stochastic ensemble, the initial/boundary conditions and all parameterizations remain constant across all simulations, with the selected parameterizations as follows:
microphysics—Ferrier scheme (Ferrier et al. 2002)
planetary boundary layer—Mellor–Yamada–Janjić (MYJ; Janjić 1994) scheme
cumulus—Kain–Fritsch scheme (Kain 2004)
longwave radiation—Rapid Radiation Transfer Model (RRTM; Mlawer et al. 1997; Cavallo et al. 2011)
shortwave radiation—Dudhia (1989) scheme
land surface—Noah land surface model (Chen and Dudhia 2001a,b)
Regarding the parameterization ensembles, a multiphysics approach is taken whereby the cumulus, microphysics, and planetary boundary layer (PBL) parameterizations are allowed to vary between model runs. Parameterizations are a source of uncertainty within numerical weather models; therefore, focusing on their contribution to model bias should allow for a more descriptive spread at short lead times. Additionally, by comparing the various ensemble members to observations (which is beyond the scope of this study and held for future research), the ideal set of parameterizations can be chosen for subsequent deterministic forecasts (e.g., Hariprasad et al. 2014; Ratna et al. 2014). For the multiphysics ensemble, the initial/boundary conditions remain constant across all simulations; however, the cumulus, PBL, and microphysics schemes vary between ensemble members while the radiation and land surface schemes remained unchanged. The time steps for the radiation and cumulus schemes are set at 9 and 5 min, respectively, for all simulations. The parameterization schemes used to produce this 12-member ensemble, which reflect schemes commonly used for operational WRF simulations, are as follow:
microphysics
Lin scheme (Lin et al. 1983)
Ferrier scheme (Ferrier et al. 2002)
Thompson scheme (Thompson et al. 2004)
planetary boundary layer
MYJ scheme (Janjić 1994)
Mellor–Yamada–Nakanishi–Niino scheme (MYNN2; Nakanishi and Niino 2004)
cumulus
Kain–Fritsch scheme (Kain 2004)
Betts–Miller–Janjić scheme (Janjić 1994, 2000)
longwave radiation
shortwave radiation
Dudhia scheme (Dudhia 1989)
land surface
Noah land surface model (Chen and Dudhia 2001a,b).
c. Statistical testing
To determine if the ensemble simulations with varying horizontal spatial grid spacings produce unique spread values, the interquartile range (IQR) is calculated for each grid point over all ensemble members. Since each atmospheric variable is characterized by different probability density functions (i.e., gamma distribution for specific humidity and precipitation, Gaussian distribution for temperature and geopotential height), the IQR is chosen to represent spread since it makes no underlying assumptions on the distribution of the data.
For quantification of the magnitude of the spread within each ensemble, the median IQR over all grid points in the domain is calculated for each forecast hour of each event. In some cases the maximum IQR is calculated to augment the results but was not presented. Although any number of variables could be used to define the ensemble spread, the IQRs of 500-hPa geopotential height and 850-hPa temperature are used to define the progression of spread during each 24-h ensemble forecast. These variables are chosen since they incorporate both kinematic and thermodynamic characteristics of the atmosphere and are, therefore, considered to be representative of the general atmospheric state at any given time.
To determine the differences in the statistical distribution of spread within each ensemble, a cumulative density function is generated to describe the IQR values at a specific grid spacing and simulation forecast hour for a specific variable. For the sake of brevity, only the 12- and 24-h forecast lead times are included for analysis since they represent spread during the middle and end of the simulations. This should incorporate the issue of increasing spread with increasing forecast hour as the model error propagates and compounds through the simulation.
To test if the various fields of IQR calculated at each grid spacing and variable are part of the same underlying distribution, the two-sample Kolmogorov–Smirnov (K–S) test is used, which quantifies a distance between the empirical distribution functions of two samples. The null distribution of the two-sample K–S statistic is calculated under the hypothesis that the samples are drawn from the same distribution. In each case, the distributions considered under the null hypothesis are continuous distributions but are otherwise unrestricted, with sample sizes equal to the number of grid points in each simulation (n = 30 276 for 9 km, n = 10 816 for 15 km, n = 3844 for 25 km, and n = 1936 for 35 km). The two-sample K–S test is one of the most useful and general nonparametric methods for comparing two samples, as it is sensitive to differences in both the location and shape of the empirical cumulative distribution functions of the two samples. Over the five test cases used in this study, the K–S test is employed to define if the simulation of each variable using the 9-km grid spacing is significantly different than those of other grid spacings at a 95% confidence interval.
3. Results
a. Time series analysis of spread magnitude
Analysis of the median IQR of 500-hPa geopotential heights shows that ensemble spread tends to increase rapidly during the first 3–6 h of each simulation, after which it continues to increase or remain constant through the end of the simulation (Fig. 2). This is logical considering the first few hours of each simulation contain potential error due to the “cold start” approach, where the model must gradually build up momentum to adjust to the initial and lateral boundary conditions (Cavallo et al. 2016). This is also referred to as the “spinup” of the model. The exception to this is case 5, which shows an increase in spread through hour 10 for the multiphysics simulation, after which spread decreases. Ensemble spread is case dependent, as it is influenced by factors such as the type of forcing (i.e., kinematic versus thermodynamic) as well as the scale of the event being simulated; therefore, it is not viable to directly compare the patterns of spread between each case. However, spread should increase throughout a simulation as a result of the nature of the model uncertainty growth (Klocke and Rodwell 2014). As such, the patterns of spread shown in Fig. 2 are not inconsistent with expectations for ensembles.
Median IQRs of 500-hPa geopotential height for multiphysics and stochastic ensembles at various horizontal grid spacings over the five selected 24-h simulations. Note that the vertical scales vary for each case to highlight differences between the respective time series.
Citation: Weather and Forecasting 31, 6; 10.1175/WAF-D-16-0030.1
Relative to the stochastic ensembles, the uncertainty of 500-hPa geopotential heights within the multiphysics ensembles is generally lower at all horizontal grid spacings except for case 1. Case 1 was characterized by widespread precipitation and instability across the study area; therefore, these results display the sensitivity of the model results to the choice of parameterization. The remaining cases show that the stochastic initial condition perturbations play a greater role in determining ensemble uncertainty than does the choice of parameterization during events with weak synoptic and/or mesoscale forcing across the domain. More relevant, however, is the fact that the level of uncertainty generated within the various stochastic ensembles is relatively consistent across events and horizontal grid spacings. While the relative magnitude and pattern of uncertainty growth during the simulations differs between cases, the range in uncertainty for the stochastic ensembles is roughly equal to the range of uncertainty among the multiphysics ensembles (with the exception of case 2). Also, for the stochastic ensembles, horizontal grid spacing leads to no discernible differences in uncertainty, such that all the ensembles show the same overall pattern of uncertainty growth and magnitude throughout the simulations.
In comparison, analysis of 500-hPa geopotential heights within the multiphysics ensembles indicates that spread tends to increase with horizontal grid spacing, such that the 9-km (35 km) ensembles generally had the highest (lowest) overall ensemble spread at all forecast lead times. However, the difference in spread among the various horizontal grid spacings is not linear, with the difference in median IQR between the 9- and 15-km simulations being less than half the difference between the 15- and 25-km simulations. Although the actual magnitude of the median IQR among the study cases varied considerably, with a maximum for case 1 and minimum for case 2, the 9- and 15-km simulations showed similar values of spread at all forecast lead times, with minimal deviation between the time series after the initial increase in spread during the first few hours of each simulation. This implies that the 15-km model framework can be used to quantify the median uncertainty in the 9-km simulation, albeit with a slightly lower overall spread.
The same argument cannot be made with respect to the maximum spread, as opposed to the median, of 500-hPa geopotential height within each multiphysics ensemble (not shown), where the 9-km simulations generally showed the highest maximum IQR at some point during the simulation. The maximum in the 9-km IQR tends to coincide with an increase in subgrid-scale features related to cloud and precipitation processes that characterized case 1. As such, the variation in the parameterization schemes leads to an increase in model spread due to the sensitivity of the simulations to the choice of the cumulus, microphysics, and PBL schemes. For the remaining cases where precipitation-based subgrid-scale processes do not dominate the simulations, despite a few short peaks in maximum IQR, there is general agreement in the spread among the ensembles at varying horizontal grid spacings. This implies that specific areas within the model domain that exhibit a peak in ensemble uncertainty due to an increase in subgrid-scale processes (i.e., convective precipitation) will likely exhibit a substantially higher level of spread at smaller horizontal grid spacings. However, on case study days where synoptic-scale processes likely dominate, the same general magnitude of IQR exists between ensemble simulations, allowing the resulting spread values to be used interchangeably among ensembles of varying grid spacing.
Analysis of ensemble spread within the 850-hPa temperature field (Fig. 3) shows that the multiphysics ensemble produces comparable magnitudes of uncertainty relative to the stochastic ensembles for cases where convective precipitation exists (i.e., cases 1, 3, and 4). This is logical considering the sensitivity of 850-hPa temperatures to surface-based convection. As with 500-hPa geopotential height uncertainty, the spread of 850-hPa temperature within the stochastic ensembles remains relatively consistent between all cases in terms of both magnitude and the rate of growth over time. Unlike with 500-hPa geopotential heights, however, there is a noticeable difference in the behavior of the stochastic ensemble spread with respect to horizontal grid spacing. In general, the smaller (larger) horizontal grid-spacing simulations produce higher (lower) levels of uncertainty, although in many cases the 35-km simulations deviate from a group containing the simulations for the other three grid spacings. This indicates that the uncertainty produced by the 35-km ensemble is unique relative to those of the simulations using the other grid spacings, which is perhaps due to the importance of small-scale processes and/or circulations that are not being properly resolved at this horizontal grid spacing. As such, at smaller grid spacings where smaller-scale processes are being resolved, there is likely more nonlinear interaction between scales that makes the spread increase.
As in Fig. 2, but for 850-hPa temperature.
Citation: Weather and Forecasting 31, 6; 10.1175/WAF-D-16-0030.1
The result of these interactions is more pronounced in the multiphysics ensembles, where there are substantial differences in ensemble spread between simulations of various horizontal grid spacings (Fig. 3). As with the stochastic ensembles, the simulations with smaller horizontal grid spacings produce greater ensemble spreads, although the pattern of uncertainty growth over the simulations remains consistent within each case. This indicates that model grid spacing does play a role in the generation, growth, and magnitude of uncertainty within the simulations with respect to low-level thermodynamic conditions; therefore, uncertainty metrics developed within one model framework based on horizontal grid spacing cannot be applied to another if lower-level atmospheric features are the focus of the forecast.
b. Analysis of cumulative density functions of ensemble spread
To compare the distribution of the model uncertainty over the study domain relative to the horizontal grid spacing for the two types of ensembles (multiphysics and stochastic), a cumulative density function (CDF) of IQR is calculated for each case and model grid spacing based on 500-hPa geopotential height and 850-hPa temperature. A two-sample K–S test statistic is then applied to the 500-hPa geopotential height and 850-hPa temperature fields to determine if the ensemble simulations at varying horizontal grid spacings are statistically different. In comparing the expected values of the IQR CDFs of each of these variables across the five test cases relative to the 9-km ensemble, results show that at the 12-h forecast the multiphysics ensemble spread in 500-hPa geopotential heights is significantly different (p < 0.05) for all grid spacings in case 1, while cases 3–5 only show significant differences between either the 25- or 35-km simulations (Table 2). For the 24-h forecast time, cases 1 and 3 exhibit significant differences between ensembles at all grid spacings. As described in Table 1, case 1 is markedly different than the other cases in that it is more dynamic in terms of synoptic-scale forcings with attendant subgrid-scale processes, whereas the other cases are more slowly varying (especially case 2). As such, subgrid-scale processes would play a larger role in each simulation; therefore, the multiphysics ensemble would generate the most spread. However, cases 3–5 do have some level of forcing based on either synoptic-scale patterns (i.e., short-wave trough) or surface-based convective processes; therefore, variations in parameterizations should logically change the way uncertainty is generated, especially at larger horizontal grid spacings.
Expected values of IQR CDFs. Boldface values indicate that the specified distribution is significantly different (p < 0.05) relative to the 9-km simulation for that case and ensemble type based on a two-sample K–S test. For reference, the sample sizes for the various grid spacings are 9 km, n = 30 276; 15 km, n = 10 816; 25 km, n = 3844; and 35 km, n = 1936.
For the spread associated with the stochastic ensemble simulations of 500-hPa geopotential heights, only the ensembles at larger grid spacings (i.e., 25 and 35 km) show significantly different ensemble spreads (p < 0.05) than the 9-km simulations for cases 1, 3, and 5 at the 12-h forecast (with the exception of the 25-km ensemble for case 1). For the 24-h forecast, however, all but case 2 (characterized by quiescent synoptic conditions) show significant differences between the 25- and 35-km ensembles, while all grid spacings are statistically different from the 9-km ensembles for cases 1 and 3. These results imply that model uncertainties based on synoptic-scale forcing are better represented using a stochastic ensemble approach, and that in general if moderate to strong synoptic conditions exist within the simulation domain, ensemble spread based on a simulation with larger horizontal grid spacing cannot be used to describe uncertainty in simulations with smaller horizontal grid spacing. Although an exact limit cannot be defined based on these results, it would generally be accepted that the uncertainty values (based on 500-hPa geopotential height) between the 9- and 15-km ensemble simulations are comparable except in cases with strong synoptic forcing.
Cases 3 and 4 represent days with predominantly surface-based forcing instead of upper-level dynamics; therefore, it is reasonable to hypothesize that the simulation of low-level temperature fields would show the greatest sensitivity to horizontal grid spacing. This is the case, as indicated by the significantly different (p < 0.05) IQR distributions based on the multiphysics ensemble simulations for 850-hPa temperature at larger grid spacings (>25 km; Table 2) at both the 12- and 24-h simulation times. In general, these results indicate that conditions with stronger forcing mechanisms (either kinematic or thermodynamic) lead to varying ensemble uncertainty with horizontal grid spacing. Interestingly, however, the ensemble spread for the stochastic simulations show no significant differences at any horizontal grid spacing for any case during the 12-h forecast, with differences only arising in cases 2, 4, and 5 for the 24-h forecast.
These results are intriguing in that they imply that the stochastic scheme produces greater ensemble spread for synoptically driven atmospheric processes while the multiphysics scheme produces greater ensemble spread for surface-based or thermodynamic atmospheric processes. This is logical, considering that thermodynamic processes are generally subgrid scale; therefore, parameterizations are required for proper simulation. The ramifications of this are that the ensemble method used to produce the model spread will be a deciding factor in how influential the horizontal grid spacing is, which means the specific atmospheric conditions that exist within the simulation domain must be taken into account if ensemble spread from one model is to be applied to a forecast from a model with a different horizontal grid spacing.
For illustrative purposes, and to augment the discussion regarding differences between the CDFs of ensemble spread at varying horizontal grid spacings, the CDF for the 24-h forecasts is generated for the 500-hPa geopotential height field (Fig. 4) and the 850-hPa temperature field (Fig. 5). The most striking feature is the distinct difference in the curves between the multiphysics and stochastic ensembles. In general (with case 1 being the primary exception), the CDFs for the multiphysics ensembles show a much higher frequency of low (or zero) IQR values for either variable, which is generally reflected in the associated expected values (Table 2). This is especially true for 500-hPa geopotential heights (Fig. 4). Regardless of shape, however, it is apparent based on the CDFs and the associated expected values E(x) that there is no general relationship between the horizontal grid spacing and E(x). The only discernable pattern, which is verified by the results of the two-sample K–S test results (Table 2), is that the ensemble uncertainty for simulations at larger horizontal grid spacing is generally lower over the model domain, as evidenced by the lower expected values, and that this pattern is most apparent in 1) the multiphysics ensembles and 2) the 850-hPa temperature uncertainty (Fig. 5).
CDFs of IQRs of 500-hPa geopotential height over the WRF simulation domain for the multiphysics and stochastic ensembles at various horizontal grid spacings over the five selected 24-h simulations. Note that the horizontal scales vary for each case to highlight differences between the respective CDFs at lower bin values.
Citation: Weather and Forecasting 31, 6; 10.1175/WAF-D-16-0030.1
As in Fig. 4, but for 850-hPa temperature.
Citation: Weather and Forecasting 31, 6; 10.1175/WAF-D-16-0030.1
c. Spatial analysis of ensemble spread
Maps of ensemble uncertainty during the 24-h forecast period for case 1 (7–8 February) are generated for all ensemble methods and horizontal grid spacings to provide a geographic representation of model spread. Only one case is analyzed because of space constraints; however, this case displays substantial magnitudes and variations in uncertainty, and is a good representation of what model results would look like on days with moderate to strong synoptic forcing. For reference, these maps correspond to the data presented in the CDFs from the previous section. Although this analysis is qualitative based on visual interpretation, it does provide a geographic perspective on the patterns of uncertainty across the simulation domain. This can help in identifying if there is consistency in the spatial representation of model spread among the various simulations.
The IQR patterns for 500-hPa geopotential height in the 9-km multiphysics ensemble (Fig. 6) show a peak in uncertainty values along the Southern California and northern Mexico coastline, extending eastward into southern Arizona. The same general pattern is evident at the larger horizontal grid spacings, although the area of maximum uncertainty tends to be more focused along the Baja Peninsula and less along the Southern California coast as grid spacing increases. The results from the stochastic ensemble also show a maximum region of uncertainty along the California/Mexico coastline, although the area extends slightly more to the west with no corresponding maximum over southern Arizona.
IQRs of 500-hPa geopotential height spread (m) across the WRF simulation domain for all horizontal grid spacings and ensemble types. Data are valid for the 24-h forecast of the 7–8 Feb 2012 event (case 1).
Citation: Weather and Forecasting 31, 6; 10.1175/WAF-D-16-0030.1
Spread in 850-hPa temperatures (Fig. 7) for the multiphysics ensemble is organized into multiple small areas of maximum uncertainty across the simulation domain at a 9-km grid spacing, with these areas combining into two general areas of maximum uncertainty with increasing horizontal grid spacing. The aggregation of the areas of highest IQR progresses gradually as the grid spacing changes, which shows that although horizontal grid spacing influences the detail in the patterns of uncertainty over the simulation domain, the various multiphysics ensembles do agree on the general location of maximum uncertainty. For the stochastic ensemble, uncertainty is substantially less than that of the multiphysics ensemble, which is also shown in the associated CDFs (Fig. 5). Additionally, despite variations in the spatial detail of where the uncertainty is maximized, spread tends to exhibit the highest values off the coast of Southern California and in the extreme southern tip of Nevada.
As in Fig. 6, but for 850-hPa temperature spread (°C).
Citation: Weather and Forecasting 31, 6; 10.1175/WAF-D-16-0030.1
4. Discussion
Based on results utilizing the IQR of 500-hPa geopotential height as a measure of ensemble spread, there are two primary patterns of note. First, the influence of horizontal grid spacing does play a role in the magnitude of uncertainty, especially at longer forecast lead times and for events with stronger synoptic forcing (i.e., case 1). Second, the patterns of uncertainty are noticeably different between the multiphysics and stochastic ensembles, with uncertainty in the stochastic ensembles increasing relatively consistently during the simulation period for all cases. This is not necessarily the case for the multiphysics ensemble, where uncertainty is maximized at different times. With regard to the stochastic ensembles, there are substantial (and in some cases significant) differences in uncertainty between the different variations of horizontal grid spacing; however, unlike with the multiphysics ensembles, there is no general pattern regarding which simulation has the highest or lowest values. As such, it is difficult to quantify any discernible biases between the uncertainties at the varying horizontal grid spacings.
With regard to the 850-hPa temperature uncertainty, the influence of horizontal grid spacing is much more noticeable for both types of ensembles. In general, horizontal grid spacing is inversely proportional to the magnitude of the uncertainty, such that simulations with smaller horizontal grid spacing have higher ensemble spread. This is most noticeable with the 35-km simulations, which show the greatest deviations relative to the other simulations. These results indicate that for days characterized as having moderate-to-strong synoptic forcing with precipitation (i.e., cases 1, 3, and 4), both grid spacing and ensemble type play a large role in the magnitude and temporal evolution of model spread.
Based on these results, there is a clear indication that the type of event being simulated and the primary atmospheric processes related to that event largely determine the magnitude and patterns of model uncertainty. Not surprisingly, the type of ensemble being used also influences the amount of uncertainty generated in the associated model runs. All of these factors appear to outweigh the individual impacts of horizontal grid spacing on ensemble spread; however, at longer forecast lead times the influence of horizontal grid spacing does begin to play an important role in uncertainty magnitude. As such, utilizing a modeling framework with a larger horizontal grid spacing for the purposes of generating ensemble uncertainty, and then using those uncertainty metrics to assess the uncertainty of simulations at smaller grid spacing, would be inadvisable if the grid spacings are substantially different or at longer forecast lead times (>12 h). Based on the time series of uncertainty for the five cases presented here, the 9-km simulations could be used in conjunction with the 15-km ensemble uncertainty with an acceptable bias, and during weak synoptic conditions the 25-km ensemble uncertainty could be used. This is especially true for forecast lead times less than 12 h, which is promising for nowcasting applications. For longer forecast lead times, simulations with larger horizontal grid spacings produce a much lower ensemble spread, which means a forecaster utilizing those values in association with a 9-km deterministic simulation would substantially underestimate the uncertainty.
Analysis of the 12- and 24-h forecast uncertainty shows that although ensemble spread may show an acceptable bias between horizontal grid spacing relative to only magnitude, spatial patterns often show a marked difference between ensemble type and horizontal grid spacing. This decreases the utility of substituting ensemble uncertainty at different horizontal grid spacings to a single model run with small grid spacing, since the magnitude and/or spatial pattern of uncertainty may not be comparable. Relative to the spatial patterns of uncertainty, results show that as horizontal grid spacing increases, the overall magnitude of uncertainty tends to decrease across the domain. As a result, areas of maximum uncertainty tend to smooth out and blend with the surrounding areas, leading to a loss of detail in the geographic representation of uncertainty. However, the difference between the 9- and 15-km ensemble spreads shows the smallest degree of deviation, and in fact the prominent areas of uncertainty tend to spatially align. As such, it can be argued that the ensemble spread values from the 15-km model framework can be used to describe uncertainty within the 9-km model framework.
For time-critical forecasting purposes this could be deemed an advantage since it allows for a more rapid assessment of uncertainty across the model domain; however, removal of small-scale features at larger grid spacings could remove, or average out, important features that would influence forecaster decisions. For example, at smaller grid spacings there is a clear bimodal distribution of high 850-hPa temperature IQR values off the coast of southern California and the Baja Peninsula for the multiphysics ensemble for case 1. These features are also apparent in the 25-km ensemble; however, for the 25-km ensemble the two distinct modes are replaced by a single area of higher IQR. Although this area does show elongation with nodes indicative of the individual areas shown at smaller grid spacings, a forecaster could easily misidentify this feature as a single area of uncertainty, when in fact there are two distinct areas with a substantial geographic displacement.
This pattern, whereby areas of spread are smoothed out as grid spacing increases, could be seen as an advantage or disadvantage for forecasters, depending on the application. A critical note here is that these model results are only meant to reflect the precision of the model not its accuracy. As such, it cannot be determined whether one ensemble product is more or less accurate than another, only that the uncertainty information provided by the various ensembles is statistically different with respect to horizontal grid spacing.
5. Conclusions
This study shows that under certain conditions, a 9- and 15-km multiphysics and/or stochastic ensemble over a regional simulation domain shows comparable values and patterns of spread, indicating that an ensemble generated at 15 km can be used in association with a 9-km deterministic model run, particularly for lead times less than 12 h. Given the difference in computational time required to complete a model run at the different horizontal grid spacings, this result shows that it is possible to provide a combination of deterministic and ensemble model products to operational forecasters using regional model output. More importantly, the simulations can potentially be done on portable equipment given the right model setup and framework. This provides a powerful tool for those needing a forward-deployed, operational modeling system where rapid analysis and forecasting of atmospheric properties is required.
Considering the operational practicality of ensemble generation, it should be noted that the maintenance of ensembles (either multimodel, multiphysics, stochastic, or mixtures thereof) can become computationally burdensome and is inherently at risk for programmatic bugs and/or mistakes. As such, future work focused on a comparison of stochastically perturbed physics tendencies (SPPTs) with the multiphysics ensembles presented in this paper would be beneficial, as ensembles generated using SPPTs could potentially show substantial practical improvements by enabling the use of a single model with a single physics configuration.
For future work related to ensemble spread and accuracy, comparison of ensemble members with observations can provide better characterizations of the spread of the ensemble relative to that of the observations as well as enable optimization of the set of parameterizations to be used to improve subsequent deterministic forecasts. Further investigation using metrics associated with the skill of the ensemble members can provide valuable information regarding the reliability of the ensembles in forecasting applications. Additionally, the inclusion of perturbed initial condition/lateral boundary condition ensemble members, along with data assimilation approaches to remove spinup issues in the first 0–6 h of the simulation, will be considered. Regarding quantification and visualization of ensemble spread, the model uncertainty varies based on what atmospheric variable is being examined and the vertical level in question; therefore, true quantification of model spread must be accomplished using three-dimensional analysis. This in itself is a relatively straightforward undertaking since it only involves adding a third dimension to the underlying data matrix; however, the true difficulty in such an analysis involves visualization and interpretation of the results. As a result, future research involves the development and application of strategies for the operational quantification and visualization of three-dimensional ensemble uncertainty metrics.
In relation to the assessment and quantification of the role of subgrid-scale processes on the various simulations, additional work is planned to address the importance of parameterized processes (i.e., convective cloud cover and precipitation) for the various grid spacings and meteorological conditions. Such work will help provide scientific justification for the role of parameterized processes in ensemble spread, and will also help in defining optimum parameter schemes for simulating various types of weather events at the regional scale. Within this context, it would be interesting to assess ensembles at convection-allowing grid spacings to see if, for example, a 15–9-km ensemble can sufficiently estimate the PDF of a convection-allowing ensemble.
Acknowledgments
This work was funded through the U.S. Department of Army Research Laboratory (Agreement W911NF-14-2-0058). The authors thank the anonymous reviewers for their extremely helpful comments and suggestions on the manuscript.
REFERENCES
Aligo, E. A., Gallus W. A. Jr., and Segal M. , 2009: On the impact of WRF model vertical grid resolution on Midwest summer rainfall forecasts. Wea. Forecasting, 24, 575–594, doi:10.1175/2008WAF2007101.1.
Berner, J., 2013: WRF implementation details and version history of a stochastic kinetic-energy backscatter scheme (SKEBS). National Center for Atmospheric Research Tech. Note, Boulder, CO, 20 pp. [Available online at http://www.cgd.ucar.edu/~berner/skebs_in_wrf.pdf.]
Berner, J., Ha S.-Y. , Hacker J. P. , Fournier A. , and Snyder C. , 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 1972–1995, doi:10.1175/2010MWR3595.1.
Bernier, N., and Bélair S. , 2012: High horizontal and vertical resolution limited-area model: Near-surface and wind energy forecast applications. J. Appl. Meteor. Climatol., 51, 1061–1078, doi:10.1175/JAMC-D-11-0197.1.
Bonnardot, V., and Cautenet S. , 2009: Mesoscale atmospheric modeling using a high horizontal grid resolution over a complex coastal terrain and a wine region of South Africa. J. Appl. Meteor. Climatol., 48, 330–348, doi:10.1175/2008JAMC1710.1.
Bryan, G. H., Wyngaard J. C. , and Fritsch J. M. , 2003: Resolution requirements for the simulation of deep moist convection. Mon. Wea. Rev., 131, 2394–2416, doi:10.1175/1520-0493(2003)131<2394:RRFTSO>2.0.CO;2.
Buizza, R., 1997: Potential forecast skill of ensemble prediction and spread and skill distributions of the ECMWF Ensemble Prediction System. Mon. Wea. Rev., 125, 99–119, doi:10.1175/1520-0493(1997)125<0099:PFSOEP>2.0.CO;2.
Cavallo, S. M., Dudhia J. , and Snyder C. , 2011: A multilayer upper-boundary condition for longwave radiative flux to correct temperature biases in a mesoscale model. Mon. Wea. Rev., 139, 1952–1959, doi:10.1175/2010MWR3513.1.
Cavallo, S. M., Berner J. , and Snyder C. , 2016: Diagnosing model error from time-averaged tendencies in the Weather Research and Forecasting Model. Mon. Wea. Rev., 144, 759–779, doi:10.1175/MWR-D-15-0120.1.
Chen, C. S., Lin Y. L. , Peng W. C. , and Liu C. L. , 2010: Investigation of a heavy rainfall event over southwestern Taiwan associated with a subsynoptic cyclone during the 2003 Mei-Yu season. Atmos. Res., 95, 235–254, doi:10.1016/j.atmosres.2009.10.003.
Chen, F., and Dudhia J. , 2001a: Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Mon. Wea. Rev., 129, 569–585, doi:10.1175/1520-0493(2001)129<0569:CAALSH>2.0.CO;2.
Chen, F., and Dudhia J. , 2001b: Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part II: Preliminary model validation. Mon. Wea. Rev., 129, 587–604, doi:10.1175/1520-0493(2001)129<0587:CAALSH>2.0.CO;2.
Clark, A. J., Gallus W. A. Jr., and Chen T. C. , 2007: Comparison of the diurnal precipitation cycle in convection-resolving and non-convection-resolving mesoscale models. Mon. Wea. Rev., 135, 3456–3473, doi:10.1175/MWR3467.1.
Clark, A. J., Gallus W. A. Jr., and Chen T. C. , 2008: Contributions of mixed physics versus perturbed initial/lateral boundary conditions to ensemble-based precipitation forecast skill. Mon. Wea. Rev., 136, 2140–2156, doi:10.1175/2007MWR2029.1.
Clark, A. J., Gallus W. A. Jr., Xue M. , and Kong F. Y. , 2009: A comparison of precipitation forecast skill between small convection-allowing and large convection-parameterizing ensembles. Wea. Forecasting, 24, 1121–1140, doi:10.1175/2009WAF2222222.1.
Deng, A., and Coauthors, 2009: Update on WRF-ARW end-to-end multi-scale FDDA system. Preprints, 10th WRF Users' Workshop, Boulder, CO, National Center for Atmospheric Research, 1.9. [Available online at http://www2.mmm.ucar.edu/wrf/users/workshops/WS2009/abstracts/1-09.pdf.]
Done, J., Davis C. , and Weisman M. , 2004: The next generation of NWP: Explicit forecasts of convection using the Weather Research and Forecast (WRF) model. Atmos. Sci. Lett., 5, 110–117, doi:10.1002/asl.72.
Du, J., and Coauthors, 2004: The NOAA/NWS/NCEP Short Range Ensemble Forecast (SREF) system: Evaluation of an initial condition versus multiple model physics ensemble approach. Preprints, 20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., 21.3. [Available online at http://ams.confex.com/ams/pdfpapers/71107.pdf.]
Du, J., McQueen J. , DiMego G. , Toth Z. , Jovic D. , Zhou B. , and Chuang H. , 2006: New dimension of NCEP Short-Range Ensemble Forecasting (SREF) system: Inclusion of WRF members. Preprints, WMO Expert Team Meeting on Ensemble Prediction System, Exeter, United Kingdom, WMO. [Available online at http://www.emc.ncep.noaa.gov/mmb/SREF/WMO06_full.pdf.]
Du, J., DiMego G. , Jovic D. , Ferrier B. , Yang B. , and Zhou B. , 2015: Short range ensemble forecast (SREF) system at NCEP: Recent development and future transition. Preprints, 27th Conf. on Weather Analysis and Forecasting/23th Conf. on Numerical Weather Prediction, Chicago, IL, Amer. Meteor. Soc., 2A.5. [Available online at https://ams.confex.com/ams/27WAF23NWP/webprogram/Paper273421.html.]
Dudhia, J., 1989: Numerical study of convection observed during the Winter Monsoon Experiment using a mesoscale two-dimensional model. J. Atmos. Sci., 46, 3077–3107, doi:10.1175/1520-0469(1989)046<3077:NSOCOD>2.0.CO;2.
Dumais, R., Henmi T. , Passner J. , Jameson T. , Haines P. , and Knapp D. , 2004: A mesoscale modeling system developed for the U.S. Army. U.S. Army Research Laboratory Final Rep. ARL-TR-3183, 50 pp.
Dumais, R., Raby J. W. , Wang Y. , Raby Y. R. , and Knapp D. , 2012: Performance assessment of the three-dimensional wind field Weather Running Estimate-Nowcast and the three-dimensional wind field Air Force Weather Agency weather research and forecasting wind forecasts. U.S. Army Research Laboratory Tech. Note ARL-TN-0514, 30 pp.
Dumais, R., Kirby S. , and Flanigan R. , 2013: Implementation of the WRF four-dimensional data assimilation method of observation nudging for use as an ARL Weather Running Estimate-Nowcast. U.S. Army Research Laboratory Final Rep. ARL-TR-6485, 24 pp.
Dyer, J. L., 2011: Analysis of a warm-season surface-influenced mesoscale convective boundary in northwest Mississippi. J. Hydrometeor., 12, 1007–1023, doi:10.1175/2011JHM1326.1.
Eckel, F. A., and Mass C. F. , 2005: Aspects of effective mesoscale, short-range ensemble forecasting. Wea. Forecasting, 20, 328–350, doi:10.1175/WAF843.1.
Errico, R., and Baumhefner D. , 1987: Predictability experiments using a high-resolution limited-area model. Mon. Wea. Rev., 115, 488–504, doi:10.1175/1520-0493(1987)115<0488:PEUAHR>2.0.CO;2.
Ferrier, B. S., Jin Y. , Lin Y. , Black T. , Rogers E. , and DiMego G. , 2002: Implementation of a new grid-scale cloud and precipitation scheme in the NCEP Eta model. Preprints, 19th Conf. on Weather Analysis and Forecasting/15th Conf. on Numerical Weather Prediction, San Antonio, TX, Amer. Meteor. Soc., 10.1.
Fritsch, J. M., and Carbone R. E. , 2004: Improving quantitative precipitation forecasts in the warm season: A USWRP research and development strategy. Bull. Amer. Meteor. Soc., 85, 955–965, doi:10.1175/BAMS-85-7-955.
Grasso, L. D., 2000: A numerical simulation of dryline sensitivity to soil moisture. Mon. Wea. Rev., 128, 2816–2834, doi:10.1175/1520-0493(2000)128<2816:ANSODS>2.0.CO;2.
Hagedorn, R., Hamill T. M. , and Whitaker J. S. , 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part I: Two-meter temperatures. Mon. Wea. Rev., 136, 2608–2619, doi:10.1175/2007MWR2410.1.
Hamill, T. M., and Colucci S. J. , 1997: Verification of Eta–RSM short-range ensemble forecasts. Mon. Wea. Rev., 125, 1312–1327, doi:10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2.
Hamill, T. M., and Colucci S. J. , 1998: Evaluation of Eta–RSM ensemble probabilistic precipitation forecasts. Mon. Wea. Rev., 126, 711–724, doi:10.1175/1520-0493(1998)126<0711:EOEREP>2.0.CO;2.
Hamill, T. M., Hagedorn R. , and Whitaker J. S. , 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part II: Precipitation. Mon. Wea. Rev., 136, 2620–2632, doi:10.1175/2007MWR2411.1.
Hariprasad, K. B. R. R., Srinivas C. V. , Singh A. B. , Vijaya Bhaskara Rao S. , Baskaran R. , and Venkatraman B. , 2014: Numerical simulation and intercomparison of boundary layer structure with different PBL schemes in WRF using experimental observations at a tropical site. Atmos. Res., 145–146, 27–44, doi:10.1016/j.atmosres.2014.03.023.
Hart, K. A., Steenburgh W. J. , and Onton D. J. , 2005: Model forecast improvements with decreased horizontal grid spacing over finescale intermountain orography during the 2002 Olympic Winter Games. Wea. Forecasting, 20, 558–576, doi:10.1175/WAF865.1.
Hong, S., and Lee J. , 2009: Assessment of the WRF model in reproducing a flash-flood heavy rainfall event over Korea. Atmos. Res., 93, 818–831, doi:10.1016/j.atmosres.2009.03.015.
Houtekamer, P. L., Lefaivre L. , Derome J. , Ritchie H. , and Mitchell H. L. , 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev., 124, 1225–1242, doi:10.1175/1520-0493(1996)124<1225:ASSATE>2.0.CO;2.
Janjić, Z. I., 1994: The step-mountain eta coordinate model: Further developments of the convection, viscous sublayer, and turbulence closure schemes. Mon. Wea. Rev., 122, 927–945, doi:10.1175/1520-0493(1994)122<0927:TSMECM>2.0.CO;2.
Janjić, Z. I., 2000: Comments on “Development and evaluation of a convection scheme for use in climate models.” J. Atmos. Sci., 57, 3686–3686, doi:10.1175/1520-0469(2000)057<3686:CODAEO>2.0.CO;2.
Kain, J. S., 2004: The Kain–Fritsch convective parameterization: An update. J. Appl. Meteor., 43, 170–181, doi:10.1175/1520-0450(2004)043<0170:TKCPAU>2.0.CO;2.
Kain, J. S., and Coauthors, 2008: Some practical considerations regarding horizontal resolution in the first generation of operational convection-allowing NWP. Wea. Forecasting, 23, 931–952, doi:10.1175/WAF2007106.1.
Klocke, D., and Rodwell M. J. , 2014: A comparison of two numerical weather prediction methods for diagnosing fast-physics errors in climate models. Quart. J. Roy. Meteor. Soc., 140, 517–524, doi:10.1002/qj.2172.
Kober, K., Craig G. C. , Keil C. , and Dörnbrack A. , 2012: Blending a probabilistic nowcasting method with a high-resolution numerical weather prediction ensemble for convective precipitation forecasts. Quart. J. Roy. Meteor. Soc., 138, 755–768, doi:10.1002/qj.939.
Kong, F., Droegemeier K. K. , and Hickmon N. L. , 2006: Multiresolution ensemble forecasts of an observed tornadic thunderstorm system. Part I: Comparison of coarse- and fine-grid experiments. Mon. Wea. Rev., 134, 807–833, doi:10.1175/MWR3097.1.
Kong, F., Droegemeier K. K. , and Hickmon N. L. , 2007: Multiresolution ensemble forecasts of an observed tornadic thunderstorm system. Part II: Storm-scale experiments. Mon. Wea. Rev., 135, 759–782, doi:10.1175/MWR3323.1.
Lee, S. W., Lee D. K. , and Chang D. E. , 2011: Impact of horizontal resolution and cumulus parameterization scheme on the simulation of heavy rainfall events over the Korean Peninsula. Adv. Atmos. Sci., 28, 1–15, doi:10.1007/s00376-010-9217-x.
Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409–418, doi:10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2.
Leutbecher, M., and Palmer T. N. , 2008: Ensemble forecasting. J. Comput. Phys., 227, 3515–3539, doi:10.1016/j.jcp.2007.02.014.
Li, L., Li W. , and Jin J. , 2014: Improvements in WRF simulation skills of southeastern United States summer rainfall: Physical parameterization and horizontal resolution. Climate Dyn., 43, 2077–2091, doi:10.1007/s00382-013-2031-2.
Lin, Y.-L., Farley R. D. , and Orville H. D. , 1983: Bulk parameterization of the snow field in a cloud model. J. Climate Appl. Meteor., 22, 1065–1092, doi:10.1175/1520-0450(1983)022<1065:BPOTSF>2.0.CO;2.
Liu, H., and Xue M. , 2008: Prediction of convective initiation and storm evolution on 12 June 2002 during IHOP_2002. Part I: Control simulation and sensitivity experiments. Mon. Wea. Rev., 136, 2261–2282, doi:10.1175/2007MWR2161.1.
Liu, H., Zhang D. L. , and Wang B. , 2010: Impact of horizontal resolution on the regional climate simulations of the summer 1998 extreme rainfall along the Yangtze River Basin. J. Geophys. Res., 115, D12115, doi:10.1029/2009JD012746.
Liu, Y., Bourgeois A. , Warner T. , Swerdlin S. , and Hacker J. , 2005: Implementation of observation-nudging based FDDA into WRF for supporting ATEC test operations. Preprints, 6th WRF/15th MM5 Users' Workshop, Boulder, CO, National Center for Atmospheric Research, 10.7. [Available online at http://www2.mmm.ucar.edu/wrf/users/workshops/WS2005/abstracts/Session10/7-Liu.pdf.]
Lowrey, M., and Yang Z. , 2008: Assessing the capability of a regional-scale weather model to simulate extreme precipitation patterns and flooding in central Texas. Wea. Forecasting, 23, 1102–1126, doi:10.1175/2008WAF2006082.1.
Mass, C. F., Ovens D. , Westrick K. , and Colle B. A. , 2002: Does increasing horizontal resolution produce more skillful forecasts? Bull. Amer. Meteor. Soc., 83, 407–430, doi:10.1175/1520-0477(2002)083<0407:DIHRPM>2.3.CO;2.
Mlawer, E. J., Taubman S. J. , Brown P. D. , Iacono M. J. , and Clough S. A. , 1997: Radiative transfer for inhomogeneous atmosphere: RRTM, a validated correlated-k model for the long-wave. J. Geophys. Res., 102, 16 663–16 682, doi:10.1029/97JD00237.
Molteni, F., Buizza R. , Palmer T. N. , and Petroliagis T. , 1996: The ECMWF Ensemble Prediction System: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122, 73–119, doi:10.1002/qj.49712252905.
Nakanishi, M., and Niino H. , 2004: An improved Mellor–Yamada level-3 model with condensation physics: Its design and verification. Bound.-Layer Meteor., 112, 1–31, doi:10.1023/B:BOUN.0000020164.04146.98.
Nutter, P., Stensrud D. , and Xue M. , 2004: Effects of coarsely resolved and temporally interpolated lateral boundary conditions on the dispersion of limited-area ensemble forecasts. Mon. Wea. Rev., 132, 2358–2377, doi:10.1175/1520-0493(2004)132<2358:EOCRAT>2.0.CO;2.
Palmer, T., 2000: Predicting uncertainty in forecasts of weather and climate. Rep. Prog. Phys., 63, 71–116, doi:10.1088/0034-4885/63/2/201.
Petch, J. C., 2006: Sensitivity studies of developing convection in a cloud-resolving model. Quart. J. Roy. Meteor. Soc., 132, 345–358, doi:10.1256/qj.05.71.
Ratna, S. B., Ratnam J. V. , Behera S. K. , Rautenbach C. J. deW. , Ndarana T. , Takahashi K. , and Yamagata T. , 2014: Performance assessment of three convective parameterization schemes in WRF for downscaling summer rainfall over South Africa. Climate Dyn., 42, 2931–2953, doi:10.1007/s00382-013-1918-2.
Rife, D. L., and Davis C. A. , 2005: Verification of temporal variations in mesoscale numerical wind forecasts. Mon. Wea. Rev., 133, 3368–3381, doi:10.1175/MWR3052.1.
Roebber, P., Schultz D. , Colle B. , and Stensrud D. , 2004: Toward improved prediction: High-resolution and ensemble modeling systems in operations. Wea. Forecasting, 19, 936–949, doi:10.1175/1520-0434(2004)019<0936:TIPHAE>2.0.CO;2.
Sanyal, J., Zhang S. , Dyer J. , Mercer A. , Amburn P. , and Moorhead R. , 2010: Noodles: A tool for visualization of numerical weather model ensemble uncertainty. IEEE Trans. Visualization Comput. Graphics, 16, 1421–1430, doi:10.1109/TVCG.2010.181.
Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., doi:10.5065/D68S4MVH.
Stensrud, D. J., Bao J. , and Warner T. T. , 2000: Using initial condition and model physics perturbations in short-range ensemble simulations of mesoscale convective systems. Mon. Wea. Rev., 128, 2077–2107, doi:10.1175/1520-0493(2000)128<2077:UICAMP>2.0.CO;2.
Thompson, G., Rasmussen R. M. , and Manning K. , 2004: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part I: Description and sensitivity analysis. Mon. Wea. Rev., 132, 519–542, doi:10.1175/1520-0493(2004)132<0519:EFOWPU>2.0.CO;2.
Toth, Z., and Kalnay E. , 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317–2330, doi:10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2.
Toth, Z., and Kalnay E. , 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125, 3297–3319, doi:10.1175/1520-0493(1997)125<3297:EFANAT>2.0.CO;2.
Vukicevic, T., and Paegle J. , 1989: The influence of one-way interacting lateral boundary conditions upon predictability of flow in bounded numerical models. Mon. Wea. Rev., 117, 340–350, doi:10.1175/1520-0493(1989)117<0340:TIOOWI>2.0.CO;2.
Warner, T. T., Peterson R. A. , and Treadon R. E. , 1997: A tutorial on lateral boundary conditions as a basic and potentially serious limitation to regional numerical weather prediction. Bull. Amer. Meteor. Soc., 78, 2599–2617, doi:10.1175/1520-0477(1997)078<2599:ATOLBC>2.0.CO;2.
Weidle, F., Wang Y. , and Smet G. , 2016: On the impact of the choice of global ensemble in forcing a regional ensemble system. Wea. Forecasting, 31, 515–530, doi:10.1175/WAF-D-15-0102.1.
Weisman, M. L., Skamarock W. C. , and Klemp J. B. , 1997: The resolution dependence of explicitly modeled convective systems. Mon. Wea. Rev., 125, 527–548, doi:10.1175/1520-0493(1997)125<0527:TRDOEM>2.0.CO;2.
Weisman, M. L., Davis C. , Wang W. , Manning K. W. , and Klemp J. B. , 2008: Experiences with 0–36-h explicit convective forecasts with the WRF-ARW model. Wea. Forecasting, 23, 407–437, doi:10.1175/2007WAF2007005.1.
Xue, M., and Martin W. J. , 2006: A high-resolution modeling study of the 24 May 2002 dryline case during IHOP. Part I: Numerical simulation and general evolution of the dryline and convection. Mon. Wea. Rev., 134, 149–171, doi:10.1175/MWR3071.1.
Zhao, Q., Qin X. , Jin Y. , McLay J. , and Reynolds C. , 2015: Time-expanding sampling for ensemble-based data assimilation applied to conventional and satellite observations. Wea. Forecasting, 30, 855–872, doi:10.1175/WAF-D-14-00108.1.
Zupanski, D., Zhang S. Q. , Zupanski M. , Hou A. Y. , and Cheung S. H. , 2011: A prototype WRF-based ensemble data assimilation system for dynamically downscaling satellite precipitation observations. J. Hydrometeor., 12, 118–134, doi:10.1175/2010JHM1271.1.
