1. Introduction
The Weather Research and Forecasting Model (WRF; Skamarock et al. 2008) is formulated as a mesoscale model with grid nesting, which allows portions of the model domain to be simulated at higher resolution. If nesting is pursued to very fine scales, the large-eddy simulation (LES) approach can be used for the highest-resolution domains. Grid nesting, coupled with WRF’s many physics parameterizations, can provide more realistic boundary conditions for LES than are commonly used. The nested LES can, in theory, improve the accuracy of simulated flow parameters, compared to the outer domain, by using higher resolution. The inflow and nesting procedures can, however, contribute errors to the nested LES domain, potentially overshadowing the benefits of higher resolution. The nature and extent of these errors are the focus of this study.
WRF supports two options for nesting LES, two way, as discussed by Moeng et al. (2007), and one way, discussed here. Both have specific and independent applications. Until recently, most atmospheric LES studies have focused on homogeneous atmospheric boundary layers (ABLs), which permit periodic lateral boundary conditions. However, for inhomogeneous ABLs, different lateral boundary conditions are required. One-way nesting serves this need, via simulation on an outer, coarser domain, which provides realistic turbulent inflow for the nested domain, within which an inhomogeneous ABL can develop due to surface and/or forcing inhomogeneities. Two-way nesting, for which flow on the nest affects the outer domain, permits study of the effect of highly resolved ABL flows on mesoscale circulations.
While WRF (and similar models) provides the computational framework for nested simulations, many challenges involving flow and model parameters at nest interfaces remain. Keating et al. (2004) tested methods to instigate turbulence in laminar inflows used to force LES and found that large-scale structures at inlet boundaries strongly govern turbulence downstream. Piomelli et al. (2006) investigated LES of turbulent plane channel flow and noted significant effects on both the resolved and subfilter-scale (SFS) stresses arising from mesh changes. Vanella et al. (2008) examined isotropic turbulence and found that sudden mesh refinement introduces perturbations that could sometimes be reduced by filtering and turbulence model choices. They also found that sudden filter width discontinuities generated smaller errors than gradual ones, which, by maintaining larger eddy viscosities, delayed onset of smaller scales. Use of a dynamic eddy viscosity SFS stress model reduced errors near grid discontinuities, relative to a constant-coefficient model. Their simulations did not involve solid boundaries, as in the ABL.
Moeng et al. (2007) examined nested LES using WRF for both shear-driven and free-convective ABLs. Successful simulation required alteration of the SFS stress model, careful selection of nest size, and use of relaxation zones along nest edges. Their simulations produced similar turbulence structure and statistics in both domains, although wind speed profiles from the shear-driven simulations still showed departures from the expected similarity profile.
The present study complements these investigations by further examining the performance of fine LES nested within coarse LES, using WRF. We focus on the influence of the SFS stress model on near-surface flow properties. Several new SFS stress models recently implemented into WRF have shown improvements in single-domain simulations (Mirocha et al. 2010; Kirkil et al. 2012). Here, we explore their performance and seek to provide guidance for nested applications.
2. LES and the SFS stress model
The LES technique is based upon explicit resolution of energy-producing scales near the classic inertial range of three-dimensional turbulence. These scales provide most of the stress, turbulent kinetic energy, and transport, within a flow. The SFS stress model parameterizes the effects of unresolved scales, which primarily provide energy transfer for resolved scales. Mesoscale boundary layer schemes, in contrast, parameterize the effects of the entire turbulence spectrum, assumed to be unresolved and horizontally homogeneous, on vertical distributions. Because of the resolved turbulence in LES, and the role of mesh spacing in capturing turbulent structures and in formulating the length scale used by many SFS stress models, changes in mesh spacing at nest interfaces may more strongly impact nested LES than nested mesoscale simulations.
Nested LES applications using WRF will likely involve at least two LES domains, with the fine LES nested within a coarse LES, all possibly nested within a mesoscale simulation. This ensures that grid refinement ratios remain small. Errors arising from the mesoscale simulation, the bounding LES, and the nesting approach will all impact the fine LES. Here we investigate the coarse LES to fine LES transition. The more complicated problem of the mesoscale-to-LES interface, not addressed herein, will also require examination.
a. LES approach
b. Subfilter-scale stress models
The SFS stress model can strongly influence LES performance. Several SFS stress models using various formulations have been implemented into the Advanced Research Weather Research and Forecasting Model (WRF-ARW), version 3.1, to examine the sensitivity of nested LES performance to the SFS stress model.1 These models do not all account for compressibility; however, as their intended applications within WRF involve flows with low turbulent Mach numbers, the key parameter for compressibility effects on SFS stresses, compressibility effects can be ignored (Samtaney et al. 2001; Kosović et al. 2002). Detailed descriptions of these models may be found in Kirkil et al. (2012) and references therein; below we present a succinct overview of their formulations.
1) Constant-coefficient Smagorinsky models
WRF also contains a variant of Eq. (4) that uses SFS turbulence kinetic energy e. However, as the modifications to extend this model to nested LES, as proposed by Moeng et al. (2007), are not in the public WRF release at the time of this study, this model is not considered here.
2) Lagrangian-averaged scale-dependent dynamic Smagorinsky model
Well-known errors arising from the use of constant CS values in Eq. (4) can be ameliorated by computing it dynamically, using information from the smallest resolved scales (Germano et al. 1991). The Lagrangian-averaged scale dependent (LASD) model computes the scale dependence of CS using two explicit spatial (tophat) filters, then uses a power-law dependence to extrapolate CS at the subfilter scale. The LASD model is stabilized via Lagrangian averaging, accomplished using weighted time histories for the averaged quantities along path lines (Bou-Zeid et al. 2005, 2008).
3) Nonlinear backscatter and anisotropy model
In addition to the first term in the braces, which resembles the SMAG closure, the NBA1 model includes nonlinear terms (at the tensor level) that provide both SFS anisotropies and energy backscatter. These parameters are controlled by the backscatter coefficient Cb. The NBA13 model and those described in 2b(1) are included in the WRF release (NBA1 since version 3.2)
4) Dynamic reconstruction model
The eddy-viscosity component of the DRM is computed using the scale-invariant dynamic procedure of Wong and Lilly (1994). The eddy viscosity coefficients are stabilized using two applications of a tophat filter of width 4Δxi.
3. Methodology
This study examines the accuracy of nested LES in WRF, investigating errors arising from both the coarser bounding simulations providing lateral boundary conditions for the nests, and from the transition of flow and model parameters near nest interfaces. Understanding these errors is crucial to interpreting results in more complicated setups. These errors are herein identified using a simplified computational configuration. We simulate neutral flow over flat, rough terrain. A geostrophic wind Ug constant in time and height, oriented in the x direction, forces the flow, as in many LES investigations (e.g., Andren et al. 1994; Chow et al. 2005; Mirocha et al. 2010; Kirkil et al. 2012). A Coriolis parameter of f = 0.0001 s−1 (≈45°N) is used. Fifth- and third-order finite differencing is used for horizontal and vertical advection, respectively.
Simulations with an outer domain (d01) and one nest embedded within (d02) are conducted. Periodic lateral boundary conditions are used on d01, allowing d01 to provide turbulent inflow for d02. Nesting is one way, using a mesh refinement ratio of three for the horizontal grid spacing. WRF does not currently support mesh refinement in the vertical direction for concurrent nested simulations.
This study examines the evolution of flow beyond the coarse-to-fine interface as the flow enters and develops within d02. The d02 solutions are evaluated against stand-alone (SA) simulations, conducted on computational meshes identical to d02, but using periodic lateral boundary conditions. Using identical computational domains, geostrophic forcing, and surface roughness, differences in flow properties between the d02 and SA simulations must arise from aspects of the grid nesting procedure. We examine the development of several flow parameters within d02 to understand the nature of those errors, and to identify potential mitigation strategies.
Different mesh spacings in the horizontal (Δx = Δy) and vertical (Δz) directions were prescribed to investigate sensitivity to these parameters. Horizontal domain sizes varied due to the different mesh spacings; however, domain depths were all approximately 1400 m, varying slightly due to different values of Δz1, and use of a stretched vertical mesh (increasing Δz by 5% per grid cell with height). Physical and computational dimensions of all domains are shown in Table 1. The upper boundary condition is w = τij = 0 (w is the vertical velocity). A Rayleigh damping layer with a coefficient of 0.003 s−1 is applied to the upper 400 m of each domain.
Physical and computational dimensions of all simulation domains, with Δx and Δz1 the horizontal and vertical grid spacings, respectively; α = Δx/Δz1 the grid aspect ratio; Lx, Ly, and Lz are the physical dimensions; and nx, ny, and nz are the number of computational grid points in the x, y, and z directions, respectively. Superscript 1 indicates the first grid point above the surface.
Near-wall stresses for dynamic models
The LASD model has shown good near-surface performance in a pseudospectral solver (Bou-Zeid et al. 2005); however, in WRF, as with the DRM, near-wall stresses are underpredicted (Kirkil et al. 2012), likely due to numerical damping near the wall of the smallest resolved scales required by the dynamic approach. This disparate performance of the LASD model using different numerical methods underscores the importance of the numerical solution procedure and its impact on SFS model performance. The LASD model in WRF, therefore, also utilizes the near-wall stress model given by Eq. (8), with slightly different parameter values.
Previous LES studies using WRF have found that, for neutral flow over flat terrain, with z0 = 0.1 m, the SMAG and NBA1 models’ near-surface wind speed profiles agree best with the expected logarithmic similarity solution using computational meshes with grid aspect ratios at the surface of α = Δx/Δz1 ≅ 4 (Mirocha et al. 2010; Kirkil et al. 2012). With these values of α and z0, the LASD and DRM achieve good near-surface performance in WRF using 
Figure 1 shows time- and plane-averaged profiles of the vertical flux of x momentum τ13 from all of the SFS models, including 
Time- and plane-averaged profiles of components of the vertical flux of x momentum τ13 from simulations using periodic lateral boundary conditions, and using horizontal grid spacings of Δx (=Δy) = 33 m, and vertical grid spacings at the surface of Δz1 ≅ 8.25 m, yielding grid aspect ratios at the surface of α = Δx/Δz1 ≅ 4.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
4. Results
All simulations were initialized dry, with mean u and υ profiles matching the specified geostrophic forcing, and potential temperature profiles of θ = 300 K. Small perturbations, φ ∈ [−0.5, 0.5], obtained from a pseudorandom uniform distribution, were added to u, υ, and θ as a decreasing cubic function of height up to 500 m, to instigate turbulence. The vertical spacings of WRF’s pressure-based vertical coordinate were approximated from prescribed Δz values using the hypsometric equation (e.g., Holton 1992). Surface heat and moisture fluxes were set to zero. No atmospheric or land surface process models were used.
All of the simulations were run for 12 h of physical time prior to analysis. For all simulations the first 8 h used the LASD SFS stress model. For simulations using different SFS models, the SFS model was switched for the remaining 4 h, reducing computation while providing ample time for the simulations to equilibrate to the different SFS stress models.
a. Flow structures
Figure 2 shows the flow at the beginning of the 12th hour from simulations on computational meshes as described above. The mean flow direction, given by the orientation of the resolved structures, indicates rotation near the surface due to the Coriolis acceleration.
Instantaneous contours of the x-direction velocity component u from the (a),(d),(g),(j) outer (d01) and (b),(e),(h),(k) nested (d02) domains of nested simulations, and from (c),(f),(i),(l) stand-alone simulations (SA), at 97.4 m, corresponding to the 10th computational grid point above the surface; Δx = 99 m for d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 8.25 m, yielding α ≅ 12 on d01 and α ≅ 4 on d02 and SA.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Contours show instantaneous cross sections of the x component of velocity u at 97.4 m (the 10th computational grid point) above the surface, using Ug = 10 m s−1 and z0 = 0.1 m. Results are shown for each SFS stress model, SMAG (top), NBA1 (top middle), LASD (bottom middle), and DRM0 (bottom). The left, middle, and right columns show d01, d02, and SA, respectively. The dotted rectangles in d01 show the locations of d02. These simulations used a mesh refinement ratio of 3, with Δx = 99 m on d01 and Δx = 33 m d02 and SA. As at the time of this study, WRF supported mesh refinement for only the horizontal directions, but not the vertical (for concurrently nested simulations), all of the domains used Δz1 ≅ 8.75 m. As such, the same value of α could not be specified on both the coarse and fine domains. [For the results shown in Figs. 2–12, α ≅ 4 was specified on d02 (and SA), with α ≅ 12 on d01. For these simulations, the LASD and DRMO values of cw on d01 were increased to 0.9 to account for the large d01 α values. The sensitivity of simulation results to cw is examined in section 4b.]
Significant differences are apparent in Fig. 2 regarding the characteristics of the resolved structures produced by the various SFS stress models. These differences are most notable on d01 and near the d02 inlet boundaries. The d01 solutions all contain longer spatial correlations than the SA solutions, also discernable on d02. The SMAG model produces the largest structures, while the dynamic models (LASD and DRM0) produce the smallest, with the NBA1 model generally in between.
Another difference is the rate at which the flow structures transition from resembling those on d01 to those on SA during transit through d02 (middle column of Fig. 2). Each flow develops smaller structures with increasing distance through d02; however, the distances over which these changes occur vary. For the simulations depicted in Fig. 2, at the specified height, the dynamic models transition most rapidly, with the LASD model somewhat faster than the DRM0, the NBA1 model slightly slower, and the SMAG model slowest of all. Reducing the SMAG model’s Cs from its default WRF value of 0.25 to 0.18 yielded little difference in these results.
Figure 3 depicts the same information as Fig. 2 for simulations forced with Ug = 5 m s−1. For each model, flow equilibration on d02 is somewhat slower. The SMAG model significantly delays the onset of smaller-scale turbulence due to the failure to represent much turbulence on d01. The d02 SMAG solution does not capture the scales apparent on the SA domains until close to its outflow boundaries. The other solutions, each of which use less, or more variably dissipative SFS stress models, are able to capture small scales of turbulence much more rapidly on d02. While noting that Δx = 99 m on d01 is near the coarse-resolution limit for acceptable LES of these flows, Fig. 3 highlights the increasing role of the SFS model as this limit is approached. Figure 3 also demonstrates that, provided the coarse LES domain adequately resolves the large scales of turbulence, reducing the mean wind speed does not appreciably change the distance required for smaller-scale structures to emerge on the fine LES nested within. Reducing the mean wind speed increases the advection time of the eddies, but also reduces the turbulence levels; this increases eddy turnover times and slows the formation of smaller structures on the finer grid due to the downscale cascade process.
As in Fig. 2, but forced with a geostrophic wind speed Ug reduced by a factor of 2, to 5 m s−1.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Figure 4 depicts the same information as Fig. 2 from higher-resolution simulations, for which the grid spacings in all three directions were reduced by a factor of 3 relative to those of Fig. 2. The cross sections are again shown at the 10th model grid point above the surface, in this case corresponding to a height of 32.5 m. Strong morphological similarities between the resolved structures among Figs. 2 and 4 are apparent, despite differences in the mesh resolution, and height above the surface, indicating that the equilibration rates of the flow structures within d02 depend more strongly upon the number of computational grid points than on physical distance. The equilibration distances on the finer mesh were much shorter near the 97.4-m height of Fig. 4, which occurred at twice the number of grid points above the surface (not shown).
As in Fig. 2, but from simulations with grid spacings reduced by a factor of 3 in each direction, to Δx = 33 m for d01 and Δx = 11 m for d02 and SA, with Δz1 ≅ 2.75 m. Horizontal domain dimensions were also reduced, so that the same number of grid points as the simulations shown in Fig. 2 was used in each lateral direction. Contours are also shown at the 10th computational grid point above the surface, here corresponding to a height of 32.5 m.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Higher-resolution simulations were also conducted for the lower wind speed case (not shown). Morphological similarities were again apparent among the NBA1, LASD, and DRM0 simulations conducted at the different resolutions. The higher-resolution SMAG simulations, unlike those in Fig. 4, were able to capture turbulence structures on d01.
Spectra
The different equilibration rates of flow structures on d02 among the SFS stress models can also be examined from velocity spectra, which depict the relative occurrences of resolved structures of different sizes. Figure 5 shows spectra of u, taken in the y direction, from each domain. The d01 and SA spectra are indicated by dashed and solid gray lines, respectively, while black lines depict six x locations within d02. The averaging time was 4 h beginning at hour 12.
(a)–(d) Mean spectra of u taken in the y direction, at two heights above the surface. Spectra from d01 (thick gray dashed) and SA (thick gray solid) are time- and x-direction averages, while those from d02 are averaged in time only, and plotted at six distances from the d02 x-direction inlet boundary, as indicated in (b). Simulations used Δx = 99 m on d01 and Δx = 33 m on d02 and SA, with Δz1 ≅ 8.25 m, yielding α ≅ 12 on d01 and α ≅ 4 on d02 and SA.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
While the d01 and SA spectra are averaged both in time and in the x direction, the d02 spectra are averaged in time only, at each x location, to show their evolution from the d02 x-direction inflow boundary. The d02 and SA spectra are computed using u velocities from grid points [30 < j < 210], where j is the y-direction index (see Figs. 2–4), yielding 180 grid points, hence 91 Fourier modes. Values within 30 grid points of the j boundaries were omitted to reduce low-frequency bias near the j boundaries on d02. Spectra from SA and d01 are taken from the same y locations, with those from d01 using one-third as many j indices due to coarser grid spacing. Hamming windows are applied to all spectra calculations.
Figure 5 confirms the conclusions gleaned from Figs. 2–4 regarding the sizes of the spatially correlated structures on different domains and their equilibration rates within d02. The SMAG model equilibrates most slowly, and the dynamic models most rapidly. While each simulation eventually develops similar smaller-scale spectral content on d02 to that on SA, reduction of the sizes of the large structures is much slower, with the low-frequency biases persisting throughout d02. All simulations produce greater low-frequency power on d01 than on d02 and SA, a feature most pronounced using the SMAG model. Energies at higher frequencies are greater using the dynamic models, due to their lowering of the dissipation rate. The equilibration rates differ with height as well, with lower elevations showing shorter adjustment ranges for the dynamic models.
The different rates of equilibration of the spectra among the SFS stress models can also be investigated by plotting the differences between the spectral power on d02 and SA, at a frequency resolved on d02 and SA, but not on d01, as a function of distance within d02. Figure 6 shows the relative difference, ΔF = (Fx,d02 − FSA)/FSA, where F corresponds to k2L/2π = 30, which is well resolved on d02 and SA, but not on d01 (see Fig. 5); Fx,d02 is the time average of the y-direction spectra at each x location on d02; and FSA is the time- and x-direction average on SA. Here L is the distance over which the spectra were computed. Spectra are shown from both the coarser- (Figs. 6a–c) and finer-resolution (Figs. 6d–f) simulations, shown in Figs. 2 and 4. Three heights above the surface are shown to indicate the height dependence of the equilibration rate for each model, and to show differences due to horizontal resolution. Because of significant variability in the spectral power with distance in d02, Fx,d02 was smoothed using 11-point running means in the x direction to better elucidate trends.
Relative differences in spectral power coefficient, ΔF = (Fx,d02 − FSA)/FSA, with F corresponding to k2L/2π = 30, between simulations on d02 and SA, as functions of distance from the d02 x-direction inlet boundary; Fx,d02 are the time-average values at different x locations within d02, while FSA are SA time- and plane-average values. (a)–(c) Spectra from simulations using Δx = 99 m for d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 8.25 m for all domains, as in Fig. 2. (d)–(f) Simulations on domains for which the grid spacings were reduced by a factor of 3 in all directions, as in Fig. 4. All simulations used grid aspect ratios of α ≅ 12 on d01 and α ≅ 4 on d02 and SA.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Each d02 solution eventually develops similar power at k2L/2π = 30 to that on SA, however, at different rates. While the equilibration rates on d02 at each height are similar between the coarser- and finer-resolution simulations, the physical distances differ by a factor of 3. The LASD model generally exhibits the most rapid equilibration at the lower heights, with the NBA1 and DRM0 somewhat slower, and the SMAG slowest of all. At greater heights, the equilibration rates of the NBA1 and LASD models become comparable. At the greatest heights, the SMAG simulations significantly overpredict spectral power on d02, relative to that on SA, likely due to the gradual downscale cascade of the anomalously large structures advecting into d02 from d01. The DRM, in contrast, underpredicts spectral power on d02 relative to that on SA. One hypothesis for the DRM’s behavior is that the RSFS component, by modeling stresses arising from scales between the explicit and implicit filters, suppresses formation of the smallest resolvable scales (see the resolved component of τ13 in Fig. 1d in relation to the other models), hence, delaying formation of smaller eddies from the downscale cascade, upon entry into d02. Nearer to the surface, surface interactions contribute to the breakup of larger eddies, augmenting the formation of smaller scales.
The dependence of the spectral equilibration rate on Ug and z0 was examined with additional simulations using Ug = 20 m s−1 and z0 = 0.01 and 0.2 m, on the coarser-resolution domains (not shown). For both parameters, the largest and smallest values produced the shortest and longest equilibration distances, respectively; however, their effects on the results were weak. For the SMAG model, which, for the Ug = 5 m s−1 had so little resolved structure on d01 that equilibration on d02 was extremely slow (Figs. 3a,b), only small differences were observed between the two higher wind speed cases. These results suggest that if a flow has sufficient resolved turbulence at the inlet boundaries, development of smaller-scale structure on d02 is much more strongly governed by the SFS stress model than the wind speed or z0.
b. Mean profiles
In addition to smaller scales of turbulence, other flow parameters show different evolutions within d02 with different SFS stress models. Figure 7 shows profiles of mean wind speed U on each domain. The domain-averaged values on d01 and SA are indicated by the dashed and solid gray lines, respectively, with the black lines again depicting time averages at different x locations within d02. The averaging time was again 4 h beginning at hour 12.
(a)–(d) Profiles of mean wind speed U in the lower ABL, from simulations using Δx = 99 m for d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 8.25 m, yielding α ≅ 12 on d01 and α ≅ 4 on d02 and SA. Profiles on d01 and SA are averaged in time and over all x and y grid points. Profiles on d02 are averaged in time and in the y direction, over [30 < j < 210], at six distances from the d02 x-direction inlet boundary, as indicated in (b).
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
The most obvious difference among the various SFS stress models is the range of discrepancies between the d01 and SA values, and the correspondingly slower equilibration for the SMAG and NBA1 SFS models that feature the larger discrepancies. The large discrepancies between the d01 and SA solutions are due primarily to the different α on the different domains, with α ≅ 12 on d01 and α ≅ 4 on SA (and d02). Adjusting cw in Eq. (8) for the LASD and DRM0 simulations significantly improves the prediction of U on d01; however, this is not feasible with the SMAG and NBA1 simulations, which do not possess an equivalent parameter.
Figure 8 shows the influence of the value of cw used on d01 on both the domain-averaged U profiles on d01 (Figs. 8a,c) and the time-averaged d02 U profiles far from the x-direction inlet boundary (Figs. 8b,d). The d02 solution is seen to be strongly impacted by the cw value on d01 due to d01 providing lateral boundary conditions for d02, even though the optimal value of cw corresponding to α ≅ 4 is used on d02. Using the value of cw optimized for α ≅ 4 on domains with α ≅ 12 significantly degrades those models’ performances, resulting in similar equilibration rates to those of the SMAG and NBA1 models (not shown).
(a)–(d) Profiles of U from LASD and DRM0 simulations, using three different values of the near-wall stress model coefficient cw on d01, relative to its value on d02 and SA. The simulations used Δx = 99 m for d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 8.25 on all domains, yielding α ≅ 12 on d01 and α ≅ 4 on d02 and SA. The cw values used on d02 and SA, optimized for α ≅ 4, are shown in (b). The different values of cw specified on d01, shown in (a), compensate for the larger α values on d01. (a),(c) Time- and plane-averaged profiles on d01, corresponding to the three cw values and (b),(d) the corresponding time- and y-averaged profiles on d02, from i = 320, which is near the x-direction exit boundary. The thick gray lines depict time- and plane-averaged SA solutions.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
For the DRM0 simulations, specifying cw = 0.9 on d01 provides close agreement between the SA solution and those on both d01 and d02. For the LASD simulations, however, while specifying cw = 0.9 on d01 provides close agreement between the SA and the d02 solutions, the d01 solutions contain significant errors. While this study focuses on flow near the surface, we note that the values of cw that yield the closest agreement between d02 and SA below 120 m do not maximize agreement farther aloft, where the d02 profiles gradually converge to those from d01 with increasing height (not shown).
Figure 9 depicts the same information as Fig. 7, for the higher-resolution cases, showing the same patterns among the models, however, with the significant adjustments of the d02 profiles confined to a narrower layer above the surface. The significant changes of wind speed within d02 are confined near the surface due to the total mass flow within d02 being constrained by the d01 solution. For the higher-resolution LASD simulation, the bias of the d02 U profile could be removed by further increasing cw on d01, albeit with degraded agreement on d01 (see Figs. 8a,b).
(a)–(d) Profiles of U, as in Fig. 7, but from simulations with grid spacings reduced by a factor of 3 in each direction, to Δx = 33 m for d01 and Δx = 11 m for d02 and SA, and with Δz1 ≅ 2.75 m for all domains, yielding α ≅ 12 on d01 and α ≅ 4 on d02 and SA.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Figure 10 shows the U profiles in the lower ABL relative to the log law, the expected similarity solution, where U is a logarithmic function of z. Profiles of U scaled by 
(a)–(d) Mean profiles of wind speed U scaled by surface friction velocity u*, as functions of height above the surface scaled by the effective boundary layer depth H. The simulations used Δx = 99 m for d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 8.25 on all domains, yielding α ≅ 12 on d01 and α ≅ 4 on d02 (and SA). Gray lines depict time- and plane-averaged profiles from d01 and SA, as well as the expected similarity profile (the log law), as indicated in (a). Black lines show nested domain (d02) profiles, as in Fig. 7.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Agreement with the log law on SA and d02, where α ≅ 4, is good for all but the SMAG model, which exhibits characteristic overshoot near the surface (e.g., Mirocha et al. 2010; Kirkil et al. 2012). The SMAG and NBA1 simulations each show poor agreement on d01, where α ≅ 12. The good agreement of the LASD and DRM0 solution with the log law on d01 is due to the larger value of cw used on d01. Without this adjustment, agreement with the log law on d01 is comparable to or worse than the SMAG and NBA1 solutions (not shown).
A benefit of the LES technique is that most of the stresses within the flow are resolved, (except near the surface), which can, in principle, lead to more accurate prediction of the total stress than provided by mesoscale boundary layer schemes. Figure 11 depicts vertical profiles of the total (resolved plus SFS) vertical flux of x momentum τ13. The resolved component of τ13 was computed as 
As in Fig. 7, but for the total (resolved + SFS) vertical flux of x momentum τ13.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Figure 11 shows that the d01 and SA τ13 profiles are each nearly linear, as expected. Agreement of the magnitudes of τ13 predicted on d01 and SA is closer for the LASD and DRM0 simulations, again due to the increase of cw on d01, than for the SMAG and NBA1 models, for which the large α value on d01 results in overprediction of the d01 stress. As with the U profiles, without the d01 cw correction, prediction of the d01 τ13 profiles on d01 by the dynamic models is comparable or worse than the SMAG and NBA1 solutions (not shown).
Figure 11 shows considerable changes in the τ13 profiles at different locations within d02 for each model. Upon entrance of the flow into d02, the magnitudes of τ13 decrease significantly at the first few grid points above the surface. In the SMAG and NBA1 models, this reduction arises from explicit dependence on the length scale l, which changes by a factor of 
The dynamic models base their SFS stresses upon resolved velocities near the grid scale. The lack of these resolved scales near the d02 inlet boundary, which have not yet formed, results in those models predicting smaller values of CS, hence, smaller SFS stresses. Lagrangian averaging reduces this problem since its integration (averaging) time is constructed to reduce the relative weights of the local coefficients when they are very small, thus retaining an averaged coefficient that is large enough to produce nonnegligible SFS stresses. In addition, the height of the near-wall stress parameterization also depends on Δx, which results in an immediate reduction of the magnitudes of the near-wall stresses at the inlet boundaries as well.
Irrespective of the mechanism, the imbalances of τ13 (and τ23) between the surface and the lower ABL decelerate the near-surface flow upon entry into d02 for each model (see e.g., Figs. 7 and 9), via reduced momentum transport from aloft. The magnitudes of the near-surface τ13 deficits attenuate with passage through d02, as smaller scales of turbulence develop and influence both the SFS and resolved stresses. However, each of the τ13 profiles retains significant biases in relation to the SA values.
Another benefit of the LES technique is that, as with the stresses, the scales of motion containing most of the turbulence kinetic energy, 
As in Fig. 7, but for resolved turbulent kinetic energy K.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Figure 12 shows significant variability in the resolved component of K predicted by each SFS stress model. Both the SMAG and NBA1 simulations feature large differences in both the heights and magnitudes of the K maxima on d01 versus SA, while those differences are smaller for the dynamic models. The DRM0 simulations feature the smallest resolved K values overall on each domain, due to the explicit filter that smoothes the resolved velocity field (as discussed in relation to Fig. 6). On d02, as with τ13, significant increases in the magnitudes of K are observed for each model, but especially SMAG and NBA1. The overpredictions of K on d02 eventually begin to decrease for all, except for the SMAG simulations, with increasing distance within d02. While strongly impacting the U profiles, varying cw on d01 for the dynamic models has little effect on the large τ13 and K overpredictions on d02 (not shown).
Dependence on α
Because of the large biases from the SMAG and NBA1 models arising from large α values on d01, and the need for larger d01 cw values for the dynamic models, simulations using identical forcing, but on meshes using α ≅ 4 on d01 and α ≅ 1.33 on d02 (and SA) were conducted. For this configuration, the optimal α value occurs on d01, rather than d02. One drawback of smaller α is that, for the same Δx, Δz1 is increased, in this case from 8.25 to 24.75 m. One potential benefit of smaller α, however, relevant to complex terrain, is reduced errors in the horizontal gradients computed along WRF’s terrain-following grid (e.g., Lundquist et al. 2010).
Figure 13 shows U profiles, similar to Fig. 7, from the smaller-α simulations. Comparison with Fig. 7 shows similar characteristics, albeit slower convergence on d02 for the SMAG simulation. The good agreement of the dynamic model simulations was achieved using the same value of cw optimized for α ≅ 4, on all domains, showing that dependence of the dynamic models’ U profiles on cw is weaker for smaller α values.
Profiles of U, as in Fig. 7, but from simulations computed on meshes using smaller aspect ratios α. Simulations used Δx = 99 m for d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 24.75, yielding α ≅ 4 on d01 and α ≅ 1.33 on d02 and SA.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Figure 14 shows U profiles from simulations with the mesh spacing reduced by a factor of 3 in each direction, on each domain, relative to those of Fig. 13. As with the larger-α simulations (Figs. 7 and 9), the use of finer mesh resolution for the smaller-α simulations again reduces discrepancies between the U profiles on the various domains. We note that the mesh used in Fig. 14 has the same Δz as the coarser-resolution, larger-α simulations shown in Fig. 7.
Mean U profiles, as in Fig. 13, from simulations using smaller α domains, but with the grid spacing reduced by a factor of 3 in each direction, with Δx = 33 m for d01 and Δx = 11 m for d02, and Δz1 = 8.25 m for all domains, yielding α ≅ 4 on d01 and α ≅ 1.33 on d02 and SA.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
The accuracy of the d02 and SA U profiles using the dynamic models can be further improved using smaller values of cw on d02 and SA due to the above discussed dependence of the near-surface flow speed on the value of cw relative to α (not shown). This sensitivity causes the disagreements among both the LASD and, more noticeably, the DRM0 U profiles shown in Fig. 14, which are amplified relative to those shown in Fig. 13 by the finer mesh resolution. As Figs. 7 and 8 demonstrate that these discrepancies are correctable via adjustment of cw, Fig. 14 shows the potential for errors from using unadjusted values. As discussed in relation to Fig. 8, the d02 U profiles from the dynamic models shown in Fig. 14 also converge with those from d01 farther aloft (not shown).
Figure 15 shows the agreement of the U profile with the log law, as in Fig. 10, from simulations with Δx = 99 (as in Fig. 13) and smaller α. Unlike the larger-α simulations, for which the SMAG and NBA1 solutions showed the greatest departures from the log law on d01, here the d01 profiles are the most logarithmic, since α ≅ 4 on d01, and the more significant departures now occur on d02 and SA. Again, agreement between the SA and d02 profiles can be improved for the dynamic model simulations using a slightly smaller value of cw on d02 (and SA, not shown).
As in Fig. 10, but from simulations using smaller-α domains, with Δx = 99 m for d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 24.75, yielding α ≅ 4 on d01 and α ≅ 1.33 on d02 and SA.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
Figure 16 shows profiles of τ13 from coarser-resolution, smaller-α simulations. Comparison with larger-α simulations (Fig. 11) shows little improvement from smaller-α domains. Likewise, the K values from smaller-α domains show little improvement over the larger-α simulations (not shown).
As in Fig. 11, but from simulations using smaller-α domains, with Δx = 99 m for d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 24.75, yielding α ≅ 4 on d01 and α ≅ 1.33 on d02 and SA.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
c. Equilibration rates and errors on d02
To further examine the evolution of flow parameters near the surface within d02, the relative differences between the d02 and SA values of U and K are computed analogously to those of the spectral power coefficient shown in Fig. 6.
Figure 17 shows the relative difference of U, ΔU = (Ux,d02 − USA)/USA, at three heights above the surface within the lowest 100 m, as a function of distance within d02, using both larger-α (Figs. 17a–c) and smaller-α1 (Figs. 17d–f) domains. Here, Ux,d02 is the time- and y-direction average on d02, at each x location, and USA is the time and plane average on SA. Averaging time was again four hours beginning at hour 12. The larger-α LASD and DRM0 simulations (top panels) used larger values of cw = 0.9 on d01, to account for the large value of α ≅ 12, with their optimal values for α ≅ 4 used on d02 and SA. On the smaller α domains (bottom panels), those models used the values optimized for α ≅ 4 on all domains.
Relative differences in mean wind speed, ΔU = (Ux,d02 − USA)/USA, between d02 and SA, as functions of distance from the d02 x-direction inlet boundary, at three heights above the surface, as in Fig. 6. (a)–(c) Results from simulations using Δx = 99 on d01 and Δx = 33 m for d02 and SA, with Δz1 ≅ 8.25 on all domains, yielding α ≅ 12 on d01 and α ≅ 4 on d02 and SA. (d)–(f) Results from simulations using the same Δx on each domain, but with Δz1 ≅ 24.75 on all domains, yielding α ≅ 4 on d01 and α ≅ 1.33 on d02 and SA. The near-wall stress model parameter cw for the DRM0 and LASD simulations was increased to 0.9 on d01 to account for large d01 α values in (a)–(c).
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
As with the U profiles shown in Figs. 7 and 13, the equilibration process within d02 depends strongly on the SFS stress model, however, some elements are common to all simulations. The U near the surface decelerates upon entry into d02 for each simulation due to the mechanisms described in section 4b. However, the extents of the departures and their equilibration rates differ markedly. The SMAG simulations generally produce the largest discrepancies, and equilibrate more slowly. The NBA1 solutions also feature large discrepancies, however, those equilibrate more quickly, eventually agreeing closely with the SA values. The dynamic models generally achieve the smallest discrepancies near the inlet boundaries, and the fastest equilibrations.
Figure 18 shows ΔK, defined analogously to ΔU, using both larger-α (Figs. 18a–c) and smaller-α (Figs. 18d–f) domains, as in Fig. 17. Differences in K between d02 and SA are generally much larger than for U, but unlike U, increase with height. The ΔK also shows some dependence on the SFS stress model. The SMAG and NBA1 simulations again produce larger ΔK, and slower equilibration, with the SMAG the slowest. The dynamic models again produce smaller, though still considerable ΔK values, and faster equilibration.
As in Fig. 17, but for resolved turbulent kinetic energy K.
Citation: Monthly Weather Review 141, 3; 10.1175/MWR-D-11-00263.1
5. Discussion and conclusions
Successful nesting of a higher-resolution LES within a coarser-resolution LES, in WRF, is challenging. The approach used in this study improved the simulation of some flow parameters, but not others. The nested fine LES (d02) captured smaller scales of turbulence, and generally improved profiles of average wind speed U near the surface, over the coarse LES (d01) providing it with inflow. Performance relative to the stand-alone solution (SA) varied, depending upon domain configuration and the SFS stress model. For some parameters, the d02 solution showed poorer agreement with the SA solution than the coarse LES solution (d01), due to errors contributed by the nesting approach. While more sophisticated SFS stress models performed better in many respects, significant errors remained in key parameters, within all of the nested domains, regardless of domain configuration or SFS stress model. Reducing the errors common to all simulations investigated herein will likely require changes to other aspects of the WRF, or the nesting strategy, beyond the SFS stress model. In particular, this study suggests the potential benefits of a method to seed the inflow into the smaller nested domain with the missing scales, synthetically or otherwise. Some further tests were performed to complement the analysis and are summarized below.
The vertical flux of x momentum τ13 was generally not well predicted on nested domains (d02), particularly near the inlet boundaries, because of immediate reductions of the magnitudes of the SFS stresses above the surface upon entry into d02. The sudden imbalances between the surface stresses and the SFS stresses just above decelerate the near-surface flow. The τ13 errors on d02 generally diminish with increasing distance from the inlet boundary, but remain significant.
The impacts of abrupt changes of the mixing length l used by the SMAG and NBA1 models, at the nest boundaries, were examined by instead reducing l gradually. Consistent with Vanella et al. (2008), this approach had little impact on the errors on d02, and delayed the onset of smaller scales (not shown). We believe this delay results from a lag between the decrease of the SFS stress component, and the emergence of turbulence structures that increase the resolved component, resulting in temporary deficits of total stress. Abruptly reducing l creates larger deficits near the inflow boundaries, however, those recover more quickly than when l is gradually reduced; gradual reduction lessens the magnitude of those deficits, but spreads them over greater distances.
For the dynamic models, as the near wall stresses from Eq. (8) are similar in magnitude to resolved and SFS components near the surface (see Figs. 1c,d), simulations were conducted holding the height factor hw constant, thus removing its dependence on the horizontal grid spacing. While this approach reduced τ13 and U anomalies near the d02 inlet boundaries somewhat, other anomalies persisted (not shown). For the larger-α simulations, reducing hw on d01 to equal that on d02 (and SA) underpredicted the near-surface stress on d01. Increasing the coefficient cw on d01 still produced errors on d01, by distributing the stronger stress over too shallow a layer, and also reintroduced abrupt changes at the nest interfaces. For the smaller-α simulations, increasing hw on d02 (and SA) to the d01 value resulted in an excessively deep layer of near-wall stress on d02, which, when countered by reducing cw, again created anomalies on d01, and reintroduced abrupt changes at nest interfaces (not shown).
Significant overpredictions of the resolved turbulence kinetic energy K were observed for all simulations, irrespective of changes of α or the near-wall stress model parameters. A proposed mechanism for the overpredictions of K (and τ13) on d02 is that the large structures entering d02 from d01 do not have scales responsible for the downscale energy cascade immediately upon entering the finer d02 mesh. As such, the resolved K associated with those structures grows until smaller scales form. The dynamic models, which feature both the least large-scale bias on d01, and the fastest evolutions on d02, produce smaller K anomalies overall.
The SMAG and NBA1 models were more robust to changes in the grid aspect ratio at the surface, α = Δx/Δz1, as the dynamic models required tuning of near-wall stress parameters. However, with proper adjustment of cw, the dynamic models provided the best performance for the flow variables investigated. Guidelines for adjustment of the near-wall stress model parameters can be developed (see e.g., Chow et al. 2005). The large errors in K and τ13 remain disconcerting, however, as explicit simulation of the structures responsible for these quantities motivate the LES approach, and justify its considerable computational expense.
Overall, this study indicates that extensive buffer zones are required for equilibration of flow parameters to nearly steady values on nested domains. Vertical mesh refinement would likely reduce both equilibration distances and anomalies, by allowing use of constant α and, for the dynamic models, constant cw, on all domains. For the SMAG and NBA1 models, vertical mesh refinement would exacerbate the abrupt change of l at nest interfaces; however, as the greatest τ13 and U anomalies occur near the inflow boundaries, with α more strongly impacting the profiles downstream, vertical nesting would likely improve results overall. This study points to the need to develop this capability for concurrently nested simulations in WRF. A grid-size-independent length scale could also potentially ameliorate some impacts from abrupt grid-size changes at nest boundaries.
Despite the difficulties encountered in the nested LES investigated herein, nested simulations using Reynolds-averaged or mesoscale parameterizations are unlikely to perform better, as quantities such as τ13 and K are fully parameterized in such simulations, and the assumptions upon which those parameterizations are based are violated at high resolutions and under inhomogeneous forcing scenarios. Improving nested-LES performance in models such as WRF remains the best pathway to enable finescale atmospheric modeling in real-world setups. This study elucidates shortcomings of existing approaches, and identifies directions for future research.
Acknowledgments
This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory (LLNL) under Contract DE-AC52-07NA27344, and was supported by both the Laboratory Directed Research and Development (LDRD) program, and the U.S. DOE Office of Energy Efficiency and Renewable Energy. NSF funding is acknowledged by EBZ from CBET-1058027 and FKC from ATM-0645784.
REFERENCES
Andren, A., A. R. Brown, J. Graf, P. J. Mason, C.-H. Moeng, F. T. M. Nieuwstadt, and U. Schumann, 1994: Large eddy simulation of a neutrally stratified boundary layer: A comparison of four codes. Quart. J. Roy. Meteor. Soc., 120, 1457–1484.
Bou-Zeid, E., C. Meneveau, and M. B. Parlange, 2005: A scale-dependent Lagrangian dynamic model for large eddy simulation of complex turbulent flows. Phys. Fluids, 17, 025105, doi:10.1063/1.1839152.
Bou-Zeid, E., N. Vercauteren, M. B. Parlange, and C. Meneveau, 2008: Scale dependence of subgrid-scale model coefficients: An a priori study. Phys. Fluids, 20, 115106, doi:10.1063/1.2992192.
Brown, A. R., J. M. Hobson, and N. Wood, 2001: Large-eddy simulation of neutral turbulent flow over rough sinusoidal ridges. Bound.-Layer Meteor., 98, 411–441.
Chow, F. K., R. L. Street, M. Xue, and J. H. Ferziger, 2005: Explicit filtering and reconstruction turbulence modeling for large-eddy simulation of neutral boundary layer flow. J. Atmos. Sci., 62, 2058–2077.
Germano, M., U. Piomelli, P. Moin, and W. H. Cabot, 1991: A dynamic subgrid-scale eddy viscosity model. Phys. Fluids A, 3, 1760–1765.
Holton, J. R., 1992: An Introduction to Dynamic Meteorology. 3rd ed. Academic Press, 511 pp.
Keating, A., U. Piomelli, E. Balaras, and H.-J. Kaltenbach, 2004: A priori and a posteriori tests of inflow conditions for large-eddy simulation. Phys. Fluids, 16, 4696–4712.
Kirkil, G., J. D. Mirocha, F. K. Chow, and E. Bou-Zeid, 2012: Implementation and evaluation of dynamic subfilter-scale stress models for large-eddy simulation using WRF. Mon. Wea. Rev., 140, 266–284.
Kosović, B., 1997: Subgrid-scale modeling for the large-eddy simulation of high-Reynolds-number boundary layers. J. Fluid Mech., 336, 151–182.
Kosović, B., D. I. Pullin, and R. Samtaney, 2002: Subgrid-scale modeling for large-eddy simulations of compressible turbulence. Phys. Fluids, 14, 1511–1522.
Lilly, D. K., 1967: The representation of small-scale turbulence in numerical experiment. Proc. IBM Scientific Computing Symp. on Environmental Sciences, White Plains, NY, IBM, 195–210.
Lundquist, K. A., F. K. Chow, and J. K. Lundquist, 2010: Numerical errors in flow over of steep topography: Analysis and alternatives. Preprints, 14th Conf. on Mountain Meteorology, Olympic Valley, CA, Amer. Meteor. Soc., 10.1. [Available online at https://ams.confex.com/ams/pdfpapers/173801.pdf.]
Mirocha, J. D., J. K. Lundquist, and B. Kosović, 2010: Implementation of a nonlinear subfilter turbulence stress model for large-eddy simulation in the Advanced Research WRF model. Mon. Wea. Rev., 138, 4212–4228.
Moeng, C.-H., J. Dudhia, J. B. Klemp, and P. P. Sullivan, 2007: Examining two-way nesting for large eddy simulation of the PBL using the WRF model. Mon. Wea. Rev., 135, 2295–2311.
Piomelli, U., S. Kang, F. Ham, and G. Iaccarino, 2006: Effect of discontinuous filter width in large-eddy simulations of plane channel flow. Proc. 2006 Summer Program, Stanford, CA, Center for Turbulence Research, 151–162. [Available online at http://ctr.stanford.edu/ctrsp06/piomelli.pdf.]
Samtaney, R., D. I. Pullin, and B. Kosović, 2001: Direct numerical simulation of decaying compressible turbulence and shocklet statistics. Phys. Fluids, 13, 1415–1430.
Skamarock, W. C., and Coauthors, 2008: A description of the advanced research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp. [Available online at http://www.mmm.ucar.edu/wrf/users/docs/arw_v3_bw.pdf.]
Smagorinsky, J., 1963: General circulation experiments with the primitive equations. Mon. Wea. Rev., 91, 99–152.
Vanella, M., U. Piomelli, and E. Balaras, 2008: Effect of grid discontinuities on large-eddy simulation statistics and flow fields. J. Turbul., 9, 1–23.
Wong, V. C., and D. K. Lilly, 1994: A comparison of two dynamic subgrid closure methods for turbulent thermal-convection. Phys. Fluids, 6, 1016–1023.
