1. Introduction
Accurate downscaling of time-varying mesoscale (Δx > 1 km) meteorological data to microscale resolutions (Δx < 10 m) has recently been shown to improve microscale simulations of scalar transport in an urban environment (Wiersema et al. 2020). While these simulations demonstrate the importance of downscaling, they also highlight the associated difficulties, such as resolving the effects of complex terrain on the flow, configuring grids of vastly different resolutions, and appropriately representing turbulence at various scales. The goal of this study is to improve microscale predictions for transport and dispersion over complex urban topography by improving the downscaling and representation of turbulence within a multiscale simulation. We focus on quantifying model performance at predicting turbulence kinetic energy (TKE) through use of skill metrics and calculations of spectra. We also examine the role of modeling physical processes that are important to the production of TKE yet are often neglected in simulations of flow over urban terrain. The effect of including surface heat fluxes and incorporating perturbations at the lateral boundaries (designed to speed the development of turbulence) is examined by quantifying the fidelity of the simulated flow fields for TKE, wind fields, and scalars.
The simulations presented here build upon those from Wiersema et al. (2020), which demonstrated a multiscale configuration of the Weather Research and Forecasting (WRF) Model for prediction of transport and mixing of a passive tracer gas (SF6) released during the Joint Urban 2003 (JU2003) field experiment in Oklahoma City, Oklahoma (Allwine and Flaherty 2006). The multiscale simulation used a series of five telescoping nested domains that downscale regional and mesoscale meteorology to large-eddy simulation (LES) scales. The simulation yielded excellent agreement with observed concentrations of SF6, which was hypothesized to result in part from the dynamic downscaling of large-scale meteorological fields and in part from resolving turbulence on the microscale domain. The work presented here further examines this hypothesis.
As in Wiersema et al. (2020), we simulate transport and mixing of a tracer gas (SF6) released during the JU2003 field campaign and compare simulations using the multiscale setup with microscale-only simulations that are nested within a larger domain using periodic boundary conditions. The immersed boundary method (Lundquist et al. 2012; Bao et al. 2018; Arthur et al. 2020), which is an alternative gridding technique that enables the model to simulate flow over complex terrain, is used to handle flow around resolved buildings. Vertical grid refinement (Daniels et al. 2016; Mirocha and Lundquist 2017) is used to ensure high-quality grids while downscaling over a large range of resolutions. The multiscale method of Wiersema et al. (2020) does not require a priori knowledge (i.e., local observations) but instead downscales a mesoscale meteorological forecast product. In this way, the highest resolution domain receives lateral boundary conditions that include meteorological features and turbulence from all scales that are resolved within the telescope of nested domains that span from mesoscale to microscale resolutions.
One of the difficulties associated with downscaling from the mesoscale to the microscale is transitioning from domains using a planetary boundary layer (PBL) scheme to those using a LES turbulence closure, which in practice means transitioning from a smooth flow field, where all turbulence is modeled, to a field where a portion of the turbulence is resolved. The cell perturbation method (CPM) of Muñoz-Esparza et al. (2015), which is detailed in section 4, has previously been demonstrated to improve model behavior when downscaling from mesoscale to large-eddy simulation. The CPM introduces small temperature perturbations on patches of grid cells where there is inflow to the nested domain. These perturbations promote the development of turbulence and are applied here when transitioning from a coarse resolution domain with a PBL scheme to a fine resolution domain using a LES turbulence closure. Two multiscale simulations (with and without the CPM) are examined here, with the goal of evaluating the effect of the CPM on turbulent quantities.
In addition to the multiscale simulations with and without the CPM, we evaluate two versions of the microscale-only configuration; one with neutral conditions and the other, hereinafter referred to as microscale-only HFX, that includes a surface heat flux matching that from the multiscale simulation. Many past microscale-only simulations of JU2003 IOP 3, such as Lundquist et al. (2012) and Wiersema et al. (2020), have assumed the atmosphere within the urban core is well mixed and can be adequately represented with neutral stability and without a surface heat flux. Despite fully developed turbulent inflow to the 2-m domain, the neutral microscale-only simulation severely underestimates near-surface TKE, as shown and discussed in section 6. Because the multiscale simulations include a surface heat flux and are not neutral, it is reasonable to question how the lack of surface heating impacts the microscale-only results and whether it contributes to the TKE underestimate. For this reason, we evaluate a variation of the microscale-only configuration with a surface heat flux.
Performance of the four simulations (microscale-only, microscale-only HFX, multiscale and multiscale CPM) is qualitatively evaluated in section 5 and quantitatively evaluated in sections 6 and 7 by comparing the simulations with observations of meteorological conditions and tracer gas concentrations during JU2003. These comparisons include statistical measures of model skill for the prediction of wind speed, wind direction, tracer concentration and TKE. In addition, spectra of predicted TKE are analyzed and compared with sonic anemometer observations.
2. Joint Urban 2003 field campaign
The Joint Urban 2003 (JU2003) atmospheric dispersion study in Oklahoma City was led by the Defense Threat Reduction Agency and the U.S. Department of Homeland Security with the objective of investigating flows downwind of tall buildings and in street canyons, and tracer dispersion around and downwind of tall buildings. A detailed overview of the field campaign can be found in (Allwine and Flaherty 2006). Numerous and extensive simulations have been performed using models of varying degrees of fidelity and a focus on urban meteorology and tracer dispersion within the central business district of Oklahoma City during the JU2003 intensive observational periods (IOPs) (Burrows et al. 2007; Chan and Leach 2007; Gowardhan et al. 2011; Hanna et al. 2011; Li et al. 2018; Lundquist et al. 2012; Wiersema et al. 2020).
JU2003 consists of 10 IOPs, each with an 8-h duration. The IOPs vary in many ways, including time of day, atmospheric stability, synoptic forcing, and whether SF6 was released continuously or as a puff. The simulations presented here focus on the first continuous tracer release during IOP 3, between 1600 and 1630 UTC 7 July 2003. This particular release was chosen because of consistent synoptic forcing and because it has been the focus of several previous JU2003 studies, including Wiersema et al. (2020).
Several observational datasets from JU2003 are used in this study for initializing and forcing the microscale-only simulations and also for analyzing model performance and the simulations’ skill. These observations include a sonic anemometer at the SF6 release location that was deployed by the Field Research Division of the NOAA Air Resources Laboratory (ARL-FRD), a SODAR deployed by Argonne National Laboratory (ANL) in the botanical gardens located to the southwest of the SF6 release location, 11 portable weather information display stations (PWIDS) with propeller and vane anemometers and 15 Super PWIDS with sonic anemometers deployed by the Dugway Proving Grounds (DPG), 20 programmable integrating gas samplers (PIGS) deployed by NOAA ARL-FRD, and 19 “blue box” integrating gas samplers deployed by Lawrence Livermore National Laboratory (LLNL). The LLNL blue-box gas samplers are sited relatively near to the SF6 release location, and the ARL-FRD gas samplers are sited farther afield.
3. Model configurations
Version 3.8.1 of the WRF Model is used in this work (Skamarock et al. 2008). The relative sizes and positions of domains in the microscale-only and multiscale model configuration are displayed in Fig. 1. It is important to note that the microscale-only and multiscale simulations are configured such that the highest resolution domains, those with horizontal resolution of 2 m, are as similar as is possible so that differences in the simulation results are primarily due to the forcing method.
Positioning of grids in (a) the two-domain microscale-only configuration and (b) six-domain multiscale model configuration. For the multiscale configuration, a map showing state boundaries is overlaid on the 4.95-km domain and contours of ground elevation above sea level are shown on the 1.65-km, 330-m, 30-m, and 10-m domains. The 2-m domains include contours of building heights above ground level. The blue and red star-shaped symbols indicate the SF6 release location and the ANL mini-SODAR, respectively. A blue arrow in (a) indicates the mean wind direction below 100 m AGL to which the microscale-only simulation 10-m domain is tuned to match.
Citation: Monthly Weather Review 150, 12; 10.1175/MWR-D-22-0056.1
Topography and structures within the central business district are represented using an immersed boundary method (IBM). While the IBM is enabled for the 10- and 2-m domains, buildings are included only on the 2-m domain because they are inadequately resolved on the 10-m domain. Details of the IBM are included in Wiersema et al. (2020) and the urban geometry is shown in Fig. 1. The IBM used here is called the velocity reconstruction IBM (VR-IBM) because velocities are reconstructed at grid points adjacent to the immersed boundary and within the fluid domain, using the law of the wall (i.e., the logarithmic law). There are minor differences between the VR-IBM algorithm used in these simulations and that used in Wiersema et al. (2020). For those interested in the details of the IBM, these simulations closely follow the VR-IBM algorithm described in Arthur et al. (2020). Note that surface fluxes of heat and moisture are not included in the IBM domains because WRF’s surface physics parameterizations (i.e., the land surface model and radiation schemes) are not yet applicable at the urban microscale. The development and validation of surface parameterizations for the urban microscale is left to future work. As in Wiersema et al. (2020), scalar variables (i.e., SF6) are not yet treated using the IBM; however, this has minimal effect on our results because the wind fields prevent advection of scalars through the immersed boundary and diffusion of scalar across the immersed boundary is negligible.
a. Microscale-only configuration
The microscale-only simulation in this study uses a two-domain one-way nested configuration with horizontal grid resolutions of 10 and 2 m. The parent 10-m domain has periodic lateral boundary conditions and is allowed to “spin up” for 6 h and 50 min prior to the initialization of the 2-m domain. This two-domain configuration has the advantage of providing well-developed turbulent inflow to the 2-m simulation, where presence of the 2-m simulation does not affect the solution on the parent domain. Due to the simplified nature of the microscale-only simulation, it is necessary to specify idealized initial conditions and forcing for the 10-m domain.
Initial atmospheric conditions are specified as neutral and dry. Forcing is applied by the addition of a uniform pressure gradient throughout the 10- and 2-m domains. The magnitude of this pressure gradient is tuned to maintain agreement as best as possible between velocities in the 2-m domain and time-averaged JU2003 observations, shown in Fig. 2. These observations include the ANL mini-SODAR, DPG PWIDS 10 and 11, DPG Super PWIDS 17 and 20, and the NOAA ARL-FRD sonic anemometer sited at the release location. Data from each of these instruments are temporally averaged over the 30-min SF6 release period. Near-surface velocities are calculated using the ARL-FRD sonic anemometer for 2 m AGL and an average of the four nearby DPG stations for 8 m AGL. The ANL mini-SODAR observations are used to calculate velocities between 15 and 135 m AGL, above which there are no further observations. The resulting time-averaged wind speed and direction at the location of the ANL mini-SODAR for the microscale-only simulations is included in Fig. 2 (green and blue lines). Although the simulated wind speed and direction match observations well aloft, significant bias is observed near the surface (below 40 m) despite tuning of the pressure gradient forcing. As is common with idealized microscale simulations, this configuration does not include transient forcing or the effects of regional meteorology.
Vertical profiles of horizontal wind speed and direction above the ANL mini-SODAR that are time- averaged over the 30-min SF6 release period. Results from the four simulations are displayed as solid lines. Observations from the ANL mini-SODAR, an average of nearby DPG PWIDS and Super PWIDS, and the ARL-FRD sonic anemometer are displayed with symbols.
Citation: Monthly Weather Review 150, 12; 10.1175/MWR-D-22-0056.1
The vertical grids each use a near-surface grid aspect ratio of Δx/Δz = 2.0 for the first 100 m AGL of the 10-m domain and 25 m AGL for the 2-m domain. Above the near-surface, the spacing of grid levels increases with a constant stretching coefficient, (zk+1 − zk)/(zk − zk−1), of 1.016 and 1.027 for the 10- and 2-m domains, respectively, until reaching a maximum grid aspect ratio of Δx/Δz = 0.5 that is maintained until reaching the model top at 400 m AGL.
The microscale-only HFX simulation includes a surface heat flux applied to the 10-m domain. This surface heat flux alters the development and scale distribution of turbulence within the 10-m domain, which is then downscaled into the 2-m domain of interest. The magnitude (407 W m−2) is determined by spatially averaging the surface heat flux from a section of the multiscale 30-m domain coincident with the 10-m domain and then temporal averaging over the IOP 3 release window.
b. Multiscale configuration
The multiscale model configuration uses six nested domains with horizontal resolutions of 4.95 km, 1.65 km, 330 m, 30 m, 10 m, and 2 m. The positions of the six domains are shown in Fig. 1, and additional configuration details are included in Table 1. For this set of simulations, since the predominant wind direction on the LES domains is consistent and known for this case, the 10- and 2-m domains are positioned in the northeast quadrant of their respective parent domains such that the “fetch” distances between inflow boundaries of the nested domains is increased relative to a centered arrangement.
Six-domain multiscale model configuration for JU2003 simulations. The labels used in the table are TF = terrain-following coordinate, IBM = immersed boundary method, MYJ = Mellor–Yamada–Janjić planetary boundary layer turbulence parameterization, Smag = 3D Smagorinsky turbulence closure scheme, WSM3 = WRF single-moment 3-class microphysics scheme, RRTM = Rapid Radiative Transfer Model longwave radiation model, and KF = Kain–Fritsch cumulus parameterization.
Unlike the microscale-only simulation, the multiscale simulation does not use any JU2003 observations for forcing. Initial conditions and mesoscale forcing are supplied by the North American Regional Reanalysis (NARR; Mesinger et al. 2006). By only relying on a meteorological forecast product, such as the NARR dataset, the multiscale configuration is running as a “forecast” and, if computational resources were significantly more powerful, the simulation could be used in a predictive capacity. Forcing strategies for limited-area atmospheric simulations, such as the multiscale simulation, vary greatly in complexity and can have a large impact on the simulation accuracy (Warner et al. 1997; Davies 2014). The detailed evaluation and comparison of inflow boundary conditions for LES are covered by review papers (e.g., Tabor and Baba-Ahmadi 2010) and are not the focus of this study.
The computational burden of the multiscale simulation is reduced by delaying the start times of each nested domain, which allows for development of the flow within a parent domain prior to initializing a child domain. This “spinup” time is crucial because it allows for each domain to advance sufficiently beyond the initial conditions and develop features of scales not resolved in the initial flow field or the corresponding parent domain. Beginning with the 4.95-km domain, the six domains have start times of 0300, 0600, 0900, 1200, 1500, and 1550 UTC. The continuous SF6 release begins at 1600 UTC, and the simulation is terminated at 1630 UTC, corresponding to the shutoff of the SF6 release.
Terrain-following coordinates and the standard WRF bottom boundary conditions are used on the 4.95-km, 1.65-km, 330-m, and 30-m domains. The velocity reconstruction immersed boundary method is used on the 10- and 2-m domains with a spatially homogeneous roughness length z0 = 0.1 m. The Mellor–Yamada–Janjić planetary boundary layer scheme is enabled on the 330-m domain and its parents. At LES resolutions (30, 10, and 2 m), the three-dimensional Smagorinsky turbulence closure scheme is used with a coefficient Cs = 0.18.
As with the microscale-only configuration, the vertical levels of each domain are carefully chosen. A model top of 200 hPa is used for all domains in the multiscale configuration, meaning that more vertical grid levels are necessary for the 10- and 2-m domains relative to their microscale-only counterparts. Near-surface vertical grid levels for the 10- and 2-m multiscale domains are selected to match, as closely as possible, the vertical grid levels from the microscale-only configuration. Above 400 m AGL, which is the height of the model top of the microscale-only configuration, the vertical grid levels are stretched in height at a constant rate of (zk+1 − zk)/(zk − zk−1) = 1.05. This stretching greatly reduces the computational costs but results in highly elongated grid cells with small aspect ratios (Δx/Δz ≪ 1) near the model top. Any error associated with poor grid quality is restricted to the topmost levels of the 10- and 2-m domains, where it does not affect the near-surface flow.
The multiscale 10- and 2-m domains do not use a land surface model, longwave radiation scheme, or shortwave radiation scheme. The absence of these physics parameterizations is a potential shortcoming of the multiscale configuration; however, these parameterizations are not intended for use at microscale resolutions and should be applied to microscale domains with caution. In extensive tests (not shown), disabling the IBM and enabling these physics parameterizations on the 10-m domain significantly degrades the simulation accuracy. More specifically, enabling the physics parameterizations exacerbates the overestimates in the 10-m domain of near-surface wind speeds and turbulence kinetic energy. These overestimates arise, in part, because the model does not resolve the effects of buildings, trees and vegetation, parked vehicles, and other flow obstructions. Future research to improve the representation of surface heterogeneity in intermediate resolution domains could focus on the use of IBM to resolve large structures, such as warehouses and office buildings, while the effects of smaller scale obstructions could be represented with a virtual building drag model, such as that demonstrated by Chan and Leach (2007).
4. Cell perturbation method
In nested multiscale models, traversing the turbulence “gray zone” involves the transition from a planetary boundary layer (PBL) scheme used on coarse resolution domains to a large-eddy simulation (LES) turbulence closure model used on high resolution domains. In contrast to turbulent flow resolved in a high-resolution LES, the flow on coarse resolution domains using a PBL scheme is smooth with minimal resolved turbulence. When an LES domain is nested within a parent domain using a PBL scheme the nested LES domain requires a prohibitively long distance to fully develop turbulence (i.e., fetch) (Mirocha et al. 2013). The CPM (Muñoz-Esparza et al. 2014, 2015; Muñoz-Esparza and Kosović 2018) greatly reduces the fetch necessary for turbulence development by introducing small temperature perturbations along the lateral inflow boundaries. The CPM has previously been demonstrated to improve LES simulations, including those by Lee et al. (2019), which used the CPM to develop turbulence in a microscale-only simulation over urban terrain forced using fixed inflow velocity profiles; by Connolly et al. (2021) in multiscale simulations over complex mountainous terrain of the Perdigão field campaign; and simulations of flow over the Askervein Hill by Sauer and Muñoz-Esparza (2020) using the FastEddy model with the CPM, which displayed excellent agreement with observations of near-surface winds and speedup of flow over the hill.
Here we apply potential temperature perturbations to the coarsest LES domain, the 30-m domain, on patches of 8 × 8 grid points in the horizontal dimensions and 2 grid points in the vertical dimension, with a constant perturbation magnitude within each patch. The patch dimensions were selected to be large enough that the perturbations would not be quickly dissipated by the WRF Model. As discussed in Muñoz-Esparza et al. (2014), the WRF Model’s finite-difference schemes introduce numerical diffusion that rapidly dissipates energy for
Because Γk depends upon the vertical grid level, meteorological variables (i.e., velocities), and grid variables (i.e., resolution, patch sizes, and number of patches), it is difficult to provide a typical value. For reference, if considering a case with p = 3 patches, n = 8 grid points per patch, Δ = 30 m and
5. Qualitative analysis of simulation results
A visual comparison of the results from the four JU2003 IOP3 simulations reveals several noteworthy differences in the predictions of near-surface vertical velocity, SF6 concentrations and turbulence kinetic energy (TKE) discussed in more detail below.
a. Vertical velocity
The contours of instantaneous vertical velocity at 8 m AGL in Fig. 3 display clear differences in strength and size of eddies in the 10-m domains of the four simulations. Including the surface heat flux on the 10-m domain of the microscale-only simulation noticeably increases the strength of vertical motions visible in (Fig. 3c). The flow characteristics inherited from the 10-m domains persist near the inflow (southern) boundary of the 2-m domains but the visible differences between the simulations are reduced as the flow traverses through the business district and the flow is disrupted by buildings.
Contours of instantaneous vertical velocity at 8 m AGL at 1600 UTC, the start time of the SF6 release, from the (a),(c) 10- and (b),(d) 2-m domains of the microscale-only simulations and the (e),(h) 30-, (f),(i) 10-, and (g),(j) 2-m domains of the multiscale simulations.
Citation: Monthly Weather Review 150, 12; 10.1175/MWR-D-22-0056.1
Because of the periodic boundary conditions, the scale distribution of turbulence in the 10-m domain of the microscale-only simulations, (Figs. 3a,c), appears horizontally consistent. The 30- and 10-m domains of the multiscale simulations, (Figs. 3e,f,h,i), show the development of smaller scales as flow progresses from the southwest domain corner to the northeast. As expected, the multiscale 30-m domains include scales of turbulence much larger than those seen in the microscale-only simulations. The multiscale 10-m domains, (Figs. 3f,i), show less variation in vertical velocity relative to the periodic microscale-only 10-m domains, (Figs. 3a,c); however, there are large coherent and energetic structures that are downscaled in the multiscale simulations from the 30-m parent domains. The relative smoothness of the multiscale 10-m domains is likely due to the lack of a surface heat flux, but also might indicate an absence of important roughness elements, such as trees and buildings, which will later be discussed in more detail.
b. SF6 concentrations
Figure 4 includes horizontal contours at 2.5 m AGL of SF6 concentrations time-averaged over the 30-min release period. Comparison of the two microscale-only simulations reveals evidence for the importance of including a surface heat flux for the development of turbulence on the 10-m domain. With the surface heat flux, the microscale-only plume spreads farther laterally and shows improved agreement with observations, particularly to the northwest of the release location. Plumes from both multiscale simulations have more lateral spread than those from the microscale-only simulations. This increased width of the multiscale plumes is due to time-varying shifts in the inflow direction of the 2-m domain that are not replicated when using the idealized forcing of the microscale only configuration. Visually, the multiscale simulations show improved agreement with the observations, especially at sensors west of the SF6 release location.
Horizontal contours of SF6 concentrations at 2.5 m above ground level time-averaged over the SF6 release window (1600–1630 UTC) from the four simulations. Concentration observations are overlaid as filled circles, with the larger symbols corresponding to the LLNL blue-box gas samplers and small symbols corresponding to the NOAA ARL-FRD gas samplers. The transparent star-shaped symbols indicate the SF6 release location.
Citation: Monthly Weather Review 150, 12; 10.1175/MWR-D-22-0056.1
Transport of SF6 upwind, relative to the predominant flow direction, from the release location is observed in both multiscale simulations and is visible in Fig. 4. This upwind transport does not appear in the microscale-only simulations because it results from the large-scale flow variability downscaled from the intermediate resolution domains. Intermittent periods of near-zero wind speed and highly variable wind direction, including brief periods of flow reversal, occur at the SF6 release location in both multiscale simulations and in the JU2003 observations (see Fig. 10 in Wiersema et al. 2020). It is during these intermittent periods that coherent large-scale flow patterns may induce substantial upwind transport of the SF6 plume. When the CPM is enabled, the additional small-scale turbulence generated in the 30-m domain assists with breaking down these coherent flow patterns, which reduces, but does not eliminate, the upwind transport.
c. Turbulence kinetic energy
At Super PWIDS locations, observations were logged every 1/10 s. The three velocity components are output from the model at each time step, which is 1/75 s for the 2-m domain. Because of storage limitations, velocities at every model time step are only saved at the grid point nearest to a Super PWIDS and the six adjacent grid points (north, south, east, west, above, and below). Because a single grid point may not be the best choice for representing a real-world station, the predicted TKE is calculated at each Super PWIDS as the average of TKE in the seven gridpoint cluster. Calculations of TKE at locations other than the Super PWIDS stations, such as the horizontal contours of TKE in Fig. 5 and vertical profiles of TKE in Fig. 6, are performed using simulation history files with a 3-s output interval. Differences in the spatial averaging and temporal resolution of the velocity fields will result in small variations between the two types of TKE calculations.
Horizontal contours of resolved TKE at 8 m above ground level time averaged over the SF6 release period (1600–1630 UTC) from the four simulations. TKE calculated from the DPG Super PWIDS observations is overlaid using filled circles. Contours of TKE are calculated using data output every 3 s.
Citation: Monthly Weather Review 150, 12; 10.1175/MWR-D-22-0056.1
Vertical profiles of average TKE during the SF6 release period (1600–1630 UTC) from the Δ = 2-m domains along a transect between the ANL mini-SODAR and DPG PWIDS 03.
Citation: Monthly Weather Review 150, 12; 10.1175/MWR-D-22-0056.1
To verify that the two TKE calculations result in a comparable estimate, we calculate TKE along vertical profiles at the locations of ANL mini-SODAR and DPG PWIDS 03 using both data sources from the multiscale CPM simulation. TKE calculated with the two data sources is compared at 20 grid points per vertical profile, which yields a mean absolute difference of 0.11 m2 s−2 and a standard deviation of the absolute differences of 0.14 m2 s−2. At each grid point evaluated, the percent difference between TKE calculated with different data sources is found as the ratio of the absolute difference and average of the TKE calculations. For the two profiles evaluated, the average percent difference is 2.5%, which lends confidence in the calculations of TKE using simulation history files with the lower temporal resolution of 3 s.
A similar spatial pattern of resolved TKE is produced by all four simulations, which can be seen in Fig. 5, which shows the horizontal contours of resolved TKE at 8 m AGL time-averaged over the SF6 release period (1600–1630 UTC). Regions of elevated TKE develop throughout the simulation and are pronounced where buildings obstruct the flow and channel it through narrow gaps (see UTM coordinates 634500 easting, 3926000 northing in Fig. 5), at the intersection of street canyons (see coordinates 634850, 3925900), and in the lee of tall buildings (see coordinates 634850, 3926050). Both multiscale simulations resolve more TKE relative to the microscale-only simulations.
Qualitatively, the microscale-only simulation without a surface heat flux stands out for resolving less TKE than the other three simulations and appears to underestimate TKE at locations of DPG Super PWIDS. The microscale-only HFX simulation shows improved agreement with the DPG Super PWIDS and more closely resembles results from the multiscale simulations, which indicates the importance of including the surface heat flux in the microscale-only simulation. This visual analysis is confirmed by the model skill metrics for TKE that are discussed in the following section.
Figure 6 shows vertical TKE profiles along a transect between the ANL mini-SODAR and DPG PWIDS 03. This transect was selected to loosely follow the predominant wind direction, pass near the SF6 release location and provide a transition between the inflow and outflow boundaries of the 2-m domain. The location of each vertical profile within the 2-m domain is shown in the map overlaid in the top left corner of Fig. 6. As is also seen in Fig. 5, the four simulations produce similar trends but with different magnitudes. In each vertical profile, the microscale-only simulation resolves less TKE than the other three simulations, which, once again, indicates the importance of including a surface heat flux. In general, the multiscale simulations produce similar profiles; however, enabling the CPM often causes a slight increase in TKE, especially in the lee of buildings, which is most obvious in the profile located at a distance of approximately 500 m along the transect.
When advancing from south to north along the transect in Fig. 6, there is an increase in the maximum height that TKE is influenced by the buildings. In these simulations, the region upwind of the inflow boundaries consists of single-story structures and parking lots. If taller structures were present upstream of the inflow, then the 2-m domain would need to be expanded to ensure sufficient distance over which the flow could develop prior to reaching the SF6 release location.
6. Statistical analysis of model skill
In the above equations, Xo is the set of observational data and Xp are the corresponding predictions from a simulation, N is the number of observations, ϕi is the difference between observed and predicted wind directions, and |Ui| is the predicted wind speed. Values for Xo and Xp are time-averages over the 30-min tracer release period. An overbar indicates averaging over all locations. Tables of Xo and Xp, where the X represents wind speed, wind direction, SF6 concentration, or TKE, are available in the online supplemental material.
Skill scores evaluating wind speeds, direction and SF6 concentrations predicted by the microscale-only and multiscale simulations are graphically represented in Fig. 7. These skill scores are comparable to those produced by the five-domain multiscale setup from Wiersema et al. (2020) who observed that microscale-only simulations performed best at predicting wind speeds and a multiscale simulation performed best at predicting SF6 concentrations. It is not surprising that the microscale-only simulations, the only simulations with forcing based upon local JU2003 observations, would best predict wind speed and direction. The microscale-only simulations were provided inflow that was tuned to ensure agreement with local observations, whereas the multiscale simulations were provided a meteorological forecast product with a horizontal resolution of 32 km. In addition, the WRF Model and parameterizations used here have been developed to simulate regional-scale weather and have historically struggled to correctly predict near-surface wind speeds (Olson et al. 2019). Model advances, such as the CPM and the IBM used here, can potentially improve predictions of the near-surface wind speeds at microscale resolutions.
Model skill test results evaluating (top) horizontal wind speed and wind direction, (middle) SF6 concentration, and (bottom) turbulence kinetic energy for the microscale-only simulations with and without a surface heat flux and the multiscale simulations with and without the CPM. Simulation results are evaluated against DPG PWIDS and Super PWIDS, LLNL blue-box, and NOAA ARL FRD integrated tracer samplers. The thick black lines represent the score of a perfect model.
Citation: Monthly Weather Review 150, 12; 10.1175/MWR-D-22-0056.1
Each simulation yields negative FB scores for wind speed, indicating that all simulations overestimate near-surface wind speeds. We hypothesize that this is due to the absence of drag induced by buildings in the intermediate resolution domains and is also due to missing small-scale sources of drag in the 2-m domain, such as vegetation and architectural features. Enabling the CPM results in a further decrease in the FB score, which is likely due to a downward transport of high momentum flow from aloft because of increased vertical mixing. Overall, the microscale-only simulations display the highest model skill for the prediction of wind speed and direction, followed by the multiscale simulation and the multiscale CPM simulation.
Similar to the simulations from Wiersema et al. (2020), the multiscale simulations outperform the microscale-only simulations at predicting SF6 concentrations. Relative to the microscale-only simulations, the multiscale simulations show similar or improved skill for every metric evaluating prediction of SF6 concentrations. Enabling CPM results in a slight decrease in the model skill scores for the prediction of SF6 concentrations.
The microscale-only simulation’s large and positive FB score for TKE indicates a substantial underestimate. The upper limit of turbulence length scales supported on the microscale-only simulation’s 10-m domain may be partly responsible for the underestimation of TKE. The multiscale simulations develop energetic, large-scale, coherent flow structures in the domains with 30-, 330-, and 1650-m horizontal resolution, which the microscale-only simulation is unable to reproduce. The microscale HFX simulation displays near-perfect skill predicting TKE and higher skill predicting wind speed and direction than the multiscale simulations, and yet the microscale-only HFX simulation shows significantly less skill predicting SF6 concentrations. This supports the importance of accurately representing both the mean flow characteristics and the flow variability. These microscale-only simulations are unable to reproduce the flow variability that is necessary to achieve the best predictions of SF6 concentrations. That this is evident in JU2003 IOP3 simulations should emphasize its importance. Very few other simulations will have as near-idealized a region of interest as that seen in JU2003, with flat ground topography, near-neutral stability, consistent synoptic forcing, and mostly unobstructed upstream flow. If large-scale variability is important here, then it is likely to be important everywhere.
Overall, the microscale-only HFX simulation best predicts TKE, and both multiscale simulations overestimate TKE, but the magnitude of this error is less than the underestimate by the microscale-only simulation. Enabling CPM increases the overestimation of TKE and results in slightly reduced model skill for both TKE and SF6. Both multiscale simulations show improved model skill at predicting SF6 and TKE relative to the microscale-only simulation. Adding a surface heat flux to the microscale-only simulation yields a slight increase in model skill at predicting SF6 concentrations, but not enough of an increase to match the model skill of the multiscale simulations.
When CPM is enabled, the model skill scores marginally deteriorate for almost every category. The omission of roughness elements from the intermediate domains, such as buildings and trees, results in the stubborn persistence of relatively large-scale and energetic turbulence introduced by the CPM. We hypothesize that the additional vertical mixing accompanying these large-scale flow features is primarily responsible for the increased overestimates of wind speed and TKE, which has the secondary effect of lowering model skill at predicting SF6 concentrations.
7. Spectral analysis of turbulent kinetic energy
DPG Super PWIDS 17 (SP17), located 8 m AGL at the SF6 release location, is well sited to observe flow with relatively unobstructed upstream conditions. SP17 is the most reliable observation of unobstructed flow leading into the central business district because directly upwind from the sonic anemometer are botanical gardens, single-story structures, and parking lots. Figure 8 shows TKE spectra at SP17 calculated using the sonic anemometer observations and output at every time step (1/75 s) from the four simulations during the SF6 release window (1600–1630 UTC).
Frequency spectra of resolved TKE at 8 m above ground level at the SF6 release location. Included on this plot are observations from the DPG Super PWIDS 17 and results from the four simulations. The large symbols represent frequency-binned results, which more clearly display trends that are obscured as a result of the number of points in the energy spectra and overlap between results from the different simulations. The frequency of 0.05 s−1, which is referenced in the discussion, is denoted with an enlarged tick mark.
Citation: Monthly Weather Review 150, 12; 10.1175/MWR-D-22-0056.1
The SP17 observations display excellent agreement with the expected −5/3 power-law scaling within the inertial subrange. All four simulations produce spectra showing trends expected from a high-resolution LES, with similar magnitude and slope as the observations at low frequencies (large-scale turbulence), and deviation from the observed spectra at high frequencies (small-scale turbulence). This underprediction at high frequencies is expected and arises because the spectra includes only TKE resolved in the model velocity terms, and not TKE resolved by the subgrid-scale (LES closure) model.
Relative to the microscale-only simulation, the multiscale simulation spectra have improved agreement with SP17 within the inertial subrange, particularly at frequencies greater than 0.05 s−1. This extension of the resolved inertial subrange within multiscale simulations is due to the energetic low-frequency motions generated on the intermediate resolution domains. When downscaled and allowed to develop on a higher resolution domain, these large-scale motions break down into smaller, higher frequency motions. The microscale-only simulation does receive fully developed turbulent flow at the lateral boundaries of the 2-m domain, but the 10-m parent domain does not resolve the larger scales of turbulence that are generated and downscaled using the telescoping grids of the multiscale simulation.
When a surface heat flux is added to the microscale-only simulation, the turbulence developed in the 10-m parent includes more resolved TKE at the lower frequencies (larger scales) relative to the neutral microscale-only simulation. Because of the turbulent cascade, TKE from these larger eddies increases TKE at smaller scales, which is visible in Fig. 8 because the microscale-only HFX simulation resolves more TKE at all scales than does the neutral microscale-only simulation.
The microscale-only HFX simulation and both of the multiscale simulations overestimate the energy in large-scale motions starting approximately at frequencies less than 0.05 s−1. The overestimation can be partially attributed to the flat topography and lack of roughness elements within the 10-m domain that would disrupt the flow. For the multiscale simulations, this overestimation may also result from persistent large-scale flow structures that are not realistically cascading into smaller scale motions. The use of a more sophisticated LES turbulence closure model, such as the dynamic reconstruction method (Chow et al. 2005; Chow and Street 2009), might help to reduce this buildup of energy in the large scales. In the LES domains, extending the fetch—the distance between the inflow boundaries and grid refinement interfaces—may also improve the results by providing additional time for the flow to develop smaller scales of turbulence prior to downscaling.
Configuring a multiscale simulation requires striking a difficult-to-optimize balance between the simulation’s computational costs, the resolution and extent of each grid, and the degree of physical accuracy that can be expected. The sensitivity of terrain-following coordinates and different IBM algorithms to grid resolution for simulations over complex terrain has been explored (Bao 2018; Wiersema 2019; Arthur et al. 2020); however, significantly more research is required before any “best practices” can be decisively stated.
Enabling the CPM in the multiscale simulation results in a clear increase in the energy resolved at frequencies greater than 0.04 s−1. As expected, the CPM promotes the development of turbulence on the 30-m domain, and highly energetic turbulent features that develop are then downscaled through the 10-m domain and into the 2-m domain, where they are a major contributor to the increase in resolved TKE seen in Fig. 8.
8. Conclusions
The simulations and analysis within this paper demonstrate that, relative to microscale-only simulations, multiscale simulations can often improve predictions of transport, dispersion, and TKE. Four simulation configurations are compared: two traditional microscale-only simulations and two six-domain multiscale simulations. These simulations predict wind flow and the transport and mixing of SF6 throughout the central business district of Oklahoma City during IOP3 of the JU2003 field campaign. The microscale-only simulations use a two-domain nested configuration with idealized boundary conditions and forcing that is tuned using local observations from JU2003. The multiscale simulations use a six-domain nested configuration, which spans from the mesoscale to the microscale, with initial conditions and forcing provided by NARR, a meteorological forecast product.
Several statistical measures of model skill are used to evaluate the performance of each simulation versus JU2003 observations. As was observed in Wiersema et al. (2020), the microscale-only simulations demonstrate the most skill at predicting wind speed and direction, which is attributed to tuning of the simulation forcing to match JU2003 observations. The multiscale simulations show the most skill at predicting SF6 concentrations. This increased skill of the multiscale simulation results from downscaling of large-scale turbulence through a telescoping series of nested domains. These results demonstrate the importance of including large scales when forcing a microscale simulation. Even though the microscale-only simulation with a surface heat flux best predicts TKE, the absence of large-scale turbulence results in a more constrained plume of SF6 in comparison with both the multiscale simulations and the JU2003 observations.
Enabling the CPM of Muñoz-Esparza et al. (2015) was found to promote the development of turbulence and improve the transition from a parent domain using a PBL scheme to a LES child domain; however, the associated increase in downward transport of high momentum flow in this case exacerbated the existing positive wind speed bias in the 30- and 10-m domains. Care should be taken when developing general conclusions about the CPM because this study evaluates a single region, during a short time window, and a unique model configuration. Future research should focus on evaluating the CPM under a diverse range of stability and forcing conditions, and on reducing the positive wind speed bias by improving the representation of poorly resolved obstacles, such as buildings, and including their effects on flow in the intermediate resolution domains. Additionally, future studies should consider more advanced turbulence closures, such as the dynamic reconstruction model (Chow et al. 2005), that may improve the development of turbulence at intermediate resolutions and promote the breakup of persistent large-scale flow features (Mirocha et al. 2013).
Perhaps the most interesting and important implication of these results is that the microscale-only HFX simulation, which better predicted wind speed, direction and TKE, relative to the multiscale simulations, did not best predict SF6 concentrations because of the lack of large-scale variability in the flow. The importance of large-scale variability in simulations of JU2003 IOP 3, an experiment with near-idealized conditions, indicates that large-scale variability is likely to be of importance for all microscale simulations. As multiscale modeling becomes more computationally feasible and commonplace, modelers should strive to include and resolve the effects of intermediate scales and the variability they introduce to the microscale domains.
Acknowledgments.
This work was partially supported by the LLNL Laboratory Directed Research and Development Program as Project 18-ERD-049 and the LLNL postdoctoral program. This document has been reviewed and approved for release (LLNL-JRNL-829389). This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
Data availability statement.
A publicly released version of the WRF-IBM code, which includes a microscale-only JU2003 test case, is openly available from Zenodo (https://doi.org/10.5281/zenodo.6081827).
REFERENCES
Allwine, K. J., and J. E. Flaherty, 2006: Joint Urban 2003: Study overview and instrument locations. Pacific Northwest National Laboratory Tech. Rep. PNNL-15967, 92 pp., https://www.pnnl.gov/publications/joint-urban-2003-study-overview-and-instrument-locations.
Arthur, R. S., K. A. Lundquist, D. J. Wiersema, J. Bao, and F. K. Chow, 2020: Evaluating implementations of the immersed boundary method in the weather research and forecasting model. Mon. Wea. Rev., 148, 2087–2109, https://doi.org/10.1175/MWR-D-19-0219.1.
Bao, J., 2018: An improved immersed boundary method for atmospheric boundary layer simulations over complex terrain. Ph.D. thesis, University of California, 95 pp.
Bao, J., K. A. Lundquist, and F. K. Chow, 2018: Large-eddy simulation over complex terrain using an improved immersed boundary method in the Weather Research and Forecasting Model. Mon. Wea. Rev., 146, 2781–2797, https://doi.org/10.1175/MWR-D-18-0067.1.
Burrows, D. A., E. A. Hendricks, S. R. Diehl, and R. Keith, 2007: Modeling turbulent flow in an urban central business district. J. Appl. Meteor. Climatol., 46, 2147–2164, https://doi.org/10.1175/2006JAMC1282.1.
Calhoun, R., F. Gouveia, J. Shinn, S. Chan, D. Stevens, R. Lee, and J. Leone, 2004: Flow around a complex building: Comparisons between experiments and a Reynolds-averaged Navier–Stokes approach. J. Appl. Meteor. Climatol., 43, 696–710, https://doi.org/10.1175/2067.1.
Chan, S. T., and M. J. Leach, 2007: A validation of FEM3MP with Joint Urban 2003 data. J. Appl. Meteor. Climatol., 46, 2127–2146, https://doi.org/10.1175/2006JAMC1321.1.
Chang, J. C., and S. R. Hanna, 2004: Air quality model performance evaluation. Meteor. Atmos. Phys., 87, 167–196, https://doi.org/10.1007/s00703-003-0070-7.
Chow, F. K., and R. L. Street, 2009: Evaluation of turbulence closure models for large-eddy simulation over complex terrain: Flow over Askervein hill. J. Appl. Meteor. Climatol., 48, 1050–1065, https://doi.org/10.1175/2008JAMC1862.1.
Chow, F. K., R. L. Street, M. Xue, and J. H. Ferziger, 2005: Explicit filtering and reconstruction turbulence modeling for large-eddy simulation of neutral boundary layer flow. J. Atmos. Sci., 62, 2058–2077, https://doi.org/10.1175/JAS3456.1.
Chow, F. K., B. Kosović, and S. Chan, 2008: Source inversion for contaminant plume dispersion in urban environments using building-resolving simulations. J. Appl. Meteor. Climatol., 47, 1553–1572, https://doi.org/10.1175/2007JAMC1733.1.
Connolly, A., L. van Veen, J. Neher, B. J. Geurts, J. Mirocha, and F. K. Chow, 2021: Efficacy of the cell perturbation method in large-eddy simulations of boundary layer flow over complex terrain. Atmosphere, 12, 55, https://doi.org/10.3390/atmos12010055.
Daniels, M. H., K. A. Lundquist, J. D. Mirocha, D. J. Wiersema, and F. K. Chow, 2016: A new vertical grid nesting capability in the Weather Research and Forecasting (WRF) Model. Mon. Wea. Rev., 144, 3725–3747, https://doi.org/10.1175/MWR-D-16-0049.1.
Davies, T., 2014: Lateral boundary conditions for limited area models. Quart. J. Roy. Meteor. Soc., 140, 185–196, https://doi.org/10.1002/qj.2127.
Gowardhan, A. A., E. R. Pardyjak, I. Senocak, and M. J. Brown, 2011: A CFD-based wind solver for an urban fast response transport and dispersion model. Environ. Fluid Mech., 11, 439–464, https://doi.org/10.1007/s10652-011-9211-6.
Hanna, S., and Coauthors, 2011: Comparisons of JU2003 observations with four diagnostic urban wind flow and Lagrangian particle dispersion models. Atmos. Environ., 45, 4073–4081, https://doi.org/10.1016/j.atmosenv.2011.03.058.
Knievel, J. C., G. H. Bryan, and J. P. Hacker, 2007: Explicit numerical diffusion in the WRF Model. Mon. Wea. Rev., 135, 3808–3824, https://doi.org/10.1175/2007MWR2100.1.
Lee, G.-J., D. Muñoz-Esparza, C. Yi, and H. J. Choe, 2019: Application of the cell perturbation method to large-eddy simulations of a real urban area. J. Appl. Meteor. Climatol., 58, 1125–1139, https://doi.org/10.1175/JAMC-D-18-0185.1.
Li, H., G. Cui, and Z. Zhang, 2018: A new scheme for the simulation of microscale flow and dispersion in urban areas by coupling large-eddy simulation with mesoscale models. Bound.-Layer Meteor., 167, 145–170, https://doi.org/10.1007/s10546-017-0323-5.
Lundquist, K. A., F. K. Chow, and J. K. Lundquist, 2012: An immersed boundary method enabling large-eddy simulations of complex terrain in the WRF Model. Mon. Wea. Rev., 140, 3936–3955, https://doi.org/10.1175/MWR-D-11-00311.1.
Mesinger, F., and Coauthors, 2006: North American Regional Reanalysis. Bull. Amer. Meteor. Soc., 87, 343–360, https://doi.org/10.1175/BAMS-87-3-343.
Mirocha, J. D., G. Kirkil, E. Bou-Zeid, F. K. Chow, and B. Kosović, 2013: Transition and equilibrium of neutral atmospheric boundary layer flow in one-way nested large-eddy simulations using the Weather Research and Forecasting Model. Mon. Wea. Rev., 141, 918–940, https://doi.org/10.1175/MWR-D-11-00263.1.
Mirocha, J. D., and K. A. Lundquist, 2017: Assessment of vertical mesh refinement in concurrently nested large-eddy simulations using the Weather Research and Forecasting Model. Mon. Wea. Rev., 145, 3025–3048, https://doi.org/10.1175/MWR-D-16-0347.1.
Muñoz-Esparza, D., and B. Kosović, 2018: Generation of inflow turbulence in large-eddy simulations of nonneutral atmospheric boundary layers with the cell perturbation method. Mon. Wea. Rev., 146, 1889–1909, https://doi.org/10.1175/MWR-D-18-0077.1.
Muñoz-Esparza, D., B. Kosović, J. Mirocha, and J. van Beeck, 2014: Bridging the transition from mesoscale to microscale turbulence in numerical weather prediction models. Bound.-Layer Meteor., 153, 409–440, https://doi.org/10.1007/s10546-014-9956-9.
Muñoz-Esparza, D., B. Kosović, J. van Beeck, and J. Mirocha, 2015: A stochastic perturbation method to generate inflow turbulence in large-eddy simulation models: Application to neutrally stratified atmospheric boundary layers. Phys. Fluids, 27, 035102, https://doi.org/10.1063/1.4913572.
Olson, J. B., and Coauthors, 2019: Improving wind energy forecasting through numerical weather prediction model development. Bull. Amer. Meteor. Soc., 100, 2201–2220, https://doi.org/10.1175/BAMS-D-18-0040.1.
Sauer, J. A., and D. Muñoz-Esparza, 2020: The FastEddy® resident-GPU accelerated large-eddy simulation framework: Model formulation, dynamical-core validation and performance benchmarks. J. Adv. Model. Earth Syst., 12, e2020MS002100, https://doi.org/10.1029/2020MS002100.
Skamarock, W. C., 2004: Evaluating mesoscale NWP models using kinetic energy spectra. Mon. Wea. Rev., 132, 3019–3032, https://doi.org/10.1175/MWR2830.1.
Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-475+STR, 113 pp., https://doi.org/10.5065/D68S4MVH.
Tabor, G. R., and M. H. Baba-Ahmadi, 2010: Inlet conditions for large eddy simulation: A review. Comput. Fluids, 39, 553–567, https://doi.org/10.1016/j.compfluid.2009.10.007.
Warner, T. T., R. A. Peterson, and R. E. Treadon, 1997: A tutorial on lateral boundary conditions as a basic and potentially serious limitation to regional numerical weather prediction. Bull. Amer. Meteor. Soc., 78, 2599–2618, https://doi.org/10.1175/1520-0477(1997)078<2599:ATOLBC>2.0.CO;2.
Wiersema, D. J., 2019: Mesoscale to microscale atmospheric modeling over complex terrain. Ph.D. thesis, University of California, 89 pp.
Wiersema, D. J., K. A. Lundquist, and F. K. Chow, 2020: Mesoscale to microscale simulations over complex terrain with the immersed boundary method in the Weather Research and Forecasting Model. Mon. Wea. Rev., 148, 577–595, https://doi.org/10.1175/MWR-D-19-0071.1.
