1. Introduction
Large-eddy simulation (LES) is widely recognized as a promising method to precisely represent turbulence. Many atmospheric models have started to employ this technique because the grid spacing in high-resolution simulations approaches O(10–100) m, where large eddies in the planetary boundary layer (PBL) are resolved explicitly. In the near future, recent computer technology developments are expected to help conduct global atmospheric simulations with a horizontal resolution of O(10–100) m (Satoh et al. 2019). Global LES models are expected to improve the representation of the turbulence process and solve several deficiencies observed in the existing global climate models. For example, global LES may provide a significantly better representation of shallow clouds at the top of PBL, which is a critical issue for maintaining the energy balance of global climate.
In the LES, resolved flow and subgrid scale (SGS) effects are separated with a specific wavelength in the inertial subrange by introducing a spatial filter. The filtered governing equations directly calculate the resolved flow while the SGS effect due to the unresolved eddies is parameterized using turbulence models. For example, the Smagorinsky–Lilly turbulence model is a commonly used model (Smagorinsky 1963; Lilly 1962) where the SGS effect is modeled using the eddy viscosity. Here, numerical errors due to the discretization of the filtered equations need to be sufficiently smaller than the SGS effect; otherwise, a physical meaning to the SGS terms can disappear. Thus, we need to design the discretization of governing equations while considering a balance between the SGS terms and the numerical errors with other terms.
The problem of the numerical errors dominating over the SGS terms has been historically discussed in the research field of computational fluid dynamics (CFD) (e.g., Rogallo and Moin 1984; Vreman et al. 1994; Ghosal 1996; Brandt 2006). In the context of meteorological simulations, we investigated the requirement for the order of accuracy of the advection schemes in the stencil method for PBL turbulence applications (Kawai and Tomita 2021, hereafter KT21). Two ratios associated with numerical diffusion and dispersion errors were derived in KT21 to provide numerical criteria with an acceptable order of accuracy: Rdiff and Rdisp. The term Rdiff represents the ratio of decay time with the SGS terms to that of the numerical diffusion error terms; Rdisp represents the ratio of phase speed due to the error of the advection scheme to that of the SGS terms. KT21 concluded that the seventh-order or eighth-order accuracy is required for keeping both Rdisp and Rdisp less than 10−1 at wavelengths longer than eight grid lengths for grid spacing simulations of O(10) m.
A finite difference method (FDM) or finite volume method (FVM) with totally second-order spatial accuracy is often employed in state-of-the-art global nonhydrostatic models (e.g., Tomita and Satoh 2004; Skamarock et al. 2012; Zängl et al. 2015). However, the theory of KT21 suggested that the numerical accuracy of the dynamical cores needs to be considerably higher than the second-order to perform atmospheric LES precisely. Although enlarging the used stencil can achieve such high-order accuracy in conventional FVM approaches (e.g., Ullrich et al. 2010; Ullrich and Jablonowski 2012), the formulations tend to become complex because of the distinction between the cell-center point and cell-averaged values. Further, a more significant problem is the deterioration of the computational locality in the high-order stencil methods; this issue can lead to high communication costs between computer nodes in massive parallel computers.
To avoid these problems, we now focus on a discontinuous Galerkin method (DGM), originally introduced by Nitsche (1971) and Reed and Hill (1973). As mentioned in Nair et al. (2005), the DGM can be interpreted as a type of finite-element method (FEM) that permits the discontinuities of the numerical solution at interelement interfaces. The numerical fluxes at the element interfaces are calculated using approximate Riemann solvers like the FVM with piecewise reconstruction. Thus, the high-order numerical accuracy can be easily achieved by increasing the degrees of freedom (DOF) within a finite element. The DGM also achieves computational locality by employing the element-wise calculation of flux terms.
The literature (e.g., Marras et al. 2016; Gassner and Winters 2021) indicates that DGM has been used in a wide range of research fields, including geophysics and meteorology. In the field of CFD, LES based on high-order element-based methods such as DGM has been successfully applied to solve industrial flow problems, which are characterized by high Reynolds numbers and complex geometry (e.g., Mengaldo et al. 2021). For atmospheric simulations, Giraldo et al. (2002) applied a nodal DGM (e.g., Hesthaven and Warburton 2007) using high-order nodal Lagrange polynomials associated with the Legendre–Gauss–Lobatto (LGL) points to the spherical shallow-water equations. Nair and Tufo (2007) and Choi et al. (2006) constructed a three-dimensional global hydrostatic atmospheric dynamical core based on DGM for the horizontal discretization. Regarding the computational performance of DGM, Nair et al. (2009) evaluated their dynamical core on a massively parallel computer. Further, DGM has been used in nonhydrostatic mesoscale modeling (e.g., Giraldo and Restelli 2008; Restelli and Giraldo 2009; Kelly and Giraldo 2012).
In contrast to DGM’s success in simulating large-scale and mesoscale flows in the atmosphere, the suitability of DGM for atmospheric LES in smaller scale flows such as PBL turbulence is yet to be investigated extensively. Considering the goal of realizing high-resolution atmospheric simulations in near future, one important research direction is clarifying the wavelength range in which numerical errors for dissipation and dispersion are sufficiently smaller than the SGS term under DGM of a certain order. Following the fundamental idea in KT21, the numerical criteria should be extended to the DGM framework.
We encountered a new difficulty in DGM, which does not exist in the grid-point method. KT21 expressed numerical errors associated with the spatial discretization of advection terms by utilizing the modified equations of the one-dimensional linear advection equation obtained using the finite difference approximations of various order. Unfortunately, such a straightforward approach is impractical when using DGM. This is because it is significantly complex to derive the modified equation in DGM at high polynomial order. Thus, a different approach is required for DGM. Recent studies proposed approaches to quantify the numerical errors of DGM such as the Fourier eigenanalysis (e.g., Moura et al. 2015b; Alhawwary and Wang 2018; Mengaldo et al. 2018; Moura et al. 2022; Tonicello et al. 2021) and the nonmodal analysis (Fernandez et al. 2019). In the context of implicit LES, in which the numerical dissipation plays the role of turbulent models, Moura et al. (2017) indicated that their guideline (referred to as the “1% rule”), which is based on the eigenanalysis (Moura et al. 2015b), is useful to estimate the wavenumber at which the artificial dissipation has a dominant role on energy spectra in underresolved turbulent simulations. Following these studies, we overcome our issue in the derivation of Rdiff and Rdisp by applying the eigenanalysis.
We focus on four factors in this study: the polynomial order, the temporal accuracy, the intensity of modal filter, and the choice of numerical flux. First, we evaluate the order of polynomials satisfying our numerical criteria in the semidiscrete analysis. Second, based on this semidiscrete analysis, we examine the low-order temporal scheme acceptable in high-order spatial discretization. Third, we investigate sensitivity of the order and decay coefficient of the modal filter, which is often necessary to stabilize the simulations. As indicated by previous studies (e.g., Gassner and Beck 2013; Winters et al. 2018), numerical dissipation with high-order schemes is insufficient to prevent the aliasing-driven instability related to nonlinear terms. In our numerical experiments, although the turbulent model adds the eddy diffusion, the total dissipation remains insufficient near the two grid scale. Thus, we require the modal filter. In this case, the artificial parameters (i.e., order and decay coefficient) need to be selected based on our numerical criteria when including the modal filter effect. Fourth, we identify how to define the numerical flux at the boundary elements, which is a peculiar problem in DGM. In previous studies on atmospheric models using DGM, the Rusanov flux (Rusanov 1961) was often adopted as the numerical flux as it has a simple formulation. Further, numerical results can be inessential to the numerical flux with an increase in polynomial order and spatial resolution (e.g., Hesthaven and Warburton 2007). However, previous studies suggest that the Rusanov flux is inappropriate for a low Mach number flow in some situations (e.g., Ullrich et al. 2010; Moura et al. 2017; Mengaldo et al. 2018). Thus, we investigate the effect of numerical flux in the atmospheric LES using DGM by changing the coefficient included in the numerical stabilization term following Mengaldo et al. (2018) and Bassi et al. (2017).
To validate the indication from derived numerical criteria, we perform a series of numerical experiments on an idealized PBL turbulence, following KT21. The numerical criterion indicated by Rdiff can be validated directly because the effect of numerical diffusion appears directly on the energy spectra. In contrast, as described in section 3d in KT21, it is difficult to directly validate the criterion for numerical dispersion, i.e., Rdisp. It should be noted that Rdiff and Rdisp are derived using the same procedure. Therefore, it is natural to consider that the numerical criterion for the numerical dispersion will be valid if the validity of the numerical criterion for the numerical diffusion is confirmed.
Although we follow the eigenanalysis shown in the previous works to quantify the numerical errors with DGM, our study has the following two novelties: first, the introduction of new indices for LES, which enables the evaluation of different kinds of numerical schemes, and second, investigating the validity of the indices by conducting an atmospheric LES experiment. By extending the numerical criteria in KT21 to the DGM framework, the comparison between FDM (or FVM) and DGM becomes easy through a series of our studies. This implies that our methodology can be used to discuss the two types of numerical methods regarding their order of accuracy on the same ground. Furthermore, using our newly constructed LES model based on DGM, we theoretically investigate the polynomial order necessary for LES in the case of atmospheric PBL turbulence and validate the numerical criteria.
Our proposed method for deriving numerical criteria is applicable to not only DGM but also similar numerical methods, e.g., spectral difference method (SDM) (e.g., Liu et al. 2006) and spectral volume method (SVM) (e.g., Wang 2002). The SDM is closely related to a class of nodal DGM based on a strong form that provides a differential form. We consider that the findings obtained in this study can be applied to SDM in a relatively direct manner. Although SVM can avoid the volume integral required in DGM, there still exists an issue that needs to be overcome: the partitioning of the spectral volume into sub control volumes needs to be investigated as it determines the numerical accuracy and stability for the significantly high-order scheme (Harris and Wang 2009). In this study, we prefer fewer constraints with arbitrary order accuracy when exploring the numerical behavior of various order schemes.
The rest of this paper is organized as follows: In section 2, the amplification factor and phase shift error are derived based on the eigensolution approach when the one-dimensional linear advection equation is semidiscretized by the nodal DGM. These values are used to formulate Rdiff and Rdisp. Section 3 presents the behavior of the numerical diffusion and numerical dispersion errors in the DGM. The quantification of Rdiff and Rdisp reveals the polynomial order necessary for the atmospheric LES. However, some gaps still exist between such a simplified analysis based on the semidiscrete formula and the actual numerical experiments. To bridge these gaps, we investigate the impacts of temporal discretization, modal filtering for numerical stability, and the hyperupwinding in low Mach number flows, on the numerical criteria. In section 4, to validate the numerical criteria, we perform a series of numerical experiments of PBL turbulence using our newly developed dynamical core based on nodal DGM. We show the simulation results with focus on the dependence of the energy spectra on the polynomial order. In addition, we evaluate the effect of temporal discretization, the order and decay coefficient in the modal filter, and the numerical flux in the idealized numerical experiment. In section 5, we discuss the choice of numerical flux and the reduction of artificial filter. Moreover, the problems that arise when using high-order DGM for atmospheric LES are mentioned. Finally, in section 6, we summarize our study regarding the numerical accuracy necessary for LES when using DGM and provide the future direction for our work.
2. Formulation of numerical criteria in the DGM framework
Two indices for estimating numerical errors, Rdiff and Rdisp, are formulated in the framework of the nodal DGM. KT21 used a modified equation to derive the indices in the case of FDM. We will face serious difficulty if we derive the modified equation of DGM in the same manner: if there are one or two DOFs in an element, the corresponding modified equation can be derived via algebraic manipulations (Moura et al. 2015a). However, if the DOFs are more than two, such a manipulation becomes too difficult. Further, we must derive not only the leading error term but also the successive high-order error terms for the precise assessment of the numerical dispersion and dissipation properties at the short wavelengths. To avoid such complex manipulation, this study quantifies Rdiff and Rdisp by the temporal eigensolution analysis (e.g., Moura et al. 2015b; Alhawwary and Wang 2018).
a. Semidiscrete equation of one-dimensional linear advection equation
Although this study uses the strong form with the nodal representation associated with the LGL points, previous studies used the weak form with the modal representation (Moura et al. 2015b; Alhawwary and Wang 2018). This difference in representations does not essentially affect the results of eigenanalysis in the one-dimensional linear advection equation. We use the strong form with the nodal representation because of the easy generalization of our numerical criteria derivation to other schemes. For example, in nonlinear fluid simulations, the split-form nodal DGM is sometimes used for preventing aliasing errors (e.g., Gassner et al. 2016; Gassner and Winters 2021). Further, it is useful in the flux reconstruction scheme that provides a unified formulation to cover various numerical methods with locally defined multiple degrees of freedom (e.g., Huynh 2007; De Grazia et al. 2014; Asthana et al. 2015; Mengaldo et al. 2016).
b. Numerical diffusion and dispersion errors derived by eigensolution analysis
We perform the temporal eigenanalysis to investigate spatial numerical errors in Eq. (6). Assuming a wave with a real wavenumber k as the initial condition, we derive its numerical frequency
c. Derivation of Rdiff and Rdisp based on eigensolution analysis
3. Quantification of numerical criteria in the DGM framework
We first investigate the behavior of the numerical diffusion and numerical dispersion errors for the semidiscretized linear advection equation in Eq. (6). Based on the results of this investigation, we discuss the polynomial order necessary for the atmospheric LES in the DGM framework by evaluating Rdiff in Eq. (19) and Rdisp in Eq. (20). Subsequently, to bridge the gap between the simplified consideration based on the semidiscrete analysis and the actual atmospheric LES, we investigate the impact of the following factors on our numerical criteria: temporal discretization errors, modal filters for numerical stabilization, and the effect of hyperupwinding at low Mach numbers.
a. Fundamental behavior of numerical errors in DGM
The modified wavenumber Km is obtained by solving the eigenvalue problem in Eq. (8). The real [Re(Km)] and imaginary [Im(Km)] parts respectively indicate the numerical dispersion and numerical dissipation errors. Figures 1a–d show them for p = 1 and 3, respectively. The black solid lines present the primary modes; they accurately follow the exact relation in the short wavenumber range. For p = 1, Fig. 1a shows that Re(km) for the primary mode gradually deviates from the exact value with K ≃ π/4 as the boundary. It has a peak at K ∼ 5π/8 and then rapidly decreases with K. Figure 1b indicates that the primary mode with K > π/4 dissipates by a factor of at least 1/e after the waves advance by a few grid lengths. Thus, in terms of the numerical dispersion, the primary mode poorly resolved at K > π/4 is effectively dissipated. The end of well-resolved wavelength extends to short wavelength with an increase in p. For example, for p = 3, it extends to K ∼ 3π/8, as shown in Figs. 1c and 1d.
The secondary modes are represented by the colored lines in Fig. 1. They are generally damped in the well-resolved wavenumber range, whereas their behavior seems to correct the error of the primary mode in the higher wavelength range. For the precise evaluation, we need the following combined analysis. The results indicate the true behaviors of numerical dispersion and diffusion errors considering both the primary and secondary modes. Figure 2 shows the amplification factor G in Eq. (10) and the phase difference between the exact and numerical solutions Δψ in Eq. (11). The parameters used here are three different times s = 10, 100, and 1000 with uΔt/Δxeff = 6.25 × 10−3. In each figure, the solid lines represent G and Δψ for the combined modes, and the dashed lines are for the primary modes. After enough time, the solid and dashed lines almost overlap in the low wavenumber range. Thus, the primary mode almost characterizes the numerical errors after enough time owing to the rapidly damped secondary modes.
Considering this aspect in the range of relatively long wavelengths such as K < π/2, it is sufficient to examine only the primary mode after sufficient time. However, the above results are the case of the semidiscrete analysis that considers discretization only in the spatial direction. When considering temporal discretization, it is not clear if the effect of the secondary modes can be ignored in the target wavelength range because of the possibility of the interaction between eigenmodes caused by the time discretization error. The same is true when introducing modal filters. The interaction between the eigenmodes occurs when the filter is adopted. Thus, hereafter, we use the combined mode for all discussions.
b. Evaluation of Rdiff and Rdisp based on the semidiscrete analysis
We discuss the dependence of Rdiff and Rdisp on the effective grid spacing (Δxeff) and wavelength (l). Although the acceptable value of the two indices can be arbitrary chosen, we set the criteria in which they should be below 10−1 in the range of 8Δxeff or more for a resolution of O(10) m. These criteria were also employed in KT21. The parameters in these indices were set as |u| = 5 m s−1 and m = 2. When a fully explicit temporal scheme is adopted, the acoustic wave restricts the time step. We set the Courant number against the advective velocity cr,DG so that |u|Δt/Δxeff = 6.25 × 10−3. This setting is based on the numerical stability in the actual LES experiments performed in section 4.
Figure 3 shows the temporal evolutions from s = 1 to s = 1000 for various polynomial orders (p = 1, 3, 4, and 7). In the figure, the solid red line and color shade represent Rdiff, while the dashed black line represents Rdisp. It should be noted that s is the nondimensional time level. For s = 1000, the waves advance about 6Δxeff when using the above value of cr,DG. The damping effect of the numerical viscosity near grid scales settles down at a relatively early stage; the value of Rdiff reduces with an increase in s. On the other hand, Rdiff increases with s at l > 8, which is our target wavelength range. This aspect appears remarkably for p = 1; however, it is not significant with an increase in p. Thus, we can say that p = 1 is not acceptable because of the strong numerical viscosity at l > 8. With respect to numerical dissipation, p = 3 is still less accurate at a resolution of O(10) m. For p = 4, the lines of Rdiff and Rdisp equal to 10−1 are located below l = 8 in the O(10) m spatial resolution. Therefore, at least p = 4 is required to satisfy our numerical criteria with both numerical dissipation and dispersion.
Finally, we compare the Rdiff and Rdisp diagrams between FDM (KT21) and DGM. The comparison of Figs. 4a and 4b or that of Figs. 4c and 4d suggests that the Rdiff and Rdisp diagrams for upwind DGM with the pth-order polynomial corresponds to that for the (2p + 1)-order upwind FDM. Thus, compared to FDM with the same order, the numerical errors for DGM are significantly small. This is because DGM has spectral superconvergence property at the well-resolved wavenumber range, which has been extensively investigated in previous works (e.g., Hu and Atkins 2002; Ainsworth 2004; Marchandise et al. 2008). According to these works, the numerical dispersion and diffusion errors with the primary mode in the one-dimensional linear advection equation are known to converge like (khe)2p+3 and (khe)2p+2, respectively. These orders of spectral convergence rate are considerably higher than the actual formal accuracy determined by the projection errors [
c. Influence of temporal discretization
In section 3b, we treat the semidiscrete equation and do not consider temporal discretization. It is preferable to adopt the same order temporal scheme as that of spatial discretization to prevent the temporal errors from contaminating the spatial accuracy. On the other hand, KT21 showed that the low order of the temporal schemes compared to the spatial order in the FDM can be accepted. This comes from considerably smaller temporal errors than the spatial errors when the full explicit scheme is used for low Mach number flows. We examine such a possibility also in the DGM framework. In DGM, Runge–Kutta (RK) schemes are often used as the temporal scheme. We perform a full discrete analysis based on Alhawwary and Wang (2018) to evaluate the impact of RK scheme on Rdiff and Rdisp. Here, we investigate the third-order RK with three stages (RK3) and fourth-order RK with 10 stages (RK4s10) (Ketcheson 2008). Although the number of stages in RK4s10 is greater than that in the classical fourth-order RK with four stages, the former is often more efficient for the spatial operator of the upwind DGM because it allows a longer stable time step normalized by the number of right-hand side evaluations.
Figure 5 shows Rdiff and Rdisp diagrams using the full-discrete analysis for the case of a small Courant number against advection (cr,DG ≪ 1). For p = 4, the change of Rdiff and Rdisp in the temporal discretization is small even for RK3. The spatial errors characterize the leading numerical errors in this case. For p = 7 with RK3, the difference from the semidiscrete analysis exists at long wavelengths e.g., l > 10; however, such numerical errors are themselves significantly small compared to our numerical criteria. For RK4s10, the change in Rdiff and Rdisp is so small that the diagrams are indistinguishable between the semidiscrete and full-discrete analyses, even for the high-order polynomial of p = 7. We can conclude that the fourth-order temporal discretization gives a sufficient accuracy and the third-order one achieves acceptable accuracy for high-order DGM such as p ≥ 4. Note that this is valid for the case of full explicit scheme with a small Courant number against advection, i.e., cr,DG ≪ 1.
As indicated in KT21, this modest requirement for temporal-order accuracy needs to be carefully evaluated as cr,DG approaches unity. In this case, the effect of the leading temporal error terms becomes essential to the total errors. Figure 6 shows Rdiff and Rdisp diagrams for the nearly maximum stability of the Courant number against advection. We show the results for cr,DG = 3.125 ×10−1/(p + 1). If the value of cr,DG is slightly increased by 5%, numerical instability occurs for p = 7 with RK3. Similar to the case of small cr,DG, we present the results at the time when the wave advances about 6Δxeff. Figure 6 indicates that the numerical diffusion with RK3 contaminates the effect of SGS terms, and RK3 no longer satisfies our numerical criteria for the O(10) m grid spacing. In addition, it contaminates high accuracy at well-resolved wavelengths for high-order DGM. RK4s10 is acceptable; however, these temporal errors affect extremely accurate solutions at well-resolved wavelengths. When RK4s10 is adopted, we can use a longer time step (by a factor of about 4 for p = 7), compared to that when using RK3. The results for such scenarios are presented in the rightmost column of Fig. 6. The larger time step in RK4s10 still satisfies our numerical criteria although the error increase by about one order of magnitude in exchange for an efficient temporal integration. We can conclude that the fourth-order temporal accuracy is acceptable under our numerical criteria for the Courant number against advection which is associated with the stability limit of upwind high-order DGM.
d. Impact of modal filter
Figure 7 shows the effect of the decay coefficient for pm = 32. First, we note the differences between the usual hyperdiffusion (with ∂2n/∂x2n operator) and modal filter based on expansion polynomials. The latter affects both numerical diffusion and numerical dispersion and has an impact not only on short wavelengths but also on relatively long wavelengths in the Fourier spaces. We can easily understand that the result for the upwind DGM with p = 1 would be degraded into the first-order scheme by essentially removing the highest mode. Figure 7 indicates that the decay coefficient αm should be less than O(10−2) for p = 4, whereas αm = O(10−1) is allowed for a higher-order DGM such as p = 7.
To investigate the effect of the order of modal filter, Fig. 8 shows Rdiff and Rdisp diagrams for pm = 32, 16, 8 in which αm is fixed to 10−2. According to Hesthaven and Kirby (2008), the filtering error with pmth-order filter decreases as
In summary, to maintain the required order derived in section 3b, the decay coefficients with a modal filter should be less than O(10−2) for p = 4, and the order of the modal filter should be higher than 2p + 2 for αm = O(10−2).
e. Effect of hyperupwinding with upwind numerical flux
In previous atmospheric simulations using DGM (e.g., Giraldo and Restelli 2008; Restelli and Giraldo 2009), the Rusanov flux (Rusanov 1961) (or equivalently local Lax–Friedrichs flux) is often adopted as the numerical flux because of its simplicity. However, some previous studies indicated that the formulation can be inappropriate for low Mach number flows (e.g., Ullrich et al. 2010; Moura et al. 2017; Mengaldo et al. 2018). This is because the decay coefficient with the stabilization terms is proportional to the sum of the sound speed and local wind speed, and it is too large for the advective time scale. Such a situation is referred to as “overupwinding” or “hyperupwinding” in Mengaldo et al. (2018). In the context of atmospheric simulations, the problem with the Rusanov flux for the low Mach number flows has been investigated in FVM (e.g., Ullrich et al. 2010). On the other hand, the treatment of numerical flux is considered inessential when p increases, as pointed out by Hesthaven and Warburton (2007). For smooth fields, the jump between left and right states at the cell boundary decreases as the order of accuracy or spatial resolution increases; as a result, there is less discontinuity, and the difference with numerical fluxes becomes small because all numerical fluxes approach a single value according to the condition of consistency. Thus, we expect that the choice of numerical fluxes becomes unimportant when the high-order DGM is used. Following Mengaldo et al. (2018), we tested the impact of hyperupwinding by increasing β.
Figures 9 and 10 show the effect of hyperupwinding with an upwind numerical flux for β = 70 assuming that |u| = 5 m s−1 and the sound speed is 350 m s−1. In low-order DGM such as p = 1, there are remarkable differences between the standard upwinding (β = 1) and the hyperupwinding (β = 70) cases. At the early stage (e.g., s = 10) as shown in Fig. 9, the numerical dissipation near the grid scales for the hyperupwinding is stronger compared to that for the standard upwinding. Interestingly, after enough time (e.g., s = 1000) as shown in Fig. 10, the numerical dissipation for hyperupwinding becomes smaller than that for the standard upwinding. However, the differences between the standard and hyperupwinding becomes negligible with an increase in p. The numerical errors for the hyperupwinding case vary with wavelength more nonmonotonically than that with the standard upwinding. As indicated by the rightmost column in Figs. 9 and 10, these behaviors with hyperupwinding tend to disappear when using the modal filter. Therefore, as far as we adopt p ≥ 4 with the modal filters, we expect that the hyperupwinding associated with the Rusanov flux for low Mach number flows does not affect our numerical criteria.
4. Validation of numerical criteria via numerical experiment of idealized PBL turbulence
We conduct a series of idealized numerical experiments of PBL turbulence to validate our numerical criteria indicated by Rdiff and Rdisp. Except for the discretization method, the experimental setup is the same as that in Nishizawa et al. (2015, hereafter N15) and KT21. While KT21 used SCALE-RM (N15; Sato et al. 2015) based on a conservative FDM with high-order advection schemes, this study uses the newly developed LES model using the DGM described in appendix A.
a. Experimental setup
The experimental setup is summarized as follows. The computational domain is a square of 9.6 × 9.6 km2 with a double periodic boundary and an altitude of 3 km. The effective grid spacing, defined as Δxeff = he,i/(p + 1), is fixed to 10 m to compare the computational results in the same spatial resolution. Radiation and moist processes are absent to focus on the turbulent process. In the turbulent model, the filter length is set to 2Δxeff based on the discussions in N15. Consistent with this filter length, KT21 removed the two-gridscale flow structures using an explicit high-order numerical filter or inherent numerical diffusion in the upwind advection schemes. In this study, the corresponding filtering is performed by the inherent numerical diffusion with the upwind numerical flux. The e-folding time for near two-gridscale waves is comparable to that in KT21. The initial atmosphere has stable stratification with a vertical gradient of potential temperature of 4 K km−1 and added random perturbations with an amplitude of 1 K. The initial wind is 5 m s−1 in the x1 direction. A constant heat flux with 200 W m−2 is imposed at the surface.
The four factors (i.e., the polynomial order, the order of temporal schemes, the order and decay coefficient in the modal filter, and the calculation approach for the numerical flux) are changed in the series of numerical experiments. Table 1 summarizes the parameters in each of the experiments. To investigate the effect of polynomial order, we consider cases with p = 1, 3, 4, 5, 7, and 11, which are referred to as P1, P3, P4, P5, P7, and P11 experiments, respectively (see series 1 in Table 1). The P7 experiment is the “control experiment.” The DGM model needs some modal filters to suppress numerical instability. In the series of experiment 1, the effects of the modal filter should be set small enough to extract the effect of the polynomial order. For this purpose, the order and decay time coefficient of the modal filter are fixed as pm = 32 and Δt/αm = Δt/10−3, respectively. Such a large order of the modal filter is sufficiently scale-selective in terms of the polynomial orders, and the decay time is the minimum value for the stable computation in the high-order DGM. Attention to the temporal errors should also be given; to minimize the temporal errors in this experimental series, the 10-stage and fourth-order RK scheme is used for the inviscid terms. The time step is set to Δt = 0.0125 s except in the case of P11. If we assume the typical wind speed to be approximately 5 m s−1, the Courant number against the advective velocity is estimated as cr,DG ∼ 6.25 × 10−3/(p + 1). Note that the fourth-order temporal scheme was sufficient for the eighth-order advection scheme based on FDM (see section 4b in KT21). For the numerical flux, we adopt a modified version of the Rusanov flux for low Mach number flow following Bassi et al. (2017), referred to as the LM Rusanov flux in this study. The coefficient in the stabilization term is given by the component of wind speed perpendicular to the element boundary. For further detail, see Eq. (A12) in appendix A. Based on the simulation results in section 5.3 of Bassi et al. (2017), we consider a similar calculation method appropriate for flows with low Mach number.
Series of numerical experiments of an idealized PBL turbulence.
To investigate the effect of the order of temporal accuracy in DGM, the 10-stage and fourth-order RK scheme used in experimental series 1 is replaced with a three stage and third-order RK scheme (Shu and Osher 1988). Compared to the control experiment, the time step is reduced by a factor of 0.4 to ensure numerical stability. These experiments are referred to as P3(/P4/P7)_RK3 experiments (see series 2 in Table 1).
To examine the behavior of modal filter, we conduct P3, P4, and P7 experiments wherein the order of the modal filter and decay time are changed (see series 3 in Table 1). In the P3(/P4/P7)_pm32_αm×10 experiments, the decay coefficient αm increases by a factor of 10, i.e., αm = 10−2. As for the influence of the order in the modal filter, pm reduces from 32 to 16 and to 8, respectively, in P3(/P4/P7)_pm16_αm×10 and P3(/P4/P7)_pm8_αm×10 experiments. In these experiments, we use αm = 10−2, which is appropriate to identify the change in a different quantity with respect to the order of modal filter. One important conclusion in section 3e is that the upwind numerical flux for p = 1 provides a considerable amount of numerical diffusion even without the modal filter. To confirm this consequence, we additionally conduct the case of p = 1 without the modal filter; this is referred to as P1_MFoff experiment.
Further, we investigate the effect of two different approaches of numerical flux calculation (see series 4 in Table 1). In experiments 1, 2, and 3, we adopt the LM Rusanov flux. To evaluate the effect of hyperupwinding with the original Rusanov flux, we conduct the experiments of P1, P3, P4, and P7 by replacing the coefficient in the stabilization term by the sum of the sound speed and the component of wind speed perpendicular to the element boundary. For further details, see Eq. (A11) in appendix A. These experiments are called P1(/P3/P4/P7)_hyperupwind experiments.
Finally, the workflow of the experiments is explained as follows. For the control experiment, the integration time is 4 h. To reduce the computational cost, the output at 3 h of the control experiment is used as the initial condition of the other experiments whose integration time is 1 h. We analyze the results for the last 30 min.
b. Results from numerical experiment
Before performing the detailed analysis of the numerical simulations, our DGM model is validated by comparing the vertical structures of PBL obtained in KT21, which are detailed in appendix B; we confirm that the DGM model satisfactorily reproduces the PBL turbulent profiles obtained in KT21 except for the case of p = 1. In this subsection, we present the results from experiment series 1–4, which is focused on the energy spectra, and investigate whether the results are consistent with the indications described in sections 3b and 3e.
1) Impact of polynomial order
We describe the impact of polynomial order on the kinetic energy spectra in the interior of the PBL. Figure 11 shows the density-weighted energy spectra of the three-dimensional velocity at a 500 m height for various orders of the polynomial. For experiments with p ≥ 4, the energy spectra obey the −5/3 power law in the range of eight grid lengths or more. This result is consistent with the numerical criteria with numerical diffusion (Rdiff). Thus, p = 4 with a relatively weak modal filter is acceptable. When p ≥ 5, the effect of numerical diffusion decreases further. The energy spectra for P5 follow that of the control experiment (P7) or P11 in the range of five grid lengths or more. Further increasing p may reduce the errors due to the numerical diffusion at short wavelengths. However, as indicated in the result of p = 11, this effect has little impact on the energy spectra because the SGS terms play a dominant role in energy dissipation. Thus, we excluded the case of p = 11 in experimental series 2–4. It is sufficient to investigate the case of p = 7 as a sufficiently high-order polynomial case when we discuss the impact of the temporal scheme, modal filter, and numerical flux on our numerical criteria.
The problem of the modal filter in P1 appears also in the energy spectra represented by the cyan line in Fig. 11. In this case, the modal filter lets the energy spectra significantly dissipate in the range of the short wavelengths; it also affects even large-scale flows. Although we can stably perform the simulation at p = 1 without the modal filter (P1_MFoff), the energy spectra drop from the −5/3 power law at wavelengths larger than the eight grid lengths. This is consistent with Fig. 4a, which indicates that p = 1 does not satisfy our requirement, i.e., Rdiff < 10−1, even without the modal filter. Figure 12 shows the horizontal structures of the turbulence for the various orders of the polynomial. In P1 and P1_MFoff, the small-scale turbulent structure near the updraft convective region is not resolved, compared to that for the higher-order DGM (p ≥ 3).
Figure 13 shows the comparison of the energy spectra in the DGM (Fig. 11) with that in FDM, obtained by KT21. To highlight the difference among these schemes, all spectra are normalized with that of CD8ND8, which is the most reliable result in KT21 using the eighth-order central scheme and eight-order explicit numerical diffusion. The energy spectra for P1 and P1_MFoff experiments have more diffusive features than those in CD2ND2 (the second-order central advection scheme with the second-order explicit numerical diffusion) which is the lowest-order experiment in KT21. When we focus on the range of wavelengths between 5Δxeff and 8Δxeff, the seventh-order upwind advection scheme (UD7) well matches the result from P5. The spectra from the fifth-order upwind advection scheme (UD5) are located between that of P3 and P4. In the range of wavelengths shorter than 5Δxeff, the energy spectra in P3 ∼ P5 drop significantly compared with that in UD5 and UD7. This behavior indicates that the effect of numerical dissipation, caused by the upwind numerical flux and modal filter, becomes stronger toward the grid scale compared to that for the corresponding upwind FDM.
Figure 14 shows Rdiff and Rdisp diagrams for the experimental series 1. These indices for DGM are pictured when the waves advance about 6Δxeff, as performed in section 3. These diagrams for Rdiff reasonably explain the influence of numerical diffusion in the upwind DGM and FDM on the energy spectra. Indeed, the effect of numerical diffusion for P1 on the energy spectra is the most significant among all the schemes. The Rdiff diagrams for P3 and P4 match well for UD5 and UD7, respectively. As for P7 and P11, the wavelength at Rdiff = 10−1 is well below eight grid lengths, which is similar to CD8ND8. Figure 14 tends to underestimate the impact of numerical dissipation in the DGM near the grid scales because the damping effect near the grid scales is characterized by Rdiff at a relatively early stage, as discussed in section 3e. In addition, for p = 1, the increase in Rdiff with time appears to be significant at long wavelength. This would be the reason why the energy spectra of P1_MFoff is more diffusive than that of UD3 although the two Rdiff diagrams (Figs. 14b,e) are close.
2) Impact of temporal schemes
We investigate the effect of temporal discretization by replacing the fourth-order RK scheme with the third-order RK scheme for p = 3, 4, and 7 (see the experimental series 2 in Table 1). Figure 15 shows the energy spectra of P3_RK3, P4_RK3, and P7_RK3 as well as P3, P4, and P7, respectively. The change in the temporal scheme has no impact on the spectra. This fact indicates that the low-order RK scheme is also acceptable for high-order DGM when fully explicit schemes are adopted for low Mach number flows (i.e., for a very small Courant number against advection). For a significant high-order DGM such as p = 7, Fig. 5 shows that the superconvergence property at l > 10 is degraded by decreasing the temporal accuracy. However, the additional numerical dissipation caused by the temporal discretization is still small and ineffective against the results. Thus, there is little change in the energy spectra even if we use the RK3 scheme, as indicated in KT21.
The temporal splitting or implicit strategy is often employed by separating the fast wave terms and advection terms. In this case, the corresponding Courant number against advection approaches unity. In this scenario, the accuracy of the temporal scheme becomes more important; this possibility is suggested in Fig. 6.
3) Impact of modal filter
We examine the sensitivity of energy spectra to the modal filter (see the experimental series 3 in Table 1). Figures 16 and 17 show the energy spectra when we change the decay coefficient (αm) and order (pm) of the modal filter for p = 3, 4, and 7.
Section 3d implies that if αm increases by a factor of 10 when pm = 32, p = 3 will no longer satisfy our criterion of Rdiff < 10−1 in the range of 8Δxeff or more. Our results here, as indicated by the solid and dashed blue lines in Fig. 16, are consistent with the results in section 3d. The spectra for P3_pm32_αm×10 deviates from that of the control experiment (P7) at about 10 grid lengths. If p is significantly large, increasing αm by a factor of 10 has little impact on the flow structure of wavelengths longer than eight grid lengths. This is because the modal filter with a large pm retains the majority of modes.
Figure 8 indicates that the impact of the order of modal filter is as follows: The change in Rdiff is small if pm decreases from 32 to 16 when αm = 10−2. However, for higher-order DGM, the decrease in the order of the modal filter degrades the superconvergence property at long wavelengths. In the actual simulation, the difference between P3_pm32_αm×10 and P3_pm16_αm×10 is significantly small when comparing the solid blue line with the dashed purple one in Fig. 17b; the two lines are almost identical. For P7_pm16_αm×10 indicated by the yellow dashed line, the change with the lower-order modal filter can be observed at wavelengths shorter than five grid lengths. However, the effect on superconvergence is not evident in the energy spectra. We consider that even if the superconvergence is affected, the numerical error itself is sufficiently small at eight grid lengths or more (as shown in the lower panel of Fig. 8). Thus, there is no significant change in the energy spectra for this wavelength range.
The energy spectra for p = 3 indicates further diffusive field when the order of modal filter pm decreases by 8. However, it is less diffusive at wavelengths shorter than 20 grid length, compared to that for P1_MFoff. The energy spectra for p = 7 with pm = 8 resembles that for P4_pm16_αm×10 at wavelengths longer than 5 grid lengths. This is consistent with the fact that the wavelength of Rdiff = 10−1 is about 10 grid lengths for the two cases (see the case of p = 4 with pm = 16, αm = 10−2 and p = 7 with pm = 8, αm = 10−2 in Fig. 8).
Although the choice of αm and pm is artificial and depends on the experimental setup, we believe that our analysis described in section 3d provides a proper guideline with the modal filter in the turbulent simulations.
4) Effect of hyperupwinding with Rusanov flux
Figure 18 shows the hyperupwind effect with original Rusanov flux on the energy spectra. As indicated in section 3e, the difference between the original Rusanov and LM Rusanov numerical fluxes has little impact on the energy spectra for high-order DGM such as p ≥ 3. We consider that the results of the small difference are also related to the use of filter length with the two grid lengths (2Δxeff) in the turbulence model and the modal filter. This is because these factors contribute to dissipating small-scale flow structures, and this can dominate the effect of numerical fluxes in high-order DGM. For high-order DGM, Mengaldo et al. (2018) showed the impact of numerical flux on the under-resolved vortical flows, which mimics grid turbulence at infinite Reynolds number; in their experiment using the Rusanov flux and Roe flux, the former numerical flux produced the spurious reflection and small-scale noises in the under-resolved and sharp mesh coarsening regions for the low Mach number case, whereas the latter numerical flux suppressed such numerical behavior. We consider that the sensitivity of the numerical flux relates to their experimental setup where physical diffusion was not included.
For p = 1, the two numerical fluxes result in remarkably different energy spectra; the result for the original Rusanov flux indicates a more diffusive field across the spectrum of wavelengths. As discussed in section 3e, the hyperupwinding enhances the damping effect at short wavelengths for the early stage. For p = 1, this damping enhancement for the early stage extends to a relatively long wavelength; the line of Rdiff = 101, which indicates the numerical dissipation completely dominates over the eddy viscosity, reaches the eight grid lengths in the O(10) m resolution (see the top panel of Fig. 9). We believe this results in a more diffusive spectrum for P1_hyperupwind.
5. Discussion
a. Choice of numerical flux
When using a high-order DGM for future global LES, an interesting question is: between the original Rusanov flux and LM Rusanov flux, which numerical flux should be preferred? In practical meteorological simulations, we need to treat a considerably wider dynamical range of wind velocities compared to the idealized numerical experiment considered in this study. For example, if the horizontal scale of the computational domain is wider than O(1000) km, atmospheric phenomena with strong winds, such as a typhoon or jet, can be included. In such a case, it is naturally considered that the original Rusanov flux would be appropriate because it not only satisfies our numerical criteria for the LES of PBL turbulence (if p ≥ 4), but also provides more reliable numerical stability for scenarios accompanied by high-speed wind. As the wind speed increases, the dynamic range of the propagating speed of the sound wave (i.e., the sum of the wind and sound speeds) is extended. Thus, it would be preferable to adopt a more robust numerical flux if our numerical criteria are satisfied.
We note that more sophisticated numerical fluxes exist. These schemes provide small numerical dissipation. When the numerical viscosity is designed to resolve sharp gradients of the flow structures, the Roe-type scheme (Roe 1981) considers all eigenvalues of the inviscid flux Jacobian or the family of advection upstream splitting method (AUSM) (e.g., Liou and Steffen 1993) divides the inviscid flux into the convective and pressure components. Furthermore, for these numerical fluxes, previous studies have proposed modified methods for the low Mach number flows, the low Mach number fix for Roe’s approximate Riemann solver (e.g., Rieper 2011) and the AUSM for all speed flows (e.g., Liou 2006). However, such numerical fluxes are generally constructed by a complex nonlinear function. More sophisticated flux operators may need to be revisited when we adopt the temporally implicit scheme or consider the consistency of mass fluxes in tracer advection calculation. Investigating the effect of sophisticated numerical fluxes is an interesting subject; however, it is beyond the scope of this study.
b. Artificial filter for numerical stability
In the LES model based on DGM described in appendix A, we use a computationally efficient strategy with low memory cost by assuming the use of linear mapping for the local coordinate and via collocation with the integration of nonlinear terms. We note that the accuracy of the numerical quadrature with the LGL nodes is not sufficient to precisely integrate the nonlinear terms. Such a situation produces aliasing errors, and numerical instability can occur if the stabilization mechanism is not sufficient. Fortunately, in the numerical simulations performed in this study, dissipation effects with the upwind numerical flux, eddy diffusion, and weak modal filter (i.e., αm = 10−3 and pm = 32) were sufficient to prevent the numerical instability.
Because the strength of modal filter required to ensure the numerical stability is low, the polynomial order satisfying our numerical criteria is almost the same as that in the case where no modal filter is used. However, if we need to enhance the modal filter to control the aliasing-driven instability, the effect of modal filter is no longer a secondary factor of our numerical criteria. Alternatively, we can avoid or reduce the use of modal filter by using sophisticated and robust strategies that can support the numerical theory, such as the split-form nodal DGM with a summation-by-parts property (e.g., Gassner et al. 2016; Gassner and Winters 2021).
c. Realizing atmospheric LES using high-order DGM
The results in section 4b suggest that there is little advantage in further increasing p ≥ 5. This is partly because the SGS turbulent model used in this study plays a too dominant role in energy dissipation in the range shorter than the eight grid lengths. At such short wavelengths, the energy spectra drop with a slope steeper than that of the −5/3 power law even for significantly high-order DGM. Another reason can be attributed to the definition of the filter length for the SGS scheme, which is double the effective grid spacing (2Δxeff); this leads to a slight ambiguity regarding physical dissipation. However, high-order DGM itself, such as p ≥ 5, can still increase the effective resolution to about four grid lengths, which can reduce the numerical diffusion and dispersion errors. It is instinctive to exploit such a high accuracy of the DGM to decrease the difference of the −5/3 power law at short wavelengths. To achieve this, the turbulent model used in the DGM framework needs to be carefully revisited. One of the essential keys here is to clearly define the cutoff wavelength and adopt a “cutoff-like” spatial filter.
As can be inferred from the introduction of the effective wavelength Δxeff, the cutoff wavelength is slightly vague in the DGM. The modal filter is useful to define the desirable cutoff wavelength because it qualitatively works to remove the components at short wavelengths. However, the modal filters related to local polynomial expansions do not precisely provide the cutoff-like behavior in the wavenumber space. The design of cutoff-like spatial filter that clearly defines the cutoff wavenumber is a focus for future work.
6. Summary
We investigated the behavior of numerical dissipation and numerical dispersion associated with upwind DGM to consider the suitability of the discontinuous Galerkin method (DGM) for atmospheric LES. We extended the numerical criteria derived in KT21 to the DGM framework. The difficulty with obtaining the modified equation for high-order DGM was overcome using the temporal eigensolution analysis (e.g., Moura et al. 2015b; Alhawwary and Wang 2018).
We focused on the polynomial order (p) that satisfies our numerical criteria. Similar to KT21, we required numerical dissipation and numerical dispersion with advection terms that would not dominate the diffusion and dispersion, respectively, with eddy viscosity terms at wavelengths longer than eight grid lengths. In the targeted grid spacing of O(10) m, at least p = 4 was required to satisfy our numerical criteria in the case of upwind numerical flux. Compared to the conventional FDM with the same order, the superconvergence properties of the DGM resulted in significantly small numerical errors at well-resolved wavelengths. However, even for a significantly high-order DGM such as p = 11, the numerical errors at wavelengths shorter than five grid lengths dominate the effect of the eddy viscosity terms.
Furthermore, we discussed issues to bridge the gap between the simplified theoretical consideration in the semidiscrete analysis and the actual scenarios involved in atmospheric LES: the impact of temporal discretization and modal filter, and the hyperupwinding problem for low Mach number flows. For the temporal accuracy, the third-order RK scheme is acceptable even for high-order DGM such as p = 7 and the fourth-order scheme is sufficient if the time step in fully explicit temporal schemes is restricted by fast waves. A modal filter is a useful strategy for enhancing numerical stability in actual fluid simulations with DGM; however, the decay coefficient and the order should be considered carefully. To maintain the numerical criteria for the case without the modal filter, we should adopt a highly scale-selective 16th- or 32nd-order modal filter with the decay time of O(10−3Δt) where Δt is a time step in fully explicit temporal scheme. When using a stronger modal filter, we need to increase p to satisfy our numerical criteria. Considering numerical flux, the Rusanov flux is often adopted in atmospheric simulations owing to its simplicity. However, it is not naturally appropriate for low Mach number flows as this situation would lead to hyperupwinding. We investigated the impact of hyperupwinding on our numerical criteria by increasing the decay coefficient β. We found that the difference between hyperupwinding and standard upwinding decreased with increasing values of p. Thus, the choice of the numerical flux did not affect our numerical criteria for the high-order DGM.
To evaluate the validity of the numerical criteria through idealized numerical experiments, we constructed a new LES model based on a nodal DGM with upwind numerical flux and conducted a series of idealized PBL turbulence experiments with the same setting as that used in KT21.
We analyzed the energy spectra for various p to investigate its effect on small-scale turbulence. In the p ≥ 4 experiments using a significantly high-order modal filter with small decay coefficient (the 32nd-order modal filter with decay time scale 103Δt), the energy spectra followed the −5/3 power law at the wavelength range longer than the eight grid lengths. This result is consistent with the theoretical estimate; i.e., the criterion for numerical diffusion is satisfied for p = 4 with a weak, high-order modal filter. When increasing p to 5, the spectrum follows that of the control experiment (p = 7) at a wavelength range longer than five grid lengths. As suggested theoretically, the modal filter seriously contaminated the turbulent statistics for p = 1. Even for p = 1 without the modal filter, the energy spectrum deviates from the −5/3 power law at wavelengths much longer than eight grid lengths.
It is worth considering how these DGM results relate to the FDM results in KT21. Focusing on the range of wavelengths between 2Δxeff and 8Δxeff, we compared the energy spectra between the two methods. At wavelengths between 5Δxeff and 8Δxeff, we found that the energy spectrum from the seventh-order upwind scheme (UD7) corresponded well with that of p = 5 and the spectrum from the fifth-order upwind scheme (UD5) was located between those of p = 3 and p = 4. At shorter wavelengths, p = 3, 4, 5 yielded more diffusive energy spectra, compared to those for UD5 and UD7. This is because of the numerical dissipation caused by upwind numerical flux and modal filter. This relationship is reasonably interpreted using Rdiff diagrams for the corresponding schemes. We note that Rdiff diagrams depends on time, in particular, for p = 1.
Based on this aforementioned knowledge for the polynomial order, we examined the effect of temporal scheme, the modal filter, and the numerical flux. For the temporal scheme, temporal accuracy has minor influence on energy dissipation when the full explicit temporal scheme is used. The numerical results indicated that the third-order scheme showed acceptable errors even for high-order DGM such as p = 7. For the modal filter, it is difficult to determine appropriate values of the order and decay coefficient a priori. We evaluated whether the Rdiff diagram that includes the effect of the modal filter can be help us define a range of acceptable parameter values. For example, the Rdiff diagram suggests that p = 3 is unacceptable when the decay coefficient increases by a factor of 10 (the decay time scale of 102Δt). This theoretical suggestion is consistent with the energy spectra obtained in the actual LES experiment. We believe that the Rdiff diagram including the effect of the modal filter provides a useful guideline while setting these artificial parameters. Finally, for the numerical flux, we confirmed that when p is sufficiently large, the choice of the numerical flux is not an essential factor. Only for p = 1, the hyperupwinding with the original Rusanov flux resulted in a more diffusive spectrum than that obtained in the case of the modified numerical flux for low Mach number flows in our LES experiment. These numerical results are consistent with the expectation of Rdiff diagram.
Our LES experiments indicated that further increasing p ≥ 5 has little advantage. We consider that this is related to the SGS turbulent model used in this study and the filter length defined as double the effective grid spacing. The design of spatial filter is a focus for future work because the modal filter related to local polynomial is difficult to clearly define the cutoff wavenumber. In addition, it is important to further decrease the numerical dissipation due to upwind numerical flux and aliasing errors with nonlinear terms to enhance the effective resolution. To overcome the latter issue in computationally efficient ways, applying the split form nodal DGM (e.g., Gassner et al. 2016) to the atmospheric LES may be a promising approach. This is another research direction for future work.
Acknowledgments.
This research was supported by the JST AIP Grant JPMJCR19U2, Japan; MEXT KAKENHI Grant JP20H05731; Moonshot R&D Grant JPMJMS2286; and the Foundation for Computational Science (FOCUS) Establishing Supercomputing Center of Excellence, Japan. The numerical analysis in this study was performed using the Oackbridge-CX supercomputer at the University of Tokyo and the supercomputer Fugaku at RIKEN (Project ID: ra000005 and hp200271). The authors are grateful to Team SCALE for providing the SCALE version 5.3.6 used for the comparison with the simulation results in this study. We thank Dr. Seiya Nishizawa, Dr. Hiroaki Miura, and anonymous reviewers for their valuable comments and suggestions. We thank Editage (www.editage.jp) for English language editing.
Data availability statement.
Source codes used in this study are available from the Zenodo repository (https://doi.org/10.5281/zenodo.7519839), where we have provided supplemental material for the derivation of Eqs. (12) and (13) and enlarged views of Figs 3, 9, and 10 in this paper. All data obtained from the numerical experiments has been deposited in the local storage at RIKEN R-CCS.
APPENDIX A
Description of Atmospheric LES Model Using DGM
a. Governing equations
The governing equations of LES are the same as that in N15 and KT21. In a dynamical process, the three-dimensional fully compressible equations of a dry atmosphere are solved in the three-dimensional Cartesian coordinates [denoted by x = (x1, x2, x3)]. The turbulent process is represented by the Smagorinsky–Lilly model (Smagorinsky 1963; Lilly 1962) that considers the stratification effect (Brown et al. 1994).
To reduce the inaccurate numerical representation when the atmospheric state is largely in hydrostatic balance, the density and pressure (thus ρθ) are decomposed as ϕ(x, t) = ϕhyd(x) + ϕ′(x, t) where ϕhyd denotes a variable satisfying the hydrostatic balance. Using its perturbation variables, the vertical inviscid flux [Eq. (A2)] and source term [Eq. (A3)] for the vertical momentum equation are replaced with
b. Spatial discretization
The governing equations in Eq. (A1) are spatially discretized by the nodal DGM with a strong form (e.g., Hesthaven and Warburton 2007). Equation (A1) is treated as a system of first-order differential equations based on Bassi and Rebay (1997) to evaluate the eddy diffusion terms with the second-order differential operator.
c. Temporal discretization and modal filtering
The resulting semidiscretized equation in Eq. (A13) is an ordinary differential equation with respect to time. As a temporal discretization, the full explicit Runge–Kutta (RK) schemes with strong stability preserving (SSP) property (e.g., Gottlieb et al. 2001) is adopted. We basically use a 10-stage and fourth-order RK scheme proposed by Ketcheson (2008). Besides the large stability region and SSP property, the 10-stage and fourth-order RK scheme has the advantage of saving memory storage, compared to that for the classical four stage and fourth-order RK scheme.
APPENDIX B
A Validation of the DG Model
We validate the model before performing the detailed analysis of the numerical simulations because the dynamical core of the model is newly developed. Figure B1 shows the vertical structures of the PBL averaged in the final 30 min. The shaded area in Fig. B1 represents the range of our previous results in KT21. For all experiments except for p = 1, the results from the DGM model well reproduce the PBL turbulent profiles obtained in KT21; their lines are mostly in the shaded range. Thus, the major aspects of the vertical profiles in PBL do not much depend on the polynomial order at p ≥ 3. As shown in Fig. B1a, the height of PBL reaches 1200 m, and the sharp vertical gradient of the potential temperature in the surface boundary layer can be well resolved. Figure B1b shows the sum of the resolved and SGS eddy heat flux. It decreases linearly with the height from the surface to the top of the PBL. The variance and skewness of the vertical wind are shown in Figs. B1c and B1d. The maximum of variance around the midheight of PBL does not monotonously increase with the polynomial order; the maximum variance for P11 is slightly smaller than that for P4, P5, or P7. We speculate that this behavior is related to the energy pile in more than the 10-grid wavelength range; this is also discussed in relation to the numerical energy dissipation in shorter wavelengths. The positive skewness in the interior of PBL shown in Fig. B1d indicates that the upward wind is more intense than the downward wind. Figure B2 shows the horizontal structure of the vertical wind at a 500-m height for the control experiment. It has polygonal structures, i.e., mainly hexagons, with a horizontal scale of about 2–3 km. Thus, the DGM model used in this study yields results comparable to our previous study for the basic aspects of the PBL.
However, the solution for P1 is seriously contaminated because the modal filter effectively removes the highest mode, as seen in Fig. B1. Its turbulence statistics significantly deviate from the results obtained from the other experiments. Thus, the use of a modal filter for p = 1 is not recommended as indicated in Fig. 7.
REFERENCES
Ainsworth, M., 2004: Dispersive and dissipative behaviour of high order discontinuous Galerkin finite element methods. J. Comput. Phys., 198, 106–130, https://doi.org/10.1016/j.jcp.2004.01.004.
Alhawwary, M., and Z. J. Wang, 2018: Fourier analysis and evaluation of DG, FD and compact difference methods for conservation laws. J. Comput. Phys., 373, 835–862, https://doi.org/10.1016/j.jcp.2018.07.018.
Asthana, K., M. R. López-Morales, and A. Jameson, 2015: Non-linear stabilization of high-order flux reconstruction schemes via Fourier-spectral filtering. J. Comput. Phys., 303, 269–294, https://doi.org/10.1016/j.jcp.2015.09.041.
Bassi, C., A. Abbà, L. Bonaventura, and L. Valdettaro, 2017: Large eddy simulation of gravity currents with a high order DG method. Commun. Appl. Ind. Math., 8, 128–148, https://doi.org/10.1515/caim-2017-0007.
Bassi, F., and S. Rebay, 1997: A high-order accurate discontinuous finite element method for the numerical solution of the compressible Navier–Stokes equations. J. Comput. Phys., 131, 267–279, https://doi.org/10.1006/jcph.1996.5572.
Brandt, T., 2006: A priori tests on numerical errors in large eddy simulation using finite differences and explicit filtering. Int. J. Numer. Methods Fluids, 51, 635–657, https://doi.org/10.1002/fld.1144.
Brown, A. R., S. H. Derbyshire, and P. J. Mason, 1994: Large-eddy simulation of stable atmospheric boundary layers with a revised stochastic subgrid model. Quart. J. Roy. Meteor. Soc., 120, 1485–1512, https://doi.org/10.1002/qj.49712052004.
Choi, H.-W., R. Nair, and H. Tufo, 2006: A scalable high-order discontinuous Galerkin method for global atmospheric modeling. Parallel Computational Fluid Dynamics 2006, 8 pp., https://www.image.ucar.edu/staff/rnair/parcfd06.pdf.
De Grazia, D., G. Mengaldo, D. Moxey, P. E. Vincent, and S. J. Sherwin, 2014: Connections between the discontinuous Galerkin method and high-order flux reconstruction schemes. Int. J. Numer. Methods Fluids, 75, 860–877, https://doi.org/10.1002/fld.3915.
Durran, D. R., 2010: Numerical Methods for Fluid Dynamics with Applications to Geophysics. Vol. 32, Springer, 516 pp.
Favre, A., 1983: Turbulence: Space-time statistical properties and behavior in supersonic flows. Phys. Fluids, 26, 2851–2863, https://doi.org/10.1063/1.864049.
Fernandez, P., R. C. Moura, G. Mengaldo, and J. Peraire, 2019: Non-modal analysis of spectral element methods: Towards accurate and robust large-eddy simulations. Comput. Methods Appl. Mech. Eng., 346, 43–62, https://doi.org/10.1016/j.cma.2018.11.027.
Gassner, G. J., and A. D. Beck, 2013: On the accuracy of high-order discretizations for underresolved turbulence simulations. Theor. Comput. Fluid Dyn., 27, 221–237, https://doi.org/10.1007/s00162-011-0253-7.
Gassner, G. J., and A. R. Winters, 2021: A novel robust strategy for discontinuous Galerkin methods in computational fluid mechanics: Why? When? What? Where? Front. Phys., 8, 500690, https://doi.org/10.3389/fphy.2020.500690.
Gassner, G. J., A. R. Winters, and D. A. Kopriva, 2016: Split form nodal discontinuous Galerkin schemes with summation-by-parts property for the compressible Euler equations. J. Comput. Phys., 327, 39–66, https://doi.org/10.1016/j.jcp.2016.09.013.
Ghosal, S., 1996: An analysis of numerical errors in large-eddy simulations of turbulence. J. Comput. Phys., 125, 187–206, https://doi.org/10.1006/jcph.1996.0088.
Giraldo, F. X., and M. Restelli, 2008: A study of spectral element and discontinuous Galerkin methods for the Navier–Stokes equations in nonhydrostatic mesoscale atmospheric modeling: Equation sets and test cases. J. Comput. Phys., 227, 3849–3877, https://doi.org/10.1016/j.jcp.2007.12.009.
Giraldo, F. X., J. S. Hesthaven, and T. Warburton, 2002: Nodal high-order discontinuous Galerkin methods for the spherical shallow water equations. J. Comput. Phys., 181, 499–525, https://doi.org/10.1006/jcph.2002.7139.
Gottlieb, S., C.-W. Shu, and E. Tadmor, 2001: Strong stability-preserving high-order time discretization methods. SIAM Rev., 43, 89–112, https://doi.org/10.1137/S003614450036757X.
Harris, R., and Z. J. Wang, 2009: Partition design and optimization for high order spectral volume schemes. 47th AIAA Aerospace Sciences Meeting Including The New Horizons Forum and Aerospace Exposition, Orlando, FL, Aerospace Research Central, AIAA 2009-1333, https://doi.org/10.2514/6.2009-1333.
Hesthaven, J. S., and T. Warburton, 2007: Nodal Discontinuous Galerkin Methods: Algorithms, Analysis, and Applications. Springer Science & Business Media, 502 pp.
Hesthaven, J. S., and R. M. Kirby, 2008: Filtering in Legendre spectral methods. Math. Comput., 77, 1425–1452, https://doi.org/10.1090/S0025-5718-08-02110-8.
Hindenlang, F. J., G. J. Gassner, and D. A. Kopriva, 2020: Stability of wall boundary condition procedures for discontinuous Galerkin spectral element approximations of the compressible Euler equations. Spectral and High Order Methods for Partial Differential Equations ICOSAHOM 2018, S. J. Sherwin et al., Eds., Springer International Publishing, 3–19.
Hu, F. Q., and H. L. Atkins, 2002: Eigensolution analysis of the discontinuous Galerkin method with nonuniform grids: I. One space dimension. J. Comput. Phys., 182, 516–545, https://doi.org/10.1006/jcph.2002.7184.
Huynh, H. T., 2007: A flux reconstruction approach to high-order schemes including discontinuous Galerkin methods. 18th AIAA Computational Fluid Dynamics Conf., Miami, FL, Aerospace Research Central, AIAA 2007-4079, https://doi.org/10.2514/6.2007-4079.
Kawai, Y., and H. Tomita, 2021: Numerical accuracy of advection scheme necessary for large-eddy simulation of planetary boundary layer turbulence. Mon. Wea. Rev., 149, 2993–3012, https://doi.org/10.1175/MWR-D-20-0362.1.
Kelly, J. F., and F. X. Giraldo, 2012: Continuous and discontinuous Galerkin methods for a scalable three-dimensional nonhydrostatic atmospheric model: Limited-area mode. J. Comput. Phys., 231, 7988–8008, https://doi.org/10.1016/j.jcp.2012.04.042.
Ketcheson, D. I., 2008: Highly efficient strong stability-preserving Runge–Kutta methods with low-storage implementations. SIAM J. Sci. Comput., 30, 2113–2136, https://doi.org/10.1137/07070485X.
Lilly, D. K., 1962: On the numerical simulation of buoyant convection. Tellus, 14, 148–172, https://doi.org/10.3402/tellusa.v14i2.9537.
Liou, M.-S., 2006: A sequel to AUSM, Part II: AUSM+-up for all speeds. J. Comput. Phys., 214, 137–170, https://doi.org/10.1016/j.jcp.2005.09.020.
Liou, M.-S., and C. J. Steffen, 1993: A new flux splitting scheme. J. Comput. Phys., 107, 23–39, https://doi.org/10.1006/jcph.1993.1122.
Liu, Y., M. Vinokur, and Z. Wang, 2006: Spectral difference method for unstructured grids I: Basic formulation. J. Comput. Phys., 216, 780–801, https://doi.org/10.1016/j.jcp.2006.01.024.
Marchandise, E., N. Chevaugeon, and J.-F. Remacle, 2008: Spatial and spectral superconvergence of discontinuous Galerkin method for hyperbolic problems. J. Comput. Appl. Math., 215, 484–494, https://doi.org/10.1016/j.cam.2006.03.061.
Marras, S., and Coauthors, 2016: A review of element-based Galerkin methods for numerical weather prediction: Finite elements, spectral elements, and discontinuous Galerkin. Arch. Comput. Methods Eng., 23, 673–722, https://doi.org/10.1007/s11831-015-9152-1.
Mengaldo, G., D. De Grazia, F. Witherden, A. Farrington, P. Vincent, S. Sherwin, and J. Peiro, 2014: A guide to the implementation of boundary conditions in compact high-order methods for compressible aerodynamics. Seventh AIAA Theoretical Fluid Mechanics Conf., Atlanta, GA, Aerospace Research Central, AIAA 2014-2923, https://doi.org/10.2514/6.2014-2923.
Mengaldo, G., D. De Grazia, P. E. Vincent, and S. J. Sherwin, 2016: On the connections between discontinuous Galerkin and flux reconstruction schemes: Extension to curvilinear meshes. J. Sci. Comput., 67, 1272–1292, https://doi.org/10.1007/s10915-015-0119-z.
Mengaldo, G., D. De Grazia, R. C. Moura, and S. J. Sherwin, 2018: Spatial eigensolution analysis of energy-stable flux reconstruction schemes and influence of the numerical flux on accuracy and robustness. J. Comput. Phys., 358, 1–20, https://doi.org/10.1016/j.jcp.2017.12.019.
Mengaldo, G., D. Moxey, M. Turner, R. C. Moura, A. Jassim, M. Taylor, J. Peiró, and S. Sherwin, 2021: Industry-relevant implicit large-eddy simulation of a high-performance road car via spectral/hp element methods. SIAM Rev., 63, 723–755, https://doi.org/10.1137/20M1345359.
Moura, R. C., S. J. Sherwin, and J. Peiró, 2015a: Modified equation analysis for the discontinuous Galerkin formulation. Spectral and High Order Methods for Partial Differential Equations ICOSAHOM 2014, T. J. Barth et al., Eds., Lecture Notes in Computational Science and Engineering, Vol. 106, Springer, 375–383.
Moura, R. C., S. J. Sherwin, and J. Peiró, 2015b: Linear dispersion–diffusion analysis and its application to under-resolved turbulence simulations using discontinuous Galerkin spectral/hp methods. J. Comput. Phys., 298, 695–710, https://doi.org/10.1016/j.jcp.2015.06.020.
Moura, R. C., G. Mengaldo, J. Peiró, and S. J. Sherwin, 2017: On the eddy-resolving capability of high-order discontinuous Galerkin approaches to implicit LES/under-resolved DNS of Euler turbulence. J. Comput. Phys., 330, 615–623, https://doi.org/10.1016/j.jcp.2016.10.056.
Moura, R. C., L. D. Fernandes, A. F. C. Silva, G. Mengaldo, and S. J. Sherwin, 2022: Spectral/hp element methods’ linear mechanism of (apparent) energy transfer in Fourier space: Insights into dispersion analysis for implicit LES. J. Comput. Phys., 471, 111613, https://doi.org/10.1016/j.jcp.2022.111613.
Nair, R. D., and H. M. Tufo, 2007: Petascale atmospheric general circulation models. J. Phys., 78, 012078, https://doi.org/10.1088/1742-6596/78/1/012078.
Nair, R. D., S. J. Thomas, and R. D. Loft, 2005: A discontinuous Galerkin transport scheme on the cubed sphere. Mon. Wea. Rev., 133, 814–828, https://doi.org/10.1175/MWR2890.1.
Nair, R. D., H.-W. Choi, and H. M. Tufo, 2009: Computational aspects of a scalable high-order discontinuous Galerkin atmospheric dynamical core. Comput. Fluids, 38, 309–319, https://doi.org/10.1016/j.compfluid.2008.04.006.
Nishizawa, S., H. Yashiro, Y. Sato, Y. Miyamoto, and H. Tomita, 2015: Influence of grid aspect ratio on planetary boundary layer turbulence in large-eddy simulations. Geosci. Model Dev., 8, 3393–3419, https://doi.org/10.5194/gmd-8-3393-2015.
Nitsche, J., 1971: Über ein variationsprinzip zur lösung von dirichlet-problemen bei verwendung von teilräumen, die keinen randbedingungen unterworfen sind. Abh. Math. Semin. Univ. Hamburg, 36, 9–15, https://doi.org/10.1007/BF02995904.
Nordström, J., and A. R. Winters, 2021: Stable filtering procedures for nodal discontinuous Galerkin methods. J. Sci. Comput., 87, 17, https://doi.org/10.1007/s10915-021-01434-x.
Reed, W. H., and T. R. Hill, 1973: Triangular mesh methods for the neutron transport equation. Los Alamos Scientific Laboratory Tech. Rep. LA-UR-73-479, 23 pp., https://www.osti.gov/biblio/4491151.
Restelli, M., and F. X. Giraldo, 2009: A conservative discontinuous Galerkin semi-implicit formulation for the Navier–Stokes equations in nonhydrostatic mesoscale modeling. SIAM J. Sci. Comput., 31, 2231–2257, https://doi.org/10.1137/070708470.
Rieper, F., 2011: A low-mach number fix for Roe’s approximate Riemann solver. J. Comput. Phys., 230, 5263–5287, https://doi.org/10.1016/j.jcp.2011.03.025.
Roe, P. L., 1981: Approximate Riemann solvers, parameter vectors, and difference schemes. J. Comput. Phys., 43, 357–372, https://doi.org/10.1016/0021-9991(81)90128-5.
Rogallo, R. S., and P. Moin, 1984: Numerical simulation of turbulent flows. Annu. Rev. Fluid Mech., 16, 99–137, https://doi.org/10.1146/annurev.fl.16.010184.000531.
Rusanov, V. V., 1961: Calculation of interaction of non-steady shock waves with obstacles. J. Comput. Math. Phys. USSR, 1, 267–279.
Sato, Y., S. Nishizawa, H. Yashiro, Y. Miyamoto, Y. Kajikawa, and H. Tomita, 2015: Impacts of cloud microphysics on trade wind cumulus: Which cloud microphysics processes contribute to the diversity in a large eddy simulation? Prog. Earth Planet. Sci., 2, 23, https://doi.org/10.1186/s40645-015-0053-6.
Satoh, M., B. Stevens, F. Judt, M. Khairoutdinov, S.-J. Lin, W. M. Putman, and P. Düben, 2019: Global cloud-resolving models. Curr. Climate Change Rep., 5, 172–184, https://doi.org/10.1007/s40641-019-00131-0.
Shu, C.-W., and S. Osher, 1988: Efficient implementation of essentially non-oscillatory shock-capturing schemes. J. Comput. Phys., 77, 439–471, https://doi.org/10.1016/0021-9991(88)90177-5.
Skamarock, W. C., J. B. Klemp, M. G. Duda, L. D. Fowler, S.-H. Park, and T. D. Ringler, 2012: A multiscale nonhydrostatic atmospheric model using centroidal Voronoi tesselations and C-grid staggering. Mon. Wea. Rev., 140, 3090–3105, https://doi.org/10.1175/MWR-D-11-00215.1.
Smagorinsky, J., 1963: General circulation experiments with the primitive equations: I. The basic experiment. Mon. Wea. Rev., 91, 99–164, https://doi.org/10.1175/1520-0493(1963)091<0099:GCEWTP>2.3.CO;2.
Tomita, H., and M. Satoh, 2004: A new dynamical framework of nonhydrostatic global model using the icosahedral grid. Fluid Dyn. Res., 34, 357, https://doi.org/10.1016/j.fluiddyn.2004.03.003.
Tonicello, N., R. C. Moura, G. Lodato, and G. Mengaldo, 2021: Fully-discrete spatial eigenanalysis of discontinuous spectral element methods: Insights into well-resolved and under-resolved vortical flows. arXiv, 2111.13891v1, https://doi.org/10.48550/arXiv.2111.13891.
Ullrich, P. A., and C. Jablonowski, 2012: Mcore: A non-hydrostatic atmospheric dynamical core utilizing high-order finite-volume methods. J. Comput. Phys., 231, 5078–5108, https://doi.org/10.1016/j.jcp.2012.04.024.
Ullrich, P. A., C. Jablonowski, and B. Van Leer, 2010: High-order finite-volume methods for the shallow-water equations on the sphere. J. Comput. Phys., 229, 6104–6134, https://doi.org/10.1016/j.jcp.2010.04.044.
Vreman, B., B. Geurts, and H. Kuerten, 1994: Discretization error dominance over subgrid terms in large eddy simulation of compressible shear layers in 2D. Commun. Numer. Methods Eng., 10, 785–790, https://doi.org/10.1002/cnm.1640101004.
Wang, Z., 2002: Spectral (finite) volume method for conservation laws on unstructured grids: Basic formulation. J. Comput. Phys., 178, 210–251, https://doi.org/10.1006/jcph.2002.7041.
Winters, A. R., R. C. Moura, G. Mengaldo, G. J. Gassner, S. Walch, J. Peiro, and S. J. Sherwin, 2018: A comparative study on polynomial dealiasing and split form discontinuous Galerkin schemes for under-resolved turbulence computations. J. Comput. Phys., 372, 1–21, https://doi.org/10.1016/j.jcp.2018.06.016.
Zängl, G., D. Reinert, P. Rípodas, and M. Baldauf, 2015: The ICON (ICOsahedral non-hydrostatic) modelling framework of DWD and MPI-M: Description of the non-hydrostatic dynamical core. Quart. J. Roy. Meteor. Soc., 141, 563–579, https://doi.org/10.1002/qj.2378.