1. Introduction
Numerical weather prediction (NWP) models usually incorporate horizontal diffusion or a filter scheme to control the grid-scale noise, which cannot be resolved for a given horizontal resolution, as well as to mimic the horizontal mixing (Boyd 1996; Gelb and Gleeson 2001; Majewski et al. 2002; Tomita and Satoh 2004; Knievel et al. 2007; Ullrich and Norman 2014). When the nonlinear model is time marched, the energy cascade, from large to small scale, is generated and leads to the accumulation of energy and enstrophy at the grid scale in the models. These unresolved scale disturbances tend to be aliased to the resolved scale during marching-in-time and will finally contaminate the numerical solution of the model. The horizontal diffusion process in the numerical model acts as the sink of energy and enstrophy so that it can prevent the buildup of spurious energy and enstrophy (Smagorinsky 1963; Shapiro 1970; Leith 1971; Xue 2000; Skamarock 2004; Cheong et. al 2004; Jablonowski and Williamson 2011; Marras et al. 2015).
When numerical noise is intended to be suppressed by diffusion schemes, it is required that well-resolved scales are diffused as little as possible. As discussed by several studies (MacVean 1983; Gelb and Gleeson 2001; Skamarock 2004; Jablonowski and Williamson 2011), second-order diffusion schemes are not scale selective because physically meaningful scales are dampened too much. Fourth-order diffusion schemes, which are more scale selective, have been most widely used in NWP models, and higher-order (6th- and 8th-order) diffusion schemes have also been adopted in some NWP models. The high-order diffusion scheme guarantees not only model stability, but also robustness at the model-resolvable scale. The effective resolution, defined as the decaying of the tail of the model kinetic energy spectra near the grid scale (Skamarock 2004), can also be increased by the high-order diffusion scheme. Therefore, the grid-scale-resolving performance of the global models is eventually dependent on the diffusion or filter schemes.
The numerical diffusion is implemented in several ways, depending on the numerical methods used to discretize partial differential equations. In the conventional spectral method using spherical harmonics functions (SHFs), the implicit horizontal diffusion (or high-order spectral filter), which is free of Courant–Friedrichs–Lewy (CFL) restrictions, is carried out in spectral space with ease by taking advantage of the fact that the eigenfunctions of the Laplace equation in the spherical coordinate system are SHFs (Sardeshmukh and Hoskins 1984; Collins et al. 2004; Takahashi et al. 2006). The double Fourier series spectral method (Cheong 2000), employing finite Fourier series both in the longitude and latitude, deals with the only tridiagonal matrix used for the implicit hyperdiffusion by successive inversion of the multiple Helmholtz equation matrices comprising the high-order filter matrix (Cheong et al. 2004). The gridpoint-based global models, incorporating horizontal discretization methods such as the finite difference method (FDM), the finite volume method, and the spectral element method (SEM) with nodal basis, usually adopt the explicit diffusion by adding an artificial term in the governing equation. Among a variety of the gridpoint models, the global models based on the cubed-sphere SEM are now widely accepted in the global modeling community because of their accuracy, grid flexibility, and parallel scalability (Giraldo and Rosmond 2004; Wang et al. 2007; Dennis et al. 2012; Evans et al. 2013; Choi and Hong 2016). A favorable choice of the order of diffusion scheme for the model is usually limited to the 4th-order diffusion (Dennis et al. 2012) because of the CFL restriction on the determination of a proper viscosity coefficient (Jablonowski and Williamson 2011; Cheong and Kang 2015), which becomes more serious as the order of the diffusion increases. Although the restriction can be avoided by the implicit treatment of the diffusion scheme, the implicit diffusion requiring inversion of the Helmholtz equation–type filter matrix can adversely affect the computational efficiency as well as the scalability.
Recently, Kang and Cheong (2017, hereafter KC17) developed a computationally efficient and parallel scalable high-order spatial filter for the spectral element model on the cubed sphere, where the filter equation is the high-order Helmholtz equation that corresponds to the implicit time differencing of a diffusion equation employing the high-order spherical Laplacian. Two types of the high-order filter, the global and local-domain high-order filters, were developed using the global and local-domain high-order Helmholtz equation matrices, respectively. As shown in KC17, the local-domain high-order filter has better computational efficiency than the global high-order filter without significant degradation of accuracy since it deals with the local filter matrix, consisting of 5 × 5 elements on the cubed sphere. Such a computational efficiency of local-domain filter over the global-domain filter was also demonstrated by Park et al. (2011) using the Fourier spectral method. By adopting the matrix operation method, which makes filtering a matrix–vector multiplication process, two high-order filters achieve near-perfect parallel scalability. However, the effect of high-order filters on the time-dependent flow was verified only in a two-dimensional shallow water equation model.
In the present study, the high-order filter for the cubed-sphere spectral element model developed by KC17 was implemented in a three-dimensional spectral element dry hydrostatic dynamical core (SEDC), and the performance of the high-order filter was evaluated by comparing it with the explicit diffusion, which is widely used for the spectral element dynamical core on the cubed sphere. The cubed-sphere SEDC is introduced in the following section, and the implementation of the high-order filter in the dynamical core is described briefly in section 3. The performance and computational efficiency of the high-order filter are evaluated in section 4. The present study is summarized in section 5.
2. The cubed-sphere spectral element dry hydrostatic dynamical core
A schematic of a cubed sphere displayed with the global relief data. Each face (thick red solid lines) of the cubed sphere is divided into 4 × 4 elements (thick black solid lines), and each element has 4 × 4 GLL grid points (thin black solid lines).
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
To diminish the numerical instability due to the oscillatory nature of the SEM operators during time integration of the system of the primitive equations (Taylor et al. 2009), a diffusion process should be implemented in the dynamical core. The biharmonic-type (∇4) explicit hyperviscosity operator is widely used, as in other dynamical cores incorporating the SEM, such as the spectral element version of the Community Atmosphere Model (CAM-SE) (Neale et al. 2010; Dennis et al. 2012) and Nonhydrostatic Unified Model of the Atmosphere (NUMA; Giraldo et al. 2013). In the present study, we adopted the explicit diffusion used in the CAM-SE. This hyperviscosity operator incorporating auxiliary vectors is solved explicitly using a mixed finite element integrated-by-parts formulation following Giraldo (1999). The hyperviscosity operator is applied to the prognostic variable, except for surface pressure, for each element after the last stage of SSP-RK3. The performance of the explicit diffusion is compared to the implicit diffusion, which will be presented in the next section.
It is well known that the cubed-sphere spectral element model has excellent parallel scalability. In the CAM-SE, the space-filling curve algorithm is employed for parallelization (Dennis et al. 2012). In the present study, parallelization of the hydrostatic dynamical core is very similar to that of the high-order filter (KC17), except for the exchange of the data at the boundaries of each element. The hydrostatic dynamical core constructed in this study was tested on the Tachyon II supercomputer system of the Korean Institute of Science and Technology Information (KISTI), having thousands of cores. The SEDC has the near-perfect scalability up to 5400 cores for a 0.5° global resolution (Ne = 60, Np = 4) with 30 vertical layers (Fig. 2), which is comparable to the parallel scalability of other dynamical cores incorporating the SEM, such as CAM-SE (Evans et al. 2013) and NUMA (Müller et al. 2018).
Speedup of the parallel computation for the dynamical core (a solid line with circles) on Tachyon II supercomputer system of KISTI, where the grid resolution is given as Ne = 60 and Np = 4, and the number of vertical layers is 30. The maximum number of processors used for the computation is 5 400. The dashed line shows perfect parallel scalability.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
3. Implementation of the high-order filter
The global and local-domain high-order filters, which are implicit-type filters, were developed by KC17. In this section, we briefly explain the procedure for constructing and implementing of the high-order filter.
The implementation of the global and local-domain high-order filters in the three-dimensional dynamical core is very similar to that in the shallow water model (KC17), except for the application of the horizontal filter for each vertical layer. In the dry hydrostatic dynamical core, the high-order filter was applied to the prognostic variables, such as u, υ, and T, except for surface pressure at every time step after the SSP-RK3 time-stepping procedure. Parallelization of the high-order filters is also described in KC17 (in their section 4), and the parallel scalability of the global and local-domain high-order filters was almost ideal, scaling up to 1350 cores.
The viscosity coefficient of the high-order filter can be defined theoretically using the eigenvalue of the Laplacian operator for the cubed-sphere SEM (Cheong and Kang 2015; KC17), as in spectral filters for the spectral transform models (MacVean 1983; Roeckner et al. 2003; Jablonowski and Williamson 2011). However, in the present study, we used the diffusion coefficient for the explicit diffusion in CAM-SE (Dennis et al. 2012) to see the effect of the different diffusion schemes having the same diffusion coefficient.
4. Performance evaluation of the high-order filter
In this section, the performance of the high-order filter is evaluated through test cases for the dynamical core presented by the DCMIP (Ullrich et al. 2012). The baroclinic instability test (referred to as test case 4–1-0 in DCMIP) and the quiescent atmosphere in the presence of surface topography test (test case 2–0-0 in DCMIP) were chosen. The model configurations for the test cases are given in Table 1.
Model configuration of the dynamical core for each test case chosen in this study. Test cases 4–1-0 and 2–0-0, which are named in the DCMIP documents, indicate the baroclinic instability test and the quiescent atmosphere in the presence of the surface topography test, respectively. Values in parentheses denote those values used for the higher-resolution simulation.
a. Baroclinic instability test
For this test, a small Gaussian-type perturbation of zonal wind was added to the balanced steady-state model atmosphere. The perturbation growth in the form of baroclinic waves occurred during 15 days from the time of integration, after which the waves broke, merged, and dissipated. As recommended in the DCMIP test case document (Ullrich et al. 2012), the dynamical core was run for 30 days. The spherical harmonics hydrostatic dynamical core (SHDC; Simmons and Burridge 1981) with T120 resolution, employing the 8th-order spectral filter and semi-implicit time stepping, was also used as a reference model for the comparison. In this dynamical core, the prognostic variables are filtered out in spherical harmonics spectral space.
Figure 3 and 4 show the zonal wind field (η = 0.69) and surface pressure field at day 9, respectively. The zonal wind and surface pressure fields of the six experiments exhibited very similar distribution even though different diffusion schemes and dynamical cores were used, since perturbations grew linearly at this time, as in other studies (Jablonowski and Williamson 2006; Cheong 2006; Lauritzen et al. 2010). It appears that the difference between different orders of diffusion schemes (e.g., 4th- and 8th-order local-domain filter for the SEDC) is larger than the difference between different dynamics (e.g., the SEDC and SHDC). As can be seen in the right panels of Fig. 3, slightly larger amplitudes in the zonal wind fields were observed in the results for which the higher-order (i.e., 8th order) diffusion schemes were used. After day 9, the baroclinic wave continued to evolve with time until day 15 (not shown), after which it was broken and mixed nonlinearly until the end of the simulation. As shown in Fig. 5, the spatial distributions of the zonal wind fields at day 30 are very complicated. It is hard to find common features among the results, but it is evident that the 8th-order diffusion leaves more small-scale perturbations than the 4th-order diffusion.
Zonal wind for the baroclinic instability test (η = 0.69) at day 9 produced by the dynamical core using (a) 4th-order global filter, (b) 8th-order global filter, (c) 4th-order local-domain filter, (d) 8th-order local-domain filter, (e) 4th-order explicit diffusion, and (f) the spherical harmonics spectral model with an 8th-order filter. The contour interval is 4 m s−1. Numerals in the upper right of each panel indicate minimum and maximum values of the zonal wind field.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
As in Fig. 3, but for the sea level pressure field. The contour interval is 10 hPa.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
As in Fig. 3, but for day 30. The contour interval is 6 m s−1, and the thick lines indicate zero-contour line.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
To analyze the above features more quantitatively and gain an appreciation for the characteristics of the model diffusion, the kinetic energy spectrum at day 30 on that eta level was calculated. Because of the unstructured nature of the cubed-sphere grid system, the interpolation of the zonal and meridional wind fields from the cubed-sphere grid to the Gaussian latitude–longitude grid is required to calculate the kinetic energy spectra. We interpolated the horizontal wind fields using the spectral method, after which the fields were transformed into the spherical harmonics spectral space to compute the kinetic energy spectrum. Figure 6 shows the kinetic energy spectra (η = 0.69) at day 30. As expected, it is clearly seen that the kinetic energy spectra produced by the 8th-order diffusion possess more energy than those produced by the 4th-order diffusion, regardless of the different dynamics (e.g., the SEDC and the SHDC) and different filtering schemes (e.g., the explicit diffusion, the global and local-domain filters, and the spectral filter), and they have larger values at spherical wavenumbers larger than 40. Furthermore, the kinetic energy spectra of the 8th-order diffusion exhibit a slope slightly steeper than −3, whereas those of the 4th-order diffusion show a slope significantly steeper than −3. The kinetic energy of the 8th-order diffusion near higher wavenumbers is larger by almost an order of magnitude, compared with that of the 4th-order diffusion.
Kinetic energy spectra (η = 0.69) at day 30 produced by the dynamical core using the 4th-order global filter (blue solid), the 8th-order global filter (blue dots), the 4th-order local-domain filter (red solid), the 8th-order local-domain filter (red dots), the spherical harmonics spectral model with the 8th-order spectral filter (gray solid), and the 4th-order explicit diffusion (black solid).
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
For the simulations done with 1° horizontal resolution (as above), the dynamical cores incorporating the global and local-domain 8th-order filters and the spherical harmonics spectral filter produced kinetic energy spectra behaving as k−3, which is a canonical structure of the atmospheric synoptic-scale kinetic energy spectrum (Lindborg 1999; Skamarock et al. 2014). On the other hand, the dynamical cores using the 4th-order diffusion could not generate the canonical kinetic energy spectrum. Successful simulation of the model-derived canonical spectrum is dependent on the configuration of the model diffusion. This trend becomes more conspicuous as the horizontal resolution increases, since there is a transition between synoptic-scale and mesoscale regimes behaving approximately as k−3 to k−5/3 (Lindborg 1999; Skamarock 2004; Lovejoy et al. 2009; Waite and Snyder 2009; Skamarock et al. 2014). The spectral slope of −5/3 does not appear in the results of the simulation with 1° resolution since a grid scale of that moderate horizontal resolution still belongs to synoptic scale. To simulate mesoscale dynamics, we carried out the same test with higher resolution by configuring Ne = 120, Np = 4 for the SEDC and T480 for the SHDC, which are approximately equal to a 0.25° horizontal grid spacing at the equator. The time step size was set to 30 s and 90 s for the SEDC and the SHDC, respectively. In this test, we omitted the experiment with the global high-order filter for the SEDC because of its similarity to the local high-order filter.
Figure 7 shows the global kinetic energy spectra (η = 0.69) from the initial time to day 30 for the higher-resolution simulation. From initial time to day 9, the kinetic energy spectra of all experiments showed similar patterns because the baroclinic wave grew linearly during that period. After day 9, different characteristics of the spectral tail between 4th-order diffusion (Figs. 7a,c) and 8th-order diffusion (Figs. 7b,d) became clear at high wavenumbers larger than 100. The kinetic energy spectra at day 30 (a red solid line) using the 8th-order filter (Figs. 7b,d) reveal the transition from the synoptic scale to mesoscale, consistent with the high-resolution model-derived kinetic energy spectra in other studies (Hamilton et al. 2008; Terasaki et al. 2009; Evans et al. 2013; Skamarock et al. 2014). The transition of the kinetic energy spectra for the 8th-order diffusion simulation begins at approximately wavenumber 100, whereas the spectra of the 4th-order diffusion kept the slightly shallow slope of −3 (Figs. 7a,c). The spectral characteristics for the SEDC with the 4th-order diffusion are like those of Evans et al. (2013, their Fig. 6), although the simulation was different. The slope of the spectra of the 8th-order diffusion is slightly shallower than −5/3 after the transition and starts to decay at approximately wavenumber 300, where the kinetic energy spectra of the 8th-order diffusion are larger by almost an order of magnitude, compared to those of the 4th-order diffusion. It seems that the effective resolution of the SEDC, as defined by Skamarock (2004), increased by using the 8th-order filter. It is worth noting that the spectra of the 8th-order filter for different dynamics (i.e., the SEDC and SHDC) exhibited very similar slopes.
Kinetic energy spectra (η = 0.69) for the high-resolution (about 0.25° at the equator) simulation of (a) the SEDC with a 4th-order filter, (b) the SEDC with an 8th-order filter, (c) the SEDC with a 4th-order explicit diffusion, and (d) the SHDC with an 8th-order spectral filter. From initial time to day 27 with a 3-day interval, colors of the solid lines of the kinetic energy spectra gradually change from light gray to black. The red solid line indicates the kinetic energy spectra at day 30.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
b. Quiescent atmosphere in the presence of the surface topography
The numerical filter or diffusion scheme should successfully remove the numerical noise to guarantee a stable run of the dynamical core, even in a very noisy model atmosphere. The steady-state atmosphere at rest in the presence of the moderately steep orography test case (DCMIP test case 2–0-0) is a stricter test case than the baroclinic instability test in terms of the numerical instability of the dynamical core. As the complexity of the bottom topography increases, the numerical noise due to the pressure gradient term associated numerical error may increase. This numerical noise appears as the spurious flow during time integration.
The Schär-type surface topography used in the experiments of the quiescent atmosphere in the presence of orography for (a) the CTL, (b) the H20, and (c) the O05 experiments. The embedded plot at the lower left of each panel is the surface height (m) along the equator.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
Figure 9 shows the time series of the maximum wind speed (u2 + υ2)1/2 for each experiment. For all diffusion schemes, the H20 experiment has the largest numerical error, compared to other experiments. It seems that the SEDC had more difficulty in resolving the steep topography than the complex one. The maximum wind speed for all experiments of the explicit diffusion was about 2 times larger than that of the local-domain high-order filter. In particular, the H20 experiment using the explicit diffusion shows the largest error. It is worth noting that the time evolution of the H20 experiment using the explicit diffusion fluctuated with time, which means that the dynamical core incorporating the explicit diffusion was unstable. Contrary to this, the time series of the local-domain high-order filter was smooth and increased gradually with time. Regardless of the magnitude of the viscosity coefficient, furthermore, the local high-order filter always shows better performance than the explicit diffusion (lower panels of Fig. 9). Sensitivity of both filters to the viscosity coefficient is quite similar, but it seems that the local high-order filter can suppress the numerical noise better, especially at early stages of the simulation. For the smaller viscosity coefficient, both diffusion schemes are more sensitive than the larger viscosity coefficient, but error of the local high-order filter is still lower. The maximum wind speed using the global high-order filter is very similar to that of the local-domain high-order filter (not shown).
The time series of the maximum wind speeds using (left) the explicit diffusion and (right) the local-domain high-order filter. Symbols ν2.0x and ν0.5x indicate experiments with a double and halved diffusion coefficient, respectively.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
Shown in Fig. 10 are the horizontal distributions of the wind speed at the lowest level by day 6. Although the dynamical cores start with a resting atmosphere, the spurious flow was generated and spread over the mountain. Because of the shape of the surface topography, the circular pattern of the wind speed field was dominant. High wind speed, especially, was generated commonly in all experiments around the center of the topography. The strongest wind is observed in the H20 experiment, whereas the spurious flow is most widely distributed for more complex topography (O05 experiment). It is worth noting that the results of the global and local-domain high-order filters are almost the same, even though different types of bottom topography were used for the experiments. As compared to the two high-order filters (i.e., the global and local-domain high-order filter), the wind speed using the explicit diffusion shows larger magnitudes (i.e., larger numerical error) and a greater spread in pattern, especially for the H20 case, which is the experiment with the steeper mountain. To see the vertical distribution of the wind speed, we determined the azimuthal average of the wind speed field above the center of the mountain (λm, θm) by noticing that the wind speed field has a circular pattern. The radius–eta cross section of the azimuthally averaged wind speed is shown in Fig. 11. For both the explicit diffusion and the local-domain high-order filter, the spurious wind is vertically spread over the entire model layer, and it is strong near the center of the mountain, as expected from the horizontal distribution. The flow oscillated horizontally but stretched vertically because the error source stems from the wavy form of the surface topography. For the local-domain high-order filter, a more stretched flow near the top of the model was observed around the center of the mountain. However, in the middle and lower atmospheres, the spurious flow using the local-domain high-order filter was significantly alleviated, particularly in the radius from about 10° to about 60°, regardless of the shape of the surface topography.
The horizontal distribution of the wind speed (m s−1) at the lowest model level by day 6 using the (left) explicit diffusion, (center) 4th-order global filter, and (right) 4th-order local-domain filter for the (top) CTL, (middle) H20, and (bottom) O05 test cases, where EXP, GIMP, and LIMP represent the explicit diffusion, the global high-order filter, and the local-domain high-order filter, respectively.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
The radius–η cross section of the wind speed (m s−1), azimuthally averaged over the center of topography, using the (left) explicit diffusion and (right) 4th-order local-domain filter for the (top) CTL, (middle) H20, and (bottom) O05 test cases, where EXP, GIMP, and LIMP represent the explicit diffusion, the global high-order filter, and the local-domain high-order filter, respectively.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
The dynamical core incorporating the implicit diffusion, as shown, had fewer numerical errors than that using the explicit diffusion in the noisy circumstance. This may result from the efficient removal of the small-scale numerical noise by the implicit diffusion. To see this feature quantitatively, we display the vertically averaged kinetic energy spectrum by day 6 in Fig. 12. As it should be, it is clearly seen that a decrease in kinetic energy across all experiments is observed for the local-domain high-order filter. Most of all, the kinetic energy spectrum at wavenumbers approximately larger than 60, for the local-domain high-order filter, is much weaker than that for the explicit diffusion. Thus, the high-order filter diminishes the small-scale numerical noise more efficiently than the explicit diffusion, and eventually, weaker spurious flow (or less numerical noise) was generated by the surface topography. The largest differences in the kinetic energy spectra between the two different diffusion schemes (i.e., the local-domain high-order filter and the explicit diffusion) were noted for the H20 experiments, while the O05 experiments showed the smallest differences. It might be thought that the high-order filter can contribute to the stability of the SEDC by reducing the numerical errors arising from steep gradient of topography.
The vertically averaged kinetic energy spectra at day 6 using the local-domain high-order filter (solid line) and the explicit diffusion (dashed line) for the (a) CTL, (b) H20, and (c) O05 experiments, where EXP and LIMP represent the explicit diffusion and the local-domain high-order filter, respectively.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
c. Effect of the high-order filter on the time step size
The high-order filters used in this study are implicit-type filters that are free of the CFL restriction. Furthermore, especially for the cubed-sphere SEM, the high-order filters can prevent the numerical noise from the C0 enforced element boundaries by taking advantage of the filter weights covering several elements (KC17). Therefore, the spectral element dynamical core with the high-order filter can take larger time step size than that with the explicit diffusion. The same test case used in the previous section was chosen to test the limitation of the time step size. The H20 experiment was carried out to demonstrate more clearly the difference between the implicit and explicit diffusion schemes. The model configurations were also the same, except for the time step size and the filter order. The results are shown in Table 2, where the crisscross notation “×” means that the dynamical core blew up before day 6. It is evident that the explicit diffusion is much more vulnerable to the numerical instability than the implicit diffusion. It was possible to set the time step size of the 6th-order global and local-domain filters to 3 times larger than that of the explicit diffusion. Although the dynamical core employing the 4th-order explicit diffusion was successfully terminated by day 6, the numerical error was larger than that for the high-order filters. As the diffusion order increases, the time step size of the explicit diffusion is reduced, while that of the high-order filters is maintained. It is worth noting that the numerical errors of the high-order filters for different time step sizes were very similar to one another for the same order of diffusion, while that of the 4th-order explicit diffusion was largely dependent on the time step size.
The maximum wind speed (m s−1) at day 6 for the simulation of the steady-state atmosphere at rest with surface topography, where EXP, GIMP, and LIMP represent the explicit diffusion, the global high-order filter, and the local-domain high-order filter, respectively. Numerals in the first row represent the time step size in seconds, and the symbol × denotes the model blow-up before day 6.
d. Computational efficiency of the local-domain high-order filter
The local-domain high-order filter, which incorporates multiple cells, was developed to reduce the computational cost of the global high-order filter that requires inversion of the huge global filter matrix and the huge sparse matrix–vector multiplication for the filtering. As shown in the previous sections, it was found that performance of the local-domain high-order filter was comparable to that of the global high-order filter. Thus, the local-domain high-order filter can take advantage of the computational efficiency without degrading the performance. Using the same test as in section 4b, the computational efficiency of the local-domain high-order filter was investigated.
Figure 13 shows the wall-clock time of the SEDC with three types of 4th-order diffusion schemes run for 6 days of time integration using 25 cores. The SEDC with the global high-order filter, as expected, required the longest time to complete the simulation. The wall-clock time of the local-domain high-order filter was reduced by a factor of 1.5, compared to the global high-order filter, while it was larger than that of the explicit diffusion by a factor of 1.6.
Wall-clock time histogram for 6 days of time integration using 25 processors in parallel computing of the dynamical cores adopting the explicit diffusion (EXP), the 4th-order local-domain filter (LIMP), and the 4th-order global filter (GIMP). Numerals in the histogram indicate the wall-clock time in seconds.
Citation: Monthly Weather Review 146, 7; 10.1175/MWR-D-17-0226.1
5. Summary
The global and local-domain high-order filters developed by KC17 were implemented in the three-dimensional spectral element dry hydrostatic dynamical core on the cubed sphere, and the performance was evaluated using dynamical core test cases. The high-order filter was applied to the prognostic variables, except for surface pressure, after the RK3 time stepping. The filter matrix for both high-order filters was constructed only once before time integration and it was applied to the prognostic variables repeatedly at every time step. The parallel scalability of the dynamical core and high-order filter was near perfect, as in other spectral element dynamical cores, such as CAM-SE and NUMA.
Two kinds of dynamical core test results produced by DCMIP demonstrate the usefulness of the high-order filter implemented in the cubed-sphere hydrostatic dynamical core. In comparison to the explicit diffusion, which is widely used in the cubed-sphere spectral element model, the baroclinic instability test showed that the effective resolution increased by using the 8th-order filter, especially for the high-resolution simulation, maintaining the model stability simultaneously. The performance of the high-order filter was also tested using the steady-state atmosphere at rest with the surface orography. Over all experiments carried out in this test, the global and the local high-order filter can control the grid-scale numerical noise more efficiently than explicit diffusion in any cases, as revealed by vertically averaged kinetic energy spectra. Although the numerical procedure for the filtering was somewhat different, the global and local-domain high-order filters produced very similar results, as was expected.
Computational efficiency of the high-order filter was investigated in terms of the time step size and the wall-clock time of the dynamical core. It was found that the dynamical core using the high-order filter could take time step sizes several times larger than that using the explicit diffusion. The computational time of the dynamical core using the local-domain high-order filter decreased by a factor of 1.5, compared to the global high-order filter, while it increased only by a factor of 1.6, compared to the explicit diffusion. Evidently, the local-domain high-order filter is more computationally efficient without undergoing significant performance degradation, compared to the global high-order filter.
Acknowledgments
The authors thank anonymous reviewers for constructive comments. This research was supported by the Korea Institute of Atmospheric Prediction Systems in 2017. Hyun-Gyu Kang acknowledges the financial support of the postdoctoral fellow program through the BK21 Plus Project of the Graduate School of Earth Environmental Hazard System (Grant 21A20151713014).
REFERENCES
Boyd, J., 1996: The erfc-log filter and the asymptotics of the Euler and Vandeven sequence accelerations. Proceedings of the Third International Conference on Spectral and High Order Methods, A. V. Ilin and L. Ridgway Scott, Eds., University of Houston, 267–276.
Cheong, H.-B., 2000: Double Fourier series on a sphere: Applications to elliptic and vorticity equations. J. Comput. Phys., 157, 327–349, https://doi.org/10.1006/jcph.1999.6385.
Cheong, H.-B., 2006: A dynamical core with double Fourier series: Comparison with the spherical harmonics method. Mon. Wea. Rev., 134, 1299–1315, https://doi.org/10.1175/MWR3121.1.
Cheong, H.-B., and H.-G. Kang, 2015: Eigensolutions of the spherical Laplacian for the cubed-sphere and icosahedral-hexagonal grids. Quart. J. Roy. Meteor. Soc., 141, 3383–3398, https://doi.org/10.1002/qj.2620.
Cheong, H.-B., I.-H. Kwon, and T.-Y. Goo, 2004: Further study on the high-order double-Fourier-series spectral filtering on a sphere. J. Comput. Phys., 193, 180–197, https://doi.org/10.1016/j.jcp.2003.07.029.
Choi, S.-J., and S.-Y. Hong, 2016: A global non-hydrostatic dynamical core using the spectral element method on a cubed-sphere grid. Asia-Pac. J. Atmos. Sci., 52, 291–307, https://doi.org/10.1007/s13143-016-0005-0.
Collins, W. D., and Coauthors, 2004: Description of the NCAR Community Atmosphere Model (CAM 3.0). NCAR Tech. Note NCAR/TN-464+STR, National Center for Atmosphere Research, Boulder, CO, 226 pp., http://www.cesm.ucar.edu/models/atm-cam/docs/description/description.pdf.
Dennis, J. M., and Coauthors, 2012: CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model. Int. J. High Perform. Comput. Appl., 26, 74–89, https://doi.org/10.1177/1094342011428142.
Evans, K. J., P. H. Lauritzen, S. K. Mishra, R. B. Neale, M. Taylor, and J. J. Tribbia, 2013: AMIP simulation with the CAM4 spectral element dynamical core. J. Climate, 26, 689–709, https://doi.org/10.1175/JCLI-D-11-00448.1.
Gelb, A., and J. P. Gleeson, 2001: Spectral viscosity for shallow water equations in spherical geometry. Mon. Wea. Rev., 129, 2346–2360, https://doi.org/10.1175/1520-0493(2001)129<2346:SVFSWE>2.0.CO;2.
Giraldo, F. X., 1999: Trajectory calculations for spherical geodesic grids in Cartesian space. Mon. Wea. Rev., 127, 1651–1662, https://doi.org/10.1175/1520-0493(1999)127<1651:TCFSGG>2.0.CO;2.
Giraldo, F. X., and T. Rosmond, 2004: A scalable spectral element Eulerian atmospheric model (SEE-AM) for NWP: Dynamical core tests. Mon. Wea. Rev., 132, 133–153, https://doi.org/10.1175/1520-0493(2004)132<0133:ASSEEA>2.0.CO;2.
Giraldo, F. X., J. F. Kelly, and E. M. Constantinescu, 2013: Implicit-explicit formulations of a three-dimensional nonhydrostatic unified model of the atmosphere (NUMA). SIAM J. Sci. Comput., 35, B1162–B1194, https://doi.org/10.1137/120876034.
Gottlieb, S., 2005: On high order strong stability preserving Runge-Kutta and multi step time discretizations. J. Sci. Comput., 25, 105–128, https://doi.org/10.1007/BF02728985.
Hamilton, K., Y. O. Takahashi, and W. Ohfuchi, 2008: Mesoscale spectrum of atmospheric motions investigated in a very fine resolution global general circulation model. J. Geophys. Res., 113, D18110, https://doi.org/10.1029/2008JD009785.
Jablonowski, C., and D. L. Williamson, 2006: A baroclinic instability test case for atmospheric model dynamical cores. Quart. J. Roy. Meteor. Soc., 132, 2943–2975, https://doi.org/10.1256/qj.06.12.
Jablonowski, C., and D. L. Williamson, 2011: The pros and cons of diffusion, filters and fixers in atmospheric general circulation models. Numerical Techniques for Global Atmospheric Models, P. H. Lauritzen et al., Eds., Lecture Notes in Computational Science and Engineering Series, Vol. 80, Springer, 381–493.
Kang, H.-G., and H.-B. Cheong, 2017: An efficient implementation of a high-order filter for a cubed-sphere spectral element model. J. Comput. Phys., 332, 66–82, https://doi.org/10.1016/j.jcp.2016.12.001.
Knievel, J. C., G. H. Bryan, and J. P. Hacker, 2007: Explicit numerical diffusion in the WRF Model. Mon. Wea. Rev., 135, 3808–3824, https://doi.org/10.1175/2007MWR2100.1.
Lauritzen, P. H., C. Jablonowski, M. A. Taylor, and R. D. Nair, 2010: Rotated versions of the Jablonowski steady-state and baroclinic wave test cases: A dynamical core intercomparison. J. Adv. Model. Earth Syst., 2 (15), https://doi.org/10.3894/JAMES.2010.2.15.
Leith, C. E., 1971: Atmospheric predictability and two-dimensional turbulence. J. Atmos. Sci., 28, 145–161, https://doi.org/10.1175/1520-0469(1971)028<0145:APATDT>2.0.CO;2.
Lindborg, E., 1999: Can the atmospheric kinetic energy spectrum be explained by two-dimensional turbulence? J. Fluid Mech., 388, 259–288, https://doi.org/10.1017/S0022112099004851.
Lovejoy, S., A. Tuck, D. Schertzer, and S. J. Hovde, 2009: Reinterpreting aircraft measurements in anisotropic scaling turbulence. Atmos. Chem. Phys., 9, 5007–5025, https://doi.org/10.5194/acp-9-5007-2009.
MacVean, M. K., 1983: The effects of horizontal diffusion on baroclinic development in a spectral model. Quart. J. Roy. Meteor. Soc., 109, 771–783, https://doi.org/10.1002/qj.49710946206.
Majewski, D., D. Liermann, P. Prohl, B. Ritter, M. Buchhold, T. Hanisch, G. Paul, and W. Wergen, 2002: The operational global icosahedral–hexagonal gridpoint model GME: Description and high-resolution tests. Mon. Wea. Rev., 130, 319–338, https://doi.org/10.1175/1520-0493(2002)130<0319:TOGIHG>2.0.CO;2.
Marras, S., M. A. Kopera, and F. X. Giraldo, 2015: Simulation of shallow-water jets with a unified element-based continuous/discontinuous Galerkin model with grid flexibility on the sphere. Quart. J. Roy. Meteor. Soc., 141, 1727–1739, https://doi.org/10.1002/qj.2474.
Müller, A., M. Kopera, S. Marras, L. C. Wilcox, T. Isaac, and F. X. Giraldo, 2018: Strong scaling for numerical weather prediction at petascale with the atmospheric model NUMA. Int. J. High Perform. Comput. Appl., https://doi.org/10.1177/1094342018763966, in press.
Nair, R. D., S. J. Thomas, and R. D. Loft, 2005: A discontinuous Galerkin transport scheme on the cubed sphere. Mon. Wea. Rev., 133, 814–828, https://doi.org/10.1175/MWR2890.1.
Neale, R. B., and Coauthors, 2010: Description of the NCAR Community Atmosphere Model (CAM 4.0). NCAR Tech. Note NCAR/TN-485+STR, 212 pp., www.cesm.ucar.edu/models/ccsm4.0/cam/docs/description/cam4_desc.pdf.
Park, J.-R., H.-B. Cheong, and H.-G. Kang, 2011: High-order spectral filter for the spherical-surface limited area. Mon. Wea. Rev., 139, 1256–1278, https://doi.org/10.1175/2010MWR3440.1.
Patera, A., 1984: A spectral element method for fluid dynamics: Laminar flow in a channel expansion. J. Comput. Phys., 54, 468–488, https://doi.org/10.1016/0021-9991(84)90128-1.
Roeckner, E., and Coauthors, 2003: The atmospheric general circulation model ECHAM5. Part I: Model description. Max Planck Institute for Meteorology Tech. Rep. 349, 127 pp., https://www.mpimet.mpg.de/fileadmin/publikationen/Reports/max_scirep_349.pdf.
Rønquist, E., 1988: Optimal spectral element methods for the unsteady three-dimensional incompressible Navier–Stokes equations. Ph.D. thesis, Massachusetts Institute of Technology, 176 pp.
Sardeshmukh, P. D., and B. I. Hoskins, 1984: Spatial smoothing on the sphere. Mon. Wea. Rev., 112, 2524–2529, https://doi.org/10.1175/1520-0493(1984)112<2524:SSOTS>2.0.CO;2.
Shapiro, R., 1970: Smoothing, filtering, and boundary effects. Rev. Geophys. Space Phys., 8, 359–387, https://doi.org/10.1029/RG008i002p00359.
Simmons, A. J., and D. M. Burridge, 1981: An energy and angular-momentum conserving vertical finite-difference scheme and hybrid vertical coordinates. Mon. Wea. Rev., 109, 758–766, https://doi.org/10.1175/1520-0493(1981)109<0758:AEAAMC>2.0.CO;2.
Skamarock, W. C., 2004: Evaluating mesoscale NWP models using kinetic energy spectra. Mon. Wea. Rev., 132, 3019–3032, https://doi.org/10.1175/MWR2830.1.
Skamarock, W. C., S.-H. Park, J. B. Klemp, and C. Snyder, 2014: Atmospheric kinetic energy spectra from global high-resolution nonhydrostatic simulations. J. Atmos. Sci., 71, 4369–4381, https://doi.org/10.1175/JAS-D-14-0114.1.
Smagorinsky, J., 1963: General circulation experiments with the primitive equations: I. The basic experiment. Mon. Wea. Rev., 91, 99–164, https://doi.org/10.1175/1520-0493(1963)091<0099:GCEWTP>2.3.CO;2.
Takahashi, Y. O., K. Hamilton, and W. Ohfuchi, 2006: Explicit global simulation of the mesoscale spectrum of atmospheric motions. Geophys. Res. Lett., 33, L12812, doi: 10.1029/2006GL026429.
Taylor, M., J. Tribbia, and M. Iskandarani, 1997: The spectral element method for the shallow water equations on the sphere. J. Comput. Phys., 130, 92–108, https://doi.org/10.1006/jcph.1996.5554.
Taylor, M., A. St. Cyr, and A. Fournier, 2009: A non-oscillatory advection operator for the compatible spectral element method. Computational Science—ICCS 2009, Lecture Notes in Computer Science Series, D. Hutchison et al., Eds., Vol. 5545, Springer, 273–282.
Terasaki, K., H. Tanaka, and M. Satoh, 2009: Characteristics of the kinetic energy spectrum of NICAM model atmosphere. SOLA, 5, 180–183, https://doi.org/10.2151/sola.2009–046.
Tomita, H., and M. Satoh, 2004: A new dynamical framework of nonhydrostatic global model using the icosahedral grid. Fluid Dyn. Res., 34, 357, https://doi.org/10.1016/j.fluiddyn.2004.03.003.
Ullrich, P. A., and M. R. Norman, 2014: The Flux-Form Semi-Lagrangian Spectral Element (FF-SLSE) method for tracer transport. Quart. J. Roy. Meteor. Soc., 140, 1069–1085, https://doi.org/10.1002/qj.2184.
Ullrich, P. A., C. Jablonowski, J. Kent, P. H. Lauritzen, R. D. Nair, and M. A. Taylor, 2012: Dynamical Core Model Intercomparison Project (DCMIP) test case document. DCMIP Summer School, 83 pp., http://earthsystemcog.org/projects/dcmip-2012/.
Waite, M. L., and C. Snyder, 2009: The mesoscale kinetic energy spectrum of a baroclinic life cycle. J. Atmos. Sci., 66, 883–901, https://doi.org/10.1175/2008JAS2829.1.
Wang, H., J. Tribbia, F. Baer, A. Fournier, and M. Taylor, 2007: A spectral element version of CAM2. Mon. Wea. Rev., 135, 3825–3840, https://doi.org/10.1175/2007MWR2058.1.
Xue, M., 2000: High-order monotonic numerical diffusion and smoothing. Mon. Wea. Rev., 128, 2853–2864, https://doi.org/10.1175/1520-0493(2000)128<2853:HOMNDA>2.0.CO;2.
Zhang, H., and M. Rančić, 2007: A global eta model on quasi-uniform grids. Quart. J. Roy. Meteor. Soc., 133, 517–528, https://doi.org/10.1002/qj.17.
