a. The OMEGA grid structure

A unique feature of the OMEGA model is its unstructured grid. The flexibility of unstructured grids facilitates the gridding of arbitrary surfaces and volumes in three dimensions. In particular, unstructured grid cells in the horizontal dimension can increase local resolution to better capture topography or the important physical features of atmospheric circulation flows and cloud dynamics. The underlying mathematics and numerical implementation of unstructured adaptive grid techniques have been evolving rapidly, and in many fields of application there is recognition that these methods are more efficient and accurate than the structured logical grid approach used in more traditional codes (Baum and Löhner 1994; Schnack et al. 1998). To date, however, unstructured grids and grid adaptivity have not been used in the atmospheric science community (Skamarock and Klemp 1993). OMEGA represents the first attempt to use this CFD technique for atmospheric simulation.

OMEGA is based on a triangular prism computational mesh that is unstructured in the horizontal dimension and structured in the vertical (Figs. 1 and 2). The rationale for this mesh is the physical reality that the atmosphere is highly variable horizontally, but always stratified vertically. While completely unstructured three-dimensional meshes have been used for other purposes (Baum et al. 1993; Luo et al. 1994), the benefit of having a structured vertical dimension in an atmospheric grid is a significant reduction in the computational requirements of the model. Specifically, the structured vertical grid enables the use of a tridiagonal solver for implicit solution of both vertical advection and vertical diffusion. Since in many grids the vertical grid spacing is one or more orders of magnitude smaller than the horizontal grid spacing, the ability to perform vertical operations implicitly relaxes the limitation on the time step.

In discussing unstructured grids, it is first necessary to define the elemental dimensional objects that describe the properties of a volumetric mesh. The lowest-order object of a grid is the vertex, which is specified by its position (x, y, z). An edge conveys the connectivity of the mesh and is defined by the indices of its starting and terminating vertices. The face represents the interface area between adjacent volumetric cells and can be described from the list of edges that bound it, or by the sequence of vertices that form its corners. The cell or control volume is in turn specified by the list of faces that contain it. In this volumetric mesh, scalar quantities are defined in the cell centroid, while the vector quantities are defined at the center of vertically stacked faces (Fig. 1).

On an unstructured grid, the number of edges that meet at a vertex is arbitrary. Consequently there is no longer a simple algebraic construct that can be used to deduce the relationship of indices for the various elemental objects, as in the case of structured grids that have been used as the basic structure for atmospheric and ocean circulation models up until now. Rather, the formation of the grid is tied to the actual solution of the model equations and to the topography. This means that the initial grid can be readily adapted to surface features or other fixed terrain features as well as the initial weather.

An important feature of the unstructured triangular grid methodology is the calculation of the normal to each face, which is required to calculate the flux across the face. Since these normals must be computed, there is no benefit from orienting the grid in any particular fashion, so long as the numerical resolution is sufficient to evaluate the critical fluxes. This leads to a natural separation between the coordinate system for the fundamental equation set and the grid structure. The coordinate system can be as simple as possible (such as Cartesian) while the grid structure, in this coordinate system, is extremely complex. OMEGA uses a rotating Cartesian coordinate system, but the grid structure is terrain following. Figure 2 shows a rotating Cartesian coordinate system in which the origin is the center of the earth, the z axis passes through the North Pole, the x axis passes through the intersection of the equator and the prime meridian, and the y axis is orthogonal to both.

In this coordinate frame, the equations of motion are in their simplest possible form (without going into a nonrotating frame that would lead to unusual boundary conditions as the surface terrain moved through the grid) with only two terms that are somewhat nonconventional:gravity and the Coriolis acceleration. Gravity in this frame is directed in the radial (−r̂) direction, which implies that it potentially has components in all three coordinate directions. The Coriolis force is by definition −2ρΩ × v and likewise has components in all three directions.

An important aspect of the OMEGA grid structure is that vertically stacked cells all possess the same footprint. This is accomplished by creating a surface grid and then projecting radials from the center of the earth through the vertices of this grid. Horizontal layers are constructed by specifying a set of vertices along each radial (Fig. 2). Because the OMEGA grid structure results in the mixing of the earth-relative horizontal and vertical components, it is essential that the numerical scheme be able to separate these. Given a grid structure that may be a few meters in vertical resolution and a few kilometers or a few tens of kilometers in horizontal resolution, the numerical resolution must be accurate to better than 1 part in 10⁵.

b. Fundamental equation set

In this section, we document the fully elastic nonhydrostatic equation set used in OMEGA, including the applicable assumptions. For brevity, we classify the five mixing ratios for water substances into two groups: precipitating water substances, Q_p (where the subscript p designates either rain or snow), and nonprecipitating water substances, Q_n (where n designates either ice crystals, water vapor, or water droplets). Furthermore, we cast the equations in their conservative form consistent with the fully elastic mass-conservation equation (∂ρ/∂t = −∇ · ρV). This form is better suited for the state-of-the-art upwind advection schemes we have used, which have significantly less numerical diffusion (important for the finer resolution we are trying to achieve) than the schemes used in other nonhydrostatic models such as the Terminal Area Simulation System (Proctor 1987).

We begin by decomposing the atmospheric pressure and density into a time-invariant hydrostatic base state and a perturbation upon that state such that

View Expanded

We then have the following equation set for the conservation of mass, momentum, and energy.

Conservation of mass:

View Expanded

where the subscript a refers to aerosols or gases. In Eqs. (4), ρ represents the dry air density, calculated from the total density as

ρρ_totalQ_vapor(5)

Conservation of momentum:

View Expanded

where the density used in the calculation of momentum is the total density.

Conservation of energy:

View Expanded

Terms have been arranged such that the conservative advection terms appear on the left side of each equation. The source terms on the right side of the momentum equation also include buoyancy and gravitational effects, −(ρ − ρ₀)gr̂ (where r̂ is the radial unit vector), and the Coriolis force (−2ρΩ × V). Subgrid-scale turbulence contributions, F, are discussed in detail later in this section. For the remaining equations; T is the temperature, L_i and S_i denote the latent heat and rate of phase conversion of either vaporization, fusion, or sublimation; and W_p represents the terminal velocity of each of the precipitating water substances. In addition, S_i depends on the microphysics that governs the rate of phase transitions and W_a depends on the assumed size distribution and mass of the hydrometeors. Also, M_n and M_p are the nonprecipitating and precipitating microphysics source terms. Finally, subscript a refers to transport of aerosol or gas.

The initial energy density E is calculated from the total pressure as

View Expanded

and throughout the simulation pressure is diagnosed from energy density using this relationship. Potential temperature, θ, and temperature, T, are related by Poisson’s equation,

View Expanded

where p_ref = 1000 mb, κ = R_d/c_p = 0.286, R_d is the gas constant for dry air, and c_p is the specific heat of dry air at constant pressure. Finally, S_R represents the contribution of radiation flux to heating the atmosphere.

c. Hydrodynamic solver

The hydrodynamic elements of the OMEGA model are based on numerical methods of solution of the Navier–Stokes equations on an unstructured grid in the horizontal direction and a structured grid in the vertical. In the calculation of momentum, the pressure gradient, Coriolis, and buoyancy terms are calculated explicitly along with the advection terms. An implicit vertical filter and an explicit horizontal filter are applied to the vertical momentum. The calculation of the new momentum at each time step thus involves several steps, which are described below. All implicit operations are performed by tridiagonal matrix inversion.

1) Advection—Implementation of the finite volume upwind solver

In this section we describe the implementation on unstructured grids of the multidimensional positive definite advection transport algorithm (MPDATA) originally developed on regular grids by Smolarkiewicz (1984), Smolarkiewicz and Clark (1986), and Smolarkiewicz and Grabowski (1990). The resulting scheme is second-order accurate in time and space, conservative, combines the virtues of the MPDATA (e.g., ability to separately ensure monotonicity and positive definiteness) with the flexibility of unstructured grids (Baum and Löhner 1994), is compatible with adaptive mesh algorithms (Fritts 1988), and can run efficiently on highly parallel computers. Below, we describe the essential methodology and demonstrate the method on two-dimensional passive advection test problems.

In discussing unstructured grids, it is necessary to define the nomenclatures. To reiterate, the basic control volume element in our structured–unstructured computational domain is a truncated triangular prism. Each prism is bounded by five faces. For advection across each face, it is convenient to define a local coordinate system with its origin located at the center of the face. Each face separates the left-hand side (lhs) from the right-hand side (rhs) such that the flow from the lhs cell to the rhs cell is considered positive. For stability, the advected variable, hereafter denoted as Ψ, is placed at the cell centroid, while the velocity vector is defined on the cell face at the origin of the local coordinate system. Figure 3 shows the basic arrangement of the variables on a two-dimensional grid.

In its explicit form MPDATA adapts naturally to the above construct. As posed by Smolarkiewicz, the algorithm can be generalized to the following steps.

1. At each cell face the low-order flux is found in conservative form using the standard first-order-accurate “upwind” scheme.

2. The advected variable is integrated using the low-order flux.

3. At each cell face, the low-order scheme is expanded in a Taylor series and the truncation error in the flux is explicitly identified.

4. The error term is cast in the form of error velocity, Ve.

5. The correction velocity, Vc (=−Ve), is optionally limited to preserve monotonicity of the advected variable (Smolarkiewicz and Gradowski 1990).

6. Replacing V with Vc, steps 1 through 5 are repeated a chosen number of times (=Nc) to achieve greater accuracy.

Although Smolarkiewicz derived Vc for a rectangular grid, it can be generalized to a grid with arbitrary control volume shape as long as the bounding faces are flat. Consider the generic advection equation,

View Expanded

To compute the change in Ψ from time t = t₀ to t₀ + Δt, it is necessary to integrate the flux Ψ_jV_j through each face j during the period Δt:

View Expanded

Here a_j = a_jê_n is the area vector of face j, where ê_n denotes the unit vector normal to the face and pointing from left to right. For this integral, second-order accuracy in space is achieved automatically by placing F = Ψ_jV_j at the center of each face (in practice, Ψ_j at the faces is obtained by interpolation). Similarly, to ensure second-order accuracy in time, Ψ_jV_j should be evaluated at t = t₀ + Δt/2. Assuming that a leapfrog algorithm is used (i.e., V_j is defined at t = t₀ + Δt/2), we need only expand Ψ_j in a Taylor’s series as

View Expanded

where the superscript 0 denotes an evaluation at t = t₀. Substituting (12) for Ψ_j in (11) and performing the time integral, Eq. (11) becomes

View Expanded

Now substituting (10) for ∂Ψ/∂_t in (13) we have

View Expanded

If we further let υ_n = V_j · ê_n denote the component of the velocity normal to the face (and thus V_n = υ_nê_n), then the first-order upwind flux term is given by

View Expanded

Now letting δ_L denote the vector pointing from the cell centroid on the left to the face, and δ_R the vector pointing from the face to the cell centroid on the right, we can write

_Ljδ_L∇_jRjδ_R∇_j(16)

Similarly, we will let

δ_LjLδ_RRj(17)

Now we can rewrite (15) as

ΔΨ^upwind_j = {V_nΨ⁰_j − 0.5|υ_n|ê_n(δ_LΨ + δ_RΨ)} · a_jΔt.(18)

The correction term is the difference between (14) and (18). After some algebraic manipulation, this correction term can be written as

View Expanded

where Ψ = (Ψ_R + Ψ_L)/2. The correction velocity is now

View Expanded

In actual implementation, the correction term becomes

View Expanded

where Ψ̃ is the value of the advected quantity following the first-order upwind advection step. We limit the number of correction steps to N_c = 1, since additional correction steps are not cost effective. The effect of additional correction steps is to bring the solution closer to second-order accuracy, which is very nearly achieved with just one correction step; the additional accuracy attainable is limited.

To compare with known results (Smolarkiewicz 1984;Smolarkiewicz and Clark 1986; Smolarkiewicz and Grabowski 1990), we performed the rotating cone test on triangular grids. Figure 4 shows the mesh and the initial placement of the passive scalar. Figure 5 shows the solutions after six revolutions of the cone. Figure 5a shows the results of the first-order “upwind” scheme (N_c = 0). As shown, the original cone shape has completely diffused. In Fig. 5b, we show the case with one correction step. This case has a background value of zero, which when coupled with the positive definiteness aspect of the algorithm, generates extra numerical diffusion that automatically ensures monotonicity. Nevertheless, the cone shape has been substantially preserved. We have also performed other standard tests such as the multiple vortex test suggested by Staniforth (1987). In general, all of our results are consistent with the previous tests.

The claimed near-second-order accuracy of our scheme is demonstrated by testing how the accumulated error in the computation scales with grid size. Three measurements of error are used for the rotating cone test: the root-mean-square error over the whole mesh, the root-mean-square error over an area extended by the original base of the cone, and deviation in the height of the cone. The mesh sizes range from one-and-a-half to one-half of those shown in Fig. 4, that is, a factor of 3 variation. Figure 6 shows that except for the leftmost data point, all three measurements of error scale approximately as Δ² (the dotted lines indicate the slopes of both Δ² and Δ scaling for comparison). The leftmost data points were generated from runs with the largest grid size.

2) Implicit acoustic damping

One of the major goals of the OMEGA model development is to achieve a simulation capability that transcends scales through the use of variable resolution. The accurate portrayal of aerosols in the planetary boundary layer will require fine resolution in the structured vertical direction and selective locations of fine horizontal resolution. Accurate, stable numerical solutions require that the time step of integration be determined by the finest grid spacing. This problem is magnified when using the fully compressible set of nonhydrostatic equations, since fast-moving sound waves are inherent in the solution. Indeed, in the structured direction, the CFL time step can become too restrictive. To overcome this restriction, the growth and propagation of acoustic waves are controlled through the application of vertical and horizontal filters that act on the vertical component of the momentum.

The momentum equation is integrated using a semi-implicit, fractional time step scheme. After explicit calculation of the advection and source terms, an implicit calculation is performed in which a correction term is added to the vertical component of momentum. This term, which has the effect of slowing the growth and speed of sound waves, is described below. The next partial step consists of the application of the filter described in the following section. Like the implicit correction, this term is based on and added to the vertical momentum. The implicit calculation of the eddy diffusion term completes the momentum advance to time step n + 1.

The first implicit calculation solves the following equation for :

View Expanded

where wⁿ is the vertical component of momentum at the beginning of the nth time step and w^n̂ is the value after the addition of the (explicitly calculated) terms representing advection, the Coriolis force, the pressure gradient contribution, and the buoyancy effect. Here, υ_s is the sound speed, and c_s is an empirically chosen coefficient. A stability analysis of the above equation shows that c_s must be greater than 1.0. The default value currently used in OMEGA is 2.5.

Eddy diffusion is also treated implicitly in the radial direction. The fractional step approach is necessary because the acoustic damping term described above effectively increases the inertia at short wavelengths, that is, ρ_effective ∼ ρ(1 + k²), where k is the wavenumber in the direction of propagation. This can reduce the action of the eddy diffusion term unless the operations are split into separate fractional steps.

Again, this implementation of the dispersion operator removes the severe time step restriction that can result from diffusion flux along the finescale radial direction. A separate explicit treatment is used for momentum diffusion in the horizontal direction.

3) Acoustic filtering

The purpose of filtering in the OMEGA model is to remove from the solution a particular type of undesirable feature: high-frequency oscillations in the pressure field. Since these oscillations are reminiscent of the undesirable oscillations occurring in spectral solutions, it is possible to borrow a filtering algorithm from spectral methodology, in which filtering can be accomplished in either physical or modal space. The filter applied in OMEGA is analogous to the Vandeven filter of order three when defined in physical space on a structured grid (cf., Karniadakis and Sherwin 1999). This filter, which is similar to a classical cosine filter, has the form of a second-order artificial viscosity term when the grid spacing is uniform.

Since the OMEGA grid is structured in the vertical direction, calculation of the vertical component of the filter is straightforward, while the horizontal component requires special consideration. The horizontal term is based on the horizontal component of the Laplacian, which is calculated using Green’s formula:

View Expanded

where A is the area enclosed by a curve. The discretized form becomes

View Expanded

where A is now the area of the triangle, n_i is the outward normal vector to the ith edge, and dL_i is the length of the ith edge. The normal derivative is defined as

View Expanded

where w_i is the value of w at the centroid of the triangle that shares the ith edge of the target triangle, w_c is the value of w at the centroid of the target triangle, and d_i is the distance between the two centroids.

This approximation is first-order accurate only if the line segment connecting the centroids is perpendicular to the shared edge; that is, this line segment coincides with the outward normal vector. To improve the accuracy of the approximation, we define a “ghost” triangle whose edges are the perpendicular bisectors of the line segments connecting the centroid of the target triangle with the centroids of its neighbors. The area and edge lengths of this ghost triangle are used in the formula for ∇²w. Figure 7 shows the ghost triangle constructed around the centroid of the inner cell.

4) Horizontal diffusion

The horizontal diffusion term in the momentum equation can be expanded as

∇K_d∇u∇K_d∇uK_d²u(26)

where K_d is the horizontal diffusion coefficient, given by

K_dD²(27)

Here D is the deformation rate, and Δ is the horizontal grid spacing (McCorcle 1988). Assuming local deformation gradients are small, this can be written as

∇K_d∇uK_d²u(28)

We define orthonormal vectors x and y lying in a plane tangent to the surface of the earth at the centroid of the triangle in which the deformation rate is sought. Projecting the velocity u onto this local coordinate system gives u and υ, the horizontal components of velocity. By assuming the velocity to be locally linear in x and y, it is possible to express u and υ as a_ux + b_uy + c_u and a_υx + b_υy + c_υ, respectively. For the horizontal deformation rate D we will use the approximation

View Expanded

Using the values of u and υ at the vertices of the triangle, it is then possible to solve for b_u and a_υ, which gives the local deformation rate as D = b_u + a_υ. In this way, actual horizontal deformation is obtained, as opposed to deformation in the plane of the triangle. Since the horizontal grid spacing is squared in the expression for the deformation coefficient, we use the area of the triangle. The Laplacian of each component of the velocity is calculated using the discretization of Green’s formula based on the ghost triangles described in the previous section.

d. Model physics

This section briefly discusses the OMEGA model physics including atmospheric turbulence, cloud microphysics, convective parameterization, and radiation transport. Atmospheric turbulence affects, among other things, the rate at which surface moisture and energy enters the atmosphere. Cloud microphysics and convective parameterization are important in the vertical distribution of this moisture and energy and in the formation of precipitation. And finally, radiation transport is important in determining the solar heating of the earth’s surface and the atmosphere.

1) Turbulence and the planetary boundary layer

The parameterization of turbulence in OMEGA [i.e., the forcing terms, F, in Eqs. (4), (6), and (7)] is divided into two parts: horizontal and vertical. The horizontal diffusion is a function of the deformation of momentum (discussed in the previous section), while the vertical diffusion is implemented using a multilevel planetary boundary layer model. The atmospheric boundary layer in OMEGA is treated separately as the viscous sublayer, the surface layer, and the transition layer.

The viscous sublayer is defined as the level near the ground (z < z₀, z₀ is the surface roughness) where the transfer of the dependent variables by molecular motion becomes important. Temperature and specific humidity in this layer are related to their ground surface values (Deardorff 1974). The surface layer is that part of the planetary boundary layer immediately above the viscous sublayer where small-scale turbulence dominates the transfer of energy, moisture, and momentum, and the vertical variation of the vertical fluxes is roughly less than 10%. In this layer, Monin–Obukhov similarity theory provides an acceptable framework for the interaction between the atmosphere and the underlying surface and virtually all numerical models make use of it one way or the other. In the OMEGA model, the revised formulas purposed by Beljaars and Holtslag (1991) are used. Finally, the vertical turbulent exchange above the surface layer is parameterized using the so-called level 2.5 closure technique developed by Mellor and Yamada (1974). This scheme (Mellor and Yamada 1982) has been selected from the hierarchy of available closure schemes, because it is a good compromise between accuracy and efficiency, and because it has been applied to many atmospheric modeling studies. The level 2.5 turbulence closure is based on the following prognostic equation for the turbulent kinetic energy, e:

View Expanded

The OMEGA model requires as boundary conditions the temperature and humidity at the roughness level. These must be calculated prognostically taking the interaction of the atmosphere and the land surface into account. Numerical studies have demonstrated that contrasts in land surface characteristics (e.g., soil texture and moisture, vegetation type) can generate local circulations as strong as sea breezes (McCorcle 1988; Chang and Wetzel 1991). In consequence, landscape variations strongly influence atmospheric dispersion patterns. The land surface module implemented in the OMEGA model is based on the scheme proposed by Noilhan and Planton (1989) and uses worldwide datasets for soil type, land use/land cover, vegetation index, climatological sea surface temperature, climatological subsurface temperature, and climatological soil moisture (discussed in section 5). These surface characteristics are used as primary parameters, while the spatial variation of secondary parameters such as surface roughness, albedo, thermal diffussivity of the soil, etc. are specified based on the primary parameters.

Finally, the surface of water bodies such as lakes and oceans is represented in the OMEGA model rather simply. Because of the large heat capacity of water and strong surface mixing, the surface temperature of water bodies, T_sea, is initially specified based climatological data and then assumed to be constant during the period of the mesoscale simulation. The air just above the water surface is assumed to be fully saturated by water vapor. The spatial variation of surface roughness length, z₀, is determined using the Charnock’s relationship (Charnock 1955) over the water surface.

4) Radiation

Because of the divergent behavior of different parts of the atmospheric radiation spectra, it is convenient to develop separate parameterizations for long and short wavelengths. The radiative source–sink term in the conservation of energy relation [Eq. (7)] can then be written as

View Expanded

where the terms on the right-hand side represent the temperature change resulting from longwave and shortwave radiative divergence flux in the vertical direction. The divergence of radiative energy in the horizontal direction is neglected, since its variation is much larger in the vertical direction on the mesoscale. The method of parameterizing this vertical flux takes into account the absorption of shortwave radiation by water vapor and the longwave energy emitted by water vapor and carbon dioxide using the computationally efficient technique of Sasamori (1972). It is essentially similar to the one used by Mahrer and Pielke (1977).

e. Model initial conditions

The function of the OMEGA data preprocessor is to convert a diverse mixture of real-time atmospheric and surface data for input into an OMEGA simulation. The process produces two datasets. One dataset specifies the initial values of all the model’s dependent variables as well as the surface-based model parameters. A second dataset consists of the time-dependent behavior of all of the model’s dependent variables along the lateral boundaries of the simulation domain. The OMEGA preprocessor can ingest real-time atmospheric data from NCEP and FNMOC. The core of the OMEGA analysis and initialization procedure consists of seven steps: 1) ingestion of the available data prepared by the PREPDAT module and the preliminary INIT file created by the GENINIT module, 2) objective analysis of sea surface temperature data, 3) processing and quality control of surface and upper air data, 4) objective analysis of surface and upper air data, 5) adjustments to ensure that the analyzed data meet desired constraints, 6) generation of objective analysis performance statistics, and 7) output of the initialization data to a modified INIT file.

The multivariate algorithm in the OMEGA data preprocessor is based on that described by DiMego (1988), which adapts in a natural way to the unstructured grid. The analysis proceeds horizontally layer by layer, cell by cell. The analysis is performed at the top center of each OMEGA grid cell as well as at the bottom of the lowest layer of cells. Corrections to the first-guess field for height and wind are computed using the multivariate algorithm while those for relative humidity are computed using the simpler univariate algorithm. Corrections are computed for an entire layer and smoothed with a four-point smoother before being added to the first guess. The smoother sets the value at a given cell to 50% of the original value plus 1/6 the value at each of the three adjacent cells. The height corrections are converted to pressure corrections since OMEGA defines grid locations by absolute altitude. The variables are then vertically interpolated to cell centroids. Temperature is analyzed only at the surface. At cell centroids, it is derived hydrostatically from the pressure at the top and bottom of the cell. If a level is less than 10 mb from the last analyzed level, the analysis at this level is skipped. The corrections to the first guess are then interpolated from the corrections at the nearest analyzed layers above and below. This reduces the problem of noisy temperature fields due to small differences in the height correction at closely spaced analysis layers.

The surface temperature data are used to hydrostatically adjust the surface height increment in the lowest levels as in DiMego (1988). At the ground, the surface height increment will be used directly. In the first few model levels above the ground (up to about 40 mb above the surface), the surface height increment is hydrostatically corrected by assuming that the surface temperature perturbation is valid over this layer. The corrected height increment is considered to be valid at the model level. In effect, a minisounding is created from each surface observation. The profile extends about 40 mb above the ground. Only the nearest height increment to a given level is included in the analysis matrix at any given grid point. This has the effect of influencing the thickness field near the ground so that the hydrostatically derived temperature field closely resembles a direct surface temperature analysis.

The final process in the preparation of the initialization file is the adjustment of the vertical temperature profile. Since height rather than temperature is analyzed by the 3D OI scheme, temperature profiles will sometimes include regions of superadiabatic lapse rates or temperature spikes. This is particularly true near the surface where closely spaced analysis layers mean that a small error in the height analysis at one level can produce large temperature anomalies in the layers above and below. Spikes are removed, one at a time, starting with the most significant spike. The spike is removed by adjusting the height of the layer interface lying between the grid point that contains the spike and the adjacent grid point that most resembles a spike of the opposite sign. This method is based on the assumption that a small error in the height analysis at one layer interface relative to the interfaces above and below will introduce a warm–cold dipole in the temperature field at the layers immediately above and below. Once all significant spikes are removed, superadiabatic layers are removed in a similar manner. Often, the removal of temperature spikes also removes superadiabatic layers so that this final step is not needed. The adjustment of the intermediate height is done so that temperature lapse rate between the grid points above and below is set to the mean lapse rate of a layer extending one level above and below any contiguous layers containing a spike or superadiabatic lapse rate.

f. Model boundary conditions

In cloud-scale models, in which the timescale is small enough to assume that the environment outside the model domain is unchanged during the simulation, inflow boundaries are usually prescribed from environmental conditions. In mesoscale and larger-scale models this assumption can no longer be made. The environment outside the computational domain has to be derived from larger-scale forecast tools such as the Nested Grid Model, the Eta Model, or the Medium Range Forecast model run at NCEP, or the Navy Operational Global Atmospheric Prediction System (NOGAPS) run at the Navy Fleet Numerical Meteorology and Oceanography Center. Boundary conditions at user-specified time intervals are calculated based on one of these forecasts, and linear interpolation is used to determine boundary values at intermediate times.

The lateral boundaries are open and should allow the unimpeded flow of air. At outflow boundaries interior values rather than forecast data are used in the calculation of the fluxes across the boundary faces. To allow the propagation of acoustic disturbances across the boundaries, we use a radiative boundary condition with a uniform phase speed on outflow boundaries. At the end of each time step, a “nudging” scheme is applied to the new values of all variables, which nudges values in cells immediately inside the boundary to the forecast values in the ghost cells across the boundary. At inflow boundaries forecast data are used to calculate both the velocity normal to each boundary face and the fluxes across the face.

At the top of the computational domain, initial values of density and potential temperature are calculated assuming a hydrostatic balance, and these values remain fixed throughout the simulation. A homogeneous Neumann boundary condition is applied to the horizontal components of momentum, while the vertical momentum is assumed to be zero. Thus the upper boundary is a rigid, free-slip surface. To minimize reflection off of this boundary, a diffusive “sponge” is applied to the vertical component of the momentum. The coefficient of the sponge is exponential and, thus, approaches zero rapidly below the top few layers of the grid.

The surface boundary conditions applied in the OMEGA model are formulated with the aid of Monin–Obukhov similarity theory. This theory allows one to determine the values of the model’s prognostic variables at a diagnostic level between the first and second computational levels.

At the ground surface, a no-slip condition is imposed. The hydrostatic equation is used to obtain the surface pressure from level 2 (the lowest atmospheric layer in the model), while the surface density, ρ, is computed from the known surface pressure and ground surface temperature using the ideal gas law. For all Eulerian tracers (Q_t) and for the turbulent kinetic energy, a zero flux boundary condition is assumed at the ground.

a. Eulerian dispersion model

The Eulerian dispersion model embedded within the OMEGA model assumes that the released gas instantaneously and uniformly fills the OMEGA cell containing the source. The Eulerian tracers are also currently assumed to be massless and hence are advected by the mean wind and diffused by its turbulence components. The conservation equation for the tracer field can then be written as

View Expanded

where χ is the tracer concentration, F_χ represents subgrid-scale turbulent transport, and S_χ represents the tracer sources/sinks. Sources and sinks (S_χ) are specified via their source strength, latitude, longitude, and altitude locations.

In general, Eulerian models are well suited to handle complicated physical processes such as reactive nonlinear chemistry and wet deposition. These models are also well suited to handle a large number of geographically fixed sources, since their gridpoint calculations are independent of the number of sources. However, Eulerian models suffer from several limitations. First, the treatment of advection in Eulerian models usually introduces artificial numerical diffusion and sometimes spurious oscillatory behavior, a problem when advecting nonnegative physical quantities such as concentrations. Second, use of the gradient-transfer hypothesis limits these models to timescales much larger than the turbulence integral timescale and to pollutant spatial scales much larger than the turbulence integral spatial scales. Third, the pollutant mass being modeled must have a spatial extent at least equal to four or more horizontal and vertical grid increments in order for the gradients to be adequately defined and the advection phase errors minimized. Finally, these characteristics make point and line sources particularly difficult to treat in Eulerian models.

b. Lagrangian particle dispersion model

The Lagrangian particle dispersion model implemented in the OMEGA model applies the Monte Carlo technique to simulate dispersion of nonbuoyant and dynamically passive pollutants released from multiple emission sources with different geometry and release duration. The model can be used for a computer visualization of the OMEGA flow field in complex terrain with the aid of particle distributions and trajectories. The Lagrangian particle dispersion model basically simulates the dispersion of pollutants by tracking a large set of Lagrangian particles moving, at each time step, with pseudovelocities. These pseudovelocities simulate the effects of the two basic dispersion components: 1) transport due to the mean wind, and 2) diffusion due to turbulent velocity fluctuations (Boybeyi et al. 1995). In addition, particle velocity is adjusted so that particles are allowed to fall out with a vertical slip velocity due to gravity. Subsequent positions of each particle (x, y, z) representing a discrete element of pollutant mass are computed from the following:

View Expanded

where u, υ, and w are the resolvable-scale components of wind velocity obtained directly from the OMEGA model, and u′, υ′, and w′ are the corresponding subgrid-scale turbulent velocity fluctuations whose statistics are derived from the boundary layer formulations used in the OMEGA model.

An important advantage of turbulent kinetic energy closure implemented in the OMEGA model is that it allows one to determine a variety of turbulent variables including the Lagrangian turbulent statistics. The Lagrangian turbulent statistics necessary to run the Lagrangian particle dispersion model are obtained from the level 2.5 Mellor and Yamada scheme (cf. Uliasz 1990).

Lagrangian particle dispersion models have a number of advantages over other simulation techniques. First, Lagrangian particle dispersion models combine all uncertainties into the correct determination of the pseudovelocities. Second, a Lagrangian framework is more natural for modeling turbulent diffusion, which is a Lagrangian phenomenon. Third, Lagrangian particle models do not suffer from computational phase dispersion because no advection terms need to be considered in the mass conservation equations. Fourth, diffusion over small space and time scales, including emissions from multiple-point pollutant sources, can be treated easily by these models. Fifth, Lagrangian particle models are usually able to incorporate more turbulence properties than other simulation techniques due to their ability to incorporate random turbulent components. Finally, Lagrangian particle dispersion models are flexible, conceptually simple, and computationally inexpensive when a reasonable number of particles are used. In principle, Lagrangian particle dispersion models can provide a degree of resolution and accuracy to complex flow solutions not obtainable by other simulation techniques, if the mean flow, Lagrangian timescales, and turbulent statistics are provided. However, they are not well suited for modeling the dispersion of nonlinear reacting pollutants and the question of statistical sampling error must be addressed when estimating pollutant concentrations in areas of low particle density, sometimes requiring the release of a very large number of particles.

c. Probabilistic puff model

The simpler version of the Lagrangian particle dispersion model allows us to track the motion of puffs with a Gaussian distribution of pollutant mass. Continuous plumes are commonly represented by subdividing them into discrete, overlapping puffs. Puff models are conceptually simple and this simplicity usually translates into relatively straightforward solutions to the mathematical and computer programming requirements.

The probabilistic puff model implemented in the OMEGA model generates a new group of Lagrangian puffs (N) at a user’s defined time interval (Δt). The number of puffs released depends on local wind speed and turbulent diffusion at the source. Here N is chosen in such a way that the distance between two puffs should be approximately equal to half of the horizontal standard deviation, σ_h. Each puff generated represents a discrete element of the total material released into the atmosphere.

The puffs are moved using the OMEGA model mean wind field only. Locations of each puff center are described by Eqs. (34)–(36) where turbulent components are omitted. Puff diffusion, the distribution of material within each puff (around the puff centroid) in space according to a Gaussian function whose σ_x, σ_y, and σ_z values increase with travel time or distance, is estimated based on Taylor’s homogeneous diffusion theory. It is assumed that the theory is applicable over short timescales (cf. Uliasz 1990 or Arya, 1999), such as the time step, Δt, used to transport puffs in OMEGA:

View Expanded

where σ_u is the variance of wind velocity component and T^u_L is the Lagrangian integral timescale. These turbulence characteristics are obtained from the turbulence closure scheme used in the OMEGA model. In a similar way σ_y and σ_z are determined (Uliasz 1990).

As the puff grows, the local representation of the turbulence and velocity fields using puff centroid location becomes increasingly inaccurate. When the meteorological fields are inhomogeneous, the accuracy of the calculation can only be maintained by splitting puffs into smaller components that can sample the variations in the meteorology explicitly. A puff splitting scheme is implemented in such a way that when the growth of a puff extends either vertically or horizontally over one cell area of the OMEGA model grid, then the puff is divided into two or more puffs such that the sum of their masses equals the mass of the initial puff.

The key element to the practical application is the elimination of duplicate puffs. The generation of new puffs, especially due to vertical diffusion, can quickly overwhelm even the fastest and largest computers. For this reason, overlapping puffs with Gaussian concentration distributions are merged when they are separated by about σ_h or less. This reduces the number of puffs used in the probabilistic puff model. Every 10 min all adjacent puffs are checked to see if the puff centroids are within σ_h of each other. If so, the material from both is merged into a single puff. The centroid of the merged puff is assigned new coordinates and σ_h values that are averages of those in the original puffs. Puffs are also removed from further consideration when the puff centroid point is outside of the modeling domain. This approach should cause no problems within the area of interest, except when a wind reversal brings puffs back into the modeling domain.

The concentration within each puff is distributed in space according to a Gaussian function whose σ_x, σ_y, and σ_z values increase with travel time or distance. Knowledge of puff location, the material in it, and its geometrical dimensions allows concentration to be determined anywhere by summing contributions from each puff. Local concentration values are receptor oriented and based on summing contributions from each puff using generalized Gaussian distribution (Sykes et al. 1986).

Observations have shown that concentration fluctuations are sometimes at least as large as the mean. Therefore, there is a growing need for methods of predicting fluctuations of concentration. For example, some accidentally released gases are toxic and their flammability is determined by short-term concentration levels [e.g., the accidental release of MIC gas at Bhopal (Boybeyi et al. 1995)]. In this emergency case, the prediction of concentration fluctuations is crucial. On the other hand, the magnitude of turbulent fluctuations of concentration determines the uncertainty in air quality models. Uncertainty calculation is also important in most of emergency hazard predictions. Wilson et al. (1982a,b) were concerned with simulating concentration fluctuations from a continuous source observed in a wind tunnel and proposed a simplified empirical model based on the standard Gaussian formula. This formulation for the concentration fluctuation calculation is used in the OMEGA probabilistic puff model.

Puff models are conceptually simple, and computationally inexpensive. These models are also used widely in atmospheric dispersion studies. However, puff models usually ignore entrainment, cloud venting, and other mixing effects with the ambient environment, and they cannot accommodate nonlinear chemistry. Finally, treatment of multiple pollutant sources in puff models may quickly become very complicated.

a. Static grid adaptivity

The total number of grid points necessary to perform a successful numerical computation that recovers the correct physics can be greatly reduced in an unstructured grid. By this we mean that the recovery of the model physics at the smallest length scale resolved does not require the complete domain to have the same resolution. The resources of the numerical and computational machinery are focused on the regions of importance. This is especially significant for three-dimensional hydrodynamic problems, where our experience has shown the resulting economy can make the difference between tractability and intractability (cf., Baum et al. 1993).

In OMEGA, the adaptation of the unstructured grid takes place through a variety of grid operations (Figs. 9–11). The first is vertex addition, which is followed by vertex reconnection. Figure 9 illustrates these two steps when some activity that would indicate a need for more resolution is noted in two cells. The vertex addition step is accomplished by adding a vertex at the centroid of each affected cell and connecting it to the vertices of the cell. The reconnection step then involves the evaluation of each new cell to see if it is possible to create grid cells with lower aspect ratios by removing an edge and reconnecting the vertices.

Figure 10 shows the reverse process, in which the grid is coarsened through the process of vertex deletion. Like vertex addition, vertex deletion is followed by a vertex reconnection step. It is important to note that the goal of grid adaptation is not to move the grid, but rather to refine the grid in advance of any important physical process that might require additional grid resolution, and to coarsen the grid behind the region. This differentiates the method from the adaptation techniques described by Dietachmeyer and Drogemeier (1992), which used vertex movement to adapt to atmospheric features.

A different type of grid modification is also shown in Fig. 11. In this figure we show vertex relaxation, in which the vertices are allowed to move as a mass-spring system, and edge bifurcation, which is equivalent to vertex addition in the special case of an edge cell. Vertex relaxation is used to keep the aspect ratio as near unity as possible to minimize numerical errors in computing gradients.

The OMEGA grid in the “static” adaptation case is adapted a priori to resolve static features such as terrain gradients, land–water boundaries, and/or any other feature that the user includes in the adaptation scheme. The grid does not change during the course of the OMEGA run. The OMEGA grid automatically adapts to static features of the underlying topography such as complex terrain and land–water boundaries.

An example of the flexibility of the OMEGA grid is shown in Fig. 12. This figure shows a grid generated for the southeastern United States and Mexico. The grid was adapted to the underlying topography, the land–water boundaries, and to the initial weather conditions. The synoptic situation chosen was Hurricane Linda at 0000UTC on 10 September 1997. In this example, we have broken the grid-generation process into different steps for illustration. The first grid shown was generated by adapting to gradients in elevation (refining the grid in mountainous areas), the second to gradients in the land–water index (refining the grid in coastal areas), the third to the hurricane, and the fourth to all of these criteria. The final surface grid consists of approximately 7300 triangles with edges ranging from roughly 15 to 300 km.

In addition to the automatic static adaptation to geographical features, the grid can be refined in one or more specified geographical areas, such as theaters of operation, by the creation of up to 99 rectangular subdomains in which higher resolutions are specified. Within each subdomain, grid generation is governed by the user-specified minimum and maximum resolutions. Subdomains can overlap to create a high-resolution region of almost any shape, or to produce a high-resolution area within an intermediate-resolution region. In addition to subdomains, the user can specify a location on the grid and a radius of influence around it; the grid generator will then refine the grid within the region of influence. A smoothing process is also performed on the resolution specifications to avoid high expansion ratios at the boundaries of subdomains. A very important point, however, must be made: the result of the use of subdomains and/or circular region refinement is a single, not nested, grid with variable resolution.

An important feature of the use of an unstructured grid is the ability to simulate mountains and coastal features without the “stair-step” geometry required by nested grid models (cf., Zhang et al. 1999). Triangular grids can naturally follow the coastline better, leading to improved land–water circulations, and can better capture the geometry of mountainous regions. This is especially important for near-surface simulations.

b. Dynamic grid adaptation

The grid adaptation described so far is a “static” adaptation; adaptation is implemented in the grid generator prior to the beginning of an OMEGA run, and the grid does not change during the simulation. OMEGA also has the ability to adapt its grid during a simulation to different criteria such as frontal activity, convection, hurricanes, and/or a pollutant plume. This enables atmospheric features that require additional grid points for adequate simulation to be resolved as they appear. Thus through the combination of adaptation methods (Figs. 9–11) and criteria (Figs. 12a–c), the grid can be coarse where the circulation is regular and smooth, but greatly refined where there are sharp gradients, where there are discontinuities in the flow, where topographic features are important, or where model physics or dispersion source terms require fine resolution. The underlying philosophy of dynamic adaptation is to provide a refined grid to resolve important physical events and features as the simulation is in progress.

The dynamic grid adaptation in OMEGA consists of four major steps: 1) at a predetermined time step specific variables or their gradients are evaluated to see if they meet the adaptivity criteria, 2) the mesh is refined where these criteria are satisfied, 3) the physical variables are interpolated to new cell centers, and finally 4) the mesh is coarsened where the criteria are not met. For example, if there are sharp discontinuities such as propagating fronts the mesh is refined ahead of the front so that a refined mesh already exists when the front arrives at a specific location. As the front propagates, the refined mesh at the previous location of the front is coarsened to the resolution specific to the background mesh. This type of coarsening and refining is performed at predetermined time step increments.

Physical variables, which are cell centered, are interpolated to vertices before each adaptation cycle using the pseudo-Laplacian weighted averaging scheme. As new cells are created or removed locally a simple linear interpolation is done to assign values to new vertices and cell centers. All the diagnostic variables are calculated for new cells, the particle location array is updated, underlying terrain is updated, and the mesh is remasked to calculate new time steps for the time-splitting scheme. The original base-state variables are also saved, and as new cells are created, base-state profiles are generated for these cells using the initial base state. At the beginning of a simulation the minimum elevation is saved. As the grid is refined or coarsened, no cell is allowed to have elevation lower than the initial minimum elevation value. This is to ensure that no extrapolation is done in the calculation of the base state. The whole procedure requires only a few user inputs at the beginning of the simulation and after the initial inputs, it is completely automated. After the grid has been modified all the physical variables are interpolated to the new mesh and the diagnostic variables are calculated.

Setting the right criteria for adaptation is very important. There is a significant cost associated with a grid adaptation; hence, the ideal criteria are those that require minimum computational effort to evaluate yet indicate key regions requiring additional resolution. If the adaptation criteria are particle locations representing a pollutant plume or puff, then it becomes a simple step of locating the cells containing particles, and then tagging those cells for refinement. This method can become costly if there are multiple release sources and the frequency of particle releases is very high. The alternative method is adapting to the flow itself. This is not as straightforward as tagging the cells for particle locations. Atmospheric flows are almost always turbulent and highly stratified in the vertical. The gradients are often weak and hard to detect. One advantage in simulating atmospheric flows, however, is the large temporal scale of the simulation (usually 12–48 h). Therefore, even if an event is missed at first, the chances of capturing it at a later time are higher.

The OMEGA grid dynamically adapts to any criteria specified by the user. For example, OMEGA can provide high-fidelity solutions in the regions of particle plumes by refining the grid around the particle locations (Fig. 13) and coarsening the grid where high resolution is not required (e.g., areas from which the plume has already passed). The criteria for adaptation is set by defining a Gaussian function around each particle. The region close to particles therefore gets higher weighting and is tagged for refinement; areas devoid of particles and consisting of relatively flat terrain are tagged for coarsening.

Dynamic adaptation was demonstrated for a simple flow, but including the effects of sulfate chemistry in Sarma et al. (1999). This case demonstrated how the use of dynamic adaptation could capture 85% of the total sulfate production computed by a uniform high-resolution simulation, in 20% of the computational time. For our comparison here, we present the results from two pairs of simulations: 1) a high-resolution static simulation compared against an initially coarse-resolution dynamically adapting simulation and 2) a coarse-resolution static simulation compared against a dynamically adapting simulation. The former demonstrates that the dynamically adapting solution does, in fact, match that of a static solution using high resolution at all times while consuming considerably fewer computational cycles. The latter demonstrates that dynamic adaptation can improve the quality of a simulation by resolving terrain features that might otherwise be missed due to the use of static resolution.

The first pair of simulations, illustrating the advantage of dynamic grid adaptation over global refinement, is shown in Fig. 13. The first case (global refinement case) had a domain of roughly 300 km × 400 km covering parts of Nebraska, South Dakota, Iowa, and Minnesota. A variable grid resolution of 5 km to 15 km was specified. The grid had roughly 2300 cells in the horizontal layer with 30 vertical levels. In the vertical, a stretched grid was used and the first level near earth’s surface had a thickness of 15 m. A particle source was defined in the southwest corner of the domain. The simulation was initialized using the U.S. Navy’s NOGAPS gridded data. The simulation was run for 15 h.

The second case (dynamic adaptation case) used identical parameters to the global refinement simulation except for the initial grid size. The starting grid had roughly 600 cells in the horizontal (not shown). In this case a variable resolution of 10–36 km was prescribed in the initial grid. The adaptation criteria was set to particle locations. A refinement cycle was invoked after every hour of simulation time. Cells were tagged by defining a Gaussian function around the particles. Tagged cells were refined and all physical fields were interpolated at each adaptation cycle. A coarsening cycle was invoked after every 3 h to remove triangles from areas where higher resolution was not needed. This was done by identifying the cells that did not contain particles and were in the relatively flat regions and then deleting them. The final grid after 15 h of simulation had roughly 1100 cells and the resolution varied from 6 to 37 km. This simulation required less than 60% of the CPU time yet produced nearly identical results compared to global refinement. The simulation required only the initial user inputs, after which the whole process was seamless and automated.

In each adaptation cycle, OMEGA updates the underlying terrain. Thus, the refinement cycle can resolve finer topographic features, improving the solution. The second pair of simulations demonstrates this fact (Fig. 14). The simulations were conducted in the Four Corners region of the United States, which is characterized by complex terrain features. The domain and the initial conditions for both simulations were identical. The simulation domain was 300 km × 300 km in size. The initial grid had roughly 450 cells with a varying resolution from 7 to 42 km. In the first simulation, the grid was not refined during the simulation. Figure 14a shows the particle locations 12 h into the simulation. In the second simulation, the grid was adapted dynamically during the simulation. The adaptation criteria was set to particle locations. Figure 14b shows the particle locations for the second simulation after 12 h. The final grid had roughly 900 cells in a horizontal layer. As can be seen from the figure, by resolving finer terrain features as the grid dynamically adapted to the evolving plume, a different and presumably more accurate representation of the flow field (since it will be affected by the terrain) and hence plume trajectory were achieved. This figure schematically shows that fine spatial discretization of the computational domain may result in a more accurate solution of any complex computational problem.

a. Worldwide datasets

A key advantage of the OMEGA model is its worldwide datasets. The OMEGA model has eight major worldwide databases for terrain elevation, land–water distribution, soil type, land use–land cover, climatological vegetation index, climatological sea surface temperature, climatological subsurface temperature, and climatological soil moisture (Table 2). In addition to these datasets, the worldwide mapping dataset from the Digital Chart of the World is also included in the OMEGA modeling system.

The first two characteristics (terrain elevation and land–water distribution) are determined from global datasets with two different resolutions: 1) moderate resolution of 5 arc-minutes (10 km), and 2) high resolution of 30 arc-second (1 km). The other characteristics rely on newer datasets from various sources. For example, a 1° global soil-type database was created from the Global Ecosystems Database (GED) CD-ROM (Webb et al. 1992). The 12 types used in the OMEGA preprocessor are sand, loamy sand, silt loam, loam, sandy clay loam, silty clay loam, silt, clay loam, sandy clay, silty clay, clay, and peat.

In OMEGA, vegetation type is defined for each grid cell based on three different datasets. The first dataset is derived from the global monthly Normalized Difference Vegetation Index (NDVI) datasets covering the period 1985–90 published on the Global Ecosystems Database CD-ROM, based on biweekly calibrated global vegetation index (GVI) data (EDC-NESDIS 1992). These monthly datasets were averaged to form a set of monthly climatology files with a resolution of 10 arc-minutes (18 km). The second dataset is derived from a series of high-resolution Advanced Very High Resolution Radiometer (AVHRR) datasets (Eidenshink 1992) over the conterminous Unites States published by the Earth Resources Observation System Data Center of the United States Geological Survey (USGS). The datasets were averaged from 1990 to 1993 to produce biweekly climatological datasets covering the conterminous United States with a resolution of 2 arc-minutes (about 4 km). The third database is a USGS biweekly dataset covering the globe with a resolution of 30 arc-seconds (1 km).

Similarly, three separate land use datasets were also created for the OMEGA model. Each set uses the Biosphere–Atmosphere Transfer Scheme (BATS) set of 18 land cover categories, with an additional 19th category added for urban land cover. The categories are included for crop/mixed farming, short grass, evergreen needleleaf tree, deciduous needleleaf tree, deciduous broadleaf tree, evergreen broadleaf tree, tall grass, desert, tundra, irrigated crop, semidesert, ice cap/glacier, bog or marsh, inland water, ocean, evergreen shrub, deciduous shrub, mixed woodland, and urban. A 2 arc-minute (4 km) land use dataset covering the United States was formed from a 1-km BATS dataset and AVHRR data. A 30 arc-minute (55 km) global dataset was formed from the Olson (1992) World Ecosystems dataset on the Global Ecosystems Database CD-ROM. Also a 30 arc-second (1 km) global dataset was formed from USGS.

OMEGA uses a global sea surface temperature climatological dataset produced by the USGS (Schweitzer 1993). The data have been processed into a biweekly global, 12 arc-minute resolution (about 20 km) dataset. This data does not extend beyond about 70° latitude in either hemisphere. Some areas are not covered resulting in large lakes without data; for example the Caspian Sea in central Asia is completely missed.

A monthly global subsurface temperature climatology was built from monthly average surface air temperatures from the Global Ecosystems Database CD-ROM (Legates and Willmott 1992). These data were built by using monthly average surface air temperature as an estimate of subsurface temperature. The dataset resolution is 30 arc-minutes. Finally, Budyko’s simple soil moisture budget as described by Sellers (1965) was used to generate soil moisture estimates from the temperature and precipitation data with a 30 arc-minute resolution (Legates and Willmott 1992). This simple approach is far from perfect, but seems to give reasonable results for a range of climatic types. An iterative application of the Budyko equations was used to obtain volumetric soil water contents that balanced the annual water cycle.

With the exception of the land–water and terrain elevation databases each of the other surface characteristics databases are organized as sets of latitude–longitude tiles. The global-scale databases use 30° × 30° tiles, while the U.S. databases consist of 5° × 5° tiles. FORTRAN logic was devised to allow databases at both resolutions (U.S. and global coverages) to be used simultaneously. The higher-resolution data are applied to the OMEGA grid first, then the coarser-resolution data are used in any grid cells that have not been filled with values. After all of the available databases have been searched, the database values that have accumulated in each OMEGA grid cell are averaged to obtain the final value. In the case of the discrete fields (land–water, soil type and land use), the dominant (most frequent) value in each cell is considered to represent the entire cell.

b. Automation

Another key advantage of OMEGA is its integrated Graphical User Interface, XOMEGA (Fig. 15). XOMEGA provides a user-friendly method for the rapid reconfiguration of the model. Starting from a browse map of the world, an operator can use XOMEGA to set up and start a simulation anywhere in the world in a matter of minutes. XOMEGA simplifies the process of defining the computational domain, generating a grid with appropriate resolutions, acquiring the meteorological data files, building the initial and boundary conditions for the OMEGA model, specifying the source characteristics for the ADM model if there are any, running the OMEGA model, and finally analyzing the OMEGA model results.

In addition to XOMEGA, an X-windows graphics postprocessing and analysis tool has been developed for the OMEGA model in order to analyze the OMEGA model results. XGRID brings a wealth of functionality to the OMEGA modeling system and includes the ability to overlay OMEGA output on mapping data from the Digital Chart of the World. XGRID also permits the examination of vertical cross sections of the OMEGA output, and the creation of standard skew T–logp diagrams. Using XGRID, the user can examine both meteorological and ADM results. A scripting capability makes it easy to create routine products automatically, creating either Postscript or GIF files, which can be spooled into a printer or incorporated into other products, or MPEG animations.

c. Runtime issues

The runtime requirement for the operational models is the primary issue driving the development. The OMEGA model is CPU bound. Normally less than 1% of the CPU is used for input/output and memory management. OMEGA is dominated by vector operations, which are inherently faster than scalar operations and hence good execution speeds are being achieved. OMEGA also showed a computational intensity ratio of flops to memory references of 0.99.

The optimization level used in the OMEGA model maximizes inlining and vectorization. Inlining is a technique to reduce CPU associated with subroutine/function calls. It can cost up to 150 clock cycles to call a single argument subprogram. Vectorization allows a single operation to be performed on vectors of operands instead of sequentially for each operand. With the optimization level used in the OMEGA model, the overall computational speed achieved is slightly over 40 Mflops.

Parallelization is another state-of-the-art technique to boost computational speedup by multiprocessing. However, unstructured grid models tend to demand significantly more synchronization and geometry mapping overhead in their parallelization algorithm. A version of OMEGA with explicit parallelization using the Message Passing Interface (MPI) library is undergoing testing on parallel workstation platforms. The functionality of MPI’s allows (a) parallelization in an overarching manner in the OMEGA model time advancement loop, making message passing minimal; (b) pinpointing solutions to the idle time, or the imbalanced load problem, such as techniques accounting for the time-masked subcycling and dynamic domain-decomposition; and (c) transportability.

Parallelization by domain decomposition of the model domain into contiguous subdomains is currently under development and testing. Figure 16 shows an example of the technique applied to a typical OMEGA grid. The figure shows how the solution area is decomposed into 16 subdomains resulting in roughly the same number of cells in each of the subdomains. On top of the communication overheads, stacking and unstacking arrays for lumped message passing is an inherent overhead for unstructured grid models. These extra CPU costs are both related to message passing; however, by far the majority of the total overhead is due to uneven load balancing among the processors. For application on massively parallelized machines the scheme is designed to utilize over 100 processors. MPI command calls facilitate the parallelization of the time advancement loop of the model.

The parallel efficiency of OMEGA is roughly 95%. Amdahl’s law (1967) states that the speedup achieved by using N processors of a numerical algorithm that is f_p percent parallelized is

View Expanded

Figure 17a shows a large grid that was constructed for testing the parallel version of OMEGA. This grid has 13 935 cells in each of 31 levels. The grid resolution ranged from 5 to 50 km. Figure 17b shows the performance of OMEGA for 1–30 processors. Also shown are the theoretical ideal performance curves for 95% and 99% parallelization.

d. Portability issues

OMEGA produces a variety of output files (both binary and ASCII) depending on the mode of operation. The frequency of output is controlled by the user and is generally set for hourly. In order to allow OMEGA output from one system to be read and/or visualized on different hardware, we have developed a machine-independent, packed binary output data format for OMEGA.

The packed binary format is a form of compression that results in files that are written and read on the byte level by routines written in the C programming language. These files are now completely portable and may be cross ported to any machine, including PCs. There is a small penalty in runtime, which is more than compensated for by savings in postprocessing. The packed binary format files are also smaller and more efficient to read. In general, data are written using eight bytes to represent one number. With packed binary, this same number may be represented by as little as one byte. Although accuracy diminishes as fewer bytes are used, for most purposes, we have found that the use of two bytes, which results in accuracy to five significant digits, is sufficient and requires one-quarter the amount of disk space used generally.