1. Introduction

The new generation of atmospheric spectral radiance sensors has much higher radiometric sensitivity and spatial and spectral resolution than previous sensors and has a much larger number of channels. These enhancements and the growth in the number of high-resolution sensors place increasing demands on the computational aspects of remote sensing. Atmospheric radiative transfer (RT) models are an integral part of the modeling and simulation of scene radiances for use in sensor design and simulation. They are also used for extracting geophysical information from the radiometric data as part of the training for statistical inversion approaches, within physical inversion algorithms, and within direct assimilation of measured radiances into atmospheric numerical models.

This paper describes a fast and accurate radiative transfer modeling technique, optimal spectral sampling (OSS). The OSS approach was designed specifically to address the need for highly accurate real-time monochromatic radiative transfer calculations (including the Jacobians) for any class of multispectral, hyperspectral, or ultraspectral sensor at any spectral region from the microwave through the ultraviolet (Moncet 2000; Moncet et al. 2001a). Validation capabilities and multisensor data assimilation are enhanced with across-the-spectrum capability, allowing the same radiative transfer model to be used with multiple sensors to ensure spectral consistency.

The key problem in designing an accurate, yet computationally efficient, radiative transfer model resides in the way the monochromatic path transmittance for a particular viewing geometry is integrated over the spectral interval of the measurement:

View Expanded

where ϕ(ν) is the response function of the measurement instrument and T_l(ν) represents the transmittance at wavenumber ν for a path from the top of the atmosphere (pressure, p = 0) to the bottom of atmospheric layer l; T_l(ν) relates to the path optical depth τ_l(ν) as

View Expanded

View Expanded

where k_m is the absorption coefficient for gas m, θ is the temperature, q is the specific density of the molecule within the layer, μ is the secant of the viewing zenith angle, and g is the gravitational acceleration.

A number of approaches have been developed to address this problem, with varying degrees of success. One widely used class of technique (McMillin and Fleming 1976, 1977; McMillin et al. 1979; Eyre and Woolf 1988; Eyre 1991; Strow et al. 2003; Matricardi 2003) parameterizes the “effective optical depth” in each atmospheric layer as a function of atmospheric pressure, temperature, and gas concentration along the measurement path as a sum of H terms:

View Expanded

where T^ref represents an exact transmittance profile corresponding to a reference atmosphere and the X_i are functions of layer and path-integrated (pressure weighted) temperature and gas concentrations. The weights a_li are determined by least squares regression to minimize the modeling error for an ensemble of globally representative atmospheric profiles (i.e., the “training” set). Here, we use the term “total path transmittance regression” (TPTR) to denote this technique.

One advantage of the TPTR technique is that it requires the evaluation of only a single exponential to compute the transmittance for each atmospheric level and molecular constituent, thereby minimizing the total number of calculations. While there is a computationally efficient analytic solution to the calculation of the radiometric Jacobians, the overall efficiency of this calculation is affected by the dependence of the optical depths on profile variables above the layer under consideration (Strow et al. 2003). Furthermore, the overall radiometric accuracy of the method can be difficult to control because it depends largely on the choice of predictors (X) that are selected on an empirical basis and lack a direct physical basis. A set of X developed for a downlooking sensor may be inadequate for an uplooking or limb-viewing sensor. Similarly, this method is not practical for use with airborne sensors because the coefficients (a) are obtained from a least squares fit to the path properties and a different set is required for each altitude of the sensor. Additional constraints arise from the fact that the formulation is not directly adaptable to nonpositive instrument line shape (ILS) functions, such as sinc functions produced by (unapodized) Fourier transforms of interferograms sampled over a limited range of optical path difference (McMillin et al. 1998; Barnet et al. 2000).

A basic limitation of these TPTR methods is that they cannot directly and accurately handle surface reflectance of downwelling radiation. The downwelling radiation R^↓(ν) reflected at the surface contributes a component of the top-of-atmosphere (TOA) radiance for a channel:

View Expanded

where r is the surface reflectivity and T_sfc is the transmittance to the surface from the top of the atmosphere. For this discussion, we treat the surface reflection as specular. The basic concept of band models makes no provision for the treatment of spectral correlations between the downwelling radiance at the surface, the atmospheric transmittance, and the reflectance within the channel bandwidth; that is, (4) is treated as

View Expanded

where the overbar denotes the band average of the individual variable with weighting by ϕ(ν). Inherently, there is a negative spectral correlation between R^↓(ν) and T_sfc(ν), and this approximation leads to substantial errors in TOA radiances over highly reflective surfaces, even for the idealized case where r is constant over the band (Fig. 1). The magnitude of these errors depends on atmospheric opacity and viewing angle. These models thus need a separate instrument-dependent correction to mitigate the effect.

Similarly, this TPTR parameterization has inherent accuracy limits when applied to cloudy atmospheres with moderate cloud optical depths because it has no means to respond to the cloud-induced changes in the atmospheric spectrum on subchannel spectral scales, from line centers to line wings.

OSS is fundamentally a monochromatic approach, and that property alleviates many of the limitations listed above and makes the model generic and flexible. OSS can accurately treat surface reflectance and spectral variations of the Planck function and surface emissivity within the channel passband, given that the proper training is applied. In addition, the method can maintain its accuracy when applied to multiple scattering calculations—an important factor for treating cloudy radiances—and is directly applicable to nonpositive ILS. One advantage of OSS over existing methods is that its numerical accuracy is selectable such that it is capable of fitting reference transmittance calculations arbitrarily closely.

The OSS method was initially developed for supporting sensor system design trade-off studies and performance evaluations, such as those conducted in the context of the Cross-Track Infrared Sounder (CrIS) and the Conically Scanning Microwave Imager/Sounder (CMIS) payloads of the National Polar-Orbiting Operational Environmental Satellite System (NPOESS) and the Hyperspectral Environmental Suite for the Geostationary Operational Environmental Satellite (GOES) system. OSS is well suited to system development applications because, with only minor configuration changes, it can be tailored to a wide range of remote sensing problems: downlooking (satellite sensors), uplooking (ground-based sensors), and aircraft or balloon (variable viewing and altitude ranges), including limb and line-of-sight measurements. OSS was furthermore adopted for use within the baseline operational retrieval algorithms for CrIS and CMIS (Moncet et al. 2001a, b). OSS is used also by NASA (Liu et al. 2003) in their analysis of the aircraft sensor data such as that obtained by the NPOESS Airborne Sounding Testbed Interferometer.

This paper focuses on application to infrared remote sensing. The scope is limited to the theory and practical implementation of methods for modeling transmittances and radiances over measurement bands. Issues related to selection of a computational pressure grid and subgrid layer parameterizations (e.g., Matricardi 2003; Clough et al. 1992) are not addressed here because the OSS method is general with respect to these issues.

2. The OSS method

The band-averaged transmittance in a homogenous (constant pressure, temperature, and composition) layer of the atmosphere with a single absorber can be expressed as

View Expanded

where k is the spectral absorption coefficient, u denotes the absorber amount (mass per unit area), and Δν typically encompasses at least several absorption lines. A discrete approximation to (6), called exponential sum fitting of transmissions (ESFT), is

View Expanded

Wiscombe and Evans (1977) present a method to optimally select the k_i, using a non-linear fitting formulation, and the weighting factors w_i. Similar results can be obtained with the k-distribution approach, wherein the w_i and k_i are obtained from an optimized trapezoidal quadrature over the cumulative distribution function of k within the interval Δν (Goody and Yung 1989; Thomas and Stamnes 1999).

The main difficulties encountered in attempting to apply the ESFT technique to vertically inhomogeneous atmospheres directly (i.e., without invoking effective path properties) is in maintaining consistency among the different layers with respect to the choice of the representative k values. Most attempts to extend the application of the ESFT to inhomogeneous atmospheres (e.g., Lacis and Oinas 1991; Goody et al. 1989) rely on the assumption that the absorption coefficients in different altitude regimes are perfectly correlated (the correlated-k approach), an assumption that breaks down, for a single gas, in the presence of lines with different strengths or for highly temperature-dependent line strengths (e.g., West et al. 1990) and, for gas mixtures, when the relative concentration of the absorbers in the mixture differs in the two altitude regimes. Although an efficient mapping algorithm was developed by West et al. (1990) to reduce the errors due to the above assumption in the context of a single gas, none of the proposed approaches deals explicitly with gas mixtures. The problem can usually be avoided either by treating gas mixtures as a single gas (e.g., Mlawer et al. 1997; Sun and Rikus 1999), which may lead to significant errors when the composition of the mixture changes along the path, or by making use of the multiplication property for band transmittances (e.g., Armbruster and Fischer 1996), which is only valid in a few specific regimes. The impact of these approximations on the accuracy of the radiative transfer calculations is, however, limited by the fact that in most problems the radiation originates from a restricted altitude regime with quasi-homogenous thermodynamic properties, making it possible to tune the model parameters for that regime and for the interval considered. A rigorous solution to the inhomogeneous problem requires a full characterization of the multivariate probability distribution, f (k₁₁, k₁₂, . . . , k_1m; k₂₁, . . . , k_lm), of the absorption coefficients in all layers l and for all active constituents m. Simultaneously solving for all the representative k values in this context may not be feasible.

The OSS method avoids these problems. Rather than directly selecting k_i, we explicitly select spectral points ν_i (as described in section 3) from which a spectroscopic model can provide the corresponding k(ν_i):

View Expanded

This minor distinction from (7) becomes critical in the extension from the single homogeneous layer to the heterogeneous case of multiple layers and mixtures of gases. With no further approximation and no new assumptions, we can extend (8) to this case:

View Expanded

where subscripts l and m refer to layers and molecular species, respectively. In (9), each i refers to a monochromatic calculation at ν_i (an OSS “node”) over all layers and gases. In this approach, the characterization of f (k₁₁, k₁₂, . . . , k_1m; k₂₁, . . . , k_lm) is implicit. The absorption coefficients k_lm(ν_i) can be derived directly from a line-by-line model (or a tabulated version) with no further assumptions about the dependence of k on pressure, temperature, and gas mixtures, as enter in the correlated-k approach. This OSS formulation applies also to the more general case in which we approximate the transmittance in a sensor channel with response function ϕ:

View Expanded

For most sounding applications, it is advantageous to determine the OSS parameters (nodes and weights) by directly fitting radiances rather than transmittances. The OSS approximation for the average radiance in a channel is expressed as

View Expanded

where R is the channel-averaged radiance and R(ν_i) are the monochromatic radiances computed at the selected OSS nodes. This approach has the advantages that (i) it automatically emphasizes atmospheric levels located near the peak of the weighting function (which contribute the most to the outgoing radiances) in the optimization process and (ii) it provides a mechanism for taking into account smoothly varying functions such as the Planck function, cloud optical properties, or surface emissivity in the determination of the model parameters.

One distinct advantage of OSS is that the error tolerance for fitting (11) is selected a priori by the user, even in the multilayer case. This feature allows the user flexibility to tailor the fitting to balance the radiometric accuracy requirements dictated by the application and the computation time constraints. A tighter threshold will provide a fit with a larger number of nodes, at the cost of more calculations.

3. Search procedure

With the so-called “localized” training described in this paper, we train an OSS model for a given high-spectral-resolution sensor by considering one channel at a time. We defer detailed discussion of an alternative “global” training method, which gains computational economy by treating the channel set as a whole and which is more applicable to broad spectral domains.

The OSS node selection for a particular sensor channel starts from M uniformly spaced spectral locations spanning the channel bandwidth. These locations constitute a set ℑ_M. A line-by-line model is used to compute the monochromatic radiances at those spectral locations for a globally representative ensemble of atmospheric profiles and a range of surface conditions and viewing geometries (section 4). Channel radiances, R_s, for each member s of the set of S training scenes are produced by spectrally integrating the monochromatic output of the line-by-line model over the channel response function. For any given subset of ℑ_M that contains N nodes (denoted ℑ_N), we can compute the rms difference between R_s and the weighted sum of monochromatic radiances R_s(ν_i) associated with the N selected nodes, computed over the ensemble of training profiles:

View Expanded

The optimal weights w′ = w₁, . . . , w_N−1 for the first N − 1 nodes are obtained by solving an overdetermined linear system of the form

View Expanded

using a robust least squares regression technique, where 𝗔 = 𝗬𝗬^T, b = 𝗬y, 𝗬 is a matrix of dimension (N − 1) × S, whose columns contain the radiance differences [R_s(ν_i) − R_s(ν_N); i = 1, . . . , N − 1], and y is the vector of S reference channel radiance differences [R_s − R_s(ν_N)]. Then w_N is derived from

View Expanded

This approach is designed to ensure that the sum of the weights is exactly equal to unity, which is not inherently necessary, but which contributes to robustness of the solution. For the trivial case N = 1, w_N = 1.

We use an automated search to identify the smallest subset ℑ_N for which the rms difference is less than a prescribed tolerance: that is, ε_N < ε_tol. Two algorithms have been implemented for selecting the OSS nodes. The first approach is based on the search method Wiscombe and Evans (1977) applied in the context of ESFT. This approach (subsequently referred to as the W/E approach) starts with N = 1 and performs an exhaustive search among all possible one-member subsets ℑ₁ for the spectral location that produces the lowest ε₁. Once the first node has been determined, ε₁ is compared to ε_tol. If ε₁ is less than the prescribed tolerance, the procedure stops. Otherwise, N is incremented by one and the algorithm proceeds, as above, by successively adding all the elements of ℑ_M − ℑ_N−1 to ℑ_N−1 to form M − (N − 1) new ensembles ℑ_N, and by finding, among all these candidate combinations, the one that produces the lowest ε_N. The weights w_i are recomputed for each trial combination. This incremental process is repeated until the solution satisfies ε_N < ε_tol. When we use training profile sets that are stratified by view angle (section 4), we continue until ε_N < ε_tol is satisfied at each of the angles. This approach ensures that the OSS model performance is uniform across the range of air mass paths. The impact of this strategy is most apparent at large incidence angles (60° and above).

With the W/E method, the addition of a new node does not modify the previously selected nodes unless the vector w at a given step (for the selection that produced the lowest ε_N) contains one or more negative weights, indicating that this solution is of dubious robustness. In the context of positively defined instrument functions, all weights are required to be positive (or no more than slightly negative with an increasing tolerance for negativity as N grows). Large negative coefficients arise when a newly added node carries redundant spectral information, in which case matrix 𝗔 becomes ill conditioned. Such situations arise most frequently when N becomes large (N > ∼10) and the solution path has reached a local minimum. In this case, one of the elements of ℑ_N−1 associated with a negative weight is dropped and replaced by ν_N and the step is reinitialized. Wiscombe and Evans (1977) provide a full discussion of this so-called “term-dropping” procedure. We augment this procedure by monitoring the determinant of matrix 𝗔. The positive-w criterion cannot be used when dealing with nonpositive ILS such as interferometric sinc functions, for which we rely solely on the determinant check. It is useful, when term dropping has been done, to attempt to refine the W/E solution once convergence has been reached by applying the entire search procedure a second time while restricting the candidate nodes to the ℑ_N obtained in the first pass, rather than starting with the full set ℑ_M. This second pass is aimed at eliminating any redundant elements from the initial selection.

The second node selection approach uses a standard Monte Carlo (MC) search (Landau and Binder 2000). Like the W/E method, the MC method proceeds by successive increments of N. This method differs from the W/E method in the way ℑ_N is obtained for N > 1. At each step, the method builds an initial solution ℑ⁰_N by adding to ℑ_N−1 a new element selected at random from ℑ_M. A number of attempts are then made to modify this solution and produce a new solution ℑ^j_N, where j indicates the number of “successful” attempts (see below), by substituting one randomly chosen element of ℑ^j−1_N with another randomly chosen element of ℑ_M − ℑ^j−1_N. This process stops when the number of successful trials has reached J, which is currently set to

View Expanded

If, after the series of substitutions is completed, at least one of the trial solutions satisfies our convergence criterion ε_N < ε_tol, then the solution providing the best performance is selected as the ultimate solution and the procedure stops. Otherwise, ℑ_N is set equal to the result of the last substitution, ℑ^J_N, and N is incremented.

In the substitution process, a new trial solution ℑ^j_N is accepted with a probability:

View Expanded

where ε^j_N denotes the rms error associated with ℑ^j_N. Practically, the acceptance criterion is z < P, where z is a random number from a uniform distribution over [0, 1]. This strategy is based on the heat-bath algorithm (e.g., Binder and Heermann 1997). In the above equation, β is an adjustable parameter used to control the convergence of the search procedure. If β is large, then P ≈ 1/2 and a substitution is accepted or rejected with an equal probability. In this case, the method is equivalent to making a large number of independent attempts to find the global minimum by randomly exploring the entire ensemble of combinations. In the other limit, when β = 0, the procedure systematically accepts the substitution when ε^j_N < ε^j−1_N. In this configuration, the MC search is analogous to the W/E procedure and always looks for minimizing the error at each intermediate step. In practice, we found that the MC method combines the best attributes of the W/E and random selection when the number of rejected (J_rej) and accepted (J_acc) solutions differ approximately by a factor of 2. Under such conditions, the method preserves the ability to reduce the rms error at each trial, as in the W/E approach, and at the same time the ability to escape from local minima and find a path toward the global minimum. Note that the proper value of β cannot be determined in advance. In practice, the proper setting is found by providing an initial value for β based on β = (ε⁰_N/2.5) at the beginning of each step and by monitoring J_rej and J_acc. If J_rej/J_acc goes beyond the range from 1.5 to 2.5, then β is increased or decreased (which forces the algorithm to accept a larger or smaller number of combinations with ε^j_N > ε^j−1_N) and the procedure continues. Constraints on the sign of the weights or determinant of 𝗔 (see above) are included in the framework of the MC approach by ensuring that solutions that do not satisfy our quality acceptance criteria are systematically rejected.

A comparison of the performances of the W/E and MC approaches is provided in Fig. 2. The W/E method has the advantage that it is faster than the MC method, with the number of trial combinations on the order of 2MN and 10MN, respectively. On the other hand, failure to converge toward the optimal solution is not uncommon with the W/E method when N becomes large. The performance of the MC method, for the test cases considered (e.g., Fig. 2), equates that of a global search. The efficiency of the MC approach in relation to a global search is explained by the fact that, for a given N, there are many combinations of nodes that provide equally good fits.

The search procedure (with either approach) becomes time consuming when M exceeds a few thousand (i.e., for Δν ≳ 1 cm⁻¹ in the infrared). In such cases, it is possible to divide Δν into smaller subintervals and apply the OSS selection to each subinterval. The width of the subintervals is then progressively increased until it becomes comparable to or even larger than the width of the instrument function’s primary response mode. At each increase in the size of the subintervals, a new search is applied using the nodes selected in the previous step as the initial set from which to select in the current step. It is only in the final selection step that the desired sensor instrument function is applied. The algorithm backtracks to smaller subintervals if necessary to meet the requirement for error <ε_tol. When dealing with instruments whose channels extensively overlap, the use of large preliminary subintervals tends to encourage sharing of nodes between neighboring channels, thus reducing the total number of nodes needed for the channel set as a whole. The preliminary node selections can furthermore be saved and reused with different sensors.

4. Training dataset

Atmospheric sets commonly used in remote sensing applications for fast-model training and validation include the diverse 52-profile set developed from the European Centre for Medium-Range Weather Forecasts (ECMWF) global model (http://www.metoffice.gov.uk/research/interproj/nwpsaf/rtm/) and the 48-profile set developed at the University of Maryland, Baltimore County (UMBC) (http://cimss.ssec.wisc.edu/itwg/groups/rtwg/rtwg.html). The atmospheric data are specified on the standard 101-level UMBC grid (Strow et al. 2003), the same grid used for the RT calculations in the current OSS model.

The starting point for building our training dataset was the 52-profile ECMWF set. This set was replicated four times to form a total of five subsets, each of which was assigned a different view angle (or zenith angle) value ranging from 0° and 60° (specifically, 0.00°, 36.87°, 48.19°, 55.15°, and 60.00°). The range of scan angle used for training is not an inherent property of OSS but was tailored here for application to operational cross-track scanning instruments. The total number of scenes in the final training set is 260. Each profile in the set was assigned a random surface emissivity (between 0.7 and 1.0) and a random surface pressure. Water vapor and ozone are the only variable molecules considered in the present study. In recent applications, the OSS training has been expanded to include variability of other trace gases, including synoptic, seasonal, and secular trends associated with climate change.

To ensure robustness of the OSS RT model, the atmospheric dataset used for training must encompass the full range of atmospheric conditions encountered in nature and include realistic vertical structure. Because adequate measurements of temperature and molecular concentrations in the upper atmosphere are lacking, profiles are generally extrapolated from the lower part of the atmosphere, thereby introducing artificial correlations between the lower and upper atmosphere as well as profile-to-profile correlations. In addition, profiles often tend to be smoother than in nature, either because of the vertical coarseness of the radiosonde records or because of the various physical and numerical constraints used in weather or chemistry models. Exploitation of these artificial vertical correlations, if left uncorrected, is inherent to the OSS training process (and other fast RT methods) and results in a reduction in the number of nodes needed to fit the reference radiances within required accuracy threshold. While the reduced node set is sufficient to fit the training data, it is not robust when applying OSS to real atmospheres (or different sources of atmospheric data), so errors may be larger than those obtained in the training.

This shortcoming of profile datasets was remedied for OSS by adding random perturbations to the temperature and variable constituent concentration profiles (with the exception of water vapor) at some appropriately selected pressure levels. This random component (ζ) was vertically interpolated to the 101-level model grid. Additional random perturbations were applied at each one of the 101 pressure levels used by the current model. This second random component is denoted by ξ. This process has the effect of reducing the interlevel correlations in the temperature and molecular concentration profiles. The resulting value of profile variable x at level l is expressed as

View Expanded

where I denotes the vertical interpolation function used with the first random component and A and B are the magnitudes of the random components ζ and ξ, respectively. The pressure levels at which ζ is specified and the perturbation amplitude factors A and B depend on x. These parameters were determined empirically by ensuring that the performance of the OSS model was uniform across all the independent datasets available for this study [i.e., UMBC, thermodynamic initial guess retrieval (TIGR), and NOAA-88; Chedin et al. 1985; Seemann et al. 2003]. Each of the 260 profiles has a different random realization. The set of profiles modified according to (17) is subsequently referred to as the “noisy” ECMWF set.

5. OSS forward model

b. Optical depth calculations

In the OSS model, the total absorption optical depth in a layer is obtained as the sum of the contributions of the fixed gases (which are treated as a single constituent), water vapor, and the dry variable constituents:

View Expanded

The superscript 0 is used to indicate that these terms are computed along the vertical path, and subscripts “fix” and “var” respectively refer to the fixed and variable dry gas groups. The layer optical depth for an individual molecule m from any of these three groups is obtained as

View Expanded

where k_m is the absorption coefficient of molecule m, derived from lookup tables, and u⁰_m is the molecular amount (in units of mass per unit area). To obtain u⁰_m for a layer, we typically assume that the trend between adjacent pressure grid levels is logarithmic in pressure and mixing ratio.

For dry gases, the infrared OSS model considers only the pressure and temperature dependences of the absorption coefficient. This treatment does not provide an explicit treatment of the self-broadening in the computation of the line width and (consistent with our reference LBLRTM line-by-line model) makes no provision for the impact of the different collision efficiency of the water molecules (for which data are not available from the HITRAN spectroscopic database). In the infrared model, water vapor affects the optical depth calculations of the dry gases only through its impact on the molecular amounts.

Molecular amounts for the molecules in the fixed gas group are derived using

View Expanded

where f_m is the layer mixing ratio of molecule m, assumed invariant with time and location, and u⁰_dry is the dry air amount. The latter must be recomputed for each profile to account for its dependence on water amount in a layer at constant pressure.

Using (20), the optical depth of the combined fixed gases is expressed as

View Expanded

where f_fix is the layer total mass mixing ratio of the fixed gases and k_fix represents the effective absorption coefficient for the fixed gas mixture: f_fix is computed offline, based on the U.S. Standard Atmosphere (Anderson et al. 1986), and stored in the absorption coefficient file header. This header also contains f_m for all the variable dry gases.

Water vapor treatment differs from that of the other gases in that its strong self-broadening component must be treated explicitly. The water vapor absorption coefficient (continuum + lines) in the infrared is assumed to vary linearly with layer-specific humidity¹ q_w:

View Expanded

where k*_w and δk_w are the zero offset and slope of a linear fit. This assumption ignores the nonlinear dependence of the absorption on the line width and may lead to nonnegligible errors in monochromatic calculations in the near wing and center of water vapor lines. These errors are damped in the process of convolving the monochromatic results with the instrument function and quickly average out as the width of the channel increases beyond a few line half-widths.

The coefficients k̃_fix, k_m∈var, k*_w, and δk_w are currently stored for each RT model layer at ten temperatures roughly spanning the expected range of air temperatures in each altitude range. The coefficient values for a layer at temperature θ in the interval [(θ_i−1 + θ_i)/2, (θ_i + θ_i+1)/2] are obtained using a three-point Lagrange interpolation scheme,

View Expanded

With this temperature grid, interpolation errors do not exceed 0.05 K (rms < 0.02 K). For temperatures outside the range of the lookup table, these values are obtained by extrapolation from the nearest table entries.

c. Radiance and Jacobian calculations in the clear sky

The discretized form of the radiative transfer equation for nonscattering atmospheres used in OSS to compute monochromatic clear-sky TOA radiances in the infrared is given by

View Expanded

where B_l represents the effective Planck function in the layer, ε_sfc and r^sol_sfc correspond respectively to the surface emissivity and solar bidirectional reflectance, F₀ is the extraterrestrial solar irradiance, and the superscript “sol” refers to the solar angle. According to our convention, the first layer (l = 1) corresponds to the topmost layer and the Lth layer is located near the surface. The terms T_l and T ′_l respectively represent the transmittances from space to level l and from the surface to level l, which are computed using the layer optical depths. In (24), as well as in all subsequent equations, it is implicit that we are dealing with a single node and the wavenumber index has been dropped for the sake of simplicity.

Differentiating the thermal part of Eq. (24) with respect to absorber amounts and temperature, leads to the following expression for the derivatives of the TOA thermal radiances with respect to those variables:

View Expanded

View Expanded

where

View Expanded

View Expanded

and where Σ⁻_l−1 represents the contribution to the TOA radiance of the downwelling thermal emission originating from the layers above level l and Σ⁺_l represents the contribution of the upwelling emission (including the reflected thermal radiation) at level l; that is,

View Expanded

View Expanded

Here n denotes a computational level. The computation of the solar part is trivial and will not be discussed here.

One advantage of dealing with monochromatic transmittances [instead of average channel transmittances, as in Eq. (3)], is that, in this case, perturbing the temperature or composition of a layer alters the optical properties of only that layer [e.g., Eqs. (25) and (26)]. This property greatly simplifies the computation of the Jacobians. The following procedure (which is also applicable to limb viewing geometry) is used in OSS for simultaneously computing radiances and Jacobians. In a first pass, layers are merged in the downward direction from the top of the atmosphere to the surface to compute Σ⁻_L and the terms D⁻_l (for l = 1, L). With each addition of a new layer l, the procedure first computes and stores D⁻_l and then updates Σ⁻_l as

View Expanded

Once Σ⁻_L has been computed, it is scaled by the surface reflectivity and the surface emission term is added to form Σ⁺_L. This pass (including the computation of ground-to-level transmittances) is not executed if the atmospheric opacity is large. The second pass proceeds in similar fashion, by successively adding the layers from the surface upward in the direction of the observer to compute R = Σ⁺₀ and the terms D⁺_l (for l = 1, L). Finally, the derivatives with respect to temperature and molecular amounts for all variable molecules are obtained by applying (25) and (26). The derivatives of optical depth with respect to temperature and molecular amount are computed at the same time as the optical depths and input to the RT algorithm. For gases other than water vapor, ∂τ⁰_l/∂u⁰_lm is simply equal to the absorption coefficient k_lm, which is obtained at no computational cost as a by-product of the optical depth calculations. Derivatives of layer optical depth with respect to water vapor amount are computed as

View Expanded

which is approximated by

View Expanded

The last two terms in (32) and (33) account for the small dependence of the dry gases on water vapor concentration in a layer at constant pressure. Differentiating (22) leads to

View Expanded

6. Performance of the method

a. Dependencies

The number of nodes (a driver of computation time) depends on the specified radiometric accuracy of the model. For a given accuracy threshold, the number of nodes for a channel depends also, in a somewhat complex way, on the channel bandwidth, the range of the atmospheric opacity, and the number of active molecular species and the way their absorption overlaps both spectrally and vertically. To illustrate some of these dependencies, we have trained OSS models for an idealized spaceborne scanning radiometer measuring the TOA radiances in contiguous square impulse (“boxcar”) channel response functions of uniform width, covering the 650–850 cm⁻¹ domain.

Figure 3 shows the number of selected nodes for 5-cm⁻¹ boxcar widths and accuracy thresholds 0.05 and 0.01 K with carbon dioxide as the only absorber, with carbon dioxide and water vapor, and with carbon dioxide, water vapor, and ozone. When CO₂ alone is considered, the average number of nodes needed to describe the TOA radiances in the 5 cm⁻¹ wide intervals is largest in the 650–750 cm⁻¹ region (up to 10 and 18 for the 0.05-K and 0.01-K accuracy thresholds, respectively) and decreases sharply beyond 750 cm⁻¹. In the center of the CO₂ ν₂ band (667 cm⁻¹), the absorption spectrum is characterized by strong, well-separated lines—a structure that gives rise to a large range of atmospheric opacities within the individual 5 cm⁻¹ wide channels—and a large distribution of altitudes that contribute to each channel’s radiance (i.e., broad weighting functions). In this situation, a relatively large number of nodes is needed to model the impact on the channel radiances of temperature variations at all altitudes spanned by the weighting function. Other parts of the infrared spectrum where weighting functions are broad or double-peaking (as occurs when two or more gases absorb in distinct altitude regimes), hence showing a local maximum in number of nodes, include the 1042 cm⁻¹ O₃ band, the 1310 cm⁻¹ CH₄ band, and the CO₂ ν₃ band (2349 cm⁻¹). In contrast, spectral regions containing single-species absorption features concentrated in the troposphere, such as regions with only water vapor absorption (∼1600 cm⁻¹), require fewer nodes to meet a given radiometric threshold. An additional factor for water vapor is the strong continuum absorption, which reduces the transparency of the atmosphere between lines, tends to sharpen the weighting functions and makes the number of points used by the method relatively small in those regions. The variability in constituent concentration has only a moderate impact on the number of OSS nodes required to meet the accuracy threshold.

It is apparent from Fig. 3 that the addition of H₂O and O₃ causes a significant increase in the number of nodes selected for the model with 0.01-K accuracy. It is noteworthy that the increase in number of nodes due to O₃ in the 700–750 cm⁻¹ region is at least as pronounced as the increase due to H₂O beyond 700 cm⁻¹, despite the fact that the impact of variations in O₃ concentration on the TOA radiances is smaller than the impact of H₂O variations (typically less than 1.5 K compared to up to 15–20 K for H₂O, Fig. 4). The relatively large impact of O₃ on the number of nodes is attributed to a combination of two factors: (i) the greater complexity of the O₃ absorption spectrum, which results in a weaker correlation between O₃ and CO₂ absorption spectra and (ii) the span of the O₃ weighting functions.

The dependence of the number of nodes on the width of the channels is illustrated in Fig. 5. The dependence is most pronounced in the 700–750 cm⁻¹ and 750–800 cm⁻¹ intervals. In the 650–700 cm⁻¹ region, there is a marked increase in the number of nodes when going from 0.1 to 0.5 cm⁻¹ (i.e., up to scales comparable to the spacing between adjacent CO₂ lines), but the number of selected nodes reaches a plateau as resolution further degrades. The structure of the CO₂ absorption spectrum does not change significantly across the 650–700 cm⁻¹ spectral domain, which makes the probability distribution of the absorption/radiance in a channel relatively independent of the width of the instrument function. In the 700–750 cm⁻¹ and 750–800 cm⁻¹ regions, the rapid changes in the CO₂ absorption characteristics in the wing of the ν₂ band, combined with changes in the absorption characteristics of O₃ and H₂O, cause the number of nodes to keep increasing with the width of the channels beyond scales of tens of wavenumbers. For this reason, modeling wideband imaging radiometers in these regions requires more nodes per channel than for high-spectral-resolution sounders. By comparison, the computational requirements for TPTR schemes are independent of the channel bandwidth, so they become more competitive with OSS, with respect to speed, for wide channels, although the accuracy of these schemes is limited by their failure to take account of the Planck function and surface emissivity variations across the bandwidth.

For the sake of completeness, it is useful to compare the predicted computational performance of the OSS approach to that of the frequency sampling method, also known as the radiance sampling method (RSM) (Tjemkes and Schmetz 1997). The RSM treats the spectral radiances within a channel bandwidth as a continuous random process (e.g., Sneden et al. 1976; Peytremann 1974). The mean radiance for a channel is estimated by sampling realizations of this process at N fixed discrete spectral locations:

View Expanded

The relationship between the RSM and the k-distribution method is explained in Tjemkes and Schmetz (1997). Like OSS, the RSM method is a monochromatic approach and its computational performance is proportional to the number of samples used to estimate R. The main conceptual difference between the two approaches is that in OSS, each ν_i is associated with “unique” optical properties and OSS makes explicit use of weights to represent the frequency of occurrence of R(ν_i) within the channel bandwidth, whereas in the RSM approach, this weighting is contained in the number of spectral locations sharing statistically similar optical properties. The methods also have different strategies for choosing the ν_i. Figure 6 plots OSS and RSM modeling errors as a function of the average number of spectral samples used with both approaches, for 0.5 and 5 cm⁻¹ wide boxcar functions covering the range 740–760 cm⁻¹. Several sampling strategies may be adopted for RSM, but in practice a uniform spectral sampling strategy works best (e.g., Tjemkes and Schmetz 1997). For Fig. 6, the offset of the uniform spectral grid was optimized for each channel. It is apparent from this figure that the OSS weighting scheme reduces the number of spectral samples required for achieving a specified accuracy by one to three orders of magnitude. This example illustrates the effectiveness of the OSS weighting scheme in eliminating redundancies in the spectral radiances.

7. Conclusions

The OSS method provides a way to approximate channel-average radiances by generalizing the ESFT or k-distribution concept to vertically inhomogeneous atmospheres with multiple absorbers. With this method, one can achieve selectably high fidelity to a reference line-by-line model with orders of magnitude reduction in computational cost.

In the present paper, we focused on the application of the OSS method to infrared sensors in the thermal regime. We demonstrated that only a few monochromatic RT operations, on average, are required to achieve accuracies on the order of a few hundredths of a kelvin for the AIRS high-spectral-resolution sensor. This efficiency, when combined with the efficiency with which our RT model produces the Jacobians required for the radiance inversion, makes the method very well suited to remote sensing applications. Several characteristics of the OSS method that distinguish it from other methods are summarized in Table 3.

Some of the characteristics in Table 3 are particularly relevant for atmospheric data assimilation systems. Algorithm speed and memory usage are often major concerns, and trade-offs against radiometric accuracy may be beneficial. The introduction of new sensor data sources into an assimilation system may be facilitated by the robustness and simplicity of the OSS method, accommodating new spectral and geometric properties without any need to develop new empirical formulations or to validate empirical coefficients. The speed with which the necessary coefficients can be generated is an additional asset of OSS.

The localized training described in this paper minimizes the number of nodes required to reconstruct radiances in each single (isolated) channel. While this criterion provides very good performance for AIRS and near-optimal performance for wideband imaging instruments such as MODIS, modification can be made to the search method to significantly benefit applications to high-spectral-resolution sounders. When ILS from different channels overlap extensively (such as sinc functions), the localized training does not explicitly enforce a maximum reuse of nodes among the channels. Even when the ILSs do not strongly overlap (like AIRS), some further reduction of node requirements can be obtained by treating the channel set as a whole and recognizing that nodes in one spectral region can be used to model radiances for channels in distant spectral locations with similar absorption properties. An alternative global OSS training approach selects nodes for the channel set as a whole, while employing node clustering to efficiently account for spectral correlations, and thus condenses the information of the full channel set into a minimal number of nodes. This alternative will be presented as a follow-up to the current paper.

Recently, the OSS method has been extended to deal accurately and efficiently with cloudy atmospheres in which scattering processes are important. We have also extended the OSS training to include all variable molecular species that are relevant to infrared sounding of the earth’s atmosphere. These developments will be reported separately. Improvements in the handling of the solar term and additional validation work in the near-infrared and visible domains, as well as modeling of nonlocal thermodynamic equilibrium effects in the near-infrared and microwave domain, are left for future work.

Acknowledgments. The work reported here was supported in part by the Integrated Program Office of the National Polar-Orbiting Operational Environmental Satellite System and by the Joint Center for Satellite Data Assimilation through NOAA award NA04NES4400005. Additional support was internal AER funding. We thank Xu Liu, who participated in the testing and validation of versions of the NPOESS/CrIS RT model and of alternate methods for extension of the OSS training process to nonpositive ILS, and Ryan Aschbrenner, who assisted with some computations. We also thank S. Hannon and L. Strow for access to their UMBC training dataset and R. Saunders and M. Matricardi for the profile dataset derived from ECMWF profiles produced by F. Chevallier.