1. Introduction
Radiative transfer (RT) processes not only impact the Earth–atmosphere energy balance but also provide key information relevant to atmospheric remote sensing (Goody and Yung 1989; Liou 2002). Therefore, the transfer of radiation in the atmosphere is an essential problem in atmospheric modeling and measurements (Chandrasekhar 1960; Goody and Yung 1989; Liou 2002), and significant effort has been devoted to solving the RT equation under different circumstances and with differing degrees of accuracy. Since molecular absorption of atmospheric gases has significant spectral variation (absorption coefficients can change by a few orders of magnitude within a spectral region of 0.01 cm−1 or less), high spectral resolution (HSR) is normally required for accurate RT calculations. This implies that tens of thousands, if not more, of accurate RT simulations have to be performed for even a narrow spectral region. The resolution of monochromatic RT calculations is basically controlled by the width of the absorption line—stratospheric Doppler broadened lines are typically much narrower than tropospheric pressure-broadened lines. However, even for relatively broad absorption lines, scattering due to molecules, aerosols, and clouds makes the simulations complex and time consuming. The above considerations make it impractical to employ HSR simulations for applications such as satellite retrievals or weather and climate modeling.
On the other hand, HSR measurements can provide unique information about trace gases and other atmospheric constituents (e.g., Crevoisier et al. 2004; O’Dell et al. 2012). Such HSR instruments are on both polar orbiting and geostationary satellites. Examples include the Atmospheric Infrared Sounder (Aumann et al. 2003), the Infrared Atmospheric Sounding Interferometer (Hilton et al. 2012), the Cross-track Infrared Sounder (Han et al. 2013), the Visible Infrared Imaging Radiometer Suite (Yang et al. 2017), Orbiting Carbon Observatory-2 (Eldering et al. 2017) and TANSAT (Liu et al. 2013). Ground-based HSR instruments have also been developed for gas and aerosol retrievals [e.g., Total Carbon Column Observing Network (Wunch et al. 2011) and California Laboratory for Atmospheric Remote Sensing (Fu et al. 2014)]. Analysis of these HSR observations and development of retrieval algorithms require accurate and computationally efficient forward RT simulations. A distinction should be made between HSR RT simulations and HSR measurements. The former is normally performed at monochromatic wavelengths with a spectral resolution much finer than that of the actual measurements; the latter can be visualized as the convolution of the former over the instrument spectral response function.
Several rigorous RT schemes, for example, discrete ordinates (Chandrasekhar 1960; Liou 1973), Monte Carlo (Plass and Kattawar 1968), and adding–doubling (Twomey et al. 1966) methods, have been developed to perform accurate RT simulations, particularly for scattering atmospheres. Meanwhile, a HSR line-by-line model has to be applied to compute the gaseous absorption (Clough et al. 1992). The approach of using a line-by-line model coupled with exact RT calculations is reasonable for sensitivity studies or model validation for limited atmospheric scenarios; however, it becomes impractical for applications (such as radiance simulations for satellite retrieval/assimilation or flux simulations for climate and weather forecasting) that involve a large number of simulations over vast spatiotemporal domains. As a result, several fast RT methods have been developed for a variety of applications. Generally, there are two approaches to improve the efficiency of RT simulations. One approach achieves computational efficiency by accelerating every RT simulation but keeping the number of independent RT simulations unchanged (Wang et al. 2013; Liu et al. 2015). The second approach alleviates the computational burden by decreasing the number of RT simulations needed to compute HSR radiances (Liu et al. 2006; Moncet et al. 2008). The accuracy and efficiency of several fast RT approaches have also been investigated (Clough et al. 2005; Oreopoulos et al. 2012; Randles et al. 2013; Aumann et al. 2018).
A variety of fast RT models have been employed for HSR simulations (Duan et al. 2005; Natraj et al. 2005; Liu et al. 2006; Moncet et al. 2008; O’Dell 2010; Spurr et al. 2013; Wang et al. 2015; Kopparla et al. 2016; Somkuti et al. 2017). These models include two varieties of principal component analysis (PCA)-based techniques. Natraj et al. (2005) perform PCA on spectrally binned sets of optical properties; costly multiple-scattering RT calculations are only done for a few PCA-derived optical states. This approach will henceforth be referred to as optical-property-based PCA (Opt-PCA). On the other hand, Liu et al. (2006) perform rigorous RT calculations at only a small number of representative wavelengths, and then compute radiances at higher spectral resolution using PCA in the radiance domain. This methodology employs PCA in radiance space, and will be referred to as radiance-based PCA (Rad-PCA).
Both Opt-PCA and Rad-PCA have been tested and validated for various applications. Opt-PCA was first applied to scalar radiance simulations (no polarization) at high resolution in and around the O2 A-band (Natraj et al. 2005), and later expanded to RT modeling with polarization (Natraj et al. 2010). Subsequently, the PCA procedure was analytically differentiated and Jacobians developed for the PCA-based radiation fields (Spurr et al. 2013). A slightly different method for calculating the Jacobians was demonstrated by Efremenko et al. (2014a). Further work was also done to extend the technique to broadband radiances and fluxes (Kopparla et al. 2016), to improve treatment of aerosol scattering (Kopparla et al. 2017), and to demonstrate application for retrievals (Somkuti et al. 2017). Efremenko et al. (2014b) discussed Opt-PCA within the general framework of dimensionality reduction in optical property space. Rad-PCA is designed to fully exploit a large amount of information from spectral radiances (Liu et al. 2006; Matricardi 2010; Havemann et al. 2018). The technique has been extended for solar spectrum simulations (Liu et al. 2016; Yang et al. 2016), and been applied to retrievals of atmospheric profiles, surface characteristics, and cloud properties under all-sky conditions, as well as in satellite data assimilation (Liu et al. 2009; Matricardi and McNally 2014; Wu et al. 2017).
In this study, we combine the advantages of Opt-PCA and Rad-PCA to further enhance the computational efficiency of HSR RT simulations with negligible loss of accuracy. The new technique compresses spectral information in both optical property and radiance domains, and will hence be referred to as spectral data compression (SDCOMP) model. To the best of the authors’ knowledge, this is the first attempt at unifying the optical and radiance-based PCA approaches to both reduce the number of monochromatic wavelengths at which RT calculations are performed and optimize the RT calculation efficiency at each wavelength. Widely utilized fast RT models, such as the ones developed by Liu et al. (2006), Matricardi (2010), Moncet et al. (2015), and Havemann et al. (2018), fail to do the latter. A similar methodology (that couples the PCA-based approaches in a different manner) was recently applied for radiance computations in the Hartley-Huggins band of ozone (del Águila et al. 2019). However, their work only included the effects of gas absorption and Rayleigh scattering (no aerosols or clouds). Further, its performance for wider spectral ranges is still unclear.
The paper is organized as follows. The new model is introduced in section 2. Section 3 discusses the performance (in terms of accuracy and computational efficiency) of the algorithm for both solar and infrared spectral regions. We provide concluding remarks in section 4.
2. The SDCOMP model
Rad-PCA saves computational time by performing monochromatic RT calculations on a reduced set of representative wavelengths and extending these results to the whole spectral range using PCA in the radiance domain. On the other hand, Opt-PCA accelerates the simulations by efficient computation at each wavelength through PCA in the optical property (e.g., optical thickness, single-scattering albedo) domain. Clearly, the two PCA techniques are complementary; we combine the two approaches and investigate the performance of the unified model for simulations of HSR top of the atmosphere radiance. This study considers a solar spectral region between 0.75 and 0.92 μm and an infrared spectral region between 2000 and 2250 cm−1 to illustrate the performance of the method for both solar and thermal radiation sources. Spectral resolutions of 0.05 and 0.005 cm−1 are used for the solar and infrared regions, respectively; both scenarios contain 50 000 monochromatic wavelengths. It should be noted that this study is intended to present the general methodology of SDCOMP; the spectral resolution is not chosen to meet any specific application. Clearly, finer spectral resolutions should be used for more accurate results; this will be done in future studies. Indeed, the numerical efficiency of our method will increase with finer spectral resolution since the computation time of SDCOMP is a very weak function of spectral resolution; on the other hand, the computation time of exact RT methods scales linearly with the number of wavelengths.
Figure 1 shows schematically the SDCOMP methodology. The basic idea is to extend Opt-PCA results (left) at representative wavelengths to all required HSR wavelengths using the Rad-PCA approach (right). Opt-PCA is executed on a much coarser wavelength grid than the desired HSR grid. Opt-PCA follows a three-step procedure. First, the spectral region is split into bins, where each bin is characterized by grouping optical properties (such as atmospheric-layer trace gas optical thickness values or single-scattering albedos) that are similar within the bin. Second, PCA is performed on the optical properties so as to obtain the eigenstates with the largest contributions to the variance. Finally, accurate RT calculations (using 32 computational quadrature angles in this work) are carried out only for a few PCA-projected states.
Flowchart to illustrate the basic concept of the SDCOMP RT model.
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
Radiances from exact RT simulations (including absorption, scattering and, in the thermal infrared, emission effects) for a large variety of atmospheric conditions are needed to calculate the PC vectors used by Rad-PCA. For this purpose, we use an atmospheric profile dataset developed by the Copernicus Atmospheric Monitoring Service (CAMS) of the European Centre for Medium-Range Weather Forecasts (ECMWF), which is a collection of representative model atmospheres (Eresmaa and McNally 2016). The dataset includes 40 000 atmospheric profiles on a 60-level vertical grid from the surface to 0.1 hPa covering realistic annual and diurnal variations in temperature, specific humidity, and the mixing ratio of ozone, carbon monoxide, nitrogen dioxide, sulfur dioxide, and formaldehyde.
For consideration of aerosol scattering, we employ simulations of the Modal Aerosol Dynamics Model for Europe (MADE) (Ackermann et al. 1998). MADE provides vertical profiles of aerosol optical thickness for five aerosol types (black carbon, organic carbon, accumulation mode sea salt, coarse mode sea salt, and sulfate). Corresponding spectrally dependent optical properties for each aerosol type are precalculated using Mie theory with size distribution parameters and refractive indices from the Optical Properties of Aerosols and Clouds (OPAC) database (Hess et al. 1998), except for black carbon, where they are taken from d’Almeida et al. (1991).
For the infrared spectral region, we use monthly mean emissivity from the University of Wisconsin–Madison Baseline Fit Emissivity Database (Seemann et al. 2008). For the visible to near-infrared spectral range, we use monthly mean surface reflectivity spectra based on the visible and near-infrared bidirectional reflectance distribution function (BRDF) atlas (Vidot et al. 2018), which was developed for the fast RT model RTTOV (Saunders et al. 2018).
The High-Resolution Transmission 2016 (HITRAN2016) molecular spectroscopic database (Gordon et al. 2017) is used to obtain gas absorption optical thickness profiles; all gases that absorb in the relevant spectral ranges are considered in the simulations. The RT calculations for the training dataset employ the linearized discrete ordinate radiative transfer (LIDORT) model (Spurr et al. 2001).
Figure 2 presents some examples of the atmospheric temperature, water vapor, and aerosol profiles and the surface albedo spectra; for each parameter, 100 different scenarios are shown in the figure. The figure clearly illustrates the large dynamic range of the parameters, which is required to obtain a representative HSR radiance dataset to be used by Rad-PCA for training. The top panel of Fig. 3 shows the resulting reflectance spectra, based on the above profiles, within the solar spectral region. We calculated spectra for 7000 randomly selected atmospheric scenarios: (i) 5000 scenarios are used as training spectra to calculate the PC vectors, (ii) 1000 scenarios are used to choose the representative wavelengths, and (iii) 1000 scenarios are used for model evaluation (SDCOMP accuracy). The bottom panel of Fig. 3 shows the first five PCs, as well as the average reflectances, in the solar spectral region. The average reflectance clearly shows the dominant gas absorption features, whereas the PC vectors capture spectral variations of the reflectances (due to absorption, scattering, and surface reflection) from one wavelength to another. In this study, we use the first 100 PCs since the results become stable and additional PCs do not further improve the accuracy. This can be modified based on the spectral information within the wavelength range of interest.
Examples of atmospheric temperature, water vapor, and aerosol optical depth (AOD) profiles, and surface albedo spectra used in the simulations.
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
(top) Examples of simulated reflectance spectra at the top of the atmosphere used for training Rad-PCA. (bottom) Corresponding reflectance mean and first five principal components (PC-1 to PC-5). To illustrate the curves on the same plot, PC-1 to PC-5 are shifted by adding 0.08, 0.12, 0.16, 0.20, and 0.24, respectively, to their values.
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
It should be noted that the accurate monochromatic RT simulations used here for the training and validation datasets are computationally expensive; however, since the training is performed only once during the model development stage, the large computational burden is acceptable. On the other hand, Opt-PCA simulations are accurate enough for the calculation of the training dataset. We have tested SDCOMP based on training with Opt-PCA results (not shown here), and have found errors due to this process to be negligible. This procedure will make future SDCOMP development for wider spectral regions much more efficient.
Liu et al. (2006) selected the representative wavelengths to predict PC scores by considering correlation coefficients between radiances at different wavelengths. This study uses an alternative, iterative method to pick the wavelengths. It should be noted that neither method might provide the “optimal” choice; more research is required to optimize the procedure. Figure 4 shows schematically the iterative method. We start with an equal-spaced grid in the spectral domain, and iteratively modify the grid based on the ensemble errors of calculated SDCOMP radiances. We follow two rules for the modification. First, wavelengths at which SDCOMP radiances have the largest errors (compared to an “exact” calculation using the LIDORT RT model) replace the closest grid point from the prior iteration, so that the “quasi uniform” distribution of the grid will not be significantly affected. Second, the total number of grid points will be kept the same, and only ~25% of the grid points are changed during each iteration. Typically, after approximately three to four iterations, convergence is achieved; that is, the overall errors cannot be improved any more by changing the grid (keeping the total number of points fixed). Note that, after convergence, if SDCOMP errors at particular wavelengths are still significant, we can add those monochromatic wavelengths to the grid to further reduce computational errors.
Schematic of the iterative method to choose the representative monochromatic wavelengths.
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
Figure 5 illustrates the key steps of the aforementioned iterative procedure. First, 400 monochromatic points are chosen uniformly within the given spectral region. Based on this initial grid, SDCOMP results at the full HSR set of 50 000 wavelengths are obtained for 1000 different atmospheric scenarios and evaluated against accurate radiances/reflectances at the top of the atmosphere from LIDORT calculations. The average relative errors for the HSR wavelength grid are illustrated in Fig. 5a. Figure 5b illustrates a particular spectral region (corresponding to the region with blue background in Fig. 5a) with relatively large relative errors; the blue circles (eight overall for this region) indicate the initial equally spaced grid points. Clearly, the choice of uniformly separated grid points introduces significant errors; ~100 (out of 50 000) relative errors are over ~2%, which we set as the threshold for grid replacement. The gray-shaded background of Fig. 5 indicates the threshold. We replace some of the initial points with wavelengths at which the relative errors are larger than the threshold; the arrows between Figs. 5b and 5c illustrate this procedure. The replacement is done such that the total number of representative wavelengths does not change between iterations. SDCOMP is then invoked again to perform calculations based on the updated grid, and the new errors are shown in Fig. 5d. It is evident that the average errors become much smaller, overall, although there are still some regions with large errors (yellow-shaded region in Fig. 5d). The procedure is repeated with a new but much smaller error criterion (~0.4%, i.e., gray-shaded region in Fig. 5d); again, most wavelengths that give large errors can be included in the grid for the next iteration. It should be noted that once a wavelength is picked to replace an initial one, it will not be replaced in the later iterations. The errors become stable after three to four iterations, and little improvement obtained with further modification.
Example to illustrate the iterative method for choosing the representative wavelengths. (a) Average relative errors at 50 000 wavelengths by using 400 equally spaced monochromatic representative wavelengths. (b) As in (a), but for a much narrower spectral range [blue region in (a)]; corresponding representative wavelength grid points are indicated by blue circles. (c) Modified grid points for next iteration (green circles) by replacing some of the prior ones with those that produce large (as defined by values in the gray-shaded area) relative errors. (d) Average relative errors after adjustment.
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
For the solar spectral region in Fig. 5, a grid with 400 representative wavelengths after the iteration is enough to give very accurate results. However, for infrared wavelengths, significant overlap between absorption by multiple gases, and the relatively dense nature of the absorption lines, make the relationship between radiances at different wavelengths much more complicated; therefore, the simple iterative scheme discussed above is not enough to obtain similar accuracy levels as that at solar wavelengths. To address this issue, wavelengths at which the errors are large are added as additional grid points. Note that additional wavelengths imply more Opt-PCA calculations. However, Opt-PCA computes only approximate (2S and single scatter) RT solutions at these points; these are computationally efficient and take negligible time compared to exact RT calculations. Once the representative grid is obtained (offline), it is not changed for any other arbitrary user-generated scenarios. Using 5000 (instead of 1000) training atmospheres did not result in any significant changes, either in the chosen reference wavelengths or in the radiances.
3. SDCOMP performance
(top) Example of reflectance at the top of the atmosphere computed using exact radiative transfer simulations. (bottom) Corresponding relative errors produced by Opt-PCA (blue), Rad-PCA (red), and SDCOMP (green).
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
Figure 7 illustrates the probability and cumulative probability distributions of spectral mean REs for 1000 different atmospheric trace gas and aerosol scenarios. The 1000 profiles are different from those used for the Rad-PCA training and those employed to choose the representative wavelengths, thereby providing an unbiased representation of SDCOMP errors. Most mean REs are less than 0.1%. Over 90% of Rad-PCA monochromatic results have REs less than 0.05%. Opt-PCA and SDCOMP provide slightly larger errors, but most of the errors (over 70% of the monochromatic REs) are less than 0.05%. SDCOMP uses Opt-PCA calculations at the reference wavelengths, whereas Rad-PCA employs exact RT calculations at those wavelengths. Therefore, SDCOMP REs are, by design, worse than those from Rad-PCA. The larger errors for Opt-PCA are because PCA is performed on the optical properties (that have a nonlinear relationship with the radiance), whereas Rad-PCA employs PCA directly on the radiance field. Finally, SDCOMP accrues errors due to PCA in both optical property and radiance domains and is therefore the least accurate of the three models. Probability distribution results for the 5000 training atmospheres are almost indistinguishable from those obtained here, demonstrating that the testing profiles span a wide range of atmospheric, surface and viewing geometry conditions, and also corroborating the robustness of the training process.
(left) Probability distributions and (right) cumulative probability distributions of spectrally averaged Opt-PCA, Rad-PCA, and SDCOMP relative errors for 1000 atmospheric trace gas and aerosol scenarios.
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
Figure 8 is similar to Fig. 6 but shows brightness temperatures (BTs) given by accurate simulations and the corresponding BT differences (BTDs) due to the three models. Since there are many more absorbing gases and more finely spaced spectral lines in the infrared, 800 representative wavelengths and 200 PCs are required for the monochromatic calculations (compared to 400 wavelengths and 100 PCs in the near-infrared). The BTDs are generally under 0.1 K; less than 1% of all results have BTDs larger than 0.1 K. Compared to the solar region, the differences are less systematic in the infrared. The average BTDs for Opt-PCA, Rad-PCA, and SDCOMP are approximately 0.012, 0.012, and 0.017 K, respectively. For the IR simulations, since the computations of the monochromatic gas absorptions themselves are quite time consuming, the efficiency gained by combining the two PCA approaches is even more significant; this will be discussed in greater detail at the end of the section.
As in Fig. 6, but for results in the infrared expressed in terms of brightness temperature.
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
Figure 9 is similar to Fig. 7 but shows the probability and cumulative probability distributions of the mean BTDs of the three models. The average BTDs for Opt-PCA and Rad-PCA show largest probability less than 0.03 K; the probability also decreases sharply as BTD increases. Over 90% of the results show BTDs less than 0.05 K; 99% show BTDs less than 0.1 K. The BTDs for SDCOMP are slightly larger, but also mostly under 0.1 K (over 95%). It should be noted that all REs and BTDs reported in this study are at monochromatic wavelengths, which are expected to be much larger than those for convolved radiances, representative of typical instrument channel measurements, used for assimilation of hyperspectral sounder data.
As in Fig. 7, but for results in the infrared expressed in terms of brightness temperature differences.
Citation: Journal of the Atmospheric Sciences 77, 6; 10.1175/JAS-D-19-0238.1
Figures 6–9 show that the accuracy of SDCOMP is similar to that of Opt-PCA and Rad-PCA. On the other hand, SDCOMP is more computationally efficient than the other two PCA-based models, as demonstrated in Table 1. To avoid the speed up of the fast RT approaches to be dependent on the choice of the accurate RT model, we employ LIDORT with 32 streams as the numerically exact RT model for all four scenarios. An “exact” model requires 50 000 accurate monochromatic RT calculations. In comparison, Rad-PCA needs only 400 accurate RT simulations for the solar spectral region; radiance-based PCA is used to extend these results to the full set of 50 000 wavelengths. The PCA process takes negligible computational time. For Opt-PCA, a combination of 2S and SS calculations is performed at every wavelength (i.e., 50 000 total); these take negligible computational time as well. Following the treatment by Kopparla et al. (2017), the spectral region is split into 11 intervals (bins) based on the total gas absorption optical thickness; a maximum of (2k + 1) accurate RT simulations are needed for each bin, where k is the number of PCs that are retained. We employ four PCs in this study, implying that no more than 99 (2 × 4 + 1 = 9 calculations for each bin) accurate simulations are required. For SDCOMP, the 400 accurate Rad-PCA simulations are replaced by fast 2S/SS calculations. Further, for the bin RT calculations, it should be noted that bins containing less than nine wavelengths are merged into an adjacent bin. Since there are far fewer monochromatic wavelengths for SDCOMP (~400) compared to Opt-PCA (50 000), several bins are either empty or have very few wavelengths. In practice, only about 50 accurate calculations are necessary. Table 1 lists only the main RT components required for the three PCA-based models; computation times that are shared by all three models (e.g., modeling setup, input/output) are not considered here. Considering that the computational times for exact LIDORT simulations differ dramatically for different atmospheric scenarios, we will not provide absolute CPU times here; numbers in Table 1 simply demonstrate the speed up compared to exact calculations.
Comparison of relative computational efficiency of the three models for the spectral range between 0.75 and 0.92 μm.
The situation for the infrared region is slightly different. Table 2 compares the relative efficiencies of the three PCA approaches compared to the exact solution. There are about 800 representative wavelengths (compared to 400 for the solar case) for Rad-PCA; therefore, more accurate RT simulations are needed for all PCA models compared to the solar scenario. Meanwhile, the difference in the number of accurate RT simulations required for Opt-PCA and SDCOMP becomes smaller. However, in this spectral region, computational times for the modeling setup (mainly calculation of gas absorption optical thicknesses) and accurate RT simulations are comparable, with a ratio approximately 3:10. Thus, if we assume that each accurate RT simulation uses a computational unit of 1.0, the corresponding model setup will take an additional computational unit of ~0.3. For the exact RT (50 000 exact RT simulations and 50 000 setups) and SDCOMP (800 exact RT simulations and 800 setups) simulations, over 20% of the total computational time is spent in the modeling setup; the fraction becomes over 99% for Opt-PCA (50 000 setups and ~100 exact RT simulations). The advantage of SDCOMP compared with Opt-PCA is clearly due to the significant decrease in modeling setup time. SDCOMP is three orders of magnitude faster than the exact simulation.
Comparison of relative computational efficiency of the three models for the spectral range between 2000 and 2250 cm−1.
4. Conclusions
This study utilized the complementarity of two different PCA-based approaches to develop a spectral data compression model that optimizes the advantages of the two models. In this SDCOMP approach, PCA is used twice independently: first, to compute radiances at representative wavelengths utilizing correlations in optical properties, and second, to extend radiances at these wavelengths to those at a larger set of monochromatic wavelengths using radiance correlations. Our results indicate that SDCOMP further improves the computational efficiency of the two existing PCA-based approaches while providing similar accuracy as those techniques (Figs. 6–9). The accuracy and efficiency may be further improved by optimizing the choice of representative wavelengths and/or the number of PC vectors.
Recent work has shown that the accuracy of SDCOMP can be further improved by applying machine learning techniques (Le et al. 2020). Future efforts will investigate the applicability and performance of SDCOMP over wider spectral ranges. Only aerosol scattering is considered here; results for cloudy atmospheres will be analyzed in forthcoming endeavors. Furthermore, SDCOMP may have the potential to be applied to flux and atmospheric heating-/cooling-rate simulations in weather and climate models. In order for SDCOMP to be relevant for atmospheric retrieval systems and numerical weather models, calculation of radiances alone is not enough; the model needs to also have the capability to compute Jacobians. The current SDCOMP model does not include Jacobian calculations. On the other hand, there is no difficulty per se to perform such calculations. This has already been demonstrated for Opt-PCA by Spurr et al. (2013). The same methodology will be employed for SDCOMP in future work.
Acknowledgments
This research was supported in part by the National Natural Science Foundation of China (NSFC) (41975025) and the Young Elite Scientists Sponsorship Program of CAST (2017NRC001). A portion of this research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004). V. N. acknowledges support from the NASA Earth Science U.S. Participating Investigator program (Solicitation NNH16ZDA001N-ESUSPI).
REFERENCES
Ackermann, I. J., H. Hass, M. Memmesheimer, A. Ebel, F. S. Binkowski, and U. Shankar, 1998: Modal Aerosol Dynamics model for Europe: Development and first applications. Atmos. Environ., 32, 2981–2999, https://doi.org/10.1016/S1352-2310(98)00006-5.
Aumann, H. H., and Coauthors, 2003: AIRS/AMSU/HSB on the Aqua mission: Design, science objectives, data products, and processing system. IEEE Trans. Geosci. Remote Sens., 41, 253–264, https://doi.org/10.1109/TGRS.2002.808356.
Aumann, H. H., and Coauthors, 2018: Evaluation of radiative transfer models with clouds. J. Geophys. Res. Atmos., 123, 6142–6157, https://doi.org/10.1029/2017JD028063.
Chandrasekhar, S., 1960: Radiative Transfer. Dover Publications, 393 pp.
Clough, S. A., M. J. Iacono, and J.-L. Moncet, 1992: Line-by-line calculation of atmospheric fluxes and cooling rates: Application to water vapor. J. Geophys. Res., 97, 15 761–15 785, https://doi.org/10.1029/92JD01419.
Clough, S. A., M. W. Shephard, E. J. Mlawer, J. S. Delamere, M. J. Iacono, K. E. Cady-Pereira, S.-A. Boukabara, and P. D. Brown, 2005: Atmospheric radiative transfer modeling: A summary of the AER codes. J. Quant. Spectrosc. Radiat. Transfer, 91, 233–244, https://doi.org/10.1016/j.jqsrt.2004.05.058.
Crevoisier, C., S. Heilliette, A. Chédin, S. Serrar, R. Armante, and N. A. Scott, 2004: Midtropospheric CO2 concentration retrieval from AIRS observations in the tropics. Geophys. Res. Lett., 31, L17106, https://doi.org/10.1029/2004GL020141.
d’Almeida, G. A., P. Koepke, and E. P. Shettle, 1991: Atmospheric Aerosol: Global Climatology and Radiative Characteristics. A. Deepak Publishing, 561 pp.
del Águila, A., D. S. Efremenko, V. M. Garcia, and J. Xu, 2019: Analysis of two dimensionality reduction technique for fast simulation of the spectral radiances in the Hartley-Huggins band. Atmosphere, 10, 142, https://doi.org/10.3390/atmos10030142.
Duan, M., Q. Min, and J. Li, 2005: A fast radiative transfer model for simulating high-resolution absorption bands. J. Geophys. Res., 110, D15201, https://doi.org/10.1029/2004JD005590.
Efremenko, D. S., D. G. Loyola, R. Spurr, and A. Doicu, 2014a: Acceleration of radiative transfer model calculations for the retrieval of trace gases under cloudy conditions. J. Quant. Spectrosc. Radiat. Transfer, 135, 58–65, https://doi.org/10.1016/j.jqsrt.2013.11.014.
Efremenko, D. S., A. Doicu, D. Loyola, and T. Trautmann, 2014b: Optical property dimensionality reduction techniques for accelerated radiative transfer performance: Application to remote sensing total ozone retrievals. J. Quant. Spectrosc. Radiat. Transfer, 133, 128–135, https://doi.org/10.1016/j.jqsrt.2013.07.023.
Eldering, A., and Coauthors, 2017: The Orbiting Carbon Observatory-2: First 18 months of science data products. Atmos. Meas. Tech., 10, 549–563, https://doi.org/10.5194/amt-10-549-2017.
Eresmaa, R., and A. P. McNally, 2016: Diverse profile datasets based on the CAMS atmospheric composition forecasting system. ECMWF Rep., 12 pp.
Fu, D., T. J. Pongetti, J.-F. Blavier, T. J. Crawford, K. S. Manatt, G. G. Toon, K. W. Wong, and S. P. Sander, 2014: Near-infrared remote sensing of Los Angeles trace gas distributions from a mountaintop site. Atmos. Meas. Tech., 7, 713–729, https://doi.org/10.5194/amt-7-713-2014.
Goody, R. M., and Y. L. Yung, 1989: Atmospheric Radiation. Oxford University Press, 519 pp.
Gordon, I. E., and Coauthors, 2017: The HITRAN2016 molecular spectroscopic database. J. Quant. Spectrosc. Radiat. Transfer, 203, 3–69, https://doi.org/10.1016/j.jqsrt.2017.06.038.
Han, Y., and Coauthors, 2013: Suomi NPP CrIS measurements, sensor data record algorithm, calibration and validation activities, and record data quality. J. Geophys. Res. Atmos., 118, 12 734–12 748, https://doi.org/10.1002/2013JD020344.
Havemann, S., J.-C. Thelen, J. P. Taylor, and R. C. Harlow, 2018: The Havemann-Taylor fast radiative transfer code (HT-FRTC): A multipurpose code based on principal components. J. Quant. Spectrosc. Radiat. Transfer, 220, 180–192, https://doi.org/10.1016/j.jqsrt.2018.09.008.
Hess, M., P. Koepke, and I. Schult, 1998: Optical properties of aerosols and clouds: The software package OPAC. Bull. Amer. Meteor. Soc., 79, 831–844, https://doi.org/10.1175/1520-0477(1998)079<0831:OPOAAC>2.0.CO;2.
Hilton, F., and Coauthors, 2012: Hyperspectral Earth observation from IASI: Five years of accomplishments. Bull. Amer. Meteor. Soc., 93, 347–370, https://doi.org/10.1175/BAMS-D-11-00027.1.
Kopparla, P., V. Natraj, R. Spurr, R.-L. Shia, D. Crisp, and Y. L. Yung, 2016: A fast and accurate PCA based radiative transfer model: Extension to the broadband shortwave region. J. Quant. Spectrosc. Radiat. Transfer, 173, 65–71, https://doi.org/10.1016/j.jqsrt.2016.01.014.
Kopparla, P., V. Natraj, D. Limpasuvan, R. Spurr, D. Crisp, R.-L. Shia, P. Somkuti, and Y. L. Yung, 2017: PCA-based radiative transfer: Improvements to aerosol scheme, vertical layering and spectral binning. J. Quant. Spectrosc. Radiat. Transfer, 198, 104–111, https://doi.org/10.1016/j.jqsrt.2017.05.005.
Le, T., C. Liu, B. Yao, V. Natraj, and Y. L. Yung, 2020: Application of machine learning to hyperspectral radiative transfer simulations. J. Quant. Spectrosc. Radiat. Transfer, 246, 106928, https://doi.org/10.1016/j.jqsrt.2020.106928.
Liou, K. N., 1973: A numerical experiment on Chandrasekhar’s discrete-ordinate method for radiative transfer: Application to cloudy and hazy atmospheres. J. Atmos. Sci., 30, 1303–1326, https://doi.org/10.1175/1520-0469(1973)030<1303:ANEOCD>2.0.CO;2.
Liou, K. N., 2002: An Introduction to Atmospheric Radiation. Academic Press, 583 pp.
Liu, C., P. Yang, S. L. Nasiri, S. Platnick, K. G. Meyer, C. Wang, and S. Ding, 2015: A fast visible infrared imaging radiometer suite simulator for cloudy atmospheres. J. Geophys. Res. Atmos., 120, 240–255, https://doi.org/10.1002/2014JD022443.
Liu, X., W. L. Smith, D. K. Zhou, and A. Larar, 2006: Principal component-based radiative transfer model for hyperspectral sensors: Theoretical concept. Appl. Opt., 45, 201–209, https://doi.org/10.1364/AO.45.000201.
Liu, X., D. K. Zhou, A. M. Larar, W. L. Smith, P. Schluessel, S. M. Newman, J. P. Taylor, and W. Wu, 2009: Retrieval of atmospheric profiles and cloud properties from IASI spectra using super-channels. Atmos. Chem. Phys., 9, 9121–9142, https://doi.org/10.5194/acp-9-9121-2009.
Liu, X., Q. Yang, H. Li, Z. Jin, W. Wu, S. Kizer, D. K. Zhou, and P. Yang, 2016: Development of a fast and accurate PCRTM radiative transfer model in the solar spectral region. Appl. Opt., 55, 8236–8247, https://doi.org/10.1364/AO.55.008236.
Liu, Y., Z. Cai, D. Yang, M. Duan, and D. Lü, 2013: Optimization of the instrument configuration for TANSAT CO2 spectrometer. Chin. Sci. Bull., 58, 2787–2789, https://doi.org/10.1360/972013-518.
Matricardi, M., 2010: A principal component based version of the RTTOV fast radiative transfer model. Quart. J. Roy. Meteor. Soc., 136, 1823–1835, https://doi.org/10.1002/qj.680.
Matricardi, M., and A. McNally, 2014: The direct assimilation of principal components of IASI spectra in the ECMWF 4D-Var. Quart. J. Roy. Meteor. Soc., 140, 573–582, https://doi.org/10.1002/qj.2156.
Moncet, J.-L., G. Uymin, A. E. Lipton, and H. E. Snell, 2008: Infrared radiance modeling by optimal spectral sampling. J. Atmos. Sci., 65, 3917–3934, https://doi.org/10.1175/2008JAS2711.1.
Moncet, J.-L., G. Uymin, P. Liang, and A. E. Lipton, 2015: Fast and accurate radiative transfer in the thermal regime by simultaneous optimal spectral sampling over all channels. J. Atmos. Sci., 72, 2622–2641, https://doi.org/10.1175/JAS-D-14-0190.1.
Natraj, V., X. Jiang, R.-L. Shia, X. Huang, J. S. Margolis, and Y. L. Yung, 2005: Application of principal component analysis to high spectral resolution radiative transfer: A case study of the O2 A band. J. Quant. Spectrosc. Radiat. Transfer, 95, 539–556, https://doi.org/10.1016/j.jqsrt.2004.12.024.
Natraj, V., R.-L. Shia, and Y. L. Yung, 2010: On the use of principal component analysis to speed up radiative transfer calculations. J. Quant. Spectrosc. Radiat. Transfer, 111, 810–816, https://doi.org/10.1016/j.jqsrt.2009.11.004.
Oreopoulos, L., and Coauthors, 2012: The continual intercomparison of radiation codes: Results from phase I. J. Geophys. Res., 117, D06118, https://doi.org/10.1029/2011JD016821.
O’Dell, C. W., 2010: Acceleration of multiple-scattering hyperspectral radiative transfer calculations via low-stream interpolation. J. Geophys. Res., 115, D10206, https://doi.org/10.1029/2009JD012803.
O’Dell, C. W., and Coauthors, 2012: The ACOS CO2 retrieval algorithm—Part I: Description and validation against synthetic observations. Atmos. Meas. Tech., 5, 99–121, https://doi.org/10.5194/amt-5-99-2012.
Plass, G. N., and G. W. Kattawar, 1968: Monte Carlo calculations of light scattering from clouds. Appl. Opt., 7, 415–419, https://doi.org/10.1364/AO.7.000415.
Randles, C., and Coauthors, 2013: Intercomparison of shortwave radiative transfer schemes in global aerosol modeling: Results from the AeroCom Radiative Transfer Experiment. Atmos. Chem. Phys., 13, 2347–2379, https://doi.org/10.5194/acp-13-2347-2013.
Saunders, R., and Coauthors, 2018: An update on the RTTOV fast radiative transfer model (currently at version 12). Geosci. Model Dev., 11, 2717–2737, https://doi.org/10.5194/gmd-11-2717-2018.
Seeman, S. W., E. Borbas, R. O. Knuteson, G. R. Stephenson, and H.-L. Huang, 2008: Development of a global infrared land surface emissivity database for application to clear sky sounding retrievals from multi-spectral satellite radiances measurements. J. Appl. Meteor. Climatol., 47, 108–123, https://doi.org/10.1175/2007JAMC1590.1.
Somkuti, P., H. Boesch, V. Natraj, and P. Kopparla, 2017: Application of a PCA-based fast radiative transfer model to XCO2 retrievals in the shortwave infrared. J. Geophys. Res. Atmos., 122, 10 477–10 496, https://doi.org/10.1002/2017JD027013.
Spurr, R. J. D., T. P. Kurosu, and K. V. Chance, 2001: A linearized discrete ordinate radiative transfer model for atmospheric remote sensing retrieval. J. Quant. Spectrosc. Radiat. Transfer, 68, 689–735, https://doi.org/10.1016/S0022-4073(00)00055-8.
Spurr, R. J. D., V. Natraj, C. Lerot, M. Van Roozendael, and D. Loyola, 2013: Linearization of the principal component analysis method for radiative transfer acceleration: Application to retrieval algorithms and sensitivity studies. J. Quant. Spectrosc. Radiat. Transfer, 125, 1–17, https://doi.org/10.1016/j.jqsrt.2013.04.002.
Twomey, S., H. Jacobowitz, and H. B. Howell, 1966: Matrix methods for multiple-scattering problems. J. Atmos. Sci., 23, 289–298, https://doi.org/10.1175/1520-0469(1966)023<0289:MMFMSP>2.0.CO;2.
Vidot, J., P. Brunel, M. Dumont, C. Carmagnola, and J. Hocking, 2018: The VIS/NIR land and snow BRDF atlas for RTTOV: Comparison between MODIS MCD43C1 C5 and C6. Remote Sens., 10, 21, https://doi.org/10.3390/rs10010021.
Wang, C., P. Yang, S. Platnick, A. K. Heidinger, B. A. Baum, T. Greenwald, Z. Zhang, 2013: Retrieval of ice cloud properties from AIRS and MODIS observations based on a fast high-spectral-resolution radiative transfer model. J. Appl. Meteor. Climatol., 52, 710–726, https://doi.org/10.1175/JAMC-D-12-020.1.
Wang, C., P. Yang, and X. Liu, 2015: A high-spectral-resolution radiative transfer model for simulating multilayered clouds and aerosols in the infrared spectral region. J. Atmos. Sci., 72, 926–942, https://doi.org/10.1175/JAS-D-14-0046.1.
Wu, W., and Coauthors, 2017: The application of PCRTM physical retrieval methodology for IASI cloudy scene analysis. IEEE Trans. Geosci. Remote Sens., 55, 5042–5056, https://doi.org/10.1109/TGRS.2017.2702006.
Wunch, D., and Coauthors, 2011: The Total Carbon Column Observing Network. Philos. Trans. Roy. Soc., 369A, 2087–2112, https://doi.org/10.1098/rsta.2010.0240.
Yang, J., Z. Zhang, C. Wei, F. Lu, and Q. Guo, 2017: Introducing the new generation of Chinese geostationary weather satellites, Fengyun-4. Bull. Amer. Meteor. Soc., 98, 1637–1658, https://doi.org/10.1175/BAMS-D-16-0065.1.
Yang, Q., X. Liu, W. Wu, S. Kizer, and R. R. Baize, 2016: Fast and accurate hybrid stream PCRTM-SOLAR radiative transfer model for reflected solar spectrum simulation in the cloudy atmosphere. Opt. Express, 24, A1514–A1527, https://doi.org/10.1364/OE.24.0A1514.