Variability present at a satellite instrument sampling scale (small-scale variability) has been neglected in earlier simulations of atmospheric and cloud property change retrievals using spatially and temporally averaged spectral radiances. The effects of small-scale variability in the atmospheric change detection process are evaluated in this study. To simulate realistic atmospheric variability, top-of-the-atmosphere nadir-view longwave spectral radiances are computed at a high temporal (instantaneous) resolution with a 20-km field-of-view using cloud properties retrieved from Moderate Resolution Imaging Spectroradiometer (MODIS) measurements, along with temperature humidity profiles obtained from reanalysis. Specifically, the effects of the variability on the necessary conditions for retrieving atmospheric changes by a linear regression are tested. The percentage error in the annual 10° zonal mean spectral radiance difference obtained by assuming linear combinations of individual perturbations expressed as a root-mean-square (RMS) difference computed over wavenumbers between 200 and 2000 cm−1 is 10%–15% for most of the 10° zones. However, if cloud fraction perturbation is excluded, the RMS difference decreases to less than 2%. Monthly and annual 10° zonal mean spectral radiances change linearly with atmospheric property perturbations, which occur when atmospheric properties are perturbed by an amount approximately equal to the variability of the10° zonal monthly deseasonalized anomalies or by a climate-model-predicted decadal change. Nonlinear changes in the spectral radiances of magnitudes similar to those obtained through linear estimation can arise when cloud heights and droplet radii in water cloud change. The spectral shapes computed by perturbing different atmospheric and cloud properties are different so that linear regression can separate individual spectral radiance changes from the sum of the spectral radiance change. When the effects of small-scale variability are treated as noise, however, the error in retrieved cloud properties is large. The results suggest the importance of considering small-scale variability in inferring atmospheric and cloud property changes from the satellite-observed zonally and annually averaged spectral radiance difference.
Longwave spectra observed from space have been used for retrieving atmospheric temperature vertical profiles (e.g., Wark and Fleming 1966; Chahine 1968, 1977), water vapor amounts and vertical profiles (e.g., Susskind et al. 2003), and cloud-top heights (e.g., Kahn et al. 2007; Menzel et al. 2008; Minnis et al. 2007). The atmospheric changes expected to occur in response to a forcing, therefore, can in principle be inferred from longwave spectrum changes. Retrieving atmospheric properties such as temperature and water vapor profiles, however, usually requires clear-sky scenes (e.g., Chahine et al. 2006; Susskind et al. 2006), although some recently developed retrieval algorithms allow clouds in a field of view (Zhou et al. 2007; Liu et al. 2007, 2009). As a consequence, the retrieval yield varies temporally and spatially depending on clear-sky occurrence. Temperature and water vapor profiles in atmospheric conditions that are preferentially avoided are absent in the retrieved data. Furthermore, results of cloud screening algorithms (e.g., Ackerman et al. 1998; Frey et al. 2008) and cloud property retrievals (e.g., Nakajima and King 1990; Minnis et al. 1998; Platnick et al. 2001) using visible wavelengths often contain a viewing zenith angle dependent error (e.g., Loeb and Coakley 1998; Zuidema and Evans 1998) that must also be accounted for when inferring mean cloud property changes. In order for instantaneously retrieved properties to be used to infer atmospheric changes over time, the effects of the instantaneous retrieval error on the mean value need to be investigated. While earlier studies investigated the temperature and humidity retrieval accuracy and yield (e.g., Tobin et al. 2006; Fetzer et al. 2003; Fetzer et al. 2004), as well as the effects of cloud optical thickness and particle size retrieval errors on a domain average (e.g., Kato et al. 2006; Kato and Marshak 2009), further studies are required to understand the effects on climate data.
An alternative to the retrieve-and-average approach to inferring atmospheric changes is an average-and-retrieve approach. Earlier studies have used temporally and spatially averaged spectral radiance differences from two time periods to infer atmospheric and cloud property changes. For example, Leroy et al. (2008) used clear-sky-averaged spectral radiance to estimate the temperature and humidity changes between two time periods. Huang et al. (2010) extended the optimal detection technique of Leroy et al. (2008) to all-sky conditions. The study by Huang et al. (2010) is based on climate model simulations so that cloud properties may be different from actual clouds (e.g., Norris and Weaver 2001; Bony et al. 2004). Due to computational restrictions, earlier studies have not accounted for temporal scale variabilities of less than a month and spatial scale variabilities of less than several hundred kilometers. As shown in section 2, the temporally and spatially averaged spectral radiance observed by a satellite instrument is affected by variability occurring at instantaneous temporal and spatial sampling scales. In this study, we further extend earlier studies to consider the variability present at an instantaneous sampling scale and to investigate how the variability affects inferring the atmospheric change from highly averaged spectral radiance differences.
Toward understanding the variability present at an instantaneous sampling scale by satellite instruments, three objectives of this study are
to understand how the variability present at a satellite instrument’s temporal and sampling spatial scales (small-scale variability) affects the retrieval of atmospheric and cloud property changes from highly averaged spectral radiances,
to quantify how small-scale variability influences the atmospheric and cloud property change detection method, and
to understand how clouds affect the variability and spectral signals.
In this study, we simulate instantaneous nadir-view spectral radiances using cloud properties derived from satellite observations on a 20-km footprint to achieve the objectives above. We then perturb the atmospheric properties and compute the spectral radiances, maintaining the instantaneous temporal and spatial resolutions to test the effects of variability on the spectral signal. Our simulation, therefore, differs from the earlier studies that used climate-model-derived monthly mean atmospheric properties (Huang et al. 2010). We use a linear regression to retrieve the temperature, humidity, and cloud property changes from the nadir-view spectral radiance change, similar to earlier studies (Leroy et al. 2008; Huang et al. 2010).
The retrieval is performed by seeking consistent atmospheric and cloud property changes with the observed spectral radiance change. Our retrieval goal is to infer changes that occur on a shorter time scale (up to a decade) instead of a longer time scale (~100 yr), as was sometimes used in earlier studies. As the observation continues, the trend of the atmospheric and cloud property changes can be inferred from these retrieved properties, or changes can be detected by directly applying the average-and-retrieve technique to the spectral radiance difference derived from a longer time period. Our purpose in inferring atmospheric changes contrasts with studies that find the best combination of responses to a particular external forcing modeled by climate models, and how the response changes the top-of-the-atmosphere (TOA) spectral radiance. As a result, we do not distinguish between changes caused by anthropogenic forcing and those from natural variability in this study.
Section 2 briefly explains the motivation for investigating the effects of small-scale variability and section 3 describes the modeling of spectral radiance. Section 4 describes the results of the spectral radiance change at nadir caused by atmospheric and cloud property perturbations, and tests the necessary conditions for retrieving atmospheric changes by a linear regression. Finally, the retrieval process is demonstrated in section 5 while a discussion of the findings and our conclusions are presented in sections 6 and 7, respectively.
2. Effects of observed high temporal and spatial variabilities on atmospheric change detection
We construct a simple model to understand how small-scale variability affects the mean spectral radiance and the detection of atmospheric and cloud property changes. Suppose that we try to retrieve the atmospheric and cloud property changes from two annual mean spectral radiances. The linearized annual mean spectral radiance I measured over the time period 1 is
where x is an atmospheric property such as temperature, humidity, or cloud property. The overbar indicates the annual mean, δx is its small-scale variability (i.e., deviation from the mean) present at satellite sampling scale, and n is the number of satellite observations used to determine the annual mean value. Many atmospheric properties influence the TOA spectral radiance. The different variables are expressed by the subscript j. The subscript i indicates an individual sampling during time period 1. The spectral radiance sensitivity to the variable xj at an instantaneous observation deviates from the sensitivity computed with mean atmospheric and cloud properties and is approximately equal to . Equation (1) indicates that the mean spectral radiance and the spectral radiance computed with mean atmospheric and cloud properties differ when the second term on the right side is not negligible due to the correlation between δxji and . When instantaneous radiances are measured during time period 2, atmospheric properties and the radiance sensitivity to atmospheric properties deviate from the respective mean values of time period 1:
where Δx is the atmospheric or cloud property deviation from the mean value of the time period 1 and is the spectral radiance sensitivity to the atmospheric or cloud property xj when the mean atmospheric and cloud properties are from time period 1. The second term in the bracket accounts for the deviation of the sensitivity at an instantaneous scale from the sensitivity of the mean atmospheric and cloud properties of time period 1.
When we divide Δxji in Eq. (2) into two components—the mean change of time period 2 from the mean value of time period 1 and the small-scale variability, —and take the average of radiances sampled during time period 2, we get
When we take the difference of the spectral radiance from two time periods, 2 minus 1, we get
Term 1 is the spectral radiance change caused by the mean atmospheric and cloud property changes. Term 2 is the contribution due to the correlation between the small-scale variability and mean atmospheric or cloud properties. Term 3 is caused by the correlation among the mean atmospheric and cloud properties. Also, term 3 is the nonlinear term and throughout the remainder of this paper, nonlinear refers to the correlations between two mean changes among atmospheric and cloud properties. Term 4 is caused by the small-scale variability difference between two time periods and term 5 is caused by the difference of term 2 between two time periods (i.e., a part of spectral shape uncertainty). Terms 4 and 5 are expected to be negligible when many pairs of two time periods are averaged. Considering a given pair of two time periods, however, two terms associated with small-scale variability in terms 4 and 5 do not necessarily cancel at the annual time scale. When terms 4 and 5 are not negligible, terms 2–5 become error sources in retrieving the atmospheric changes if one assumes that the radiance change is caused by mean atmospheric and cloud property changes.
3. Spectral radiance computations
To include the small-scale variability present at the satellite sampling scale, we use satellite-observed cloud fields in our simulations because, as shown in section 4b, spectral radiance variability is largely caused by clouds. Therefore, it is important to use realistic cloud fields to test the effects of variability on atmospheric and cloud change detection.
a. Cloud fields
The cloud fields used in this study were derived from Moderate Resolution Imaging Spectroradiometer (MODIS) spectral radiance observations. MODIS-retrieved cloud properties are included in the Clouds and the Earth’s Radiant Energy System (CERES) Single Scanner Footprint (SSF) product. Cloud properties derived from MODIS are output from the second edition (Ed2) of the CERES cloud algorithm (Minnis et al. 2007) based on the assumption of a single-layer overcast cloud in a 1-km pixel. As a result, there are no overlapping clouds within a CERES footprint and up to two single-layer cloud properties were kept within a CERES footprint.
b. Computation of TOA spectra viewed from nadir
Two years’ worth of MODIS-derived cloud fields from January 2003 through December 2004 are used as a control run. Temperature and water vapor profiles from the Goddard Earth Observing System Data Assimilation System (GEOS-4; Bloom et al. 2005) are used for the simulation. Ozone profiles are retrieved daily from the Solar Backscatter Ultraviolet instrument (SBUV/2; Bhartia et al. 1996). For the polar region during polar night, the ozone profiles are retrieved from the Television and Infrared Observation Satellite (TIROS) Operational Vertical Sounder (TOVS) by the algorithm of Neuendorffer (1996). Retrieved ozone profiles are sorted into daily maps of 2.5° × 2.5° grids with 24 pressure levels (Yang et al. 2011).
TOA longwave nadir-view spectra from 50 to 2760 cm−1 are computed with a 1.0 cm−1 resolution by the Principal Component-based Radiative Transfer Model (PCRTM; Liu et al. 2006) using the independent column approximation (Stephens et al. 1991; Cahalan et al. 1994). The spectroscopic High-Resolution Transmission Molecular Absorption Database (HITRAN 2000) is used for atmospheric molecular transmittance calculations. Spectral radiances are computed at Terra overpass times and are averaged to obtain zonal and global means.
We adapt a method that performs cloud radiative transfer calculations using precomputed cloud transmittances and reflectances (Yang et al. 2001; Wei et al. 2007; Huang et al. 2004; Niu et al. 2007). The complex refractive indexes of ice and water are taken from Warren (1984) and Segelstein (1981). The individual ice cloud particle size distributions are derived from various field campaigns as described by Baum et al. (2007). The single-scattering properties, such as phase function and single-scattering albedo, are derived from the finite-difference time domain method, the improved geometric optics method, or Lorenz–Mie theory, depending on the size and shape of the cloud particles. A gamma size distribution is assumed for water clouds. The surface emissivity depends on surface type but does not vary with time.
In addition to the control run, we perturb the atmospheric properties and compute the TOA spectral radiance (perturbed runs). Only the first 15 days in each month are selected for perturbation calculations due to computational constraints. Fifteen cloud and atmospheric properties listed in Table 1 are perturbed independently. In perturbing cloud properties, clouds are separated into three types depending on their top height, according to the International Satellite Cloud Climatology Project (ISCCP) cloud classifications (Rossow and Schiffer 1991). Clouds with cloud-top heights of 6.5 km or higher are classified as high-level clouds, clouds with cloud top heights of 3.5 km or lower are classified as low-level clouds, and clouds in between are midlevel clouds. For perturbation 5 in Table 1, thin cirrus clouds were defined as having an optical thickness of less than 1. The values listed in Table 1 are used to perturb the atmospheric and cloud properties uniformly at all latitudes.
* RMS computed with all ice clouds.
We make a subtle but important modification to the GEOS-4 temperature profile to make the temperature profile consistent with the MODIS-derived cloud-top heights. When the low-level cloud height is increased by 250 m in the cloud-top perturbation run, the temperature inversion present at the top of boundary layer clouds is also moved with the cloud to prevent the resulting cloud-top height from becoming higher than the temperature inversion height. To adjust the temperature profile, we first compute the lapse rate in the boundary layer with the original GEOS-4 temperature profile. Second, when the low-level cloud is moved upward by 250 m, we extend the boundary layer so that the lapse rate from the surface to the new cloud top is the same as the original lapse rate below the cloud.
Note that the boundary layer cloud-top height and the height of the temperature inversion do not necessarily agree in the control run. In the case when the cloud-top and temperature inversion heights do not agree in the control run, the temperature inversion height is adjusted in the same way described above. Hence, the temperature inversion height also agrees with the boundary layer cloud-top height in the control run.
Table 1 lists the magnitudes of the perturbations used for perturbed runs. Each perturbation magnitude is determined in one of two ways. One way is to match approximately the root-mean-square (RMS) difference of 10° zonal monthly means computed from the 2003 and 2004 atmospheres (approximately equal to the standard deviation of deseasonalized anomalies). The other way is to use the expected global mean changes between the first two decades (2000–09 and 2010–19) of a simulation of the Intergovernmental Panel on Climate Change (IPCC) Special Report on Emission Scenario (SRES) A1B forced by the National Center for Atmospheric Research (NCAR) Community Climate System Model (CCSM), version 3.0 (Collins et al. 2006). A reasonable magnitude of expected atmospheric and cloud property changes are used to test the linear relationship between these properties and TOA spectral irradiance changes in the presence of small-scale variability.
As noted in the previous section, atmosphere and cloud properties are perturbed at a 20-km spatial resolution and an instantaneous temporal resolution to include the effects of small-scale variability. Because our purpose is to compute a spectral radiance change for the atmospheric and cloud property change retrieval, all perturbations listed in Table 1 are applied to all regions uniformly. Instantaneous spectral radiances are averaged monthly at 10° latitude intervals. Monthly zonal mean spectral radiance changes due to perturbations are computed by differencing the control run and perturbed run spectral radiances. The global mean spectral radiance or brightness temperature change is computed by averaging the zonal mean spectral radiances weighted by their area, and subtracting the global mean radiance (or brightness temperature) of the control run from the global mean value of a perturbed run.
a. Spectral radiance change
Figure 1 shows the global annual mean brightness temperature difference caused by perturbations listed in Table 1. The shape of the spectral radiance changes from some perturbations is expected to be similar for the following reasons.
The contribution function of the atmospheric emission to the TOA radiance for a given wavenumber peaks at the height where the absorption optical thickness is approximately equal to one (Goody and Yung 1989). The absorption optical thickness of the clear-sky atmosphere is often less than one in the window spectral regions from 800 to 1200 cm−1 and beyond 2000 cm−1. Therefore, the emission in these spectral regions mostly comes from the surface. In spectral regions where gaseous absorption is strong, the atmosphere is opaque. Therefore, changes in the surface temperature have little effect on the TOA radiance in spectral regions of the water vapor rotation band below 500 cm−1, the ν2 vibration-rotation band centered at 1595 cm−1, and the CO2 vibration-rotation band centered at 667 cm−1. Because most low-level clouds are opaque at infrared wavelengths, increasing the low-level cloud height, which reduces the cloud-top temperature, yields a spectral radiance change similar to the surface temperature increase with an opposite sign in the window regions. However, the emissivity of low-level clouds in the window region is not spectrally constant. As a consequence, the cloud-top height perturbation gives a nonuniform effective cloud-top temperature change as opposed to a spectrally constant radiance change in response to the surface temperature perturbation. This difference leads to spectral shape differences in the window region.
The spectral shapes of the brightness temperature change due to cirrus cloud optical thickness perturbation and due to upper-tropospheric relative humidity perturbation are quite different (Fig. 1). When the cirrus cloud optical thickness is increased, the emission height changes from a lower altitude to a higher altitude in the spectral region in which clouds are translucent. In contrast, the spectral change due to upper-tropospheric humidity perturbation occurs in stronger water vapor lines where the emission from water vapor originates in the upper troposphere.
The effects of the cloud particle size change are one order of magnitude smaller than the spectral radiance change by the optical thickness, cloud height, and fraction perturbations (Fig. 1). The longwave radiance is sensitive to cloud particle size change when the clouds are translucent and the particle sizes are not so large [e.g., ~100-μm diameter for ice particles; Cooper et al. (2006)]. The distinct feature of the spectral shape can potentially help to separate cloud microphysical changes from macrophysical changes, if particle size changes occur in optically thin clouds.
b. Necessary conditions for a linear regression
To accurately retrieve atmospheric and cloud property changes by a linear regression, several conditions must be satisfied. First, the sum of the spectral changes caused by individual perturbations must be approximately equal to the spectral change caused by all individual properties perturbed simultaneously. Second, although all wavenumbers used in the retrieval do not need to respond linearly, the radiance for a given wavenumber needs to change approximately linearly in response to an atmospheric or cloud property perturbation. Third, the spectral shape caused by each atmospheric or cloud property change must be unique, meaning that a linear regression can identify individual spectral shapes from the sum of all spectral signals.
To test the effects of small-scale variability on the first condition, we combined perturbations (1 and 5–12 listed in Table 1) and computed the global annual mean spectra to evaluate whether the sum of the spectral changes computed independently is equal to the spectral change from the combined run in which all are perturbed together. Because small-scale variability is included in both the individual and combined runs, term 2 in Eq. (4) is the same in both the individual and combined runs. In addition, because the independent runs and combined run use the same time period, the small-scale variabilities in those runs are the same. Hence, terms 4 and 5 in Eq. (4) cancel out when one result is subtracted from the other. Therefore, the spectral radiance difference is due to the nonlinear term [term 3 in Eq. (4)] that is included in the combined run but excluded in the sum of the independent runs.
Figure 2 shows that the difference is less than 10% (relative difference less than 0.1) of the signal, except around 1200 and 2000 cm−1 (Fig. 2, bottom left). Note that large differences around 650 cm−1 are due to dividing by a small value, as indicated by the top-left plot in Fig. 2. A nonlinear effect can occur, for example, when a larger surface area with a higher temperature is exposed to space as a result of a mean cloud fraction decrease and a mean surface temperature increase. If the cloud fraction change is not included (i.e., the combined run includes 1, 5–8, and 12 in Table 1), the nonlinear term is 1% (Fig. 2, bottom left). Therefore, the cloud fraction change significantly contributes to the nonlinear term. Water vapor amount perturbations also cause a nonlinear effect when both the surface temperature and humidity are perturbed. The transmittance of the signal from the surface temperature change is altered as a result of a smaller atmospheric water vapor amount. The atmospheric transmission change by the humidity perturbation is, however, smaller than the cloud fraction perturbation.
In addition, Fig. 3 shows, for each 10° latitude zone, the annual mean RMS relative difference between the spectral radiance change computed by perturbing the atmospheric and cloud properties of 1 and 5–12 listed in Table 1 (solid line) simultaneously (ΔIs) and the sum of the spectral radiance change computed by perturbing these properties individually (ΔIi). The 10° mean relative RMS difference is computed from ΔIs − ΔIi over wavenumber and divided by the RMS of ΔIs computed also over wavenumber using monthly mean spectral radiances. The annual mean RMS difference is computed by averaging the monthly relative differences. The percent error in the estimated spectral radiance difference obtained by assuming linear combinations of individual perturbations expressed as an RMS difference is 10%–15%, except over Antarctica (Fig. 3). The maximum and minimum relative differences of the monthly 10° zonal means are 16% and 3%, respectively, when the two southernmost 10° latitudinal zones are excluded. Figure 3 also shows that when cloud fraction changes are excluded, the RMS difference is less than 2% for most 10° zones. The magnitude of the nonlinear term with clouds is consistent with a study by Huang et al. (2010), who perturbed the monthly mean atmospheres to test this condition. However, our results show that the magnitude of the nonlinear term changes significantly with and without cloud fraction perturbation, while the results of Huang et al. (2010) show that the magnitude and shape of the nonlinear term are similar with and without cloud property changes.
To further test the effects of small-scale variability on signal linearity (second condition listed above), we doubled the perturbation amount and evaluated whether or not the spectral radiance change is also doubled. Figure 4 shows the difference in the monthly and annual 10° zonal mean spectral changes due to Δxi and 2Δxi–Δxi. The spectral radiance changes linearly for temperature changes so that is nearly constant in a range at least twice as large as the perturbation amount listed in Table 1. Water vapor amount perturbations give a slight nonlinear response in window regions. Larger deviations are seen outside the window region for the low-level cloud-height perturbation, and somewhat smaller but significant deviations for midlevel cloud height and water cloud particle size perturbations are noted. The differences in Δxi and 2Δxi–Δxi computed from the monthly mean and annual mean are similar, indicating that for given mean atmospheric and cloud property changes, the linear relationship is nearly insensitive to the length of the temporal averaging period. A better linear response is achieved by using a longer temporal averaging period because the mean atmospheric and cloud property changes are smaller when a longer temporal period is used.
To test the uniqueness of the spectral shape given by different atmospheric and cloud property perturbations (the third necessary condition), we perform a simple linear regression. The mean spectral difference ΔI is expressed by a linear combination of the spectral signature matrix and scaling vector a:
where δIb and ε are, respectively, the modeling bias and random errors. The spectral radiance difference ΔI is a column vector of dimension m. The spectral signature matrix is an m row × n column matrix that contains distinguishing spectral radiance changes where m is the number of wavenumbers and n is the number of perturbed atmospheric properties used to compute the spectral changes. Columns of S are the average of computed at an instantaneous resolution. Because the units of elements of S are the same as ΔI, elements of a are nondimensional. Elements of a are, therefore, scaling factors to the perturbed amount Δx used to compute the elements of (Table 1). The bias and random errors caused by radiative transfer modeling and the uncertainty in molecular spectroscopy contribute to errors in the elements of . Because simulated spectral radiances are used for ΔI in this study, the modeling errors have no impact on the retrieval. Note that the effects of small-scale variability (terms 4 and 5) are estimated in section 5.
In this simple example, we use the sum of terms 1–3 as ΔI. We use the spectral radiance change ΔI computed by perturbing the variables listed in Table 1 (i.e., variables 1, 4–10, and 14) simultaneously to test whether or not the shape of the spectral radiance change given by an individual perturbation is correctly separated by a linear regression. When there is no error term, a simple linear regression for retrieving the atmospheric and cloud property changes is
For a perfect retrieval, the elements of a are all unity in this simulation. When we obtained a by neglecting a nonlinear term, the error was less than 10% except for the midlevel cloud fraction (Table 2). When the nonlinear term (term 3) is included in , as suggested by Allen and Tett (1999), a linear regression properly retrieved all elements of a. This result suggests that all spectral shapes of radiance changes used in this study are different and can be separated by a linear regression.
In summary, our results suggest that the TOA spectral radiance change can be expressed approximately by a linear combination of the TOA spectral radiance changes computed by perturbing the atmospheric properties independently. The spectral difference between the linear combination of the spectra perturbed independently and the spectrum computed by simultaneously perturbing all atmospheric variables is approximately 10%–15% expressed by the relative RMS difference over the wavenumber between 200 and 2000 cm−1 (Fig. 3). The relative difference around spectral regions 1200 and 2000 cm−1 can be as large as 100% when the cloud fraction change is included (Fig. 2). The TOA longwave spectrum changes linearly in the perturbation of the atmospheric temperature and water vapor when it is perturbed within the variability of 10° zonal deseasonalized monthly anomalies or within the magnitude of a decadal change predicted by a climate model (Fig. 4). TOA longwave spectral radiance also changes linearly in the perturbation of a cloud property except for low- and middle-level cloud-height perturbations. When the low- and midlevel cloud-height perturbations are doubled, the annual 10° zonal mean spectral radiance change at water vapor absorption bands, wavenumbers less than 500 cm−1, and between 1300 and 2000 cm−1 significantly deviates from a linear relationship. The broad spectral feature that deviates from a linear relationship is an error source when Eq. (8) is applied to ΔI when the change in the atmospheric properties or cloud properties is large. The shape of the spectral radiance changes is different such that the individual spectral radiance changes can be separated from the spectral radiance change computed by perturbing all variables simultaneously using a linear regression. When the nonlinear term is neglected in a spectral signature matrix, the retrieval error is less than 10% for most cases (Table 2). The spectral difference by midlevel cloud fraction perturbation is, however, approximately 20%. Presumably, the spectral shape of the nonlinear term is somewhat similar to the spectral shape of the midlevel cloud fraction perturbation.
5. Retrieval simulation
Two error terms (terms 4 and 5) do not affect the retrieval result of a simple simulation, as demonstrated in the previous section. In this section, we assess the effects of terms 4 and 5 in the retrieval. For the simulation, we use the annual 10° zonal mean spectral radiance in this study. In the simulation, we treat the spectral difference of the 2003 and 2004 control runs as the observed spectral difference and test whether or not temperature, humidity, and cloud property differences between the 2003 and 2004 data used for control runs can accurately be retrieved.
The smallest eigenvalue of used in this study is of the order of 10−12, while the radiance change is of the order of 10−4 W m−2 sr−1 cm. When the matrix is inverted to obtain a, small errors in the elements of are magnified, leading to large errors in a (Twomey 1977). To avoid a large error in a, we introduce a smoothing constraint:
where λ is a Lagrange multiplier. For the simulation, we use an identity matrix to minimize the variability of scaling factors in ac (Twomey 1977), which simply increases all eigenvalues by a constant amount. To determine the value of λ, we compute the RMS difference between the retrieved ac and the true values for the annual 10° zonal means as a function of λ. Based on the result of the RMS difference as a function of λ, we use λ = 1.0 × 10−7 for our simulation.
We derive ac over a 10° latitudinal zone separately using Eq. (9). We then compare ac multiplied by the perturbed value used in perturbed runs ΔxTac with the atmospheric and cloud property changes computed from atmospheric and cloud properties used to compute spectral radiances in the control run (true values). Figure 5 shows the retrieved values versus the true values. While atmospheric temperatures, especially stratospheric temperatures, are retrieved well, our results indicate that the retrieved cloud properties obtained with this average-and-retrieve approach have sizable errors and large estimated uncertainties.
The uncertainty in the retrieved values caused by the error terms (terms 4 and 5) is estimated as follows. First, we compute the difference of the spectral radiance computed from the mean instantaneous spectral radiances and the spectral radiance computed from annual 10° zonal mean atmospheric and cloud properties, . Second, we compute with 2003 and 2004 atmospheric and cloud properties and take the difference:
where subscripts 04 and 03 indicate 2004 and 2003, respectively. Because includes all terms and includes terms 1 and 3, Eq. (10) includes terms 2, 4, and 5. Term 2 is estimated from 2003 perturbed runs by multiplying the spectral radiance difference by the ratio of the actual atmospheric and cloud property changes from 2003 to 2004, Δx(04–03), by the perturbed amount Δx, . Using annual 10° zonal mean spectral radiance, we form an m × l error matrix ε′, where m is the number of wavenumbers and l is the number of 10° zones (=16 because the two southernmost zones are dropped) and the prime indicates that the mean over 10° zones is subtracted so that the mean is zero for each wavenumbers (i.e., and is an l × l matrix of which elements are all 1). We compute an n × n covariance matrix by (ε′ε′T) (see appendix B), where n is the number of retrieved variables. We set λ = 0, which gives the upper limit of the uncertainty estimate due to terms 4 and 5 to crudely account for using ε′ (the mean = 0) instead of ε. For this limit, the expression is equivalent to given by, for example, Allen and Tett (1999). We plot the square root of the diagonal elements of the resulting matrix as error bars in Fig. 5 to indicate the retrieved temperature and humidity uncertainties due to terms 4 and 5. Because the error bars for the retrieved cloud fraction and height are large and extend over the entire y range of the plot, the error bars were omitted from the retrieved cloud property plots. The uncertainties averaged over eighteen 10° zones are ±2.0 km, ±0.2, and ±0.4 for low cloud height, low cloud fraction, and high cloud fraction, respectively.
Note that brightness temperature differences can be used in the spectral signature matrix and ΔI for the retrieval, but we find that the retrieval errors are significantly smaller, especially for cloud properties, when radiance differences are used. The exact reason is unknown but when brightness temperature differences are used, the difference in the far- and near-infrared spectral regions is emphasized compared to the difference computed from radiances.
A relatively small impact of small-scale variability on achieving necessary conditions to retrieve atmospheric and cloud properties, compared with the effects on the retrieval results presented in section 5, can be seen from Eq. (4). When the spectral radiance from the control run of the same time period is subtracted from the perturbed run, the two terms in terms 4 and 5 cancel out exactly. Therefore, the spectral radiance change included in the spectral signature matrix is
When (11) is used for the retrieval from the annual and 10° zonal mean spectral radiance differences computed from two time periods, however, the two terms in terms 4 and 5 in (4) remain. As a consequence, they become a bias error in . The magnitude of these terms appears to depend on latitude. The spectral shape of terms 4 and 5 resembles the spectral shape of cloud property changes because the small-scale variability δxji is largely due to the variability of cloud properties.
There are several possible ways to reduce the retrieval error. If, for example, terms 4 and 5 are related to the standard deviation of the spectral radiance, it might be possible to estimate these terms from the mean and standard deviation of the spectral radiance. Subsequently, they are subtracted from ΔI before the inversion. Another possible way to reduce the error in the retrieval is by separating clear-sky scenes from cloudy-sky scenes because clouds are responsible for much of the small-scale variability. Using only clear-sky spectral change reduces the error in retrieving temperature and humidity profile changes. Subsequently, constraining the temperature and humidity changes in the retrieval under the all-sky conditions might reduce the error in the retrieved cloud property changes. Investigating the method to reduce the retrieval error is left for future studies.
Optimizing a linear regression using empirical orthogonal functions (EOFs) of a covariance matrix (ε′ε′T) and omitting smaller eigenvalues instead of using Eq. (9), is an alternative approach. We, however, used Eq. (9) in this study because the retrieval result is sensitive to the covariance matrix, as the results of Huang et al. (2010) imply. Perhaps the proper way to form the covariance matrix is to use temporal correlation instead of spatial correlation. We did not have enough simulated spectral radiances to form the temporal correlation in this study. Therefore, to exclude the effects of the covariance matrix, we used Eq. (9). We found, however, that applying the covariance matrix based on spatial correlation worsens the retrieval result. Unlike the instrument noise, which has a different spectral shape than the signals, terms 4 and 5 are caused by the small-scale variability of clouds and atmospheric properties. As a consequence, they could have similar spectral shapes as signals. Whether EOFs and a properly formed covariance matrix help to improve the retrieval still needs to be tested in the future.
To understand the effects of small-scale variability on atmospheric temperature, humidity, and cloud property change detection, we computed the spectral radiance using high temporal (instantaneous) and spatial (~20 km) resolutions and simulated the variability of observed radiances in the nadir direction. We tested the necessary conditions to retrieve atmospheric and cloud property changes from spatially and temporally averaged spectral radiances by a linear regression. Our results show that the annual 10° zonal mean spectral radiance changes linearly with respect to the temperature and humidity perturbations when they are perturbed either by the amount of changes expected to happen in a decade or by the RMS difference of 10° zonal monthly means between 2003 and 2004. The spectral radiance due to cloud-height perturbations changes nonlinearly outside the window region, especially for low-level clouds. The sum of the spectral changes computed by perturbing atmospheric properties independently is equal to within 10%–15% of the spectral change computed by perturbing all properties simultaneously for most spectral regions. Cloud fraction changes are largely responsible for the difference. When cloud fraction changes are excluded, the difference decreases to less than 2%. Spectral shapes of the radiance change caused by different atmospheric and cloud property changes are separated by a linear regression. Variability present at an instantaneous resolution does not affect the establishment of these conditions necessary for atmospheric and cloud property change detections by a linear regression as much as it affects the retrieval. Our simulation indicates that retrieved atmospheric and cloud property changes from the annual 10° zonal mean spectral radiance changes contain errors, especially in retrieved cloud properties, because small-scale variability affecting the mean spectral radiance from two periods does not necessarily cancel. As a consequence, the residual becomes a bias error in the spectral radiance difference computed from two time periods. Two possible ways to improve the retrieval are 1) to perform the retrieval using clear sky only and constrain the temperature and humidity changes in the all-sky retrieval and 2) to seek the relationship between the standard deviation of the spectral radiance and small-scale variability terms. Using the relationship, bias errors caused by small-scale variability can be subtracted from the spectral radiance difference. Investigating ways to improve the retrieval is left for future studies.
We thank Drs. Stephen Leroy, John Dykema, Yi Huang, Xianglei Huang, Robert Knuteson, Norman Loeb, and Oleg Dubovik for helpful discussions and suggestions and two anonymous reviewers for constructive and very helpful comments. We also thank Ms. Amber Richards for proofreading the manuscript. The work was supported by the NASA Science Directorate through the CLARREO project.
Cloud Effects on the TOA Spectral Radiance
The nadir-view radiance for clear sky is
where Bν is the Planck function, Tsfc is the surface temperature, and τν0 is the optical thickness of the atmosphere. The subscript ν indicates that the optical thickness is wavelength dependent. The nadir-view radiance for overcast cloudy sky is
where τνc is the spectral-dependent cloud optical thickness and the source function Jν is
In (A3), ω0 is the single-scattering albedo, P(μ, φ) is the phase function, μ is the cosine of the zenith angle, and φ is the azimuth angle. For the partly cloudy sky, the nadir-view radiance is assumed to be the sum of the clear-sky and overcast radiances weighted by the cloud fraction f.
Suppose both the surface skin temperature and atmospheric air temperature are perturbed by ΔTsfc and ΔT, respectively. When we expand the Planck function by a Taylor expansion and keep the first term, the nadir-view radiance difference is
The difference for cloudy sky is
The difference between the clear- and cloudy-sky differences is, therefore,
If the temperature profile above the surface does not change with the surface temperature, the second and third terms vanish, which leads to
Equation (A7) shows that ΔIcld has a different spectral dependence from ΔIclr because the spectrally dependent deference is between τν0 and τνc.
Contribution of Terms 4 and 5 to the Covariance of Retrieved Properties
From Eq. (9),
where is a constant matrix.
Applying a variance operator Var to (B3),
From the fact that is symmetric, this leads to
The contribution of the error term ε′ in ΔI to the variance of ac is obtained by applying the Var to ε′: