First comparisons of improved initialization inversion (3I) cloud parameters determined from TIROS-N Operational Vertical Sounder observations with time–space-collocated clouds from the recently reprocessed International Satellite Cloud Climatology Project (ISCCP) dataset have shown a reasonable agreement between all cloud types, with exception of the stratocumulus regions off the western coasts. Here, 3I clouds were found systematically thinner and higher than ISCCP clouds. These results have initiated a careful investigation of the methods used to convert measurements from IR sounders into cloud parameters. All existing methods get very sensitive to the chosen temperature profile toward lower cloud heights, due to a denominator approaching zero. This leads to a bias like the one seen in the comparison with ISCCP. Therefore, a new 3I cloud scheme has been developed, based on a weighted-χ2 method, which calculates the effective cloud amount from the CO2-band radiances, but weighted differently according to the effect of the brightness temperature uncertainty within an air mass on these radiances at the various cloud levels. This physically much more correct method led to unbiased 3I cloud parameters for homogeneous cloud types. The ISCCP comparison agrees much better now, especially in the stratocumulus regions where the cloud type matching improved from about 50% to 75%. In 1° grid boxes covered uniformly with the same ISCCP cloud type, the matching reaches even 87%. Remaining discrepancies in cloud classification can be explained by partly cloudy fields and differences in temperature profiles and cloud detection.
The weighted-χ2 method can be used in other IR sounder inversion algorithms, if the empirical weights, taking care of the effect of temperature profile uncertainties on the difference between clear sky and cloudy radiances for different cloud levels and spectral channels, have been reevaluated so that they can be calculated automatically by the corresponding inversion algorithm.
Infrared (IR) sounders can provide important information on atmospheric temperature and water vapor profiles as well as on cloud and surface properties (day and night). However, complex retrieval algorithms are necessary to convert measured radiances, which are backscattered or emitted by the atmosphere or clouds, into atmospheric, cloud, and surface properties. The subject of this article is a careful reinvestigation of different methods used for cloud parameter determination.
The 3I algorithm suite (Chédin et al. 1985) determines atmospheric temperature and water vapor profiles, as well as cloud and surface properties, from TIROS-N Operational Vertical Sounder (TOVS) observations (Smith at al. 1979), using the instruments: High-resolution Infrared Radiation Sounder (HIRS) and Microwave Sounding Unit (MSU). It is based on 1) the Thermodynamic Initial Guess Retrieval (TIGR) dataset, describing ∼2300 different atmospheric conditions extracted from ∼180 000 radiosonde measurements (Escobar 1993; Chevallier et al. 1998) and 2) a fast line-by-line radiative transfer model, Automatized Atmospheric Absorption Atlas (4A; Scott and Chéin 1981), simulating clear sky and cloudy radiances at 30 pressure levels. Cloud detection is performed at HIRS spatial resolution (≈17 km) by seven (night) or eight (day) threshold tests, relying very much upon comparisons of the simultaneous HIRS and MSU channels, where the latter probe through the clouds.
First comparisons with time–space collocated clouds from the recently reprocessed International Satellite Cloud Climatology Project (ISCCP) dataset (Rossow et al. 1996) have shown that cloud-top pressure and effective cloud amount obtained from the 3I algorithms were in general reliable (original 3I in Stubenrauch et al. 1999a). Even low clouds were identified by the 3I cloud scheme (Stubenrauch et al. 1996), but systematic differences could be found in the stratocumulus regions off the western coasts, where 3I clouds were found mostly thinner and higher than the ISCCP clouds.
Therefore, we have investigated carefully the methods used to convert the CO2-band emitted radiances into cloud parameters in section 2. We came to the conclusion that all existing methods get very sensitive to the chosen temperature profile toward lower cloud heights due to a denominator approaching zero, introducing a bias that tends to determine low opaque clouds as high thin clouds due to temperature profile uncertainties. The newly developed weighted-χ2 method, not including a denominator, is described in section 3. By this method, one calculates effective cloud amount by using all CO2-band emitted radiances, but with different weights depending on the effect of the brightness temperature uncertainty within an air mass on these radiances at the various cloud levels. For an evaluation of this physically more correct method, we have compared the 3I cloud parameters with time–space collocated clouds from ISCCP in section 4. Conclusions are drawn in section 5. This paper is the second in a series called “Clouds as Seen from Satellite Sounders (3I) and Imagers (ISCCP).” It explains essentially the method of the new 3I cloud scheme. More details about 3I and ISCCP datasets are given in (Stubenrauch et al. 1999a, henceforth paper 1) and in (Stubenrauch et al. 1999b, henceforth paper 3).
2. Original methods
In the IR sounder community, mostly two approaches have been developed for the determination of cloud parameters: 1) the “CO2-slicing” method and 2) “coherence of effective cloud amount” method. Both methods use the four HIRS spectral channels (numbers 4–7) in the 15-μm CO2 band (with peak signal arising from levels between 400 and 900 hPa) and the 11-μm IR window channel (number 8).
a. CO2-slicing method
One philosophy is to concentrate the determination of cloud-top pressure and effective cloud amount on high and midlevel clouds, because one estimates the instrument noise as being too high in the case of low clouds. In this case, Smith and Platt (1978), Susskind et al. (1987), Menzel et al. (1986), and Wylie and Menzel (1989) have used the CO2-slicing method: The cloud-top pressure pcld is determined by minimizing the function S in Eq. (1), slicing through the atmosphere with pairs of adjacent channels:
where νi and νj are two adjacent frequencies, Im is the measured radiance, Iclr is the retrieved clear sky radiance, Icld is the calculated radiance emitted by a homogeneous opaque single cloud layer (ɛ = 1), k are the different pressure levels, and θ is the viewing zenith angle. The effective cloud amount is then calculated by using Eq. (2) for cloud-top pressure pcld. To get complete information on all clouds, the amount of low clouds is determined using additional procedures (Wu and Susskind 1990; Wylie et al. 1994).
b. Coherence of effective cloud amount method
The 3I cloud scheme (Wahiche et al. 1986; Stubenrauch et al. 1996) tries to extract cloud information at all heights by introducing the coherence of effective cloud amount method. Therefore, the CO2-slicing method [Eq. (1)] is applied only in a first step to eliminate cloud levels k with S(pk)/Smin > 5, where Smin is the minimum of function S in Eq. (1). For the remaining cloud levels and for each of the four CO2-band channels (numbers 4–7) as well as for the 11-μm IR window channel, one calculates the effective cloud amount in Eq. (2):
with cloud pressure levels k = 1, 30 and channel numbers i = 4, . . . , 8.
Again, one assumes that all cloudy HIRS pixels are covered by a homogeneous single cloud layer. Before calculating the relative dispersion of the effective cloud amount within the spectral interval of the five HIRS channels, σ[Nɛ(pk, νi)]/Nɛ(pk), at each pressure level k, noisy channels i, with |Nɛ(pk, νi) − Nɛ(pk)| > 0.2, are first eliminated (testing in the following order: channel number 4, 5, 7, 8, and then 6). This means that only channels with an effective cloud amount within 0.2 from the frequency averaged effective cloud amount are used in the calculation at a specific pressure level k. The pressure level is eliminated if only one channel is left. Finally, the cloud top is assigned to the pressure level that gives the minimum relative dispersion σ[Nɛ(pk, νi)]/Nɛ(pk) of the effective cloud amount within the spectral interval of the remaining HIRS channels.
Local comparisons over the North Atlantic (Stubenrauch et al. 1996) with clouds identified from Advanced Very High Resolution Radiometer (AVHRR) measurements (Derrien et al. 1993) showed that even low clouds could be identified by the 3I cloud scheme.
Nevertheless, this method becomes unstable for low clouds, because the denominator in Eq. (2) goes toward zero. Figure 1 shows the difference Iclr − Icld between clear and cloudy radiances as a function of cloud level for the five HIRS spectral channels (numbers 4–8) used in the cloud parameter determination. Cloud levels 1–12 correspond to low clouds, and high clouds start from cloud level 20 upward. Results are shown for two different geographical regions: (a) stratocumulus region off the west coast of Africa and (b) Southern Hemisphere ocean. One observes that the sounder weighting functions are involved in Iclr − Icld, since, for example, Iclr − Icld of channel 4, sounding the higher atmosphere, decreases faster toward lower cloud levels than Iclr − Icld of channel 8, getting information down to the surface.
An error in the temperature profiles propagates into an error in Iclr − Icld, which translates into an uncertainty in Nɛ. From Fig. 2 one observes that this uncertainty in Nɛ is highly asymmetric: it increases with increasing Nɛ, which means with decreasing cloud height, since for the same measurement, the calculated Nɛ increases with decreasing cloud height. Calculations are shown for two examples of measured radiances: in Fig. 2a the Iclr − Im difference between clear sky and measured radiance is small: 2 W m−2 sr−1, whereas in Fig. 2b there is more contrast between clear sky and measured radiance (14 W m−2 sr−1).
Figures 3a and 3b give examples of “IR radiance profiles” in the stratocumulus region: IR radiances as seen by the five HIRS spectral channels (numbers 4–8), with maximum contributions from the atmosphere around 400, 600, 800, 900, and 1000 hPa. The profile with highest IR radiances is for a clear sky situation; then five different IR radiances are calculated for each cloud level, considering a single-layer opaque cloud. The profiles are getting colder with increasing cloud level. One example cloud profile is shown, together with the cloudy profile that was chosen by the 3I cloud algorithm. These are compared to the five IR radiances measured by HIRS. Figure 3a shows a situation in which the shape of the measured and the calculated “IR radiance profile” agree quite well, leading to a solution with a nearly opaque cloud (Nɛ = 93%). In comparison, Fig. 3b shows a case where the measured IR radiance profile and the one of the cloud level yielding the closest IR radiances cross each other. In this case the 3I cloud algorithm has chosen a solution of a much higher cloud with Nɛ = 17%. If uncertainties in measured and calculated radiances are not properly taken into account, a mismatch between cloudy and measured profiles like in Fig. 3b can lead to a bias toward a higher cloud with lower Nɛ.
3. Weighted-χ2 method
The methods used so far (sections 2a and 2b) do not take into account on one side the growing uncertainty in calculating Nɛ with increasing pk (this is when Icld gets close to Iclr, Fig. 2), and on the other side uncertainties in the determined temperature profile and in the measured radiances. Hence, especially in the case of low clouds, results can become biased toward higher clouds with lower Nɛ (Fig. 3).
To develop a method that determines the cloud parameters Nɛcld and pcld in a physically more correct way, we first define a χ2 function, like in (Eyre and Menzel 1989):
One determines Nɛcld and pcld out of the 30 cloud levels k by minimizing χ2. The σ2(νi) should correspond to the variance of the uncertainties. In a first step, we set them to the square of the instrument noise of the five HIRS spectral channels. Like in (Eyre and Menzel 1989), we suppose the noise is channel independent, and we set it to 1 W m−2 sr−1, the exact value chosen for σ2(νi) is irrelevant as long as one is interested only in finding the value of Nɛcld corresponding to the minimum χ2 and not in the magnitude of χ2.
To test the reliability of the results, we need an additional dataset: we compare to time–space collocated ISCCP cloud parameters (see Table 2 and section 4 later). Note that this comparison gives only an estimate of the quality of the 3I cloud parameters, because in the case of different atmospheric temperature profiles or heterogeneous scenes one does not expect the same results (see papers 1 and 3).
By setting σ(νi) to the instrument noise of the five HIRS spectral channels, we obtain about the same results as with the previous method (section 2b). Actually the results are slightly worse, because unlike in section 2b noisy channels have not been eliminated in the calculation.
We know that the different spectral channels used for the calculation of Nɛcld are not equally important at the different cloud levels (e.g., channel 4 is useful only in the higher troposphere, whereas channel 8 is mostly useful in the lower troposphere). Also, the uncertainty in calculating Nɛ grows with increasing pk (Fig. 2). We can introduce in the χ2 this a priori information based on physical knowledge by using empirical weights W(pk, νi) to obtain physically more correct results:
These weights should take into account the weighting functions of the different channels as well as the effect of temperature profile uncertainties on the difference between clear sky and cloudy radiances. They should be dimensionless and depend on spectral channel i and cloud level pk. We leave the instrument noise σ(νi) channel independent (1 W m−2 sr−1).
Minimizing χw2 is equivalent to dχw2/dNɛ = 0, from which one can extract Nɛ as:
One calculates Nɛ for all cloud levels k, and one chooses the solution with the minimum χw2. This method has the advantage of using all channels and all cloud pressure levels. The problem is how to define the different weights W2(pk, νi).
We study the relative standard deviation of the radiance difference Iclr(νi) − Icld(pk, νi) distributions over two different geographical regions in Figs. 4a (stratocumulus) and 4b (Southern Hemisphere ocean) as a function of cloud height for all five HIRS spectral channels. The 150 values (five spectral channels times 30 cloud levels) have been calculated as the standard deviations of the Iclr(νi) − Icld(pk, νi) distributions over the specific geographical region in a time interval of 5 days divided by the mean values of the same distributions. These quantities seem to reflect the usefulness of a spectral channel i at a cloud level k for the determination of Nɛ: the smaller the relative standard deviation of Iclr(νi) − Icld(pk, νi) within a geographical region the higher should be the weight. On one hand one can see that channel 4, sounding the higher atmosphere, leads to a higher variation σ/|Iclr(νi) − Icld(pk, νi)| for cloud levels lower than 15. On the other hand, channel 8, getting part of the surface emission, becomes useless in the determination of Nɛcld for cloud levels below 5, because the variation within the geographical region increases too much.
However, different atmospheric situations (a and b) cause different relative standard deviations of the Iclr(νi) − Icld(pk, νi) distributions. Hence, for a global data processing, one has to find a more systematic way to determine the relative uncertainties in Iclr(νi) − Icld(pk, νi).
One way is to use the TIGR brightness temperature (Tb) standard deviations of the five HIRS spectral channels within each of the five TIGR subsets of airmass types (Achard 1991) as a measure of the temperature profile uncertainty. These vary from 3 K to 10 K, as one can see in Table 1.
The radiance I can be calculated from Tb using the Planck function as follows:
The variation of Tb within a TIGR air mass propagates into a radiance variation dI:
A first estimation of the weights was
where dI is calculated as in Eq. (7) for each of the five TIGR airmass types. Here W2(pk, νi) can be calculated for each observation by the 3I algorithms.
Comparing the results to time–space collocated ISCCP cloud parameters showed that in this way the 3I clouds were much lower than the ISCCP clouds (Table 2). By studying the weights, this comes from the fact that the weights determined in this way vary too much. By taking the square root, the difference in the weights between pk and νi stays limited:
Even better results were obtained by limiting the spread between the 150 weights further:
The allowed maximal value for W2(pk, νi) is 20, and when the solution of Nɛcld exceeds 2, the solution of the second minimum of χw2 is taken. The behavior of W(pk, νi) from Eq. (8) and Eq. (10), respectively, is shown for the two example regions of stratocumulus (Figs. 5a and Fig. 6a, respectively) and Southern Hemisphere ocean (Figs. 5b and Fig. 6b, respectively) as a function of cloud level for the five HIRS spectral channels (numbers 4–8).
Table 2 summarizes the evolution of the 3I method by comparing each time the determined cloud-top temperature and effective cloud amount with time–space collocated ISCCP results, for the two example regions mentioned before.
4. Comparison with ISCCP
For an evaluation of the 3I cloud parameters, we compare them with those obtained from time–space collocated AVHRR measurements processed by ISCCP (Rossow et al. 1996) in July 1987 and January 1988. Both datasets come from the NOAA-10 polar orbiter satellite, which observes the earth at 0730 LT. One has to keep in mind that both datasets are quite different: ISCCP cloud parameters come from one IR window channel (11 μm) and one visible channel (0.6 μm; only during day) measurements with an initial spatial resolution of about 1 km × 4 km at nadir, sampled to 35 km in satellite flight direction and to 30 km perpendicular to flight direction; 3I cloud parameters are determined from the average over all cloudy HIRS pixels (about 17-km spatial resolution at nadir) within a 100 km × 100 km box, but using in addition to the IR window channel four IR CO2-band channels, sounding into the atmosphere.
Figures 7 and 8 show geographical maps of the difference between time–space collocated 3I and ISCCP cloud parameters: (a) cloud-top temperature and (b) effective cloud amount. In the ISCCP dataset, the effective cloud amount has to be calculated from the optical thickness of cloudy pixels inside a 1° grid [see section 3b(3) in paper 1]. Since ISCCP can determine cloud optical thickness, and therefore effective cloud amount, only during day and NOAA-10 morning observations in July and January are made under daylight conditions only in the summer hemisphere, results in Figs. 7 and 8 are from July in the Northern Hemisphere and from January in the Southern Hemisphere. In Fig. 7, the results are obtained from the original 3I algorithm, whereas Fig. 8 shows the results of the new 3I cloud scheme using the weighted-χ2 method. The area-weighted relative occurrence of the different colored grid cells is also given. By considering the cloud-top temperature difference in Fig. 7a, one observes over 66% of the globe an agreement within 7.5 K between ISCCP and 3I. Over land, 3I identifies warmer clouds than ISCCP, a mismatch due to the difficult distinction between partly cloudy low clouds and cirrus. Another feature is lower cloud-top temperatures in the intertropical convergence zone, mostly due to larger cirrus regions determined by 3I. Considering the effective cloud amount difference in Fig. 7b, one observes a systematic underestimation by the original 3I cloud scheme in all stratocumulus regions; in other regions with mostly low clouds the difference is less significant.
By using the weighted-χ2 method, the Nɛ structures in all stratocumulus regions disappear, leading globally to a much better agreement between ISCCP and 3I. The fraction of the globe over which the agreement between ISCCP and 3I is within 10% increases from 56% to 71%. The cloud-top temperatures in Fig. 8a do not change considerably, with an agreement within 7.5 K between ISCCP and 3I over 65% of the globe (or over 50% of the globe within 5 K). The mean values and standard deviations of area-weighted histograms of these differences are improved from −1.7 ± 10.3 K to −1.0 ± 10.4 K and from −4.4% ± 13.2% to −0.5% ± 10%. Note again that this global comparison gives only an estimate of the quality of the 3I cloud parameters, because in the case of different atmospheric temperature profiles or heterogeneous scenes one does not expect the same results (see papers 1 and 3). Disagreements over the Sahara, Antarctica, and Northern Hemisphere land can mostly be explained by misidentification between cirrus and partly cloudy low clouds. Comparisons of 3I and ISCCP zonal-mean cloud parameters are discussed in section 3b in paper 1.
The different cloud parameters can be merged into one variable: the cloud type. Table 3 shows the total match-up frequencies between 3I cloud types and time–space collocated ISCCP cloud types in six different geographical regions. For this matching study, four cloud types have been defined: high opaque (pcld < 440 hPa and Nɛ > 90%), cirrus (pcld < 440 hPa and Nɛ < 90%), midlevel (440 hPa < pcld < 680 hPa), and low-level cloud (pcld > 680 hPa). Agreement is achieved when the 3I cloud type is the same as the most frequent ISCCP cloud type within the same 1° grid. More details about the comparison can be found in section 4 of paper 1 and in paper 3. We see improvement of the match-ups with ISCCP going from original 3I to new 3I (weighted-χ2 method) in all of the six geographical regions. The magnitude of this improvement varies from 6% (NL: Northern Hemisphere Land) to 27% (STN: Northern marine stratocumulus). The cloud type match-up increases further by 10%–15% for situations in which the ISCCP pixels have the same cloud type (out of the four mentioned above) within the 1° grid, leading to a final cloud type match-up between 56% (NL) and 87% (STN).
The iterative process of comparing time–space collocated ISCCP data with 3I cloud parameters has led to a review of cloud parameter determination methods from IR sounder measurements and to a further development of these methods. By taking into account the effect of temperature profile uncertainties on the difference between clear sky and cloudy radiances for different cloud levels and spectral channels, the newly developed 3I weighted-χ2 method yields nonbiased cloud parameters at all cloud heights for homogeneous cloud types. The comparison with ISCCP reveals a considerable improvement of the 3I cloud parameters, especially in the stratocumulus regions. The remaining disagreements with ISCCP can be explained by grid heterogeneities (vertical and horizontal) or by difference in cloud detection (see papers 1 and 3).
Within the framework of the National Oceanic and Atmospheric Administration National Aeronautics and Space Administration Pathfinder Program (Maiden et al. 1994), eight years of TOVS data have already been processed by the 3I algorithms, using the weighted−χ2 method for cloud parameter determination. The data processing is still going on at the rate of 2 months of data per day, providing atmospheric temperature and water vapor profiles as well as cloud and surface parameters at a spatial resolution of 1°.
The weighted-χ2 method can be used in other inversion algorithms that transform measured IR sounder radiances into cloud parameters. In each case, the empirical weights, taking care of the temperature profile uncertainties on the difference between clear sky and cloudy radiances for different cloud levels and spectral channels, have to be reevaluated so that they can be calculated automatically by the corresponding inversion algorithm.
The 3I processing has been performed on the computers of the Institut du Développement et des Ressources en Informatique Scientifique of the Centre National de la Recherche Scientifique. We would like to thank William B. Rossow at Goddard Institute of Space Studies for many fruitful discussions, as well as the two reviewers for their useful comments.
Corresponding author address: Dr. Claudia J. Stubenrauch, Laboratoire de Météorologie Dynamique, Ecole Polytechnique, 91128 Palaiseau Cedex, France.