1. Introduction
The Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), on board the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite, was launched in April 2006 (Winker et al. 2007), in formation with the CloudSat satellite, as part of the A-Train constellation of satellites (Stephens et al. 2002). The main objectives of the CALIPSO mission are to provide global measurements of cloud and aerosol spatial distributions and optical properties to complement current measurements and improve our understanding of weather and climate. The availability of a global, multiyear set of vertically resolved measurements of the earth’s atmosphere should ultimately lead to great improvements in both weather and climate models. CALIOP is the first satellite-borne lidar optimized specifically for aerosol and cloud measurements, and is also the first polarization lidar in space. CALIOP is a dual-wavelength, polarization-sensitive elastic backscatter lidar. Its light source is an Nd:YAG pulsed laser operating at 1064 and 532 nm. The transmitted laser beam is linearly polarized, and two polarization-sensitive 532-nm receiver channels measure the degree of linear depolarization of the return signal [see details of the lidar instrument provided in Hunt et al. (2009)]. Using the two 532-nm receiver channels and a channel measuring the total 1064-nm return signal, CALIOP measures the detailed vertical distribution of aerosols and clouds along with their microphysical and optical properties.
Since June 2006, CALIOP has been acquiring a global dataset that is being processed continuously at NASA’s Langley Research Center. An overview of the CALIOP science data processing architecture is provided in Winker et al. (2009). The routine lidar data processing currently includes two levels: 1 and 2 (King et al. 2004). In the level 1 data processing, the lidar backscatter data are geolocated and calibrated. The resulting altitude-resolved profiles of attenuated backscatter coefficients (Hostetler et al. 2006; Powell et al. 2009) are reported in the CALIOP level 1B data products. These level 1 profiles are further analyzed in level 2 processing to derive the optical and physical properties of clouds and aerosols (Vaughan et al. 2004).
The level 2 processing algorithms include three primary modules: a layer detection algorithm known as the selective, iterated boundary locator (SIBYL), the scene classification algorithms (SCA), and the hybrid extinction retrieval algorithm (HERA). First, the layer-detection algorithm, SIBYL, finds features (clouds, aerosols, surface and subsurface, stratospheric layers, etc.) by searching for regions of enhanced signal in the attenuated backscatter profiles provided by the level 1 processing (Vaughan et al. 2009). After finding features, mean values of the 532- and 1064-nm attenuated backscatter, attenuated total color ratio, and volume depolarization ratio are computed for each atmospheric feature detected. These layer optical properties, along with physical properties such as top and base heights, latitude, longitude, etc., are reported in the level 2 layer products. Based on these optical and physical properties, each atmospheric feature is then classified according to type by the SCA (Liu et al. 2005). A primary function of the SCA is to select an extinction-to-backscatter ratio (also known as the lidar ratio) for each layer based on the classification results. The lidar ratio is a key parameter that is used in HERA to retrieve particulate (i.e., aerosol and/or cloud) extinction and backscatter coefficients (Young and Vaughan 2009). An accurate retrieval of layer optical properties by HERA depends critically on the selection of proper lidar ratios, which in turn requires an accurate scene classification.
SCA is composed of three main submodules: cloud–aerosol discrimination (CAD) (Liu et al. 2004), aerosol subtyping (Omar et al. 2005, 2009, hereafter OmarSI), and cloud ice-water phase discrimination (Hu et al. 2009, hereafter HuSI). The CAD algorithm is a multidimensional probability density function (PDF) based approach (Liu et al. 2004). The attributes (dimensions) included are the mean attenuated backscatter at 532 nm, the layer-integrated 1064/532-nm attenuated backscatter ratio (color ratio), and the midlayer altitude. The PDFs used in the first data release were developed based on airborne lidar measurements and measurements obtained during the Lidar In-Space Technology Experiment (LITE; Winker rt al. 1996). A new set of PDFs has been developed based on a manual classification of one full day of CALIOP measurements and is used in the current CALIOP data release (version 2). In addition, a new depolarization ratio test has also been incorporated into the version 2 CAD algorithm. In this paper we focus on the version 2 CAD algorithm. The algorithm overview and an initial assessment of the algorithm performance will be presented.
2. Algorithm outline
In this section, we provide an overview of the theoretical basis of the CALIOP CAD algorithm and describe salient aspects of the implementation. Complete details can be found in Liu et al. (2004, 2005).
a. Theoretical basis
When selecting the attributes used to discriminate clouds from aerosols, a very strong preference should be given to the intrinsic properties of features, that is, to those physical and optical characteristics of the feature that depend solely on the layer composition and are independent of particle concentration. However, in many cases—and this is especially relevant for backscatter color ratios—estimates of the intrinsic properties of a feature can only be obtained after computing (and subsequently correcting for) the extinction profile within the feature. The CALIOP CAD algorithm thus faces a dilemma: deriving the extinction solution requires an estimate of the feature lidar ratio, and this estimate in turn is derived from an assessment of layer type (e.g., OmarSI). Because the CAD is the first step of the scene classification analysis that is performed before the extinction coefficient is retrieved by HERA, some intrinsic attributes (e.g., the particulate backscatter color ratio) are not available for use and, thus, are replaced by the corresponding extrinsic feature properties (e.g., the attenuated backscatter color ratio).
b. Development of PDFs
The cloud and aerosol PDF database used in the prelaunch version for the first data release was developed mainly based on regional airborne lidar measurements acquired before CALIPSO’s launch (Liu et al. 2004). The database has since been updated based on expert manual classification of all layers detected during one full day of data acquired by CALIOP on 12 August 2006. In contrast to the automated production CAD processing that performs feature classifications in isolation, based on the properties of single vertical layer, this expert reclassification identifies features by simultaneous inspection of two-dimensional vertical–horizontal images of several different lidar measurements (attenuated backscatter, attenuated backscatter color ratio, and volume depolarization ratio). That is, in addition to χ′, β′, and midlayer altitude, this expert reclassification also makes use of additional information such as layer structures, textures, connections with the surrounding layers, and geographic locations in the assessment of feature types. Due to the large amount of data that must be examined (e.g., ∼0.3 million 5-km layers for the selected day), manual classification of layer types can be very time consuming. For this reason, a computer tool has been developed to facilitate this kind of image-based manual identification of features by immediately and interactively providing the quantitative, measured data associated with each layer. Despite this computerized assistance and the application of spatial coherence tests, there remain some cases that cannot be identified with high confidence. These cases include layers at edges of clouds, mixed layers of cloud and aerosol, and layers below dense clouds whose optical properties are not correctly measured due to the detection issues such as the detector transient response, multiple scattering, etc. These layers, labeled as no confidence in the manual classification, account for ∼2% of the selected day’s manually classified data (see Table 1).
The updated PDFs based on the one day of manual classification are used in the version 2 algorithm for the current data release. We note that even these new PDFs may not completely characterize the aerosol and cloud distributions for all seasons and all locations, and that more comprehensive PDFs are being developed based on more data for future data releases. However, as demonstrated in section 3, the CAD algorithm with the updated PDFs works well for most cases. Figure 1 presents (a) a combination scatterplot–contour plot in the backscatter–color ratio space and (b) the occurrence distribution as a function of attenuated backscatter of clouds (blue) and aerosols (red) within the 0–1-km altitude range measured by CALIOP on 12 August 2006. As seen in Fig. 1, aerosols generally have smaller backscatter intensities and attenuated backscatter color ratios when compared with clouds, and the two classes are largely separated in the backscatter–color ratio space, as expected by the prelaunch study (Liu et al. 2004) based on the Optical Properties of Aerosols and Clouds (OPAC) software package (Hess et al. 1998). While aerosols have a single-mode distribution centered at ∼0.003 km−1 sr−1 of attenuated backscatter and ∼0.45 of color ratio, clouds have a bimodal distribution centered, respectively, at ∼0.1 and ∼0.01 km−1 sr−1 of attenuated backscatter and ∼0.95 and ∼1 of color ratio. An overlap (<10% of total features) is seen mainly in a region of 0.004–0.01 km−1 sr−1 for attenuated backscatter and of 0.5–0.9 for the color ratio. In this overlap region, an unambiguous classification will not be possible and some misclassifications will occur.
Dense aerosols such as dust and smoke over or near the sources can contribute to the overlap. Dust aerosols usually have large backscatter color ratios, comparable to those of clouds, due to the large size of the dust particles. The lidar signatures for dense dust aerosols can look very similar to those of clouds in terms of attenuated backscatter and color ratio. Dense smoke is another case for which both the attenuated backscatter and the color ratio can closely resemble those characteristic of clouds. For smoke, the particulate backscatter color ratio is not generally as large as that for dust or cloud. However, the smoke extinction at 532 nm is much larger than at 1064 nm (2.5–5 times) (Pueschel and Livingston 1990). When measuring dense smoke layers, the range-dependent reduction in lidar signals due to attenuation occurs much faster at 532 nm than at 1064 nm. As a result, the attenuated color ratio will grow quickly, resulting in a large layer-averaged attenuated color ratio. Hygroscopic aerosols located in regions of high humidity (Charlson et al. 2007) may also fall into the overlap region.
Optically thin clouds, including horizontally small-scale clouds and mixed layers of cloud and aerosol (vertically adjacent cloud and aerosol layers or clouds embedded in aerosol), can also contribute to the overlap. We note, however, that these mixed-type features should be classified as “cloud” by convention (refer to section 2e) in the current CALIOP scene classification.
In addition to the natural spread of cloud and aerosol distributions, signal noise can widen the cloud and aerosol distributions and hence contribute to the overlap in the cloud and aerosol PDFs (Liu et al. 2004). The spread in the scatterplot shown in Fig. 1 is caused partly by signal noise. Because the signal-to-noise ratios (SNRs) for the measurements from which the original version 1 PDFs were derived are substantially higher than the CALIOP SNR, the version 1 processing included a module that explicitly compensated for these additional noise effects. The version 2 PDFs were developed based solely on the CALIOP measurements, and because the noise broadening effect is thus inherent in the revised distributions, this noise compensation module is disabled in the version 2 data release. We note that the PDFs for daytime and nighttime measurements are slightly different; the daytime PDFs have a somewhat broader distribution than the nighttime PDFs due to the increase in background noise during the day. However, disabling the noise compensation module has had a negligible impact on the classification results, although it can produce some additional uncertainty in the confidence value estimated for each classification.
c. Enhancements to the version 2 algorithm
For the initial classification of tropospheric features, the algorithm uses a threshold value γ′th to identify features that have very large values of γ′532. Clouds containing large fractions of horizontally oriented ice crystals are characterized by large integrated attenuated backscatter, large color ratio (close to unity), and very low depolarization ratio (close to zero) (Platt 1978). Additional threshold tests based on χ′, δ, and temperature are performed on those clouds with large γ′532 to identify cloud returns that are obviously affected by horizontally oriented ice crystals. Cloud layers identified as containing horizontally oriented ice crystals are assigned a special CAD score of 102. A more sophisticated algorithm will be incorporated in a future data release to provide improved detection of oriented ice crystals (HuSI). The classification of features with γ′532 > γ′th that do not pass the oriented ice crystal tests is less certain. The large γ′532 values could be artifacts due to an overestimation of the attenuation for overlying layers by HERA (Young and Vaughan 2009). Due to this uncertainty, these features are assigned a special CAD score of 103. All features having special CAD scores are reported in the cloud-layer products. The interpretation of the CAD scores, as well as CAD quality assurance (QA) flags reported in the vertical feature mask (VFM), is given in Table 2.
For the tropospheric features with γ′532 < γ′th, the CAD algorithm computes f using Eq. (3), based on β′532, χ′, and z. If f > 0, the feature is classified as cloud. Otherwise, an additional test is conducted that compares the layer-integrated volume depolarization ratio δ with a latitude-dependent threshold δth as defined in the top panel of Fig. 3. If δ < δth, the feature is defined as aerosol, with a CAD score equal to (100 times) the magnitude of f; otherwise, the feature is classified as cloud with a special CAD score of 101.
d. Special CAD scores
In addition to the standard CAD scores, which range between −100 and 100, the special CAD scores of −101, 101, 102, and 103 are also reported in the CALIOP cloud-layer products. These special scores distinguish features/artifacts identified by other tests as described in Fig. 2. Figure 5 presents an example of (a) the occurrence distribution and (b) the cumulative occurrence frequency of the CAD score derived from one month (July 2006) of the CALIOP 5-km-layer products data (version 2.01). Table 3 also lists frequencies for selected CAD score ranges and the special CAD scores.
The special CAD score of −101 indicates features that have a negative mean attenuated backscatter. These features should not occur based on the algorithm design. However, they occurred in the current layer products due to some unknown programming reasons. The fraction of these features is very low, ∼0.1% of the total tropospheric features, including clouds and aerosols for the one month of data shown in Fig. 5, and they should be eliminated in future data releases. Layers flagged −101 should never be used. The rest of the special scores (101, 102, and 103) correspond mostly to genuine atmospheric features. As described earlier, the special score of 101 is assigned to the features that are classified as cloud by the depolarization ratio test (the δ switch in Fig. 2). About 0.7% of the total features have CAD scores of 101, and these mostly occur at high latitudes. The special score of 102 is assigned to those features that are highly likely to contain large fractions of oriented ice crystals. These features account for ∼0.2% of the total features. We note that the one month’s worth of the data analyzed here were acquired when the lidar was operated with a nadir-pointing angle of 0.3°. The lidar-pointing angle was adjusted to 3° recently, and thereafter the fraction of horizontally oriented ice crystals should be considerably reduced. Features with special CAD scores of 101 and 102 can confidently be included in scientific applications of the data. Features with the special score of 103 may be a mixture of oriented ice crystals with other types of cloud particles, or a feature whose optical properties are artificial due to improper data processing, or may not be an atmospheric feature. Therefore, caution should be taken in the interpretation and use of this class of features.
e. Some conventions
Conventions used in the version 2 CAD algorithm are provided in the following:
(a) Only two classes of tropospheric features are defined in the CAD algorithm: “cloud” and “aerosol.” The aerosol class is defined as all airborne particles, excluding activated water droplets and frozen ice crystals. This class includes all commonly defined aerosol types (maritime, continental, dust, and smoke) (OmarSI). The cloud class is defined as an air mass containing any activated water droplets and/or ice crystals. All normally defined clouds (high-, middle-. and low-cloud families) are included in the cloud class.
(b) Based on the definition above, a mixed layer of cloud and aerosol that is erroneously identified as a single layer by the CALIOP feature finder is classified as a cloud.
(c) Fogs, in either liquid form or frozen form, are included in the cloud class.
(d) Diamond dust also belongs to the cloud class because it consists mostly of falling ice crystals.
(e) Hazes belong to the aerosol class, whereas mists are classified as clouds.
3. Initial performance assessment
a. General performance
Performance assessments of the version 2 CAD algorithm are ongoing. In this section we present several initial results. Overall, the CAD algorithm works well in most cases; manual verification of the classifications for a full day of 5-km data (299 683 features) suggests that the success rate is larger than 90% (see Table 1). Table 1 summarizes the assessment results based on the 1-day 5-km data. We note that this 1-day assessment may not be completely representative of the whole CALIOP dataset, and only provides a general idea about the CAD performance. The percentage of clouds misclassified as aerosol for daytime is smaller than that for nighttime because the daytime feature finding misses more optically thin clouds that fall in the overlap region of PDFs and are, thus, more likely to be misclassified. On the other hand, the percentage of aerosols misclassified as cloud is larger for daytime measurements. This occurs because a large fraction of the optically tenuous aerosols that fall outside of the deep overlap region of the PDFs, and therefore can be correctly classified as aerosol, are not reliably detected during daytime operations. For the one month of data shown in Fig. 5 and Table 3, ∼83% of aerosols and ∼95% of clouds have a CAD score magnitude greater than 70, which corresponds to the “high” confidence category (Table 2), indicating that clouds and aerosols are well separated.
An example of the CALIOP cloud and aerosol discrimination is given in Fig. 6. The data were acquired by CALIOP on 27 June 2007 from a nighttime orbit across central and southern Europe and North Africa. In this scene, a spatially extensive Saharan dust layer of moderate optical thickness was observed, extending from ∼39°N all the way to the right-hand-side end of the images at ∼7.4°N. This dust layer is easily identified from the depolarization ratio measurement (green–yellow–orange colors in the middle panel of Fig. 6). Dust aerosol can be well distinguished from other aerosol types based on a test on the volume depolarization ratio using a threshold of 0.06 (Liu et al. 2008a). Vertically, this dust layer extends from the surface up to several kilometers (>5 km for its highest part) and begins to detach from the surface north of 34°N (middle panel of Fig. 6). Above 34°N, other aerosol types are also observed (green–yellow–orange–red-colored features below 3 km in the top panel of Fig. 6). Aerosols between 34° and 46°N appear to be mixed with dust particles (greenish colors in the middle panel of Fig. 6). In addition, cirrus clouds in high altitudes (>8 km, red–yellow–gray colors in the top panel of Fig. 6) and relatively small-scale clouds (red–gray–white colors) in mid- and low altitudes are observed.
From the VFM image shown in the bottom panel of Fig. 6, it is seen that the CAD algorithm has identified most features correctly. In this particular scene, misclassifications happen mainly at the edges of clouds and surrounding areas, though they are not clearly visible. The few mixed layers of dust and embedded cloud (blue strips near the right-hand-side end in the bottom panel of Fig. 6) are, by convention, correctly classified as cloud. Note that when a cloud is adjacent to or embedded in an aerosol layer, the heterogeneous mix of layers can sometimes be identified as a single layer (Vaughan et al. 2009).
The features colored red in the VFM image represent those features with CAD score magnitudes smaller than 20 or equal to 103 or −101, that is, low- or no-confidence features. These “red” features account for about 9% of the total tropospheric features based on the 1-month data analysis (Table 3). In this scene they are mostly artifacts due to the noise in the signal, multiple scattering effects, or due to artificial signal enhancements caused by a nonideal detector transient response or an overestimate of the attenuation due to overlaying layers. These erroneous “pseudofeatures” occur mainly under dense clouds. They are neither cloud nor aerosol; however, because they are not properly assessed and interdicted by the feature finder, the CAD algorithm nonetheless attempts to assign them to one class or the other.
b. Misclassification types
Despite the generally high accuracy of the CAD algorithm, several types of misclassifications still occur with some frequency. Among these, the most prevalent are very dense dust and smoke aerosols lying over and close to the source regions. Because the CAD algorithm operates on individual layers, without a contextual awareness of any surrounding features, it can happen that small but strongly scattering regions within an extended aerosol layer can occasionally be labeled as cloud. This occurs because the optical properties (the attenuated backscatter and color ratio) of dense dust and smoke layers are similar to what would be expected for clouds at the same altitude. These misclassifications are often apparent from studying the level 1 browse images.
We present an extreme example of the misclassification of very dense dust layers in Fig. 7. These data were acquired on the daytime side of an orbit on 30 March 2007 as depicted by the CALIOP ground track shown in Fig. 7f. Attenuated backscatter measured by CALIOP at 532 nm and the CALIOP vertical feature mask are presented in Figs. 7a and 7b, respectively. A second vertical feature mask derived from the coincident CloudSat radar measurements (Stephens et al. 2008) is presented in Fig. 7c to help identify dense clouds.
In late March 2007, a large dust storm originated in the Gobi Desert and, in the days that followed, the resulting dust plume was transported for long distances into the North Pacific (Yumimoto et al. 2008). CALIOP observed extensive dust plumes on 30 March 2007 over the Gobi area (the yellow–orange–red–gray-colored features north of ∼46°N and below ∼8 km in Fig. 7a). These dust plumes were not observed by CloudSat (Fig. 7c) because the dust particles are small compared with the radar wavelength (millimeter). A very dense dust plume (gray-colored features in Fig. 7a) is seen between 38° and 46°N and below ∼4 km, where the lidar signal becomes totally attenuated (no surface signals are detected) and the optical depth therefore should be larger than ∼3. This dense dust plume has been misclassified by the CAD algorithm, because its optical properties are similar to what would be expected for clouds (note that the gray color of the dust layer in Fig. 7a is almost the same as the high dense cloud at 9–12 km between 39.4° and 41°N). An opaque cloud is seen on the top of the dense dust plume at an altitude of 4–5 km between 44° and 44.6°N, as observed by CloudSat (Fig. 7c). Dust particles were also lofted to higher altitudes and a dust layer is observed at 4–8-km altitude. This layer has also been misclassified, because, while it is only moderately dense, it has been lofted to relatively high altitudes. Aerosol layers of moderate to high backscatter intensity are not expected to be present at relatively high altitudes by the version 2 CAD algorithm. Currently, one set of PDFs is used globally in the CAD algorithm. In the future, latitudinally, regionally, and/or seasonally dependent sets of PDFs can be developed and used, which should greatly reduce this type of misclassification. Note that the moderate dust layer north of 37°N and below ∼4 km has been correctly classified.
Clouds (mainly gray–white-colored features) are also observed by CALIOP in Fig. 7a. All clouds in this scene have been identified correctly by the CAD algorithm. This is evident when comparing the CALIOP vertical feature mask in Fig. 7b with the CloudSat vertical feature mask in Fig. 7c. We note that, besides dust aerosols, the CloudSat radar also could not detect optically thin high clouds (e.g., the orange–red-colored feature at the left-hand-side upper corner near 12 km in Fig. 7a). On the other hand, CALIOP could not penetrate the entire layer of dense clouds due to the large attenuation of the clouds at the lidar wavelengths (532 and 1064 nm). However, as designed, CloudSat and CALIOP together can provide more complete measurements of aerosol and cloud distributions.
As mentioned earlier, the addition of more attributes can increase the degree of separation between clouds and aerosols and, consequently, improve the classification. As an example, we have derived layer-effective lidar ratios for the total attenuated dust layer and for the clouds between 41° and 50°N, using an approach for totally attenuated layers described by Platt et al. (1999). We present the results in Fig. 7d. The lidar ratio is the ratio of extinction to backscatter coefficients, and the layer effective lidar ratio is the ratio of integrated particulate backscatter to layer optical depth in the presence of multiple scattering. As seen in Figs. 7d and 7e, the dust and cloud layers are clearly separated into two groups by the effective lidar ratio: the dust layers have a value of 19.7 ± 2.2 sr and the cloud layers have a value of 9.5 ± 1.5 sr. The lidar ratio for water clouds is ∼18 sr (Pinnick et al. 1983). The effective lidar ratio can be reduced significantly in the presence of multiple scattering as demonstrated by the CALIOP measurement in cirrus clouds (Vaughan et al. 2008). The values (9.5 ± 1.5 sr) presented in Fig. 7e are typical of water clouds in the presence of multiple scattering. Lidar ratios for dust are normally higher, ranging from 30 to 80 sr at 532 nm (e.g., Liu et al. 2002; Cattrall et al. 2005; Muller et al. 2007). A value of ∼41 sr was observed by CALIOP for a moderate Saharan dust layer over the eastern North Atlantic, and a slightly larger value was measured over the Gulf of Mexico for transported Saharan dust by the NASA Langley Research Center’s High-Spectral-Resolution Lidar (HSRL) (Liu et al. 2008b). The low lidar ratio derived for this dust event may be due to differences in the particle sizes, as the particles occurring in dust storms immediately over the source regions should normally be much larger than those that are lofted and transported over large distances. The low lidar ratio may also be due to the multiple scattering. The dust layer in this example has a very high optical thickness, which can produce significant multiple scattering. The presence of multiple scattering can reduce the effective lidar ratio; the more significant the multiple scattering, the smaller the effective lidar ratio.
We have performed an analysis for one month’s worth of data (July 2006), attempting to quantify the misclassification of dense dust. This was done based on the fact that, when a dust layer is misclassified as cloud, the layer will likely be classified as an ice cloud in the downstream data processing by the ice–water discrimination algorithm (HuSI) using the depolarization ratio measurement. Dust storms occur normally in dry and hot air conditions over the source regions during daytime due to the enhanced convection in a deepened boundary layer (e.g., Ackerman and Cox 1987; N’Tchayi Mbourou et al. 1997). By checking the midlayer temperature, the misclassified dense dust layers can be identified in most cases. In our analysis to evaluate misclassified dust layers, if a layer in the dust belt region defined in this paper is classified as ice cloud and its midlayer temperature is larger than 0°C, this layer is identified as “hot cirrus.” The hot cirrus feature is not an ice cloud but a misclassification of a dust or water cloud layer or a mixture of dust and cloud. Our analysis showed that the fraction of hot cirrus is ∼1.4% of total tropospheric features. Among these hot cirrus features, mixed layers of dust and cloud should be classified as cloud by convention (section 2e). Further analyses have also been performed to partition the hot cirrus features. This analysis checks the CloudSat measurement: if no cloud is seen in the nearest CloudSat profiles, this hot cirrus layer is identified as a misclassified dust layer. Misclassification of water clouds as ice has been assessed by applying the same temperature test to the hot cirrus found in regions over remote southern oceans, where dust is not expected to be present. Applying these additional analyses showed that misclassified dust layers represent <50% of the hot cirrus features, corresponding to ∼0.7% of the total features.
Relatively dense smoke, particularly over or near source regions, is another aerosol type that can be occasionally misclassified. Its occurrence frequency is estimated to be smaller than dense dust, though we have not quantified this type misclassification.
c. Zonal distribution
Zonal distributions of cloud and aerosol features derived from the July 2006 CALIOP layer products data are presented in Fig. 8. Three modes are seen in the cloud distribution. One mode is peaked at ∼10°N, corresponding to the intertropical convergence zone (ITCZ). The other two modes are peaked at ∼65°S and in the Arctic. It is seen that aerosols are distributed mostly in low latitudes. The major dust sources are mainly located over the continents in Northern Hemisphere (Prospero et al. 2002). The dust aerosols can be transported for long distances across the tropical North Atlantic and North Pacific Oceans or to the Indian Ocean. Polluted aerosol is another aerosol type that can dominate some industrial areas in the Northern Hemisphere (Herman et al. 1997; Torres et al. 1998). These aerosol types contribute largely to the aerosol mode in the Northern Hemisphere. Smoke aerosols due to biomass burning in central and southern Africa show an annual pattern that reaches a maximum in the Southern Hemisphere winter and that can dominate the aerosol loading in the tropical South Atlantic (Herman et al. 1997; Torres et al. 1998). This is responsible for the peak seen in the tropical Southern Hemisphere. Maritime aerosols with large backscatters are often observed by CALIOP over the southern waters of the Atlantic and Pacific Oceans, and these contribute to the long tail of the southern aerosol mode.
Approaching the polar regions, the frequency of aerosol occurrence drops quickly. This behavior is expected, as the polar regions are remote from sources and aerosol layers with backscatter intensities above the CALIOP detection threshold are not often present. However, even in this relatively clean region of the globe, aerosol layers are still identified with some regularity. Polar clouds are often low-lying, optically thin, and spatially diffuse clouds, and can be composed of ice, water, or a mixture of both. The spatial distributions and optical properties of polar clouds are distinctly different from the distributions encountered at lower latitudes, and this can cause difficulties for the CAD algorithm. At present, a single set of latitude-independent PDFs is used globally for all CAD decisions. This set of PDFs is more representative of the cloud and aerosol distributions at lower latitudes than at higher latitudes. Consequently, some misclassifications of optically thin clouds (or edges of such clouds) can occasionally occur. In particular, thin ice clouds, which in the polar regions can extend from the surface to several kilometers in altitude, are sometimes misclassified as aerosol. We plan to develop latitude-dependent PDFs for future data releases to improve the classification in the polar regions.
4. Summary
In this paper, we have reviewed the theoretical basis of the CALIOP cloud and aerosol discrimination (CAD) algorithm, and described the enhancements incorporated into the version 2 algorithm that is used in the current data release (release 2). We also presented preliminary assessment results of the CAD performance based on one full day of expert manual classification and one month’s worth of CALIOP 5-km cloud and aerosol layer products. Overall, the CAD algorithm works well in most cases. The 1-day manual verification of the classifications suggests that the success rate is in the neighborhood of 90% or better.
The CAD error budget is characterized by several typical misclassification scenarios. Among these, the most prevalent are dense dust and smoke over or close to the source regions. Because the mean attenuated backscatter and mean attenuated total color ratio of these layer types are both relatively large, and thus similar to what would be expected for clouds, these features can be occasionally misclassified as cloud. The analysis of the 1-month data (July 2006), combined with the corresponding CloudSat measurements, showed that the misclassification of dust as ice clouds is ∼4% of the total features (i.e., clouds and aerosols) found. Smoke layers are misclassified as cloud less frequently than are dust layers. Though the fraction is small compared with the entire aerosol features found globally, the misclassification of optically thin clouds as aerosols may be significant if the analysis is limited in the polar regions. In the polar regions, many optically thin clouds, both ice and water, are encountered in the low atmosphere. These clouds, which can extend from the surface to several kilometers in altitude, are sometimes misclassified as aerosol.
CAD scores should be used to screen the CALIOP layer products to minimize the impacts of misclassifications. Several data screening approaches can be considered, including the following:
(a) As described earlier, features that have a special CAD score of −101 and 103 should be excluded from the data analysis in the scientific application.
(b) Detection of features underneath attenuating layers becomes more uncertain for a variety of reasons: low signal, incorrect attenuation correction, or instrumental artifacts. Confidence scores should be examined and features with score values less than 20 should be considered to be unreliable. A threshold of 20 has been used in generating the standard CALIOP vertical feature mask browse images (see, e.g., the bottom panels in Figs. 6 and 7b). A larger threshold between 20 and 90 may be selected for cloud features, depending on the application of the data. A threshold of 90 will remove ∼4.5% of the complete cloud dataset (Fig. 5b and Table 3).
(c) Misclassified dense dust layers in the defined dust belt can be identified using the temperature approach as described in section 3b. That is, for a cirrus cloud in the geographic region of 0°–50°N and 40°W–130°E, its midlayer temperature T should be compared with a threshold Tth: if T > Tth, this layer is very likely a misclassified dust layer and, thus, should be rejected from any data analysis devoted to clouds. We used Tth = 0°C in this paper, together with the CloudSat measurements, to evaluate the misclassification of dust aerosols.
(d) For dense smoke features misclassified as clouds, the sharp gradient of the attenuated total color ratio with respect to increasing layer penetration depth can be used to establish the correct classification. Due to strong spectral differences in absorption efficiency, the attenuation of smoke is much larger at 532 than at 1064 nm, and thus the backscatter signals from smoke decrease much faster at 532 than 1064 nm, causing the attenuated total color ratio to grow rapidly to large values (>1.5) in the lower regions of a smoke layer. In general, a dense smoke layer can be identified by the nature of the low-volume depolarization ratio and the high total color ratio (Liu et al. 2008a). In cases where the effective lidar ratio can be estimated, this parameter can also be used to separate smoke from clouds.
The CAD algorithm will be modified to improve the CALIOP cloud and aerosol classification for future data releases. Several things can be considered. First, a set of PDFs that is latitudinally and seasonally dependent will be developed. This step alone should greatly improve CAD performance. Second, to better separate dense dust layers from clouds at lower altitudes, we plan to incorporate volume depolarization ratios into the PDFs developed for future data releases. Recent offline tests have shown that the addition of the volume depolarization ratio can effectively distinguish dense dust layers from clouds at lower altitudes. The CloudSat measurements would also provide useful ancillary data to help in identifying dense aerosols that may be misclassified as cloud.
Acknowledgments
The authors wish to acknowledge the many and substantial contributions of the members of the CALIPSO Lidar Science Working Group. In particular, we thank Kam-Pui Lee, the chief architect of the analysis codes that produce all of the level 1 lidar data products from which the level 2 data products are derived, for his valuable support.
REFERENCES
Ackerman, S. A., and Cox S. K. , 1987: Surface weather observations of atmospheric dust over the Southwest summer monsoon region. Meteor. Atmos. Phys., 41 , 19–34.
Cairo, F., Donfrancesco G. , Adriani A. , Pulvirenti L. , and Federico F. , 1999: Comparison of various linear depolarization parameters measured by lidar. Appl. Opt., 38 , 4425–4432.
Cattrall, C., Reagan J. , Thome K. , and Dubovik O. , 2005: Variability of aerosol and spectral lidar and backscatter and extinction ratios of key aerosol types derived from selected Aerosol Robotic Network locations. J. Geophys. Res., 110 , D10S11. doi:10.1029/2004JD005124.
Charlson, R., Ackerman A. , Bender F. , Anderson T. , and Liu Z. , 2007: On the climate forcing consequences of the albedo continuum between cloudy and clear air. Tellus, 59 , 715–727.
Herman, J. R., Bhartia P. K. , Torres O. , Hsu C. , Seftor C. , and Celarier E. , 1997: Global distribution of UV-absorbing aerosols from Nimbus7/TOMS data. J. Geophys. Res., 102 , 16911–16922.
Hess, M., Koepke P. , and Schult I. , 1998: Optical properties of aerosols and clouds: The software package OPAC. Bull. Amer. Meteor. Soc., 79 , 831–844.
Hostetler, C., and Coauthors, 2006: CALIOP algorithm theoretical basis document—Part 1: Calibration and level 1 data products. PC-SCI-201, Release 1.0, NASA Langley Research Center, Hampton, VA, 66 pp. [Available online at http://www-calipso.larc.nasa.gov/resources/pdfs/PC-SCI-201v1.0.pdf].
Hu, Y., and Coauthors, 2009: CALIPSO/CALIOP cloud phase discrimination algorithm. J. Atmos. Oceanic Technol., in press.
Hunt, W., Winker D. , Vaughan M. , Powell K. , Lucker P. , and Weimer C. , 2009: CALIPSO lidar description and performance assessment. J. Atmos. Oceanic Technol., 26 , 1214–1228.
King, M., Closs J. , Spangler S. , Greenstone R. , Wharton S. , and Myers M. , 2004: EOS Data Products Handbook. Vol. 1. EOS Project Office, 260 [Available from EOS Project Science Office, Code 900, NASA Goddard Space Flight Center, Greenbelt, MD 20771; or online at http://eospso.gsfc.nasa.gov/ftp_docs/data_products_1.pdf].
Liu, Z., Sugimoto N. , and Murayama T. , 2002: Extinction-to-backscatter ratio of Asian dust observed by high-spectral-resolution lidar and Raman lidar. Appl. Opt., 41 , 2760–2767.
Liu, Z., Vaughan M. , Winker D. , Hostetler C. , Poole L. , Hlavka D. , Hart W. , and McGill M. , 2004: Use of probability distribution functions for discriminating between cloud and aerosol in lidar backscatter data. J. Geophys. Res., 109 , D15202. doi:10.1029/2004JD004732.
Liu, Z., Omar A. , Hu Y. , Vaughan M. , and Winker D. , 2005: CALIOP algorithm theoretical basis document—Part 3: Scene classification algorithms. Release 1.0, PC-SCI-202, NASA Langley Research Center, Hampton, VA, 56 [Available online at http://www-calipso.larc.nasa.gov/resources/pdfs/PC-SCI-202_Part3_v1.0.pdf].
Liu, Z., and Coauthors, 2008a: Airborne dust distributions over the Tibetan Plateau and surrounding areas derived from the first year of CALIPSO lidar observations. Atmos. Chem. Phys. Discuss., 8 , 5957–5977.
Liu, Z., and Coauthors, 2008b: CALIPSO lidar observations of the optical properties of Saharan dust: A case study of long-range transport. J. Geophys. Res., 113 , D07207. doi:10.1029/2007JD008878.
Muller, D., Ansmann A. , Mattis I. , Tesche M. , Wandinger U. , Althausen D. , and Pisani G. , 2007: Aerosol-type-dependent lidar-ratio observed with Raman lidar. J. Geophys. Res., 112 , D16202. doi:10.1029/2006JD008292.
N’Tchayi Mbourou, G., Bertrand J. , and Nicholson S. , 1997: The diurnal and seasonal cycles of wind-borne dust over Africa north of the equator. J. Appl. Meteor., 36 , 868–881.
Omar, A., Won J. , Winker D. , Yoon S. , Dubovik O. , and McCormick M. , 2005: Development of global aerosol models using cluster analysis of Aerosol Robotic Network (AERONET) measurements. J. Geophys. Res., 110 , D10S14. doi:10.1029/2004JD004874.
Omar, A., and Coauthors, 2009: The CALIPSO automated aerosol classification and lidar ratio selection algorithm. J. Atmos. Oceanic Technol., in press.
Pinnick, R. G., Jennings S. G. , Chylek P. , Ham C. , and Grandy W. T. Jr., 1983: Backscatter and extinction in water clouds. J. Geophys. Res., 88 , 6787–6796.
Pitts, M. C., Thomason L. W. , Poole L. R. , and Winker D. M. , 2007: Characterization of polar stratospheric clouds with spaceborne lidar: CALIPSO and the 2006 Antarctic season. Atmos. Chem. Phys., 7 , 5207–5228.
Platt, C. M., 1978: Lidar backscatter from horizontal ice crystal plates. J. Appl. Meteor., 17 , 482–488.
Platt, C., Winker D. , Vaughan M. , and Miller S. , 1999: Backscatter-to-extinction ratios in the top layers of tropical mesoscale convective systems and in isolated cirrus from LITE observations. J. Appl. Meteor., 38 , 1330–1345.
Powell, K., and Coauthors, 2009: CALIPSO lidar calibration algorithms. Part I: Nighttime 532-nm parallel channel and 532-nm perpendicular channel. J. Atmos. Oceanic Technol., in press.
Prospero, J. M., Ginoux P. , Torres O. , Nicholson S. E. , and Gill T. E. , 2002: Environmental characterization of global sources of atmospheric soil dust identified with the NIMBUS 7 Total Ozone Mapping Spectrometer (TOMS) absorbing aerosol product. Rev. Geophys., 40 , 1002. doi:10.1029/2000RG000095.
Pueschel, R. F., and Livingston J. M. , 1990: Aerosol spectral optical depths: Jet fuel and forest fire smokes. J. Geophys. Res., 95 , 22417–22422.
Stephens, G. L., and Coauthors, 2002: The CloudSat mission and the A-Train: A new dimension of space-based observations of clouds and precipitation. Bull. Amer. Meteor. Soc., 83 , 1771–1790.
Stephens, G. L., and Coauthors, 2008: The CloudSat mission: Performance and early science after the first year of operation. J. Geophys. Res., 113 , D00A18. doi:10.1029/2008JD009982.
Torres, O., Bhartiea P. , Herman J. , Ahmad Z. , and Gleason, 1998: Derivation of aerosol properties from satellite measurements of backscattered ultraviolet radiation: Theoretical basis. J. Geophys. Res., 103 , (D14). 17099–17110.
Vaughan, M., Young S. , Winker D. , Powell K. , Omar A. , Liu Z. , Hu Y. , and Hostetler C. , 2004: Fully automated analysis of space-based lidar data: An overview of the CALIPSO retrieval algorithms and data products. Proc. SPIE, 5575 , 16–30.
Vaughan, M., Kuehn R. , Young S. , McGill M. , Liu Z. , and Hu Y. , 2008: Validating cirrus cloud optical properties retrieved by CALIPSO. 24th Int. Laser Radar Conf., Boulder, CO, NOAA–NASA, 1090–1093.
Vaughan, M., and Coauthors, 2009: Fully automated detection of cloud and aerosol layers in the CALIPSO lidar measurements. J. Atmos. Oceanic Technol., in press.
Winker, D. M., Couch R. H. , and McCormick M. P. , 1996: An overview of LITE: NASA’s Lidar In-Space Technology Experiment. Proc. IEEE, 84 , 164–180.
Winker, D. M., Hunt W. , and McGill M. , 2007: Initial performance assessment of CALIOP. Geophys. Res. Lett., 34 , L19803. doi:10.1029/2007GL030135.
Winker, D. M., Vaughan M. A. , Omar A. H. , Hu Y. , Powell K. A. , Liu Z. , Hunt W. H. , and Young S. A. , 2009: Overview of the CALIPSO mission and CALIOP data processing algorithms. J. Atmos. Oceanic Technol., in press.
Young, S., and Vaughan M. , 2009: The retrieval of profiles of particulate extinction from Cloud Aerosol Lidar Infrared Pathfinder Satellite Observation (CALIPSO) data: Algorithm description. J. Atmos. Oceanic Technol., 26 , 1105–1119.
Yumimoto, K., Uno I. , Sugimoto N. , Shimizu A. , Liu Z. , and Winker D. M. , 2008: Adjoint inversion modeling of Asian dust emission using lidar observations. Atmos. Chem. Phys., 8 , 2869–2884.
CAD performance based on 1 day of manual classification.
CAD score interpretation and CAD QA flags reported in VFM.
Occurrence frequencies for selected CAD score ranges and special scores.