• Ackerman, S. A., and Cox S. K. , 1987: Surface weather observations of atmospheric dust over the Southwest summer monsoon region. Meteor. Atmos. Phys., 41 , 1934.

    • Search Google Scholar
    • Export Citation
  • Cairo, F., Donfrancesco G. , Adriani A. , Pulvirenti L. , and Federico F. , 1999: Comparison of various linear depolarization parameters measured by lidar. Appl. Opt., 38 , 44254432.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cattrall, C., Reagan J. , Thome K. , and Dubovik O. , 2005: Variability of aerosol and spectral lidar and backscatter and extinction ratios of key aerosol types derived from selected Aerosol Robotic Network locations. J. Geophys. Res., 110 , D10S11. doi:10.1029/2004JD005124.

    • Search Google Scholar
    • Export Citation
  • Charlson, R., Ackerman A. , Bender F. , Anderson T. , and Liu Z. , 2007: On the climate forcing consequences of the albedo continuum between cloudy and clear air. Tellus, 59 , 715727.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herman, J. R., Bhartia P. K. , Torres O. , Hsu C. , Seftor C. , and Celarier E. , 1997: Global distribution of UV-absorbing aerosols from Nimbus7/TOMS data. J. Geophys. Res., 102 , 1691116922.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hess, M., Koepke P. , and Schult I. , 1998: Optical properties of aerosols and clouds: The software package OPAC. Bull. Amer. Meteor. Soc., 79 , 831844.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hostetler, C., and Coauthors, 2006: CALIOP algorithm theoretical basis document—Part 1: Calibration and level 1 data products. PC-SCI-201, Release 1.0, NASA Langley Research Center, Hampton, VA, 66 pp. [Available online at http://www-calipso.larc.nasa.gov/resources/pdfs/PC-SCI-201v1.0.pdf].

    • Search Google Scholar
    • Export Citation
  • Hu, Y., and Coauthors, 2009: CALIPSO/CALIOP cloud phase discrimination algorithm. J. Atmos. Oceanic Technol., in press.

  • Hunt, W., Winker D. , Vaughan M. , Powell K. , Lucker P. , and Weimer C. , 2009: CALIPSO lidar description and performance assessment. J. Atmos. Oceanic Technol., 26 , 12141228.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • King, M., Closs J. , Spangler S. , Greenstone R. , Wharton S. , and Myers M. , 2004: EOS Data Products Handbook. Vol. 1. EOS Project Office, 260 [Available from EOS Project Science Office, Code 900, NASA Goddard Space Flight Center, Greenbelt, MD 20771; or online at http://eospso.gsfc.nasa.gov/ftp_docs/data_products_1.pdf].

    • Search Google Scholar
    • Export Citation
  • Liu, Z., Sugimoto N. , and Murayama T. , 2002: Extinction-to-backscatter ratio of Asian dust observed by high-spectral-resolution lidar and Raman lidar. Appl. Opt., 41 , 27602767.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., Vaughan M. , Winker D. , Hostetler C. , Poole L. , Hlavka D. , Hart W. , and McGill M. , 2004: Use of probability distribution functions for discriminating between cloud and aerosol in lidar backscatter data. J. Geophys. Res., 109 , D15202. doi:10.1029/2004JD004732.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., Omar A. , Hu Y. , Vaughan M. , and Winker D. , 2005: CALIOP algorithm theoretical basis document—Part 3: Scene classification algorithms. Release 1.0, PC-SCI-202, NASA Langley Research Center, Hampton, VA, 56 [Available online at http://www-calipso.larc.nasa.gov/resources/pdfs/PC-SCI-202_Part3_v1.0.pdf].

    • Search Google Scholar
    • Export Citation
  • Liu, Z., and Coauthors, 2008a: Airborne dust distributions over the Tibetan Plateau and surrounding areas derived from the first year of CALIPSO lidar observations. Atmos. Chem. Phys. Discuss., 8 , 59575977.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., and Coauthors, 2008b: CALIPSO lidar observations of the optical properties of Saharan dust: A case study of long-range transport. J. Geophys. Res., 113 , D07207. doi:10.1029/2007JD008878.

    • Search Google Scholar
    • Export Citation
  • Muller, D., Ansmann A. , Mattis I. , Tesche M. , Wandinger U. , Althausen D. , and Pisani G. , 2007: Aerosol-type-dependent lidar-ratio observed with Raman lidar. J. Geophys. Res., 112 , D16202. doi:10.1029/2006JD008292.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • N’Tchayi Mbourou, G., Bertrand J. , and Nicholson S. , 1997: The diurnal and seasonal cycles of wind-borne dust over Africa north of the equator. J. Appl. Meteor., 36 , 868881.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Omar, A., Won J. , Winker D. , Yoon S. , Dubovik O. , and McCormick M. , 2005: Development of global aerosol models using cluster analysis of Aerosol Robotic Network (AERONET) measurements. J. Geophys. Res., 110 , D10S14. doi:10.1029/2004JD004874.

    • Search Google Scholar
    • Export Citation
  • Omar, A., and Coauthors, 2009: The CALIPSO automated aerosol classification and lidar ratio selection algorithm. J. Atmos. Oceanic Technol., in press.

    • Search Google Scholar
    • Export Citation
  • Pinnick, R. G., Jennings S. G. , Chylek P. , Ham C. , and Grandy W. T. Jr., 1983: Backscatter and extinction in water clouds. J. Geophys. Res., 88 , 67876796.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pitts, M. C., Thomason L. W. , Poole L. R. , and Winker D. M. , 2007: Characterization of polar stratospheric clouds with spaceborne lidar: CALIPSO and the 2006 Antarctic season. Atmos. Chem. Phys., 7 , 52075228.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Platt, C. M., 1978: Lidar backscatter from horizontal ice crystal plates. J. Appl. Meteor., 17 , 482488.

  • Platt, C., Winker D. , Vaughan M. , and Miller S. , 1999: Backscatter-to-extinction ratios in the top layers of tropical mesoscale convective systems and in isolated cirrus from LITE observations. J. Appl. Meteor., 38 , 13301345.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Powell, K., and Coauthors, 2009: CALIPSO lidar calibration algorithms. Part I: Nighttime 532-nm parallel channel and 532-nm perpendicular channel. J. Atmos. Oceanic Technol., in press.

    • Search Google Scholar
    • Export Citation
  • Prospero, J. M., Ginoux P. , Torres O. , Nicholson S. E. , and Gill T. E. , 2002: Environmental characterization of global sources of atmospheric soil dust identified with the NIMBUS 7 Total Ozone Mapping Spectrometer (TOMS) absorbing aerosol product. Rev. Geophys., 40 , 1002. doi:10.1029/2000RG000095.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pueschel, R. F., and Livingston J. M. , 1990: Aerosol spectral optical depths: Jet fuel and forest fire smokes. J. Geophys. Res., 95 , 2241722422.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephens, G. L., and Coauthors, 2002: The CloudSat mission and the A-Train: A new dimension of space-based observations of clouds and precipitation. Bull. Amer. Meteor. Soc., 83 , 17711790.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephens, G. L., and Coauthors, 2008: The CloudSat mission: Performance and early science after the first year of operation. J. Geophys. Res., 113 , D00A18. doi:10.1029/2008JD009982.

    • Search Google Scholar
    • Export Citation
  • Torres, O., Bhartiea P. , Herman J. , Ahmad Z. , and Gleason, 1998: Derivation of aerosol properties from satellite measurements of backscattered ultraviolet radiation: Theoretical basis. J. Geophys. Res., 103 , (D14). 1709917110.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, M., Young S. , Winker D. , Powell K. , Omar A. , Liu Z. , Hu Y. , and Hostetler C. , 2004: Fully automated analysis of space-based lidar data: An overview of the CALIPSO retrieval algorithms and data products. Proc. SPIE, 5575 , 1630.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, M., Kuehn R. , Young S. , McGill M. , Liu Z. , and Hu Y. , 2008: Validating cirrus cloud optical properties retrieved by CALIPSO. 24th Int. Laser Radar Conf., Boulder, CO, NOAA–NASA, 1090–1093.

    • Search Google Scholar
    • Export Citation
  • Vaughan, M., and Coauthors, 2009: Fully automated detection of cloud and aerosol layers in the CALIPSO lidar measurements. J. Atmos. Oceanic Technol., in press.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., Couch R. H. , and McCormick M. P. , 1996: An overview of LITE: NASA’s Lidar In-Space Technology Experiment. Proc. IEEE, 84 , 164180.

  • Winker, D. M., Hunt W. , and McGill M. , 2007: Initial performance assessment of CALIOP. Geophys. Res. Lett., 34 , L19803. doi:10.1029/2007GL030135.

  • Winker, D. M., Vaughan M. A. , Omar A. H. , Hu Y. , Powell K. A. , Liu Z. , Hunt W. H. , and Young S. A. , 2009: Overview of the CALIPSO mission and CALIOP data processing algorithms. J. Atmos. Oceanic Technol., in press.

    • Search Google Scholar
    • Export Citation
  • Young, S., and Vaughan M. , 2009: The retrieval of profiles of particulate extinction from Cloud Aerosol Lidar Infrared Pathfinder Satellite Observation (CALIPSO) data: Algorithm description. J. Atmos. Oceanic Technol., 26 , 11051119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yumimoto, K., Uno I. , Sugimoto N. , Shimizu A. , Liu Z. , and Winker D. M. , 2008: Adjoint inversion modeling of Asian dust emission using lidar observations. Atmos. Chem. Phys., 8 , 28692884.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    (a) Scatterplot with contour lines, and (b) occurrence distribution of clouds (blue) and aerosols (red) whose midlayer altitude falls between 0 and 1 km, measured by CALIOP on 12 Aug 2006, classified based on expert manual classification. The contours in (a) represent number densities of features of 5 and 25 corresponding to ∼10% and ∼50% of the maximum density, respectively. The layer-averaged attenuated backscatter is in km−1 sr−1.

  • View in gallery

    Flowchart of the version 2 CAD algorithm.

  • View in gallery

    (top) The threshold used in the depolarization ratio test and (bottom) a map defining a dust belt used in this paper.

  • View in gallery

    An example of the volume depolarization ratio as a function of the aerosol mixing ratio (R−1) for an assumed particulate depolarization ratio of 0.3 and a molecular depolarization ratio of 0.0036.

  • View in gallery

    (a) Number of occurrences and (b) cumulative occurrence frequency of the CAD score derived from one month (July 2006) of the CALIOP layer products data (version 2.01).

  • View in gallery

    Example of the cloud and aerosol discrimination measured by CALIOP on 27 Jun 2007 from a nighttime orbit across central and southern Europe and North Africa. (top) Attenuated backscatter at 532 nm, along with the orbit track (pink line); (middle) volume depolarization ratio; and (bottom) the corresponding classification results. Feature types in the bottom panel: 0, invalid (|CAD| < 20 or CAD = 103); 1, clear air; 2, cloud; 3, aerosol; 4, stratospheric feature; 5, surface; 6, subsurface; and 7, no signal (total attenuated).

  • View in gallery

    Example of misclassification of a dust storm generated in the Gobi Desert observed on 30 Mar 2007: (a) CALIOP 532-nm attenuated backscatter and (b) VFM (same color bar as in Fig. 5), (c) CloudSat vertical feature mask, (d) effective lidar ratio at 532 nm and (e) its histogram distribution, and (f) the CALIOP track. The circled features in (a) are dust layers that have been misclassified. Note that there are small-scale clouds on the top of each circled scene (7–8 km around 37.1° and 4–5 km near 44°).

  • View in gallery

    Zonal distribution of cloud and aerosol features, derived from one month (July 2006) of 5-km-layer products, scaled to the total number of 5-km profiles measured in the 0°–10° latitude band. Features with special CAD scores of −101 and 103 have been excluded.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 674 611 51
PDF Downloads 489 452 48

The CALIPSO Lidar Cloud and Aerosol Discrimination: Version 2 Algorithm and Initial Assessment of Performance

View More View Less
  • 1 National Institute of Aerospace, Hampton, Virginia
  • | 2 National Aeronautics and Space Administration, Hampton, Virginia
  • | 3 Science Systems and Applications, Inc., Hampton, Virginia
  • | 4 National Aeronautics and Space Administration, Hampton, Virginia
© Get Permissions
Full access

Abstract

The Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite was launched in April 2006 to provide global vertically resolved measurements of clouds and aerosols. Correct discrimination between clouds and aerosols observed by the lidar aboard the CALIPSO satellite is critical for accurate retrievals of cloud and aerosol optical properties and the correct interpretation of measurements. This paper reviews the theoretical basis of the CALIPSO lidar cloud and aerosol discrimination (CAD) algorithm, and describes the enhancements made to the version 2 algorithm that is used in the current data release (release 2). The paper also presents a preliminary assessment of the CAD performance based on one full day (12 August 2006) of expert manual classification and on one full month (July 2006) of the CALIOP 5-km cloud and aerosol layer products. Overall, the CAD algorithm works well in most cases. The 1-day manual verification suggests that the success rate is in the neighborhood of 90% or better. Nevertheless, several specific layer types are still misclassified with some frequency. Among these, the most prevalent are dense dust and smoke close to the source regions. The analysis of the July 2006 data showed that the misclassification of dust as cloud occurs for <1% of the total tropospheric cloud and aerosol features found. Smoke layers are misclassified less frequently than are dust layers. Optically thin clouds in the polar regions can be misclassified as aerosols. While the fraction of such misclassifications is small compared with the number of aerosol features found globally, caution should be taken when studies are performed on the aerosol in the polar regions. Modifications will be made to the CAD algorithm in future data releases, and the misclassifications encountered in the current data release are expected to be reduced greatly.

Corresponding author address: Zhaoyan Liu, 100 Exploration Way, Hampton, VA 23666-6147. Email: zhaoyan.liu-1@nasa.gov

This article included in the CALIPSO special collection.

Abstract

The Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite was launched in April 2006 to provide global vertically resolved measurements of clouds and aerosols. Correct discrimination between clouds and aerosols observed by the lidar aboard the CALIPSO satellite is critical for accurate retrievals of cloud and aerosol optical properties and the correct interpretation of measurements. This paper reviews the theoretical basis of the CALIPSO lidar cloud and aerosol discrimination (CAD) algorithm, and describes the enhancements made to the version 2 algorithm that is used in the current data release (release 2). The paper also presents a preliminary assessment of the CAD performance based on one full day (12 August 2006) of expert manual classification and on one full month (July 2006) of the CALIOP 5-km cloud and aerosol layer products. Overall, the CAD algorithm works well in most cases. The 1-day manual verification suggests that the success rate is in the neighborhood of 90% or better. Nevertheless, several specific layer types are still misclassified with some frequency. Among these, the most prevalent are dense dust and smoke close to the source regions. The analysis of the July 2006 data showed that the misclassification of dust as cloud occurs for <1% of the total tropospheric cloud and aerosol features found. Smoke layers are misclassified less frequently than are dust layers. Optically thin clouds in the polar regions can be misclassified as aerosols. While the fraction of such misclassifications is small compared with the number of aerosol features found globally, caution should be taken when studies are performed on the aerosol in the polar regions. Modifications will be made to the CAD algorithm in future data releases, and the misclassifications encountered in the current data release are expected to be reduced greatly.

Corresponding author address: Zhaoyan Liu, 100 Exploration Way, Hampton, VA 23666-6147. Email: zhaoyan.liu-1@nasa.gov

This article included in the CALIPSO special collection.

1. Introduction

The Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), on board the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite, was launched in April 2006 (Winker et al. 2007), in formation with the CloudSat satellite, as part of the A-Train constellation of satellites (Stephens et al. 2002). The main objectives of the CALIPSO mission are to provide global measurements of cloud and aerosol spatial distributions and optical properties to complement current measurements and improve our understanding of weather and climate. The availability of a global, multiyear set of vertically resolved measurements of the earth’s atmosphere should ultimately lead to great improvements in both weather and climate models. CALIOP is the first satellite-borne lidar optimized specifically for aerosol and cloud measurements, and is also the first polarization lidar in space. CALIOP is a dual-wavelength, polarization-sensitive elastic backscatter lidar. Its light source is an Nd:YAG pulsed laser operating at 1064 and 532 nm. The transmitted laser beam is linearly polarized, and two polarization-sensitive 532-nm receiver channels measure the degree of linear depolarization of the return signal [see details of the lidar instrument provided in Hunt et al. (2009)]. Using the two 532-nm receiver channels and a channel measuring the total 1064-nm return signal, CALIOP measures the detailed vertical distribution of aerosols and clouds along with their microphysical and optical properties.

Since June 2006, CALIOP has been acquiring a global dataset that is being processed continuously at NASA’s Langley Research Center. An overview of the CALIOP science data processing architecture is provided in Winker et al. (2009). The routine lidar data processing currently includes two levels: 1 and 2 (King et al. 2004). In the level 1 data processing, the lidar backscatter data are geolocated and calibrated. The resulting altitude-resolved profiles of attenuated backscatter coefficients (Hostetler et al. 2006; Powell et al. 2009) are reported in the CALIOP level 1B data products. These level 1 profiles are further analyzed in level 2 processing to derive the optical and physical properties of clouds and aerosols (Vaughan et al. 2004).

The level 2 processing algorithms include three primary modules: a layer detection algorithm known as the selective, iterated boundary locator (SIBYL), the scene classification algorithms (SCA), and the hybrid extinction retrieval algorithm (HERA). First, the layer-detection algorithm, SIBYL, finds features (clouds, aerosols, surface and subsurface, stratospheric layers, etc.) by searching for regions of enhanced signal in the attenuated backscatter profiles provided by the level 1 processing (Vaughan et al. 2009). After finding features, mean values of the 532- and 1064-nm attenuated backscatter, attenuated total color ratio, and volume depolarization ratio are computed for each atmospheric feature detected. These layer optical properties, along with physical properties such as top and base heights, latitude, longitude, etc., are reported in the level 2 layer products. Based on these optical and physical properties, each atmospheric feature is then classified according to type by the SCA (Liu et al. 2005). A primary function of the SCA is to select an extinction-to-backscatter ratio (also known as the lidar ratio) for each layer based on the classification results. The lidar ratio is a key parameter that is used in HERA to retrieve particulate (i.e., aerosol and/or cloud) extinction and backscatter coefficients (Young and Vaughan 2009). An accurate retrieval of layer optical properties by HERA depends critically on the selection of proper lidar ratios, which in turn requires an accurate scene classification.

SCA is composed of three main submodules: cloud–aerosol discrimination (CAD) (Liu et al. 2004), aerosol subtyping (Omar et al. 2005, 2009, hereafter OmarSI), and cloud ice-water phase discrimination (Hu et al. 2009, hereafter HuSI). The CAD algorithm is a multidimensional probability density function (PDF) based approach (Liu et al. 2004). The attributes (dimensions) included are the mean attenuated backscatter at 532 nm, the layer-integrated 1064/532-nm attenuated backscatter ratio (color ratio), and the midlayer altitude. The PDFs used in the first data release were developed based on airborne lidar measurements and measurements obtained during the Lidar In-Space Technology Experiment (LITE; Winker rt al. 1996). A new set of PDFs has been developed based on a manual classification of one full day of CALIOP measurements and is used in the current CALIOP data release (version 2). In addition, a new depolarization ratio test has also been incorporated into the version 2 CAD algorithm. In this paper we focus on the version 2 CAD algorithm. The algorithm overview and an initial assessment of the algorithm performance will be presented.

2. Algorithm outline

In this section, we provide an overview of the theoretical basis of the CALIOP CAD algorithm and describe salient aspects of the implementation. Complete details can be found in Liu et al. (2004, 2005).

a. Theoretical basis

The discrimination between clouds and aerosols is performed mainly based on the differences in their optical and physical properties. The algorithm is driven by the confidence function
i1520-0426-26-7-1198-e1
where pcloud and paerosol are the multidimensional PDFs, respectively, for clouds and aerosols, as a function of attributes X1, X2, … , Xm. In addition, k is a scaling factor that is related to the ratio of the numbers of aerosol layers and cloud layers and is determined based on the measurement. It may vary depending on locations, altitudes, and seasons. Attributes X1, X2, … , Xm can be lidar observables (e.g., backscatter intensity, wavelength dependency, depolarization ratio, layer heights) or ancillary parameters (e.g., temperature, pressure, location, season). The function f is a normalized differential probability. Its value ranges from −1 to 1. Within the CALIOP level 2 layer products, a percentile (integer) value of f ranging from −100 to 100 is reported as the “CAD score” characterizing each feature.
The identification of a feature as either cloud or aerosol is performed based on the sign of the CAD score. If f (X1, X2, … , Xm) > 0, then the probability that [X1, X2, … , Xm] represents a cloud is larger than the probability that [X1, X2, … , Xm] represents an aerosol, and the feature is therefore classified as a cloud. Otherwise, if f (X1, X2, … , Xm) < 0, the cloud probability is smaller than the aerosol probability, and so the feature is classified as an aerosol. When f (X1, X2, … , Xm) = 0, the probability that the feature represented by [X1, X2, … , Xm] is a cloud is equal to its probability of being an aerosol, and thus the classification is indeterminate. In regions where | f| = 1 (or |CAD| = 100), the classification can be made unambiguously. The absolute value of the CAD score provides a confidence level for the classification: the larger the magnitude of the CAD score, the higher our confidence that the classification is correct. Theoretically, an absolute value of 100 therefore indicates complete confidence. Absolute values less than 100, corresponding to regions where the cloud and aerosol PDFs overlap, indicate some ambiguity in the classification. In this case, a definitive classification cannot be made; that is, although we can provide a “best guess” classification, this guess could be wrong. Theoretically, a success rate Rs (or a failure rate Rf), defined as the ratio of correctly classified events to the total number of events (Rf = 1 − Rs), is related to f by (Liu et al. 2005)
i1520-0426-26-7-1198-e2
The performance of the classification is limited essentially by the degree of the overlap in the cloud and aerosol PDFs; the smaller the overlap region, the more complete the separation between the cloud and aerosol distributions, and the better the classification performance. To reduce the overlap, PDFs with a larger number of attribute dimensions are desired, because the separation between the cloud and aerosol clusters is more complete in a higher-dimensional space (Liu et al. 2004). In principle, as many of the available observables as possible should be used to achieve the best performance. However, the addition of an attribute dimension may not improve the classification significantly if the attribute to be added is not totally independent from the other attributes used. Some practical issues such as computing time and the latency of any ancillary data should also be considered when selecting attribute dimensions for operational classification.

When selecting the attributes used to discriminate clouds from aerosols, a very strong preference should be given to the intrinsic properties of features, that is, to those physical and optical characteristics of the feature that depend solely on the layer composition and are independent of particle concentration. However, in many cases—and this is especially relevant for backscatter color ratios—estimates of the intrinsic properties of a feature can only be obtained after computing (and subsequently correcting for) the extinction profile within the feature. The CALIOP CAD algorithm thus faces a dilemma: deriving the extinction solution requires an estimate of the feature lidar ratio, and this estimate in turn is derived from an assessment of layer type (e.g., OmarSI). Because the CAD is the first step of the scene classification analysis that is performed before the extinction coefficient is retrieved by HERA, some intrinsic attributes (e.g., the particulate backscatter color ratio) are not available for use and, thus, are replaced by the corresponding extrinsic feature properties (e.g., the attenuated backscatter color ratio).

b. Development of PDFs

The core part of the cloud and aerosol classification is based on multidimensional PDFs of clouds and aerosols. Attributes of layer-averaged attenuated backscatter (β′), attenuated total backscatter color ratio (χ′), and midlayer heights (z) have been selected in the prelaunch version (version 1) that was used in the first lidar data release (Liu et al. 2004) and in version 2 of the CAD algorithm. The f function can then be rewritten as
i1520-0426-26-7-1198-e3
where
i1520-0426-26-7-1198-e4
i1520-0426-26-7-1198-e5
and B is the attenuated backscatter coefficient at range zi with additional corrections for the effects of molecular and ozone attenuations applied. For practical use, the PDFs are implemented as a 100 × 100 × 20 three-dimensional array. The attenuated backscatter dimension of the PDFs is logarithmic, having 100 elements starting at ln(β′) = −12 with an increment of 0.14. The attenuated total backscatter color ratio dimension also has 100 elements, starting at χ′ = 0 with increments of 0.02. The midlayer altitude dimension is binned for every 1 km from 0 to 20 km (20 elements).

The cloud and aerosol PDF database used in the prelaunch version for the first data release was developed mainly based on regional airborne lidar measurements acquired before CALIPSO’s launch (Liu et al. 2004). The database has since been updated based on expert manual classification of all layers detected during one full day of data acquired by CALIOP on 12 August 2006. In contrast to the automated production CAD processing that performs feature classifications in isolation, based on the properties of single vertical layer, this expert reclassification identifies features by simultaneous inspection of two-dimensional vertical–horizontal images of several different lidar measurements (attenuated backscatter, attenuated backscatter color ratio, and volume depolarization ratio). That is, in addition to χ′, β′, and midlayer altitude, this expert reclassification also makes use of additional information such as layer structures, textures, connections with the surrounding layers, and geographic locations in the assessment of feature types. Due to the large amount of data that must be examined (e.g., ∼0.3 million 5-km layers for the selected day), manual classification of layer types can be very time consuming. For this reason, a computer tool has been developed to facilitate this kind of image-based manual identification of features by immediately and interactively providing the quantitative, measured data associated with each layer. Despite this computerized assistance and the application of spatial coherence tests, there remain some cases that cannot be identified with high confidence. These cases include layers at edges of clouds, mixed layers of cloud and aerosol, and layers below dense clouds whose optical properties are not correctly measured due to the detection issues such as the detector transient response, multiple scattering, etc. These layers, labeled as no confidence in the manual classification, account for ∼2% of the selected day’s manually classified data (see Table 1).

The updated PDFs based on the one day of manual classification are used in the version 2 algorithm for the current data release. We note that even these new PDFs may not completely characterize the aerosol and cloud distributions for all seasons and all locations, and that more comprehensive PDFs are being developed based on more data for future data releases. However, as demonstrated in section 3, the CAD algorithm with the updated PDFs works well for most cases. Figure 1 presents (a) a combination scatterplot–contour plot in the backscatter–color ratio space and (b) the occurrence distribution as a function of attenuated backscatter of clouds (blue) and aerosols (red) within the 0–1-km altitude range measured by CALIOP on 12 August 2006. As seen in Fig. 1, aerosols generally have smaller backscatter intensities and attenuated backscatter color ratios when compared with clouds, and the two classes are largely separated in the backscatter–color ratio space, as expected by the prelaunch study (Liu et al. 2004) based on the Optical Properties of Aerosols and Clouds (OPAC) software package (Hess et al. 1998). While aerosols have a single-mode distribution centered at ∼0.003 km−1 sr−1 of attenuated backscatter and ∼0.45 of color ratio, clouds have a bimodal distribution centered, respectively, at ∼0.1 and ∼0.01 km−1 sr−1 of attenuated backscatter and ∼0.95 and ∼1 of color ratio. An overlap (<10% of total features) is seen mainly in a region of 0.004–0.01 km−1 sr−1 for attenuated backscatter and of 0.5–0.9 for the color ratio. In this overlap region, an unambiguous classification will not be possible and some misclassifications will occur.

Dense aerosols such as dust and smoke over or near the sources can contribute to the overlap. Dust aerosols usually have large backscatter color ratios, comparable to those of clouds, due to the large size of the dust particles. The lidar signatures for dense dust aerosols can look very similar to those of clouds in terms of attenuated backscatter and color ratio. Dense smoke is another case for which both the attenuated backscatter and the color ratio can closely resemble those characteristic of clouds. For smoke, the particulate backscatter color ratio is not generally as large as that for dust or cloud. However, the smoke extinction at 532 nm is much larger than at 1064 nm (2.5–5 times) (Pueschel and Livingston 1990). When measuring dense smoke layers, the range-dependent reduction in lidar signals due to attenuation occurs much faster at 532 nm than at 1064 nm. As a result, the attenuated color ratio will grow quickly, resulting in a large layer-averaged attenuated color ratio. Hygroscopic aerosols located in regions of high humidity (Charlson et al. 2007) may also fall into the overlap region.

Optically thin clouds, including horizontally small-scale clouds and mixed layers of cloud and aerosol (vertically adjacent cloud and aerosol layers or clouds embedded in aerosol), can also contribute to the overlap. We note, however, that these mixed-type features should be classified as “cloud” by convention (refer to section 2e) in the current CALIOP scene classification.

In addition to the natural spread of cloud and aerosol distributions, signal noise can widen the cloud and aerosol distributions and hence contribute to the overlap in the cloud and aerosol PDFs (Liu et al. 2004). The spread in the scatterplot shown in Fig. 1 is caused partly by signal noise. Because the signal-to-noise ratios (SNRs) for the measurements from which the original version 1 PDFs were derived are substantially higher than the CALIOP SNR, the version 1 processing included a module that explicitly compensated for these additional noise effects. The version 2 PDFs were developed based solely on the CALIOP measurements, and because the noise broadening effect is thus inherent in the revised distributions, this noise compensation module is disabled in the version 2 data release. We note that the PDFs for daytime and nighttime measurements are slightly different; the daytime PDFs have a somewhat broader distribution than the nighttime PDFs due to the increase in background noise during the day. However, disabling the noise compensation module has had a negligible impact on the classification results, although it can produce some additional uncertainty in the confidence value estimated for each classification.

c. Enhancements to the version 2 algorithm

The version 2 CAD algorithm improves upon version 1 by using additional tests to ensure the correct classification of several well-defined special cases. The flow of the version 2 CAD algorithm is presented in Fig. 2. First, the CAD algorithm reads in layer products, β532, χ′, δ, γ532, z, T, provided by SIBYL. Here, γ532 is the integrated attenuated backscatter at 532 nm from the layer top to the base and T is the midlayer temperature. In addition, δ is the volume depolarization ratio, defined by
i1520-0426-26-7-1198-e6
where β532,m,⊥, β532,p,⊥, β532,m,‖, and β532,p,‖ are perpendicular and parallel components of the molecular backscatter coefficient, β532,m, and the particulate backscatter coefficient, β532,p. Before classifying a feature, the algorithm checks the validity of the feature. This is done by checking the mean attenuated backscatter β532: if β532 < 0, the layer is erroneous and assigned a special CAD score of −101. (All features detected, including these erroneous features, are reported in the 5-km cloud-layer products. In the current data release, CAD scores are reported only in the 5-km-layer products.) Then, the algorithm determines if the layer is stratospheric or tropospheric by comparing the layer base height zbase with the tropopause height ztrop. If zbase is higher than ztrop, the feature is stratospheric; otherwise, the feature is tropospheric. The cloud and aerosol discrimination is conducted only for tropospheric features in the CAD module and no additional classification is done to distinguish between PSCs and stratospheric aerosols or among the different classes of PSCs in the current SCA. Studies on PSCs (e.g., Pitts et al. 2007) have been done that provide a foundation for classifications of PSCs in future data releases.

For the initial classification of tropospheric features, the algorithm uses a threshold value γth to identify features that have very large values of γ532. Clouds containing large fractions of horizontally oriented ice crystals are characterized by large integrated attenuated backscatter, large color ratio (close to unity), and very low depolarization ratio (close to zero) (Platt 1978). Additional threshold tests based on χ′, δ, and temperature are performed on those clouds with large γ532 to identify cloud returns that are obviously affected by horizontally oriented ice crystals. Cloud layers identified as containing horizontally oriented ice crystals are assigned a special CAD score of 102. A more sophisticated algorithm will be incorporated in a future data release to provide improved detection of oriented ice crystals (HuSI). The classification of features with γ532 > γth that do not pass the oriented ice crystal tests is less certain. The large γ532 values could be artifacts due to an overestimation of the attenuation for overlying layers by HERA (Young and Vaughan 2009). Due to this uncertainty, these features are assigned a special CAD score of 103. All features having special CAD scores are reported in the cloud-layer products. The interpretation of the CAD scores, as well as CAD quality assurance (QA) flags reported in the vertical feature mask (VFM), is given in Table 2.

For the tropospheric features with γ532 < γth, the CAD algorithm computes f using Eq. (3), based on β532, χ′, and z. If f > 0, the feature is classified as cloud. Otherwise, an additional test is conducted that compares the layer-integrated volume depolarization ratio δ with a latitude-dependent threshold δth as defined in the top panel of Fig. 3. If δ < δth, the feature is defined as aerosol, with a CAD score equal to (100 times) the magnitude of f; otherwise, the feature is classified as cloud with a special CAD score of 101.

The depolarization ratio test was added to the version 2 CAD algorithm to help in reducing possible misclassifications of cloud as aerosol. This is based on the fact that cirrus and dense water clouds normally have a depolarization ratio larger than aerosols. In dense water clouds, multiple scattering is significant (particularly for space lidars) and can produce large depolarization ratios. Aerosols, except dust, normally have small depolarization ratios, as observed from the CALIOP measurements (Liu et al. 2008a; OmarSI). Dust aerosols have large particulate depolarization ratios (δp = β532,p,⊥ / β532,p,‖) because of the nonsphericity of dust particles. Based on analyses of the CALIOP data, the value of the dust particulate depolarization ratio is normally smaller than 0.4. The volume depolarization ratio will decrease in the presence of molecular scattering that, for CALIOP, has a very small depolarization ratio of δm ∼ 0.0036. This very small value is due to the narrow bandwidth of the CALIOP receiver, which only allows the central Cabannes line of the backscattered signal to be detected. The smaller the dust concentration, the smaller the volume depolarization ratio. This is evident from the curve of the volume depolarization ratio as a function of the backscatter ratio, R = (βp + βm)/βm, presented in Fig. 4, computed using (e.g., Cairo et al. 1999; Liu et al. 2002)
i1520-0426-26-7-1198-e7
Over the dust source regions, the volume depolarization ratio can be close to the particulate depolarization ratio where the dust concentration remains very high after a dust storm occurs. For this reason, a threshold value of 0.4 has been selected for a global “dust belt.” This dust belt is a region containing the major dust sources and is located in the Northern Hemisphere, stretching from the western coast of Africa to China, covering the Sahara and Sahel regions, the Arabian Peninsula, northern India, the Tarim Basin, and the Gobi Desert (Prospero et al. 2002). In the CAD algorithm, the dust belt is defined as a larger geographic region between 0°–50°N and 40°W–130°E, indicated by the square in the bottom panel of Fig. 3. Outside of the dust belt, smaller threshold values have been selected. Approaching the North and South Poles, the threshold decreases, because both the occurrence probability and concentration of dust aerosols decreases in the higher latitudes.

d. Special CAD scores

In addition to the standard CAD scores, which range between −100 and 100, the special CAD scores of −101, 101, 102, and 103 are also reported in the CALIOP cloud-layer products. These special scores distinguish features/artifacts identified by other tests as described in Fig. 2. Figure 5 presents an example of (a) the occurrence distribution and (b) the cumulative occurrence frequency of the CAD score derived from one month (July 2006) of the CALIOP 5-km-layer products data (version 2.01). Table 3 also lists frequencies for selected CAD score ranges and the special CAD scores.

The special CAD score of −101 indicates features that have a negative mean attenuated backscatter. These features should not occur based on the algorithm design. However, they occurred in the current layer products due to some unknown programming reasons. The fraction of these features is very low, ∼0.1% of the total tropospheric features, including clouds and aerosols for the one month of data shown in Fig. 5, and they should be eliminated in future data releases. Layers flagged −101 should never be used. The rest of the special scores (101, 102, and 103) correspond mostly to genuine atmospheric features. As described earlier, the special score of 101 is assigned to the features that are classified as cloud by the depolarization ratio test (the δ switch in Fig. 2). About 0.7% of the total features have CAD scores of 101, and these mostly occur at high latitudes. The special score of 102 is assigned to those features that are highly likely to contain large fractions of oriented ice crystals. These features account for ∼0.2% of the total features. We note that the one month’s worth of the data analyzed here were acquired when the lidar was operated with a nadir-pointing angle of 0.3°. The lidar-pointing angle was adjusted to 3° recently, and thereafter the fraction of horizontally oriented ice crystals should be considerably reduced. Features with special CAD scores of 101 and 102 can confidently be included in scientific applications of the data. Features with the special score of 103 may be a mixture of oriented ice crystals with other types of cloud particles, or a feature whose optical properties are artificial due to improper data processing, or may not be an atmospheric feature. Therefore, caution should be taken in the interpretation and use of this class of features.

e. Some conventions

Conventions used in the version 2 CAD algorithm are provided in the following:

  • (a) Only two classes of tropospheric features are defined in the CAD algorithm: “cloud” and “aerosol.” The aerosol class is defined as all airborne particles, excluding activated water droplets and frozen ice crystals. This class includes all commonly defined aerosol types (maritime, continental, dust, and smoke) (OmarSI). The cloud class is defined as an air mass containing any activated water droplets and/or ice crystals. All normally defined clouds (high-, middle-. and low-cloud families) are included in the cloud class.

  • (b) Based on the definition above, a mixed layer of cloud and aerosol that is erroneously identified as a single layer by the CALIOP feature finder is classified as a cloud.

  • (c) Fogs, in either liquid form or frozen form, are included in the cloud class.

  • (d) Diamond dust also belongs to the cloud class because it consists mostly of falling ice crystals.

  • (e) Hazes belong to the aerosol class, whereas mists are classified as clouds.

3. Initial performance assessment

a. General performance

Performance assessments of the version 2 CAD algorithm are ongoing. In this section we present several initial results. Overall, the CAD algorithm works well in most cases; manual verification of the classifications for a full day of 5-km data (299 683 features) suggests that the success rate is larger than 90% (see Table 1). Table 1 summarizes the assessment results based on the 1-day 5-km data. We note that this 1-day assessment may not be completely representative of the whole CALIOP dataset, and only provides a general idea about the CAD performance. The percentage of clouds misclassified as aerosol for daytime is smaller than that for nighttime because the daytime feature finding misses more optically thin clouds that fall in the overlap region of PDFs and are, thus, more likely to be misclassified. On the other hand, the percentage of aerosols misclassified as cloud is larger for daytime measurements. This occurs because a large fraction of the optically tenuous aerosols that fall outside of the deep overlap region of the PDFs, and therefore can be correctly classified as aerosol, are not reliably detected during daytime operations. For the one month of data shown in Fig. 5 and Table 3, ∼83% of aerosols and ∼95% of clouds have a CAD score magnitude greater than 70, which corresponds to the “high” confidence category (Table 2), indicating that clouds and aerosols are well separated.

An example of the CALIOP cloud and aerosol discrimination is given in Fig. 6. The data were acquired by CALIOP on 27 June 2007 from a nighttime orbit across central and southern Europe and North Africa. In this scene, a spatially extensive Saharan dust layer of moderate optical thickness was observed, extending from ∼39°N all the way to the right-hand-side end of the images at ∼7.4°N. This dust layer is easily identified from the depolarization ratio measurement (green–yellow–orange colors in the middle panel of Fig. 6). Dust aerosol can be well distinguished from other aerosol types based on a test on the volume depolarization ratio using a threshold of 0.06 (Liu et al. 2008a). Vertically, this dust layer extends from the surface up to several kilometers (>5 km for its highest part) and begins to detach from the surface north of 34°N (middle panel of Fig. 6). Above 34°N, other aerosol types are also observed (green–yellow–orange–red-colored features below 3 km in the top panel of Fig. 6). Aerosols between 34° and 46°N appear to be mixed with dust particles (greenish colors in the middle panel of Fig. 6). In addition, cirrus clouds in high altitudes (>8 km, red–yellow–gray colors in the top panel of Fig. 6) and relatively small-scale clouds (red–gray–white colors) in mid- and low altitudes are observed.

From the VFM image shown in the bottom panel of Fig. 6, it is seen that the CAD algorithm has identified most features correctly. In this particular scene, misclassifications happen mainly at the edges of clouds and surrounding areas, though they are not clearly visible. The few mixed layers of dust and embedded cloud (blue strips near the right-hand-side end in the bottom panel of Fig. 6) are, by convention, correctly classified as cloud. Note that when a cloud is adjacent to or embedded in an aerosol layer, the heterogeneous mix of layers can sometimes be identified as a single layer (Vaughan et al. 2009).

The features colored red in the VFM image represent those features with CAD score magnitudes smaller than 20 or equal to 103 or −101, that is, low- or no-confidence features. These “red” features account for about 9% of the total tropospheric features based on the 1-month data analysis (Table 3). In this scene they are mostly artifacts due to the noise in the signal, multiple scattering effects, or due to artificial signal enhancements caused by a nonideal detector transient response or an overestimate of the attenuation due to overlaying layers. These erroneous “pseudofeatures” occur mainly under dense clouds. They are neither cloud nor aerosol; however, because they are not properly assessed and interdicted by the feature finder, the CAD algorithm nonetheless attempts to assign them to one class or the other.

b. Misclassification types

Despite the generally high accuracy of the CAD algorithm, several types of misclassifications still occur with some frequency. Among these, the most prevalent are very dense dust and smoke aerosols lying over and close to the source regions. Because the CAD algorithm operates on individual layers, without a contextual awareness of any surrounding features, it can happen that small but strongly scattering regions within an extended aerosol layer can occasionally be labeled as cloud. This occurs because the optical properties (the attenuated backscatter and color ratio) of dense dust and smoke layers are similar to what would be expected for clouds at the same altitude. These misclassifications are often apparent from studying the level 1 browse images.

We present an extreme example of the misclassification of very dense dust layers in Fig. 7. These data were acquired on the daytime side of an orbit on 30 March 2007 as depicted by the CALIOP ground track shown in Fig. 7f. Attenuated backscatter measured by CALIOP at 532 nm and the CALIOP vertical feature mask are presented in Figs. 7a and 7b, respectively. A second vertical feature mask derived from the coincident CloudSat radar measurements (Stephens et al. 2008) is presented in Fig. 7c to help identify dense clouds.

In late March 2007, a large dust storm originated in the Gobi Desert and, in the days that followed, the resulting dust plume was transported for long distances into the North Pacific (Yumimoto et al. 2008). CALIOP observed extensive dust plumes on 30 March 2007 over the Gobi area (the yellow–orange–red–gray-colored features north of ∼46°N and below ∼8 km in Fig. 7a). These dust plumes were not observed by CloudSat (Fig. 7c) because the dust particles are small compared with the radar wavelength (millimeter). A very dense dust plume (gray-colored features in Fig. 7a) is seen between 38° and 46°N and below ∼4 km, where the lidar signal becomes totally attenuated (no surface signals are detected) and the optical depth therefore should be larger than ∼3. This dense dust plume has been misclassified by the CAD algorithm, because its optical properties are similar to what would be expected for clouds (note that the gray color of the dust layer in Fig. 7a is almost the same as the high dense cloud at 9–12 km between 39.4° and 41°N). An opaque cloud is seen on the top of the dense dust plume at an altitude of 4–5 km between 44° and 44.6°N, as observed by CloudSat (Fig. 7c). Dust particles were also lofted to higher altitudes and a dust layer is observed at 4–8-km altitude. This layer has also been misclassified, because, while it is only moderately dense, it has been lofted to relatively high altitudes. Aerosol layers of moderate to high backscatter intensity are not expected to be present at relatively high altitudes by the version 2 CAD algorithm. Currently, one set of PDFs is used globally in the CAD algorithm. In the future, latitudinally, regionally, and/or seasonally dependent sets of PDFs can be developed and used, which should greatly reduce this type of misclassification. Note that the moderate dust layer north of 37°N and below ∼4 km has been correctly classified.

Clouds (mainly gray–white-colored features) are also observed by CALIOP in Fig. 7a. All clouds in this scene have been identified correctly by the CAD algorithm. This is evident when comparing the CALIOP vertical feature mask in Fig. 7b with the CloudSat vertical feature mask in Fig. 7c. We note that, besides dust aerosols, the CloudSat radar also could not detect optically thin high clouds (e.g., the orange–red-colored feature at the left-hand-side upper corner near 12 km in Fig. 7a). On the other hand, CALIOP could not penetrate the entire layer of dense clouds due to the large attenuation of the clouds at the lidar wavelengths (532 and 1064 nm). However, as designed, CloudSat and CALIOP together can provide more complete measurements of aerosol and cloud distributions.

As mentioned earlier, the addition of more attributes can increase the degree of separation between clouds and aerosols and, consequently, improve the classification. As an example, we have derived layer-effective lidar ratios for the total attenuated dust layer and for the clouds between 41° and 50°N, using an approach for totally attenuated layers described by Platt et al. (1999). We present the results in Fig. 7d. The lidar ratio is the ratio of extinction to backscatter coefficients, and the layer effective lidar ratio is the ratio of integrated particulate backscatter to layer optical depth in the presence of multiple scattering. As seen in Figs. 7d and 7e, the dust and cloud layers are clearly separated into two groups by the effective lidar ratio: the dust layers have a value of 19.7 ± 2.2 sr and the cloud layers have a value of 9.5 ± 1.5 sr. The lidar ratio for water clouds is ∼18 sr (Pinnick et al. 1983). The effective lidar ratio can be reduced significantly in the presence of multiple scattering as demonstrated by the CALIOP measurement in cirrus clouds (Vaughan et al. 2008). The values (9.5 ± 1.5 sr) presented in Fig. 7e are typical of water clouds in the presence of multiple scattering. Lidar ratios for dust are normally higher, ranging from 30 to 80 sr at 532 nm (e.g., Liu et al. 2002; Cattrall et al. 2005; Muller et al. 2007). A value of ∼41 sr was observed by CALIOP for a moderate Saharan dust layer over the eastern North Atlantic, and a slightly larger value was measured over the Gulf of Mexico for transported Saharan dust by the NASA Langley Research Center’s High-Spectral-Resolution Lidar (HSRL) (Liu et al. 2008b). The low lidar ratio derived for this dust event may be due to differences in the particle sizes, as the particles occurring in dust storms immediately over the source regions should normally be much larger than those that are lofted and transported over large distances. The low lidar ratio may also be due to the multiple scattering. The dust layer in this example has a very high optical thickness, which can produce significant multiple scattering. The presence of multiple scattering can reduce the effective lidar ratio; the more significant the multiple scattering, the smaller the effective lidar ratio.

We have performed an analysis for one month’s worth of data (July 2006), attempting to quantify the misclassification of dense dust. This was done based on the fact that, when a dust layer is misclassified as cloud, the layer will likely be classified as an ice cloud in the downstream data processing by the ice–water discrimination algorithm (HuSI) using the depolarization ratio measurement. Dust storms occur normally in dry and hot air conditions over the source regions during daytime due to the enhanced convection in a deepened boundary layer (e.g., Ackerman and Cox 1987; N’Tchayi Mbourou et al. 1997). By checking the midlayer temperature, the misclassified dense dust layers can be identified in most cases. In our analysis to evaluate misclassified dust layers, if a layer in the dust belt region defined in this paper is classified as ice cloud and its midlayer temperature is larger than 0°C, this layer is identified as “hot cirrus.” The hot cirrus feature is not an ice cloud but a misclassification of a dust or water cloud layer or a mixture of dust and cloud. Our analysis showed that the fraction of hot cirrus is ∼1.4% of total tropospheric features. Among these hot cirrus features, mixed layers of dust and cloud should be classified as cloud by convention (section 2e). Further analyses have also been performed to partition the hot cirrus features. This analysis checks the CloudSat measurement: if no cloud is seen in the nearest CloudSat profiles, this hot cirrus layer is identified as a misclassified dust layer. Misclassification of water clouds as ice has been assessed by applying the same temperature test to the hot cirrus found in regions over remote southern oceans, where dust is not expected to be present. Applying these additional analyses showed that misclassified dust layers represent <50% of the hot cirrus features, corresponding to ∼0.7% of the total features.

Relatively dense smoke, particularly over or near source regions, is another aerosol type that can be occasionally misclassified. Its occurrence frequency is estimated to be smaller than dense dust, though we have not quantified this type misclassification.

c. Zonal distribution

Zonal distributions of cloud and aerosol features derived from the July 2006 CALIOP layer products data are presented in Fig. 8. Three modes are seen in the cloud distribution. One mode is peaked at ∼10°N, corresponding to the intertropical convergence zone (ITCZ). The other two modes are peaked at ∼65°S and in the Arctic. It is seen that aerosols are distributed mostly in low latitudes. The major dust sources are mainly located over the continents in Northern Hemisphere (Prospero et al. 2002). The dust aerosols can be transported for long distances across the tropical North Atlantic and North Pacific Oceans or to the Indian Ocean. Polluted aerosol is another aerosol type that can dominate some industrial areas in the Northern Hemisphere (Herman et al. 1997; Torres et al. 1998). These aerosol types contribute largely to the aerosol mode in the Northern Hemisphere. Smoke aerosols due to biomass burning in central and southern Africa show an annual pattern that reaches a maximum in the Southern Hemisphere winter and that can dominate the aerosol loading in the tropical South Atlantic (Herman et al. 1997; Torres et al. 1998). This is responsible for the peak seen in the tropical Southern Hemisphere. Maritime aerosols with large backscatters are often observed by CALIOP over the southern waters of the Atlantic and Pacific Oceans, and these contribute to the long tail of the southern aerosol mode.

Approaching the polar regions, the frequency of aerosol occurrence drops quickly. This behavior is expected, as the polar regions are remote from sources and aerosol layers with backscatter intensities above the CALIOP detection threshold are not often present. However, even in this relatively clean region of the globe, aerosol layers are still identified with some regularity. Polar clouds are often low-lying, optically thin, and spatially diffuse clouds, and can be composed of ice, water, or a mixture of both. The spatial distributions and optical properties of polar clouds are distinctly different from the distributions encountered at lower latitudes, and this can cause difficulties for the CAD algorithm. At present, a single set of latitude-independent PDFs is used globally for all CAD decisions. This set of PDFs is more representative of the cloud and aerosol distributions at lower latitudes than at higher latitudes. Consequently, some misclassifications of optically thin clouds (or edges of such clouds) can occasionally occur. In particular, thin ice clouds, which in the polar regions can extend from the surface to several kilometers in altitude, are sometimes misclassified as aerosol. We plan to develop latitude-dependent PDFs for future data releases to improve the classification in the polar regions.

4. Summary

In this paper, we have reviewed the theoretical basis of the CALIOP cloud and aerosol discrimination (CAD) algorithm, and described the enhancements incorporated into the version 2 algorithm that is used in the current data release (release 2). We also presented preliminary assessment results of the CAD performance based on one full day of expert manual classification and one month’s worth of CALIOP 5-km cloud and aerosol layer products. Overall, the CAD algorithm works well in most cases. The 1-day manual verification of the classifications suggests that the success rate is in the neighborhood of 90% or better.

The CAD error budget is characterized by several typical misclassification scenarios. Among these, the most prevalent are dense dust and smoke over or close to the source regions. Because the mean attenuated backscatter and mean attenuated total color ratio of these layer types are both relatively large, and thus similar to what would be expected for clouds, these features can be occasionally misclassified as cloud. The analysis of the 1-month data (July 2006), combined with the corresponding CloudSat measurements, showed that the misclassification of dust as ice clouds is ∼4% of the total features (i.e., clouds and aerosols) found. Smoke layers are misclassified as cloud less frequently than are dust layers. Though the fraction is small compared with the entire aerosol features found globally, the misclassification of optically thin clouds as aerosols may be significant if the analysis is limited in the polar regions. In the polar regions, many optically thin clouds, both ice and water, are encountered in the low atmosphere. These clouds, which can extend from the surface to several kilometers in altitude, are sometimes misclassified as aerosol.

CAD scores should be used to screen the CALIOP layer products to minimize the impacts of misclassifications. Several data screening approaches can be considered, including the following:

  • (a) As described earlier, features that have a special CAD score of −101 and 103 should be excluded from the data analysis in the scientific application.

  • (b) Detection of features underneath attenuating layers becomes more uncertain for a variety of reasons: low signal, incorrect attenuation correction, or instrumental artifacts. Confidence scores should be examined and features with score values less than 20 should be considered to be unreliable. A threshold of 20 has been used in generating the standard CALIOP vertical feature mask browse images (see, e.g., the bottom panels in Figs. 6 and 7b). A larger threshold between 20 and 90 may be selected for cloud features, depending on the application of the data. A threshold of 90 will remove ∼4.5% of the complete cloud dataset (Fig. 5b and Table 3).

  • (c) Misclassified dense dust layers in the defined dust belt can be identified using the temperature approach as described in section 3b. That is, for a cirrus cloud in the geographic region of 0°–50°N and 40°W–130°E, its midlayer temperature T should be compared with a threshold Tth: if T > Tth, this layer is very likely a misclassified dust layer and, thus, should be rejected from any data analysis devoted to clouds. We used Tth = 0°C in this paper, together with the CloudSat measurements, to evaluate the misclassification of dust aerosols.

  • (d) For dense smoke features misclassified as clouds, the sharp gradient of the attenuated total color ratio with respect to increasing layer penetration depth can be used to establish the correct classification. Due to strong spectral differences in absorption efficiency, the attenuation of smoke is much larger at 532 than at 1064 nm, and thus the backscatter signals from smoke decrease much faster at 532 than 1064 nm, causing the attenuated total color ratio to grow rapidly to large values (>1.5) in the lower regions of a smoke layer. In general, a dense smoke layer can be identified by the nature of the low-volume depolarization ratio and the high total color ratio (Liu et al. 2008a). In cases where the effective lidar ratio can be estimated, this parameter can also be used to separate smoke from clouds.

The CAD algorithm will be modified to improve the CALIOP cloud and aerosol classification for future data releases. Several things can be considered. First, a set of PDFs that is latitudinally and seasonally dependent will be developed. This step alone should greatly improve CAD performance. Second, to better separate dense dust layers from clouds at lower altitudes, we plan to incorporate volume depolarization ratios into the PDFs developed for future data releases. Recent offline tests have shown that the addition of the volume depolarization ratio can effectively distinguish dense dust layers from clouds at lower altitudes. The CloudSat measurements would also provide useful ancillary data to help in identifying dense aerosols that may be misclassified as cloud.

Acknowledgments

The authors wish to acknowledge the many and substantial contributions of the members of the CALIPSO Lidar Science Working Group. In particular, we thank Kam-Pui Lee, the chief architect of the analysis codes that produce all of the level 1 lidar data products from which the level 2 data products are derived, for his valuable support.

REFERENCES

  • Ackerman, S. A., and Cox S. K. , 1987: Surface weather observations of atmospheric dust over the Southwest summer monsoon region. Meteor. Atmos. Phys., 41 , 1934.

    • Search Google Scholar
    • Export Citation
  • Cairo, F., Donfrancesco G. , Adriani A. , Pulvirenti L. , and Federico F. , 1999: Comparison of various linear depolarization parameters measured by lidar. Appl. Opt., 38 , 44254432.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cattrall, C., Reagan J. , Thome K. , and Dubovik O. , 2005: Variability of aerosol and spectral lidar and backscatter and extinction ratios of key aerosol types derived from selected Aerosol Robotic Network locations. J. Geophys. Res., 110 , D10S11. doi:10.1029/2004JD005124.

    • Search Google Scholar
    • Export Citation
  • Charlson, R., Ackerman A. , Bender F. , Anderson T. , and Liu Z. , 2007: On the climate forcing consequences of the albedo continuum between cloudy and clear air. Tellus, 59 , 715727.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herman, J. R., Bhartia P. K. , Torres O. , Hsu C. , Seftor C. , and Celarier E. , 1997: Global distribution of UV-absorbing aerosols from Nimbus7/TOMS data. J. Geophys. Res., 102 , 1691116922.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hess, M., Koepke P. , and Schult I. , 1998: Optical properties of aerosols and clouds: The software package OPAC. Bull. Amer. Meteor. Soc., 79 , 831844.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hostetler, C., and Coauthors, 2006: CALIOP algorithm theoretical basis document—Part 1: Calibration and level 1 data products. PC-SCI-201, Release 1.0, NASA Langley Research Center, Hampton, VA, 66 pp. [Available online at http://www-calipso.larc.nasa.gov/resources/pdfs/PC-SCI-201v1.0.pdf].

    • Search Google Scholar
    • Export Citation
  • Hu, Y., and Coauthors, 2009: CALIPSO/CALIOP cloud phase discrimination algorithm. J. Atmos. Oceanic Technol., in press.

  • Hunt, W., Winker D. , Vaughan M. , Powell K. , Lucker P. , and Weimer C. , 2009: CALIPSO lidar description and performance assessment. J. Atmos. Oceanic Technol., 26 , 12141228.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • King, M., Closs J. , Spangler S. , Greenstone R. , Wharton S. , and Myers M. , 2004: EOS Data Products Handbook. Vol. 1. EOS Project Office, 260 [Available from EOS Project Science Office, Code 900, NASA Goddard Space Flight Center, Greenbelt, MD 20771; or online at http://eospso.gsfc.nasa.gov/ftp_docs/data_products_1.pdf].

    • Search Google Scholar
    • Export Citation
  • Liu, Z., Sugimoto N. , and Murayama T. , 2002: Extinction-to-backscatter ratio of Asian dust observed by high-spectral-resolution lidar and Raman lidar. Appl. Opt., 41 , 27602767.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., Vaughan M. , Winker D. , Hostetler C. , Poole L. , Hlavka D. , Hart W. , and McGill M. , 2004: Use of probability distribution functions for discriminating between cloud and aerosol in lidar backscatter data. J. Geophys. Res., 109 , D15202. doi:10.1029/2004JD004732.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., Omar A. , Hu Y. , Vaughan M. , and Winker D. , 2005: CALIOP algorithm theoretical basis document—Part 3: Scene classification algorithms. Release 1.0, PC-SCI-202, NASA Langley Research Center, Hampton, VA, 56 [Available online at http://www-calipso.larc.nasa.gov/resources/pdfs/PC-SCI-202_Part3_v1.0.pdf].

    • Search Google Scholar
    • Export Citation
  • Liu, Z., and Coauthors, 2008a: Airborne dust distributions over the Tibetan Plateau and surrounding areas derived from the first year of CALIPSO lidar observations. Atmos. Chem. Phys. Discuss., 8 , 59575977.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., and Coauthors, 2008b: CALIPSO lidar observations of the optical properties of Saharan dust: A case study of long-range transport. J. Geophys. Res., 113 , D07207. doi:10.1029/2007JD008878.

    • Search Google Scholar
    • Export Citation
  • Muller, D., Ansmann A. , Mattis I. , Tesche M. , Wandinger U. , Althausen D. , and Pisani G. , 2007: Aerosol-type-dependent lidar-ratio observed with Raman lidar. J. Geophys. Res., 112 , D16202. doi:10.1029/2006JD008292.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • N’Tchayi Mbourou, G., Bertrand J. , and Nicholson S. , 1997: The diurnal and seasonal cycles of wind-borne dust over Africa north of the equator. J. Appl. Meteor., 36 , 868881.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Omar, A., Won J. , Winker D. , Yoon S. , Dubovik O. , and McCormick M. , 2005: Development of global aerosol models using cluster analysis of Aerosol Robotic Network (AERONET) measurements. J. Geophys. Res., 110 , D10S14. doi:10.1029/2004JD004874.

    • Search Google Scholar
    • Export Citation
  • Omar, A., and Coauthors, 2009: The CALIPSO automated aerosol classification and lidar ratio selection algorithm. J. Atmos. Oceanic Technol., in press.

    • Search Google Scholar
    • Export Citation
  • Pinnick, R. G., Jennings S. G. , Chylek P. , Ham C. , and Grandy W. T. Jr., 1983: Backscatter and extinction in water clouds. J. Geophys. Res., 88 , 67876796.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pitts, M. C., Thomason L. W. , Poole L. R. , and Winker D. M. , 2007: Characterization of polar stratospheric clouds with spaceborne lidar: CALIPSO and the 2006 Antarctic season. Atmos. Chem. Phys., 7 , 52075228.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Platt, C. M., 1978: Lidar backscatter from horizontal ice crystal plates. J. Appl. Meteor., 17 , 482488.

  • Platt, C., Winker D. , Vaughan M. , and Miller S. , 1999: Backscatter-to-extinction ratios in the top layers of tropical mesoscale convective systems and in isolated cirrus from LITE observations. J. Appl. Meteor., 38 , 13301345.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Powell, K., and Coauthors, 2009: CALIPSO lidar calibration algorithms. Part I: Nighttime 532-nm parallel channel and 532-nm perpendicular channel. J. Atmos. Oceanic Technol., in press.

    • Search Google Scholar
    • Export Citation
  • Prospero, J. M., Ginoux P. , Torres O. , Nicholson S. E. , and Gill T. E. , 2002: Environmental characterization of global sources of atmospheric soil dust identified with the NIMBUS 7 Total Ozone Mapping Spectrometer (TOMS) absorbing aerosol product. Rev. Geophys., 40 , 1002. doi:10.1029/2000RG000095.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pueschel, R. F., and Livingston J. M. , 1990: Aerosol spectral optical depths: Jet fuel and forest fire smokes. J. Geophys. Res., 95 , 2241722422.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephens, G. L., and Coauthors, 2002: The CloudSat mission and the A-Train: A new dimension of space-based observations of clouds and precipitation. Bull. Amer. Meteor. Soc., 83 , 17711790.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stephens, G. L., and Coauthors, 2008: The CloudSat mission: Performance and early science after the first year of operation. J. Geophys. Res., 113 , D00A18. doi:10.1029/2008JD009982.

    • Search Google Scholar
    • Export Citation
  • Torres, O., Bhartiea P. , Herman J. , Ahmad Z. , and Gleason, 1998: Derivation of aerosol properties from satellite measurements of backscattered ultraviolet radiation: Theoretical basis. J. Geophys. Res., 103 , (D14). 1709917110.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, M., Young S. , Winker D. , Powell K. , Omar A. , Liu Z. , Hu Y. , and Hostetler C. , 2004: Fully automated analysis of space-based lidar data: An overview of the CALIPSO retrieval algorithms and data products. Proc. SPIE, 5575 , 1630.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, M., Kuehn R. , Young S. , McGill M. , Liu Z. , and Hu Y. , 2008: Validating cirrus cloud optical properties retrieved by CALIPSO. 24th Int. Laser Radar Conf., Boulder, CO, NOAA–NASA, 1090–1093.

    • Search Google Scholar
    • Export Citation
  • Vaughan, M., and Coauthors, 2009: Fully automated detection of cloud and aerosol layers in the CALIPSO lidar measurements. J. Atmos. Oceanic Technol., in press.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., Couch R. H. , and McCormick M. P. , 1996: An overview of LITE: NASA’s Lidar In-Space Technology Experiment. Proc. IEEE, 84 , 164180.

  • Winker, D. M., Hunt W. , and McGill M. , 2007: Initial performance assessment of CALIOP. Geophys. Res. Lett., 34 , L19803. doi:10.1029/2007GL030135.

  • Winker, D. M., Vaughan M. A. , Omar A. H. , Hu Y. , Powell K. A. , Liu Z. , Hunt W. H. , and Young S. A. , 2009: Overview of the CALIPSO mission and CALIOP data processing algorithms. J. Atmos. Oceanic Technol., in press.

    • Search Google Scholar
    • Export Citation
  • Young, S., and Vaughan M. , 2009: The retrieval of profiles of particulate extinction from Cloud Aerosol Lidar Infrared Pathfinder Satellite Observation (CALIPSO) data: Algorithm description. J. Atmos. Oceanic Technol., 26 , 11051119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yumimoto, K., Uno I. , Sugimoto N. , Shimizu A. , Liu Z. , and Winker D. M. , 2008: Adjoint inversion modeling of Asian dust emission using lidar observations. Atmos. Chem. Phys., 8 , 28692884.

    • Crossref
    • Search Google Scholar
    • Export Citation

Fig. 1.
Fig. 1.

(a) Scatterplot with contour lines, and (b) occurrence distribution of clouds (blue) and aerosols (red) whose midlayer altitude falls between 0 and 1 km, measured by CALIOP on 12 Aug 2006, classified based on expert manual classification. The contours in (a) represent number densities of features of 5 and 25 corresponding to ∼10% and ∼50% of the maximum density, respectively. The layer-averaged attenuated backscatter is in km−1 sr−1.

Citation: Journal of Atmospheric and Oceanic Technology 26, 7; 10.1175/2009JTECHA1229.1

Fig. 2.
Fig. 2.

Flowchart of the version 2 CAD algorithm.

Citation: Journal of Atmospheric and Oceanic Technology 26, 7; 10.1175/2009JTECHA1229.1

Fig. 3.
Fig. 3.

(top) The threshold used in the depolarization ratio test and (bottom) a map defining a dust belt used in this paper.

Citation: Journal of Atmospheric and Oceanic Technology 26, 7; 10.1175/2009JTECHA1229.1

Fig. 4.
Fig. 4.

An example of the volume depolarization ratio as a function of the aerosol mixing ratio (R−1) for an assumed particulate depolarization ratio of 0.3 and a molecular depolarization ratio of 0.0036.

Citation: Journal of Atmospheric and Oceanic Technology 26, 7; 10.1175/2009JTECHA1229.1

Fig. 5.
Fig. 5.

(a) Number of occurrences and (b) cumulative occurrence frequency of the CAD score derived from one month (July 2006) of the CALIOP layer products data (version 2.01).

Citation: Journal of Atmospheric and Oceanic Technology 26, 7; 10.1175/2009JTECHA1229.1

Fig. 6.
Fig. 6.

Example of the cloud and aerosol discrimination measured by CALIOP on 27 Jun 2007 from a nighttime orbit across central and southern Europe and North Africa. (top) Attenuated backscatter at 532 nm, along with the orbit track (pink line); (middle) volume depolarization ratio; and (bottom) the corresponding classification results. Feature types in the bottom panel: 0, invalid (|CAD| < 20 or CAD = 103); 1, clear air; 2, cloud; 3, aerosol; 4, stratospheric feature; 5, surface; 6, subsurface; and 7, no signal (total attenuated).

Citation: Journal of Atmospheric and Oceanic Technology 26, 7; 10.1175/2009JTECHA1229.1

Fig. 7.
Fig. 7.

Example of misclassification of a dust storm generated in the Gobi Desert observed on 30 Mar 2007: (a) CALIOP 532-nm attenuated backscatter and (b) VFM (same color bar as in Fig. 5), (c) CloudSat vertical feature mask, (d) effective lidar ratio at 532 nm and (e) its histogram distribution, and (f) the CALIOP track. The circled features in (a) are dust layers that have been misclassified. Note that there are small-scale clouds on the top of each circled scene (7–8 km around 37.1° and 4–5 km near 44°).

Citation: Journal of Atmospheric and Oceanic Technology 26, 7; 10.1175/2009JTECHA1229.1

Fig. 8.
Fig. 8.

Zonal distribution of cloud and aerosol features, derived from one month (July 2006) of 5-km-layer products, scaled to the total number of 5-km profiles measured in the 0°–10° latitude band. Features with special CAD scores of −101 and 103 have been excluded.

Citation: Journal of Atmospheric and Oceanic Technology 26, 7; 10.1175/2009JTECHA1229.1

Table 1.

CAD performance based on 1 day of manual classification.

Table 1.
Table 2.

CAD score interpretation and CAD QA flags reported in VFM.

Table 2.
Table 3.

Occurrence frequencies for selected CAD score ranges and special scores.

Table 3.
Save