• Alvarez, J. M., Vaughan M. A. , Hostetler C. A. , Hunt W. H. , and Winker D. M. , 2006: Calibration technique for polarization-sensitive lidars. J. Atmos. Oceanic Technol., 23, 683699, doi:10.1175/JTECH1872.1.

    • Search Google Scholar
    • Export Citation
  • Behrendt, A., and Nakamura T. , 2002: Calculation of the calibration constant of polarization lidar and its dependency on atmospheric temperature. Opt. Express, 10, 805817, doi:10.1364/OE.10.000805.

    • Search Google Scholar
    • Export Citation
  • Bevington, P., and Robinson D. , 2002: Data Reduction and Error Analysis for the Physical Sciences. 3rd ed. McGraw-Hill, 336 pp.

  • Borg, L. A., Holz R. E. , and Turner D. D. , 2011: Investigating cloud radar sensitivity to optically thin cirrus using collocated Raman lidar observations. Geophys. Res. Lett., 38, L05807, doi:10.1029/2010GL046365.

    • Search Google Scholar
    • Export Citation
  • Bucholtz, A., 1995: Rayleigh-scattering calculations for the terrestrial atmosphere. Appl. Opt., 34, 27652773, doi:10.1364/AO.34.002765.

    • Search Google Scholar
    • Export Citation
  • Campbell, J. R., Hlavka D. L. , Welton E. J. , Flynn C. J. , Turner D. D. , Spinhirne J. D. , Scott V. S. , and Hwang I. H. , 2002: Full-time, eye-safe cloud and aerosol lidar observation at Atmospheric Radiation Measurement Program sites: Instruments and data processing. J. Atmos. Oceanic Technol., 19, 431, doi:10.1175/1520-0426(2002)019<0431:FTESCA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Campbell, J. R., Sassen K. , and Welton E. J. , 2008: Elevated cloud and aerosol layer retrievals from micropulse lidar signal profiles. J. Atmos. Oceanic Technol., 25, 685700, doi:10.1175/2007JTECHA1034.1.

    • Search Google Scholar
    • Export Citation
  • Carswell, A. I., and Pal S. R. , 1980: Polarization anisotropy in lidar multiple scattering from clouds. Appl. Opt., 19, 41234126, doi:10.1364/AO.19.004123.

    • Search Google Scholar
    • Export Citation
  • Clothiaux, E. E., Mace G. G. , Ackerman T. P. , Kane T. J. , Spinhirne J. D. , and Scott V. S. , 1998: An automated algorithm for detection of hydrometeor returns in micropulse lidar data. J. Atmos. Oceanic Technol., 15, 10351042, doi:10.1175/1520-0426(1998)015<1035:AAAFDO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Clothiaux, E. E., Ackerman T. P. , Mace G. G. , Moran K. P. , Marchand R. T. , Miller M. A. , and Martner B. E. , 2000: Objective determination of cloud heights and radar reflectivities using a combination of active remote sensors at the ARM CART sites. J. Appl. Meteor., 39, 645665, doi:10.1175/1520-0450(2000)039<0645:ODOCHA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Comstock, J. M., Ackerman T. P. , and Mace G. G. , 2002: Ground-based lidar and radar remote sensing of tropical cirrus clouds at Nauru Island: Cloud statistics and radiative impacts. J. Geophys. Res., 107, 4714, doi:10.1029/2002JD002203.

    • Search Google Scholar
    • Export Citation
  • Comstock, J. M., Protat A. , McFarlane S. A. , Delanoë J. , and Deng M. , 2013: Assessment of uncertainty in cloud radiative effects and heating rates through retrieval algorithm differences: Analysis using 3 years of ARM data at Darwin, Australia. J. Geophys. Res. Atmos., 118, 45494571, doi:10.1002/jgrd.50404.

    • Search Google Scholar
    • Export Citation
  • Cooney, J., Orr J. , and Tomasetti C. , 1969: Measurements separating the gaseous and aerosol components of laser atmospheric backscatter. Nature, 224, 10981099, doi:10.1038/2241098a0.

    • Search Google Scholar
    • Export Citation
  • Coulter, R., 2012: Micropulse lidar (MPL) handbook. U.S. Department of Energy Tech. Rep. DOE/SC-ARM/TR-019, 12 pp. [Available online at https://www.arm.gov/publications/tech_reports/handbooks/mpl_handbook.pdf.]

  • Davis, S., and Coauthors, 2010: In situ and lidar observations of tropopause subvisible cirrus clouds during TC4. J. Geophys. Res., 115, D00J17, doi:10.1029/2009JD013093.

    • Search Google Scholar
    • Export Citation
  • Dessler, A. E., and Yang P. , 2003: The distribution of tropical thin cirrus clouds inferred from Terra MODIS data. J. Climate, 16, 12411247, doi:10.1175/1520-0442(2003)16<1241:TDOTTC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Dupont, J.-C., Haeffelin M. , Morille Y. , Comstock J. M. , Flynn C. , Long C. N. , Sivaraman C. , and Newson R. K. , 2011: Cloud properties derived from two lidars over the ARM SGP site. Geophys. Res. Lett., 38, L08814, doi:10.1029/2010GL046274.

    • Search Google Scholar
    • Export Citation
  • Feldman, D. R., L’Ecuyer T. S. , Liou K. N. , and Yung Y. L. , 2008: Remote sensing of tropical tropopause layer radiation balance using A-train measurements. J. Geophys. Res., 113, D21113, doi:10.1029/2008JD010158.

    • Search Google Scholar
    • Export Citation
  • Ferrare, R., and Coauthors, 2006: Evaluation of daytime measurements of aerosols and water vapor made by an operational Raman lidar over the Southern Great Plains. J. Geophys. Res., 111, D05S08, doi:10.1029/2005JD005836.

    • Search Google Scholar
    • Export Citation
  • Fu, Q., Hu Y. , and Yang Q. , 2007: Identifying the top of the tropical tropopause layer from vertical mass flux analysis and CALIPSO lidar cloud observations. Geophys. Res. Lett., 34, L14813, doi:10.1029/2007GL030099.

    • Search Google Scholar
    • Export Citation
  • Gobbi, G. P., 1998: Polarization lidar returns from aerosols and thin clouds: A framework for the analysis. Appl. Opt., 37, 55055508, doi:10.1364/AO.37.005505.

    • Search Google Scholar
    • Export Citation
  • Goldsmith, J. E. M., Blair F. H. , Bisson S. E. , and Turner D. D. , 1998: Turn-key Raman lidar for profiling atmospheric water vapor, clouds, and aerosols. Appl. Opt., 37, 49794990, doi:10.1364/AO.37.004979.

    • Search Google Scholar
    • Export Citation
  • Grund, C. J., and Eloranta E. W. , 1991: University of Wisconsin high spectral resolution lidar. Opt. Eng., 30, 612, doi:10.1117/12.55766.

    • Search Google Scholar
    • Export Citation
  • Hair, J. W., and Coauthors, 2008: Airborne High Spectral Resolution Lidar for profiling aerosol optical properties. Appl. Opt., 47, 67346752, doi:10.1364/AO.47.006734.

    • Search Google Scholar
    • Export Citation
  • Haladay, T., and Stephens G. , 2009: Characteristics of tropical thin cirrus clouds deduced from joint CloudSat and CALIPSO observations. J. Geophys. Res., 114, D00A25, doi:10.1029/2008JD010675.

    • Search Google Scholar
    • Export Citation
  • Hogan, R. J., and Illingworth A. J. , 2000: Deriving cloud overlap statistics from radar. Quart. J. Roy. Meteor. Soc., 126, 29032909, doi:10.1002/qj.49712656914.

    • Search Google Scholar
    • Export Citation
  • Holben, B., and Coauthors, 1998: AERONET—A federated instrument network and data archive for aerosol characterization. Remote Sens. Environ., 66, 116, doi:10.1016/S0034-4257(98)00031-5.

    • Search Google Scholar
    • Export Citation
  • Kingston, R., 1978: Detection of Optical and Infrared Radiation. Springer Series in Optical Sciences, Vol. 10, Springer-Verlag, 142 pp.

  • Liljegren, J. C., and Lesht B. M. , 1996: Measurements of integrated water vapor and cloud liquid water from microwave radiometers at the DOE ARM Cloud and Radiation Testbed in the U.S. Southern Great Plains. IGARSS ’96: 1996 International Geoscience and Remote Sensing Symposium; Remote Sensing for a Sustainable Geoscience and Remote Sensing Symposium, Vol. 3, IEEE, 1675–1677.

  • Luo, Y., Xu K.-M. , Morrison H. , McFarquhar G. M. , Wang Z. , and Zhang G. , 2008: Multi-layer Arctic mixed-phase clouds simulated by a cloud-resolving model: Comparison with ARM observations and sensitivity experiments. J. Geophys. Res., 113, D12208, doi:10.1029/2007JD009563.

    • Search Google Scholar
    • Export Citation
  • Massie, S. T., Gille J. , Craig C. , Khosravi R. , Barnett J. , Read W. , and Winker D. , 2010: HIRDLS and CALIPSO observations of tropical cirrus. J. Geophys. Res., 115, D00H11, doi:10.1029/2009JD012100.

    • Search Google Scholar
    • Export Citation
  • Matthais, V., and Coauthors, 2004: Aerosol lidar intercomparison in the framework of the EARLINET project. 1. Instruments. Appl. Opt., 43, 961976, doi:10.1364/AO.43.000961.

    • Search Google Scholar
    • Export Citation
  • Measures, R. M., 1984: Laser Remote Sensing: Fundamentals and Applications. Wiley, 510 pp.

  • Melfi, S. H., 1972: Remote measurements of the atmosphere using Raman scattering. Appl. Opt., 11, 16051610, doi:10.1364/AO.11.001605.

    • Search Google Scholar
    • Export Citation
  • Murayama, T., and Coauthors, 2001: Ground-based network observation of Asian dust events of April 1998 in east Asia. J. Geophys. Res., 106, 18 34518 359, doi:10.1029/2000JD900554.

    • Search Google Scholar
    • Export Citation
  • Newsom, R. K., 2009: Raman lidar (RL) handbook. U.S. Department of Energy Tech. Rep. DOE/SC-ARM/TR-038, 25 pp. [Available online at https://www.arm.gov/publications/tech_reports/handbooks/rl_handbook.pdf.]

  • Newsom, R. K., Turner D. D. , Mielke B. , Clayton M. , Ferrare R. , and Sivaraman C. , 2009: Simultaneous analog and photon counting detection for Raman lidar. Appl. Opt., 48, 39033914, doi:10.1364/AO.48.003903.

    • Search Google Scholar
    • Export Citation
  • Newsom, R. K., Turner D. D. , and Goldsmith J. E. M. , 2013: Long-term evaluation of temperature profiles measured by an operational Raman lidar. J. Atmos. Oceanic Technol., 30, 16161634, doi:10.1175/JTECH-D-12-00138.1.

    • Search Google Scholar
    • Export Citation
  • Oliver, B. M., 1965: Thermal and quantum noise. Proc. IEEE, 53, doi:10.1109/PROC.1965.3814.

  • Riihimaki, L. D., McFarlane S. A. , and Comstock J. M. , 2012: Climatology and formation of tropical midlevel clouds at the Darwin ARM site. J. Climate, 25, 68356850, doi:10.1175/JCLI-D-11-00599.1.

    • Search Google Scholar
    • Export Citation
  • Sassen, K., 1991: The polarization lidar technique for cloud research: A review and current assessment. Bull. Amer. Meteor. Soc., 72, 18481866, doi:10.1175/1520-0477(1991)072<1848:TPLTFC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Sassen, K., 2002: Indirect climate forcing over the western US from Asian dust storms. Geophys. Res. Lett., 29, 103-1103-4, doi:10.1029/2001GL014051.

    • Search Google Scholar
    • Export Citation
  • Stephens, G. L., and Coauthors, 2002: The CloudSat mission and the A-Train—A new dimension of space-based observations of clouds and precipitation. Bull. Amer. Meteor. Soc., 83, 17711790, doi:10.1175/BAMS-83-12-1771.

    • Search Google Scholar
    • Export Citation
  • Sun, W., Videen G. , Kato S. , Lin B. , Lukashin C. , and Hu Y. , 2011: A study of subvisual clouds and their radiation effect with a synergy of CERES, MODIS, CALIPSO, and AIRS data. J. Geophys. Res., 116, D22207, doi:10.1029/2011JD016422.

    • Search Google Scholar
    • Export Citation
  • Thorsen, T. J., and Fu Q. , 2015: Automated retrieval of cloud and aerosol properties from the ARM Raman lidar. Part II: Extinction. J. Atmos. Oceanic Technol., doi:10.1175/JTECH-D-14-00178.1, in press.

    • Search Google Scholar
    • Export Citation
  • Thorsen, T. J., Fu Q. , and Comstock J. , 2011: Comparison of the CALIPSO satellite and ground-based observations of cirrus clouds at the ARM TWP sites. J. Geophys. Res., 116, D21203, doi:10.1029/2011JD015970.

    • Search Google Scholar
    • Export Citation
  • Thorsen, T. J., Fu Q. , and Comstock J. M. , 2013a: Cloud effects on radiative heating rate profiles over Darwin using ARM and A-train radar/lidar observations. J. Geophys. Res. Atmos., 118, 56375654, doi:10.1002/jgrd.50476.

    • Search Google Scholar
    • Export Citation
  • Thorsen, T. J., Fu Q. , Comstock J. M. , Sivaraman C. , Vaughan M. A. , Winker D. M. , and Turner D. D. , 2013b: Macrophysical properties of tropical cirrus clouds from the CALIPSO satellite and from ground-based micropulse and Raman lidars. J. Geophys. Res. Atmos., 118, 92099220, doi:10.1002/jgrd.50691.

    • Search Google Scholar
    • Export Citation
  • Turner, D. D., Ferrare R. A. , Brasseur L. A. H. , Feltz W. F. , and Tooman T. P. , 2002: Automated retrievals of water vapor and aerosol profiles from an operational Raman lidar. J. Atmos. Oceanic Technol., 19, 3750, doi:10.1175/1520-0426(2002)019<0037:AROWVA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Vaughan, M. A., and Coauthors, 2009: Fully automated detection of cloud and aerosol layers in the CALIPSO lidar measurements. J. Atmos. Oceanic Technol., 26, 20342050, doi:10.1175/2009JTECHA1228.1.

    • Search Google Scholar
    • Export Citation
  • Wandinger, U., and Ansmann A. , 2002: Experimental determination of the lidar overlap profile with Raman lidar. Appl. Opt., 41, 511, doi:10.1364/AO.41.000511.

    • Search Google Scholar
    • Export Citation
  • Wang, P.-H., Minnis P. , McCormick M. P. , Kent G. S. , Yue G. K. , Young D. F. , and Skeens K. M. , 1998: A study of the vertical structure of tropical (20°S-20°N) optically thin clouds from SAGE II observations. Atmos. Res., 47–48, 599614, doi:10.1016/S0169-8095(97)00085-9.

    • Search Google Scholar
    • Export Citation
  • Wang, Z., and Sassen K. , 2001: Cloud type and macrophysical property retrieval using multiple remote sensors. J. Appl. Meteor., 40, 16651682, doi:10.1175/1520-0450(2001)040<1665:CTAMPR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wang, Z., and Sassen K. , 2002: Cirrus cloud microphysical property retrieval using lidar and radar measurements. Part I: Algorithm description and comparison with in situ data. J. Appl. Meteor., 41, 218229, doi:10.1175/1520-0450(2002)041<0218:CCMPRU>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Welton, E. J., and Campbell J. R. , 2002: Micropulse lidar signals: Uncertainty analysis. J. Atmos. Oceanic Technol., 19, 20892094, doi:10.1175/1520-0426(2002)019<2089:MLSUA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Whiteman, D. N., 2003a: Examination of the traditional Raman lidar technique. I. Evaluating the temperature-dependent lidar equations. Appl. Opt., 42, 25712592, doi:10.1364/AO.42.002571.

    • Search Google Scholar
    • Export Citation
  • Whiteman, D. N., 2003b: Examination of the traditional Raman lidar technique. II. Evaluating the ratios for water vapor and aerosols. Appl. Opt., 42, 2593, doi:10.1364/AO.42.002593.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., and Vaughan M. , 1994: Vertical distribution of clouds over Hampton, Virginia observed by lidar under the ECLIPS and FIRE ETO programs. Atmos. Res., 34, 117133, doi:10.1016/0169-8095(94)90084-1.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., and Trepte C. R. , 1998: Laminar cirrus observed near the tropical tropopause by LITE. Geophys. Res. Lett., 25, 33513354, doi:10.1029/98GL01292.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., Vaughan M. A. , Omar A. , Hu Y. , Powell K. A. , Liu Z. , Hunt W. H. , and Young S. A. , 2009: Overview of the CALIPSO mission and CALIOP data processing algorithms. J. Atmos. Oceanic Technol., 26, 23102323, doi:10.1175/2009JTECHA1281.1.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., and Coauthors, 2010: The CALIPSO mission: A global 3D view of aerosols and clouds. Bull. Amer. Meteor. Soc., 91, 12111229, doi:10.1175/2010BAMS3009.1.

    • Search Google Scholar
    • Export Citation
  • Wulfmeyer, V., Pal S. , Turner D. , and Wagner E. , 2010: Can water vapour Raman lidar resolve profiles of turbulent variables in the convective boundary layer? Bound.-Layer Meteor., 136, 253284, doi:10.1007/s10546-010-9494-z.

    • Search Google Scholar
    • Export Citation
  • Yang, Q., Fu Q. , and Hu Y. , 2010: Radiative impacts of clouds in the tropical tropopause layer. J. Geophys. Res., 115, D00H12, doi:10.1029/2009JD012393.

    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    Schematic comparison of the (left) expected molecular signal to (right) the signal measured by the lidar.

  • View in gallery
    Fig. 2.

    (a) Scattering ratio derived using the elastic and nitrogen channels, (b) scattering ratio derived using the low elastic and nitrogen channels, (c) scattering ratio derived using only the elastic channel, and (d) the depolarization ratio after the final iteration of FEX for 24 Dec 2010 at Darwin.

  • View in gallery
    Fig. 3.

    Flow diagram for feature detection in the FEX algorithm.

  • View in gallery
    Fig. 4.

    The initial threshold (dashed blue) and final threshold (solid blue) for the profile at 1958:00 UTC in Fig. 2 for the (a) scattering ratio derived using both the elastic and nitrogen channel signals, (b) scattering ratio derived using both the low-elastic-and-nitrogen-channel signals, (c) scattering ratio derived using only the elastic channel, and (d) the depolarization ratio. Both the initial profiles (i.e., the first iteration, gray line) and final profiles (i.e., the final iteration, black line) are given. Potential features identified in these ratios on the initial and final iterations are noted by the color in the bars on the right-hand side of each panel; features that have been determined to be false detections are denoted by yellow and true features denoted by blue. For reference, the expected clear-sky ratio is given as a dashed red line. Note that the scale of the x axis in each panel is nonlinear.

  • View in gallery
    Fig. 5.

    Feature mask with height vs hour on the final iteration of FEX for the ratios given in Fig. 2. Each color represents a different combination of the four ratios—scattering ratio derived using: the elastic and nitrogen channels (EN), the low elastic and nitrogen channels (ENL), and only the elastic channel (E), and the depolarization ratio (D)—that detected the feature. (a) Potential features, i.e., bins where the ratio exceeded the threshold. (b) True features, i.e., those potential features that remain after removing false detections due to random noise and applying consistency checks as discussed in section 3e.

  • View in gallery
    Fig. 6.

    (a) Scattering ratio derived using the elastic and nitrogen channels, (b) scattering ratio derived using the low elastic and nitrogen channels, (c) scattering ratio derived using only the elastic channel, and (d) the depolarization ratio after the final iteration of FEX for 10 May 2013 at Darwin. (e) Feature mask where each color represents a different combination of the four ratios that detected the feature—scattering ratio derived using: the elastic and nitrogen channels (EN), the low elastic and nitrogen channels (ENL), and only the elastic channel (E), and the depolarization ratio (D).

  • View in gallery
    Fig. 7.

    The (a) daytime and (b) nighttime cloud occurrence profiles for cloud detected by any ratio (gray), the scattering ratio derived using: the low elastic and nitrogen channels (red), the elastic and nitrogen channels (green), and only the elastic channel (blue), and the depolarization ratio (brown) at Darwin from December 2010 through December 2014.

  • View in gallery
    Fig. 8.

    The median threshold profiles (red) for points where the laser beam has not been completely attenuated and the median ratio value (blue) for points where that ratio detected a feature. Separate panels are shown for each ratio—the scattering ratio derived using: (a),(b) the low elastic and nitrogen channels, (c),(d) the elastic and nitrogen channels, and (e),(f) only the elastic channel, and (g),(h) the depolarization ratio—and for (a),(c),(e),(g) daytime and (b),(d),(f),(h) nighttime profiles. The shaded region on both the threshold and ratio profiles encompasses the 5th–95th percentiles from their respective distributions.

  • View in gallery
    Fig. 9.

    The cloud occurrence profile from the ARM DEP product (red) and from the new FEX algorithm (blue) described in this work at Darwin from December 2010 through December 2014.

  • View in gallery
    Fig. 10.

    The cloud occurrence profile from the ARM MPL cloud mask (green) (Wang and Sassen 2001) and RL-FEX cloud mask (blue) from December 2010 through August 2011 for (a) all pixels and (b) only pixels where both the ARM MPL and RL SNR is >3 in profiles where rain was not detected.

  • View in gallery
    Fig. 11.

    The cloud occurrence in transparent profiles from the RL accumulated to 2-min and 30-m bins (blue) and from the CALIPSO VFM product (red). The profile from the RL data accumulated to 9 min and 30 m, which is approximately equivalent to CALIPSO’s averaging below 8.2 km, is given in brown. The profile from the RL data accumulated to 15 min and 60 m, which is approximately equivalent to CALIPSO’s averaging above 8.2 km, is given in gray. Profiles are shown separately for the (a) daytime and (b) nighttime observations. This comparison includes CALIPSO profiles that fall within 200 km of the Darwin site from December 2010 through December 2014. The RL data are limited to times within ±2 h of CALIPSO overpasses.

  • View in gallery
    Fig. 12.

    FEX feature mask for 10 May 2013 at Darwin when the signal was accumulated into 9-min and 30-m bins. Each color represents a different combination of the four ratios that detected the feature: scattering ratio derived using the elastic and nitrogen channels (EN), the low elastic and nitrogen channels (ENL), and only the elastic channel (E), and the depolarization ratio (D).

  • View in gallery
    Fig. 13.

    The aerosol occurrence in transparent profiles from the RL at 2 min and 30 m of averaging (blue) and from the CALIPSO VFM product (red). Also given is the vertical occurrence in the subset of transparent profiles where no clouds are detected (no cloud). Profiles are shown separately for the (a) daytime and (b) nighttime observations. The shading encompasses the uncertainty due to sampling (95% confidence) determined by bootstrapping. This comparison includes CALIPSO profiles that fall within 200 km of the Darwin site from December 2010 through December 2014. The RL data are limited to times within ±2 h of CALIPSO overpasses.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 433 252 14
PDF Downloads 315 168 7

Automated Retrieval of Cloud and Aerosol Properties from the ARM Raman Lidar. Part I: Feature Detection

Tyler J. ThorsenDepartment of Atmospheric Sciences, University of Washington, Seattle, Washington

Search for other papers by Tyler J. Thorsen in
Current site
Google Scholar
PubMed
Close
,
Qiang FuDepartment of Atmospheric Sciences, University of Washington, Seattle, Washington

Search for other papers by Qiang Fu in
Current site
Google Scholar
PubMed
Close
,
Rob K. NewsomPacific Northwest National Laboratory, Richland, Washington

Search for other papers by Rob K. Newsom in
Current site
Google Scholar
PubMed
Close
,
David D. TurnerNOAA/National Severe Storms Laboratory, Norman, Oklahoma

Search for other papers by David D. Turner in
Current site
Google Scholar
PubMed
Close
, and
Jennifer M. ComstockPacific Northwest National Laboratory, Richland, Washington

Search for other papers by Jennifer M. Comstock in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

A feature detection and extinction retrieval (FEX) algorithm for the Atmospheric Radiation Measurement Program’s (ARM) Raman lidar (RL) has been developed. Presented here is Part I of the FEX algorithm: the detection of features including both clouds and aerosols. The approach of FEX is to use multiple quantities— scattering ratios derived using elastic and nitrogen channel signals from two fields of view, the scattering ratio derived using only the elastic channel, and the total volume depolarization ratio—to identify features using range-dependent detection thresholds. FEX is designed to be context sensitive with thresholds determined for each profile by calculating the expected clear-sky signal and noise. The use of multiple quantities provides complementary depictions of cloud and aerosol locations and allows for consistency checks to improve the accuracy of the feature mask. The depolarization ratio is shown to be particularly effective at detecting optically thin features containing nonspherical particles, such as cirrus clouds. Improvements over the existing ARM RL cloud mask are shown. The performance of FEX is validated against a collocated micropulse lidar and observations from the Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite over the ARM Darwin, Australia, site. While the focus is on a specific lidar system, the FEX framework presented here is suitable for other Raman or high spectral resolution lidars.

Corresponding author address: Tyler J. Thorsen, Department of Atmospheric Sciences, University of Washington, ATG 408, Box 351640, Seattle, WA 98195. E-mail: tylert@atmos.washington.edu

Abstract

A feature detection and extinction retrieval (FEX) algorithm for the Atmospheric Radiation Measurement Program’s (ARM) Raman lidar (RL) has been developed. Presented here is Part I of the FEX algorithm: the detection of features including both clouds and aerosols. The approach of FEX is to use multiple quantities— scattering ratios derived using elastic and nitrogen channel signals from two fields of view, the scattering ratio derived using only the elastic channel, and the total volume depolarization ratio—to identify features using range-dependent detection thresholds. FEX is designed to be context sensitive with thresholds determined for each profile by calculating the expected clear-sky signal and noise. The use of multiple quantities provides complementary depictions of cloud and aerosol locations and allows for consistency checks to improve the accuracy of the feature mask. The depolarization ratio is shown to be particularly effective at detecting optically thin features containing nonspherical particles, such as cirrus clouds. Improvements over the existing ARM RL cloud mask are shown. The performance of FEX is validated against a collocated micropulse lidar and observations from the Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite over the ARM Darwin, Australia, site. While the focus is on a specific lidar system, the FEX framework presented here is suitable for other Raman or high spectral resolution lidars.

Corresponding author address: Tyler J. Thorsen, Department of Atmospheric Sciences, University of Washington, ATG 408, Box 351640, Seattle, WA 98195. E-mail: tylert@atmos.washington.edu

1. Introduction

Remotely determining the vertical and temporal structure of clouds and aerosols is essential for understanding atmospheric processes and the climate system. Various remote sensors are used to characterize the atmosphere, but only active instruments, such as radars and lidars, can produce unambiguous vertical profiles of cloud and aerosol properties. While millimeter-wavelength radars (e.g., Clothiaux et al. 2000; Hogan and Illingworth 2000; Stephens et al. 2002) are quite sensitive to clouds, lidars are needed to detect the smallest of cloud particles and many optically thin targets. In tandem with cloud radars, lidars are needed to fully resolve radiative heating rate profiles (Feldman et al. 2008; Yang et al. 2010; Thorsen et al. 2013a) and top-of-the-atmosphere fluxes (Haladay and Stephens 2009; Borg et al. 2011; Sun et al. 2011).

Here we develop an algorithm for a Raman lidar to objectively determine the vertical extent of clouds and aerosols (particulates). The algorithm falls into the general category of a threshold-based method where an estimate of a clear-sky signal is made and significant returns above that are considered particulates. Variations of threshold methods have been widely used in the analysis of elastic lidar signals (e.g., Winker and Vaughan 1994; Clothiaux et al. 1998; Campbell et al. 2008; Vaughan et al. 2009). Raman lidars (RL; e.g., Goldsmith et al. 1998; Matthais et al. 2004) and high-spectral-resolution lidars (HSRL; e.g., Hair et al. 2008; Grund and Eloranta 1991) are relatively more advanced lidars that intrinsically separate returns from molecules and particulates. This work presents an algorithm developed for the Atmospheric Radiation Measurement Program’s (ARM) Raman lidars that have operated at the ARM Southern Great Plains (SGP) site near Lamont, Oklahoma (36.61°N, 97.49°W); the tropical western Pacific (TWP) Darwin, Australia, site (12.43°S, 130.89°E); and as part of the third ARM Mobile Facility (AMF3) currently stationed in Oliktok Point (OLI), Alaska (70.50°N, 149.89°W). The TWP RL will soon be moved to the eastern North Atlantic (ENA) site on Graciosa Island in the Azores (39.09°N, 28.03°W).

The ARM Raman lidar was originally conceived with the goal of measuring water vapor profiles at a high temporal and spatial resolution. Since the RL was designed to be a continuously operated instrument, automated algorithms were designed to process the data with a focus on the retrieval of water vapor and aerosol extinction profiles in the lower atmosphere (Turner et al. 2002). While cloud observations were originally considered of secondary importance for this system, studies have demonstrated that ARM RL is capable of making high-quality cloud observations (Wang and Sassen 2002; Dupont et al. 2011; Thorsen et al. 2013b). However, the identification of clouds is treated in a simple manner in current ARM RL data products, and many clouds, especially cirrus, are not identified (Thorsen et al. 2013b).

To fully realize the potential of the ARM RL, we develop an automated algorithm for feature detection and extinction retrieval (FEX). The FEX algorithm objectively identifies features (i.e., clouds and aerosols) and retrieves their extinction profiles over the extent of the troposphere. Complete details of the extinction profile retrieval are given in Thorsen and Fu (2015, hereafter Part II), while Part I here focuses on feature identification. The intent is to run FEX operationally within the ARM Data Management Facility (DMF) with the output being made available to the general user community via the ARM website (http://www.arm.gov/).

The specifications of the ARM RL are given in section 2. Section 3a describes initial signal processing, and sections 3bd detail how scattering and depolarization ratios are determined as well as the process of calibration and deriving overlap functions. Section 3e describes the method used to detect features accompanied by examples. An assessment of how well FEX discriminates features from clear sky is given in section 4. FEX is evaluated using 4 yr of RL observations at Darwin, Australia, in section 5 through comparisons to the original algorithm and that from a collocated ARM micropulse lidar (MPL; Campbell et al. 2002; Coulter 2012) and observations from the Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO; Winker et al. 2009, 2010). Summary and conclusions are given in section 6.

2. The ARM Raman lidar

The RL at the SGP site has been in near-continuous operation since 1998, and its basic design is described in Goldsmith et al. (1998). This original system has since evolved through various upgrades and modifications (Ferrare et al. 2006; Newsom 2009). Additional ARM RLs were deployed at the Darwin TWP site in December 2010 and at the AMF3 OLI site in October 2014, both with nearly the same design as the SGP RL. Specifications of the ARM RL system and the detection channels used in this work are given in Table 1. The system contains channels for detection of elastic as well as Raman-scattered light from nitrogen, water vapor, and two temperature channels. Only the elastic and nitrogen channels are used for this work. Details concerning retrieval of water vapor are given in Turner et al. (2002) and temperature in Newsom et al. (2013). Backscattered returns are collected at a vertical (temporal) resolution of 7.5 m (10 s) in two fields of view (FOV): a narrow FOV (NFOV; referred to as the “high channels”) and a wide FOV (WFOV; referred to as the “low channels”). The high-elastic-channel signal is split into copolarized and cross-polarized signals. For convenience, the prefix “high” is dropped when referring to the high-channel signals, while the prefix “low” will be included when referring to low-channel signals.

Table 1.

Specifications of the ARM RL transmitter and receiver channels used for feature detection in this study.

Table 1.

3. Feature detection algorithm

FEX’s feature detection centers around the application of signal detection theory (e.g., Kingston 1978). In the presence of noise, the problem of determining if a signal contains a contribution from particulates becomes that of comparing two probability distributions. This is shown schematically in Fig. 1, where an expected molecular signal (m) with noise is given in blue and the measured signal (S) with noise is given in red. In the presence of noise the detection threshold (τ) is set at a value larger than expected from a purely molecular signal—in this case, one standard deviation away from the expected molecular signal. In addition to setting a threshold, an important consideration is the suppression of false detections. In this work, this is partially accomplished by calculating the overlap between the and probability distributions (i.e., the purple region in Fig. 1). The essential components of feature detection are all given in Fig. 1—given the signal measured by the lidar, we need to determine 1) the noise in the measured signal, 2) the expected molecular signal, and 3) the noise in the expected molecular signal. From this we set a threshold and assess whether a true feature was detected.

Fig. 1.
Fig. 1.

Schematic comparison of the (left) expected molecular signal to (right) the signal measured by the lidar.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

a. Initial processing

FEX starts by first noting the times when automated or manual alignment “tweaks” (i.e., adjustments) are made from the instrument log files [automated alignment tweaks (Turner et al. 2002) were performed only prior to an upgrade in 2007 (Newsom 2009)]. Typically, manual tweaks are performed about once each day and automated tweaks occur every few hours.

The main input into our feature detection algorithm is the MERGE product (Newsom et al. 2009), which is routinely produced as part of the ARM archive (http://www.archive.arm.gov). MERGE applies dead-time corrections and combines raw analog voltages and dead-time-corrected photon-counting data signals into a single profile by converting the analog signals to an equivalent photon count rate. FEX converts the MERGE photon count rate data into photon counts for each channel. The subscript i throughout this paper is used to denote the respective channels considered in Table 1. Photon counts at their native 10-s and 7.5-m resolution are then accumulated (summed) into their desired time and height bins—in this study we primarily use 2 min and 30 m—to improve the signal-to-noise ratio (SNR).

Random noise in lidar signals is contributed from two main sources: 1) background noise due to solar radiation, detector dark current, and thermal noise; and 2) shot noise. Background noise is range independent and measured by the ARM RL system with pretrigger samples—that is, samples taken for ~15 μs (~300 height bins of 7.5-m height) before the laser fires. Shot noise is the uncertainty associated with the arrival of photons on the detector due to the discrete nature of the incident light. The uncertainty due to shot noise is well described by a Poisson distribution (Oliver 1965), making it straightforward to calculate since the standard deviation is equal to the square root of the signal intensity. While it is possible that excess noise beyond background and shot noise may be present, no attempt is made to determine its potential impact. Using the RL at SGP, Wulfmeyer et al. (2010) found this assumption to be a reasonable one since water vapor mixing ratio noise profiles from various methods agreed well with those estimated using Poisson statistics.

The pretrigger samples, at the native 10-s and 7.5-m resolution, are used to determine the background signal in the 2-min and 30-m data:
e1
where are the pretrigger signals, is the number of height bins accumulated (), is the number of time bins accumulated (), and is the number of pretrigger samples (). The 2-min and 30-m background-subtracted photon count profiles are then obtained from
e2
where z corresponds to the middle height of the bins being accumulated starting from height .
The range-independent background noise for each 10-s profile k is computed as the standard deviation of the pretrigger samples:
e3
where the overbar denotes the mean over all pretrigger samples. Equation (3) is then used to calculate the background noise in data accumulated to 2 min and 30 m:
e4
The range-dependent shot noise is then calculated in each channel:
e5
where is the background-subtracted photon counts in the 2-min and 30-m data. The total random noise is determined by adding the shot and background noise in quadrature:
e6
Finally, the signal-to-noise ratio is calculated as
e7

b. Scattering and depolarization ratios

Following the development by Measures (1984), the lidar equation for a background-subtracted signal due to elastic backscattering can be written as
e8
where is the number of photons received from height z, is the system constant incorporating all the lidar parameters that describe the characteristics of the optics and detectors, is the number of transmitted photons, and is the overlap function that describes the overlap between the laser beam and receiver’s FOV. Quantities and are the molecular and particulate backscatter coefficients, respectively. Terms and are the transmission due to molecules and particulates, respectively, which, for a zenith-pointing lidar like the ARM RL, are
e9
and
e10
where and are the molecular and particulate extinction coefficients, respectively.
For the signal in the nitrogen Raman channels, which contains only backscatter due to nitrogen molecules, the lidar equation can be written as
e11
Compared with Eq. (8), extra transmission terms are needed to account for the different wavelength of the return signal. In Eqs. (8) and (11), the molecular backscatter terms, and , can be expressed in terms of the molecular number concentration:
e12
and
e13
where is the molecular number profile; is the nitrogen mixing ratio; and and are the differential backscatter cross sections for Rayleigh and nitrogen Raman scattering, respectively. Equations (8) and (11) ignore the temperature dependence of the Rayleigh and nitrogen Raman backscattering, as it is very small for the narrow filter widths of the RL and a laser wavelength of 355 nm (Whiteman 2003a). The influence of multiple scattering is also not accounted for [i.e., Eqs. (8) and (11) assume that the transmitted photons and the atmosphere undergo a single interaction]. However, FEX does explicitly consider multiple scattering effects in these equation, but the details of this are discussed in Part II of this work.
To identify signals greater than expected from clear sky, we use the scattering ratio:
e14
defined as the ratio of the total (molecular and particulate) backscatter to the backscatter from just molecules. Since the elastic channel signal contains contributions from both molecular and particulate scattering and the nitrogen channel signal is a function of molecular scattering only, a scattering ratio can be formed by combining Eqs. (8) and (11) (Cooney et al. 1969; Melfi 1972). The expression for the scattering ratio defined in Eq. (14) using the high channels can be written as
e15
where and are the signals in the high perpendicular and parallel channels, respectively; and a is the Ångström exponent. The subscripts are used to denote that this scattering ratio is derived by using both the elastic and nitrogen channel signals, respectively. The quantity is a calibration constant,
e16

An example of the scattering ratio derived using the elastic and nitrogen channels is given in Fig. 2a for the high-channel signals. Separate system constants are not used for the high parallel and perpendicular channels because any difference between the two channels has already been accounted for as shown in the appendix. Separate overlap functions are not derived for the high parallel and perpendicular channels either as discussed in section 3c. The exponential term accounts for the differential particulate transmission between the elastic and nitrogen channels. For cloudy bins, this term is approximately 1 since the Ångström exponent . However, the same cannot be assumed for aerosols. Therefore, collocated Cimel sun photometer data (Holben et al. 1998) are used to determine the aerosol Ångström exponent. Using multiple years of data (1996–2007 at SGP and 2004–07 at Darwin), the median Ångström exponent at both the SGP and Darwin sites is 1.35 between 340 and 380 nm (with a standard deviation of 0.53 and 0.49 at Darwin and SGP, respectively). FEX’s classification of feature type, described in Part II, separates clouds and aerosols, allowing the proper Ångström exponent to be assigned in each bin. The differential transmission due to molecules must also be taken into account in calculating this scattering ratio. These molecular transmission terms are calculated following Bucholtz (1995) using pressure and temperature profiles from radiosondes, launched at least twice daily at all ARM facilities, which are linearly interpolated to the heights and times of the lidar profiles.

Fig. 2.
Fig. 2.

(a) Scattering ratio derived using the elastic and nitrogen channels, (b) scattering ratio derived using the low elastic and nitrogen channels, (c) scattering ratio derived using only the elastic channel, and (d) the depolarization ratio after the final iteration of FEX for 24 Dec 2010 at Darwin.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

Similarly, we construct a scattering ratio using the low elastic and nitrogen channels as
e17
where the superscript L is used to denote the low channel. Figure 2b gives an example of this scattering ratio below 9 km since the ARM RL does not record low-channel signals above that height. At the expense of increased random noise and multiple scattering, using the low-channel signals has the benefit of achieving complete overlap sooner (i.e., at a lower height above the system), allowing for a more accurate scattering ratio in the near field than the high channels. For the ARM RL, the high channels achieve complete overlap by 5 km and the low channels by 800 m (Goldsmith et al. 1998).

After determining the calibration constants and overlap functions (sections 3c and 3d), the scattering ratio using Eqs. (15) and (17) can be calculated. An advantage of deriving a scattering ratio using both the elastic and nitrogen channel signals is that, for a perfect optical system, the ratio of the two overlap functions is unity, though in practice there is usually some residual amount of overlap effects (e.g., Whiteman 2003b). Nevertheless, taking the ratio of the two overlap functions helps reduce the impact of incomplete overlap. While the differential molecular transmission is straightforward to account for using radiosonde observations, determining the differential aerosol transmission term requires an identification of aerosol, and the retrieval of the aerosol extinction profile and Ångström exponent. However, the differential aerosol transmission term is typically small (Whiteman 2003b). Therefore, even large errors in this term introduce minimal uncertainty to the scattering ratio. For the Darwin RL we find mean corrections for differential aerosol transmission ranging from 0% to 2% from 0 to 18 km. The more accurately constrained correction for differential molecular transmission ranges from 0% to 18% from 0 to 18 km. As shown in section 4, using and for feature detection is limited by the relatively weak Raman scattering process.

To produce a quantity with the smallest amount of random noise possible, the scattering ratio can also be derived using only the elastic channel signal. This is commonly done for single-channel elastic backscatter lidars for the purpose of feature detection (e.g., Winker and Vaughan 1994; Campbell et al. 2008; Vaughan et al. 2009) since Rayleigh scattering can be accurately modeled (e.g., Bucholtz 1995). The scattering ratio using only the elastic channel can be written as
e18
where
e19
Note that we do not derive the scattering ratio using only the low elastic channel, as doing so identified few additional cloud or aerosol layers.

An example of the scattering ratio derived using only the elastic channel is shown in Fig. 2c. An accurate estimate of the full profile of cloud and aerosol extinction coefficients [i.e., the term in Eq. (18)] is crucial in determining the scattering ratio in this way. In addition there is no cancellation of overlap effects, which occurs if both the elastic and nitrogen channels are used. Despite these limitations, deriving the scattering ratio using only the elastic channel results in a higher SNR. This is apparent when comparing in Fig. 2c to in Fig. 2a, where the upper portion of the daytime ice cloud layer after 2100 UTC becomes overwhelmed by noise in but is clearly visible in the image.

With the ARM RL it is also possible to identify features using the linear volume depolarization ratio:
e20
An example false-color image of depolarization is given in Fig. 2d. The depolarization ratio has the benefit of not depending on extinction as well as using only the elastic channel signals. But it is limited to identifying particulates that induce a depolarization, that is, nonspherical particles like ice and some aerosols (Sassen 1991; Gobbi 1998; Murayama et al. 2001; Sassen 2002) and liquid clouds that induce a significant amount of multiple scattering (Carswell and Pal 1980; Sassen 1991).

The four ratios given in Eqs. (15), (17), (18), and (20) (Fig. 2) are used for identifying features from 0 to 20 km and, as discussed above, each quantity has specific advantages and disadvantages. Therefore, each quantity is analyzed separately with the expectation that taken as a whole they will be complementary and provide a more complete description of the vertical extent of clouds and aerosols. To calculate these ratios, calibration constants must be determined using clear-sky regions where the ratios have known values. One can readily see that the problem at hand is circular since clear sky must be used to calibrate these quantities, but clear sky cannot be identified without the calibrated quantities themselves in our framework. In addition, we also require the particulate extinction profile, but we first must identify where the particulates are. Therefore, FEX is an iterative algorithm as illustrated in Fig. 3: going through the process of determining overlap functions (section 3c), calibrating (section 3d), calculating ratios (this section), calculating the expected molecular signals and identifying features (section 3e), and retrieving extinction (described in Thorsen and Fu 2015). These steps are then repeated utilizing the feature mask and extinction profiles from the previous iteration. Iterations continue until less than 0.1% of all pixels in the feature mask change relative to the previous iteration (typically six to seven iterations).

Fig. 3.
Fig. 3.

Flow diagram for feature detection in the FEX algorithm.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

c. Overlap functions

Following Wandinger and Ansmann (2002), a single low-channel overlap function is derived by assuming that the low-elastic and nitrogen-channel overlap functions are equal. Besides this assumption, the accuracy of this method relies on knowledge of the extinction profile. Therefore, only bins where the particulate transmission is greater than 0.9 are used to ensure the relative contribution of extinction errors remains small. In addition, only bins with “good” signal are used, which are defined as regions where the SNR [Eq. (7)] is greater than 3.

High-elastic and nitrogen-channel overlap functions are determined by similar methods to those described in Turner et al. (2002). The overlap-corrected low-channel signals are used to derive the high-channel overlap functions by taking the median value of the ratio between them in each height bin. The ratio of the signals above the height of complete overlap is used to calibrate the overlap function. Only good signals where the scattering ratio is less than 2 are used since we expect the signal in each channel to be proportional only when multiple scattering is small. Functions are computed at each height bin where suitable signals exist in at least 10% of the profiles in the period between tweaks. The standard deviation at each height bin is also required to be less than 30% of the median value. Height bins failing these requirements use an overlap function calculated using all signals for the current day being processed. When per-day values cannot be used, historical per-day values are used to determine the mean overlap function from the previous 7 days. Historical functions are also used during the first iteration, when no estimate of particulate transmission or scattering ratio exists to select the appropriate bins. Since separate polarization components are not measured in the low channels, the same overlap function is used for both the high parallel and perpendicular channels.

d. Calibration constants

Calibration constants are determined for the scattering ratios by solving for the respective constants in Eqs. (15), (17), and (18) using clear-sky bins (where the scattering ratio is 1) identified in FEX’s feature mask. In addition, only good signals are used to solve for constants. An additional calibration constant, , is calculated to match the low-elastic-channel signal to the high-elastic-channel signal (needed for the threshold calculation; see section 3e). For , in addition to clear-sky signals, signals where the scattering ratio is less than 2 are used.

All constants are determined by taking the median value over multiple bins at varying resolutions. Starting with the coarsest level, constants are determined per day, per-tweak period, and per profile. The per-profile values are used to obtain other per-profile constants by interpolating. A summary of the height regions and rules used to determine when each type of calibration type is performed is given in Table 2. When no suitable signal exists to derive any type of calibration constant, a database of historical values are used to determine the mean constant over the previous 7 days. Historical constants are always used during the first iteration of FEX since no feature mask exists yet to determine what signals are clear sky. As indicated in Table 2, height bins above the height of complete overlap are typically used to calculate constants. The lowest height bin used () also depends on the current iteration being performed with
e21
where is the current iteration, is the minimum number of iterations to be performed (), and is the absolute lowest height bin used for calibration as given in Table 2. Equation (21) helps minimize calibration bias by initially using the highest heights possible for calibration. This reduces the possibility that as-yet-undetected aerosol will bias the constants.
Table 2.

Hierarchy of calibration types and height regions used by FEX. Calibrations constants are taken as the median over all samples and interpolated constants are obtained using a smoothing spline interpolant. The >2-km and >5-km height regions are not used for deriving and , respectively. Before the minimum number of iterations (i.e., ) is reached, the height regions are increased as described in the section 3d. Also given is the multiplicative factor used to modify the probability P threshold used by the spatial filter to reduce false detections.

Table 2.

Table 3 gives the percentage of profiles for the ARM Darwin RL from December 2010 through December 2014 by calibration type: historical, per day, per tweak, interpolated, and per profile. For both high-channel scattering ratios, the majority of profiles, about 77%, are able to use a per-profile calibration constant. For , the uncertainties in the per-tweak- and per-profile-derived values are about 6%. A similar uncertainty exists in the per-profile calculation of . However, when is determined per tweak the uncertainty is larger (~16%). This larger per-tweak uncertainty is due to the additional uncertainty from the particulate transmission in each profile below the heights used for calibration. When a per-profile calibration is made, any bias in the transmission below the calibration height is absorbed by the calibration constant. For calibration constants that use low channels, a per-profile constant is used for about 60% of the time. The larger amount of random error in the low-channel signals results in larger uncertainties (~13%). Both the per-day and historical constants exhibit similar uncertainties to the per-tweak value, indicating that the calibration of the TWP system is quite stable even when alignment tweaks are performed.

Table 3.

The percentage of profiles by the calibration type used from December 2010 through December 2014 at Darwin. The mean relative uncertainties (%) in the calibration constants are given in parentheses. Uncertainties are computed by taking the standard deviation in each per-day, per-tweak, or per-profile region. The uncertainty in the historical calibration constants are determined by repeatedly (10 000 times) randomly sampling 7 days of per-profile constants, and in each random sample calculating the sum (in quadrature) of the standard deviation of the per-profile values and the mean standard deviation in the per-profile values themselves.

Table 3.

e. Feature identification

As illustrated in Fig. 1, to determine if a feature is present in any of the four ratios introduced in the previous section, the expected clear-sky ratio and its noise must be calculated. The expected clear-sky ratio is straightforward since, after calibration, we expect a scattering ratio of 1 and a depolarization of 0.4% (see the appendix). The expected clear-sky noise in each of the ratios is calculated as follows. First, is calculated, that is, the signal expected in each channel just due to molecular backscatter. This quantity is determined using the appropriate lidar equation for each channel [Eqs. (8) and (11)] by setting and evaluating all other terms. The product of all constants needed (i.e., ν, , ) can be determined from the previously determined calibration constants (, , , ). Molecular scattering terms (, ) are calculated following Bucholtz (1995) using pressure and temperature profiles from interpolated radiosondes profiles. FEX’s retrieval of particulate extinction (described in Part II) is used to evaluate the particulate transmission terms () in each channel’s lidar equation. The wavelength dependence of the particulate transmission between the elastic and nitrogen channels is accounted for using a climatological value of the aerosol Ångström exponent as described section 3b.

The random noise of each channel’s expected molecular signal is calculated by combining Eqs. (5) and (6) but with the measured signal in those equations replaced by the expected molecular signal:
e22
The expected molecular signal () and random noise () in each channel are then used to determine the expected clear-sky noise in each of the ratios by propagation of uncertainty (e.g., Bevington and Robinson 2002). The ratios in Eqs. (15), (17), (18), and (20) for molecular signals follow the general form
e23
where , , and represent the expected molecular signal in the channels needed to form the various ratios. The quantity contains all other terms in the ratio that have negligible random noise in comparison to the signal noise. For purely molecular signals and for the scattering and depolarization ratio, respectively. By propagation of uncertainty, the noise in Eq. (23) is
e24
A threshold profile is then defined for each ratio as
e25
Portions of the measured ratio profiles that exceed the threshold profile are identified as potential features—that is, values greater than one standard deviation from the expected clear-sky ratio are considered to potentially contain a feature.

An example application of these threshold profiles is given in Fig. 4 for the profile at 1958:00 UTC in Fig. 2. For (Fig. 4c), the need for an accurate estimate of particulate extinction can be seen by comparing the initial profile of (gray line) with the final profile of (black line). In the initial profile, where no extinction estimate is available, the value of falls well below the expected clear-sky value of 1 once the cloud layer above 10 km is encountered. As indicated by the color bar in Fig. 4c, this causes portions of the cloud layers to go undetected since falls below the threshold profile (dashed blue). By the final iteration, when an extinction estimate exists, does not rapidly decrease above 10 km and the full extent of the cloud layer is detected. Particulate extinction also impacts the threshold itself, which is particularly apparent in Figs. 4a and 4d. The values of and δ remain largely unchanged from the initial to the final iterations since both iterations are mostly independent of extinction, and an accurate historical calibration was available on the first iteration. Above ~16 km, and δ exceed their initial thresholds (dashed blue) at many heights although it is apparent that this is just noise. The thresholds on the final iteration (solid blue) increase relative to the initial threshold above the cloud layer base at 10 km due to the influence of extinction. It is apparent that more accurate thresholds are obtained on the final iteration since above ~16 km the thresholds follows the envelope of the noise well.

Fig. 4.
Fig. 4.

The initial threshold (dashed blue) and final threshold (solid blue) for the profile at 1958:00 UTC in Fig. 2 for the (a) scattering ratio derived using both the elastic and nitrogen channel signals, (b) scattering ratio derived using both the low-elastic-and-nitrogen-channel signals, (c) scattering ratio derived using only the elastic channel, and (d) the depolarization ratio. Both the initial profiles (i.e., the first iteration, gray line) and final profiles (i.e., the final iteration, black line) are given. Potential features identified in these ratios on the initial and final iterations are noted by the color in the bars on the right-hand side of each panel; features that have been determined to be false detections are denoted by yellow and true features denoted by blue. For reference, the expected clear-sky ratio is given as a dashed red line. Note that the scale of the x axis in each panel is nonlinear.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

The need for an iterative process is illustrated by forcing an initial calibration bias of 10% in in Fig. 4c. This high bias causes a large portion of the profile in the region to be identified as containing a potential feature since the threshold profile only accounts for random uncertainty. By the final iteration, the calibration of is improved and many of the falsely detected features are gone. However, on the final iteration, portions of the profile exceeding the threshold still exist, which are obviously just excursions above the threshold due to random noise. This is expected since, if the noise is assumed to be Gaussian distributed, there is a ~16% probability that a measured ratio will be above a one standard deviation threshold solely due to random noise and is therefore a false detection.

To reduce the rate of false detections due to random noise, the overlap between the expected molecular signal and measured signal—that is, the purple regions in Fig. 1—is used. In establishing the threshold profile, the expected ratio and noise in a clear-sky signal has already been determined [Eq. (23) and (24)]. The noise in the measured ratio is calculated by propagation of uncertainty similar to that given in Eq. (24) except that the measured signal and related noise are used in place of the expected molecular signal. Assuming the noise is Gaussian distributed, the overlap probability can then be determined as
e26
where is the measured ratio, is the noise in the measured ratio, and min indicates using the minimum value of the two probability density functions. Unlike the threshold profile [Eq. (25)], the quantity in Eq. (26) accounts for the uncertainty of measured signal relative to the expected clear-sky signal. The amount of false detections that results from applying the threshold profile is reduced by constructing a spatial filter using Eq. (26). We expect the feature occurrence to be highly spatially correlated over a 90-m by 6-min window (three range bins by three time bins) centered on a bin containing a potential feature. Assuming the noise is independent in each range and time bin, the overlap probability in this window is
e27
If a true feature is present, then we also expect the surrounding bins in this box to contain a feature; therefore, the probability P in this 90-m by 6-min box should be low. For pixels identified as a potential feature, if P is greater than the empirically determined value of , then the pixel is changed to clear sky.

The threshold profiles and spatial filter consider only sources of random noise, namely, the signal noise in each channel. Compared to the signal noise, random noise in all other components [the term in Eq. (23)] is considered negligible. We expect the molecular scattering terms calculated from radiosonde profiles to contribute a relatively small amount of random noise. The calibration constants and overlap functions are determined by averaging relatively large amounts of data, making their random noise small. The contribution of random noise by the extinction profiles is also relatively small since extinction appears inside an integral in the transmission terms.

Despite a small random noise component, calibration constants, overlap functions, and extinction may contain potential systematic noise large enough to cause false detections. Therefore, when applying the spatial filter, more restrictive thresholds are used to compensate for potential biases. The threshold for changing a potential feature to clear sky is lowered to for bins below the height of complete overlap. Depending of the type of calibration performed, the initial value of the P thresholds is modified by the multiplicative factors given in Table 2. The P threshold is decreased by an addition factor of for bins that use a historical overlap function.

After applying the spatial filter to each of the four masks separately, “consistency checks” are made among the four masks to further identify and suppress false detections due to systematic noise. The mask is most likely to be troublesome since its accuracy depends strongly on the ability to accurately determine the overlap function and particulate extinction. In addition, biases may exist in the depolarization ratio at lower heights since separate overlap functions are not determined for the high parallel and perpendicular elastic channels. To catch any potential false detections in the and δ masks, regions of high confidence clear sky (HCCS) are identified using the and masks. Pixels where the low-nitrogen-channel SNR is greater than 10 and the mask is clear are considered to be HCCS. Where the low-nitrogen-channel SNR is less than 10, the mask is used with HCCS defined as pixels where the mask is clear and the high-nitrogen-channel SNR is greater than 3. Any features that occur in the δ mask below 1 km, or at any height in the mask in these regions of HCCS are considered false detections and are removed.

An additional check for false detections is made when a per-day or historical calibration constant is used for a profile. In these cases, it is possible for the calibration constant to contain a significant bias. Therefore, when a per-day or historical calibration constant is used for a ratio, at least one other ratio must also detect the feature. Otherwise, the feature is removed from the mask. After removing false detections, the four individual feature masks are combined to create a single mask.

The identification of false detections is illustrated in the example profiles in Fig. 4, where the color bar indicates when the ratio is above the threshold value (blue or yellow) and when the algorithm has identified a false detection (yellow). Small excursions above the threshold due to random noise, like those seen in the depolarization profile (Fig. 4d), are typically identified by the spatial filter. More egregious systematic errors can be caused by a poor calibration as seen in the initial profile of in Fig. 4c below 10 km. Since the random noise is expected to be relatively small at these heights, as indicated by how close the threshold is to one, this calibration bias causes to exceed the threshold by a large enough amount that the spatial filter cannot identify these as false detections. This demonstrates the importance of the additional consistency checks made to the feature mask, which serves both to improve the accuracy of FEX and to reduce the number of iterations needed to converge to a final feature mask. In this case, this region from 2 to 10 km is determined to be HCCS using the and masks (Figs. 4a and 4b); therefore, the features there are considered false detections. On the next iteration, this clear-sky region is then used to derive an calibration constant and to correct the initial bias.

Figure 5 shows the feature mask after the final iteration with Fig. 5a showing the potential features—that is, those that exceed the threshold—and Fig. 5b showing the true features—that is, those potential features that remain after removing false detections using the logic outlined in this section. Even after achieving the best possible calibration constants, overlap functions, and extinction profiles, numerous false detections exist in Fig. 5a that are effectively identified and removed in Fig. 5b. Comparing Fig. 5b to Fig. 2 demonstrates the typically good performance of the algorithm. All major features are detected, including many small isolated cumuli throughout the day below 6 km; the thick and at times multilayered ice cloud from 10 to 17 km; the very thin cirrus layer at 17 km that exists after 2100 UTC; and a layer of aerosols in the boundary layer. From about 0600 to 1500 UTC, numerous optically thick water clouds exist that completely attenuated the RL’s signal, although small glimpses of the upper-level cloud deck are still possible during this period.

Fig. 5.
Fig. 5.

Feature mask with height vs hour on the final iteration of FEX for the ratios given in Fig. 2. Each color represents a different combination of the four ratios—scattering ratio derived using: the elastic and nitrogen channels (EN), the low elastic and nitrogen channels (ENL), and only the elastic channel (E), and the depolarization ratio (D)—that detected the feature. (a) Potential features, i.e., bins where the ratio exceeded the threshold. (b) True features, i.e., those potential features that remain after removing false detections due to random noise and applying consistency checks as discussed in section 3e.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

A second example of FEX’s feature detection is given in Fig. 6 for 10 May 2013 over Darwin. During this day some scattered low clouds and two distinct layers of ice clouds exist: an optically thicker layer from about 11 to 14 km and a very thin tropical tropopause layer (TTL) cirrus from about 15 to 17 km. The TTL cirrus layer is more clearly discerned using the depolarization ratio (Fig. 6d); that is, the contrast between the clear-sky and in-cloud values is larger than for the scattering ratios. This is reflected in the feature mask (Fig. 6e), where the TTL cirrus is captured best in the depolarization mask. The frequency with which the depolarization ratio can more easily detect thin cirrus is explored further in section 4. Despite the TTL cirrus being captured best in the depolarization ratio mask, portions of it still go undetected. In this example, more signal accumulation (or averaging) is required to confidently distinguish these portions of the cloud from clear sky. The effects of averaging are further discussed in section 5b.

Fig. 6.
Fig. 6.

(a) Scattering ratio derived using the elastic and nitrogen channels, (b) scattering ratio derived using the low elastic and nitrogen channels, (c) scattering ratio derived using only the elastic channel, and (d) the depolarization ratio after the final iteration of FEX for 10 May 2013 at Darwin. (e) Feature mask where each color represents a different combination of the four ratios that detected the feature—scattering ratio derived using: the elastic and nitrogen channels (EN), the low elastic and nitrogen channels (ENL), and only the elastic channel (E), and the depolarization ratio (D).

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

A separate color in Figs. 5b and 6e is used to represent the different combination of the four ratios that identified the feature. This example highlights the strength of using multiple quantities since all four masks taken together produce a more accurate feature mask than any single quantity. Care was taken designing the algorithm to minimize false detections, which comes at the expense of not reducing the rate of missed detections. Instead we rely on the random and systematic noise in each ratio to be different enough not to cause missed detections in the same pixels. In both Figs. 5b and 6e, and in general, we find this to be the case as small gaps in the feature mask for any one ratio are typically filled by other ratio’s masks. For example, this gap filling can be seen in Fig. 5b in the thick cloud layer from 1500 to 2400 UTC as well in the cirrus present in Fig. 6e.

FEX also computes two versions of a detection confidence score (DCS), at each pixel using Eq. (26). The first version gives the effect of random uncertainty by averaging the values over all four ratios. The DCS is reported as 1 minus this mean value. Therefore, a DCS of 0 indicates complete confidence that the bin is clear sky, while a DCS of 1 indicates complete confidence that the bin contains a feature. Values between 0 and 1 quantify the amount of ambiguity between classifying the bin as clear sky or a feature. In a similar manner, the second DCS gives an estimate of the effect of the total uncertainty by again evaluating Eq. (26) but with the noise terms ( and ) replaced by those including contributions from both random and systematic noise. The systematic noise from the calibration constants and overlap functions are estimated by taking the standard deviation over all bins used to calculate each quantity. Propagation of uncertainty is used to obtain the resulting systematic noise in each ratio. This is then added in quadrature to both the random signal noise in the expected clear-sky ratio and the measured ratio.

4. Feature discrimination

Figure 7 shows the cloud occurrence profiles obtained from FEX for December 2010 through December 2014 at Darwin for daytime (Fig. 7a) and nighttime (Fig. 7b) profiles. The overall cloud occurrence profile (gray line) is separated into the occurrence of clouds in each ratio: the scattering ratios from the low elastic and nitrogen channels (red line), from the elastic and nitrogen channels (green line), and from only the elastic channel (blue line); and from the depolarization ratio (brown line).

Fig. 7.
Fig. 7.

The (a) daytime and (b) nighttime cloud occurrence profiles for cloud detected by any ratio (gray), the scattering ratio derived using: the low elastic and nitrogen channels (red), the elastic and nitrogen channels (green), and only the elastic channel (blue), and the depolarization ratio (brown) at Darwin from December 2010 through December 2014.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

Distinct diurnal differences can be seen in the scattering ratios derived using both the elastic and nitrogen channels, which detect significantly less clouds during the daytime. Most heights also have less cloud during the daytime in the and δ masks, although the diurnal difference is much smaller. Below about 8 km, no single ratio’s mask quite matches the total cloud occurrence, an indication that a better estimate of the mean cloud occurrence is obtained by the use of all four ratios. Above about 8 km, any cloud present is almost always detected using the depolarization ratio. The mask is the second-most-sensitive method at these higher heights, but it consistently detects fewer clouds than the depolarization. For better insight into this difference, and the diurnal variation in Fig. 7, the ability of FEX to discriminate a feature from a clear-sky signal is examined in more detail. For this purpose, we quantify how the distribution thresholds at each height differ from the distribution of the in-feature values of the ratios themselves.

Figure 8 shows the median thresholds (red) and median in-feature (blue) ratios at Darwin from December 2010 through December 2014. The shaded region encompasses the 5th–95th percentiles of the distribution of thresholds and in-feature ratios at each height. The top-most and bottom-most bin of feature layers are not included in calculating these distributions since we assume these bins are some mixture of feature and clear sky. Each distribution of feature ratios is determined using only points where the respective ratio detected a feature. The calculation of threshold distributions includes all points where the laser beam has not been completely attenuated—defined as a high-parallel-elastic-channel SNR > 1.

Fig. 8.
Fig. 8.

The median threshold profiles (red) for points where the laser beam has not been completely attenuated and the median ratio value (blue) for points where that ratio detected a feature. Separate panels are shown for each ratio—the scattering ratio derived using: (a),(b) the low elastic and nitrogen channels, (c),(d) the elastic and nitrogen channels, and (e),(f) only the elastic channel, and (g),(h) the depolarization ratio—and for (a),(c),(e),(g) daytime and (b),(d),(f),(h) nighttime profiles. The shaded region on both the threshold and ratio profiles encompasses the 5th–95th percentiles from their respective distributions.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

The influence of the solar background can be determined from the lower bound (i.e., the 5th percentile) on the threshold profiles in Fig. 8. The lower bound represents points where the particulate extinction at lower heights is small to nonexistent. In addition, we expect the shot noise to be similar between day and night at a fixed height. Therefore, diurnal differences in the lower bound of this threshold can be attributed to differences in the background noise, which we expect to be dominated by the solar background. Making this comparison of the lower bound on the threshold distribution in Fig. 8, the impact of the solar background is clearly discernible for both and , which have higher thresholds during the daytime. Therefore, the diurnal difference in Fig. 7, where fewer clouds are detected during the day at most heights, is due to the solar background for and . For the δ threshold, the daytime threshold lower bound shows a small relative increase above about 16 km. Almost no difference exists between day and night in the lower bound of the threshold. Therefore, we do not expect the overall cloud occurrence to be biased by solar background noise, and the larger occurrence of nighttime cloud seen in Fig. 7 can be taken as a true physical phenomena.

The higher amount of cloud occurrence in the δ mask compared with can be explained by comparing their respective threshold distributions to their feature distributions. During both day and night, the feature distribution of δ has less overlap with its threshold distribution compared to at higher heights. In other words, the contrast between typical in-cloud depolarization values and the clear-sky value of 0.004 is greater than typical in-cloud scattering ratios and the clear-sky value of 1. This higher contrast can be seen in the example ratios and feature masks given in Fig. 6, particularly for the TTL cirrus layer. In terms of median values in Fig. 8, the ratio of feature-to-threshold is typically 3–6 times larger above 8 km for δ than (not shown). This allows for an easier detection using the depolarization ratio. However, this does not imply that a feature mask constructed using only a scattering ratio cannot detect tenuous features, just that it would require more signal accumulation (or averaging) to do so than is needed for the depolarization ratio.

The large amount of optically thin cirrus clouds that occurs at the TWP site (e.g., Thorsen et al. 2011) leads to the up to a factor of 2 difference between the cloud occurrence in the δ mask compared to . At the SGP site, where there exists a smaller amount of thin cirrus, the δ mask also detects more high cloud but only up to a maximum of 30% more than the mask (not shown).

While Fig. 8 quantifies sensitivity in terms of scattering and depolarization ratios, extinction and backscatter coefficients are potentially a more comprehensible estimate of FEX’s sensitivity. The minimum, defined as the 5th percentile, in-feature particulate extinction and backscatter coefficient detected by FEX is 7.0 × 10–3 km–1 and 2.7 × 10–4 km–1 sr–1, respectively, for 2-min time bins and 30-m height bins. Determining the particulate extinction and backscattering depends on FEX’s extinction retrieval (Thorsen and Fu 2015). In addition, values can only be determined in-feature and therefore cannot always represent the true detection thresholds.

5. Performance assessments

Assessing FEX’s performance is inherently difficult since we have no underlying truth for the physical world. Therefore, much of the testing during the development of this algorithm was qualitative: comparing the algorithm’s mask to the features identified by an expert observer looking at an image of the data. By this measure, the algorithm performance is satisfactory with acceptably low rates of false and missed detections. In this section, more quantitative assessments of performance are made mainly for cloud detection at the ARM Darwin site. Cloud detection at a tropical site like Darwin presents a challenge for a lidar due to the high levels of solar background noise and the high altitude of the tropopause, near which very optically thin cirrus frequently occur (e.g., Winker and Trepte 1998; Wang et al. 1998; Fu et al. 2007; Dessler and Yang 2003; Massie et al. 2010; Davis et al. 2010). The comparisons made in this section also rely on classifying features into cloud and aerosol, which is described in Part II.

One of the motivations for this work was to improve the cloud mask in the current generation of ARM RL products. Figure 9 shows the profile of cloud occurrence from the ARM depolarization (DEP) product. The DEP product applies static thresholds to the depolarization and scattering ratio (calculated using the nitrogen and elastic channels) to identify cloud layers: defined as where depolarization is greater than 0.05 or the scattering ratio is greater than 1.5. The comparison of the cloud occurrence profile from the DEP product to FEX is given in Fig. 9. While reasonable agreement exists below 5 km, FEX detects more clouds at high heights. The increased detection is most striking for heights above 12 km, where up to twice as many clouds are detected. A visual comparison of the two cloud masks revealed that the DEP product has not only missed detections but also has numerous false detections. This is reflected in the amount of cloud that exists in the DEP product from 18 to 20 km, a region where we expect almost no clouds to occur at this location. It is of no surprise that the simple static thresholds used in the original ARM RL cloud mask are outperformed by our more rigorous threshold algorithm. Since there is such a stark difference in complexity between these two approaches, we provide better assessments of our algorithm’s performance by comparing to other lidar observations.

Fig. 9.
Fig. 9.

The cloud occurrence profile from the ARM DEP product (red) and from the new FEX algorithm (blue) described in this work at Darwin from December 2010 through December 2014.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

a. ARM MPL comparison

The ARM Darwin site is also equipped with an MPL (Campbell et al. 2002; Coulter 2012) that transmits 6–8 μJ of 532-nm light with a PRF of 2500 Hz. Data are averaged to 2 min and 30 m, the same bin size used for the RL. The cloud mask of Wang and Sassen (2001) is applied to the backscattered signal. The Wang and Sassen (2001) algorithm is a slope method that examines the relative change in the returned signal power for strong negative and positive slopes that occur in the presence of clouds. The Wang and Sassen (2001) algorithm has been widely used in the analysis of ARM MPL signals (e.g., Comstock et al. 2002; Luo et al. 2008; Thorsen et al. 2011; Comstock et al. 2013; Riihimaki et al. 2012). Note that no aerosol products are produced using the ARM MPL; therefore, in this section, only cloud detection is compared.

The ARM MPL cloud mask (Wang and Sassen 2001) is compared to that of FEX at Darwin using over 80 000 coincident profiles from December 2010 through August 2011, a period when both instruments were operating. Figure 10a compares the cloud occurrence profiles during this period. Good agreement exists from about 5 to 9 km. Below 5 km, the ARM MPL detects more clouds at some heights and less at others than the RL. Differences can be due to several things beyond the methods used for cloud detection below 5 km. Like the RL high channels, the ARM MPL does not achieve complete overlap until 5 km, so uncertainties in overlap functions may contribute to differences in Fig. 10a. The ARM MPL data have a single generic overlap correction applied to all the data and, because of increased uncertainty in this generic correction at lower heights, no cloud mask is determined below 1 km. In addition, the different methods used to identify rain/virga may contribute to discrepancies at lower heights. The cloud mask of Wang and Sassen (2001) separates the base of rain–virga from the cloud base by comparing the increase in slope between the two. In FEX, the identification of rain is based on the retrieved lidar ratio (Thorsen and Fu 2015). Above 9 km, the ARM MPL Wang and Sassen (2001) retrieval detects significantly less clouds presumably due to the larger amount of noise in the ARM MPL measurements (Thorsen et al. 2013b).

Fig. 10.
Fig. 10.

The cloud occurrence profile from the ARM MPL cloud mask (green) (Wang and Sassen 2001) and RL-FEX cloud mask (blue) from December 2010 through August 2011 for (a) all pixels and (b) only pixels where both the ARM MPL and RL SNR is >3 in profiles where rain was not detected.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

To provide a more direct comparison of the detection algorithms themselves, only profiles without rain (as determined by FEX; see Thorsen and Fu 2015) and only pixels where both the ARM MPL and RL SNR is greater than 3 are used to calculate the cloud occurrence profile in Fig. 10b. The MPL SNR is calculated in the same way as is done for the RL [Eq. (7) and section 3a] except that background noise is inferred from high altitude (45–55 km) signals, where the contribution from molecular scattering is negligible (Welton and Campbell 2002). About 60% of RL pixels where the SNR is greater than 3 also have the MPL SNR greater than 3. In this more limited comparison (Fig. 10b), the ARM MPL and RL agree very well above 2 km. However, the ARM MPL still lacks clouds below 2 km, likely an indication that its cloud mask is adversely affected by the uncertainty in its overlap function.

To ensure that this agreement is not due to some fortuitous cancellation of errors, the error matrix for this comparison is shown in Table 4 for pixels above 2 km. When the SNR for both instruments is greater than 3, the two algorithms show good agreement with the ARM MPL detecting cloud for only ~2% of the pixels label by the RL as clear. The opposite error, when RL pixels determined to be cloudy are marked as clear by the ARM MPL, has a larger error rate of ~9%. This is likely an indication of the higher SNR required for a slope method versus a threshold method since taking the slope increases the relative amount of signal noise. If we make the same comparison for a higher MPL SNR—that is, compare the subset of pixels where the MPL SNR is greater than 6 and the RL SNR is greater than 3—then this error is reduced to 7%. Overall, we find reasonable agreement between the RL-FEX and Wang and Sassen’s (2001) ARM MPL cloud mask when both instruments have sufficient SNR.

Table 4.

Comparison between RL-FEX and ARM MPL cloud detection [which uses the algorithm of Wang and Sassen (2001)] at Darwin from December 2010 through August 2011. Only the subset of pixels above 2 km and where both the RL and MPL SNRs > 3 are compared. Percentages are given relative to the detection of RL-FEX.

Table 4.

b. CALIPSO comparison

The CALIPSO satellite (Winker et al. 2009, 2010) was launched in April 2006 into a sun-synchronous orbit providing near-global observations at approximately 0130 and 1330 (local time) for a fixed point. We use the 5-km vertical feature mask (VFM) product (Vaughan et al. 2009) from CALIPSO level 2, version 3, which, like FEX, is based on a threshold method. Since CALIPSO and the ARM RL do not make coincident measurements, only a statistical comparison can be made. CALIPSO VFM profiles that fall within 200 km of the Darwin site are compared to RL profiles from December 2010 through December 2014. The RL data are limited to times within ±2 h of CALIPSO overpasses. To fairly compare a spaceborne platform with a ground-based one, the profiles are further limited to those where the lidar beam passes through all layers unattenuated, that is, transparent profiles. For CALIPSO, transparent profiles are approximated as those with the presence of a signal return from the surface. For the RL, transparent profiles are defined as those where the high-elastic-channel SNR is greater than 1 at 18.5 km.

Figure 11 shows the cloud occurrence in transparent profiles from the RL (blue line) and CALIPSO (red line). Large differences exist with CALIPSO detecting more high clouds. As a whole, CALIPSO must perform significantly more averaging than a ground-based lidar to obtain a similar SNR due to its larger distance from the target. The occurrence of clouds is dependent on averaging since more averaging allows for relatively smaller cloud optical thicknesses to be detected. For example, given in Fig. 12 is the FEX cloud mask for the ratios in Fig. 6 but for data accumulated into 9-min bins instead of 2 min. With 9-min bins, more of the thin TTL cirrus from 16 to 18 km can be detected. However, larger amounts of averaging can smear clouds, artificially increasing the reported cloud occurrence. This can be seen in the lower layer of ice cloud (11–14 km) in Fig. 12. Small gaps that appear in this layer in 2-min bins (Fig. 6e) are smeared out in 9-min bins (Fig. 12). The opposite effect, a decrease in the reported cloud occurrence, can occur for clouds with small spatial scales as larger amounts of averaging can mix in clear-sky signals, causing the cloud to go undetected. Furthermore, our comparison is composed of transparent profiles, which also depend on averaging. Larger amounts of averaging (or signal accumulation) increases the likelihood that enough laser shots penetrate through the feature, thereby reducing the occurrence of complete attenuation. To avoid all these complications, we create a RL-FEX dataset with temporal and height bins approximately equal to CALIPSO’s spatial average.

Fig. 11.
Fig. 11.

The cloud occurrence in transparent profiles from the RL accumulated to 2-min and 30-m bins (blue) and from the CALIPSO VFM product (red). The profile from the RL data accumulated to 9 min and 30 m, which is approximately equivalent to CALIPSO’s averaging below 8.2 km, is given in brown. The profile from the RL data accumulated to 15 min and 60 m, which is approximately equivalent to CALIPSO’s averaging above 8.2 km, is given in gray. Profiles are shown separately for the (a) daytime and (b) nighttime observations. This comparison includes CALIPSO profiles that fall within 200 km of the Darwin site from December 2010 through December 2014. The RL data are limited to times within ±2 h of CALIPSO overpasses.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

Fig. 12.
Fig. 12.

FEX feature mask for 10 May 2013 at Darwin when the signal was accumulated into 9-min and 30-m bins. Each color represents a different combination of the four ratios that detected the feature: scattering ratio derived using the elastic and nitrogen channels (EN), the low elastic and nitrogen channels (ENL), and only the elastic channel (E), and the depolarization ratio (D).

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

The mean amount of horizontal averaging used by CALIPSO is determined from the reported amounts for each cloudy pixel in the VFM product. Since advection determines the amount of cloud passing through the RL FOV, the mean wind speed is obtained from the collocated radiosonde profiles at Darwin. Dividing the profile of mean horizontal averaging by the profile of mean wind speed gives the equivalent temporal bin size for the RL: a mean of 9 min for heights less than 8.2 km and 15 min for those above. Since above 8.2 km CALIPSO’s vertical averaging increases from 30 to 60 m, the same 60-m vertical bin size is used for the RL in the 15-min data.

Figure 11 shows the RL cloud occurrence profile for 9-min and 30-m bins (brown line) and 15-min and 60-m bins (gray line). For nighttime profiles in Fig. 11b, nearly all the difference between the 2–30-min RL and CALIPSO profiles can be attributed to averaging since the 15-min–60-m RL profile agrees well with CALIPSO above 8.2 km and the 9–30-min RL profile agrees well below. Making the same comparison for daytime profiles (Fig. 11a), the 15-min/60-m RL data show more cloud from about 11 to 16 km. This suggests the increased solar background may cause some optically thin clouds to go undetected by CALIPSO, although this difference is not statistically significant. Overall, when approximately the same amount of averaging is used, we find good agreement between RL-FEX and the CALIPSO VFM product.

The agreement between the RL and CALIPSO in Fig. 11 is similar to that found by Thorsen et al. (2013b). However, unlike the current study, agreement between CALIPSO and the RL was found by Thorsen et al. (2013b) when the RL data were averaged to 2 min and 30 m. The reason for this discrepancy is that the Thorsen et al. (2013b) study applied a median filter to the depolarization ratio, which was used to identify cirrus, with a moving window of 150 m by 10 min. This filter created a dataset with an effective resolution of 10 min, which we show here to be approximately equivalent to CALIPSO’s horizontal averaging.

Comparisons of aerosol vertical occurrence profile to CALIPSO were also performed in a similar manner and are given in Fig. 13. At lower heights FEX detects aerosol nearly 100% of the time as one would expect in the boundary layer. However, CALIPSO detects significantly less aerosol than FEX even for the subset of nighttime profiles with no clouds present. This limits our ability to straightforwardly assess FEX’s aerosol detection using CALIPSO, although resolving the differences in Fig. 13 will be the subject of future work. Credibility in FEX’s vertical extent of aerosol is partially demonstrated by good agreement in aerosol optical depth measured by the Aerosol Robotic Network (AERONET; Holben et al. 1998) as shown in Part II since FEX extinction retrievals are only performed in bins where aerosol is detected. Furthermore, detection of aerosol is performed in the same consistent framework as cloud detection and therefore we do not expect large inconsistencies between the two.

Fig. 13.
Fig. 13.

The aerosol occurrence in transparent profiles from the RL at 2 min and 30 m of averaging (blue) and from the CALIPSO VFM product (red). Also given is the vertical occurrence in the subset of transparent profiles where no clouds are detected (no cloud). Profiles are shown separately for the (a) daytime and (b) nighttime observations. The shading encompasses the uncertainty due to sampling (95% confidence) determined by bootstrapping. This comparison includes CALIPSO profiles that fall within 200 km of the Darwin site from December 2010 through December 2014. The RL data are limited to times within ±2 h of CALIPSO overpasses.

Citation: Journal of Atmospheric and Oceanic Technology 32, 11; 10.1175/JTECH-D-14-00150.1

6. Summary and conclusions

We have presented an automated method for the detection of features in the Atmospheric Radiation Measurement Program’s (ARM) Raman lidar (RL) measurements, which is Part I of the feature detection and extinction retrieval (FEX) algorithm. While the detection of features in FEX requires an estimate of particulate extinction, this work focused on the aspects most pertinent to feature detection. The retrieval of particulate extinction in FEX is given in Part II. The intent is to run FEX operationally within the ARM Data Management Facility (DMF) with the output being made available to the general user community via the ARM website (http://www.arm.gov/).

The feature detection approach used in FEX is to analyze multiple quantities—scattering ratios derived using elastic and nitrogen channel signals from two FOVs, the scattering ratio derived using only the elastic channel, and the total volume depolarization ratio—for the presence of features. Range-dependent detection thresholds are determined for each profile separately by calculating the expected noise in a clear-sky signal. We show that the approach of using multiple quantities provides a complementary description of the vertical and temporal extent of clouds and aerosols. Using multiple quantities also allows for additional consistency checks on the feature mask, which reduces the rate of false detections and improves the efficiency of the algorithm.

The performance of FEX’s feature detection is illustrated by the application of the algorithm to 4 years of ARM RL data over Darwin, Australia. The feature detection in FEX is found to be robust across the diurnal cycle, as no bias exists in the elastic channel detection thresholds. Sufficiently small noise in the depolarization ratio and the strong contrast between molecular and typical in-cloud values allows for the depolarization ratio to be the most effective for the detection of thin cirrus for the ARM RL system. We also presented the improvement of FEX’s cloud mask relative to the cloud mask used in the current operational ARM products. Evaluations of FEX’s cloud detection were made by comparing to the collocated ARM micropulse lidar (MPL) and to observations from the Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite, which uses independent methods for identifying clouds. RL-FEX detects more clouds than the ARM MPL due to the RL’s larger SNR. However, when the comparison is limited to when both instruments’ SNR was above 3 good agreement was found. Comparisons with CALIPSO also agreed well for transparent profiles when similar amounts of signal averaging are used, particularly for nighttime profiles.

While we have focused on one particular lidar system, the FEX framework is flexible enough to be extended to other Raman or HSRL systems that can intrinsically separate signal returns from molecules and particulates.

Acknowledgments

The Raman lidar, radiosonde, sun photometer, microwave radiometer, and micropulse lidar datasets were obtained from the ARM data archive (www.archive.arm.gov). The CALIPSO data sets were obtained from the NASA Langley Research Center Atmospheric Science Data Center. This work was greatly improved by the excellent reviews of J. Campbell, M. Vaughan, and one anonymous reviewer. This research was supported by the Office of Science (BER), U.S. Department of Energy through Grants DE-SC0010557 (T.J. Thorsen and Q. Fu) and DE-SC0006898 (D.D. Turner). It was also supported by NASA Grants NNX14AB28G (Q. Fu) and NNX13AN49G (T. J. Thorsen and Q. Fu). J. M. Comstock and R. K. Newsom were supported by both DOE ASR and ARM.

APPENDIX

Depolarization Calibration

The ARM RL separates the high elastic signal into parallel and perpendicular components. To prevent saturation of the detector, an extra neutral density filter exists in the parallel channel. Therefore, to determine the total elastic signal and the depolarization, the difference in gain between the parallel and perpendicular channels must be taken into account. In addition, the accuracy of the depolarization measurements is affected by possible misalignment of the polarizing beamsplitter to that of the transmitted beam. This misalignment induces cross talk into the signal—some portion of the perpendicular signal ends up in the parallel channel and vice versa. The relationship between the observed or apparent signals and the true signals is given by (Alvarez et al. 2006)
ea1
and
ea2
where and are the observed signals in the parallel and perpendicular channels, respectively. Terms and are the true parallel and perpendicular signals, respectively. The factor G represents the relative gain, that is, the reduction in signal by the extra neutral density filter in the parallel channel relative to the perpendicular channel. The angle ϕ is the misalignment angle between the transmitter and receiver polarization planes.

The relative gain G is taken as the ratio of daytime perpendicular to parallel channel background signal during the presence of optically thick liquid water clouds. In these situations the true depolarization is 1 since multiply-scattered sunlight through the clouds will be randomly polarized, and by using the background signals any misalignment between the transmitter and receiver polarization planes is irrelevant. Times of optically thick liquid water clouds are identified using collocated microwave radiometer (MWR) observations (Liljegren and Lesht 1996)—defined as times when the liquid water path exceeded 75 g m–2. Multiple years of collocated RL and MWR data are used to calculate a mean G offline. We find G values of 21.26 and 6.40 at the TWP and SGP site, respectively, with standard deviations of about 10% of these mean values.

Given the offline estimate of G, the misalignment angle ϕ is then determined from backscattered signals in clear sky. Forming a ratio of Eqs. (A2) to (A1) and solving for ϕ gives
ea3
where is the observed depolarization, that is, ; and is the true depolarization due to molecular scattering, that is, . The true molecular depolarization is calculated using the model of Behrendt and Nakamura (2002), and for the RL’s interference filter’s bandpass we find a value . Since we need to identify clear-sky signals, the calculation of ϕ using Eq. (A3) is performed as an addition step in the iterative portion of FEX using the same tactics as is done for the other calibration constants (section 3d). The breakdown of what resolution ϕ is determined at is given in Table A1 along with the associated uncertainties, which are about 5%.
Table A1.

The percentage of profiles by the calibration type used from December 2010 through December 2014 at Darwin for the depolarization misalignment angle. The mean relative uncertainties (%) are given in parentheses. Uncertainties are computed by taking the standard deviation in each per-day, per-tweak, or per-profile region. The uncertainty in the historical calibration constants are determined by repeatedly (10 000 times) randomly sampling 7 days of per-profile constants, and in each random sample calculating the sum (in quadrature) of the standard deviation of the per-profile values and the mean standard deviation in the per-profile values themselves.

Table A1.

Note that in this paper the calculation of signal noise detailed in section 3a is performed using the observed signals (i.e., and ). However, for the purpose of feature detection, the true signals are used (i.e., and ). Therefore, in practice, propagation of uncertainty is applied to Eqs. (A1) and (A2) to determine the noise in these true signals.

REFERENCES

  • Alvarez, J. M., Vaughan M. A. , Hostetler C. A. , Hunt W. H. , and Winker D. M. , 2006: Calibration technique for polarization-sensitive lidars. J. Atmos. Oceanic Technol., 23, 683699, doi:10.1175/JTECH1872.1.

    • Search Google Scholar
    • Export Citation
  • Behrendt, A., and Nakamura T. , 2002: Calculation of the calibration constant of polarization lidar and its dependency on atmospheric temperature. Opt. Express, 10, 805817, doi:10.1364/OE.10.000805.

    • Search Google Scholar
    • Export Citation
  • Bevington, P., and Robinson D. , 2002: Data Reduction and Error Analysis for the Physical Sciences. 3rd ed. McGraw-Hill, 336 pp.

  • Borg, L. A., Holz R. E. , and Turner D. D. , 2011: Investigating cloud radar sensitivity to optically thin cirrus using collocated Raman lidar observations. Geophys. Res. Lett., 38, L05807, doi:10.1029/2010GL046365.

    • Search Google Scholar
    • Export Citation
  • Bucholtz, A., 1995: Rayleigh-scattering calculations for the terrestrial atmosphere. Appl. Opt., 34, 27652773, doi:10.1364/AO.34.002765.

    • Search Google Scholar
    • Export Citation
  • Campbell, J. R., Hlavka D. L. , Welton E. J. , Flynn C. J. , Turner D. D. , Spinhirne J. D. , Scott V. S. , and Hwang I. H. , 2002: Full-time, eye-safe cloud and aerosol lidar observation at Atmospheric Radiation Measurement Program sites: Instruments and data processing. J. Atmos. Oceanic Technol., 19, 431, doi:10.1175/1520-0426(2002)019<0431:FTESCA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Campbell, J. R., Sassen K. , and Welton E. J. , 2008: Elevated cloud and aerosol layer retrievals from micropulse lidar signal profiles. J. Atmos. Oceanic Technol., 25, 685700, doi:10.1175/2007JTECHA1034.1.

    • Search Google Scholar
    • Export Citation
  • Carswell, A. I., and Pal S. R. , 1980: Polarization anisotropy in lidar multiple scattering from clouds. Appl. Opt., 19, 41234126, doi:10.1364/AO.19.004123.

    • Search Google Scholar
    • Export Citation
  • Clothiaux, E. E., Mace G. G. , Ackerman T. P. , Kane T. J. , Spinhirne J. D. , and Scott V. S. , 1998: An automated algorithm for detection of hydrometeor returns in micropulse lidar data. J. Atmos. Oceanic Technol., 15, 10351042, doi:10.1175/1520-0426(1998)015<1035:AAAFDO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Clothiaux, E. E., Ackerman T. P. , Mace G. G. , Moran K. P. , Marchand R. T. , Miller M. A. , and Martner B. E. , 2000: Objective determination of cloud heights and radar reflectivities using a combination of active remote sensors at the ARM CART sites. J. Appl. Meteor., 39, 645665, doi:10.1175/1520-0450(2000)039<0645:ODOCHA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Comstock, J. M., Ackerman T. P. , and Mace G. G. , 2002: Ground-based lidar and radar remote sensing of tropical cirrus clouds at Nauru Island: Cloud statistics and radiative impacts. J. Geophys. Res., 107, 4714, doi:10.1029/2002JD002203.

    • Search Google Scholar
    • Export Citation
  • Comstock, J. M., Protat A. , McFarlane S. A. , Delanoë J. , and Deng M. , 2013: Assessment of uncertainty in cloud radiative effects and heating rates through retrieval algorithm differences: Analysis using 3 years of ARM data at Darwin, Australia. J. Geophys. Res. Atmos., 118, 45494571, doi:10.1002/jgrd.50404.

    • Search Google Scholar
    • Export Citation
  • Cooney, J., Orr J. , and Tomasetti C. , 1969: Measurements separating the gaseous and aerosol components of laser atmospheric backscatter. Nature, 224, 10981099, doi:10.1038/2241098a0.

    • Search Google Scholar
    • Export Citation
  • Coulter, R., 2012: Micropulse lidar (MPL) handbook. U.S. Department of Energy Tech. Rep. DOE/SC-ARM/TR-019, 12 pp. [Available online at https://www.arm.gov/publications/tech_reports/handbooks/mpl_handbook.pdf.]

  • Davis, S., and Coauthors, 2010: In situ and lidar observations of tropopause subvisible cirrus clouds during TC4. J. Geophys. Res., 115, D00J17, doi:10.1029/2009JD013093.

    • Search Google Scholar
    • Export Citation
  • Dessler, A. E., and Yang P. , 2003: The distribution of tropical thin cirrus clouds inferred from Terra MODIS data. J. Climate, 16, 12411247, doi:10.1175/1520-0442(2003)16<1241:TDOTTC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Dupont, J.-C., Haeffelin M. , Morille Y. , Comstock J. M. , Flynn C. , Long C. N. , Sivaraman C. , and Newson R. K. , 2011: Cloud properties derived from two lidars over the ARM SGP site. Geophys. Res. Lett., 38, L08814, doi:10.1029/2010GL046274.

    • Search Google Scholar
    • Export Citation
  • Feldman, D. R., L’Ecuyer T. S. , Liou K. N. , and Yung Y. L. , 2008: Remote sensing of tropical tropopause layer radiation balance using A-train measurements. J. Geophys. Res., 113, D21113, doi:10.1029/2008JD010158.

    • Search Google Scholar
    • Export Citation
  • Ferrare, R., and Coauthors, 2006: Evaluation of daytime measurements of aerosols and water vapor made by an operational Raman lidar over the Southern Great Plains. J. Geophys. Res., 111, D05S08, doi:10.1029/2005JD005836.

    • Search Google Scholar
    • Export Citation
  • Fu, Q., Hu Y. , and Yang Q. , 2007: Identifying the top of the tropical tropopause layer from vertical mass flux analysis and CALIPSO lidar cloud observations. Geophys. Res. Lett., 34, L14813, doi:10.1029/2007GL030099.

    • Search Google Scholar
    • Export Citation
  • Gobbi, G. P., 1998: Polarization lidar returns from aerosols and thin clouds: A framework for the analysis. Appl. Opt., 37, 55055508, doi:10.1364/AO.37.005505.

    • Search Google Scholar
    • Export Citation
  • Goldsmith, J. E. M., Blair F. H. , Bisson S. E. , and Turner D. D. , 1998: Turn-key Raman lidar for profiling atmospheric water vapor, clouds, and aerosols. Appl. Opt., 37, 49794990, doi:10.1364/AO.37.004979.

    • Search Google Scholar
    • Export Citation
  • Grund, C. J., and Eloranta E. W. , 1991: University of Wisconsin high spectral resolution lidar. Opt. Eng., 30, 612, doi:10.1117/12.55766.

    • Search Google Scholar
    • Export Citation
  • Hair, J. W., and Coauthors, 2008: Airborne High Spectral Resolution Lidar for profiling aerosol optical properties. Appl. Opt., 47, 67346752, doi:10.1364/AO.47.006734.

    • Search Google Scholar
    • Export Citation
  • Haladay, T., and Stephens G. , 2009: Characteristics of tropical thin cirrus clouds deduced from joint CloudSat and CALIPSO observations. J. Geophys. Res., 114, D00A25, doi:10.1029/2008JD010675.

    • Search Google Scholar
    • Export Citation
  • Hogan, R. J., and Illingworth A. J. , 2000: Deriving cloud overlap statistics from radar. Quart. J. Roy. Meteor. Soc., 126, 29032909, doi:10.1002/qj.49712656914.

    • Search Google Scholar
    • Export Citation
  • Holben, B., and Coauthors, 1998: AERONET—A federated instrument network and data archive for aerosol characterization. Remote Sens. Environ., 66, 116, doi:10.1016/S0034-4257(98)00031-5.

    • Search Google Scholar
    • Export Citation
  • Kingston, R., 1978: Detection of Optical and Infrared Radiation. Springer Series in Optical Sciences, Vol. 10, Springer-Verlag, 142 pp.

  • Liljegren, J. C., and Lesht B. M. , 1996: Measurements of integrated water vapor and cloud liquid water from microwave radiometers at the DOE ARM Cloud and Radiation Testbed in the U.S. Southern Great Plains. IGARSS ’96: 1996 International Geoscience and Remote Sensing Symposium; Remote Sensing for a Sustainable Geoscience and Remote Sensing Symposium, Vol. 3, IEEE, 1675–1677.

  • Luo, Y., Xu K.-M. , Morrison H. , McFarquhar G. M. , Wang Z. , and Zhang G. , 2008: Multi-layer Arctic mixed-phase clouds simulated by a cloud-resolving model: Comparison with ARM observations and sensitivity experiments. J. Geophys. Res., 113, D12208, doi:10.1029/2007JD009563.

    • Search Google Scholar
    • Export Citation
  • Massie, S. T., Gille J. , Craig C. , Khosravi R. , Barnett J. , Read W. , and Winker D. , 2010: HIRDLS and CALIPSO observations of tropical cirrus. J. Geophys. Res., 115, D00H11, doi:10.1029/2009JD012100.

    • Search Google Scholar
    • Export Citation
  • Matthais, V., and Coauthors, 2004: Aerosol lidar intercomparison in the framework of the EARLINET project. 1. Instruments. Appl. Opt., 43, 961976, doi:10.1364/AO.43.000961.

    • Search Google Scholar
    • Export Citation
  • Measures, R. M., 1984: Laser Remote Sensing: Fundamentals and Applications. Wiley, 510 pp.

  • Melfi, S. H., 1972: Remote measurements of the atmosphere using Raman scattering. Appl. Opt., 11, 16051610, doi:10.1364/AO.11.001605.

    • Search Google Scholar
    • Export Citation
  • Murayama, T., and Coauthors, 2001: Ground-based network observation of Asian dust events of April 1998 in east Asia. J. Geophys. Res., 106, 18 34518 359, doi:10.1029/2000JD900554.

    • Search Google Scholar
    • Export Citation
  • Newsom, R. K., 2009: Raman lidar (RL) handbook. U.S. Department of Energy Tech. Rep. DOE/SC-ARM/TR-038, 25 pp. [Available online at https://www.arm.gov/publications/tech_reports/handbooks/rl_handbook.pdf.]

  • Newsom, R. K., Turner D. D. , Mielke B. , Clayton M. , Ferrare R. , and Sivaraman C. , 2009: Simultaneous analog and photon counting detection for Raman lidar. Appl. Opt., 48, 39033914, doi:10.1364/AO.48.003903.

    • Search Google Scholar
    • Export Citation
  • Newsom, R. K., Turner D. D. , and Goldsmith J. E. M. , 2013: Long-term evaluation of temperature profiles measured by an operational Raman lidar. J. Atmos. Oceanic Technol., 30, 16161634, doi:10.1175/JTECH-D-12-00138.1.

    • Search Google Scholar
    • Export Citation
  • Oliver, B. M., 1965: Thermal and quantum noise. Proc. IEEE, 53, doi:10.1109/PROC.1965.3814.

  • Riihimaki, L. D., McFarlane S. A. , and Comstock J. M. , 2012: Climatology and formation of tropical midlevel clouds at the Darwin ARM site. J. Climate, 25, 68356850, doi:10.1175/JCLI-D-11-00599.1.

    • Search Google Scholar
    • Export Citation
  • Sassen, K., 1991: The polarization lidar technique for cloud research: A review and current assessment. Bull. Amer. Meteor. Soc., 72, 18481866, doi:10.1175/1520-0477(1991)072<1848:TPLTFC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Sassen, K., 2002: Indirect climate forcing over the western US from Asian dust storms. Geophys. Res. Lett., 29, 103-1103-4, doi:10.1029/2001GL014051.

    • Search Google Scholar
    • Export Citation
  • Stephens, G. L., and Coauthors, 2002: The CloudSat mission and the A-Train—A new dimension of space-based observations of clouds and precipitation. Bull. Amer. Meteor. Soc., 83, 17711790, doi:10.1175/BAMS-83-12-1771.

    • Search Google Scholar
    • Export Citation
  • Sun, W., Videen G. , Kato S. , Lin B. , Lukashin C. , and Hu Y. , 2011: A study of subvisual clouds and their radiation effect with a synergy of CERES, MODIS, CALIPSO, and AIRS data. J. Geophys. Res., 116, D22207, doi:10.1029/2011JD016422.

    • Search Google Scholar
    • Export Citation
  • Thorsen, T. J., and Fu Q. , 2015: Automated retrieval of cloud and aerosol properties from the ARM Raman lidar. Part II: Extinction. J. Atmos. Oceanic Technol., doi:10.1175/JTECH-D-14-00178.1, in press.

    • Search Google Scholar
    • Export Citation
  • Thorsen, T. J., Fu Q. , and Comstock J. , 2011: Comparison of the CALIPSO satellite and ground-based observations of cirrus clouds at the ARM TWP sites. J. Geophys. Res., 116, D21203, doi:10.1029/2011JD015970.

    • Search Google Scholar
    • Export Citation
  • Thorsen, T. J., Fu Q. , and Comstock J. M. , 2013a: Cloud effects on radiative heating rate profiles over Darwin using ARM and A-train radar/lidar observations. J. Geophys. Res. Atmos., 118, 56375654, doi:10.1002/jgrd.50476.

    • Search Google Scholar
    • Export Citation
  • Thorsen, T. J., Fu Q. , Comstock J. M. , Sivaraman C. , Vaughan M. A. , Winker D. M. , and Turner D. D. , 2013b: Macrophysical properties of tropical cirrus clouds from the CALIPSO satellite and from ground-based micropulse and Raman lidars. J. Geophys. Res. Atmos., 118, 92099220, doi:10.1002/jgrd.50691.

    • Search Google Scholar
    • Export Citation
  • Turner, D. D., Ferrare R. A. , Brasseur L. A. H. , Feltz W. F. , and Tooman T. P. , 2002: Automated retrievals of water vapor and aerosol profiles from an operational Raman lidar. J. Atmos. Oceanic Technol., 19, 3750, doi:10.1175/1520-0426(2002)019<0037:AROWVA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Vaughan, M. A., and Coauthors, 2009: Fully automated detection of cloud and aerosol layers in the CALIPSO lidar measurements. J. Atmos. Oceanic Technol., 26, 20342050, doi:10.1175/2009JTECHA1228.1.

    • Search Google Scholar
    • Export Citation
  • Wandinger, U., and Ansmann A. , 2002: Experimental determination of the lidar overlap profile with Raman lidar. Appl. Opt., 41, 511, doi:10.1364/AO.41.000511.

    • Search Google Scholar
    • Export Citation
  • Wang, P.-H., Minnis P. , McCormick M. P. , Kent G. S. , Yue G. K. , Young D. F. , and Skeens K. M. , 1998: A study of the vertical structure of tropical (20°S-20°N) optically thin clouds from SAGE II observations. Atmos. Res., 47–48, 599614, doi:10.1016/S0169-8095(97)00085-9.

    • Search Google Scholar
    • Export Citation
  • Wang, Z., and Sassen K. , 2001: Cloud type and macrophysical property retrieval using multiple remote sensors. J. Appl. Meteor., 40, 16651682, doi:10.1175/1520-0450(2001)040<1665:CTAMPR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Wang, Z., and Sassen K. , 2002: Cirrus cloud microphysical property retrieval using lidar and radar measurements. Part I: Algorithm description and comparison with in situ data. J. Appl. Meteor., 41, 218229, doi:10.1175/1520-0450(2002)041<0218:CCMPRU>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Welton, E. J., and Campbell J. R. , 2002: Micropulse lidar signals: Uncertainty analysis. J. Atmos. Oceanic Technol., 19, 20892094, doi:10.1175/1520-0426(2002)019<2089:MLSUA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Whiteman, D. N., 2003a: Examination of the traditional Raman lidar technique. I. Evaluating the temperature-dependent lidar equations. Appl. Opt., 42, 25712592, doi:10.1364/AO.42.002571.

    • Search Google Scholar
    • Export Citation
  • Whiteman, D. N., 2003b: Examination of the traditional Raman lidar technique. II. Evaluating the ratios for water vapor and aerosols. Appl. Opt., 42, 2593, doi:10.1364/AO.42.002593.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., and Vaughan M. , 1994: Vertical distribution of clouds over Hampton, Virginia observed by lidar under the ECLIPS and FIRE ETO programs. Atmos. Res., 34, 117133, doi:10.1016/0169-8095(94)90084-1.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., and Trepte C. R. , 1998: Laminar cirrus observed near the tropical tropopause by LITE. Geophys. Res. Lett., 25, 33513354, doi:10.1029/98GL01292.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., Vaughan M. A. , Omar A. , Hu Y. , Powell K. A. , Liu Z. , Hunt W. H. , and Young S. A. , 2009: Overview of the CALIPSO mission and CALIOP data processing algorithms. J. Atmos. Oceanic Technol., 26, 23102323, doi:10.1175/2009JTECHA1281.1.

    • Search Google Scholar
    • Export Citation
  • Winker, D. M., and Coauthors, 2010: The CALIPSO mission: A global 3D view of aerosols and clouds. Bull. Amer. Meteor. Soc., 91, 12111229, doi:10.1175/2010BAMS3009.1.

    • Search Google Scholar
    • Export Citation
  • Wulfmeyer, V., Pal S. , Turner D. , and Wagner E. , 2010: Can water vapour Raman lidar resolve profiles of turbulent variables in the convective boundary layer? Bound.-Layer Meteor., 136, 253284, doi:10.1007/s10546-010-9494-z.

    • Search Google Scholar
    • Export Citation
  • Yang, Q., Fu Q. , and Hu Y. , 2010: Radiative impacts of clouds in the tropical tropopause layer. J. Geophys. Res., 115, D00H12, doi:10.1029/2009JD012393.

    • Search Google Scholar
    • Export Citation
Save