1. Introduction

As lidar technologies have developed and improved, researchers have proposed a number of analytical techniques to derive cloud and aerosol layer heights from signal profile samples. Platt et al. (1994) describe three such approaches: the differential zero-crossing method (e.g., Pal et al. 1992), the threshold method (e.g., Winker and Vaughan 1994), and a quantitative approach based on the clear-air scattering assumption (e.g., Sassen and Cho 1992). Subsequent efforts have modified, adapted, and advanced these methods (Young 1995; Flamant et al. 1997; Clothiaux et al. 1998; Shimizu et al. 2004). Furthermore, wavelet transform analysis has been introduced as a means for identifying particulate layer scattering structure (Cohn and Angevine 2000; Brooks 2003). No uniform cloud and aerosol layer detection algorithm can be applied to lidar measurements without limitation. This is because there is no binary solution to analyzing signal returns for particulate layer structure across the laser emission spectrum (Wang and Sassen 2001).

This paper describes a thresholding-based detection technique designed for and tested using micropulse lidar (MPL; 0.523 μm) data. MPL is an autonomous and eye-safe instrument developed in 1992 at the National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC; Spinhirne 1993). Eye safety enables unattended full-time instrument operation. Transmitted MPL pulse energies are between 5.0 and 10.0 μJ. This is less than 40% of the quoted 25.0-μJ American National Standards Institute (ANSI) eye-safety limit at the source wavelength, given the 0.20-m Schmidt–Cassegrain transmitter/receiver (transceiver) aperture and 2500-Hz pulse repetition frequency used (Campbell et al. 2002).

Easily deployed and maintained, MPL instruments detect nearly all tropospheric and lower-stratospheric clouds and aerosols, to the limit of signal attenuation, through pulse summation and geometric signal compression (Spinhirne et al. 1995). The MPL Network (MPLNET; Welton et al. 2001) is a federated group of MPL instruments deployed worldwide in support of basic science and the NASA Earth Observing System (EOS) program (Wielicki et al. 1995). Value-added network datasets are made available to the community via an online repository (http://mplnet.gsfc.nasa.gov).

Threshold-based algorithms for cloud and aerosol layer detection have been described previously for MPL instruments based on retrieved signals (Campbell et al. 1998; Clothiaux et al. 1998; Mahesh et al. 2005). However, the MPLNET archive includes data collected from many climate regimes worldwide depicting a diverse sample of cloud and aerosol macrophysical scenarios. Furthermore, our experience shows that instrument performance can vary widely across the network. Algorithms based on the highly dependent signal profile of one or a few instruments may lead to intensive maintenance and threshold tuning when applied to many instruments for a large network. Therefore, our goal is to derive an algorithm based on the statistical uncertainties of the signal profile, an independent variable, thereby making it more easily transferable between network datasets. We also describe an option for multitemporal processing, where the algorithm may be run over multiple time steps using the results of a base-resolution iteration to limit attenuation effects in longer averages. Results may then be analyzed by the user to decide the optimal resolution for a given subject of study.

Similar to Winker and Vaughan (1994), the algorithm includes a profile normalization step involving a theoretical molecular scattering profile that is used to simulate clear-sky background conditions. This is done for a section of the profile that is deemed free of particulate scattering, which most commonly occurs for free-tropospheric air. Because of the ambiguity in potential signal transmission losses below this height, the algorithm may only be applied to the portion of the profile above the normalization region and, therefore, elevated cloud and aerosol layer retrievals.

Polar stratospheric cloud (PSC) retrievals for data collected from the South Pole MPLNET instrument (90.00°S; 2.835 km MSL) are described to illustrate the algorithm. PSCs are commonly found near and above 20.0 km AGL over Antarctica (e.g., Gobbi et al. 1998), making their detection with a low-powered lidar instrument a demanding scenario, even during polar night. The algorithm is outlined in the following section, including results for the base-resolution iteration. Algorithm performance is analyzed for varying cases of solar background to evaluate the effects of ambient noise on the retrievals. This is followed by a discussion of algorithm threshold values, their variability, and the benefits of multitemporal processing. We conclude by speculating on the application of this technique for nadir-pointing instruments, such as mounted on aircraft or satellite platforms, where the entire column profile could be analyzed without limitation.

2. Methodology

The level 1.0 MPLNET data file disseminated to the research community contains the normalized relative backscatter (NRB) product described by Campbell et al. (2002). They describe the algorithm for processing raw photon count rates, recorded at prescribed temporal and spatial sampling resolutions, to an uncalibrated attenuated backscatter coefficient, after accounting for instrument correction terms. Based on the scattering form of the lidar equation (e.g., Measures 1984), they defined this parameter as

View Expanded

where n is photoelectron counts per second at range r, O_c is the optical overlap correction as a function of range (the result of field-of-view conflicts in the transceiver system), C represents a dimensional calibration constant, E is the laser-transmitted pulse energy, β is the backscatter coefficient from all types of atmospheric scattering, T is atmospheric transmittance, n_b is the background contribution of ambient light, n_ap is the contribution from afterpulse (cross talk induced by internal reflections of the laser pulse), and D is the detector photon-coincidence “dead time” factor as a function of raw count rate. The units for NRB are (photoelectrons km⁻²)/(μs μJ⁻¹).

Welton and Campbell (2002) describe the algorithm for quantifying the uncertainty of Eq. (1). Based on Bevington (1969), they rewrite Eq. (1) as

View Expanded

where

View Expanded

View Expanded

Here, δX denotes the statistical uncertainty of X based on Poisson statistics and is consistent with the nomenclature introduced by Welton and Campbell (2002). In Eq. (3), N refers to the total number of laser pulses per sample interval. A 60-s resolution setting is most common. Therefore, a sample interval, referred to as a “shot,” represents the averaging of 150 000 pulses at 2500 Hz. Uncertainty in the dead-time correction is not considered, because its magnitude is not significant at typical count rates. In Eq. (4), η(r) is the fractional uncertainty of NRB as a function of range. The inverse of η(r) is analogous to a signal-to-noise ratio (SNR).

SNR levels for the MPL may not always be adequate for postprocessing because of noise saturation. For evaluating low-level clouds, 1-min resolution NRB profiling has to be proven sufficient (e.g., Flynn et al. 2007). In the case of optically thin clouds, such as tropical cirrus (e.g., Comstock et al. 2002) or PSC, integrated time averages improve retrievals. However, practical limitations exist with longer averaging. MPL instrument performance has been shown to vary adversely over relatively short time periods from thermal effects (Campbell et al. 2000). Also, the attenuation effects of optically thick layers bias the vertical structure in averages such that they fail to resemble realistic lidar profiles.

Similar to Winker and Vaughan (1994), a normalization step is performed using a theoretical clear-sky molecular scattering profile (shown below). This step cannot successfully be completed for signal profiles constructed from long averages that are attenuation limited. Therefore, we first define a base temporal resolution, with the intent that lower-resolution (longer) averages will be processed later. The results of the base-resolution analysis will be used to screen attenuation-limited profiles from the longer averages. To initiate the PSC retrievals, we use a base-resolution setting of 0.01 fractional days (approximately 15 min), which reflects a minimum practical averaging period that yields reasonable results. From Bevington (1969), this average is defined as

View Expanded

and its relative uncertainty as

View Expanded

where χ denotes the averaged NRB(r) value beginning from that time and θ is the temporal base resolution. Here, N is the number of shots in the average.

NRB fractional uncertainties from 10 June 2003 at the South Pole using 1-min shot averaging are shown in Fig. 1a. Following Eqs. (5) and (6), fractional uncertainties for 0.01 fractional day averages are shown in Fig. 1b. Cloud layers above 15.0 km AGL are more easily delineated here by their higher relative SNR values. Note, the discontinuity near 6.0 km AGL reflects the effect of uncertainties on the overlap correction applied below this range.

After averaging, the resulting NRB profile [Eq. (5)] is normalized with a theoretical molecular scattering profile. The result is then used to initiate a clear-sky search for approximating C in Eq. (1). The molecular scattering profile comes from a standard atmosphere table for air density at polar latitudes (Environmental Science Services Administration et al. 1967). Sounding data are not always available in real time from MPLNET sites, so we, therefore, implement a standard density profile and assume a moderate relative uncertainty to compensate for any differences to the implicit assumption of their equality. From Eq. (1), this yields

View Expanded

and the uncertainty for this calculation, again based on Bevington (1969), is then

View Expanded

Because C* is an approximation to C in Eq. (1), we designate it with an asterisk to indicate the difference. The two values would be equal in the case of no transmission losses between the instrument and r resulting from cloud or aerosol presence/scattering. However, there is no inherent knowledge that this may be the case either way at this point. This is an intermediate step and the two values should not be confused. The relative uncertainty for the molecular scattering terms in Eq. (7) is estimated at a combined 5% for Eq. (8). We again stress that this could be improved with a priori knowledge of the density profile from a relevant local sounding. Note that the χ notation is dropped from this step forward so that the following equations involving NRB refer specifically to the averaged profile at the base resolution.

It is most practical to find a clear-sky/calibrating region nearest to the instrument as possible to increase the subsequent depth of the profile analyzed for cloud and aerosol. At the South Pole, blowing snow and diamond dust are common to heights approaching 1.0 km above ground (e.g., Mahesh et al. 2003, 2005). However, to simulate conditions common among most climate regimes, where aerosols are frequently present within and just above the boundary layer, we simulate the clear-sky search here beginning from 3.0 km AGL. A clear-air slot is considered found by determining the first range bin working upward that satisfies the requirements

View Expanded

View Expanded

where

View Expanded

Equation (9) requires that C* and its uncertainty for those range bins between r and r_N [specified by Eq. (10)] fall within that of C*(r) and its uncertainty. Equation (10) establishes a number of consecutive range bins that, when averaged together with C*(r), yield a value of (r:r_N)/δ(r:r_N) that is greater than the reference threshold ɛ. Here, Δr is the system range resolution (0.030 km for the South Pole instrument). The value inside the bracket is always rounded up such that its minimum value is 1, thereby requiring r_N > r such that a minimum of two bins are necessary to satisfy Eqs. (9a)–(9b). This is done in consideration of the possible influence of noise on a single bin.

These equations are designed to be analogous to Eq. (6) where for any given NRB(r) and its corresponding uncertainty δNRB(r) we can write

View Expanded

for N = 1. By substituting Eq. (4) here, the value N for the hypothetical case η = 1/−ɛ can be written as

View Expanded

and then

View Expanded

Used in Eq. (10), ɛ represents a reference threshold value for (r:r_N)/δ(r:r_N), whereby its selection determines the number of bins necessary to satisfy Eq. (9), and solve C* below, to within a percentage equal to its inverse.

These steps are predicated on the assumption that particulate scattering structure in the lidar profile is vertically inhomogeneous relative to that of the molecular profile. Furthermore, we assume that δNRB(r) does not change significantly over the range from r to r_N. That is, as illustrated in Eq. (12), δNRB(r) is considered constant over the range N whereas, in reality, it is not. For practical purposes, however, this deviation is negligible. The assumption of vertical particulate inhomogeneity is vital, however. Any cloud and aerosol scattering present in the profile must not exhibit structure similar to that of molecular scattering within the derived uncertainties, over the number of bins N, determined using Eq. (10). Otherwise, a false positive may occur.

We illustrate the preceding steps with a sample case, shown in Fig. 2, for the 0.01 fractional day NRB average (approximately 15 min), beginning at 0.55 10 June 2003 at the South Pole. To facilitate this discussion, ɛ is set to 100.0. The averaged profile is shown in Fig. 2a from the ground to 25.0 km AGL versus the prescribed molecular scattering profile. In this case, no particulate scattering is evident in the lidar profile between the ground and near 14.0 km AGL. Normalization of the profile through Eq. (7) is shown in Fig. 2b for the range of 3.0–6.0 km AGL, with error bars derived using Eq. (8). The number of significant bins required to satisfy Eq. (9) at each point over this range is shown in Fig. 2c. In this case, the condition in Eq. (8) was first met by the range bin centered at 3.04 km AGL, using six adjacent bins. A final normalization value (C*_f ) and its uncertainty are then solved for by averaging these seven bins with applicable forms of Eqs. (5) and (6). As stated above, the final value C*_f is calculated such that C*_f/δC*_f exceeds ɛ and is accurate to within 1/−ɛ.

The NRB profile is renormalized using C*_f to derive a final value termed pseudoattenuated backscatter (PAB),

View Expanded

where

View Expanded

As implied earlier, C*_f may be defined as

View Expanded

Use of the term “pseudo” recognizes the ambiguity between C*_f and C from Eq. (1), resulting from a lack of knowledge for T ²_p, which may be less than unity due to any scattering occurring below the height (3.0 km AGL) where the clear-sky search is initiated.

A threshold for cloud and aerosol detection is reached by modifying Eq. (15) under the assumption of clear sky. The theoretical molecular scattering profile used to normalize the NRB profile above is substituted here for PAB and then added to the product of the molecular scattering coefficient multiplied by the relative uncertainty in PAB. This latter step forms an outer error bar. The threshold α is then

View Expanded

Therefore, if a subject profile depicts scattering solely from the molecular atmosphere, PAB should fall mostly within the bounds of the threshold, except for some noise, because we have superimposed the uncertainties of the profile onto that representing a clear sky. If cloud or aerosol scattering is present, these structures should then be discernable from molecular scattering by exceeding the uncertainty represented now in the threshold.

In Fig. 3, the previous example for 10 June 2003 is continued. The PAB, molecular, and threshold profiles are shown together in Fig. 3a. PAB error bars are not shown here for image clarity. Of the 733 range bins from r_b to 25.0 km AGL, 232 of them satisfy the condition

View Expanded

where r_b corresponds to the first range bin height above r_N. Equation (18) requires that any PAB value, minus its relative uncertainty, be greater than the threshold such that there is no ambiguity between what is or is not indicative of clear sky. Those points satisfying Eq. (18) in the 10 June 2003 case are shown in Fig. 3b.

Each bin satisfying Eq. (18) is next considered independently, working upward, for the possibility that it represents a particulate base height. Running averages for PAB and δPAB are initiated, based at a qualifying bin r₁, using applicable forms of Eqs. (4), (5), and (6), such that a lower boundary is considered found at r₁ when

View Expanded

where the prime denotes those bins within this range that satisfy Eq. (18). Bins that do not satisfy Eq. (18) are ignored in this step. Including such bins in the averages would most likely act to lower the relative uncertainty of the average, which makes no sense because the goal [similar to the logic in Eq. (10)] is to potentially reach a value (r₁:r_y)/δ(r₁:r_y) that exceeds the threshold ϕ. However, bin r₁ is eliminated from consideration as a base height in the event that

View Expanded

for any consecutive bins encountered before reaching r_y that do not satisfy Eq. (18) (denoted by the asterisk). As such, bins not satisfying Eq. (18) are given a separate burden from which to establish the presence of clear sky and disqualify a bin r₁ from the base-height search. Once a base height is found, the search is continued from r_y to find the corresponding layer top. Either of two criteria may be met for this to occur: Eq. (20) may be satisfied to signify a clear-sky layer and, thus, a layer top height assigned to the last bin satisfying Eq. (18); or, as the running averages for Eq. (19) are continued, a top height is considered found if the value in Eq. (19) drops below ϕ.

Layer heights derived at the base resolution for the case on 10 June 2003 are shown in Fig. 3c, using values of 100.0 for ϕ and 10.0 for κ (discussed further in the following section). Some caveats must be addressed. First, no matter what the settings are for ϕ and κ, there is the possibility that optically/geometrically thin layers will go undetected with this technique. This may be exacerbated in the case for a low-resolution instrument setting at the point of data retrieval. The South Pole MPL is nominally set to 0.030-km resolution. However, other MPLNET sites frequently use a 0.075-km setting. Second, the objective threshold in Eq. (17) is not adjusted for transmission. For the single-channel instrument, uncertainty in the type of layer detected (i.e., aerosol, liquid water, or ice) inhibits the calculation of an optical depth by way of an extinction-to-backscatter ratio, and a subsequent adjustment of the threshold. Therefore, for multilayer cloud scenes, the algorithm is attenuation limited. Similarly, this also affects the detection of the top height for a given layer. There is no means for determining whether or not the actual top height is ever reached or whether the profile has become noise saturated. A threshold could be designed using SNR to interpret bins above a layer top height to be indicative of useable signal. However, this is beyond the scope of this work.

For the PAB profile in Fig. 3a (beginning from r_b) an attenuated scattering ratio (ASR; Measures 1984) can be defined as

View Expanded

where β(r) and T ²(r) represent total backscatter and transmission, respectively, and the prime denotes an attenuation-limited parameter. Equation (21) is only solved for bins within cloud layers identified by the algorithm. All other regions may be set to 1.0. As implied above, PAB within the boundaries of a defined layer may fall below the threshold in Eq. (17). ASR for those bins may be smoothed over using an interpolative technique. For the results shown here, we use a one-dimensional form of the Hanning window (Blackman and Tukey 1959) with no temporal coordinate and 0.150-km spatial half-width.

From Eqs. (17) and (18), an unattenuated minimum detectable scattering ratio can be defined as

View Expanded

An example for Eq. (22) is shown in Fig. 4 for the 10 June 2003 case. The signal threshold and attenuated molecular scattering profile are shown in Fig. 4a, the minimum detectable scattering ratio and smoothed ASR profile in Fig. 4b, and the unsmoothed ASR profile in Fig. 4c. At the base resolution during polar night, the MPL is sensitive to particulate layers with a scattering ratio under 2.0 up to near 16.0 km AGL, and approaching 10.0 to near 25.0 km AGL.

A composite example of these data is shown in Fig. 5. Level 1.0 MPLNET data from the South Pole for 1–10 June 2003 are displayed (Fig. 5a) along with corresponding fractional uncertainties (Fig. 5b), each at the 1-min shot system resolution. Algorithm results derived at the 0.01 fractional day base resolution are shown in Fig. 6a, along with corresponding minimum detectable scattering ratios (Fig. 6b). Gaps reflect either missing data or periods of low-cloud presence where the normalization step was not possible, C*_f was not solved, and the data were not analyzed.

Background measurements at the South Pole during the polar night are extremely low [approximately 1.0 × 10⁻⁴ PhE μs⁻¹; photoelectrons (PhE)]. This leads to a low relative signal uncertainty in Eq. (17), and a subsequently low objective threshold. Shown in Fig. 7 are the minimum detectable scattering ratios for four additional scenarios relative to nominal South Pole conditions: 1.000, 0.100 (Figs. 7a,b; 1.0–20.0), 0.010, and 0.001 PhE μs⁻¹ (Figs. 7c,d; 1.0–10.0). These data were derived by modeling an MPL signal profile using the standard atmosphere molecular scattering profile discussed above, an instrument output energy value of 5.0 μJ, and a calibration factor C of 20.0 [Eq. (1)]. The latter two settings approximate those values encountered with the South Pole instrument in 2003. The algorithm was run from 3.0 km AGL using the tunable threshold settings prescribed above. The data are plotted versus the natural logarithm of the temporal averaging period. Therefore, the value 10⁻³ approximates the 1-min resolution, which equals the sampling resolution of the instrument data system. A solid line is used to delineate regions of the profile where the algorithm derived no data. That is, based on Eqs. (9) and (10), for noisier signal averages the minimum number of bins required to calibrate the profile is increased. In each case, some of the column is lost for analysis to this step. The worst case corresponds with the highest background at 1-min resolution (Fig. 7a).

At the base resolution used above, 0.01 fractional days, algorithm performance is fair (β_{r min} < 10.0) for all scenarios below 15.0 km AGL, except in the highest solar background case (Fig. 7a; analogous to tropical and subtropical conditions, based on experience). Performance for the other cases is compromised only at the highest levels at this temporal setting. Multitemporal resolution is discussed further below. However, performance at 1-min resolution is poor for the two highest background scenarios (β_{r min} < 20.0; Figs. 7a,b). When averaging is extended to one-tenth of a day, performance is naturally improved. Note that all of these results are greatly improved upon when the instrument calibration factor reaches a second order of magnitude and/or the output energy of the instrument is increased (not shown). It is not uncommon to find MPLNET instruments where the calibration factor reaches above 100.0, or where the output laser energy per pulse approaches 10.0 μJ (T. A. Berkoff 2007, personal communication).

3. Threshold testing

In this section, tests are described where both ϕ and κ were varied to compare algorithm output and describe the influence of each on particulate layer height retrievals. Here, ɛ was kept at a constant 100.0 for these tests. Although its variance would obviously induce some consequence to the final results, its influence is confined mostly to the calibration steps and not those affecting the layer search. Because Eqs. (13), (19), and (20) relate to the signal averages and their relative uncertainty, each tunable threshold has some measure of statistical relevance. That is, considering Eqs. (10) and (13), N bins are required, such that (r:r_N)/δ(r:r_N) exceeds ɛ. For a value of ɛ set to 100.0 or greater, C*_f is accurate to within 1%. The thresholds ϕ and κ are not applied necessarily in the same direct manner. That is, the running averages in Eqs. (19) and (20) do not always reflect those for consecutive bins [depending on Eq. (18)]. However, the goal is still to find a value (r₁:r_y)/δ(r₁:r_y) that exceeds ϕ so as to differentiate particulate scattering from the clear sky, and vice versa, for (r₁:r_y)/δ(r₁:r_y) and κ.

From Eq. (19), ϕ is most influential for the detection of a layer base height. Two tests were run with ϕ set at 200.0 and 50.0, a factor of 2 greater than and less than the value used in the examples above. For these runs, κ was kept constant at 10.0. The results are shown in Figs. 8a,b, respectively, for 1–10 June 2003, and may be compared directly with Fig. 6a (ϕ = 100.0). Setting ϕ to 200.0 results in fewer actual PSC detected relative to the lower setting. This is consistent, given that a greater number of bins satisfying Eq. (18) must be present before the conditions of Eq. (19) are met and a base height is considered to be found. This effect is pronounced for the weakly scattering PSC, and is best seen by comparing Figs. 6a and 8a for the optically thin elements observed on 8–9 June centered near 15.0 km AGL. Lowering ϕ to 50.0 (Fig. 8b) induces an opposite effect. More actual PSC are apparent, relative to Fig. 6a, because the restraint discussed in Eq. (18) is weakened. Weakly scattering elements are detected near and below 15.0 km AGL on 9–10 June, for example. As will be shown in the next section, retrievals using longer time iterations for these days depict a lower layer present to near 12.5 km AGL. However, numerous false positives occurring below 10.0 km AGL are also apparent, though their ASRs are near 1.0 and are difficult to see in the image. For this case, setting ϕ to 100.0 is a compromise between including as many clouds as possible in the sample and the goal of limiting false positives.

In Eq. (20), varying κ may influence the detection of a base height, because the detection of a clear slot can end the base height search in Eq. (19) before one is found. However, κ has a greater influence on the identification of a layer top height. Two tests were run where κ varied from 20.0 to 5.0, or a factor of 2 greater than and less than the value of 10.0 used above; ϕ was kept constant at 100.0. The results are shown in Figs. 9a,b, respectively, and may be compared directly to Fig. 6a. At a setting of 20.0 (Fig. 9a), two effects are seen. First, because Eq. (20) is not as easily surpassed in the low SNR regions at upper levels, the retrieved PSC layers exhibit more vertical depth. The dark vertical striping apparent in the image indicates cases where Eq. (19) was satisfied, but a corresponding layer top height was not found before the algorithm reached 25.0 km AGL (the prescribed ceiling for these retrievals). For such cases, the algorithm assumes the base height discovery to be erroneous, and no layer is recorded. Lowering κ to 5.0 produces an opposite effect. Fewer cloud layers are found because Eq. (20) is more easily satisfied during the layer base search. Layer tops are found at lower heights relative to κ values of 10.0 and 20.0. This finding is also consistent because of the influence of Eq. (20). Here, κ must be managed to consider the SNR properties of the signal relative to the need for retrieving as much of the cloud as is reasonable.

As discussed, it is not practical when processing many network datasets to have an excessive number of tuning thresholds in any algorithm. Therefore, an algorithm consisting of three constraints is reasonable. We have described practical values for ɛ, ϕ, and κ for PSC retrievals using the South Pole MPL operated in winter 2003 (100.0, 100.0, and 10.0). For other instruments and experimental goals these values will vary as a function of the relative difference in output power, optical system efficiency, and averaging period, because these parameters directly influence SNR performance. From testing the algorithm with other MPLNET instruments, for example, optimal ɛ values can reach as low as 75.0 at 1-min resolution from lower SNR. We are confident that each threshold may be varied in real time as a function of outgoing pulse energies and/or the system calibration constant in Eq. (1). Further testing and development of this aspect, however, is continuing. Still, we wish to stress that the settings used in this study may not always yield appropriate results, and may need to be optimized before being implemented.

4. Temporal retrieval variability

Producing datasets at multiple temporal resolutions offers users the choice for an optimal result that best meets the objectives of their study. The base temporal resolution is defined above as the minimum resolution that yields meaningful results. For PSC retrievals, 0.01 fractional days were used and acceptable performance was demonstrated. Depending on the type of cloud or aerosol being studied, an optimal resolution exists that produces the most efficient results. In the case of persistent features, such as PSC, this resolution actually may be longer than the setting chosen here. But, a practical means is also necessary for screening profiles that exhibit strong attenuation effects with range. Averaged profiles biased by attenuated signal structures limit the accuracy of the molecular normalization step in Eqs. (7) and (8). Profiles resulting from such averages cease to resemble actual lidar profiles under these conditions. To screen these data out of longer averages, the base-resolution results are used. Base-resolution profiles are grouped into longer averages depending on whether or not C*_f is solved, therefore providing an objective means for profile rejection. From Eqs. (6) and (17), the minimum detectable scattering ratio [Eq. (22)] lowers, in response, as a function of iteration length t.

ASR retrievals are shown in Fig. 10 at the base resolution and for 0.02 (∼28 min), 0.04 (∼57 min), and 0.10 (144 min) fractional day averages. Profiles are offset such that the ASR scale between successive retrievals varies linearly from 1.0 to 5.0. With decreasing temporal resolution, the retrievals show the PSC to have greater a vertical extent (∼12.5–25.0 km AGL) and to be one contiguous layer. By using lower-resolution averaging intervals, the minimum detectable scattering ratio drops because the PAB uncertainties used to create the screening threshold in Eq. (18) are lowered. At higher temporal resolutions, the retrievals are limited and clear slots are apparent in the retrievals. ASR values frequently peak over 5.0 for higher temporal resolution retrievals, whereas the longer-averaged retrievals fall mostly below this value. With longer averages, signal structure is gradually diluted. At the full 1-day temporal resolution from 10 June (not shown), the maximum ASR value peaks just over 2.0, despite the layer being mostly persistent visually (Fig. 6a). These results may appear confusing at first. However, they are an artifact of the signal becoming more distinguishable relative to uncertainty in longer averages, which, in turn, allows for more of the PSC to be retrieved with high confidence.

5. Conclusions

We have described a statistical method for analyzing micropulse lidar (MPL) data for elevated cloud and aerosol layer heights based on the signal uncertainties of the level 1.0 Micropulse Lidar Network (MPLNET) backscatter product. The algorithm is designed for high performance in the case of weak signal-to-noise ratio (SNR) profiles, and may be run at multiple temporal resolutions in order to yield an optimized product depending on the subject of study. We outline an automated method for solving the calibration constant in the lidar equation. However, this step limits use of this algorithm nearest the ground for zenith-viewing lidars because of the likelihood of it being solved above the planetary boundary layer, which may or may not contain clouds and aerosols and be subject to transmission losses. We propose that this work be paired with previous thresholding algorithms designed for MPL instruments so that cloud and aerosol retrievals for the entire vertical column are possible. However, stand-alone MPLNET data products may be designed based solely on the output of this algorithm, including retrievals optimized for optically thin clouds, such as polar stratospheric clouds and tropopause-level cirrus clouds.

The algorithm is predicated on three conjectures and uses one objective and three tunable thresholds. First, calibration is based on the assumption that vertical signal structures are persistent relative to that of the molecular clear sky. False positives may occur in the case where any particulate scattering present exhibits structure that parallels that of molecular scattering. Second, signal from one range bin is never considered to be significant on its own, and instead is analyzed relative to adjacent bins. This is designed to limit the effects of noise on a single bin. Last, calibrated backscatter signals are analyzed for particulate and clear-sky signals by relating their structure and uncertainties as superimposed onto a theoretical clear-sky profile. This step yields an objective threshold from which to analyze all calibrated signals. Any of the three tunable thresholds may be altered with some consequence, which may be then used to interpret the results. We have proposed reasonable values for each tunable threshold based on a study of polar stratospheric clouds (PSCs) at the South Pole in June 2003.

The algorithm is gradually limited for instruments exhibiting relatively low SNR performance. Lowering each tunable threshold may be necessary to best balance the desire to identify as much cloud and aerosol as possible relative to the negative influence of noise. Algorithm performance for multiple ambient background scenes is also addressed. Although the balance of our work is spent focusing on PSC retrievals during polar night, we model minimum detectable algorithm signals for varying background count rates over four orders of magnitude. This work shows that some temporal averaging may always be necessary to yield particulate layer retrievals above 15.0 km, depending on the optical efficiency and output energy of the instrument.

MPLNET serves as a unique test bed for lidar algorithm development because it represents an emerging global network with a common instrument design and source wavelength. The increasing visibility of lidar instruments and datasets, highlighted by recent satellite-based deployments (Winker et al. 2003; Spinhirne et al. 2005), offers researchers an increasing database for cloud and aerosol observations. Optimized techniques for identifying their presence, particularly in low SNR profiles, work to the benefit of the entire atmospheric science community. Although this algorithm has inherent limitations for zenith-pointing instruments, this would not be the case for nadir-pointing ones, such as on aircraft or satellite platforms. Under these conditions, aerosols and other particulate matter are unlikely to be present in the air mass directly adjacent to the lidar. The calibration step could be achieved without concern for particulate transmission losses and/or cloud and aerosol presence between the instrument and the region used to solve it. Therefore, layer retrievals could be performed over the full column.