1. Introduction
Understanding microphysical processes in clouds is key to understanding radiative feedbacks on Earth’s climate (Boucher et al. 2013). The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) central facility has been providing data to advance climate research since 1989 (Mather and Voyles 2013). At the ARM SGP central facility, a vertically pointed Doppler lidar has measured cloud microphysical and aerosol properties since 2011 (Newsom and Krishnamurthy 2022). In addition, four other Doppler lidars have been operating at auxiliary sites in the SGP since 2016. The Doppler lidar emits a laser beam at a 1548-nm wavelength and records the phase, amplitude, and frequency of the backscattered radiation. These phases and amplitudes are processed into Doppler power spectral density functions ρ(υ) as a function of Doppler velocity υ. For a vertically pointed Doppler lidar, ρ(υ) shows the distribution of vertical aerosol motion as well as fall speeds of precipitation. These ρ(υ) can be used to retrieve raindrop size distributions and fall velocities (Wei et al. 2021, 2019; Aoki et al. 2016; Träumner et al. 2010). Figure 1 shows an example of a ρ(υ) taken inside a precipitating cloud at 510 m above ground level at the ARM SGP site. The two peaks in ρ(υ), one at υ = −5 m s−1 and another near υ = 0 m s−1, probably correspond to falling raindrops and cloud droplets and aerosols, respectively. Such bimodal distributions have been observed in the past in Doppler lidar measurements of clouds, precipitation, and aerosols (Wei et al. 2021, 2019; Kalthoff 2013; Träumner et al. 2010; Lottman et al. 2001). Therefore, the ρ(υ) from the ARM Doppler lidar contains vital information about the distribution of clouds, aerosols, and precipitation that is needed for investigating cloud microphysical processes.
An example ρ(υ) from the ARM Doppler lidar sampled in a region of cloud particles and raindrops in a cloud during 4 Aug 2017 at 0055:18 UTC at 360 m above the ARM Doppler lidar.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
Two months of ρ(υ) from the Doppler lidar takes around 2.3 TB of hard drive space. Therefore, transferring and analyzing the complete Doppler spectra and their moments are not optimal. Because of this constraint, only the signal-to-noise ratio (SNR) and modal Doppler velocity from the ρ(υ) from the ARM Doppler lidar are made available to the end user. Since ρ(υ) can contain multiple peaks corresponding to different species present in the volume, not including the full ρ(υ) omits information on the distribution of the cloud and precipitation particle fall speeds. For example, in Fig. 1, the modal Doppler velocity of 0 m s−1 would only show the presence of a cloud. However, there is a second mode of ρ(υ) at approximately −5 m s−1 that shows the presence of rain. Therefore, having the full ρ(υ) instead of the modal Doppler velocity shows the correct picture of the spatial distribution of cloud and precipitation particles recorded by the ARM Doppler lidar, crucial for estimating rainfall rates and validating cloud-resolving model simulations.
The advent of edge computing in the past decade has provided new opportunities for processing data in real time at remote locations such as the ARM SGP site. In particular, the Argonne National Laboratory’s Waggle node, shown in Fig. 2, is an edge computing platform designed to perform lightweight machine learning and inference tasks using data collected at a field observation site. In particular, the need for improved cloud and rain particle fall speed data from the ARM Doppler lidar motivated the ARMing the Edge field experiment at the DOE ARM SGP site. During the ARMing the Edge experiment in 2021, a Waggle node was installed at the SGP site to both monitor cloud cover and use machine learning to harness measurements from other ARM instruments at the site. One goal of ARMing the Edge is to develop a processing pipeline, using the Waggle node, for both processing raw Doppler lidar data and identifying time periods where clouds and rain are present. From here, the Waggle node can send processed ρ(υ) to the ARM data center only during cloudy and rainy time periods and discard the complete spectra during clear-air periods.
The Waggle node installed at the ARM SGP site. The ground-facing camera and the Stevenson shield that houses the environmental sensors and microphone are clearly visible. On the top of the compute box is the sky-facing fish-eye camera (partially visible). The compute box hosts the NVIDIA NX compute module, power supply, and networking components.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
Given that a Waggle node is capable of running machine learning and deep learning models at the observation site, we were motivated to develop machine learning techniques for classifying Doppler lidar data and identifying features of interest at the edge. Before machine learning became popular, Doppler lidar classification algorithms typically looked at gradients and peaks of the SNR to determine the presence of clouds (Binietoglou et al. 2018; Newsom et al. 2019). Recently, machine learning algorithms have been used to classify lidar data that can potentially identify clouds and boundary layers in an automated manner. For example, Farhani et al. (2021) and Yang et al. (2021) used density-based spatial clustering of applications with noise (DBSCAN) to find regions of biomass burning, clouds, and aerosols in Doppler lidar data. Zeng et al. (2019) used fuzzy k-means clustering in order to distinguish aerosols from clouds in Cloud–Aerosol Lidar with Orthogonal Polarization (CALIOP) observations. Furthermore, random forest–based planetary boundary layer height (PBLH) retrievals have been developed for the ARM Doppler lidar at the ARM SGP site (Krishnamurthy et al. 2021). Artificial neural networks have been used to reconstruct Doppler lidar observations of liquid clouds above the maximum height of penetration of the lidar beam for Cloudnet liquid cloud retrievals (Kalesse-Los et al. 2022). Machine learning is hence quickly becoming a useful tool for analyzing Doppler lidar data.
Deep neural networks (DNNs), in particular, convolutional neural networks (CNNs), can automatically discern spatial characteristics of the data at multiple scales to describe the features of the dataset (LeCun et al. 2015). While DNNs can be computationally expensive to train and use for predictions, the Waggle node carries a Xavier NX graphical processing unit (GPU), which can vastly reduce this computational expense. CNNs also typically require at least thousands of images for adequate training. Given that the ARM Doppler lidar has recorded around a year of data and collects thousands of samples a day, there is an ample number of Doppler lidar data for training DNNs. These two unique aspects of the ARMing the Edge experiment provide a new opportunity to explore DNNs in addition to traditional machine learning algorithms for identifying time periods of interest in Doppler lidar data.
This paper provides a detailed overview of machine learning algorithms deployed on a Waggle node to identify clear, cloudy, and rainy time periods from raw ARM Doppler lidar data. First, section 2 provides an overview of the data and the preprocessing steps. Next, we develop various machine learning algorithms for the classifier in section 3. In section 4, we evaluate the performance of the machine learning algorithms. Section 5 provides a summary of the results.
2. Data and preprocessing
a. ARM Doppler lidar
At the ARM SGP site, there are five Doppler lidars, all manufactured by Halo Photonics. Four of the Doppler lidars, E32, E37, E39, and E41, form a rectangle about 50–60 km long in the north–south direction and 60–70 km in the east–west direction. In the center of this rectangle is a fifth Doppler lidar, C1, located at the ARM SGP central facility in Lamont, Oklahoma (36.6°N, 97.5°W). For this study, we use the C1 Doppler lidar to test the algorithm, since we have in-phase (I) and quadrature (Q) spectral data from this Doppler lidar available from August to October 2017. All of the Doppler lidars can store I/Q data if requested by the user, but they do not do so under normal operation. The C1 Doppler lidar operates at a wavelength λ of 1548 nm and pulse repetition frequency τ of 10 KHz and samples I/Q signals at a 50-MHz sampling rate. We utilized raw Doppler lidar data from August and September 2017 where I/Q data are available. The Doppler lidar conducted two scanning strategies during this period: vertically pointing stares and plan position indicator (PPI) scans at eight azimuth angles at 60° elevation. For training the machine learning models, vertical stares are used, as they provide a greater spatial resolution view of the clouds and precipitation than the PPI scans. We also utilized the autocorrelation functions that provide the I/Q components of the signal to ensure that any algorithm starts from the raw I/Q signal data to eliminate any preprocessing steps before the algorithm is run. Therefore, using raw I/Q signal data ensures that any delay between data retrieval and inference is minimized for real-time inference on the Waggle node.
b. Other observations
For this study, we utilized the rawinsonde data from August and September 2017 overlapping the ARM Doppler lidar data collected at the ARM SGP site. Rawinsondes were launched four times per day, providing vertical profiles of temperature, dewpoint temperature, and winds within 100 km of the site. Furthermore, a Vaisala, Inc., ceilometer provided additional measurement of the heights of the cloud base at the ARM SGP site (Morris 2016).
3. Machine learning algorithms
There are a multitude of machine learning algorithms that can potentially be used to identify scenes from ARM Doppler lidar data. This section introduces methods based on both supervised learning, which requires labeling of data before training, and unsupervised algorithms that can learn features from data without any a priori handcrafted knowledge inserted by humans.
a. Supervised learning
Machine learning algorithms can be trained using supervised, unsupervised, or reinforcement learning techniques. Supervised learning involves training by means of backpropagating errors that originate in the discrepancy between the label assigned to a sample and the prediction that the network produces by processing such a sample. The supervision can be done by regression or classification. Regression is the task of predicting a continuous quantity, while classification is the task of predicting a discrete class label. In this paper, we approach this problem as a classification problem. Basically, our algorithms attempt to learn how to classify samples from the data using discrete labels assigned by hand. Therefore, before testing the applicability of supervised learning algorithms, the ARM Doppler lidar dataset must be labeled into three different categories by hand: clear, cloudy, and rainy. This was done by examining each 5-min time–height cross section of SNR and Doppler velocity from 30 July to 30 September 2017, such as those in Fig. 3, and searching for cloudy layers shown by sharp (>1 dB km−1) vertical gradients in the range-corrected SNR in a stratified layer (i.e., Fig. 3b) and falling particles shown by range-corrected SNR > 10 dB and Doppler velocities < 1 m s−1 (i.e., Fig. 3c). In addition, these hand labels were confirmed against radar reflectivity from the Next Generation Weather Radar data from KOUN in Norman, Oklahoma (NOAA/NWS Radar Operations Center 1991), and images from the Geostationary Operational Environmental Satellite (GOES) data (GOES-R Algorithm Working Group and GOES-R Series Program 2017) as a second check to ensure that rain and clouds were present over the ARM Doppler lidar. If clouds were present for less than half of the 5-min time period, then the 5-min time period was classified as clear. Of the 11 805 five-minute time periods analyzed here, 67.2% (7938) were classified as clear, 28.2% (3339) as cloudy, and 4.3% (509) as rainy.
Example images of range-corrected SNR labeled as (a) clear, (b) cloudy, and (c) rainy. (d) As in (c), but for Doppler velocity.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
For training any supervised learning algorithm in this study, we first randomly chose 60% of the 5-min periods in the ARM Doppler lidar dataset for training. Then, another 20% of the 5-min periods that do not belong to the training dataset were chosen for the test dataset. Last, the remaining 20% of the 5-min periods that were not selected for testing or training were chosen for the validation dataset. The same three datasets were used to test, train, and validate all supervised learning models for the remainder of the article. The class imbalance shown here can potentially introduce training biases. Therefore, in an attempt to mitigate this bias, two other versions of the training data were generated: one by oversampling the cloudy time periods and randomly drawing replacement samples (as many samples as there were clear-air time periods), and the other by undersampling the clear time periods to match the number of cloudy time periods in the same manner.
Hand-labeled images are prone to human bias and ambiguity about how to label situations where isolated clouds are present. Therefore, it is important to assess the accuracy of these hand labels, as it affects model prediction accuracy. However, finding a ground truth for assessing hand-label accuracy is difficult, as there are three different candidate measurements at the SGP site that are sensitive to different types of clouds and measure different sample volumes: the ceilometer, the total sky imager (TSI), and the micropulse lidar (MPL). This can cause the total amounts of the cloudy time periods to vary by up to 15% (Wagner and Kleiss 2016), with the ceilometer less sensitive to clouds with bases higher than 3.66 km. Therefore, any accuracy calculation will be an estimate. To estimate the accuracy of the hand labeling, we compared the hand labels with cloud detections from the ceilometer. Since the ceilometer determines whether a cloud is present at a 15-s resolution, for this comparison, a time period is considered cloudy by the ceilometer if clouds with bases below 3.66 km are detected for at least 2.5 min of the 5-min time period. The ceilometer shows clear conditions for 93.5% of the hand-labeled clear time periods and cloudy conditions for 83.5% of the hand-labeled cloudy periods. The total number of clear periods from hand labeling and the ceilometer differs by 2.0% and the number of cloudy periods by 4.0%. All these differences are within those seen in previous studies comparing automated cloud cover techniques using ARM SGP data (Wagner and Kleiss 2016). Therefore, while the hand labels are prone to human error and bias, these comparisons demonstrate that there is reasonable agreement between the hand labels and cloud detections from the ceilometer.
1) Gradient-boosted trees
To train a GBT, the ARM Doppler lidar images need to be reduced to a number of features that will enable the GBT to be trained at speeds reasonable for analysis. We reduced the dimensionality by grouping the ARM Doppler lidar SNR observations into 5-min intervals. The observations were then reduced to a statistical coverage product P>x(z), used previously for radar data by May and Lane (2009), by calculating the percentage of the 5-min time period where δ at a given height level is greater than threshold x. P>0.1dB(z), P>0.5dB(z), P>1dB(z), P>2dB(z), and P>5dB(z) were all calculated for each 5-min time period.
There are many different combinations of P>x(z), listed in Table 1, that can be used as input features to the GBT. We determined the best possible combination to use by training nine candidate models using the different combinations as input features for the GBT. When training each model, early stopping was used to avoid overfitting. The average testing and training loss is computed from a fivefold cross validation of the Doppler lidar dataset for each boosting iteration. Training is stopped when the testing loss does not show a further decrease for 100 iterations, since this indicates that further training will cause the GBT to be overfitted on the training data. Finally, the GBT model has numerous hyperparameters that need to be chosen in order to construct the model, including the maximum tree depth and the percentage of the dataset to subsample for training. To find the optimal values for these parameters, an automated grid search was performed to find hyperparameters that maximize the fivefold cross-validation mean validation accuracy using Hyperopt (Bergstra et al. 2013).
Accuracy, specificity, and sensitivity of the GBT-based classifier in the validation dataset.
After each of these models is trained, they are evaluated on the validation dataset. Tables 1 and 2 show the specificity, sensitivity, and accuracy of the GBT model for each of the trained models when we assume that a negative result is a clear sky and a positive result is a cloudy or rainy time period. Table 1 shows that the accuracy of all models ranges from 73%– to 77% for the validation dataset. The sensitivity values range from 92% to 98% for any given set of Px(z) in Table 1. However, the specificity ranges from 24%– to 43% in Table 1. This demonstrates that the GBT-based model is more prone to classifying some cloudy and rainy events as clear. For the training dataset in Table 2, the accuracy ranges from 77.0% to 91.3%, the sensitivity from 96.8% to 99.7%, and the specificity from 29.0% to 66.5%. In general, this shows that, regardless of the choice of input parameters, the GBT model will likely underestimate the number of cloudy and rainy time periods while generally classifying all clear time periods as clear. In addition, training these models with the over- and undersampled versions of the training data to mitigate biases due to imbalanced data did not improve these results (not shown). The high-sensitivity and low-specificity values for the training dataset in Table 2 would indicate that this model is likely to overfit the clear-air time periods. Therefore, no matter the choice of parameters used, this guarantees that the algorithm will likely provide a truly cloudy time period to the user when it identifies one at the cost of excluding some cloudy periods. Therefore, we use P>1dB, P>2dB(z), and P>5dB(z) as input features for the GBT-based model.
2) Convolutional neural networks
In the previous section, we determined the best way to reduce the dimensionality of the raw input data to feed the classifiers. That is the classical machine learning approach, in which humans judge and choose the most suitable options in order to obtain the most representative set of features to train the classifiers. Another way to reduce the dimensionality and get suitable features from the image data is by using deep learning. Deep learning came to automate the process of feature extractions by connecting it to the loss functions on the final layers of the classifiers. In this way, human judgements do not determine the best features to be extracted from the inputs. Such features are rather automatically extracted following the premises imposed by the loss functions at the end of the pipeline.
Specifically, we use convolutions of an image to represent the features of the image as inputs to a neural network. These types of networks are commonly known as CNNs. CNNs can be configured in a multitude of ways to capture features in the image at different scales. To begin with, there are many published CNNs already trained on image data that can be adapted for use on the Doppler lidar data. Two commonly used CNNs are VGG19 (Simonyan and Zisserman 2015) and ResNet50 (He et al. 2015). VGG19 is a CNN that is 19 layers deep, with the number of channels used to describe the image in each layer increasing from 64 to 512 as the depth increases. The depth of VGG19 allows it to extract a wide range of features at varying spatial scales. VGG19 uses a rectified linear unit (ReLU) activation function (Agarap 2019) in its convolutional layers and a softmax activation function in its three fully connected layers. ResNet50 is a residual deep neural network that is 50 layers deep but designed to be easier to train than VGG19, as it has skip connections between residual convolutional layers that enable the backpropagation of the gradient through fewer layers during training. This mitigates the vanishing gradient problem, where the gradient of the loss function becomes infinitesimally small as it backpropogates through the network, preventing neural network training convergence (Bengio et al. 1994). In addition, ResNet50 has fewer filters than VGG19 and a larger kernel size (7 × 7) than VGG19 (3 × 3), making it less complex and therefore faster than VGG19. As shown in Fig. 4, these CNNs extract the image features and then feed their output into three fully connected layers that classify the image.
Illustration of how pretrained CNNs are used to classify Doppler lidar data. The 96 × 128 image is an input to a feature extraction layer that is either VGG19 or ResNet50 with ImageNet weights. The feature extraction layer then has a three-layer-deep, 512-node-wide fully connected layer using the ReLU activation function. The outputs of the three-layer block are inputs to a three-node layer with softmax activation functions to give the class.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
To utilize these CNNs for developing an image classifier for the ARM Doppler lidar data, we used a process called transfer learning, in which a model developed to recognize image features is used as a starting point for ARM Doppler lidar image classification. To utilize these pretrained models, the predetermined weights from VGG19 and ResNet50 are used to construct the CNN shown in Fig. 4. When training these CNNs, only the bottom layer of VGG19 and ResNet50 and the weights in the fully connected layers are allowed to change over each epoch. For training, the Adam optimizer (Kingma and Ba 2017) was used to minimize the binary cross-entropy loss function (Zhang and Sabuncu 2018). Images of 5-min δ from the Doppler lidar are used as inputs to the CNN. In addition, we tested these models with 5-min images of a range-corrected SNR and found less than 2% change in the testing accuracy. Therefore, either a range-corrected SNR or δ can be used for these models. The input dataset is divided into testing, training, and validation in the same manner as used for training the GBT-based model. The CNNs are trained until the accuracy of the predictions over the testing dataset has not increased for 100 epochs. Training the CNN classifier while allowing the weights of the last to the bottom-most four layers of either ResNet50 or VGG19 to vary showed similar results to those presented here (not shown). All CNN-based models were trained using TensorFlow 2.4 (Abadi et al. 2016).
b. Unsupervised learning
The previous section demonstrated how to leverage supervised learning for classification of ARM Doppler lidar images. One limitation of supervised learning with hand labels is that there is potential for human error and subjectivity in hand labeling. Another limitation of hand labeling is that it is extremely labor intensive and impractical to perform on multimonth datasets. However, unsupervised learning attempts to classify these images without the need for hand labeling. A successful unsupervised learning technique helps to demonstrate that, even without labor-intensive and subjective hand labeling, automated identification of such images is possible. Therefore, this section explores the applicability of unsupervised learning algorithms in classifying ARM Doppler lidar images.
The images were first scaled to be 96 pixels wide and 128 pixels long. However, this results in 12 288-dimensional inputs, which is too large to be used by clustering techniques. The 96 × 128 pixel images used in this dataset, therefore, need to be reduced before being utilized in unsupervised learning algorithms. One way to do this is to train a convolutional autoencoder that will perform a series of convolutions of the image to extract its features (Chen et al. 2021). Figure 5 shows a diagram of a CNN encoder–decoder structure that is designed to extract such features. The features are then encoded into a latent space before they are again fed into a convolutional network that decodes the encoding back into the original image. The end goal is to build the encoder on the left side of Fig. 5, but since the outputs of the encoder are not known a priori, the entire encoder–decoder network must be trained at once. For this training process, the image is both the input and the output. It was found that training the network in Fig. 5 to 1100 epochs produced an autoencoder that was able to reconstruct the 5-min images of δ from the ARM Doppler lidar. The encodings are 24 dimensional, ideal for clustering algorithms.
An illustration of the convolutional autoencoder. The boxes show the layer in the Keras backend of TensorFlow that is used, followed by the number of nodes, the Keras pooling technique, and the activation function of each layer.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
The next step is to organize the encodings in the latent-space phase space into clusters. To do so, we performed k-means clustering on the encodings. k-means clustering, given a prescribed number of clusters, will assign a label to the cluster whose mean is closest in distance to the point. Then, it will repeat this process iteratively until the clusters do not change. One limitation of k-means clustering is that the number of clusters must be chosen beforehand. The number of clusters can be chosen either based on the number of expected clusters from theoretical considerations or through automated objective techniques. Given that there is no prior knowledge of how many clusters to expect in our dataset, we chose to use an objective technique called the elbow method (Bholowalia and Kumar 2014). The elbow method works by performing the k-means clustering algorithm while examining how the sum of squared errors (SSE), or inertia, decreases as the number of clusters increases. The number of clusters is determined by looking for a sharp inflection point in Fig. 6, called the “elbow.” However, for this particular case, the curve in Fig. 6 is smooth, and therefore, the “elbow” is less obvious. Figure 6 shows that this “elbow” could be anywhere between 6 and 10 clusters. When 10 clusters were used, section 4c demonstrates that the algorithm was able to sort cloudy time periods by cloud-base height. When analyzing the data when six clusters were used, the clusters no longer sorted cloudy time periods by cloud-base height, showing that important information is lost by having too few clusters. However, after 10 clusters, the inertia changes by less than 10% as the number of clusters increases. Therefore, little information is likely to be gained by sorting the data into more than 10 clusters. Therefore, we sorted the 24-dimensional encodings into 10 different clusters.
The training inertia for the k-means clustering method applied over the 24-dimensional encodings generated by the autoencoder as a function of the number of clusters.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
4. Results
a. Evaluation of supervised learning algorithms
Three different types of supervised learning algorithm candidates have been developed. To select the best candidate for use on the Waggle node, the performance of the algorithms must be evaluated. To evaluate the performance of the GBT-, VGG19-, and ResNet-based algorithms, Fig. 7 shows confusion matrices comparing the algorithm classifications with the hand labels for the 20% of the dataset set aside for validation. Misclassifying a rainy event as cloudy and vice versa would have no effect on the Waggle node’s decision on whether to include full Doppler spectra for the time period. Also, attempting to train the VGG19- and ResNet-based models with all three classes prevented these models from classifying all three classes. Therefore, we further grouped rainy and cloudy time periods into one class for these comparisons. Since only 3.2% of the time periods are classified as rainy, grouping cloudy and rainy time periods together helps mitigate training biases due to an imbalanced distribution of classes, which likely prevented the convergence of the three-class model training. Two separate versions of these models trained on the over- and undersampled versions of the training data were evaluated against the validation dataset. However, these attempts did not improve the validation accuracy, sensitivity, or specificity for any of the models (not shown). In addition, we compare these results with classifying time periods as cloudy, with more than 50% of vertical profiles with an identified peak in the range-corrected SNR, as detected by the algorithm used by Newsom et al. (2019). For this method, the entire dataset is shown in Fig. 7, as no training data are used to tune this algorithm.
Confusion matrix comparing model-predicted and true classifications for the (a) GBT-based model, (b) VGG19-based model, and (c) ResNet50-based model and (d) using the cloud-detection methodology of Newsom et al. (2019). The comparison over the validation dataset is shown in (a)–(c), and the comparison over the whole dataset is shown in (d).
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
In Fig. 7a, clear time periods are identified correctly by the GBT-based algorithm for 1459 (91.7%) of 1590 time periods. For cloudy and rainy time periods, the GBT-based algorithm identifies 317 (41.5%) of 764 cloudy time periods correctly, showing demonstrably less skill in identifying cloudy time periods. Overall, this shows that the GBT-based algorithm is not an ideal candidate algorithm for identifying cloudy time periods in lidar data.
When comparing the results of the GBT-based algorithm with those from the VGG19- and ResNet50-based algorithms, it is clear that the ResNet50-based algorithm is outperforming both the GBT- and VGG19-based algorithms. In Fig. 7b, the VGG19 algorithm does not identify any cloudy or rainy cases, while ResNet50 identifies 97.6% of clear cases and 94.7% of cloudy cases correctly. This is likely due to the fact that VGG19 is prone to the vanishing gradient problem, as the gradient of the loss function must backpropagate through 19 layers during training (Bengio et al. 1994; Basodi et al. 2020). The testing loss after training the VGG19-based neural network for 50 epochs remains constant, suggesting that the gradient of the loss function is approaching zero. ResNet50 includes skip connections between residual layers in the neural network to remedy this problem, and therefore, the gradient can backpropagate through fewer layers, allowing convergence. ResNet50 also vastly outperforms the technique of Newsom et al. (2019) in identifying cloudy cases. While the Newsom et al. (2019) technique correctly identifies 96.7% of the clear-air periods, it misclassifies 39.2% of the cloudy time periods as clear. Therefore, the ResNet50-based algorithm is the most capable algorithm here to distinguish between clear and cloudy cases. In addition, since the machine learning–based models are trained before they are installed on the Waggle node, inference of the cloud cover from an image takes under 1 s on the Waggle node for any of these algorithms. Therefore, from both speed and accuracy standpoints, the ResNet50-based model is superior to the other candidates presented here and therefore used on the Waggle node to classify clear and cloudy time periods for ARM Doppler lidar data processing.
b. Examination of clusters from unsupervised learning
To give a qualitative overview of the images in each cluster, Fig. 8 shows the mean range-corrected SNR as a function of range and time of day for each cluster. In clusters 1 and 7, range-corrected SNR values greater than 15 dB are only observed during the afternoon hours, with scattered returns throughout all height levels. This indicates that cloudiness is likely more scattered and more common in the afternoon hours for these clusters. In clusters 3, 5, 8, 9, and 10, values greater than 15 dB are found within 1.5 km throughout the day, indicating more persistent cloudiness in these clusters. Finally, in clusters 4 and 6, range-corrected SNR values greater than 15 dB are generally present in the lowest 1.5 km, indicating that clouds and rain are likely most present in these clusters. Therefore, from a quick examination of mean range-corrected SNR values from each cluster, it appears that the clustering technique is capable of clustering clouds with differing base heights into clusters as well as identifying when there are cloudy conditions.
The mean range-corrected SNR as a function of height and time of day of all nonnoise (SNR > 1 dB) returns for the given cluster.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
Figure 9 shows the frequency distribution of the hand labels in each cluster. In clusters 1, 2, and 7, at least 80% of the time periods in these clusters were identified as clear. Therefore, these clusters likely correspond to clear-air conditions. In clusters 3, 4, 5, 6, 8, 9, and 10, more than 80% of the time periods were identified as cloudy or rainy. Therefore, these clusters likely correspond to time periods with clouds and rain. This shows that the unsupervised clustering technique is capable of identifying time periods with cloudy conditions and time periods with clear conditions. However, given that none of the clusters has more than 94% of the clusters as clear, cloudy, or rainy, this shows that the ResNet50-based algorithm is superior in distinguishing between clear and cloudy time periods when compared with the unsupervised learning algorithm. However, given the 80% or higher frequencies (or 20% or lower) of clear conditions in each cluster, the unsupervised learning technique is capable of categorizing clear and cloudy cases into distinct clusters with better skill than the VGG19-based method. However, the Newsom et al. (2019) and GBT-based algorithms are more able to classify clear cases than unsupervised learning, since they correctly detect 96% of clear cases. However, these two methods are less able to categorize cloudy cases than the unsupervised learning algorithm, since those two methods can only categorize 20%–45% of them correctly, less than the percentage of cloudy and rainy cases per cloud-dominated cluster in Fig. 9.
The distribution of hand labels for the automatically inferred clusters.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
c. Analysis of cloud-dominated clusters
The distribution of hand labels in the given clusters showed that 3, 4, 6, 7, 8, 9, and 10 were cloud dominated. To quantitatively demonstrate the spatial extent of the clouds in each cluster, Fig. 10 shows frequency histograms of the range-corrected SNR sorted by height. At altitudes less than 1 km, there is a persistent peak of range-corrected SNR values less than 10 dB in all clusters in Fig. 10. Since this peak persists throughout all clusters, it probably corresponds to aerosols. Looking at heights greater than 1 km, there are peaks in the frequency distribution of range-corrected SNRs > 10 dB at 1.5 km in cluster 8, 2 km in cluster 3, and 2.5 km in cluster 10. This shows that the clustering algorithm is identifying time periods with peaks in the frequency distribution of SNRs > 10 dB, which can correspond to groups of clouds with similar cloud-base heights.
Frequency distribution of the range-corrected SNR as a function of height for each cluster. Numbers indicate percentage of 5-min time periods identified as belonging to the cluster.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
To check if these peaks in the range-corrected SNR distribution correspond to groups of clouds with similar cloud-base heights, the normalized frequency distribution of cloud bases from the ceilometer is shown in Fig. 11 for each cluster. In cluster 3, the peak frequency of cloud bases is at 1.7 km. For cluster 4, this peak is at 1 km, consistent with low-based clouds that are likely to produce rain. For cluster 8, the peak frequency of cloud bases is at 1.7 km; it is at 4.5 km for cluster 9 and 2–3 km for cluster 10. Therefore, given the close proximity of the peaks in the SNR distribution in Fig. 10 to the peaks seen in Fig. 11, it is likely that these peaks correspond to clouds with similar cloud-base heights. For the cloud-dominated clusters, the ceilometer found a cloud base at least 78% of the time in clusters 3, 4, 5, 7, 9, and 10 and as much as 98% of the time in cluster 6. Clusters 5 and 9 had peaks at ∼4 km in altitude, which is consistent with observations of low- and midlevel (bimodal) cloud-base heights over the SGP using ceilometers and cloudy images from ground-based camera (Dematties et al. 2023). This further shows that unsupervised learning using CNNs and k-means clustering is capable of finding time periods that are predominantly cloudy in the lidar data. The ceilometer cloud-base observations being above 1 km also demonstrates that the maximum SNR frequency of SNRs < 10 dB at less than 1 km is likely due to dust and aerosol in the planetary boundary layer. In addition, the retrieved cloud-base heights from the ARM Doppler lidar using the Newsom et al. (2019) technique are shown in Fig. 11. In clusters 4 and 6, there are more than 6763 higher cloud-base height values retrieved below 1 km from the ceilometer than the Newsom et al. (2019) technique from the ARM Doppler lidar. Both of these clusters have the highest number of rainy time periods, so this suggests that rainfall obscures the peak and vertical gradients in SNR in the ARM Doppler lidar imagery, preventing peak detection during rainy time periods. Therefore, this shows that the advantage of both the ResNet50- and autoencoder-based approaches is that they can detect events with clouds during rainy conditions.
Frequency distribution of the cloud-base height from the ceilometer (blue) and retrieved using the cloud-detection technique of Newsom et al. (2019) (red) for each cluster. Numbers indicate percentage of time periods in the cluster with no cloud base identified by the ceilometer.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
To provide even more evidence that the clustering algorithm is identifying time periods with groups of clouds with similar cloud-base heights, the average profiles of temperature, dewpoint, and horizontal winds for each cluster are shown in skew T diagrams in Fig. 12. In cluster 3, air is saturated with respect to water at 875 hPa. In cluster 4, the air is saturated from the surface to 850 hPa, indicating a near-surface environment where clouds and rain are likely to persist. In cluster 6, this saturated layer extends from the surface to 800 hPa. In cluster 8, the air at 850 hPa is saturated with respect to water, and it is so at 750 hPa in cluster 10. These saturated air layers in the mean vertical profiles of temperature and dewpoint at different pressure levels further support the notion that the unsupervised clustering technique is capable of identifying time periods with groups of clouds with similar cloud-base heights.
Skew T plot of the average temperature and dewpoint from the rawinsonde for each cluster. The temperature inversion is indicated in the title of each plot.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
d. Analysis of clear-air clusters
The previous section used the ARM ceilometer and rawinsonde observations to further describe the characteristics of the cloud-dominated clusters. This section looks again at rawinsonde observations, time of occurrence, and ARM Doppler lidar–retrieved PBL heights in order to provide physical interpretations of the other clusters. The characteristics of the PBL are influenced by insolation heating of the surface, creating buoyant motions that mix the boundary layer in order to create a vertical profile of temperature that is dry adiabatic. During the overnight hours, radiative cooling causes the surface to cool, and hence, the warm less dense air rises over the surface. This causes the temperature to increase with height during the overnight hours within the lowest 2 km of the atmosphere. This also influences the height of the near-surface aerosol layer, making it higher during the afternoon as it is lifted and lower during the overnight hours. This motivated the creation of Fig. 13 to examine whether the clustering algorithm is capable of identifying characteristics of the PBL through the near-surface aerosol signature. In addition, we also present the frequency distribution of the retrieved PBLH from the ARM Doppler lidar value-added product in each clear-air-dominated cluster in Fig. 14.
Frequency distribution of hours of day in central daylight time of each time period for the given clear-air-dominated cluster.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
Frequency distribution of PBLH retrieved from the ARM Doppler lidar for each cluster. The numbers indicate the percentages of PBLH that are less than 150 m.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-22-0062.1
In Fig. 13, cluster 2 is preferentially observed during the nighttime hours. In addition, cluster 2 occurs mainly from 1800 local time to 0700 local time in Fig. 13. Furthermore, there is a strong (4.0°C+) inversion near the surface that is evident in Fig. 12 for clusters 2 and 6. Furthermore, in Fig. 14, the Doppler lidar–retrieved PBLH is lower than 150 m for 66% or more of the time. Therefore, this indicates shallow boundary layers, which are common during nighttime hours in the continental United States (Seidel et al. 2012; Zhang et al. 2020). The presence of shallow boundary layers formed by radiative cooling–induced subsidence in clear sky conditions is then likely the identifying characteristic of cluster 2. The cloud-base heights, when clouds are present, are primarily greater than 3.5 km, indicating mostly scattered mid- to high-level clouds.
In Fig. 13, clusters 1 and 7 have no preferential time of occurrence. Furthermore, the distributions of PBLH in Fig. 14 show a peak in values lower than 500 m in cluster 2, but there is no peak in the distribution for clusters 1 and 7. This therefore shows that the PBLH is widely variable in the time periods labeled as clusters 1 and 7. Therefore, the clustering is unable to discern the location of the boundary layer, and given the 54% or greater retrieved values of PBLH lower than 150 m, it is limited in its ability to sort out cases with nocturnal inversions.
Given that in clear sky conditions, PBLH retrievals have used gradients of the SNR in Doppler lidar data (Demoz et al. 2006; Sawyer and Li 2013; Su et al. 2020), it is not surprising that the clustering is capable of clustering some of the times in nocturnal fair weather conditions together. Random forests have already been able to retrieve the PBLH from the ARM Doppler lidar (Krishnamurthy et al. 2021). Additionally, Yang et al. (2021) have used DBSCAN and random forests to identify aerosols in Doppler lidar data in Iceland. However, a limitation of this clustering technique is that there is no clear preferential clustering of PBLH to a specific value for a given cluster in Fig. 14, and two of the three clusters do not have any time preference. Therefore, while the clustering algorithm is able to identify some of the time periods when the planetary boundary layer decoupled with the surface at night, its capability in examining the diurnal cycle of the PBL is nonexistent. Therefore, whether deep learning can be used to retrieve PBLH and whether it can improve these current techniques remain open questions for future research.
5. Conclusions
The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) facility has operated a Doppler lidar at the ARM Southern Great Plains (SGP) site in order to profile the boundary layer winds, turbulence, and cloud properties. These microphysical properties are crucial for advancing our understanding of cloud and aerosol processes. For example, the Doppler spectra contain information about the types of hydrometeors that are present by showing how the intensity of scattered radiation varies with the Doppler velocity. Bimodal distributions similar to those shown in Fig. 1 show a clear separation between aerosol-induced and precipitation-induced modes. Similar distributions are routinely observed in vertically staring Doppler spectra at all sites when precipitation is present.
However, because of the large numbers of data produced by the ARM Doppler lidar, it has not been practical to collect and store the raw spectra. During ARMing the Edge, the Argonne National Laboratory’s Waggle node, capable of performing ML training and inference on data from remotely deployed instruments, provided a strong motivation for the development of algorithms optimized for edge computing. Several ML algorithms were developed to select time periods of interest for which these Doppler spectra would be most useful for further analysis and deployed on the Waggle node at the ARM SGP site. These ML algorithms capable of running on the Waggle node that can identify clear, cloudy, and rainy time periods in ARM Doppler lidar data were evaluated against hand labeling.
To develop this algorithm, we explored both supervised and unsupervised learning to train the algorithm. We tested both gradient-boosting trees and convolutional neural networks for identifying ARM Doppler lidar images by the presence of clouds and rain. We found that using a ResNet50-based classifier trained on a hand-labeled dataset classified 97.6% of the clear-air images and 94.7% of cloudy images correctly, outperforming gradient-boosting trees and traditional peak detection algorithms. Unsupervised learning techniques that combined convolutional autoencoders with k-means clustering also identified 10 clusters whose broad characteristics are summarized in Table 3. The unsupervised learning technique was able to sort time periods that had frequency distributions of range-corrected SNRs > 10 dB with peaks at different heights in each cloud-dominated cluster. Examining the cloud-base heights measured by the ceilometer showed that these peaks in range-corrected SNRs likely corresponded to groups of clouds with similar cloud-base heights. Additionally, the unsupervised learning algorithm was able to discern the presence of clouds during rainy times, unlike the SNR peak detection algorithm.
Summary of clusters identified by the unsupervised learning algorithm.
Preliminary testing of the algorithm has already started on the Waggle node at the SGP site. The ResNet50-based algorithm, starting from ARM Doppler lidar I/Q data, can provide classifications on an hour of vertically pointing stare data in under 2 min while running on a Waggle node. Therefore, running the ResNet50-based algorithm on a vertical stare can identify time periods where the Waggle node will then run processing of full Doppler spectra and all moments on the vertical stare data during cloudy and rainy conditions for users interested in cloud microphysics. Future work will optimize the design of this data pipeline using Waggle toward automatically providing ARM Doppler lidar data in regions of interest to ARM users. This will also enable processing of the Doppler lidar data, which will expand the number of training data available for developing and validating machine learning techniques and developing climatology of cloud and precipitation particle fall speeds from the ARM Doppler lidar.
Acknowledgments.
Argonne National Laboratory’s work was supported by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research, under Contract DE-AC02-06CH11357. The data were obtained from the Atmospheric Radiation Measurement (ARM) user facility, a U.S. Department of Energy (DOE) Office of Science user facility managed by the Biological and Environmental Research Program. This work was supported by the ANL Laboratory Directed Research and Development program. The DOE ARM user facility supported the work under the field campaign AFC 07056 “ARMing the Edge: Demonstration of Edge Computing.” The Sage project is funded through the U.S. National Science Foundation’s Mid-Scale Research Infrastructure program, NSF-OAC-1935984. The authors appreciate the valuable input provided by three anonymous reviewers, which greatly improved the quality of the paper.
Data availability statement.
The ARM SGP data [https://doi.org/10.5439/1181954 (ceilometer) and https://doi.org/10.5439/1021460 (rawinsonde)] and ARM Doppler lidar autocorrelation function (https://doi.org/10.5439/1393859) can be obtained online. The models created and tested here are available on GitHub (https://www.github.com/rcjackson/arming_the_edge). In addition, the model implemented to run on the Waggle node is available online (https://www.github.com/rcjackson/plugin-weatherclassification).
REFERENCES
Abadi, M., and Coauthors, 2016: TensorFlow: Large-scale machine learning on heterogeneous distributed systems. TensorFlow, https://www.tensorflow.org/.
Agarap, A. F., 2019: Deep learning using rectified linear units (ReLu). arXiv, 1803.08375v2, https://doi.org/10.48550/arXiv.1803.08375.
Aoki, M., H. Iwai, K. Nakagawa, S. Ishii, and K. Mizutani, 2016: Measurements of rainfall velocity and raindrop size distribution using coherent Doppler lidar. J. Atmos. Oceanic Technol., 33, 1949–1966, https://doi.org/10.1175/JTECH-D-15-0111.1.
Basodi, S., C. Ji, H. Zhang, and Y. Pan, 2020: Gradient amplification: An efficient way to train deep neural networks. Big Data Min. Anal., 3, 196–207, https://doi.org/10.26599/BDMA.2020.9020004.
Bengio, Y., P. Simard, and P. Frasconi, 1994: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Network, 5, 157–166, https://doi.org/10.1109/72.279181.
Bergstra, J., D. Yamins, and D. Cox, 2013: Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. Proc. 30th Int. Conf. on Machine Learning, Atlanta, GA, PMLR, 115–123, https://proceedings.mlr.press/v28/bergstra13.html.
Bholowalia, P., and A. Kumar, 2014: EBK-means: A clustering technique based on elbow method and k-means in WSN. Int. J. Comput. Appl., 105, 17–24, https://research.ijcaonline.org/volume105/number9/pxc3899674.pdf.
Binietoglou, I., G. D’Ámico, H. Baars, L. Belegante, and E. Marinou, 2018: A methodology for cloud masking uncalibrated lidar signals. EPJ Web Conf., 176, 05048, https://doi.org/10.1051/epjconf/201817605048.
Boucher, O., and Coauthors, 2013: Clouds and aerosols. Climate Change 2013: The Physical Science Basis, T. F. Stocker et al., Eds., Cambridge University Press, 571–657.
Chen, M., X. Shi, Y. Zhang, D. Wu, and M. Guizani, 2021: Deep feature learning for medical image analysis with convolutional autoencoder neural network. IEEE Trans. Big Data, 7, 750–758, https://doi.org/10.1109/TBDATA.2017.2717439.
Chen, T., and C. Guestrin, 2016: XGBoost: A scalable tree boosting system. Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, New York, NY, ACM, 785–794, http://doi.acm.org/10.1145/2939672.2939785.
Dematties, D., and Coauthors, 2023: Let’s unleash the network judgment: A self-supervised approach for cloud image analysis. Artif. Intell. Earth Syst., 2, e220063, https://doi.org/10.1175/AIES-D-22-0063.1.
Demoz, B., and Coauthors, 2006: The dryline on 22 May 2002 during IHOP 2002: Convective-scale measurements at the profiling site. Mon. Wea. Rev., 134, 294–310, https://doi.org/10.1175/MWR3054.1.
Farhani, G., R. J. Sica, and M. J. Daley, 2021: Classification of lidar measurements using supervised and unsupervised machine learning methods. Atmos. Meas. Tech., 14, 391–402, https://doi.org/10.5194/amt-14-391-2021.
Friedman, J. H., 2001: Greedy function approximation: A gradient boosting machine. Ann. Stat., 29, 1189–1232, https://doi.org/10.1214/aos/1013203451.
Friedman, J. H., 2002: Stochastic gradient boosting. Comput. Stat. Data Anal., 38, 367–378, https://doi.org/10.1016/S0167-9473(01)00065-2.
GOES-R Algorithm Working Group and GOES-R Series Program, 2017: NOAA GOES-R series advanced baseline imager (ABI) level 2 cloud and moisture imagery products (CMIP). NOAA National Centers for Environmental Information, accessed 15 November 2019, https://doi.org/10.7289/V5736P36.
He, K., X. Zhang, S. Ren, and J. Sun, 2015: Deep residual learning for image recognition. arXiv, 1512.03385v1, https://doi.org/10.48550/arXiv.1512.03385.
Kalesse-Los, H., W. Schimmel, E. Luke, and P. Seifert, 2022: Evaluating cloud liquid detection against Cloudnet using cloud radar Doppler spectra in a pre-trained artificial neural network. Atmos. Meas. Tech., 15, 279–295, https://doi.org/10.5194/amt-15-279-2022.
Kalthoff, N., 2013: KITcube—A mobile observation platform for convection studies deployed during HyMeX. Meteor. Z., 22, 633–647, https://doi.org/10.1127/0941-2948/2013/0542.
Kingma, D. P., and J. Ba, 2017: Adam: A method for stochastic optimization. arXiv, 1412.6980v9, https://doi.org/10.48550/arXiv.1412.6980.
Krishnamurthy, R., R. K. Newsom, L. K. Berg, H. Xiao, P.-L. Ma, and D. D. Turner, 2021: On the estimation of boundary layer heights: A machine learning approach. Atmos. Meas. Tech., 14, 4403–4424, https://doi.org/10.5194/amt-14-4403-2021.
LeCun, Y., Y. Bengio, and G. Hinton, 2015: Deep learning. Nature, 521, 436–444, https://doi.org/10.1038/nature14539.
Lottman, B. T., R. G. Frehlich, S. M. Hannon, and S. W. Henderson, 2001: Evaluation of vertical winds near and inside a cloud deck using coherent Doppler lidar. J. Atmos. Oceanic Technol., 18, 1377–1386, https://doi.org/10.1175/1520-0426(2001)018<1377:EOVWNA>2.0.CO;2.
Mather, J. H., and J. W. Voyles, 2013: The ARM Climate Research Facility: A review of structure and capabilities. Bull. Amer. Meteor. Soc., 94, 377–392, https://doi.org/10.1175/BAMS-D-11-00218.1.
May, P. T., and T. P. Lane, 2009: A method for using weather radar data to test cloud resolving models. Meteor. Appl., 16, 425–432, https://doi.org/10.1002/met.150.
Morris, V. R., 2016: Ceilometer instrument handbook. USDOE Tech. Rep. DOE/SC-ARM-TR-020, 26 pp., https://doi.org/10.2172/1036530.
Newsom, R. K., and R. Krishnamurthy, 2022: Doppler lidar (DL) instrument handbook. USDOE Tech. Rep. DOE/SC-ARM-TR-101, 59 pp., https://doi.org/10.2172/1034640.
Newsom, R. K., C. Sivaraman, T. Shippert, and L. D. Riihimaki, 2019: Doppler lidar wind value-added product. USDOE Tech. Rep. DOE/SC-ARM/TR-148, 18 pp., https://doi.org/10.2172/1238069.
NOAA/NWS Radar Operations Center, 1991: NOAA Next Generation Radar (NEXRAD) level 2 base data. NOAA National Centers for Environmental Information, accessed 15 November 2019, https://doi.org/10.7289/V5W9574V.
Sawyer, V., and Z. Li, 2013: Detection, variations and intercomparison of the planetary boundary layer depth from radiosonde, lidar and infrared spectrometer. Atmos. Environ., 79, 518–528, https://doi.org/10.1016/j.atmosenv.2013.07.019.
Seidel, D. J., Y. Zhang, A. Beljaars, J.-C. Golaz, A. R. Jacobson, and B. Medeiros, 2012: Climatology of the planetary boundary layer over the continental United States and Europe. J. Geophys. Res., 117, D17106, https://doi.org/10.1029/2012JD018143.
Simonyan, K., and A. Zisserman, 2015: Very deep convolutional networks for large-scale image recognition. arXiv, 1409.1556v6, https://doi.org/10.48550/arXiv.1409.1556.
Su, T., Z. Li, and R. Kahn, 2020: A new method to retrieve the diurnal variability of planetary boundary layer height from lidar under different thermodynamic stability conditions. Remote Sens. Environ., 237, 111519, https://doi.org/10.1016/j.rse.2019.111519.
Träumner, K., J. Handwerker, A. Wieser, and J. Grenzhäuser, 2010: A synergy approach to estimate properties of raindrop size distributions using a Doppler lidar and cloud radar. J. Atmos. Oceanic Technol., 27, 1095–1100, https://doi.org/10.1175/2010JTECHA1377.1.
Wagner, T. J., and J. M. Kleiss, 2016: Error characteristics of ceilometer-based observations of cloud amount. J. Atmos. Oceanic Technol., 33, 1557–1567, https://doi.org/10.1175/JTECH-D-15-0258.1.
Wei, T., H. Xia, J. Hu, C. Wang, M. Shangguan, L. Wang, M. Jia, and X. Dou, 2019: Simultaneous wind and rainfall detection by power spectrum analysis using a VAD scanning coherent Doppler lidar. Opt. Express, 27, 31 235–31 245, https://doi.org/10.1364/OE.27.031235.
Wei, T., H. Xia, B. Yue, Y. Wu, and Q. Liu, 2021: Remote sensing of raindrop size distribution using the coherent Doppler lidar. Opt. Express, 29, 17 246–17 257, https://doi.org/10.1364/OE.426326.
Yang, S., F. Peng, S. von Löwis, G. N. Petersen, and D. C. Finger, 2021: Using machine learning methods to identify particle types from Doppler lidar measurements in Iceland. Remote Sens., 13, 2433, https://doi.org/10.3390/rs13132433.
Zeng, S., and Coauthors, 2019: Application of high-dimensional fuzzy k-means cluster analysis to CALIOP/CALIPSO version 4.1 cloud–aerosol discrimination. Atmos. Meas. Tech., 12, 2261–2285, https://doi.org/10.5194/amt-12-2261-2019.
Zhang, Y., K. Sun, Z. Gao, Z. Pan, M. A. Shook, and D. Li, 2020: Diurnal climatology of planetary boundary layer height over the contiguous United States derived from AMDAR and reanalysis data. J. Geophys. Res. Atmos., 125, e2020JD032803, https://doi.org/10.1029/2020JD032803.
Zhang, Z., and M. R. Sabuncu, 2018: Generalized cross entropy loss for training deep neural networks with noisy labels. arXiv, 1805.07836v4, https://doi.org/10.48550/arXiv.1805.07836.
