1. Introduction
The Weather Surveillance Radar-1988 Doppler (WSR-88D) has the primary functions of detecting hydrometeors, their distribution/intensity, and their Doppler characteristics. This allows for the detection, interrogation, and warning of flash floods, thunderstorms, tornadoes, heavy snowfall events, etc. In the early 2010s, the WSR-88D fleet was upgraded to include dual-polarization capabilities, expanding the functionality to include hydrometeor characterization via the inclusion of the differential reflectivity ZDR, correlation coefficient ρhv, and estimates of differential phase ϕDP. While the experienced forecaster can deduce hydrometeor types based on these new estimates (combined with the original moments of horizontal reflectivity factor Z, radial velocity V, and spectrum width W), the hydrometeor classification algorithm (HCA) was developed to provide an estimated hydrometeor type as a Level-III product in the Open Radar Product Generator (ORPG; Ryzhkov et al. 2005; Scharfenberg et al. 2005; Heinselman and Ryzhkov 2006; Park et al. 2009) that could serve as a source for non-human-in-the-loop applications.
The HCA operates on the principle of fuzzy logic (Straka and Zrnić 1993; Straka 1996), using the aforementioned estimates combined with three derived estimates: the logarithm of specific differential phase KDP (LKDP), standard deviation of Z (SD[Z]), and standard deviation of ϕDP (SD[ϕDP]; Park et al. 2009). An estimate of the hydrometeor type can be made through a series of trapezoidal functions (some in one dimension, and some in two dimensions), weights on these functions, confidence vectors, a collection of restrictions (described below), and knowledge of the melting-layer height and depth. The initial work on hydrometeor classification took place during the Joint Polarization Experiment (JPOLE) in 2003 (Ryzhkov et al. 2005; Scharfenberg et al. 2005), with further refinements made by Park et al. (2009). A modified Park et al. (2009) version was eventually included in the WSR-88D ORPG (as the “original” HCA), and a number of slight adjustments have been made in the years since, such as the hybrid hydrometeor classification (HHC) algorithm (Apffel et al. 2015). The current version of the HCA on the WSR-88D can identify 10 different classes: ground clutter/anomalous propagation (GC), biological scatterers (BI), ice crystals (IC), dry snow (DS), wet snow (WS), rain (RA), heavy rain (HR), big drops (BD), graupel (GR), and hail/rain (RH). An unknown (UK) class is also available for cases where certain minimum requirements are not met.
The Radar Operations Center and the National Severe Storms Laboratory have investigated incorporating additional classes and subclasses. Recent additions to the HCA allow subclassification of the RH class into hail (HA), large hail (LH), and giant hail (GH) by virtue of the hail size discrimination algorithm (HSDA; Ryzhkov et al. 2013; Ortega et al. 2016). Other work has investigated the addition of a tornadic debris class (Snyder and Ryzhkov 2015), while Mahale et al. (2014) proposed a method for identifying three-body scattering in hail spikes. There have also been studies investigating winter precipitation types, including at the ground (Elmore 2011) and aloft (Thompson et al. 2014). As of build 18 in the ORPG, only the additional HSDA hail size subclasses have been added to the original HCA. A number of changes to the HCA trapezoidal membership functions have been made in the years since the original HCA implementation, including recent changes to the DS category that affect the transition between DS, WS, and GR near the melting layer (B. Klein 2019, personal communication). The build 19 (operational deployment expected in 2020) HCA specifics are discussed in the next section.
The purpose of this paper is to present a new WSR-88D HCA class known as inanimate (IN). IN covers a series of nonweather target types that do not logically fall under the existing HCA categories but are often mislabeled as BI, BD, DS (within and above the melting layer), and UK. These targets, which maintain similar features in the radar estimates, are chaff, sea clutter, and some types of combustion debris and radio frequency interference (RFI). This study focuses on the performance of the IN class in chaff and sea clutter, since combustion debris/RFI can take various forms based on the burning/interference sources. Combustion debris, in particular, will occasionally present as IN but can even be classified as precipitation depending on the source of combustion. Therefore, IN is not intended as a classification that can reliably capture combustion debris and RFI. However, future work on an RFI class would be impactful given the increasing use of the S band for wireless data links, and so on.
The IN class was developed using a series of 60 human-truthed cases of weather, BI, and IN target types (20 per type). The approach to human truthing was the same as that described in Kurdzo et al. (2018). A lasso tool was utilized to circle areas of different target types using subject matter expert opinions based on thousands of previous case observations. A genetic algorithm (Goldberg 1989) was used to optimize the trapezoidal membership functions, weights, and restrictions based on a scaled cost function that awarded correct detections and penalized changes to existing classes in the non-IN cases (and non-IN gates in the IN cases). Results show excellent IN detection characteristics in IN cases, virtually zero false alarms in weather, and minimal changes of BI classifications in biological cases (results are quantified in the following sections).
The IN class has been built upon the existing build-18 HCA but, for the time being, will only be included in a new product within the ORPG known as the aviation classification algorithm (ACA) for testing purposes. The ACA is an identical representation of the build-18 HCA with the addition of the IN class. Although IN may eventually be incorporated in the operational HCA, the ACA also exists as an alternative to the existing HCA for other Federal Aviation Administration (FAA) algorithms being developed based on the current HCA rules in order to alleviate the need for recalibration at later dates.
This paper is organized as follows: the data and methods section describes the ORPG simulator used for this study, the selection of cases, human truthing, the genetic algorithm, the cost function development, and the choice of restrictions. The results section details results in multiple types of cases, including chaff (Kurdzo et al. 2018), sea clutter (Ryzhkov et al. 2002), GC, and BI. Results are shown from build 18 of the ORPG (not the ORPG simulator) and are described in terms of accuracy, both quantitatively and qualitatively. Last, the discussion and summary section describes the use cases for the IN class, including the possibility of future subclassing. The use of IN in a WSR-88D chaff detection algorithm (CDA) is discussed (Kurdzo et al. 2017a), and a summary of the results is presented.
2. Data and methods
a. Data collection and processing
Development of the IN class required the use of various Level-III radar products and derived products. Level-III data are necessary for development due to the fact that the recombined Level-III data are used in the operational ORPG, meaning that all training and optimization must be done at Level III. The ORPG inputs Level-II data and outputs Level-III data. The archived Level-III WSR-88D data at the National Centers for Environmental Information (NCEI) do not include all of the derived products included in the HCA algorithm, such as SD[Z] and SD[ϕDP]. For example, ϕDP was not a Level-III product in the ORPG at the time of IN development. To alleviate this problem, the ORPGSim software suite was developed (Kurdzo et al. 2017b, 2018). Leveraging access to the ORPG C/C++ code, a nearly one-to-one translation of all critical ORPG functions was made in the MATLAB software package. The MATLAB-based ORPGSim performs polar-to-Cartesian transformation, recombination into nonsuper resolution, the dual-polarization preprocessor, and a series of modules including the HCA and quantitative precipitation estimation product generators. The ORPGSim version used in this study is based on the code in ORPG build 17. From extensive testing in build 18 on the C/C++-based ORPG, the minor differences between build 17 and the current build 18 do not cause any ill effects on the IN class.
Level-II data were collected via the NCEI archive hosted on Amazon Web Services. The Level-II data were processed in ORPGSim to create all relevant Level-III products and derived products. Sixty cases were selected manually: 20 chaff/sea-clutter cases, 20 GC/BI cases, and 20 general weather cases spanning a variety of seasons, locations, and intensities. Each case consisted of just one volume scan. Upper tilts were used due to the frequent observations of IN-type targets (i.e., chaff) reported in Kurdzo et al. (2018). Restrictions on the number of cases are explained below. At the outset, the original goal of this study was to create a chaff classification. However, it was quickly realized that the statistical representations of chaff and sea clutter were very similar in the data ingested into the HCA. In addition, after training on chaff and sea clutter, many cases of combustion debris and several RFI cases were also captured by the new class. The class was therefore named “inanimate” on the basis of the type of targets being detected. This IN class provides input to a separate CDA to meet the original goal of an FAA request by distilling chaff from sea clutter (and other targets).
b. Fuzzy logic
The current HCA is based on fuzzy logic, including six membership functions for each class (each with their own weighting for each class), confidence vectors, and threshold values. Each radial of Level-II data is processed through the ORPG for product generation, and the resulting products/estimates are passed to the HCA through a preprocessor module. The Z, ZDR, ρhv, LKDP, SD[Z], and SD[ϕDP] each have a membership function for each class. The variable referred to as LKDP is a logarithmic value of specific differential phase KDP using a base-10 logarithm and is set to −30 if KDP is less than or equal to 10−3 ° km−1. The Z, ZDR, and ρhv are smoothed along range. Horizontal reflectivity factor Z uses a 1-km uniform averaging window, while ZDR and ρhv use a 2-km uniform averaging window. Noise corrections for ZDR and ρhv are also performed as discussed in Schuur et al. (2003) and Park et al. (2009).
The membership function is represented by a trapezoid in either one or two dimensions and varies from 0 to 1 in amplitude. For example, the one-dimensional functions are defined by four points, from left to right along the possible values of each estimate [see Fig. 1, adapted from Park et al. (2009)]: the starting point at value 0, the beginning of the maximum at value 1, the end of the maximum at value 1, and the final point at value 0. Depending on where in the trapezoid a data point falls, it will be assigned a value from 0 to 1.
One-dimensional trapezoidal function, where the values x1–x4 are the four points that define the trapezoid for a given polarimetric variable x.
Citation: Journal of Applied Meteorology and Climatology 59, 5; 10.1175/JAMC-D-19-0271.1
The HCA process, including the input of data, testing relative to membership function, addition of weights and confidence vectors, and the summation of score for HCA output.
Citation: Journal of Applied Meteorology and Climatology 59, 5; 10.1175/JAMC-D-19-0271.1
Membership function parameters, including the added IN class. Reflectivity-dependent parameters f1–f3, g1, and g2 are defined by Eqs. (2)–(6), respectively. Changes and additions made since Park et al. (2009) as of build 19 in the ORPG are marked with an asterisk.
Matrix of weights, including the added IN class, by variable. Changes and additions made since Park et al. (2009) as of build 19 in the ORPG are marked with an asterisk.
In general, when designing a new class for the HCA, we did not want to change any of the original correct categorizations (to the maximum extent possible). Of course, it is also desirable to capture as many of the new target types as possible in the new class. For these reasons, all of the existing classes, membership functions, weights, and thresholds must be considered when designing a solution. In an attempt to change as little as possible with the existing HCA, new membership functions cannot easily be added, nor can the values or weights be changed. However, additional thresholding options can be added, which are described below.
c. Data sources for thresholds
During each processing run within ORPGSim, a copy of all relevant HCA-related estimates were saved in a database demarcated based on case type. The saved data included Z, V, W, ZDR, ρhv, LKDP, SD[Z], SD[ϕDP], and the original, nonaltered HCA classification. Despite not being part of the six estimates used in the fuzzy logic membership functions, V is used for thresholding in the GC class, and W showed promise as a thresholding variable. It is important to note that Z, ZDR, and ρhv are used as thresholds in other classes. Therefore, the remaining variables were all explored for thresholding possibilities.
Through the optimization process described below, three variables displayed usefulness for thresholding in the development of the IN class: V, W, and SD[ϕDP]. In the final optimizations, each of these variables was allowed to vary through a range of possible values to provide hard cutoffs for the classification of IN. V and W were used in “standard” ways, that is, as single thresholds. SD[ϕDP] was utilized differently. It was discovered that there were two useful cutoff points in SD[ϕDP], one for the IN class, and one to define the “ice” classes (i.e., DS, WS, and IC). This was because false alarms and missed detections were occurring in areas within and above the melting layer where the characteristics of SD[ϕDP] changed.
To resolve this issue, one change was made to the existing HCA in its ACA form. Each of the ice classes was given an additional threshold of SD[ϕDP]. This resulted in two SD[ϕDP] threshold definitions: SD[ϕDP]IN and SD[ϕDP]ICE. The updated versions of the threshold values, plus the additional thresholds for SD[ϕDP]ICE and all of the thresholds for IN, are presented in Table 3. The values for these new parameters are discussed in the following subsection.
Hard thresholds used to suppress incorrect HCA classes. Changes and additions made since Park et al. (2009) as of build 19 in the ORPG are marked with an asterisk.
d. Optimization
The optimization procedure was carried out using a genetic algorithm (GA). This algorithm type is based on evolutionary computing, and utilizes random “mutations” of bits in order to attempt to find a global optimum solution (Goldberg 1989). GAs were chosen for this study due to their relative flexibility and ability to be parallelized for large datasets. It is important to note that other optimization techniques could certainly achieve similar results. One challenge with GAs is that despite their ability to be put in parallel with multiple workers/processes, they can be quite computationally complex. For this reason, the choice to use a GA limited the study to 60 total cases for optimization. In the future, a larger size of training cases may be utilized if a more-efficient optimization technique is explored. For now, we acknowledge this fact as a limitation in our results.
The GA was hosted and run on the Massachusetts Institute of Technology Lincoln Laboratory Supercomputing Center grid (LLSC; Bliss et al. 2006). Each case was loaded onto an LLSC node, with precomputed products and derived products. In addition to the previously mentioned products, for the IN cases, all instances of IN returns were manually truthed so that their locations were known (see Kurdzo et al. 2017b). This involved manually drawing polygons around areas of chaff and sea clutter for each case. This “truth” mask was passed along with the rest of the data for each IN case and used for scoring (described below).
At each iteration of the GA, 28 values are passed into the fitness function that can be varied between iterations. These values are the four trapezoidal points for each membership function (20 total; LKDP is not included, since KDP is not calculated for nonweather targets, that is, targets with a low ρhv value), the weight values (5 total, since LKDP is not included), and cutoff thresholds for W, SD[ϕDP]IN, and SD[ϕDP]ICE. The cutoff thresholds are a maximum value for W, a minimum value for SD[ϕDP]IN, and a maximum value for SD[ϕDP]ICE. Although ϕDP is expected to rise as ρhv decreases (Doviak and Zrnić 1993), some IN cases display ρhv near 1.0 despite an elevated ϕDP. Therefore, the SD[ϕDP] cutoffs aid in preventing false alarms in ice with slightly lowered ρhv values.
The cutoff for minimum V was set to 1. Integer programming was used for computational simplicity (Eiben and Smith 2007), allowing for specific ranges and resolutions of each variable in the GA. These values are listed in Table 4. The bounds were chosen empirically based on observation and experimentation with the GA.
Minimum value, maximum value, and resolution of the 28 variables included in the genetic algorithm. Values were chosen empirically on the basis of experimentation.
Computational complexity precluded the inclusion of additional cases, due primarily to memory constraints. This was because every case had to be loaded for every generation of the GA. Despite only using 60 total cases, the results presented in the next section appear to be robust and, most important, do not lead to false alarms in weather.
3. Results
a. Genetic algorithm solutions
The optimized values for the IN class are presented in Tables 1–3. The bounds from Table 4 were used as inputs to the GA. A few important aspects of the comparison between Tables 1 and 4 must be pointed out. First, LKDP is not included in the optimization because, like the GC and BI classes, KDP is not calculated for low values of ρhv. Additionally, the membership values for ZDR and SD[ϕDP] fall outside of the bounds of those set in Table 4. This is because, as Table 2 shows, the ZDR and SD[ϕDP] membership functions were given zero weight in the final aggregation, despite the ranges specified. For this reason, the values in Table 1 were set to “default” values for ZDR, SD[ϕDP], and LKDP. Note that for ZDR, the entire range of the field is represented. This is empirical but is described in detail in Kurdzo et al. (2018). A similar observation was noted in the SD[ϕDP] (i.e., a vast range was noted).
It is hypothesized that the reason for zero weight being placed upon the ZDR and SD[ϕDP] membership functions in the score aggregation by the GA is due to the wide range of the variables. The examples shown in Fig. 2 of Kurdzo et al. (2018) demonstrate that ZDR can span the entire spectrum, from −7.9 to 7.9 dB in chaff. This is also true for other IN target types. If a range of values that spans all possibilities exists, the importance or value of the variable goes to zero. It has been qualitatively observed that SD[ϕDP] also spans a very large range in IN targets. It has also been observed that both ZDR and ϕDP roughly equally span the possible ranges, that is, are relatively “flat” functions in their distributions (Kurdzo et al. 2018).
Looking at the relevant values, Z and ρhv are weighted heavily in the IN class (a full value of 1.0), while SD[Z] is weighted at 0.8. IN is most comparable to the DS class in the Z field, with membership values ranging from 3 to 45 dBZ. This is not all inclusive of the IN class, as cases of chaff have been observed at over 60 dBZ (Murphy et al. 2016), but it does cover the vast majority of cases. The ρhv membership function extends lower than the BI class, but also extends to the top of the range of values. Unlike ZDR, however, ρhv is a critical tool for differentiating between weather and nonweather echoes. Values less than approximately 0.97–0.99 are indicative of (at the very least) mixed precipitation types, and as the values get lower, nonmeteorological echoes are likely (Bringi and Chandrasekar 2001). Note that unlike ZDR, which roughly equally spans the possible ranges, the membership function values for ρhv are heavily weighted toward the lower values. This is consistent with IN being a nonweather classification. The SD[Z] values are very similar to the BI class and offer some differentiation between GC and weather.
The threshold values for the IN class are detailed in Table 3. The V was set to 1 m s−1 empirically to avoid too many false alarms in GC. This does cause some issues along the zero isodop, but these problems have not been particularly pronounced in testing due to the relatively thin nature of the zone in which V is ±1 m s−1. Note that in large areas of low V, a reduction in IN detections may result. This is a consequence of having such a threshold. Note that W is a key attribute for the IN class when considering thresholding. In our experience, IN targets rarely display W exceeding 2–3 m s−1. When it comes to chaff, for example, this can be thought of in terms of the targets acting as a tracer (Kurdzo et al. 2018). However, in cases of clear-air turbulence, for example, chaff may not be detected as IN. The GA determined that a value of 2.5 m s−1 was optimal for preventing misclassifications. Many GC/AP targets display W values much higher than this 2.5 m s−1 cutoff.
The final threshold values are both related to SD[ϕDP]. A number of misclassifications were happening in the ice classes, which was a problem in areas of weather within and above the melting layer. This was mitigated by including SD[ϕDP]IN and SD[ϕDP]ICE thresholds. The SD[ϕDP]ICE threshold is a “maximum” value and is for the DS, WS, and IC classes, while SD[ϕDP]IN is a “minimum” value and is solely for the IN class. The GA determined that IN very rarely produces SD[ϕDP] below 24°, while the ice classes rarely produce SD[ϕDP] above 44°. A significant number of additional thresholds were added in the other classes within the ORPG since Park et al. (2009), as seen in Table 3, but only the SD[ϕDP]ICE, SD[ϕDP]IN, V, and W thresholds were added as part of this work.
b. Data examples
A number of representative performance examples of IN successes (and relative “failures”) are presented in this section in order to characterize performance of the IN ACA class. These examples have been separated into three different categories, each with a “success” and a relative “failure” example. The purpose of this is to show both the strengths and limitations of the IN classification. The three categories are chaff, ground clutter, and sea clutter. It is important to note that none of these cases were used in training of the algorithm.
Each case is human truthed in order to manually select the gates that are expected to be registered as IN. For example, in the chaff and sea-clutter cases, areas of these types of targets are manually outlined in order to differentiate them from non-IN targets. These outlines/masks are then used for validation in the results that follow. The results are summarized in Table 5, which shows the six cases and their types, the number of IN gates, the number of non-IN (NIN) gates, the IN “hit” count and percentage, and the IN false alarm count and percentage.
Performance values for the six cases presented in this study.
It is acknowledged that human truthing is a subjective process. While there are relatively few confirmations of chaff releases, for example, subject matter experts were able to discern different target types at maximum likelihood based on previous observations in the literature (e.g., Kurdzo et al. 2018; Murphy et al. 2016). The same is true for sea clutter. Subject matter experts are generally able to use the polarimetric estimates to consistently separate nonweather targets from weather targets, and IN-type targets from BI targets. Cases that were relatively “clear cut” were used for this section.
As part of the results, the reader is reminded that the original HCA, and by extension the analogous ACA, is an imperfect algorithm. In the vast majority of instances, the HCA approach leads to output that is “speckled”/noisy, even in seemingly obvious classifications where a smooth coherency is expected, such as the clutter bloom (the area of enhanced clutter near the radar; Lakshmanan et al. 2010). This is evidenced by the fact that the HCA cannot be used exclusively to remove nonweather returns (Krause 2016). The IN classification was originally meant to detect chaff, and given the follow-on image processing algorithms to process the IN class presented in Kurdzo et al. (2017a), the density of IN detections has proven to be successful in identifying areas of chaff targets. More on this topic is presented in the discussion and summary section.
1) Chaff examples
Cases 1 and 2 are chaff cases, with case 1 taking place on 8 August 2016 at KBYX (1758:43 UTC), and case 2 taking place on 6 May 2016 at KDOX (0113:22 UTC). The areas of chaff can be clearly delineated in Figs. 3 and 4 in the ZDR, ρhv, and ϕDP fields in each case. The characteristics include ZDR spanning the entire range from −7.9 to 7.9 dB, lowered ρhv (although not always), and high/spatially variable ϕDP (Kurdzo et al. 2018). In the original HCA, at the bottom left of Figs. 3 and 4, the results are mixed. In case 1, the predominant HCA output is BI, with some BD interspersed. In case 2, BI and BD are prevalent, but an area of DS is evident above the melting layer. A relatively large area of UK is also present.
Reflectivity (dBZ), differential reflectivity (dB), correlation coefficient, differential phase (°), original HCA output, and ACA output with IN class included for case 1 (for the 0.5° tilt), a chaff case from 8 Aug 2016 at KBYX.
Citation: Journal of Applied Meteorology and Climatology 59, 5; 10.1175/JAMC-D-19-0271.1
As in Fig. 3, but for case 2, a chaff case from 6 May 2016 at KDOX.
Citation: Journal of Applied Meteorology and Climatology 59, 5; 10.1175/JAMC-D-19-0271.1
As discussed in Kurdzo et al. (2018), the appearance of high and variable ϕDP in chaff is similar to that seen in clutter (Ryzhkov et al. 2005). A random-phase sum of the horizontal and vertical echo components leads to a highly spatially variable ϕDP field. In the case of a lack of common alignment, such as in chaff, sea clutter, or combustion debris, the horizontal and vertical components would be random due to the fact that the return is a random-phase sum of the contributing scatterers.
The ACA with IN, shown in the bottom right of Figs. 3 and 4, includes all new IN classifications in black. Of the gates manually determined to be chaff, approximately 55% and 42% are labeled as IN in cases 1 and 2, respectively (as depicted in Table 5). In case 1, the secondary class is BI, but there are also areas of BD and other classes. In case 2, relatively large areas of BD, DS, and UK remain, but the majority of BI gates have been changed to IN in the updated ACA.
Case 1 is considered to be a relative success for the algorithm, whereas case 2 is regarded as a relatively difficult case for the algorithm. However, despite the probability of detection (POD) being below 43% in case 2, the user of such data can clearly define the area most likely to contain IN targets. This is even more apparent in case 1. What may be considered more important is that non-IN classes should be changed as little as possible. In testing, there are virtually zero changes to precipitating classes with IN included in the ACA. Most of the false alarms occur in the clutter bloom, where GC and BI can be incorrectly classified as IN. In case 1, however, only 0.76% of non-IN-truthed gates were incorrectly identified as IN (false alarms), and in case 2, that number rises to 2.32%.
2) Ground-clutter examples
Cases 3 and 4 are ground-clutter cases, with case 3 taking place on 20 May 2014 at KICT (0600:56 UTC), and case 4 taking place on 23 July 2018 at KPOE (1227:01 UTC). These are the only included cases that do not contain IN targets, since weather-only cases contain, at most, a small handful of changed gates (often in the single digits outside of the clutter bloom). Of course, it is hoped that ground-clutter cases are identified as either GC or BI, as is typical in the existing HCA. Any IN detections in these cases are considered false alarms. Since there are no truthed IN gates in cases 3 and 4, the number of manually truthed IN gates is 0 and the IN hit counts/percentages are not relevant in Table 5. The data for cases 3 and 4 are presented in Figs. 5 and 6.
As in Fig. 3, but for case 3, a ground-clutter case from 20 May 2014 at KICT.
Citation: Journal of Applied Meteorology and Climatology 59, 5; 10.1175/JAMC-D-19-0271.1
As in Fig. 3, but for case 4, a ground-clutter case from 23 Jul 2018 at KPOE.
Citation: Journal of Applied Meteorology and Climatology 59, 5; 10.1175/JAMC-D-19-0271.1
In both cases, the clutter bloom is characterized by low Z, a range from ~0 to 7.9 in ZDR, lowered ρhv, and a wide range of ϕDP. It can be seen that some of these characteristics overlap with those in cases 1 and 2, meaning there is the potential for IN false alarms in the clutter bloom. Case 3, a ground-clutter/bird-migration case (Chilson et al. 2012; Stepanian and Horton 2015; Horton et al. 2016), consists of a majority of BI classifications in the original HCA (shown in Fig. 5), with some areas of BD to the east, and areas of UK to the north and along the fringes of the clutter bloom. In the ACA with IN, there is a “speckling” of IN classifications, concentrated especially to the north and west of the radar, leaving the BD and UK classifications largely intact. The false alarm rate (FAR) for case 3 is 1.07%, which is considered to be relatively good. This consideration is quantified further in the discussion and summary section based on previous studies.
Case 4 is an example of strong ducting, evidenced by moderate Z returns in discrete areas around, within, and outside the main clutter bloom (see Fig. 6). A wide range of ZDR, ρhv, and ϕDP are observed. In particular, compared to case 3, the ϕDP field is much more variable, similar to what was seen in the chaff cases in Figs. 3 and 4. Although not shown, compared to other clutter cases, case 4 contains lower W estimates, especially to the distant southwest of the radar. The original HCA classified most of the clutter bloom as BI, with a mix of many other classes. Note that there is precipitation to the southeast indicated by semidiscrete convective cells. The ACA with IN classifies a large swath of the clutter bloom as IN to the north of the radar, and classifies many of the strong clutter returns due to anomalous propagation to the southwest of the radar as IN. The returns to the southwest do primarily fall just off the coast and into the Gulf of Mexico, but also extend over land. Because of this discrepancy, we have categorized these IN detections as false alarms, although this may be overly conservative. The FAR is higher than any other case, at 6.60%, which may be due to our conservative approach. It is important to note that this is one of the highest FAR cases we can recall among thousands of test cases.
3) Sea-clutter examples
Cases 5 and 6 are sea-clutter cases, another target type that is intended to be classified by the IN class. Case 5 takes place on 28 July 2016 at KMHX (1805:53 UTC), while case 6 takes place on 26 August 2016 at KLGX (1410:19 UTC). These cases are characterized by areas of sharp transition in Z and ZDR across the entire spectrum, lowered ρhv, and high spatial variability of ϕDP (see Figs. 7 and 8 ). There are a couple of notable differences between the cases. The ZDR in case 5 tends to be negative in the sea clutter, while there is a mix of negative and positive values in case 6. Both ρhv and ϕDP are more variable in case 5, with a wider range of values. The sea clutter in both cases is characterized in the original HCA by a mix of BI and UK below the melting layer, and BI/DS within and above the melting layer, an artifact due to melting-layer restrictions in the algorithm.
As in Fig. 3, but for case 5, a sea-clutter case from 28 Jul 2016 at KMHX.
Citation: Journal of Applied Meteorology and Climatology 59, 5; 10.1175/JAMC-D-19-0271.1
As in Fig. 3, but for case 6, a sea-clutter case from 26 Aug 2016 at KLGX.
Citation: Journal of Applied Meteorology and Climatology 59, 5; 10.1175/JAMC-D-19-0271.1
In case 5, the sea clutter is generally well classified as IN in the bottom-right panel of Fig. 7, with much of the BI and UK being replaced by IN. The areas of combined higher ρhv and lower ϕDP tend to remain classified as BI, UK, and/or DS. In our experience, this combination is often indicative of sea spray rather than sea clutter, the difference being that the radar beam is reflecting off sea spray that has been lofted. This is indicated by higher ρhv values from the lofted droplets. The IN POD was 52.84% in case 5, with a 1.95% FAR in the clutter bloom on land. Note that there is a streak of RFI to the southwest of the radar that does get classified partially as IN.
Case 6 retains a lower POD of 32.59%, with an elevated FAR of 4.66%. The primary cause of the elevated FAR is thought to be the relative lack of gates with returns/classifications on land, leading to a small sample size for the case. However, the significantly lowered POD of IN within the area of sea clutter offshore is of more interest. The area just offshore to the southwest of the radar is characterized by a distinctly lower and less-variable ϕDP, similar to that seen in sea spray. This characteristic is indicative of liquid water, which is supported by areas of ρhv values near 1.0 in the same region. Although there are elevated ρhv values in patches farther offshore, the area immediately offshore demonstrates a more-concentrated area of elevated ρhv. This area can be seen as a continuous patch of IN in the bottom-right panel of Fig. 8. Although these missed detections may fall under the scope of sea spray, they are still considered as IN targets in the POD/FAR calculations in Table 5.
4. Discussion and summary
The operational WSR-88D HCA is the primary algorithmic approach for determining hydrometeor types at beam level. Although it is able to generally define areas of different types of hydrometeors, biological targets, and ground clutter, the current classifications have some limitations and do not cover all target types. Some of these targets, such as chaff and sea clutter, maintain similar characteristics, and these characteristics are separable enough from other target types to create a new class. This new class, which has been named inanimate, separates out these targets, providing a more-proper classification for these targets rather than forcing them into existing classes.
Results from ACA testing with the IN class show promise for discriminating chaff and sea clutter from weather, ground-clutter, and biological targets. Although the results are not perfect, they are qualitatively in line with the performance of other classes and provide a new capability not in the present HCA. This is evidenced by the results shown in Figs. 3–8, where the IN class areas can be relatively easily determined by a user of the ACA. In successful cases, the areas of IN exceed 50% coverage, while even in the less-successful cases, 30%–40% coverage is not atypical. This has anecdotally allowed our team and other early adoption users of the new IN class to reliably determine areas of chaff and sea clutter.
One of the primary reasons for development of the IN class was to aid in the development of a CDA. Much of the reason for success in the CDA has been due to the density of IN observations, not the absolute percentage of detections (Kurdzo et al. 2017a). With relatively simple image processing techniques (e.g., dilation and closing), it is a straightforward process to identify clusters of IN targets and fill in the gaps of detections. For this reason, it must be stressed that despite the seemingly low values of actual detection rate, the IN classification is serving its purpose well in its original goal of allowing for the detection of chaff. In fact, the clustering for areas of sea clutter also will provide for significant advances in sea-clutter detection and characterization in future renditions of the ACA/HCA.
Historically, different types of HCAs have not been routinely tested for accuracy on a quantitative level (e.g., Thompson et al. 2014). Part of the reason for this is that it is often quite difficult to get “truth” for precipitation types, especially in situ at beam level. Some validation has been attempted for surface-based HCA classifications in Elmore (2011), while studies have looked at surface-based hail validation using the HCA (Heinselman and Ryzhkov 2006) and the HSDA (Ryzhkov et al. 2013; Ortega et al. 2016). One of the only methods to verify beam-level HCA classifications is via aircraft measurements. One such field program was described in Williams et al. (2015).
In Williams et al. (2015), verification between aircraft observations (primarily via particle imagers) using the spheres, needles, dendrites, and irregulars (SNDI) algorithm (Korolev and Sussman 2000) is attempted. The HCA was verified against SNDI data using categorization thresholds presented in Williams et al. (2015, their Table 7-1). By definition, >25% of a target presence was considered to be “when no one category is expected to dominate, but a category is expected to be present.” Additionally, >50% was defined as “when a category of particles is expected to be the primary type of particle found but allowing for a mix of other particles.” Given these definitions, it can be argued that the IN classification does a sufficient job in “successful” cases, achieving or exceeding the 50% threshold. Additionally, in the not-as-successful cases, the threshold for presence of a category is met or exceeded. In all of the non-IN cases, the “not present” category (<10%) is not exceeded.
As we move forward, the IN category will be included in the ACA within the operational ORPG starting in build 19, where it can be evaluated for possible inclusion in the HCA in the future. One goal is to break the IN class into multiple subclasses, similar to the multiple hail subclasses now in the operational HCA (Ryzhkov et al. 2013; Ortega et al. 2016). Using a variety of techniques, IN could be separated into chaff, sea clutter, and “other.” The forthcoming CDA is a target-specific example of such. It will be important to assess these different classes for interference between each other before combination in the operational ORPG.
Sea clutter is often simultaneously present with atmospheric returns. The two scattering processes generate separate Doppler spectra, a basic difference from chaff, which flows with the wind and thus returns signals in the same Doppler bins as any other passive tracer present (including hydrometeors). One result is that the effective W can be fairly wide if the two Doppler spectra are far apart, so a W threshold may exclude some cases of sea clutter from IN. In general, having two spectra with different polarimetric characteristics makes discrimination problematic in the base data domain because of the irreversible mixing. Spectral polarimetry may have the potential to improve target classifications; however, the radar back-end software may be too antiquated to handle such processing.
Initial attempts at subclassification have shown some success. The results in Kurdzo et al. (2017a) show that it is possible to separate chaff from the rest of the IN class with relatively high accuracy using machine learning. In that work, the IN class is grouped into cells that are tested against human-truthed training data. A support vector machine is used to separate chaff from nonchaff in the IN cells. Work in this area is near completion, and it is expected that a CDA will be implemented in the ORPG in build 20.
Further refinement of the IN class is possible via the use of additional cases with more computational resources. Additional thresholds could be included, and more cases would likely result in more-robust results. However, at this stage, the algorithm has performed sufficiently well to be transferred to the ORPG. In the near future, the WSR-88D software will be modified to allow for a wider range of ZDR values, meaning that some of the HCA classes may need to be retuned for optimal performance.
Acknowledgments
This paper is approved for public release, and distribution is unlimited. This material is based upon work supported by the Federal Aviation Administration under Air Force Contract FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Federal Aviation Administration. The authors thank John Cho and Timothy Bonin for reviewing an early version of the paper, Mark Veillette for assistance with human truthing of the IN data, and Earle Williams for discussions about chaff characteristics. Alexander Ryzhkov, Dusan Zrnić, Valery Melnikov, Todd Murphy, and Ryan Wade provided useful feedback regarding chaff and sea clutter, and the NEXRAD Data Quality committee also gave helpful critiques during development. The authors thank Steve Kim for his support of this project. We also thank the three anonymous reviewers who helped to improve the originally submitted paper substantially.
REFERENCES
Apffel, K. R., A. Reynolds, and D. Zaff, 2015: Improving the quantitative precipitation estimate for hydrometeors classified as dry snow by polarimetric radars. NOAA/National Weather Service Eastern Regional Tech. Attachment, 10 pp., https://www.weather.gov/media/erh/ta2015-06.pdf.
Bliss, N. T., R. Bond, J. Kepner, H. Kim, and A. Reuther, 2006: Interactive grid computing at Lincoln Laboratory. Lincoln Lab. J., 16, 165–216.
Bringi, V. N., and V. Chandrasekar, 2001: Polarimetric Doppler Weather Radar: Principles and Applications. Cambridge University Press, 636 pp.
Chilson, P. B., and Coauthors, 2012: Partly cloudy with a chance of migration: Weather, radars, and aeroecology. Bull. Amer. Meteor. Soc., 93, 669–686, https://doi.org/10.1175/BAMS-D-11-00099.1.
Doviak, R. J., and D. S. Zrnić, 1993: Doppler Radar and Weather Observations. Dover Publications, 562 pp.
Eiben, A. E., and J. E. Smith, 2007: Introduction to Evolutionary Computing. Natural Computing Series, 2nd ed. Springer, 300 pp.
Elmore, K. L., 2011: The NSSL hydrometeor classification algorithm in winter surface precipitation: Evaluation and future development. Wea. Forecasting, 26, 756–765, https://doi.org/10.1175/WAF-D-10-05011.1.
Goldberg, D., 1989: Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley, 372 pp.
Heinselman, P. L., and A. V. Ryzhkov, 2006: Validation of polarimetric hail detection. Wea. Forecasting, 21, 839–850, https://doi.org/10.1175/WAF956.1.
Horton, K. G., B. M. Van Doren, P. M. Stepanian, W. M. Hochachka, A. Farnsworth, and J. F. Kelly, 2016: Nocturnally migrating songbirds drift when they can and compensate when they must. Sci. Rep., 6, 21249, https://doi.org/10.1038/srep21249.
Korolev, A., and B. Sussman, 2000: A technique for habit classification of cloud particles. J. Atmos. Oceanic Technol., 17, 1048–1057, https://doi.org/10.1175/1520-0426(2000)017<1048:ATFHCO>2.0.CO;2.
Krause, J. M., 2016: A simple algorithm to discriminate between meteorological and nonmeteorological radar echoes. J. Atmos. Oceanic Technol., 33, 1875–1885, https://doi.org/10.1175/JTECH-D-15-0239.1.
Kurdzo, J. M., B. J. Bennett, M. S. Veillette, D. J. Smalley, E. R. Williams, and M. F. Donovan, 2017a: WSR-88D chaff detection and characterization using an optimized hydrometeor classification algorithm. 18th Conf. on Aviation, Range, and Aerospace Meteorology, Seattle, WA, Amer. Meteor. Soc., J2.5, https://ams.confex.com/ams/97Annual/webprogram/Manuscript/Paper301430/ARAM%202017.pdf.
Kurdzo, J. M., M. S. Veillette, B. J. Bennett, D. J. Smalley, E. R. Williams, and M. F. Donovan, 2017b: Development of a new inanimate class for the WSR-88D hydrometeor classification algorithm. 38th Conf. on Radar Meteorology, Chicago, IL, Amer. Meteor. Soc., 286, https://ams.confex.com/ams/38RADAR/mediafile/Manuscript/Paper320366/AMS%20RADAR%202017.pdf.
Kurdzo, J. M., E. R. Williams, D. J. Smalley, B. J. Bennett, D. C. Patterson, M. S. Veillette, and M. F. Donovan, 2018: Polarimetric observations of chaff using the WSR-88D network. J. Appl. Meteor. Climatol., 57, 1063–1081, https://doi.org/10.1175/JAMC-D-17-0191.1.
Lakshmanan, V., J. Zhang, and K. Howard, 2010: A technique to censor biological echoes in radar reflectivity data. J. Appl. Meteor. Climatol., 49, 453–462, https://doi.org/10.1175/2009JAMC2255.1.
Mahale, V. N., G. Zhang, and M. Xue, 2014: Fuzzy logic classification of S-band polarimetric radar echoes to identify three-body scattering and improve data quality. J. Appl. Meteor. Climatol., 53, 2017–2033, https://doi.org/10.1175/JAMC-D-13-0358.1.
Murphy, T. A., R. A. Wade, and B. C. Carcione, 2016: Observations and operational considerations of the 4 June 2013 chaff event in northern Alabama. J. Oper. Meteor., 4, 34–45, https://doi.org/10.15191/NWAJOM.2016.0403.
Ortega, K. L., J. M. Krause, and A. V. Ryzhkov, 2016: Polarimetric radar characteristics of melting hail. Part III: Validation of the algorithm for hail size discrimination. J. Appl. Meteor. Climatol., 55, 829–848, https://doi.org/10.1175/JAMC-D-15-0203.1.
Park, H. S., A. V. Ryzhkov, D. S. Zrnić, and K.-E. Kim, 2009: The hydrometeor classification algorithm for the polarimetric WSR-88D: Description and application to an MCS. Wea. Forecasting, 24, 730–748, https://doi.org/10.1175/2008WAF2222205.1.
Ryzhkov, A. V., P. Zhang, R. Doviak, and C. Kessinger, 2002: Discrimination between weather and sea clutter using Doppler and dual-polarization weather radars. Proc. 27th General Assembly of the International Union of Radio Science, Maastricht, Netherlands, International Union of Radio Science, F1.P.3, http://www.ursi.org/proceedings/procGA02/papers/p1383.pdf.
Ryzhkov, A. V., T. J. Schuur, D. W. Burgess, P. L. Heinselman, S. E. Giangrande, and D. S. Zrnić, 2005: The Joint Polarization Experiment: Polarimetric rainfall measurements and hydrometeor classification. Bull. Amer. Meteor. Soc., 86, 809–824, https://doi.org/10.1175/BAMS-86-6-809.
Ryzhkov, A. V., M. R. Kumjian, S. M. Ganson, and P. Zhang, 2013: Polarimetric radar characteristics of melting hail. Part II: Practical implications. J. Appl. Meteor. Climatol., 52, 2871–2886, https://doi.org/10.1175/JAMC-D-13-074.1.
Scharfenberg, K. A., and Coauthors, 2005: The Joint Polarization Experiment: Polarimetric radar in forecasting and warning decision making. Wea. Forecasting, 20, 775–788, https://doi.org/10.1175/WAF881.1.
Schuur, T. J., A. V. Ryzhkov, and P. L. Heinselman, 2003: Observations and classification of echoes with the polarimetric WSR-88D radar. NOAA/National Severe Storms Laboratory Rep., 46 pp.
Snyder, J. C., and A. V. Ryzhkov, 2015: Automated detection of polarimetric tornadic debris signatures using a hydrometeor classification algorithm. J. Appl. Meteor. Climatol., 54, 1861–1870, https://doi.org/10.1175/JAMC-D-15-0138.1.
Stepanian, P. M., and K. G. Horton, 2015: Extracting migrant flight orientation profiles using polarimetric radar. IEEE Trans. Geosci. Remote Sens., 53, 6518–6528, https://doi.org/10.1109/TGRS.2015.2443131.
Straka, J. M., 1996: Hydrometeor fields in a supercell storm as deduced from dual-polarization radar. Preprints, 18th Conf. on Severe Local Storms, San Francisco, CA, Amer. Meteor. Soc., 551–554.
Straka, J. M., and D. S. Zrnić, 1993: An algorithm to deduce hydrometeor types and contents from multi-parameter radar data. Preprints, 26th Conf. on Radar Meteorology, Norman, OK, Amer. Meteor. Soc., 513–515.
Thompson, E. J., S. A. Rutledge, B. Dolan, V. Chandrasekar, and B. L. Cheong, 2014: A dual-polarization radar hydrometeor classification algorithm for winter precipitation. J. Atmos. Oceanic Technol., 31, 1457–1481, https://doi.org/10.1175/JTECH-D-13-00119.1.
Williams, E. R., M. F. Donovan, D. J. Smalley, R. G. Hallowell, E. Griffin, K. T. Hood, and B. J. Bennett, 2015: The 2013 Buffalo Area Icing and Radar Study (BAIRS). MIT Lincoln Laboratory Rep. ATC-419, 132 pp.
