• Berenguer, M., , Sempere-Torres D. , , Corral C. , , and Sanchez-Diezma R. , 2006: A fuzzy logic technique for identifying nonprecipitating echoes in radar scans. J. Atmos. Oceanic Technol., 23, 11571180, doi:10.1175/JTECH1914.1.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. M., 1995: Neural Networks for Pattern Recognition.Oxford University Press, 512 pp.

  • Chandrasekar, V., , Kernen R. , , Lin S. , , and Moisseev D. , 2013: Recent advances in classification of observations from dual polarization weather radars. Atmos. Res., 119, 97111, doi:10.1016/j.atmosres.2011.08.014.

    • Search Google Scholar
    • Export Citation
  • Chanthavong, V., , Holmes J. , , Keränen R. , , Paris D. , , Selzler J. , , Siggia A. , , and Stordell T. , 2010: Mitigation of sea clutter and other nonstationary echoes based on general purpose polarimetric echo identification. Proc. Sixth European Conf. on Radar in Meteorology and Hydrology, Sibiu, Romania, National Meteorological Administration, P3.7. [Available online at http://www.erad2010.org/pdf/POSTER/03_Quality/07_ERAD2010_0307.pdf.]

  • Charalampidis, D., , Kasparis T. , , and Jones W. , 2002: Removal of nonprecipitation echoes in weather radar using multifractals and intensity. IEEE Trans. Geosci. Remote Sens., 40, 11211132, doi:10.1109/TGRS.2002.1010899.

    • Search Google Scholar
    • Export Citation
  • da Silveira, R. B., , and Holt A. R. , 2001: An automatic identification of clutter and anomalous propagation in polarization-diversity weather radar data using neural networks. IEEE Trans. Geosci. Remote Sens., 39, 17771788, doi:10.1109/36.942556.

    • Search Google Scholar
    • Export Citation
  • Efron, B., 1979: Bootstrap methods: Another look at the jackknife. Ann. Stat., 7, 126, doi:10.1214/aos/1176344552.

  • Figueras i Ventura, J., , and Tabary P. , 2013: The new French operational polarimetric radar rainfall rate product. J. Appl. Meteor. Climatol.,52, 1817–1835, doi:10.1175/JAMC-D-12-0179.1.

  • Friedrich, K., , Hagen M. , , and Einfalt T. , 2006: A quality control concept for radar reflectivity, polarimetric parameters, and Doppler velocity. J. Atmos. Oceanic Technol., 23, 865887, doi:10.1175/JTECH1920.1.

    • Search Google Scholar
    • Export Citation
  • Fulton, R., , Breidenback D. , , Miller D. , , and O’Bannon T. , 1998: The WSR-88D rainfall algorithm. Wea. Forecasting, 13, 377395, doi:10.1175/1520-0434(1998)013<0377:TWRA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Giuli, D., , Gherardelli M. , , Freni A. , , Seliga T. A. , , and Aydin K. , 1991: Rainfall and clutter discrimination by means of dual-linear polarization radar measurements. J. Atmos. Oceanic Technol., 8, 777789, doi:10.1175/1520-0426(1991)008<0777:RACDBM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Gourley, J. J., , Tabary P. , , and du Chatelet J. P. , 2007: A fuzzy logic algorithm for the separation of precipitating from nonprecipitating echoes using polarimetric radar observations. J. Atmos. Oceanic Technol., 24, 14391451, doi:10.1175/JTECH2035.1.

    • Search Google Scholar
    • Export Citation
  • Grecu, M., , and Krajewski W. , 2000: An efficient methodology for detection of anomalous propagation echoes in radar reflectivity data using neural networks. J. Atmos. Oceanic Technol., 17, 121129, doi:10.1175/1520-0426(2000)017<0121:AEMFDO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Heidke, P., 1926: Berechnung des erfolges und der gute der windstarkvorhersagen im sturmwarnungsdienst. Geogr. Ann., 8, 301349.

  • Holt, A., , McGuinness R. , , Torlaschi E. , , and Bebbington D. , 1992: On the use of polarization-diversity to aid recognition of ground clutter and anomalous propagation in radar echoes of storms. Proc. Open Symp. on Wave Propagation and Remote Sensing, Ravenscar, North Yorkshire, United Kingdom, URSI, 2.4.1–2.4.5.

  • Islam, T., , Rico-Ramirez M. , , Han D. , , and Srivastava P. , 2012: Artificial intelligence techniques for clutter identification with polarimetric radar signatures. Atmos. Res., 109–110, 95113, doi:10.1016/j.atmosres.2012.02.007.

    • Search Google Scholar
    • Export Citation
  • Istok, M. J., and Coauthors, 2009: WSR-88D dual polarization initial operational capabilities. Preprints, 25th Conf. on Int. Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Phoenix, AZ, Amer. Meteor. Soc., 15.5. [Available online at https://ams.confex.com/ams/89annual/techprogram/paper_148927.htm.]

  • Kessinger, C., , Ellis S. , , and Van Andel J. , 2003: The radar echo classifier: A fuzzy logic algorithm for the WSR-88D. Preprints, Third Conf. on Artificial Intelligence Applications to the Environmental Science, Long Beach, CA, Amer. Meteor. Soc., P1.6. [Available online at https://ams.confex.com/ams/annual2003/techprogram/paper_54946.htm.]

  • Lakshmanan, V., 2012: Automating the Analysis of Spatial Grids: A Practical Guide to Data Mining Geospatial Images for Human and Environmental Applications. Geotechnologies and the Environment, Vol. 6, Springer, 320 pp.

  • Lakshmanan, V., , Fritz A. , , Smith T. , , Hondl K. , , and Stumpf G. J. , 2007a: An automated technique to quality control radar reflectivity data. J. Appl. Meteor. Climatol., 46, 288305, doi:10.1175/JAM2460.1.

    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., , Smith T. , , Stumpf G. J. , , and Hondl K. , 2007b: The Warning Decision Support System–Integrated Information. Wea. Forecasting, 22, 596612, doi:10.1175/WAF1009.1.

    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., , Zhang J. , , and Howard K. , 2010: A technique to censor biological echoes in radar reflectivity data. J. Appl. Meteor. Climatol., 49, 435462, doi:10.1175/2009JAMC2255.1.

    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., , Zhang J. , , Hondl K. , , and Langston C. , 2012: A statistical approach to mitigating persistent clutter in radar reflectivity data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 5, 652662, doi:10.1109/JSTARS.2011.2181828.

    • Search Google Scholar
    • Export Citation
  • Liu, H., , and Chandrasekar V. , 1998: An adaptive neural network scheme for precipitation estimation from radar observations. IGARSS ’98: Sensing and Managing the Environment; 1998 IEEE International Geoscience and Remote Sensing Symposium Proceedings, T. I. Stein, Ed., Vol. 4, IEEE, 1895–1897, doi:10.1109/IGARSS.1998.703687.

  • Lucas, D. J., , Goodman L. S. , , and Fabozzi F. J. , 2006: Collateralized Debt Obligations: Structures and Analysis.Frank J. Fabozzi Series, Wiley Finance, 528 pp.

  • Lynn, R., , and Lakshmanan V. , 2002: Virtual radar volumes: Creation, algorithm access, and visualization. Preprints, 21st Conf. on Severe Local Storms, San Antonio, TX, Amer. Meteor. Soc., P5.3. [Available online at https://ams.confex.com/ams/SLS_WAF_NWP/techprogram/paper_47546.htm.]

  • Ortega, K., , Smith T. , , Manross K. , , Scharfenberg K. , , Witt A. , , Kolodziej A. , , and Gourley J. , 2009: The severe hazards analysis and verification experiment. Bull. Amer. Meteor. Soc., 90, 15191530, doi:10.1175/2009BAMS2815.1.

    • Search Google Scholar
    • Export Citation
  • Otsu, N., 1979: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern., 9, 6266, doi:10.1109/TSMC.1979.4310076.

    • Search Google Scholar
    • Export Citation
  • Park, H., , Ryzhkov A. , , Zrnić D. , , and Kim K. , 2009: The hydrometeor classification algorithm for the polarimetric WSR-88D: Description and application to a MCS. Wea. Forecasting, 24, 730748, doi:10.1175/2008WAF2222205.1.

    • Search Google Scholar
    • Export Citation
  • Rico-Ramirez, M. A., , and Cluckie I. D. , 2008: Classification of ground clutter and anomalous propagation using dual-polarization weather radar. IEEE Trans. Geosci. Remote Sens., 46, 18921904, doi:10.1109/TGRS.2008.916979.

    • Search Google Scholar
    • Export Citation
  • Ryzhkov, A. V., 2007: The impact of beam broadening on the quality of radar polarimetric data. J. Atmos. Oceanic Technol., 24, 729744, doi:10.1175/JTECH2003.1.

    • Search Google Scholar
    • Export Citation
  • Ryzhkov, A. V., , Giangrande S. E. , , Melnikov V. M. , , and Schuur T. J. , 2005: Calibration issues of dual-polarization radar measurements. J. Atmos. Oceanic Technol., 22, 1138–1155, doi:10.1175/JTECH1772.1.

    • Search Google Scholar
    • Export Citation
  • Steiner, M., , and Smith J. , 2002: Use of three-dimensional reflectivity structure for automated detection and removal of nonprecipitating echoes in radar data. J. Atmos. Oceanic Technol., 19, 673686, doi:10.1175/1520-0426(2002)019<0673:UOTDRS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Stumpf, G., , Witt A. , , Mitchell E. D. , , Spencer P. L. , , Johnson J. T. , , Eilts M. D. , , Thomas K. W. , , and Burgess D. W. , 1998: The National Severe Storms Laboratory mesocyclone detection algorithm for the WSR-88D. Wea. Forecasting, 13, 304326, doi:10.1175/1520-0434(1998)013<0304:TNSSLM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Sun, J., , and Wilson J. W. , 2003: The assimilation of radar data for weather prediction. Radar and Atmospheric Science: A Collection of Essays in Honor of David Atlas, Meteor. Monogr., No. 52, Amer. Meteor. Soc., 175–198, doi:10.1175/0065-9401(2003)030<0175:TAORDF>2.0.CO;2.

  • Zhang, J., , Wang S. , , and Clarke B. , 2004: WSR-88D reflectivity quality control using horizontal and vertical reflectivity structure. Preprints, 11th Conf. on Aviation, Range, and Aerospace, Hyannis, MA, Amer. Meteor. Soc., P5.4. [Available online at https://ams.confex.com/ams/11aram22sls/techprogram/paper_81858.htm.]

  • Zrnić, D., , and Ryzhkov A. , 1998: Observations of insects and birds with a polarimetric radar. IEEE Trans. Geosci. Remote Sens., 36, 661668, doi:10.1109/36.662746.

    • Search Google Scholar
    • Export Citation
  • Zrnić, D., , and Ryzhkov A. , 1999: Polarimetry for weather surveillance radars. Bull. Amer. Meteor. Soc., 80, 389406, doi:10.1175/1520-0477(1999)080<0389:PFWSR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Zrnić, D., , Ryzhkov A. , , Straka J. , , Liu Y. , , and Vivekanandan J. , 2001: Testing a procedure for automatic classification of hydrometeor types. J. Atmos. Oceanic Technol., 18, 892913, doi:10.1175/1520-0426(2001)018<0892:TAPFAC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • View in gallery

    The QC algorithm of this paper. Data are shown in rectangular boxes, and processing steps are shown in ellipses.

  • View in gallery

    In the simple classifier, the three best univariate predictors are used to classify range gates if the virtual volume is incomplete, such as after a VCP change. The fourth graph shows that classifying a pixel as being precipitation if any two of these conditions are met is better than requiring all or only one of these conditions to be true. In each graph, the three curves depict the POD, FAR, and HSS. The threshold at which the HSS is maximized and the maximum HSS itself are also shown.

  • View in gallery

    One reason that the simple classifier introduced in this paper is better than just a plain ρHV filter is that it correctly handles nonuniform beamfilling: (a) reflectivity and (b) ρHV. Note the drop in ρHV values all along the strong reflectivity echo. A ρHV filter will censor such areas, but because the (c) Z and Zdr values are not affected, the simple classifier will not. (d) The number of criteria satisfied at each pixel is shown.

  • View in gallery

    Pixels are preclassified based on Zdr and ρHV, so as to reduce the number of pixels presented to the NN.

  • View in gallery

    The cumulative distribution function of reflectivity within the training set is shown. The tranche thresholds (10 and 20 dBZ) are indicated by dashed lines.

  • View in gallery

    The piecewise linear fuzzy membership function to represent the rule that the centroids are separated well in reflectivity value.

  • View in gallery

    Bimodal clustering is carried out close to the radar, and connectivity-based clustering is farther away from the radar. Pixelwise probability of precipitation obtained from the NN is averaged within a cluster to yield the final probability field. (top) Four figures show data from KNQA on 21 Sep 2012, while (bottom) four figures show data from KRLX on 1 Oct 2012. In each set (a) reflectivity and (b) clusters with each object shown in a different color, (c) pixelwise probability (red = 1)—note the speckles of red that indicate misclassification, and (d) postprocessed probability—note that the speckles have vanished because of being averaged in with all the other correctly classified pixels.

  • View in gallery

    The impact of postprocessing the output of the pixelwise NN. Compare these HSS values with those of the univariate predictions in Fig. 2.

  • View in gallery

    (left) A meteorologist manually classified the training data into two categories in order to train the NN. (right) The data close to the radar have been identified as bad (blue) and the echoes farther away as good (red).

  • View in gallery

    The QC algorithm of this paper applied to a few of the cases of the training dataset. (left) The images are of the raw reflectivity composite. (right) The images depict the QCed data. (a),(b) Data from KEWX around 1000 UTC 6 May 2012 show significant artifacts being removed. (c),(d) Data from KFTG around 2100 UTC 24 Feb 2013 show snow being retained. (e),(f) Data from KJAX around 0900 UTC 23 Sep 2012 show bioscatter being removed. (g),(h) Data from KNQA around 1100 UTC 21 Sep 2012 show ground clutter being removed.

  • View in gallery

    The QC algorithm of this paper applied to a few of the cases of the independent dataset. (left) The images are of the raw reflectivity composite. (right) The images depict the QCed data. (a),(b) Data from KAMA around 1800 UTC 15 Feb 2012 show ground clutter and bioscatter near the radar being removed but light precipitation being retained. (c),(d) Data from KCLE around 1700 UTC 15 Feb 2012 show snow being retained. (e),(f) Data from KTLX around 2300 UTC 9 Mar 2013 show bioscatter and wind turbine clutter being removed. (g),(h) Data from KMHX around 0100 UTC 27 Aug 2011 show data from a hurricane being retained.

  • View in gallery

    Performance of QC methods on an independent dataset.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1 1 1
PDF Downloads 2 2 2

Quality Control of Weather Radar Data Using Polarimetric Variables

View More View Less
  • 1 Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, and National Severe Storms Laboratory, Norman, Oklahoma
© Get Permissions
Full access

Abstract

Because weather radar data are commonly employed in automated weather applications, it is necessary to censor nonmeteorological contaminants, such as bioscatter, instrument artifacts, and ground clutter, from the data. With the operational deployment of a widespread polarimetric S-band radar network in the United States, it has become possible to fully utilize polarimetric data in the quality control (QC) process. At each range gate, a pattern vector consisting of the values of the polarimetric and Doppler moments, and local variance of some of these features, as well as 3D virtual volume features, is computed. Patterns that cannot be preclassified based on correlation coefficient ρHV, differential reflectivity Zdr, and reflectivity are presented to a neural network that was trained on historical data. The neural network and preclassifier produce a pixelwise probability of precipitation at that range gate. The range gates are then clustered into contiguous regions of reflectivity, with bimodal clustering carried out close to the radar and clustering based purely on spatial connectivity farther away from the radar. The pixelwise probabilities are averaged within each cluster, and the cluster is either retained or censored depending on whether this average probability is greater than or less than 0.5. The QC algorithm was evaluated on a set of independent cases and found to perform well, with a Heidke skill score (HSS) of about 0.8. A simple gate-by-gate classifier, consisting of three simple rules, is also introduced in this paper and can be used if the full QC method is not able to be applied. The simple classifier has an HSS of about 0.6 on the independent dataset.

Corresponding author address: Valliappa Lakshmanan, CIMMS, University of Oklahoma, 120 David L. Boren Blvd., Norman, OK 73072. E-mail: lakshman@ou.edu

Abstract

Because weather radar data are commonly employed in automated weather applications, it is necessary to censor nonmeteorological contaminants, such as bioscatter, instrument artifacts, and ground clutter, from the data. With the operational deployment of a widespread polarimetric S-band radar network in the United States, it has become possible to fully utilize polarimetric data in the quality control (QC) process. At each range gate, a pattern vector consisting of the values of the polarimetric and Doppler moments, and local variance of some of these features, as well as 3D virtual volume features, is computed. Patterns that cannot be preclassified based on correlation coefficient ρHV, differential reflectivity Zdr, and reflectivity are presented to a neural network that was trained on historical data. The neural network and preclassifier produce a pixelwise probability of precipitation at that range gate. The range gates are then clustered into contiguous regions of reflectivity, with bimodal clustering carried out close to the radar and clustering based purely on spatial connectivity farther away from the radar. The pixelwise probabilities are averaged within each cluster, and the cluster is either retained or censored depending on whether this average probability is greater than or less than 0.5. The QC algorithm was evaluated on a set of independent cases and found to perform well, with a Heidke skill score (HSS) of about 0.8. A simple gate-by-gate classifier, consisting of three simple rules, is also introduced in this paper and can be used if the full QC method is not able to be applied. The simple classifier has an HSS of about 0.6 on the independent dataset.

Corresponding author address: Valliappa Lakshmanan, CIMMS, University of Oklahoma, 120 David L. Boren Blvd., Norman, OK 73072. E-mail: lakshman@ou.edu

1. Introduction

Weather radar data are routinely employed for precipitation estimation (Fulton et al. 1998), hail detection (Ortega et al. 2009), nowcasting (Lakshmanan et al. 2007b), numerical weather prediction (Sun and Wilson 2003), and severe weather diagnosis (Stumpf et al. 1998). Such automated applications operating on weather radar data are adversely affected by nonmeteorological radar echoes, such as insects, birds, sun spikes, test patterns, ground clutter, and anomalous propagation (AP).

Quality control (QC) algorithms to remove such nonmeteorological echoes (e.g., by Grecu and Krajewski 2000; Charalampidis et al. 2002; Steiner and Smith 2002; Kessinger et al. 2003; Zhang et al. 2004; Lakshmanan et al. 2007a, 2012) concentrate primarily on anomalous propagation and ground clutter because such contamination consists of high reflectivity values. Low-reflectivity contamination that persists over long periods of time, such as that caused by biological targets, is difficult to remove. It requires specialized classification procedures, such as the linear fit of reflectivity to distance that was proposed in Lakshmanan et al. (2010). Even though the reflectivity values are low, such low-valued contamination when accumulated over long periods leads to significant problems in estimating precipitation. A second shortcoming of existing nonpolarimetric quality control methods is that they operate on a volume-scan basis [virtual volumes in the case of Lakshmanan et al. (2007a)] and can therefore not remove contamination at lower-elevation scans of the radar while retaining precipitation echoes at higher-elevation scans.

The deployment of polarimetric radar (Zrnić and Ryzhkov 1999) in the operational weather radar network in the United States (Istok et al. 2009) offers the opportunity to take advantage of polarimetric moments to better address the issue of biological targets (Zrnić and Ryzhkov 1998) and to improve existing quality control methods. Quality assessment methods that take advantage of polarimetric moments have been devised and carried out on a variety of research and quasi-operational C- and S-band polarimetric weather radars (Friedrich et al. 2006). In addition, a hydrometeor classification algorithm (HCA) that attempts to classify each range gate into one of several meteorological and nonmeteorological categories has also been developed on research radar systems (Zrnić et al. 2001; Park et al. 2009; Chandrasekar et al. 2013). Although the calibration issues of dual-polarization radar measurements (Ryzhkov et al. 2005) could be addressed case by case in research studies, an automated real-time algorithm needs to be tolerant of shortcomings in calibration across a radar network. Therefore, there is a need for a fully automated quality control application that is capable of censoring weather radar data in real time fully utilizing the polarimetric moments.

Techniques such as a fuzzy logic approach (e.g., Kessinger et al. 2003; Berenguer et al. 2006) or a neural network (NN) (e.g., Grecu and Krajewski 2000; Liu and Chandrasekar 1998; Lakshmanan et al. 2007a) have achieved success in separating contamination using nonpolarimetric radar. More recent studies have begun to incorporate polarimetric variables with the aforementioned technique(s) with better results (e.g., da Silveira and Holt 2001; Giuli et al. 1991; Holt et al. 1992; Gourley et al. 2007; Rico-Ramirez and Cluckie 2008; Chanthavong et al. 2010]. Such techniques are commonly referred to as artificial intelligence (AI) methods [see Islam et al. (2012) for a comparison] but with notable distinctions. For example, fuzzy logic is primarily heuristics driven by human experience, although some aspects of the membership function may be based on analysis of observations, as in Gourley et al. (2007). Neural networks, unlike fuzzy logic, are completely data driven (i.e., it falls into the category of supervised learning methods) and can be proved to be optimal approximations of underlying distributions.

Fuzzy logic–based identification methods for data quality classification have been developed by Gourley et al. (2007) and Chanthavong et al. (2010), with the method of Gourley et al. (2007) being employed operationally in France (Figueras i Ventura and Tabary 2013) as a preprocessor for a rainfall-rate estimation algorithm. Gourley et al. (2007) set fuzzy membership functions and weights based on kernel densities derived from a single day of data, whereas Chanthavong et al. (2010) allow these to be tweaked by the end user. This paper extends such previous work on data quality classification based on polarimetric variables by carrying out extensive data-driven analysis on a diverse set of cases for the selection of thresholds and weights. Although this paper describes the impact of quality control primarily on the radar reflectivity field, the same QC mask may be applied to remove artifacts from other moments.

The rest of this paper is organized as follows. The quality control algorithm is described in section 2, and the setup of the machine learning training regimen is explained in section 3. The algorithm is applied to a set of independent data cases, and the skill of the algorithm on those independent cases is quantified and discussed in section 4.

2. Algorithm

The QC algorithm operates on six moments available from the Weather Surveillance Radar-1988 Doppler (WSR-88D) polarimetric radar: reflectivity (Z), velocity (V), spectrum width (SPW), correlation coefficient (ρHV), differential reflectivity (Zdr), and differential phase on propagation (ϕDP). The velocity data are the raw data, before any dealiasing is applied. The data are collected tilt by tilt and placed into virtual volumes (Lynn and Lakshmanan 2002) consisting of the latest moment data at each tilt. The probability of precipitation is computed using a neural network once all six moments have been received for the tilt. The reflectivity value at each range gate within that tilt is either retained or removed based on the computed probability value. Unlike Lakshmanan et al. (2012), who correct the reflectivity value by subtracting out an estimated ground clutter power, this algorithm has no way to estimate the power due to ground clutter. The QC process is shown as a flowchart in Fig. 1 and described in the remainder of this section.

Fig. 1.
Fig. 1.

The QC algorithm of this paper. Data are shown in rectangular boxes, and processing steps are shown in ellipses.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

a. Simple classifier

If the volume coverage pattern (VCP) of the radar is changed by the operator, then the elevation angles received from the radar will change. In such cases, the entire volume is cleared and a new virtual volume1 is built. Because the QC algorithm operates on virtual volumes, the QC algorithm is faced with partial data during the first volume after a VCP change. Such partial data are subjected to a simple classification method.

The simple classifier, applied only when sufficient data are not present, retains the reflectivity at the range gate if it meets at least two of these three conditions:

  1. Reflectivity value ≥ 3 dBZ
  2. ρHV ≥ 0.9
  3. |Zdr| < 2.3 dB
These thresholds were selected by maximizing the univariate Heidke skill score (HSS; Heidke 1926) over the training dataset (see Fig. 2; section 3). In Fig. 2, the probability of detection (POD) refers to the probability of detecting good data, so that a POD of 1 and a false alarm rate (FAR) of 1 implies that no echoes are censored. Because reflectivity values on the WSR-88D are quantized in intervals of 0.5 dBZ, the rule Z > 3.1 is equivalent to Z ≥ 3.
Fig. 2.
Fig. 2.

In the simple classifier, the three best univariate predictors are used to classify range gates if the virtual volume is incomplete, such as after a VCP change. The fourth graph shows that classifying a pixel as being precipitation if any two of these conditions are met is better than requiring all or only one of these conditions to be true. In each graph, the three curves depict the POD, FAR, and HSS. The threshold at which the HSS is maximized and the maximum HSS itself are also shown.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

Because the simple classifier is composed of only criteria on the values at the range gate itself (without any dependence on virtual volumes or even the complete tilt and without difficult-to-implement operations, such as clustering), it may be useful in situations where the full QC method of this paper is not able to be applied. In particular, it should be noted that the simple classifier is better than a purely ρHV-based filter in terms of its HSS. One reason that it is better can be inferred from a case of nonuniform beam filling (Ryzhkov 2007), shown in Fig. 3. In such areas, the reflectivity and Zdr criteria both hold—it is only the ρHV field that is impacted by nonuniform beam filling.

Fig. 3.
Fig. 3.

One reason that the simple classifier introduced in this paper is better than just a plain ρHV filter is that it correctly handles nonuniform beamfilling: (a) reflectivity and (b) ρHV. Note the drop in ρHV values all along the strong reflectivity echo. A ρHV filter will censor such areas, but because the (c) Z and Zdr values are not affected, the simple classifier will not. (d) The number of criteria satisfied at each pixel is shown.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

The robustness of the thresholds used in the simple classifier can be gauged using bootstrapping (Efron 1979), a resampling technique for estimating the confidence interval of nondifferentiable statistics. In bootstrapping, a sample of the same size as the training dataset is created by random sampling with replacement of that dataset. The optimal threshold was computed on the sample by determining the threshold that maximizes the Heidke skill score. By repeating this process 100 times, we were able to nonparametrically estimate the 95% confidence bounds of the four thresholds used in the simple classifier. The confidence intervals for Zdr, ρHV, and the number of criteria are zero width and centered around 2.3, 0.9, and 2, respectively. The 95% confidence interval for the reflectivity value is centered around 3 dBZ, but it also includes 2.5 dBZ. It is clear, therefore, that all four thresholds are quite robust.

b. Preclassification

Some range gates are preclassified, so as to ensure that the features provided as input to the neural network are computed appropriately. For example, at far ranges from the radar, only reflectivity data may have been collected and polarimetric data such as Zdr and ρHV may be unavailable. To avoid presenting the NN with missing data, such range gates are preclassified as meteorological if their reflectivity value is above −14 dBZ. One set of features used as input to the NN consists of local variance estimates. Because the local variance estimates will be poor at the edges of echoes, such gates are preclassified as “do not care” and are assigned a pixelwise probability of precipitation of 0.5. Because of the postprocessing, these pixels will take on the characteristics of the inside of the echoes.

Other pixels are preclassified based on the graphs in Fig. 4 (these graphs differ from those of Fig. 2, in that the range of values on the x axis is larger) so as to reduce the amount of data on which the neural network needs to be trained. The thresholds for preclassification were chosen so as to retain more than 99.9% of the meteorological echoes while removing as much of the nonmeteorological echoes as possible. Thus, pixels with ρHV less than 0.6 or |Zdr| greater than 6 were preclassified as nonmeteorological, and the probability of precipitation at those range gates was set to zero. Pixels with reflectivity values below −14 dBZ were also assumed to be nonmeteorological.

Fig. 4.
Fig. 4.

Pixels are preclassified based on Zdr and ρHV, so as to reduce the number of pixels presented to the NN.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

c. Feature computation

At each range gate, a number of features based on the values of the radar moments are computed. A neural network was trained with these features, and the output of the neural network is used to classify the pixel as either meteorological or nonmeteorological.

The first set of features corresponds to the values of the radar moments at the given range gate: the absolute of the velocity value, SPW, the reflectivity value in dBZ, Zdr, ρHV, and the ϕDP.

In addition, a set of vertical features (Grecu and Krajewski 2000; Lakshmanan et al. 2007a) is computed from the virtual volume of Z:

  1. The maximum dBZ in the vertical column that includes the given range gate.
  2. The maximum height at which the Z value is greater than −14 dBZ.
  3. The Z value (in dBZ) at 3 km in height from tilts greater than 1°.
  4. The difference in Z values between the lowest tilt and next higher tilt where the elevation is more than 1°; thus, in a VCP where the elevation angles are 0.5°, 0.9°, and 1.45°, the difference will be between the 0.5° and 1.45° tilts.
Because a mechanically rotating radar senses different tilts at different times, any 3D feature will be subject to misalignment of features as they move between the times that they are sensed at different tilts. We do not explicitly account for this error (by, e.g., moving radar echoes along their computed trajectory). However, because this error is present in the training data as well as at run time, this error is implicitly factored in when the 3D features are weighted relative to other features.

The third set of features consists of the local variance, computed in a 5 × 5 neighborhood centered around the range gate. The local variance of Z, Zdr, and ρHV is computed as in Park et al. (2009).

Finally, the result of the simple classifier (the number of criteria met) is also computed and used as an input to the neural network.

d. Tranches

A total of four neural networks were trained. The first neural network is used for range gates where the velocity data are unavailable or range folded. Velocity data may be unavailable at some ranges with valid reflectivity values because, on the Next Generation Weather Radar (NEXRAD) system, the range of some reflectivity tilts is longer than that of the corresponding velocity tilt. The next three neural networks deal with tranches2 based on the Z value at the range gate, with the data ranges chosen so as to equalize the number of range gates in each of the tranches (see Fig. 5).

Fig. 5.
Fig. 5.

The cumulative distribution function of reflectivity within the training set is shown. The tranche thresholds (10 and 20 dBZ) are indicated by dashed lines.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

Tranching based on the Z values was carried out because of limitations of our neural network training software (it could not handle more patterns than the maximum 32-byte signed integer) and so as to allow each tranche to “specialize” in removing a particular variety of nonmeteorological echo. Thus, the Z < 10 tranche would be focused mainly on insects, the 10 ≤ Z < 20 tranche on birds, and Z > 20 mostly on widespread bioscatter and anomalous propagation (while retaining good data, such as nonuniform beamfilling).

The set of computed features at a range gate is presented to one of the four neural networks depending on the velocity and reflectivity values at that range gate. The architecture of the neural network employed in this study is such that the probability that a range gate has meteorological echo (pixelwise probability) is computed from the features, xis, using
e1
where S is the sigmoid or logistic function S(x) = 1/(1 + ex) and tanh is the hyperbolic tangent function tanh (x) = (e2x − 1)/(e2x + 1). The weights (ws), biases (bs and bhs), and the number of hidden nodes (the number of js) were obtained by numerical optimization on a training dataset and then used unchanged in real time (i.e., the weights are not updated based on new data). Because a total of four neural networks were trained (one for each tranche), there are four sets of ws and bs.

e. Postprocessing

It is clear that, in the QC algorithm thus far, spatial connectivity has been taken into account only indirectly (by preclassifying pixels on the edges of storms and by the use of local variance features). However, adjacent pixels tend to have the same (precipitation/nonprecipitation) classification. Spatial correlation of the probability values is explicitly incorporated into the QC algorithm through postprocessing the pixelwise probability of the precipitation field.

Similar to Lakshmanan et al. (2007a), data from the reflectivity tilt are clustered into contiguous regions of reflectivity, and the pixelwise probabilities are averaged within each region. Each region of reflectivity is then either retained or censored depending on whether the average probability of precipitation is greater than or less than 0.5.

The tilt is first smoothed and speckle filtered using a 2.5 km × 2.5 km median filter (i.e., the filter operates on the polar grid and uses more radials close to the radar and fewer radials farther away from the radar). The median filter is clamped to a minimum size of 5 × 5 (i.e., it will comprise of at least five radials and five gates at each radial). Bimodal clustering is carried out until N gates and connectivity-based clustering beyond N gates, where N is the closest gate that is at least 150 km in range or 2 km in height from the radar.

Farther away from the radar, where the beam heights are relatively far from the ground, it is enough to simply cluster all contiguous pixels with valid reflectivity values and average their pixelwise probabilities to decide whether to retain a cluster. Close to the radar, however, it is common for nonmeteorological echoes, such as clutter or insects, to be adjacent to meteorological echoes. Therefore, close to the radar bimodal clustering is carried out, whereby the pixels with valid reflectivity values are first divided into two categories and clusters are created to consist of contiguous areas of the same category. Thus, when ground clutter (high reflectivity) is mixed in with precipitation (relatively lower reflectivity), the two areas will belong to two different categories and enable the higher-reflectivity pixels to be censored. Similarly, when bioscatter (low reflectivity) is mixed in with precipitation (relatively higher reflectivity), the two areas will belong to two different categories but this time enabling the lower-reflectivity regions to be censored.

In bimodal clustering, a histogram of reflectivity values within N gates of the radar is created and the method of Otsu (1979) is employed to determine the threshold that best separates the reflectivity values close to the radar into two categories. Then, the centroid of all the range gates below the threshold (range1, azimuth1, Z1) is computed, as is the centroid of the ranges above the threshold (range2, azimuth2, Z2).

The radar reflectivity field close to the radar is assumed to have two distinct categories near the radar if the two centroids are separated well in both reflectivity value and range. This rule is expressed using fuzzy logic:
e2
where and are piecewise linear fuzzy membership functions that are zero at differences of 5 dBZ and 5 km and one at differences of 15 dBZ and 15 km, respectively (see Fig. 6).
Fig. 6.
Fig. 6.

The piecewise linear fuzzy membership function to represent the rule that the centroids are separated well in reflectivity value.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

If it is determined that the radar reflectivity field has two distinct categories (i.e., if fbimodal > 0.5), then range gates are then assigned to one of the two categories by evaluating a fuzzy membership function at each range gate within the range N of the radar:
e3
where p is the pixelwise probability of precipitation; and fZ and fr are piecewise linear fuzzy membership functions that are zero at Z1 and range1 and one at Z2 and range2, respectively. All pixels beyond a range of N are assigned to the second category. On the other hand, if it is determined that the radar reflectivity field does not have two distinct categories near the radar, then all the pixels are assigned to a single category.

Once the pixels have been categorized, region growing (Lakshmanan 2012) is carried out to cluster the image into regions of reflectivity. The process is illustrated on two sample cases in Fig. 7. The pixelwise probabilities are averaged within each cluster (a weighted average is computed, with the weight given by the dBZ value at the range gate). This average probability is used to determine whether to retain all the echoes within the cluster. Postprocessing by averaging the probability of precipitation within a cluster helps to correct for the occasional misclassification of range gates by the neural network.

Fig. 7.
Fig. 7.

Bimodal clustering is carried out close to the radar, and connectivity-based clustering is farther away from the radar. Pixelwise probability of precipitation obtained from the NN is averaged within a cluster to yield the final probability field. (top) Four figures show data from KNQA on 21 Sep 2012, while (bottom) four figures show data from KRLX on 1 Oct 2012. In each set (a) reflectivity and (b) clusters with each object shown in a different color, (c) pixelwise probability (red = 1)—note the speckles of red that indicate misclassification, and (d) postprocessed probability—note that the speckles have vanished because of being averaged in with all the other correctly classified pixels.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

It is interesting to note that, on the training dataset, postprocessing the output of the neural network by incorporating continuity constraints serves both to stabilize and to decrease the overall skill (see Fig. 8). The best HSS on the pixelwise NN is 0.78, but it is attained at a threshold of 0.3 and is quite sensitive to the choice of this threshold. On the other hand, the postprocessed NN attains its maximum skill at 0.5 and is relatively insensitive to the choice of threshold. Thus, postprocessing trades skill for robustness. As expected, the HSS values obtained by using the neural network are much larger than those obtainable from univariate predictions (see Fig. 2).

Fig. 8.
Fig. 8.

The impact of postprocessing the output of the pixelwise NN. Compare these HSS values with those of the univariate predictions in Fig. 2.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

3. Machine learning

The neural networks for the four tranches were trained on data that were hand truthed. Cases were identified, taking care to include both “good” (rain, snow, etc.) data and “bad” (birds, insects, AP, artifacts, etc.) data. To avoid the problem of partial virtual volumes, two consecutive volume scans were obtained for each data case and training was carried out on the second volume scan’s data. The cases chosen for training are shown in Table 1.

Table 1.

Data cases used for training the neural networks. AFB = air force base.

Table 1.

A meteorologist examined echoes on radar in the second volume scan of each data case and identified those that did not correspond to precipitation (see Fig. 9). Radar data and other observations, such as rain gauges, were employed to determine whether a radar echo was to be retained or be censored. From the areas of bad data marked by the meteorologist, a classification was created for each range gate. The goal of the neural network was to mimic this human classification. Human classification does not get every range gate correct—in particular, when storm cells are embedded inside bioscatter, not all bioscatter is identified by the meteorologist. However, since the number of training patterns is in the millions, minor classification errors by humans can be ignored.

Fig. 9.
Fig. 9.

(left) A meteorologist manually classified the training data into two categories in order to train the NN. (right) The data close to the radar have been identified as bad (blue) and the echoes farther away as good (red).

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

The efficiency of preclassification and of the simple classifier can be quantified by comparing the performance of those rules (such as ρHV > 0.9) against the human classification. Graphs such as those in Figs. 2 and 4 were created by varying the thresholds and computing the probability of detection, false alarm rate, and Heidke skill score of the rule as applied to the training dataset and employing the human classification as truth.

Range gates that are not preclassified are to be classified by the neural network. Consequently, the input features and the truth value (the “pattern”) were presented to a backpropagation neural network, with one pattern for each range gate. The patterns were sampled so that the data presented to the neural network had equal numbers of meteorological and nonmeteorological patterns. In the training dataset, there were more meteorological patterns (1s) than nonmeteorological patterns (0s). Therefore, not all the 1s were used; instead, the 1s used were randomly selected so that there were equal numbers of 0s and 1s. This random selection ratio varied from 3% in the case of the tranch covering Z values greater than 20 dBZ to 50% in the case of the tranche covering Z values less than 10 dBZ.

The neural network consisted of one layer of input nodes, one layer of hidden nodes, and a single output node. Following the recommendations in Bishop (1995), a sigmoid function was used as the activation function of the input and output nodes, the activation function of the hidden nodes was tanh, and the error measure being minimized was the cross entropy. The cross-entropy error measure is given by , where N is the total number of patterns in the dataset, y is the output of the neural network, the first summation is carried out over the nonmeteorological patterns, and the second over the meteorological ones. Different choices were tried for the number of hidden nodes (in the range 6–12) and the number of hidden nodes at which the cross entropy was minimum was chosen as the final architecture. The neural network was trained on the patterns extracted from the training dataset, but the training was subjected to early stopping. Early stopping was carried out by retaining a second dataset, termed the validation dataset, and stopping the neural network training when the error measure on the validation dataset started to increase. The validation set consisted of radar volume scans approximately 30 min from the training dataset. These cases tend to exhibit the same phenomena as those of the training dataset, but slightly changed. The validation set was subject to the same processing steps (preclassification, virtual volumes, tranching, sampling) as the training dataset. The validation set, by functioning as a slightly perturbed version of the training set, limits overfitting.

Besides the list of input features listed in section 2, several other candidate features, such as the texture feature of Steiner and Smith (2002), a simplified hydrometeor classification algorithm (HCA; Park et al. 2009), surface temperature at the radar site (Lakshmanan et al. 2010), and various statistics of radar moments computed within a cluster, were experimented with during the study. These inputs were discarded when their inclusion did not improve the HSS of the pixelwise neural network on the training dataset. However, a formal feature importance study has yet to be carried out.

The performance of the trained neural network and postprocessing on a few selected cases of the training dataset is shown in Fig. 10. The performance of the QC algorithm should be assessed only on independent cases and this will be done in section 4.

Fig. 10.
Fig. 10.

The QC algorithm of this paper applied to a few of the cases of the training dataset. (left) The images are of the raw reflectivity composite. (right) The images depict the QCed data. (a),(b) Data from KEWX around 1000 UTC 6 May 2012 show significant artifacts being removed. (c),(d) Data from KFTG around 2100 UTC 24 Feb 2013 show snow being retained. (e),(f) Data from KJAX around 0900 UTC 23 Sep 2012 show bioscatter being removed. (g),(h) Data from KNQA around 1100 UTC 21 Sep 2012 show ground clutter being removed.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

4. Results

The algorithm was applied to a set of independent cases and its skill scores computed. A list of the cases is shown in Table 2 and a few of the cases are depicted in Fig. 11. It should be noted that these cases are truly independent—no attempt was made to match the diversity or frequency of cases in the training dataset. Thus, for example, the independent dataset includes hurricane cases but the training dataset did not.

Table 2.

Independent data cases used for evaluating the QC algorithm.

Table 2.
Fig. 11.
Fig. 11.

The QC algorithm of this paper applied to a few of the cases of the independent dataset. (left) The images are of the raw reflectivity composite. (right) The images depict the QCed data. (a),(b) Data from KAMA around 1800 UTC 15 Feb 2012 show ground clutter and bioscatter near the radar being removed but light precipitation being retained. (c),(d) Data from KCLE around 1700 UTC 15 Feb 2012 show snow being retained. (e),(f) Data from KTLX around 2300 UTC 9 Mar 2013 show bioscatter and wind turbine clutter being removed. (g),(h) Data from KMHX around 0100 UTC 27 Aug 2011 show data from a hurricane being retained.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

The performance of the QC method of this paper (results shown in Fig. 12) was first evaluated quantitatively on the independent dataset. Figure 12a depicts the performance of the sophisticated radar QC algorithm of Lakshmanan et al. (2007a, 2010) that was developed before the U.S. weather radar network was upgraded to provide polarimetric information. Comparing the performance difference between Figs. 12a and 12f indicates the increase in skill possible through the use of polarimetric variables to perform quality control. There are advantages to the QC method of this paper beyond just the numerical increase shown in the figure. The QC method censors data gate by gate, whereas Lakshmanan et al. (2007a) censor data vertical column by vertical column, making it impossible to retain good data aloft if the lower tilts are contaminated by, say, wind turbine clutter. Second, the QC method of Lakshmanan et al. (2010) employs surface temperature at the radar site to determine whether snow is likely at the location. Therefore, the QC algorithm of Lakshmanan et al. (2010) performs particularly poorly in the transitional months of spring and fall, which were not part of this evaluation dataset. The QC method of this paper does not use surface temperature as one of its inputs and so, this should not be a concern. However, future work will ensure that this is the case.

Fig. 12.
Fig. 12.

Performance of QC methods on an independent dataset.

Citation: Journal of Atmospheric and Oceanic Technology 31, 6; 10.1175/JTECH-D-13-00073.1

Figures 12b–d demonstrate the quality control performance achievable with simple rules on polarimetric variables. The sophisticated nonpolarimetric QC method of Lakshmanan et al. (2010) appears to outperform the simple classifier that consists just of thresholds on the polarimetric variables. However, because the recommended threshold for the nonpolarimetric QC method was 0.5, not 0.9, it could be argued that the polarimetric variables do perform better. Nevertheless, an optimal weighted combination of the radar moments and 3D features provides performance that is inarguably better than a purely nonpolarimetric QC (cf. Fig. 12e and 12a). Postprocessing to take advantage of spatial consistency check further improves the QC performance (cf. Figs. 12e and 12f).

The computational efficiency of the QC algorithm introduced in this paper is data dependent (e.g., gates that do not meet the preclassifier criteria are processed cheaply, and postprocessing via region growing would be cheap on tilts with little or no data). Therefore, “worst case” performance was measured on a volume scan containing significant meteorological and nonmeteorological activity. On a Linux desktop computer with a 64-bit Intel i7–3820 3.60-GHz CPU, the algorithm took 119 s (0.5 s of CPU time) to process that volume scan. Since the fastest WSR-88D volume scans are 4 min apart, this algorithm is capable of processing the data in real time. Further, since 118.5 of those seconds were spent on input and output, the actual performance on real-time infrastructure where the data are already available will be better.

The results shown in Fig. 12 demonstrate that the QC technique of this paper performs well on the independent dataset with 95% of good data and about 20% of bad data retained at a threshold of 0.5. On the independent dataset, the postprocessed output is more skilled than the pixelwise neural network (see Fig. 12e). Both these methods outperform the simple classifier suggested in section 2a (see Fig. 12d) and the simple classifier in turn outperforms univariate thresholds on Zdr or ρHV (see Figs. 12b and 12c).

The QC method introduced in this paper consists of creating a pattern vector at each range gate. The pattern consists of the values of the polarimetric and Doppler moments and local variance of some of these features, as well as 3D features computed in a virtual volume sense. Any patterns that cannot be preclassified based on ρHV, Zdr, and Z are presented to a neural network that was trained on historical data. The neural network and preclassifier combine to produce a pixelwise probability of precipitation at each range gate. The range gates are then clustered into contiguous regions of reflectivity, with bimodal clustering carried out close to the radar and clustering based purely on spatial connectivity farther away from the radar. The pixelwise probabilities are averaged within each cluster, and the cluster is either retained or censored depending on whether this average probability is greater than or less than 0.5. The QC algorithm was evaluated on a set of independent cases and found to perform well, with a Heidke skill score (HSS) of about 0.8. A simple gate-by-gate classifier was also introduced in this paper and can be used if the full QC method is not able to be applied (such as in the case of partial data, or if a “quick and dirty” QC method is desired for research studies). The simple classifier has an HSS of about 0.6 on the independent dataset.

Acknowledgments

Funding for the authors was provided by NOAA’s Office of Oceanic and Atmospheric Research under NOAA–OU Cooperative Agreement NA11OAR4320072, U.S. Department of Commerce. The technique described in this paper has been implemented as part of the Warning Decision Support System–Integrated Information (WDSS-II; Lakshmanan et al. (2007b)) as the program w2qcnndp. It is available for free download (at http://www.wdssii.org/). The weights and biases of Eq. (1) are also available as a text file separate from the code itself.

REFERENCES

  • Berenguer, M., , Sempere-Torres D. , , Corral C. , , and Sanchez-Diezma R. , 2006: A fuzzy logic technique for identifying nonprecipitating echoes in radar scans. J. Atmos. Oceanic Technol., 23, 11571180, doi:10.1175/JTECH1914.1.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. M., 1995: Neural Networks for Pattern Recognition.Oxford University Press, 512 pp.

  • Chandrasekar, V., , Kernen R. , , Lin S. , , and Moisseev D. , 2013: Recent advances in classification of observations from dual polarization weather radars. Atmos. Res., 119, 97111, doi:10.1016/j.atmosres.2011.08.014.

    • Search Google Scholar
    • Export Citation
  • Chanthavong, V., , Holmes J. , , Keränen R. , , Paris D. , , Selzler J. , , Siggia A. , , and Stordell T. , 2010: Mitigation of sea clutter and other nonstationary echoes based on general purpose polarimetric echo identification. Proc. Sixth European Conf. on Radar in Meteorology and Hydrology, Sibiu, Romania, National Meteorological Administration, P3.7. [Available online at http://www.erad2010.org/pdf/POSTER/03_Quality/07_ERAD2010_0307.pdf.]

  • Charalampidis, D., , Kasparis T. , , and Jones W. , 2002: Removal of nonprecipitation echoes in weather radar using multifractals and intensity. IEEE Trans. Geosci. Remote Sens., 40, 11211132, doi:10.1109/TGRS.2002.1010899.

    • Search Google Scholar
    • Export Citation
  • da Silveira, R. B., , and Holt A. R. , 2001: An automatic identification of clutter and anomalous propagation in polarization-diversity weather radar data using neural networks. IEEE Trans. Geosci. Remote Sens., 39, 17771788, doi:10.1109/36.942556.

    • Search Google Scholar
    • Export Citation
  • Efron, B., 1979: Bootstrap methods: Another look at the jackknife. Ann. Stat., 7, 126, doi:10.1214/aos/1176344552.

  • Figueras i Ventura, J., , and Tabary P. , 2013: The new French operational polarimetric radar rainfall rate product. J. Appl. Meteor. Climatol.,52, 1817–1835, doi:10.1175/JAMC-D-12-0179.1.

  • Friedrich, K., , Hagen M. , , and Einfalt T. , 2006: A quality control concept for radar reflectivity, polarimetric parameters, and Doppler velocity. J. Atmos. Oceanic Technol., 23, 865887, doi:10.1175/JTECH1920.1.

    • Search Google Scholar
    • Export Citation
  • Fulton, R., , Breidenback D. , , Miller D. , , and O’Bannon T. , 1998: The WSR-88D rainfall algorithm. Wea. Forecasting, 13, 377395, doi:10.1175/1520-0434(1998)013<0377:TWRA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Giuli, D., , Gherardelli M. , , Freni A. , , Seliga T. A. , , and Aydin K. , 1991: Rainfall and clutter discrimination by means of dual-linear polarization radar measurements. J. Atmos. Oceanic Technol., 8, 777789, doi:10.1175/1520-0426(1991)008<0777:RACDBM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Gourley, J. J., , Tabary P. , , and du Chatelet J. P. , 2007: A fuzzy logic algorithm for the separation of precipitating from nonprecipitating echoes using polarimetric radar observations. J. Atmos. Oceanic Technol., 24, 14391451, doi:10.1175/JTECH2035.1.

    • Search Google Scholar
    • Export Citation
  • Grecu, M., , and Krajewski W. , 2000: An efficient methodology for detection of anomalous propagation echoes in radar reflectivity data using neural networks. J. Atmos. Oceanic Technol., 17, 121129, doi:10.1175/1520-0426(2000)017<0121:AEMFDO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Heidke, P., 1926: Berechnung des erfolges und der gute der windstarkvorhersagen im sturmwarnungsdienst. Geogr. Ann., 8, 301349.

  • Holt, A., , McGuinness R. , , Torlaschi E. , , and Bebbington D. , 1992: On the use of polarization-diversity to aid recognition of ground clutter and anomalous propagation in radar echoes of storms. Proc. Open Symp. on Wave Propagation and Remote Sensing, Ravenscar, North Yorkshire, United Kingdom, URSI, 2.4.1–2.4.5.

  • Islam, T., , Rico-Ramirez M. , , Han D. , , and Srivastava P. , 2012: Artificial intelligence techniques for clutter identification with polarimetric radar signatures. Atmos. Res., 109–110, 95113, doi:10.1016/j.atmosres.2012.02.007.

    • Search Google Scholar
    • Export Citation
  • Istok, M. J., and Coauthors, 2009: WSR-88D dual polarization initial operational capabilities. Preprints, 25th Conf. on Int. Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Phoenix, AZ, Amer. Meteor. Soc., 15.5. [Available online at https://ams.confex.com/ams/89annual/techprogram/paper_148927.htm.]

  • Kessinger, C., , Ellis S. , , and Van Andel J. , 2003: The radar echo classifier: A fuzzy logic algorithm for the WSR-88D. Preprints, Third Conf. on Artificial Intelligence Applications to the Environmental Science, Long Beach, CA, Amer. Meteor. Soc., P1.6. [Available online at https://ams.confex.com/ams/annual2003/techprogram/paper_54946.htm.]

  • Lakshmanan, V., 2012: Automating the Analysis of Spatial Grids: A Practical Guide to Data Mining Geospatial Images for Human and Environmental Applications. Geotechnologies and the Environment, Vol. 6, Springer, 320 pp.

  • Lakshmanan, V., , Fritz A. , , Smith T. , , Hondl K. , , and Stumpf G. J. , 2007a: An automated technique to quality control radar reflectivity data. J. Appl. Meteor. Climatol., 46, 288305, doi:10.1175/JAM2460.1.

    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., , Smith T. , , Stumpf G. J. , , and Hondl K. , 2007b: The Warning Decision Support System–Integrated Information. Wea. Forecasting, 22, 596612, doi:10.1175/WAF1009.1.

    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., , Zhang J. , , and Howard K. , 2010: A technique to censor biological echoes in radar reflectivity data. J. Appl. Meteor. Climatol., 49, 435462, doi:10.1175/2009JAMC2255.1.

    • Search Google Scholar
    • Export Citation
  • Lakshmanan, V., , Zhang J. , , Hondl K. , , and Langston C. , 2012: A statistical approach to mitigating persistent clutter in radar reflectivity data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 5, 652662, doi:10.1109/JSTARS.2011.2181828.

    • Search Google Scholar
    • Export Citation
  • Liu, H., , and Chandrasekar V. , 1998: An adaptive neural network scheme for precipitation estimation from radar observations. IGARSS ’98: Sensing and Managing the Environment; 1998 IEEE International Geoscience and Remote Sensing Symposium Proceedings, T. I. Stein, Ed., Vol. 4, IEEE, 1895–1897, doi:10.1109/IGARSS.1998.703687.

  • Lucas, D. J., , Goodman L. S. , , and Fabozzi F. J. , 2006: Collateralized Debt Obligations: Structures and Analysis.Frank J. Fabozzi Series, Wiley Finance, 528 pp.

  • Lynn, R., , and Lakshmanan V. , 2002: Virtual radar volumes: Creation, algorithm access, and visualization. Preprints, 21st Conf. on Severe Local Storms, San Antonio, TX, Amer. Meteor. Soc., P5.3. [Available online at https://ams.confex.com/ams/SLS_WAF_NWP/techprogram/paper_47546.htm.]

  • Ortega, K., , Smith T. , , Manross K. , , Scharfenberg K. , , Witt A. , , Kolodziej A. , , and Gourley J. , 2009: The severe hazards analysis and verification experiment. Bull. Amer. Meteor. Soc., 90, 15191530, doi:10.1175/2009BAMS2815.1.

    • Search Google Scholar
    • Export Citation
  • Otsu, N., 1979: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern., 9, 6266, doi:10.1109/TSMC.1979.4310076.

    • Search Google Scholar
    • Export Citation
  • Park, H., , Ryzhkov A. , , Zrnić D. , , and Kim K. , 2009: The hydrometeor classification algorithm for the polarimetric WSR-88D: Description and application to a MCS. Wea. Forecasting, 24, 730748, doi:10.1175/2008WAF2222205.1.

    • Search Google Scholar
    • Export Citation
  • Rico-Ramirez, M. A., , and Cluckie I. D. , 2008: Classification of ground clutter and anomalous propagation using dual-polarization weather radar. IEEE Trans. Geosci. Remote Sens., 46, 18921904, doi:10.1109/TGRS.2008.916979.

    • Search Google Scholar
    • Export Citation
  • Ryzhkov, A. V., 2007: The impact of beam broadening on the quality of radar polarimetric data. J. Atmos. Oceanic Technol., 24, 729744, doi:10.1175/JTECH2003.1.

    • Search Google Scholar
    • Export Citation
  • Ryzhkov, A. V., , Giangrande S. E. , , Melnikov V. M. , , and Schuur T. J. , 2005: Calibration issues of dual-polarization radar measurements. J. Atmos. Oceanic Technol., 22, 1138–1155, doi:10.1175/JTECH1772.1.

    • Search Google Scholar
    • Export Citation
  • Steiner, M., , and Smith J. , 2002: Use of three-dimensional reflectivity structure for automated detection and removal of nonprecipitating echoes in radar data. J. Atmos. Oceanic Technol., 19, 673686, doi:10.1175/1520-0426(2002)019<0673:UOTDRS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Stumpf, G., , Witt A. , , Mitchell E. D. , , Spencer P. L. , , Johnson J. T. , , Eilts M. D. , , Thomas K. W. , , and Burgess D. W. , 1998: The National Severe Storms Laboratory mesocyclone detection algorithm for the WSR-88D. Wea. Forecasting, 13, 304326, doi:10.1175/1520-0434(1998)013<0304:TNSSLM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Sun, J., , and Wilson J. W. , 2003: The assimilation of radar data for weather prediction. Radar and Atmospheric Science: A Collection of Essays in Honor of David Atlas, Meteor. Monogr., No. 52, Amer. Meteor. Soc., 175–198, doi:10.1175/0065-9401(2003)030<0175:TAORDF>2.0.CO;2.

  • Zhang, J., , Wang S. , , and Clarke B. , 2004: WSR-88D reflectivity quality control using horizontal and vertical reflectivity structure. Preprints, 11th Conf. on Aviation, Range, and Aerospace, Hyannis, MA, Amer. Meteor. Soc., P5.4. [Available online at https://ams.confex.com/ams/11aram22sls/techprogram/paper_81858.htm.]

  • Zrnić, D., , and Ryzhkov A. , 1998: Observations of insects and birds with a polarimetric radar. IEEE Trans. Geosci. Remote Sens., 36, 661668, doi:10.1109/36.662746.

    • Search Google Scholar
    • Export Citation
  • Zrnić, D., , and Ryzhkov A. , 1999: Polarimetry for weather surveillance radars. Bull. Amer. Meteor. Soc., 80, 389406, doi:10.1175/1520-0477(1999)080<0389:PFWSR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Zrnić, D., , Ryzhkov A. , , Straka J. , , Liu Y. , , and Vivekanandan J. , 2001: Testing a procedure for automatic classification of hydrometeor types. J. Atmos. Oceanic Technol., 18, 892913, doi:10.1175/1520-0426(2001)018<0892:TAPFAC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
1

The fixed set of elevations scanned by a radar is termed a volume scan. Conventionally, in the United States, a volume scan is defined as starting with the lowest elevation scan and consisting of the tilts scanned until the lowest elevation scan is repeated in 4–6 min. However, defining a volume scan as starting at the lowest tilt is arbitrary and so, a virtual volume was defined by Lynn and Lakshmanan (2002) as the set of latest scans available at every elevation. Defining the virtual volume choronologically makes it possible to have a complete volume at the end of every tilt scanned by the radar and therefore enables near-real-time processing of radar data even by algorithms that require a volume scan of radar data.

2

In other words, the space of input feature vectors was tranched (or sliced) into four divisions. Given an input vector, it is possible to determine to which tranche it belongs: feature vectors without valid velocity belong to the first tranch, while feature vectors with Z less than 10 dBZ belong to the second, those with Z between 10 and 20 dBZ belong to the third, and those above 20 dBZ belong to the fourth. The neural network corresponding to each tranche is trained only on those feature vectors that it would have to classify at run time. The word tranche comes from financial statistics (Lucas et al. 2006, 464–466), and it gained wider notoriety when mortgages were tranched into collaterized debt obligations, so as to equalize interest yield and risk.

Save