1. Introduction

Since the beginning of weather radar, the suppression of ground clutter echoes (GCE) has been a major concern and continues to be an important consideration for data quality control. In the early days of weather radar, finite impulse response (FIR), infinite impulse response (IIR), and spectral-based clutter filters were investigated. By the 1990s, IIR filters were the preferred choice for clutter mitigation with the U.S. National Weather Service (NWS) adopting such a filter (Sirmans 1987). By the early 2000s, radar processors had enough power to perform clutter filtering on the Doppler spectra in real time, and since then, spectral clutter filters have become the standard; e.g., the NWS uses Gaussian Model Adaptive Processing (GMAP) on the Next Generation Weather Radar (NEXRAD) (Doviak et al. 2000; Siggia and Passarelli 2004; Ice et al. 2004, 2005). Important operational aspects of GMAP are as follows: 1) It operates on a single radar gate (resolution volume) of data, 2) the filter notch width is adaptable, and 3) it includes an algorithm to interpolate across the zero-velocity gap created by the filter, so that biases in reflectivity and velocity estimates are reduced. These three aspects of GMAP were difficult to accomplish with IIR and FIR filters since they typically optimally operate on a continuous stream of time series. Applying an IIR or FIR filter on a single gate of time series is problematic due to unfilled filter taps at the time series’ endpoints. A disadvantage of spectral filters is that they must multiply the time series by a window function such as the von Hann, Blackman, or Blackman–Nuttall to contain clutter leakage (Harris 1978) before a spectral notch can be applied. These window functions attenuate the time series, effectively reducing the available number of independent samples, before calculating the radar variables from any weather signal that may overlay the clutter signal, thereby increasing the standard error of the radar variable estimates. We refer to spectral filters as window-and-notch (WN) filters.

Regression filters have been investigated in the biomedical field for processing ultrasound images to suppress stationary and slow-moving tissue signal so that the blood flow signal of interest can be better estimated (Bjaerum et al. 2002; Kadi and Loupas 1995). Interestingly, the regression filter was investigated in Torres and Zrnić (1998, 1999) for NEXRAD and was found to have comparable performance to IIR filters. At that time, GMAP also became a viable option, and it was adapted in lieu of regression filtering. In Torres and Zrnić (1998), the authors indicate that a local regression filter approach was used for 64-point sequences using a 16-point kernel. In Hubbert et al. (2021), the local regression filter approach was investigated and shown to have inferior frequency response performance as compared to a global regression filter for a specific set of parameters. Local versus global regression filtering is an important distinction.

The primary advantage of the regression filter is that windowing of the time series is not required, and thus the standard error of radar variable estimates from any weather echo overlaid on the GCE is reduced, typically by 25%–50% depending on the window used, the length of the time series, and the intrinsic weather signal statistics (Harris 1978; Sachidananda and Zrnić 1999; Siggia and Passarelli 2004; Zrnić et al. 2008). This reduction in standard error is demonstrated in Hubbert et al. (2021) where the regression filter is compared to an equivalent WN clutter filter using simulated data.

Recently, a clutter mitigation solution, which combines both clutter identification and clutter filtering, was introduced, which is termed as the Clutter Environment Analysis using Adaptive Processing (CLEAN-AP) (Torres and Warde 2014). CLEAN-AP is a WN-type filter, but it uses the lag-1 autospectral density function (ASD) to help determine the bandwidth of the spectral notch. One advantage of CLEAN-AP over GMAP is that it adapts the window function based on the clutter-to-noise ratio (CNR). However, when higher power clutter signal is present, CLEAN-AP still uses the significantly attenuating windows von Hann, Blackman, and Blackman–Nuttall. CLEAN-AP clutter detection performance has been improved with the fuzzy logic algorithm Weather Environment Thresholding (WET), based on the spatial variability of the polarimetric variables (Warde and Torres 2015). Herein, the regression filter is compared to the CLEAN-AP filter performance given in Torres and Warde (2014) using simulated data.

Two important aspects for the design of an operational regression clutter filter are an adaptable stop bandwidth selection (i.e., polynomial order) and subsequent interpolation across the zero-velocity gap created by the clutter filter, both of which are addressed herein.

In this paper, the viability and performance of the global regression filter is established using time series gathered by both S-Pol and the NEXRAD KFTG (Denver, Colorado) via comparisons to WN-filtered data. The performance comparisons are accomplished in part by calculating the areal standard deviations (SDs) over plan position indicators (PPIs) of the radar variables and then making histograms of the SD differences. Reducing the areal SD of radar variable estimates, without spatial averaging, indicates reduced standard errors of the estimates. Reduction in standard errors, in general, leads to improved radar products, such as rain rates and particle identification.

This paper is organized as follows. Section 2 is a description of the regression filtering process, SD comparison methodology, and the datasets. A comparison of regression filtered to CLEAN-AP processing is given for simulated data in section 3. Section 4 provides a statistical analysis of clutter suppression and comparison of regression-filtered versus window-and-notch-filtered data based on S-Pol clutter time series. Section 5 compares PPIs of S-Pol data processed with the regression and WN filters, and section 6 compares level-2 long pulse repetition time (PRT) KFTG data to the same data processed with the regression filter. Section 7 discusses the issue of azimuthal resolution versus the SD of radar variable estimates for the regression filter. Section 8 offers a summary and conclusions.

2. Datasets and methodology

To demonstrate and validate the regression filter, three observational datasets are used.

c. Automated global regression filtering

For operational regression clutter filtering, the process should include 1) automated polynomial order selection and 2) interpolation across the zero-velocity gap created by the clutter filter so that biases to reflectivity, velocity, and spectrum width σ_υ estimates are reduced.

The required polynomial order for the regression filter can be determined for a set of data collection parameters using simulations as discussed in Hubbert et al. (2021). The polynomial order depends on 1) the CNR, 2) the radar scan rate, 3) the PRT, 4) the number of samples, and 5) the operating frequency. The scan rate, PRT, the number of samples, and the operating frequency are known a priori. Thus, only the CNR needs to be estimated in real time. The functional relationship between the required polynomial order and the CNR for various sets of the other radar parameters is established via simulations and is given in appendix A. Estimating the CNR can be done in two ways: 1) from the power in the three-central velocity components of a Doppler spectrum (i.e., zero-velocity component and the two adjacent components) and 2) the power contained in a second- or third-order polynomial fit. To demonstrate these two techniques, we use case 1 S-Pol clutter time series. Figure 1 shows the S-Pol PPI of unfiltered SNR with the Rocky Mountains to the west of S-Pol.

View Full Size

A clear air PPI of unfiltered S-Pol SNR (dB) gathered at 2307 UTC 31 Jan 2019 at 0° elevation angle. The locations of the Denver International Airport (DIA) and KFTG are given.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

Using clutter time series from Fig. 1, Fig. 2 shows a scatterplot of the power from the three central Doppler velocity components versus the power contained in a second-order polynomial fit. The powers are very similar, clustering around the one-to-one line. Using a second-order polynomial fit provides computational savings over using an FFT to calculate the power spectrum to estimate the power of the clutter signal, though an abbreviated DFT calculation for only the central three points could be used. The second-order polynomial fit technique is used in this paper. For dual-polarization data, the CNR is estimated for both the H and V channels and the greater CNR is used to select the polynomial order for both channels.

View Full Size

S-Pol clutter power data from Fig. 1. Scatterplot of the power contained in a second-order regression fit vs the power contained in the three central velocities of the Doppler spectra. The one-to-one line is red.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

After applying a clutter filter, biases in reflectivity, velocity, and σ_υ estimates can be reduced by interpolating across the zero-velocity gap created by the filter in the Doppler spectrum, as is done in GMAP. A similar Gaussian interpolation scheme for the regression-filtered data can be done, and this interpolation algorithm is given in appendix B. Both automated order selection and Gaussian interpolation are used in this paper; however, it is known that this interpolation can cause a bias in the estimates of the dual-polarization variables, and, thus, interpolation is not used when estimating dual-polarization variables (Zrnić et al. 2008).

3. A comparison of ADVANCE and CLEAN-AP

A new clutter filter technique being considered by the NWS is CLEAN-AP, which is a WN-type spectral filter but uses the autocorrelation spectral density to help specify the filter’s notch width (Torres and Warde 2014, hereafter referred to as TW14). CLEAN-AP clutter suppression and reflectivity bias as a function of the CSR is given in Fig. 6 of TW14 and is reproduced here as Fig. 3a. The simulation parameters are as follows: the clutter signal has a CNR of 50 dB and a σ_υ of 0.28 m s⁻¹ with a mean velocity of 0 m s⁻¹; the weather signal has an SNR of 20 dB and a σ_υ of 4 m s⁻¹; the PRT is 1 ms and the time series length is 64 points. For each CSR, the weather velocity is varied across ± the Nyquist velocity. There are 1000 simulations for each CSR and velocity pair. After clutter filtering, the mean clutter suppression and bias is calculated across all velocities for each CNR. For more details, see TW14.

View Full Size

Clutter suppression and power bias comparison for simulated data: (a) borrowed from TW14, Fig. 6 and (b) similar regression-filtered data. The filters have similar performance. There is a slight reflectivity bias for the regression filter (blue line) for CSR < 0 dB since the RHOHV test was applied to detect clutter. Such low CSR clutter can bias the polarimetric variables.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

Since CMD uses the spatial variability of radar variables along a radar radial of data, the CMD portion of ADVANCE cannot be used. It is assumed that CMD would detect clutter down to 0-dB CSR, which is a very good assumption (Hubbert et al. 2009b, see their Fig. 6), especially with the addition of the RHOHV test. Thus, the regression filter is applied to all simulations where CSR ≥ 0 dB. For simulations with CSR < 0 dB, the RHOHV test is applied using a third-order regression clutter filter to determine ρ_HV of the clutter-filtered data. The CNR is estimated for each simulation, and the order selection is then determined from Eq. (A4). The polynomial orders as a function of CNR are shown in Fig. 4.

View Full Size

Polynomial order as a function of CNR for the regression filter used in Fig. 3b.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

Using the same simulation parameters as in TW14, the regression clutter filter is now compared to that standard. The CSR is varied from −30 to 72 dB in 3-dB steps, while TW14 used 5-dB steps. Figure 3b shows the clutter suppression of the regression filter with the red curve and in green for ideal clutter suppression (see TW14 for the details of this calculation). The curves lie on top of one another. The blue curve shows the average power bias for the regression filter. Almost all of the biases are less than a tenth of a decibel. Table 1 tabulates the number of clutter detections from the RHOHV test at each CSR < 0 dB. The importance of detecting and filtering clutter down to −15-dB CSR to reduce the bias in the polarimetric variables is described in Friedrich et al. (2009). Clearly, the regression filter performs as well as CLEAN-AP in clutter rejection and power bias for this simulation.

Table 1.

Clutter detected by the RHOHV test for 64 000 simulations for each CSR corresponding to the data in Fig. 3b.

View Table

The regression filter is further compared to CLEAN-AP via Fig. 8 of TW14 given here as Fig. 5. Shown are the bias and SD of power in Figs. 5a and 5b, velocity in Figs. 5c and 5d, and σ_υ in Figs. 5e and 5f as a function of simulated velocity and simulated σ_υ. The CSR is always 50 dB, and the weather has an SNR of 20 dB.

View Full Size

From TW14, Fig. 8: “…CSR of 50 dB. In (c),(d), an asterisk marks the WSR-88D requirement benchmark at a velocity of 4 m s⁻¹ and a spectrum width of 4 m s⁻¹ where the velocity bias is required to be 2 m s⁻¹ and where the SD of velocity estimates is required to be 2 m s⁻¹, respectively. (e),(f) As in (c),(d).”

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

The equivalent statistics for the regression-filtered data are given in Fig. 6. The regression filter’s order is determined according to the estimated CNR. The regression filter power bias in Fig. 6a is, in general, lower than in TW14 in the region 0–5 m s⁻¹ simulated velocity and 0.25–3 m s⁻¹ simulated σ_υ. The SD in Fig. 6b is also significantly lower than in TW14 in most areas. The velocity bias, Fig. 6c, of the regression filter shows a positive bias of about 0.25–0.75 m s⁻¹ from 0 to 8 m s⁻¹ σ_υ while the TW14 velocity bias, Fig. 5c, is negative from about −1 to −0.25 m s⁻¹ in the similar region. However, the regression velocity SDs (Fig. 6d) are significantly lower than those in TW14 (Fig. 5d). The ${\widehat{\sigma }}_{\upsilon }$ (estimated σ_υ) bias of the regression filter, Fig. 6e, decreases from about 2 m s⁻¹ to about 0.3 m s⁻¹ as the simulated σ_υ increases from 0.25 to 8 m s⁻¹ for velocities close to zero. The ${\widehat{\sigma }}_{\upsilon }$ bias in TW14 (Fig. 5e) has lower biases in the similar region. However, the ${\widehat{\sigma }}_{\upsilon }$ biases in TW14 do show a broad area with negative bias where the regression filter has very small bias (i.e., magnitudes < 0.25 m s⁻¹). SDs due to the regression filter are lower in the region close to 0 velocity where TW14 shows lower bias. The comparison of ${\widehat{\sigma }}_{\upsilon }$ SDs is mixed, with the regression SDs being lower in some areas but higher in other areas as compared to TW14. Importantly, the NEXRAD benchmark requirements are given in the caption of Fig. 5 for the points marked by the asterisks, namely, 4 m s⁻¹ simulated velocity and 4 m s⁻¹ simulated σ_υ. The same points are marked with asterisks in Fig. 6. The regression filter data meet these requirements also.

View Full Size

As in Fig. 5, but for regression-filtered simulations.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

The conclusion is that the regression filter has similar clutter rejection and reflectivity biases as does CLEAN-AP (Fig. 3). However, from Fig. 6, the regression filter offers some improvement in power and velocity biases and significantly outperforms CLEAN-AP for the power and velocity SDs. This agrees with simulation results shown previously where the regression filter was compared to a WN filter (Hubbert et al. 2021). This is expected since CLEAN-AP uses the Blackman–Nuttall, Blackman, or von Hann windows when CSR > 0 dB (see TW14, Fig. 1).

The comparisons thus far have been for the case where both the regression filter and the WN filter operate on the same number of time series samples. Use of overlapping windows such as used in the NEXRAD split cuts is examined in sections 5b and 6.

4. Case 1: S-Pol clutter suppression comparisons

The clutter filter suppression performance of the regression filter is compared to a WN filter using clutter data gathered with S-Pol from Fig. 1. The purpose is to compare the clutter suppression provided by the regression filter to the suppression provided by a WN filter. To do this, averaged scatterplots of filtered power versus unfiltered power are examined. The frequency response of the WN and regression filters is then compared.

To obtain an uncontaminated, representative subset of clutter echoes for the following analysis, the following limits are imposed: 203° < azimuth angle < 357°, 10 km < range < 50 km, CPA > 0.9, where CPA denotes the clutter phase alignment, a clutter identification metric of CMD (Hubbert et al. 2009a). CPA is used to discriminate high-quality clutter targets, i.e., clutter targets with little contamination (e.g., moving vehicles, biological targets, interference, etc., which will reduce CPA). There are 6974 total clutter data points that pass these criteria. Averaged scatterplots of signal power (uncalibrated I² + Q², no noise correction, window function attenuation compensated) after clutter filtering (referred to as filtered power) versus unfiltered echo power for each resolution volume are made for various regression and WN filters for three different unfiltered power ranges:

1. from −5 to 10 dB (Figs. 7a,b),

2. from −25 to −5 dB (Fig. 7c), and

3. from −65 to −25 dB (Fig. 7d),

View Full Size

Averaged scatterplots of filtered vs unfiltered S-Pol power for three unfiltered power ranges for time series from Fig. 1. The time series are 64-point length. The x and o symbols mark the mean locations. (a) WN-filtered data for Blackman–Nuttall, Blackman, and von Hann windows and various notch widths (points). (b) For the same power range as in (a), but for regression-filtered data with two WN curves included from (a). (c) Unfiltered powers from −25 to −5 dB showing regression data with one WN curve. (d) For unfiltered powers from −65 to −25 dB with both regression- and WN-filtered data.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

where 10 dB is the highest unfiltered power level, while −65 dB is the lowest unfiltered power level that passed the criteria with the noise floor at about −76 dB. Dividing the input powers into three separate ranges is done to better separate the plotted curves. The filtered data are binned (2- or 3-dB bins), averaged, and plotted.

Figure 7a shows averaged scatterplot curves for WN filters using Blackman–Nuttall, Blackman, and von Hann windows with 7-, 9-, and 11-point spectral notch widths for the 64-point time series. The Blackman–Nuttall and Blackman curves are nearly identical thus indicating equivalent mean clutter-rejection performance. This indicates that the clutter suppression using the Blackman window is sufficient and is preferred to the Blackman–Nuttall window since the Blackman attenuates the signal less severely (5.23 dB as compared to 5.89 dB). It is interesting that the curves have less spread for the highest input powers (7.5 dB on the horizontal axis) and have more spread at the lowest powers (around −4 dB on the horizontal axis). One might expect the opposite, i.e., the stronger the clutter power, the larger the separation between the more aggressive and less aggressive filter curves. As the clutter power increases, less aggressive windows will not contain the clutter leakage as well and the clutter power will not be suppressed as well. Stated differently, more aggressive window functions and larger notch widths are required to suppress stronger clutter signals (Siggia and Passarelli 2004). However, in this highest input clutter power region, the noise floor of the signal has increased and thus this noise power is outside all of the different filters’ notch regions. This noise has two sources: 1) the transmitter and 2) receiver saturation. One way to quantify receiver noise is with dBc, which is defined here as the decibel difference between the carrier and the noise floor of the power spectrum, many times given in decibel/hertz. In this application, the “carrier” is generated by illuminating, with a pointing radar, a very strong clutter target (i.e., near radar receiver saturation). Integrating the noise floor for S-Pol yields a CNR of about 44 dB. Thus, for large clutter signals with CNRs above about 44 dB, the noise floor will increase. Also, signal from high power clutter targets can drive the receivers into saturation, which is typically manifested by an increased noise floor. Because of the increased noise floor, regardless of the applied window function, this noise power remains spread across the spectrum. This is seen from comparing the 9-point von Hann, Blackman, and Blackman–Nuttall curves for the unfiltered power of 7.5 dB. The filtered powers are nearly identical.

Using the same unfiltered power range as in Fig. 7a, Fig. 7b shows curves for regression orders 5, 7, 9, 11, and 13 along with the Blackman–Nuttall WN curves of Fig. 7a for 7-point and 9-point notch widths (dashed curves) for comparison. As expected, as the order of the regression polynomial increases, the filter stop bandwidth increases so that more power is eliminated from the filtered time series. The fifth-order curve is located significantly above the other regression curves and indicates that the fifth-order regression filter has insufficient clutter suppression. This is particularly evident for unfiltered powers from −4 to −2 dB where all the regression filters perform similarly except for the fifth-order regression, which rejects 2–3 dB less power in a mean sense. The seventh-order regression curve is quite similar to the 7-point Blackman–Nuttall curve, and the 11th-order regression curve is very similar to the 9-point Blackman–Nuttall curve. This shows that the clutter rejection performance of the regression and WN filters can be made equivalent.

Quantitatively comparing the frequency response of the regression and WN filters is difficult since they are very different in form. However, a comparison does show that qualitatively the frequency notches of the filters largely coincide. The regression filter’s frequency response, given in Fig. 3 of Hubbert et al. (2021), can be used to compare the 7-point WN filter and the seventh-order regression filter frequency characteristics. Vertical dashed lines on that plot indicate the edge of the 7-point notch width. Figure 3 of Hubbert et al. (2021) shows that the regression filter’s frequency response approximates the frequency characteristics of the 7-point notch while clutter suppression of the seventh-order regression curve and the 7-point Blackman–Nuttall curve are similar as demonstrated in Fig. 7b.

In general, a regression filter can have the equivalent clutter suppression as a WN filter but uses a less aggressive, narrower stop bandwidth than the equivalent WN filter since the window function increases the spectrum width of the clutter signal. This is due to a property of the Fourier transform which states that multiplication in the time domain is equivalent to convolution in the frequency domain. Thus, the frequency spectrum of the window function is convolved with the spectrum of the time series, thereby increasing the clutter spectrum width. In principle, the regression filter can potentially pass more of any underlying weather echo while suppressing the clutter signal as well as a WN filter.

Figure 7c shows averaged scatterplots for the regression filter for unfiltered powers from −25 to −5 dB. The seventh- and ninth-order regression curves are about equal, indicating that a seventh-order filter is sufficient for clutter rejection for this input power range. In comparison, the fifth-order curve has about 1.5 dB less attenuation and may not offer sufficient clutter rejection. In this range of powers, the seventh-order regression and the 7-point notch Blackman curves are very similar, indicating similar clutter rejection performance.

Figure 7d shows an averaged scatterplot for unfiltered clutter powers from −65 to −20 dB in 3-dB bins for both regression and WN filters. The regression-filtered data are in solid colors, while the WN-filtered data are dashed curves. From −45- to −25-dB unfiltered power, fifth- and sixth-order regression filter curves are similar, indicating that a fifth-order regression filter may offer sufficient clutter suppression in this power range. Below −45 dB, a fourth-order regression appears to have sufficient clutter suppression. The idea is to use a high-enough-order regression filter to suppress clutter but not too high as this would unnecessarily suppress the possible underlying weather signal. It is difficult from these plots to determine what polynomial order, in general, is necessary to sufficiently suppress the clutter since this evaluation is somewhat subjective and depends on the data collection parameters. However, this section does demonstrate that the regression filter can reject practical, observational clutter signals as well as a WN filter can and that the degree of clutter suppression can be controlled by the order of the regression fit as indicated by the filter’s frequency response.

5. Case 2: S-Pol PPI data

Next, a PPI of S-Pol data gathered at 2207 UTC 13 March 2019, at 0.5° elevation angle, is processed by ADVANCE and a WN clutter filter directed by CMD. The resulting radar variable PPIs are then intercompared. The WN filter uses the Blackman window except for CNRs ≥ 75 dB where the Blackman–Nuttall window is used. The polynomial order for ADVANCE and the notch width for the WN filter are shown in Fig. 8 as a function of CNR. Notice that the polynomial order is increased to 21 for CNRs > 75 dB. For these very large clutter powers, the clutter power spectra are not Gaussian and tend to broaden. This likely occurs because of transmitter noise, receiver saturation, or other electronic noise. The bandwidth of the 21st-order regression filter approximately matches the WN filter’s 17-point notch width for these clutter signals. The comparisons between ADVANCE-processed and the WN filter–processed radar variables are given next.

View Full Size

The polynomial orders and the notch widths for the WN filter, as a function of CNR, used for clutter filtering the S-Pol PPI data.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

Figure 9a is unfiltered S-Pol reflectivity that shows widespread stratiform precipitation to the east of S-Pol with embedded GCE. To the west of S-Pol are the Rocky Mountains with the largest GCE. Figure 9b shows the CMD flag field augmented by the RHOHV test (see appendix C for details of the RHOHV test). The red marks where the filters are to be applied. Figure 9d shows the polynomial orders resulting from the automated regression order selection algorithm (see appendix A), and Fig. 9c gives the notch widths for the WN filter. White areas indicate no clutter filtering. After the PPI time series are processed by both ADVANCE and the WN filter, the following radar variables are estimated: reflectivity Z, velocity, σ_υ (based on the zero and first lags of the autocorrelation function), differential reflectivity Z_DR, copolar differential phase ϕ_DP, and ρ_HV (Bringi and Chandrasekar 2001).

View Full Size

S-Pol data gathered on 13 Mar 2019 at 0.5° elevation angle: (a) unfiltered reflectivity, (b) CMD flag with red indicating the region where the clutter filter is applied, (c) WN filter notch widths, and (d) polynomial orders.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

Figure 10a shows WN-filtered Z, Fig. 10b shows regression-filtered Z, Fig. 10c shows WN-filtered velocity, and Fig. 10d shows regression-filtered velocity. In Figs. 10a and 10b, ground clutter residue is seen west of the radar, in the region of the Rocky Mountains. The clutter residual powers are about the same for both filters. The Z images appear quite similar but closer inspection in the CMD flag areas (Fig. 9b), e.g., areas marked by the white rectangle, does show that the ADVANCE-processed Z has a smoother texture. Looking along and close to the zero-velocity isodop region of the reflectivity plots, at about 60° azimuth close to the radar, shows that ADVANCE was also able to recover more power than the WN filter did. It is assumed that the reflectivity across the zero-velocity area should be similar to the reflectivities on each side of the zero-velocity isodop. In contrast, the velocities in Figs. 10c and 10d in the white rectangle look similar. Similarly, Figs. 11a and 11b show the σ_υ comparison. The σ_υ estimates from ADVANCE and from the WN filter are again quite similar.

View Full Size

S-Pol reflectivity and velocity corresponding to Fig. 9: (a),(c) WN clutter filter data; (b),(d) ADVANCE-processed (regression filter). For the white rectangular region, the ADVANCED processing yields spatially smoother data than the WN data, especially for reflectivity. The regression filter also fills in the zero-velocity isodop better, just east-northeast of S-Pol.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

S-Pol σ_υ and Z_DR PPIs corresponding to Fig. 10: (a) WN σ_υ, (b) ADVANCE σ_υ, (c) WN Z_DR, and (d) ADVANCE Z_DR. Again, in the white rectangular region, the ADVANCED-processed data appear smoother.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

Note that the application of a window function to the time series can reduce the variance of the spectral estimates since the spectrum of the window function is convolved with the spectrum of the time series (Bringi and Chandrasekar 2001, p. 284). Velocity and σ_υ depend on the number of independent samples but also depend on the correlation of the samples; i.e., they are based on the first-lag correlation. Thus, there is an offset between the reduction of the effective number of independent samples and smoothing of the spectrum by the window. What this relationship is between spectral averaging, the number of independent samples, and estimation variance is beyond the scope of this article, but simulations indicate that the regression processing yields lower SDs of the velocity and σ_υ estimates (Hubbert et al. 2021) as compared to the equivalent WN filter processing.

The polarimetric variables Z_DR, ϕ_DP, and ρ_HV are given in a similar format in Figs. 11c, 11d and 12. For Z_DR and ϕ_DP, the ADVANCE data do appear spatially smoother. For ρ_HV, the ADVANCE and WN results appear more similar. Obviously, a more objective evaluation is needed and that is considered next.

View Full Size

S-Pol PPIs corresponding to Fig. 11: (a) WN ϕ_DP, (b) ADVANCE ϕ_DP, (c) WN ρ_HV, and (d) ADVANCE ρ_HV.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

a. Areal standard deviation comparisons

To do a quantitative PPI texture comparison, SDs are calculated over a 2D kernel of nine rays (9° in azimuth) by five gates (0.75 km in range). The 2D kernel is passed over the PPIs calculating the areal SD at each PPI data point. The calculated SDs contain variability due to the filtering but also due to the intrinsic spatial variability of the weather. Since the weather spatial variability, especially due to gradients, is the same for both filters, subtracting the SD fields eliminates this common weather variability component, thus leaving just the SD component attributable to the filtering process. The SD PPIs can then be visually compared and evaluated via histograms.

Shown in Fig. 13 is the SD comparison of WN-to-regression-filtered data for reflectivities from Figs. 10a and 10b. The SD PPIs are given in Figs. 13a and 13b for the ADVANCE- and WN-filtered data, respectively. The SDs are calculated only in the region where the clutter filters are applied. Figure 13c shows the difference field of Fig. 13b minus Fig. 13a. Most of the SD differences in Fig. 13c are negative (blue in the color scale) indicating the ADVANCE-processed data have lower SDs. Figure 13d shows the histogram of the values in Fig. 13c and demonstrates the significantly lower SDs due to the ADVANCE processing. Similarly, the SDs are calculated for velocity, Z_DR, ϕ_DP, σ_υ, and ρ_HV, and the resulting histograms of the SD differences are given in Fig. 14. The histograms show that the regression-filtered data have lower SDs than the WN-filtered data for all variables.

View Full Size

S-Pol reflectivity SD comparison between WN- and regression-filtered data corresponding to Fig. 10: (a),(b) The calculated SDs for the regression- and WN-filtered data, respectively; (c) the difference of (b) minus (a); and (d) the histogram of the difference values in (c). Negative values on the abscissa indicate that the regression-filtered data have lower spatial SDs. SDs are calculated only where the clutter filter is applied.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

SD differences as in Fig. 13d [reflectivity is repeated here in (a) for ease of comparison], but for velocity, Z_DR, ϕ_DP, σ_υ, and ρ_HV. Negative abscissa values indicate that the regression filter data have lower areal SDs than the WN filter.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

This same dataset was analyzed in Hubbert et al. (2023); however, that signal processing was different. The WN filter used the von Hann window on all gates, while here the Blackman and Blackman–Nuttall windows were used only in the CMD-flagged regions. The WN-filtered data in Hubbert et al. (2023) were created using a separate processing package, whereas here the same processing software is used for both the regression- and WN-filtered data which facilitate the SD analysis.

b. Overlapping windows for spectral clutter filtering

Using overlapping window functions for SZ phase coding (Sachidananda and Zrnić 1999) was considered by Hubbert et al. (2005) and developed for NEXRAD by Torres and Curtis (2006), and the overlapping window strategy is now used routinely by NEXRAD. It is well known that applying a window function to the time series from a scanning radar reduces the effective antenna beamwidth (EAB) as shown in Zrnić and Doviak (1976), and it is of interest to compare EABs when using various window functions to the EABs when using a rectangular window.

The two-way azimuthal effective antenna patterns (EAPs) are quantified for several window functions and scanning dwell angles, as done in Torres and Curtis (2006). The one-way, main-beam azimuthal antenna power pattern is modeled as a Gaussian shape with a 1° beamwidth (i.e., 3 dB down from the peak), and this is shown in Fig. 15 as a two-way 1° antenna pattern in green, i.e., 6 dB down at ±0.5°. When the antenna is scanned over a 1°, 0.75°, and 0.5° dwell angle, the two-way EAB becomes 1.45°, 1.26°, and 1.11° (for a rectangular window) as given by the light blue, orange, and red curves, respectively. The two-way EAP when scanning over 1° and using the von Hann, Blackman, Blackman–Nuttall, and Meza windows is given by the black, blue, dashed, and magenta curves. NEXRAD uses the Meza window in nonclutter-filtered areas when calculating the polarimetric variables.

View Full Size

Two-way EAPs. The green curve is the two-way antenna main power beam pattern, modeled as a Gaussian shape with a 1° beamwidth (i.e., 6 dB down from the peak power). The antenna pattern is scanned over 1°, 0.75°, and 0.5°, and the EAPs are shown in light blue, orange, and red, respectively. The EAPs are given for the von Hann (black), Blackman (blue), Blackman–Nuttall (dashed black), and Meza (magenta) for a 1° scan and are calculated as in Torres and Curtis (2006).

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

Since the use of a window function reduces the EAB, overlapping windows is one way to improve WN filter’s statistical performance while achieving a desired azimuthal resolution. For example, for the S-Pol case 2 above with 64 H and V sample pairs per degree azimuth, if 128 H and V sample pairs over 2° azimuth are used with the vonn Hann window, the EAB is approximately equivalent to the EAB for 1° dwell with a rectangular window. The 2° dwell von Hann window is advanced at 1° azimuthal increments. We next examine this overlapping approach for the S-Pol PPI case 2. The Blackman window is used in the nonfiltered regions to maintain the similar azimuthal resolution as in the filtered region. The two-way EAB, when using the Blackman window on 2° azimuth data, is 1.28° (calculation not shown here), while the two-way EAB for 64 points using a rectangular window is 1.45°.

Shown in Fig. 16 is S-Pol reflectivity corresponding to Fig. 10a except for processing with 128-point overlapping windows with WN clutter filtering. Since the time series length was doubled, the WN filter notch widths are doubled in length. Due to the 128-point processing as compared to the previous 64-point processing, the variance of the reflectivity estimate should decrease. Comparing the two reflectivity plots, Fig. 16–Fig. 10a, the 128-point processed reflectivity image does seem smoother. To quantify the variance for all the variables, the 128-point processed data are compared to the regression filtered via histograms of the difference of SDs as shown in Fig. 14. These difference histograms are given in Fig. 17 for Z, V, Z_DR, ϕ_DP, σ_υ, and ρ_HV, as labeled. For the 64-point processing, the peak of the Z histogram (Fig. 14a) is at about −0.4 dB, whereas the histogram maximum of the 128-point processing in Fig. 17a is at about −0.1 dB. Thus, it is apparent that the 128-point processing yields lower SDs than 64-point WN processing, not only for Z but for the other radar variables as well, which is to be expected. Note that the regression filter still has lower SDs except for σ_υ and ρ_HV, which have histograms peaks at +0.03 m s⁻¹ and +0.002. This is caused by the spectral averaging of the Blackman window as mentioned above. The 128-point WN filter does increase the computation time, as compared to 64-point regression filter processing.

View Full Size

Reflectivity corresponding to Fig. 10a, except that 128-point overlapping Blackman windows is used. The 128-point Blackman window is advanced to 64 points at a time thereby achieving 1° azimuthal resolution. The reflectivity data here appear spatially smoother than that in Fig. 10a.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Similar to Fig. 14, histograms of the difference in SDs between the 128-point WN-filtered data and regression-filtered data for the clutter-filtered region only. Negative values indicate that the regression filter produced data with lower SDs. The WN-filtered data show lower SDs than the regression-filtered data for spectrum width and ρ_HV which is due to the smoothing of the spectra from the application of the Blackman or Blackman–Nuttall window to the time series (Bringi and Chandrasekar 2001).

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

6. Case 3: KFTG long PRT data

In this section, the automated regression filter with clutter identification, ADVANCE, is applied to KFTG long PRT (LPRT) data from 2216:45 UTC 29 March 2022 at 0.5° elevation angle (NOAA NWS Radar Operations Center 1991b). For reference, Fig. 18 shows an overview of the reflectivity in Fig. 18a and the radial velocity in Fig. 18c from this NEXRAD split-cut (OFCM 2006). The location of KFTG is shown in Fig. 18a. The PRT is 3.1 ms, and the scan rate is about 12° s⁻¹ so that there are about 28 samples per degree azimuth. To detect more detailed storm features, e.g., tornadic signatures, NEXRAD uses 0.5° azimuthal resolution defined as using samples over a 0.5° dwell with a rectangular window (see Fig. 15, red curve). This is achieved by using overlapping 1° azimuth dwell time series with a window function. If clutter is present, the Blackman window is used, while the von Hann window is used in nonclutter regions. Applying a von Hann window or Blackman window to 28 points yields an EAB width that is comparable to the EAB of a rectangular window on 14 points (0.5° dwell). The window is advanced 0.5° in azimuth, and the time series is again processed thus yielding so-called superresolution data.

View Full Size

KFTG level-2 PPI at 2216:45 UTC 29 Mar at 0.5° elevation: (a) level-2 overview reflectivity, (b) level-2 overview velocity, (c) CMD flag, and (d) polynomial order for clutter filtering. The radar is located at coordinates (0, 0) km. The velocity is from a Doppler scan that uses SZ phase coding to extend the unambiguous velocity interval. Some of the missing velocity data are due to the SZ recovery algorithm which results in the well-known velocity “purple haze” regions in NEXRAD data which is included here in the white region.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

For ADVANCE processing, a rectangular 28-point window is used and centered at 0.5° azimuth increments also. The regression clutter filter is then applied to the 28 points; however, the radar variables are calculated from the central 14 points thereby achieving equivalent superresolution data. If there is no clutter, then the radar variables are simply calculated directly from the central 14 data points. The regression filter uses 28 points since GMAP operates on 28 points and this gives ADVANCE more flexibility, which is discussed below. ADVANCE could operate on contiguous 14-point time series. Note that this comparison of NEXRAD level-2 data with ADVANCE applied to level-1 data (i.e., time series) (NOAA NWS Radar Operations Center 1991a) is similar to the previous S-Pol 128-point case, which also used overlapping windows.

a. ADVANCE versus KFTG level-2 PPI comparison

Figures 18b and 18d show the CMD flagged area in red and the polynomial order for the regression filter, respectively. Figures 19a and 19b give the accompanying PPI comparison of Z for the level-2 and the ADVANCE-processed data. The level-2 Z shows censored data in white, while the ADVANCE data are uncensored. In some places, the ADVANCE PPI shows apparent valid recovered Z, while the corresponding level-2 Z is censored. For example, the censored data direct south of KFTG at 5–10-km range. This is difficult to evaluate since the reason that the level-2 Z was censored is unknown. Looking closely, the ADVANCE Z does seem smoother than the level-2 Z, i.e., the regression-filtered data appear to have lower areal SDs.

View Full Size

KFTG PPIs corresponding to Fig. 18: (a) level-2 Z, (b) ADVANCE-processed Z, (c) level-2 Z_DR, and (d) ADVANCE-processed Z_DR. The KFTG is located at coordinates (0, 0) km.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

Likewise, Z_DR, ϕ_DP, and ρ_HV are compared in Figs. 19c,d and 20a–d, and it is difficult to objectively evaluate image smoothness visually. Also, the level-2 ρ_HV has a broader area of recovery of high ρ_HV than the ADVANCE-processed data. NEXRAD uses a radial-by-radial noise correction scheme that increases ρ_HV in low SNR areas (Ivić et al. 2013), and this technique was shown to, in general, lower SD of the radar variable estimates of reflectivity and spectrum width (Ivić et al. 2014). With the ADVANCED processing, we do not use noise correction and censor the radar variables using the censoring applied to the level-2 fields.

View Full Size

KFTG PPIs corresponding to Fig. 19: (a) level-2 ϕ_DP, (b) ADVANCE ϕ_DP, (c) level-2 ρ_HV, and (d) ADVANCE ρ_HV. The radar is located at coordinates (0, 0) km.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

In the next section, the areal SDs of these radar variables are evaluated objectively as was done with the S-Pol PPIs in section 5a.

b. KFTG areal standard deviation comparison

In this section, the spatial variability of the level-2 radar variables for the KFTG LPRT superresolution data is compared to the spatial variability that results from ADVANCE processing. The 2D kernel is nine rays (one-half degree per ray or 4.5° in azimuth) by five 250-m gates. The ADVANCE-processed data are censored using the censored areas of the level-2 data for SD calculations, and the calculated SDs are from only the clutter-filtered areas.

Figure 21 demonstrates the SD comparison processing for Z for the KFTG PPI with (Fig. 21a) the Level-2 SD, (Fig. 21b) ADVANCE SD, (Fig. 21c) the ADVANCE minus Level-2 SD, and (Fig. 21d) the histogram of the data in Fig. 21c. The histogram shows that the ADVANCE-processed Z PPI has significantly lower SDs. Figure 22 shows just the SD histograms for Z_DR, ϕ_DP, and ρ_HV. These histograms also have peaks that are in the negative region, indicating that the regression-filtered data have lower SDs. This demonstrates that the ADVANCE processing produces PPIs that are smoother and thus have better data quality than the level-2 processing with GMAP, though the SD improvement is not as large as it was for the S-Pol comparisons in Fig. 14. This is due to the similarity of the EAP of the rectangular window with 0.5° dwell (used for the regression filter) to the Blackman or the Blackman–Nuttall window using a 1° dwell (Fig. 15).

View Full Size

KFTG reflectivity data at 2216:45 UTC 29 Mar at 0.5° elevation (Figs. 19a,b): (a) SD of level-2 reflectivity, (b) SD of regression filter reflectivity, (c) the difference of the SDs (level-2 SD–regression SD), and (d) histogram of (c).

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Histograms of the KFTG PPI SD difference: (a) Z_DR, (b) ϕ_DP, and (c) ρ_HV.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

It is interesting and informative to examine the SD data from nonfiltered areas for the KFTG PPI case since NEXRAD uses the Meza window in nonfiltered regions for the polarimetric variables. Figures 23a and 23b shows the SD difference for Z_DR (ADVANCE SD minus Level-2 SD) for both the filtered and nonfiltered data. Almost all of the yellow is in the nonfiltered region, while most of the blue marks are in the filtered areas. The lower level-2 Z_DR SDs (the yellow area) are due to the NEXRAD Meza window.

View Full Size

Z_DR SD comparisons as in Fig. 22a, but for both filtered and nonfiltered regions: (a) the SD difference, ADVANCE minus level 2; (b) the histogram of the data in (a). Most of the yellow regions are nonfiltered areas, while most of the blue are filtered regions (see Fig. 18c). (c),(d) As in (a) and (b), except that the regression filter uses 22-point integration, which is 0.75° in azimuth. The nonfiltered area in (c) is now more negative with less yellow than (a). The 0.75° dwell ADVANCE integration yields similar SDs as the Meza window used in the level-2 data in nonfiltered regions.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

The less-tapered Meza window is defined as (Richardson and Ivic 2019)

$$w(n)=1.5-(0.75+0.25\text{cos}\{2\pi [(n-1)+0.5]/N\}),$$(3)

where N is the number of points of the time series to be windowed and n is the index with n = 1, N. The Meza window tapers down to 0.5 at the endpoints as compared to 0 for both von Hann and Blackman windows and, on average, attenuates the windowed time series by 1.25 dB in contrast to 4.19 and 5.23 dB for the Von Hann and Blackman windows. The use of the Meza window widens the two-way EAB, effectively increasing the number of available independent samples leading to a reduction of SD of the polarimetric variable estimates. Figure 15 shows the EAB for the Meza window in magenta and the EAB for a 0.75° scan (using a rectangular window), which are 1.27° and 1.26° at the 6-dB down point, respectively. Since the EABs are nearly equal, SDs of the level-2 and 0.75° (22-point integration for the present KFTG case) ADVANCE-processed data should be very similar in the nonfiltered regions. The effect of the 22-point integration for ADVANCE processing on the SD of Z_DR is demonstrated in Figs. 23c and 23d. The predominantly yellow areas (nonfiltered data) are now roughly equal portions of yellow and blue. The Z_DR histogram (Fig. 23d) now has a peak in the negative region. This demonstrates the flexibility in ADVANCE processing, which can integrate over any number of samples of the processed time series. Since NEXRAD uses the Meza window in the nonfiltered region with acceptable spatial resolution performance, ADVANCE could also integrate on 22 samples in the filtered regions, with acceptable resolution, thus further reducing the SDs in those areas. The effect of the 22-point integration on the SD of ADVANCE processed Z_DR, in the filtered region only, is demonstrated in Figs. 23e and 23f. The Z_DR difference histogram should be compared to the 14-point integration histogram in Fig. 22a. The SD differences are now more negative and demonstrate the possible further improvement in data quality using the ADVANCED processing.

7. Resolution, variance, and the integration length with the regression filter

For both case 2 (S-Pol PPI) and case 3 (KFTG PPI), the regression filter was applied to time series samples that spanned a 1° azimuthal dwell angle. For the S-Pol case, contiguous blocks of 1° dwell samples were processed, while for the KFTG case, overlapping 1° blocks of data, centered at 0.5° azimuthal increments, were processed. After regression clutter filtering, the number of time series samples that are integrated for radar variable estimates depends on the resolution requirements of the user. The integration length choice is a trade-off between the variance of the radar variable estimate and azimuthal resolution. For example, for the KFTG case, integration over a dwell of 0.75° instead of a dwell of 0.5° produced a significant reduction in the areal SD of the Z_DR estimate (Figs. 23e,f). The SD and resolution are linked since an increase in resolution (integrating fewer samples) increases the SD of the variable estimate so that the increased PPI resolution of storm features is obscured by increased image texture. In any case, the regression filter gives the user the choice of how many samples to integrate, and this choice is not available when using a WN filter.

One way to increase azimuthal resolution with a regression filter is to use overlapping 1° dwell data, as was done for the KFTG case, and then calculating the radar variables from all the samples in the 1° window. This increases the resolution while keeping the SD of the radar variable estimates as low as possible. The radar variables could be estimated from integration on 0.5° dwell data to increase resolution at the expense of higher SDs. To quantify the increase in azimuthal resolution, the degree of correlation between two overlapping EAPs (Fig. 15) can be estimated from the autocorrelation function of the two-way EAPs. This autocorrelation function is shown in Fig. 24 for two-way EAPs for 0.5°, 0.75°, and 1° dwells, i.e., the red, orange, and light blue curves, respectively. The correlations at 0.5° lag are 0.73, 0.66, and 0.59 for 1°, 0.75°, and 0.5° curves, respectively, and these correlations indicate the degree of improved resolution as the integration dwell angle decreases. The correlation decreases 19% for 0.5° as compared to 1° integrations. This is significant but needs to be evaluated against the increased SDs due to integrating half the number of samples, which could be up to 41% (i.e., $\sqrt{2}$). The increase in SD is a function of the scanning parameters of the radar, which determines, in part, the number of equivalent independent samples (Doviak and Zrnić 1993). This issue of resolution versus SD of radar variables was addressed in Torres and Curtis (2015) who proposed range oversampling to reduce SDs, as opposed to the azimuthal oversampling suggested here. The central point is that the regression filter has the flexibility to integrate over any number of samples that are available in the rectangular data processing window, taken here as the number of samples in a 1° dwell. A spectral WN filter approach does not have this flexibility.

View Full Size

The autocorrelation function of two-way EAPs for 1°, 0.75°, and 0.5° azimuthal scans (i.e., dwells, see Fig. 15). At a lag of 0.5°, the correlations are 0.73, 0.66, and 0.59 for 1°, 0.75°, and 0.5° scans, respectively.

Citation: Journal of Atmospheric and Oceanic Technology 42, 6; 10.1175/JTECH-D-24-0029.1

• Download Figure
• Download figure as PowerPoint slide

8. Summary and conclusions

This paper introduces a new paradigm in ground clutter filtering based on global least squares polynomial regression as a filter with a frequency response that predicts how the filter affects the input signal. The literature does address local regression filters such as Savitzky–Golay; however, the global regression filter has been almost totally disregarded since it is not suited for well-established filter theory that uses convolution and impulse response theory to characterize filters such as FIR and IIR filters. The one exception we are aware of is in the medical imaging field.

The performance of the global regression clutter filter was compared to spectrum-based window-and-notch (WN) filters, which are currently popular and in widespread use (e.g., GMAP), using observational S-Pol and NEXRAD data. It was demonstrated that the regression filter suppresses clutter as well as a WN filter that requires the use of a window function, such as the Blackman or von Hann, which significantly attenuates the power in the time series. The regression filter does not use an attenuating window function. Because of this, the radar variable statistics of weather echo, with overlaid clutter, are significantly improved (i.e., they have reduced variance) as compared to WN filters with equivalent clutter rejection. Clutter suppression was demonstrated through average scatterplots of regression- and WN-filtered clutter data from S-Pol. The regression filter’s clutter suppression and radar variable recovery statistics were compared to CLEAN-AP via simulations. The SDs of the radar variable estimates were, in general, significantly lower for the regression filter.

For the ideal operational application, a ground clutter filter needs to have an automated adaptable filter bandwidth and use an interpolation algorithm to fill in the zero-velocity gap in the Doppler spectrum created by the clutter filter. In appendix A, equations were developed that specify the regression order based on the radar scan rate, the PRT, the number of points, and the estimated clutter power. It was shown that the clutter power can be estimated from either the three-central points of the Doppler spectrum or from the power removed by a second-order polynomial fit, which avoids the computational expense of an FFT. In appendix B, a Gaussian interpolation algorithm is described to fill in the zero-velocity gap. Simulations were used to demonstrate the ability of the algorithm to recover power and reduce both power and velocity estimate bias though some increase in SD of power (SNR) is seen.

A new technique to detect low-level CSR (<0 dB) signals that can bias the radar variable estimates (especially the polarimetric variables) termed the RHOHV test is described in appendix C. The algorithm uses ρ_HV since it is the most sensitive of the polarimetric variables to low CSR clutter. The algorithm is straightforward and is based on the comparison of ρ_HV calculated from filtered and unfiltered data. The RHOHV test augments CMD and is currently deployed on NEXRAD.

The complete clutter mitigation package of CMD with the RHOHV test and the automated regression clutter filter, coined ADVANCE, was applied to PPI data from both S-Pol and KFTG. For the S-pol case, the regression filter was replaced with a WN filter for the comparison. In part, the comparison was accomplished by calculating areal SD PPIs for the radar variables after clutter filtering. Histograms of the SD difference (regression-filtered minus WN-filtered data SDs) demonstrated that the regression filter produced PPIs with lower areal SDs. The SDs of the WN-filtered data can be reduced if a sliding, overlapping window is used. For the S-Pol PPI, 2° dwell data were windowed using a Blackman or Blackman–Nuttall window, thereby producing the equivalent resolution of 1° dwell data (without windowing). Histograms of the SD difference showed that the regression-filtered data had lower SDs in the clutter-filtered regions for reflectivity, velocity, Z_DR, and ϕ_DP. The conclusion is that the overlapping window technique for WN filtering offers little advantage while increasing the computational load to a 2° dwell of samples.

ADVANCE processing was compared to level-2 PPI data from the NEXRAD KFTG radar for a long PRT (3.1 ms) scan with half-degree resolution. There were 28 sample points per 1° dwell. Only reflectivity, Z_DR, ϕ_DP, and ρ_HV are recovered from the split-cut LPRT data, while velocity and spectrum width are recovered from the accompanying Doppler scan. Since superresolution was required, overlapping 28-point data windows were used in increments of 0.5° azimuth. Once again in the clutter-filtered region, the regression-filtered PPI data had lower areal SDs than the GMAP-processed level-2 data. In the nonfiltered region, NEXRAD recently switched from using a von Hann window to a Meza window, which attenuates the time series only 1.25 dB as compared to the attenuation of a von Hann window of 4.19 dB. Use of the Meza window reduces the SDs of the level-2 polarimetric variables in the nonfiltered regions. The ADVANCE clutter mitigation algorithm, based on the regression clutter filter, can likewise reduce SDs in the nonfiltered regions by increasing the integration length from 14 to 22 points (equivalently, 0.5°–0.75° dwell) for this dataset. It was shown that a two-way EAP when using the Meza window and the EAP for 22-point rectangular window were nearly identical, and therefore the SDs in the nonfiltered regions were similar for the two filtering techniques. Furthermore, the regression filter could also use 22-point integration in the filtered region to further reduce SDs there. This is not possible with GMAP processing.

There is a trade-off between azimuthal resolution and the SD of the radar variable estimates that depends on the ADVANCE integration length. The azimuthal resolution was quantified using the autocorrelation function of the EAP. The regression filter strategy allows the user to select the number of azimuthal sample points to be integrated, up to the filter block length, thereby providing an additional degree of freedom as compared to WN filters. Increasing the number of integration points from 14 to 22 increases the EAP autocorrelation at 0.5° lag from 0.59 to 0.66. For the case analyzed, the decrease in SD is likely more beneficial than the small percentage loss in resolution.

The ADVANCE clutter mitigation package of CMD with the RHOHV test and the automated regression clutter filter provides a new, real-time clutter mitigation strategy that is more flexible than WN-type filters and offers improved radar variable statistics. At its core is a simple least squares polynomial fit which is fast and easy to implement.

Data availability statement.

S-Pol moment data in Fig. 1 are available to download at https://doi.org/10.26023/KHMH-X6EP-BW0H (NCAR/EOL Remote Sensing Facility 2020). S-Pol time series is available to download at https://doi.org/10.26023/T8VD-F2C8-900K (NCAR/EOL Team 2020). The Forsythe least squares polynomial fit software was obtained from the website at https://Jean-Pierre.moreau.pagesperso-orange.fr/, which is no longer supported. A copy of the Forsythe subroutine is available upon request from the authors. The KFTG level-2 moments data are available to download at https://www.ncei.noaa.gov/metadata/geoportal/rest/metadata/item/gov.noaa.ncdc:C00345/html (NOAA NWS Radar Operations Center 1991b). The KFTG level-1 event data are available to download at https://www.ncei.noaa.gov/metadata/geoportal/rest/metadata/item/gov.noaa.ncdc:C01597/html (NOAA NWS Radar Operations Center 1991a).