A weather surveillance radar antenna intercepts thermal radiation from various sources, including the ground, the sun, the sky, and precipitation. In the radar receiver, this external radiation produces noise that adds to the receiver internal noise and results in the system noise power varying with the antenna position. If these variations are not captured, they translate into erroneous signal powers because these are computed via subtraction of noise power measurements from the overall power estimates. This may lead to biased meteorological variables at low to moderate signal-to-noise ratios if those are computed using signal power estimates. In dual-polarization radars, this problem is even more pronounced, particularly for correlation coefficient estimates that use noise power measurements from both the horizontal and vertical channels. An alternative is to use estimators that eliminate the need for noise corrections but require sufficient correlation of signals in sample time, which limits their applicability. Therefore, when the use of the latter is inappropriate, the quality of correlation coefficient estimates can be improved by computing them using sufficiently accurate noise powers measured at each antenna position. An effective technique that estimates the noise powers in real time at each scan direction and in parallel with weather data collection has been proposed. Herein, the impacts of such a technique on the estimation of the correlation coefficient are investigated. The results indicate that the use of more accurate noise power estimates can significantly reduce the bias of correlation coefficient estimates, thus visibly improving the correlation coefficient fields. This is expected because the correlation coefficient is computed using noise power measurements from both the horizontal and vertical channels.
The most significant advancement in weather radars over the last two decades is polarization diversity, which the U.S. National Weather Service has introduced to its national network of Doppler radars. This upgrade greatly enhances these radars by providing the ability to discriminate among different precipitation types (e.g., rain, hail) and nonweather scatterers (e.g., insects, ground clutter). Implementation of dual polarization on the network of Weather Surveillance Radar-1988 Doppler (WSR-88D) radars employs simultaneous transmission and reception of electromagnetic waves in horizontal (H) and vertical (V) polarizations (SHV mode). In this mode, six radar variables are measured for each radar resolution volume. These are the spectral moments: reflectivity (Ze), Doppler velocity (υ), spectrum width (συ), and the polarimetric variables: differential reflectivity (ZDR), differential phase (ΦDP), and the modulus of the cross-correlation coefficient |ρhυ(0)| (herein referred to as correlation coefficient).
A magnitude of correlation coefficient [or the copolar correlation coefficient in Bringi and Chandrasekar (2001)] is a measure of correlation between H and V backscattered fields. It is assumed herein that the effects of the antenna and receiver on the polarimetric observables can be neglected. Indeed, for well-designed weather radar H and V radiation patterns, and the receiver transfer functions need to be well matched, so they do not introduce appreciable decorrelation of received signals. Assuming these conditions are met, there are two main contributors to the bias of correlation coefficient magnitude estimates. The first is mismeasurement of noise powers in the horizontal and the vertical channels, which causes the incorrect scaling of the cross-correlation magnitude measurement in the conventional lag-0 estimator (Doviak and Zrnić 1993). For example, if the measured noise powers in H and V are higher than the true ones, estimates are biased high. In Melnikov and Zrnić (2007), it is shown that the estimates of |ρhυ(0)| based on the lag-0 estimator become increasingly biased as the signal-to-noise ratio (SNR) falls below 20 dB when errors in noise power levels are on the order of 1 dB. If periodic system calibration is used to measure the noise powers, errors stem from the fact that temporal variations of noise power caused by changes in the system may not be captured, and these can alter the system noise power by more than 1 dB within the time period of less than 3 min (Melnikov 2006). Moreover, typical noise power measurement schemes do not account for spatial variations in the external noise sources, which contribute to the noise power variability in azimuth. For example, differences in landscape surrounding the radar can create strong azimuthal variations in noise power levels (Ice et al. 2013). Also, heavy precipitation emits radiation over a broad frequency range that, when intercepted by the antenna, raises the noise power (Fabry 2001; Melnikov and Zrnić 2004). This can lead to significant overestimation or underestimation of noise powers, which can introduce significant biases in the estimates of the correlation coefficient. To alleviate this problem, an alternate estimator that computes the correlation coefficient from the estimates of lag-1 auto- and cross correlations has been proposed by Melnikov and Zrnić (2007). This estimator does not use the noise power estimates but requires sufficient coherency in sample time so that substantial information about signal parameters is present at lag-1 correlations. Another cause of bias is an inherent positive bias in the correlation coefficient estimators, which is more pronounced when the number of samples per dwell is small and the SNR falls below 20 dB. This bias is not related to noise power measurements and is present even when noise powers are accurately known. Positive biases in correlation coefficient estimates contribute to an increased number of invalid estimates (i.e., values larger than one) that are qualitatively described as “pink fringe” in the WSR-88D fields of |ρhυ(0)|. Consequently, positive biases coupled with the exceptionally high standard deviations of |ρhυ(0)| estimates render correlation coefficient fields noisy with large areas of pink fringe in regions of low to moderate SNRs.
A technique that produces more accurate estimates of noise power at every dwell (or radial) and in parallel with weather data collection has been proposed by Ivić et al. (2013). Unlike the spectral-processing-based techniques proposed in the past (Hildebrand and Sekhon 1974; Urkowitz and Nespor 1992; Siggia and Passarelli 2004), this technique makes no attempt to compute the noise power at every range location (or bin) independently. Instead, it operates under the assumption that average noise power is constant along an entire radial and uses the shape of the radial power profile in range and estimates of the SNR to search for signal-free bins. The search algorithm is based on the assumption that the noise is additive white Gaussian noise (AWGN). Once noiselike bins are identified, the technique computes the average power of these range volumes to produce an estimated noise power. Hence, such an approach requires that a sufficient number of signal-free bins are present in a radial to produce a reliable estimate of noise power. The result is that unlike the WSR-88D legacy noise calibration procedure, this technique is capable of capturing the noise power variations in azimuth and elevation in real time. The accuracy of the technique has been evaluated by Ivić et al. (2013). It has been assessed to produce estimates with an average negative bias of 0.004 dB and a standard deviation of 0.052 dB (or 1.2% of the true noise power). Given that the technique requires a sufficient number of range locations to produce an estimate, Ivić et al. (2013) assessed the failure rate of the technique to be 0.025%. Additionally, the technique had been thoroughly evaluated by the Next Generation Weather Radar (NEXRAD) Data Quality team, which recommended the technique to be accepted for implementation on the WSR-88D fleet. During this evaluation process, numerous test cases have been examined using recorded data from test radar sites with known as well as those with no known noise calibration issues. In the first case, a visible increase in the number of valid correlation coefficient estimates has been detected. In the latter test cases, either minor or no detectable differences have been observed. Given such results, the use of this technique on dual-polarized weather radars is likely to improve the correlation coefficient fields by reducing the noise-induced biases at radar sites where the legacy WSR-88D calibration procedure fails to produce sufficiently accurate results. The technique is expected to fully replace the legacy noise power calibration procedure (Ice et al. 2013).
In this paper, an analysis of typical correlation coefficient estimators as well as the impact of the radial-based noise power estimation (RBNE) on correlation coefficient estimates is presented. The paper is structured as follows. Section 2 analyzes the performance of existing correlation coefficient estimators, assuming exactly known noise powers. The bias induced by noise uncertainties is assessed in section 3, and section 4 analyzes the effects of radial-based noise power estimation on |ρhυ(0)| fields using real-time series. Finally, section 5 summarizes the main conclusions of the paper.
2. Correlation coefficient estimators
In the SHV mode, the following two estimators are analyzed herein:
where the circumflex denotes estimates; and are the estimates of powers in the H and V channels, respectively; and are the lag-1 estimates of the autocorrelation in the H and V channels, respectively; and and are the lag-0 and lag-1 cross-correlation estimates, respectively. All the quantities are calculated from the measurements of complex voltages or samples, Vh(m) and Vυ(m), in the H and V channels, respectively, at each range location. The lowercase m denotes the transmission within a radial from which the particular voltage originates (i.e., the sample number in sample time). The uppercase T stands for the pulse repetition time [T = 1/pulse repetition frequency (PRF)]. The noise power estimates, denoted by and , are in the H and V channels, respectively. Because noise power measurements inevitably introduce errors, these can be modeled as
where the asterisk denotes the complex conjugate.
Equation (1a) is the standard estimator (Doviak and Zrnić 1993) used by the WSR-88D signal processor, while Eq. (1b) has been proposed by Melnikov and Zrnić (2007) as a means to decrease the impact of noise power measurement variations on the bias of the correlation coefficient estimates. The first estimator is often referred to as the lag-0 estimator and the second as the lag-1 estimator. Expression (1a) demonstrates that the computation of the correlation coefficient via lag-0 estimator involves noise power measurement in the H and V channels (i.e., and , respectively). On the other hand, Eq. (1b) shows that the lag-1 estimator does not utilize the estimates of noise powers in the H and V channels, and therefore it is impervious to noise power mismeasurements [i.e., and are not used in Eq. (1b)]. Consequently, if the lag-0 estimator is used and noise power measurements involve large errors, then these introduce significant bias in |ρhυ(0)| estimates. In one such case, captured on a clear day, the noise power changes appreciably with each scan position (Fig. 1). It shows that the legacy WRS-88D calibration procedure is incapable of capturing noise power variations in azimuth, as shown using the estimates produced by RBNE. In the NEXRAD system, this may cause incorrect discrimination of hydrometeors as well as biological scatterers because the correlation coefficient [computed as in Eq. (1a)] is used for these purposes. Additionally, estimates produced by the lag-0 estimator are used by the differential phase smoothing algorithm in the WSR-88D processor to evaluate the quality of these estimates at each range location.
Assuming exactly known noise levels, the biases of the two estimators computed using simulations (Zrnić 1975; Torres 2001) are shown in Fig. 2. Biases were computed for the data collection parameters (i.e., M = 15 and υa = ±9 m s−1) corresponding to the surveillance scans in volume coverage patterns (VCP) of the WSR-88D used in typical operations for reflectivity and correlation coefficient measurements at the lowest tilts (e.g., 0.5°), which result in the longest unambiguous range but the shortest Nyquist cointerval. This is in contrast to Melnikov and Zrnić (2007), who showed the comparison of two estimators for M = 64 and the unambiguous velocity (υa) of ±25 m s−1, which correspond to collection parameters of data from which |ρhυ(0)| is typically computed at higher antenna elevations. The curves in Fig. 2 demonstrate that even if the noise powers are accurately known, the estimates of |ρhυ(0)|, in regions of low to moderate SNR, are still biased because of the intrinsic sensitivity of estimators (1a) and (1b) to SNR even if their values are between zero and one (i.e., valid estimates). Approximate equations that describe this behavior are shown in Melnikov and Zrnić (2007). Plots of the biases show that while at συ of 1 m s−1 the performances of the lag-0 and the lag-1 estimators are comparable, the bias of the lag-1 estimator becomes significantly larger as the spectrum width increases, whereas the opposite occurs for the lag-0 estimator. Such behavior can be rationalized by examining equations that approximately describe the biases of the lag-0 and lag-1 estimators [Eqs. (A16) and (A14) in Melnikov and Zrnić 2007]. These show the bias of both estimators to be indirectly proportional to SNR and to the equivalent number of independent samples (MI in Doviak and Zrnić 1993). As συ increases so does MI, resulting in the slight decrease in the lag-0 estimator bias. However, in case of the lag-1 estimator, the bias is also directly proportional to the spectrum width in addition to the SNR and MI. Thus, for the given conditions, the curves in Fig. 2 show that as συ increases, the latter effect becomes dominant in the lag-1 estimator, resulting in the bias increase. A similar trend is observed in Fig. 3, which shows standard deviations (SD) for the two estimators. As in the case of bias, MI increases with συ, resulting in the decrease of the lag-0 estimator SD, since it requires no coherence in sample time. The opposite occurs in the case of the lag-1 estimator because it requires substantial correlation along the sample time to produce satisfactory performance. Consequently, for the given collection parameters, insufficient correlation in sample time prevents the lag-1 estimator from producing better performance than the lag-0 estimator. Hence, if sufficiently accurate noise power estimates are available, then the results show that the lag-0 estimator is the preferred estimator for the given conditions (i.e., when the Nyquist cointerval is on the order of ±10 m s−1). Common to both estimators is that the biases and SDs increase exponentially as the SNR decreases. The biases of both estimators eventually exceed the value of ±0.01, adequate for weather applications (Balakrishnan and Zrnić 1990).
In addition to the two estimators examined herein, the multilag correlation coefficient estimators proposed by Lei at al. (2012) should be noted as well. These estimators combine estimates from several lags to compute the output. As is the case for the lag-1 estimator, the multilag estimators do not use the noise power estimates but require sufficient coherency in sample time, so that substantial information about the signal parameters is present at lags other than zero. This implies a degraded performance of these estimators for the Nyquist cointervals on the order of ±10 m s−1. Additionally, the models that the multilag estimators are based on do not take into account biases due to the finite number of samples. Aforementioned are the likely reasons that the results presented in Lei at al. (2012) have been produced for M = 128 and υa = ±25 m s−1. This suggests that the multilag estimators also require a large number of samples (usually unavailable in typical WSR-88D operations) to produce satisfactory improvements.
An analysis of the lag-0 and lag-1 estimators presents strong impetus for the use of the former in cases when the unambiguous velocities are approximately ±10 m s−1. In the network of WSR-88D radars, this is typically the case for surveillance scans at tilts below ~2.5°. Because the lag-0 estimator is susceptible to errors in noise power estimates, the noise-induced bias of the lag-0 correlation coefficient estimator is analyzed next.
3. Noise-induced bias
The contributions of δNh and δNυ to the mean value of the estimator are identified using perturbation analysis as
where the angle brackets 〈 〉 denote ensemble averages. From this equation it is possible to identify the two components of the estimator bias: the inherent bias (assuming known noise power levels), defined as
and the noise-induced bias, defined as
In this work, the focus is on the latter component given by Eq. (6). The perturbation analysis similar as in Melnikov and Zrnić (2004) is applied to terms in Eq. (6) to find their mean values. It yields
In Eq. (7), SNRh and SNRυ are SNRs in the H and V channels, respectively, while Sh and Sυ are signal powers in the H and V channels, respectively. The symbol MI stands for the equivalent number of independent samples (Doviak and Zrnić 1993).
The noise-induced bias is analyzed using Eq. (7) and the simulated time series (Zrnić 1975; Torres 2001). The analysis is carried out for δNh and δNυ values that are realistic when the WSR-88D legacy noise calibration is used (Figs. 1a, 5b, and 6b). The results in Fig. 4 demonstrate that the noise power overestimation causes significant positive bias in |ρhυ(0)| estimates as SNR diminishes. Such bias alone exceeds the desired level of ±0.01 for given δNh and δNυ values, indicating that noise power mismeasurements coupled with the positive bias inherent in the estimators (Fig. 2) may visibly degrade the correlation coefficient fields. This is investigated next.
4. Real data examples
The analysis has been carried on three sets of time series collected with WSR-88D radars located in South Kauai, Hawaii; Duluth, Minnesota; and Norman, Oklahoma. Each dataset consists of one 360° sweep at an elevation of about 0.5° using a pulse repetition time (PRT) of 3.1 ms, which is standard for reflectivity and correlation coefficient measurements on the WSR-88D at the given elevation. These settings yield an unambiguous range of 460 km and a Nyquist velocity of about ±9 m s−1. The reflectivity fields corresponding to the three collections are shown in Fig. 5a (test case 1), Fig. 6a (test case 2), and Fig. 7a (test case 3), where the numbers of samples (M) are 15, 64, and 17, respectively. The first and the third sample sizes are routinely used on the WSR-88D in the precipitation mode, executed when enough significant echoes are present (OFCM 2006). The sample size of 64 is used in the clear air mode, executed when the precipitation intensity and aerial extent are small or when there is no detectable precipitation (OFCM 2006). All the reflectivity fields were produced using the RBNE technique (Ivić et al. 2013). These noise power estimates are presented in Figs. 5b, 6b, and Fig. 7b along with the noise powers supplied by the WSR-88D legacy calibration. The first two plots show that in both test cases, the WSR-88D calibration noise powers are visibly above those of RBNE. This suggests that the estimates produced by the lag-0 estimator using the WSR-88D calibration noise powers are additionally biased high by the noise power errors at low to moderate SNRs and azimuths where the difference between RBNE and the WSR-88D calibration estimates are large. Thus, these two datasets constitute good cases to study the impact of noise power mismeasurements on the correlation coefficient estimates. Ground clutter filtering was applied to time series using the CLEAN-AP filter (Warde and Torres 2009), and point clutter removal was employed to minimize the impact of noise power increases, which affected only certain range locations. The reflectivity fields were censored with a 2-dB SNR threshold (standard for WSR-88D reflectivity and |ρhυ(0)| measurements) with respect to the RBNE noise powers.
The correlation coefficient fields in Figs. 5c and 5d were obtained from test case 1 data using the lag-0 and lag-1 estimators with the WSR-88D calibration noise power values. A visual comparison of the two fields reveals that the application of the lag-0 estimator results in a field that is spatially more coherent than the one obtained using the lag-1 estimator. This is especially visible in regions farther away from the radar, where the average συ increases because of the antenna beam broadening. Such a result is in agreement with the simulations that show the SD of the lag-1 estimator to be much larger than that of the lag-0 estimator for συ larger than ~4 m s−1. Visually, the best field in terms of spatial coherency and valid estimate coverage is obtained when the lag-0 estimator is used in combination with RBNE (Fig. 5e). The latter is supported by Table 1, which shows a substantial increase in both the valid estimate and coverage percentages when RBNE is used. Figure 5f presents a difference field obtained by subtracting |ρhυ(0)| values in Fig. 5e from those in Fig. 5c. The mean of the valid estimates with SNRs between 2 and 20 dB in Fig. 5f is 0.0166. Because the WSR-88D calibration overestimated noise powers (Fig. 5a), this corroborates the results of the simulations (shown in Fig. 4), which suggest that noise power overestimation increases the overall bias of the |ρhυ(0)| estimates.
Correlation coefficient fields for test case 2 are presented in Figs. 6c–e. In this particular case, the WSR-88D calibration overestimated noise power levels significantly more than in test case 1. As a result, the lag-1 estimator produced visibly more valid estimates than the lag-0 estimator using the WSR-88D calibration noise power values (cf. Figs. 6c and 6d). However, the use of the lag-0 estimator with the RBNE technique yielded the field (Fig. 6e) that is visually even more improved compared to those in Figs. 6c and 6d than in test case 1. Clearly, the substantial increase in the valid estimate coverage compared to Fig. 6c is due to the extensive noise power overestimation by the WSR-88D calibration. This is reflected in the results in Table 1, which shows about 28% more valid estimates and 41% larger valid estimate coverage in Fig. 6e than in Fig. 6c, whereas the differences between Figs. 5e and 5c are about 15% and 29%, respectively. The mean difference at data points with SNRs between 2 and 20 dB shown in Fig. 6f is 0.01, which further corroborates the results of simulations shown in Fig. 4.
The third test case is presented in Fig. 7. The site used to collect data is located in the plains area, and the noise power is fairly flat in azimuth when skies are clear. As opposed to the other two cases, the legacy WSR-88D calibration procedure appears to provide a noise power estimate that is fairly accurate in precipitation-free conditions. Thus, it is speculated that the variations in RBNE estimates, shown in Fig. 7b, are caused by radiation from precipitation. Consequently, it is visible in Fig. 7b that RBNE produces estimates that are higher than those from WSR-88D calibration at about half the azimuth positions. Visual comparison between correlation coefficient fields generated using the lag-0 estimator with legacy and RBNE noise powers (Figs. 7c and 7e) reveals no discernible differences to the naked eye. However, the results in Tables 1a and 1b reveal the actual slight decrease in the number of valid estimates produced by the lag-0 estimator in combination with RBNE. This is to be expected because RBNE accounted for the noise power increase caused by the precipitation, which resulted in slightly fewer valid estimates but overall more accurate correlation coefficient measurements. Figure 7f shows that the majority of significant differences (i.e., larger than −0.01) are located on the edges of the weather system far away from the radar where, because of low SNR |ρhυ(0)|, estimates have high standard deviation and are therefore unreliable. Thus, the estimate differences are located in areas that contain a large number of invalid estimates (i.e., pink fringe). This is the likely reason why the differences between Figs. 7c and 7e are not detectable by visual comparison (i.e., in both figures these are located in the pink fringe areas). In contrast to the other two test cases, the mean difference at data points with SNRs between 2 and 20 dB shown in Fig. 7f is slightly negative and amounts to −0.0015. Clearly, this is because the RBNE estimates are either higher or comparable to the legacy ones at most azimuths. As in the other two cases, the correlation coefficient field produced using the lag-1 estimator (Fig. 7d) is of diminished quality compared to those in Figs. 7c and 7e.
In this paper, a comparison of the two correlation coefficient estimators typically employed in practice is given. One is the lag-0 estimator, which uses the measurements of noise powers in the horizontal and vertical channels, and is therefore sensitive to uncertainties introduced by these measurements at low to moderate SNRs. The other is the lag-1 estimator, which does not utilize the noise power measurements. Using simulations, the two correlation coefficient estimators were compared in terms of bias and standard deviation, assuming exactly known noise powers (which removed the effects of noise power mismeasurements on the lag-0 estimator). The comparison suggests that the quality of estimates from the lag-1 estimator declines rapidly as the spectrum width increases. This is especially pronounced for data from WSR-88D surveillance scans that typically have long unambiguous ranges but narrow Nyquist cointervals. This precludes the exclusive use of lag-1 estimator in surveillance scans. Further, the sensitivity of the lag-0 estimator to errors in noise power measurements is quantified.
Errors in noise power measurements are present regardless of whether the system noise power is obtained using a calibration procedure executed in between weather scans or estimated on a radial basis from the data. The conceptual advantage of the latter is that such an approach captures noise power variations in azimuth, elevation, and time. If high-quality radial-based noise power estimates are used, then the noise-induced bias in the correlation coefficient estimates is reduced, resulting in more accurate estimates and an increased number of valid estimates in cases when the noise power is overestimated by the legacy procedure (e.g., WSR-88D calibration). In that regard, the effects of the radial-based noise power estimation on the quality of lag-0 correlation coefficient estimates were investigated. Using analytical derivations and simulations, it was shown that errors in noise power measurements induce measurable bias that is added to the inherent bias of the lag-0 correlation coefficient estimator. Next, the three real data examples were presented. RBNE produced estimates at all azimuths, thus exhibiting zero failure rates on the three datasets. In the first two, the WSR-88D calibration procedure significantly overestimated noise powers compared to those produced by RBNE. It was demonstrated herein on these two examples that the lag-0 estimator using RBNE produced visibly improved correlation coefficient fields compared to those obtained using the lag-1 estimator and the lag-0 estimator with the WSR-88D calibration noise power values. The improvements are visible as the decrease in the number of invalid correlation coefficient estimates (i.e., estimates larger than one). Moreover, the areas where the gain in valid estimates is visible exhibit consistently high correlation coefficient values (i.e., larger than 0.97). This is an indication that the noise powers overestimated by the WSR-88D calibration procedure were computed by RBNE with improved accuracy (e.g., had RBNE underestimated the noise powers, these regions would likely exhibit unusually small correlation coefficient values). This is in agreement with the results of analytical derivations and simulations, which suggests that significant noise power mismeasurements can visibly degrade the correlation coefficient fields computed with the lag-0 estimator by increasing the bias of estimates. This indicates that the radial-based noise power estimation has the potential to visibly improve the fields of the correlation coefficient in appearance and accuracy in cases when noise powers exhibit azimuthal variations in excess of 1 dB or when the legacy noise power estimation produces large errors. In the third real data example, the legacy WRS-88D calibration yielded an appropriate result, so it was speculated that the visible noise power azimuthal variations (measured by RBNE) stemmed from radiation caused by precipitation. These variations did not cause visible differences between the correlation coefficient fields produced using the WSR-88D calibration and RBNE. However, quantitative analysis revealed a slight decrease in the number of valid estimates when RBNE was used. This was expected because RBNE captured the noise power increase at azimuths where radiation from storms was present and produced values that were slightly above the values supplied by the WSR-88D calibration. Consequently, in regions where RBNE produced higher noise power estimates, the overall correlation coefficient values increased, resulting in a marginally smaller number of valid estimates but generally more accurate measurements.
The author would like to thank Sebastián Torres, Valery Melnikov, and Bradley Isom for providing comments that improved the manuscript. Funding was provided by NOAA’s Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA11OAR4320072, U.S. Department of Commerce.