The Hybrid-Scan Estimators: Exploiting WSR-88D Split Cuts to Improve the Quality of Polarimetric-Variable Estimates
1. Introduction
After its establishment, the NEXRAD program provided the U.S. National Weather Service (NWS) with one of the most critical instruments needed to achieve their mission (Crum and Alberty 1993): the Weather Surveillance Radar-1988 Doppler (WSR-88D). Furthermore, the national weather radar was enhanced with dual-polarization capabilities in 2012. With this enhancement, polarimetric variables (differential reflectivity, differential phase, and copolar correlation coefficient) have become a fundamental tool for better interpretation, forecasting, and warning of hazardous weather events. Fields of these variables complement the traditional spectral moments (reflectivity, Doppler velocity, and spectrum width) and provide forecasters with critical information (Zrnić and Ryzhkov 1999). Also, several algorithms ingest polarimetric-variable data and generate products that support NWS forecasters’ warning decision process. For these reasons, improving the quality of polarimetric-variable data on the WSR-88D has been an important long-standing effort.
To avoid unnecessary complications during the deployment of dual polarization on the NEXRAD network, the signal processor initially used simpler conventional techniques to estimate the polarimetric variables, leaving room for improved performance (Zrnić et al. 2008). Years of experience with the upgraded WSR-88D have demonstrated that polarimetric-variable estimates are very sensitive to contamination from ground clutter (Friedrich et al. 2009), interference, and noise, much more so than the spectral moments. For example, as the signal-to-noise ratio (SNR) decreases, the statistical performance of polarimetric-variable estimators degrades more rapidly than that of the spectral moments, significantly impacting their interpretation and the quality of downstream products (e.g., Ivić 2014).
WSR-88D scan strategies [also referred to as volume coverage pattern (VCP)] implement split cuts at low elevations as a means to mitigate range-and-velocity ambiguities [Office of the Federal Coordinator for Meteorological Services and Supporting Research (OFCM) 2006]. In split cuts, the same elevation angle is scanned twice using two different pulse repetition times (i.e., there are two 360° azimuthal rotations of the antenna at the same elevation angle). A long pulse repetition time (PRT) is used in the first half of the split cut (called the surveillance scan and referred to as the CS scan) for better spatial coverage, and a short PRT is used in the second half of the split cut (called the Doppler scan and referred to as the CD scan) to reduce the occurrence of velocity aliasing. Operationally, reflectivity Z, differential reflectivity ZDR, differential phase ΦDP, and copolar correlation coefficient ρHV are obtained from the CS scan, whereas Doppler velocity υ and spectrum width συ are obtained from the CD scan using a “range unfolding” technique that relies on the reflectivity from the CS scan (OFCM 2006).
While currently not used operationally, polarimetric variables estimated using data from the CD scan could result in more accurate estimates than those from the CS scan (Ivić and Isom 2014; Isom 2015; Schvartzman et al. 2017). To illustrate this, we use a conventional Monte Carlo approach with a time series simulator (Curtis 2018) to compare the statistical performance of polarimetric-variable estimates from the CS and CD scans at the lowest elevation angle in VCP 12 (OFCM 2017). The inputs to the time series simulator are the true weather signal characteristics, the Nyquist velocity, and the number of samples per dwell. For each set of input parameters, the time series simulator generates 10 000 realizations of in-phase and quadrature phase (IQ) data from which estimates of the polarimetric variables are obtained. Figure 1 shows the bias and the standard deviation of polarimetric-variable estimates from the CS and CD scans as a function of the SNR for two values of ρHV: 0.9 and 0.99. The CS-scan PRT is TS = 3.12 ms (υaS = 8.3 m s−1) with MS = 15 samples per dwell, and the CD-scan PRT is TD = 0.986 ms (υaD = 26.2 m s−1) with MD = 40 samples per dwell. A set of benchmark radar variables from the WSR-88D System Specifications (WSR-88D SS) was used for these simulations: συ = 2 m s−1 and ZDR = 0 dB. Note that depending on the value of ρHV, the range of SNRs for which CD estimates have better statistical performance than CS estimates is different. For the lower value of ρHV, CD estimates have better performance at low to medium SNR (up to ~12 dB); for the higher value of ρHV, the performance of CD estimates is better for all SNRs in the range of interest. This shows that the statistical performance of the radar-variable estimators is not only a function of the number of samples M; it also depends on the PRT and the characteristics of the weather signal. Considering the dependency on acquisition parameters only (i.e., M and PRT), at high SNR, the bias and variance of radar-variable estimates are inversely proportional to the dwell time (M × PRT). Thus, if the dwell times in the CS and CD scan are about the same (as is typically done on operational VCPs), the quality of estimates is about the same regardless of the different number of samples. However, when the SNR decreases, using the CD-scan data with the larger number of samples can result in better-quality estimates.
(left) Bias and (right) standard deviation of polarimetric-variable estimates as a function of SNR for VCP 12 parameters using CS and CD data and for ρHV = 0.90 and 0.99.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
The need to produce dual-polarization data with the best possible quality coupled with the potential data “redundancy” present in the split-cut data of most operational VCPs is a strong motivation to explore the use of polarimetric-variable estimates from the CD scan in split cuts. Operating on a single range bin at a time, the proposed hybrid-scan estimators (HSE) of the three polarimetric variables choose CD estimates if both the bias and variance of the observed estimates are expected to be lower than those of their CS-scan counterparts. Otherwise, the HSE choose estimates from the CS scan. Thus, compared to the conventional estimators that use the CS-scan data only, the HSE are expected to produce estimates with the same quality (when choosing the CS data) or estimates with better quality (when choosing the CD data). As an additional benefit, choosing a CD over a CS estimate can result in improved ground-clutter mitigation due to the typically larger number of samples per dwell and the larger Nyquist cointerval corresponding to the CD scan.
The rest of the paper is organized as follows. The hybrid-scan estimators are described in section 2. In section 3, a statistical analysis is carried out for a wide range of simulated weather-signal parameters; this confirms that, under certain conditions and compared to conventional estimators, the HSE can significantly improve the quality of the polarimetric variables. In section 4, the performance of the HSE is demonstrated by processing radar data from WSR-88D systems using both the conventional estimators (i.e., CS-scan data only) and the HSE (i.e., best choice between CS- and CD-scan data). In section 5, we conclude with a summary and recommendations for operational implementation.
2. The hybrid-scan estimators
Operating on a single range bin at a time, the HSE of the polarimetric variables choose either the CS or CD data in split cuts of WSR-88D VCPs. A high-level flowchart of the technique is shown in Fig. 2. The first step checks that the range bin under consideration does not contain unrecoverable overlaid echoes in the CD data. If there are unrecoverable overlaid echoes, the CD data cannot be used, and estimates from the CS scan are chosen. Otherwise, if both the CS and CD data are available, the HSE choose the data with better expected quality. To do this, the technique uses the expected bias and standard deviation of estimates from the CS and CD scans. If both the magnitude of the expected bias and the expected standard deviation of the CD estimates are smaller than the corresponding ones for the CS estimates, the CD data are selected. Otherwise, the CS data are selected (as with the conventional estimators). Consequently, by choosing the better of the CS or CD data, the HSE achieve the same performance (when they choose the CS data) or better performance (when they choose the CD data) than using conventional estimators. Two methods of choosing between the CS and CD data are described next: theoretical expressions or precomputed lookup tables.
Flowchart of the HSE. If there are overlaid echoes in the CD data, the CS data are chosen. Otherwise (i.e., there are no overlaid echoes in the CD data), the expected bias and standard deviation of the polarimetric variables are computed or retrieved for both the CS and CD data. If both statistics are lower for the CD data, they are chosen. Otherwise, the CS data are chosen.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
a. HSE decision using theoretical expressions
A straightforward way of choosing between the CS and CD data is by comparing their expected statistical performance using theoretical expressions. Melnikov and Zrnić (2004) derived closed-form expressions for the bias and standard deviation of the traditional polarimetric-variable estimators. These equations were derived using perturbation analyses, and they are accurate only under certain conditions. Nevertheless, the HSE decision is based on the relative performance between biases and standard deviations of estimates from the CS and CD scans, thus the accuracy of the theoretical expressions is not critical as long as their relative performance (i.e., which one is better) is preserved in the approximations. This hypothesis will be verified in the next section.
While the use of theoretical expressions is a straightforward method to choose the estimate with best expected quality, closed-form expressions may not always be available. This is the case for nontraditional estimators such as multilag estimators (Lei et al. 2012) or hybrid estimators (Ivić 2016) of the polarimetric variables. In these situations, we must rely on lookup tables as described next.
b. HSE decision using lookup tables
When closed-form expressions are not available for the bias and standard deviation of polarimetric-variable estimators, we can use lookup tables obtained through simulations. Weather-like time series data simulations can be used to derive decision tables that indicate the estimator with better statistical performance (i.e., the one from the CS or the CD scan). To generate the lookup tables, we used the simulator by Curtis (2018) to generate 100 000 realizations of weather-like IQ signals with characteristics in the following space: SNR from 2 to 20 dB in 0.5-dB steps, ρHV from 0.9 to 1 in 0.005 steps, and συ from 0.25 to 6 m s−1 in 0.25 m s−1 steps. The mean Doppler velocity was set to 0 m s−1, ΦDP to 0°, and ZDR to 0 dB because the statistical performance of polarimetric-variable estimators is independent of the signal phase (υ and ΦDP) and has only a minor dependency with ZDR in the range of interest (this was verified by comparing the lookup tables for different values of ZDR and observing negligible differences). In general, different sets of tables are needed for each set of acquisition parameters in the split cuts of the scan strategy (i.e., sets of M and PRT for the CS and CD scans). Simulated time series data are fed to the polarimetric-variable estimators; bias and standard deviations are computed for the CS and CD data as a function of SNR, ρHV, and συ; and the results are stored for use in the HSE decision process.
The number of lookup tables can be reduced in half if we combine them to follow the HSE decision logic. That is, if both the magnitude of the bias and the standard deviation of CD estimates are lower than their counterparts for the CS estimates, the corresponding entry in the lookup table is set to 1; otherwise, it is set to 0. Thus, a value of 0 will be used in the HSE decision process to choose the CS data, and a value of 1 to choose the CD data.
While the use of lookup tables provides a general approach that is compatible with any estimator, the generation of lookup tables on finer grids is time-consuming, and their storage may not be trivial. In addition, real-time access of multidimensional lookup tables can be computationally expensive. Another disadvantage is that lookup tables are derived for specific pairs of acquisition parameters (i.e., M and PRT for the CS and CD scans), and each scan strategy could require the generation of multiple sets of lookup tables.
c. HSE decision inputs
Whether the HSE decision is made using theoretical expressions or lookup tables, the process relies on knowledge of acquisition parameters (M and PRT) and signal characteristics (SNR, συ, and ρHV). Acquisition parameters are defined in the VCP and are either direct inputs when using theoretical expressions or are used to select appropriate sets of tables when relying on lookup tables. Signal characteristics are derived from the data: SNR and ρHV from the CS scan, and συ from the CD scan. The robustness of the HSE decision process to errors of estimates is studied in section 3.
If using theoretical expressions, the HSE decision inputs can be used directly in the bias and standard deviation expressions in Eqs. (1)–(5). However, if ρHV is larger than 1, a value of 1 is used in the equations. Conversely, if using lookup tables, the HSE decision inputs are used to index the set of tables for the given acquisition parameters. In this case, because the space of signal characteristics used to generate the lookup tables is only a subset of all possible situations found in practice, we must define decision rules for when the signal characteristics fall outside of the lookup table domain. Whereas one could extend the domain of the lookup tables to avoid this, our evaluation showed that the adopted domain leads to a good compromise between performance improvement and computational complexity. Also, there are cases outside the lookup table domain for which it is fairly easy to predict the performance of estimators. Otherwise, if there is no clear winner, we favor the selection of the CS data, which leads to performance equivalent to that of the conventional estimators. Based on this, when the signal characteristics fall outside of the lookup table domain, we use the following rules. For SNRs less than 2 dB, the HSE decision is irrelevant since data corresponding to these SNRs are typically censored (or thresholded). For values of ρHV larger than 1 (which are possible due to statistical errors in the CS data), the CD data are chosen as these are clearly better in all cases for ρHV values approaching 1. In addition, this provides an opportunity for the ρHV HSE to change an invalid estimate into a valid one as will be illustrated in section 4. Similarly, for συ larger than 6 m s−1, the CD data are chosen. For all other cases, the CS data are chosen as they result in identical performance as the conventional estimators.
3. Performance of the hybrid-scan estimators
In this section, we quantify the improvement provided by the HSE compared to the conventional estimators and investigate the space of signal characteristics in which these improvements are realized. We also analyze the robustness of the decision process to errors of estimates.
Figure 3 shows the differences in the bias (left column) and the standard deviation (right column) of polarimetric-variable estimates between the CS and CD scans as a function of SNR and ρHV. For these plots, we used the acquisition parameters corresponding to the lowest elevation angle in VCP 12 as in Fig. 1. For all the simulations and without loss of generality, υ, ΦDP, and ZDR were constant (0 m s−1, 0°, and 0 dB, respectively); as mentioned before, the dependency of estimator statistics on these parameters is negligible in this context. Finally, we chose a representative spectrum width value of συ = 2 m s−1, which is the median in severe storms (Fang et al. 2004). For these plots, ρHV varies from 0.9 to 1 in steps of 0.005 (unitless), which are the values typically associated with hydrometeors (except for giant hail or large, wet aggregates, which exhibit lower values of ρHV). The SNR varies from 2 to 20 dB in steps of 0.5 dB. It is important to note that as the SNR decreases from ~5 dB, the quality of the polarimetric-variable estimates degrades very rapidly (see Fig. 1). As mentioned before, it does not make sense to look at SNRs below 2 dB because the WSR-88D typically uses an SNR threshold of 2 dB to remove nonsignificant data from displays. In these plots, positive differences (in shades of blue) correspond to regions where estimates from the CD scan are statistically better (i.e., either lower bias magnitude or lower standard deviation, depending on the type of plot), and negative differences (in shades of red) correspond to regions where estimates from the CS scan are statistically better. Each plot also includes two black curves corresponding to the decision boundaries obtained with theoretical expressions (dotted line) and with lookup tables (solid line). Note that the decision boundaries separate the signal-characteristic domains in which either the CS or the CD data are chosen by the HSE (recall from the previous section that the CD data are chosen only if both the magnitude of the expected bias and the expected standard deviation are lower than their CS counterparts). The decision boundaries can be used to assess the performance of the HSE relative to that of the conventional estimators. For a specific radar variable, the HSE is better than the conventional estimator in the region left of the curve (i.e., where the CD data are chosen by the HSE). Right of the curve, both estimators have the same performance (i.e., the CS data are chosen by the HSE).
Differences in the (left) bias and (right) standard deviation of polarimetric-variable estimates between the CS and CD scans as a function of SNR (dB) and ρHV. Positive differences (blue colors) correspond to regions where the statistical performance of the CD data is better; negative differences (red colors) indicate the opposite. Decision boundaries obtained with theoretical expressions and with lookup tables are shown with dotted and solid lines, respectively.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
All plots in Fig. 3 exhibit similar behavior, except for the difference in bias of ΦDP estimates. As mentioned before, since the ΦDP estimator is unbiased, only the standard deviation of ΦDP estimates is used by the HSE to choose between the CS and CD data. Generally speaking, the HSE have better performance than the conventional estimators for low-to-medium SNR for all ρHV values in the range under analysis. As ρHV increases, the SNR range in which the HSE have better performance than the conventional estimators increases too.
The overall agreement between the solid and dotted curves in Fig. 3 indicates that the HSE decisions based on theoretical expressions and lookup tables are similar. Differences stem from the fact that theoretical expressions are approximations and the domain of the tables is discretized. However, the curves depart from each other in regions of small statistical differences between the CD and CS estimators. In other words, the region between them corresponds to very small differences in statistical performance. Finally, the transitions between positive and negative difference values are gradual, suggesting that using estimates as inputs to the HSE decision process should not result in significant performance degradation (this is quantified later).
Similar to Fig. 3, Fig. 4 shows the differences in the bias (left column) and the standard deviation (right column) of polarimetric-variable estimates between the CS and CD scans as a function of SNR and συ. The simulation parameters are the same as in Fig. 3 except that ρHV was fixed at 0.9, and συ was varied from 0.25 to 6 m s−1 in steps of 0.25 m s−1. We show the case for ρHV = 0.9 (the lower bound in our analysis) because it is more revelatory; as ρHV approaches 1, the CD data are almost always better than the CS data (see Fig. 1) resulting in trivial (almost all blue) plots. As with Fig. 3, the HSE have better performance than the conventional estimators for low-to-medium SNR for all συ values in the range under analysis. As συ increases, the SNR range in which the HSE have better performance than the conventional estimators increases too. Here, we also see the overall agreement between the solid and dotted curves indicating that the HSE decisions based on theoretical expressions and lookup tables are similar.
As in Fig. 3, but as a function of SNR (dB) and συ (m s−1).
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
As mentioned before, in practice, the HSE decision involves the use of estimates as inputs into theoretical expressions or lookup tables. Namely, the performance of the HSE depends on the quality of SNR, ρHV, and συ estimates. At low SNR, the accuracy of these parameters depends on having an accurate estimate of the noise power. On the WSR-88D, this is provided by the radial-by-radial noise estimator (Ivić et al. 2013). Whereas other radar-calibration parameters (e.g., system ZDR bias and initial system differential phase) may affect the quality of the HSE data, this is no different than the effects that would be seen in the data obtained with conventional estimators. Thus, we concentrate on quantifying the impacts to the HSE decision due to using estimates (as opposed to true values) of SNR, ρHV, and συ. Figure 5 shows the differences in the root-mean-square error (RMSE) of the polarimetric-variable estimates obtained with the practical HSE, which use estimates as inputs, and the ideal HSE, which use the true simulated values of the radar variables as inputs. For this figure, we used the same simulation parameters as in Fig. 3. The reader should note that any nonzero RMSE differences correspond to decision differences between the practical and ideal HSE. Since we accept the ideal HSE decision as the correct one, nonzero RMSE differences correspond to cases where the practical HSE made wrong decisions (i.e., they incorrectly chose the estimate with the worse expected statistical performance). Wrong decisions are more likely to occur around decision boundaries; fortunately, this is where statistical performance differences between the CS and CD data are small. That is, in these cases, decision errors are counteracted by the small performance differences, resulting in small penalties. As we move away from the decision boundaries, any wrong decisions are penalized with larger statistical performance differences. However, as we argued before, the gradual transition in the statistical performance differences between the CS and CD data coupled with the expected quality of the estimates that are inputs to the decision process makes the HSE decision robust to errors of estimates. This is confirmed in Fig. 5, where very small RMSE differences exist around the decision boundaries, converging to zero (i.e., no decision differences) away from those. For the parameters in this figure, the overall maximum absolute RMSE differences are 0.05 dB, 0.3°, and 0.007, for ZDR, ΦDP, and ρHV, respectively. These values are very small (about one order of magnitude smaller than the required precision on the WSR-88D), validating our hypothesis that the HSE decision is robust to errors of input estimates.
Differences in the RMSE of the polarimetric-variable estimates obtained with the HSE using estimates as inputs and the HSE using the true values of the radar variables as inputs as a function of SNR (dB) and ρHV. Decision boundaries obtained with theoretical expressions and with lookup tables are shown with dotted and solid lines, respectively.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
4. Demonstration using real data
In this section, the performance of the proposed technique is demonstrated by processing radar data from WSR-88D systems using both the conventional estimators and the HSE. For the HSE, the decision is based on (computationally simpler) theoretical expressions, but as shown in section 2, the performance is equivalent to using lookup tables (LUTs). We processed hundreds of cases with varied meteorological characteristics from different WSR-88Ds and verified our general expectation that the HSE improve over the conventional estimators in regions of low-to-medium SNR and relatively high correlation coefficient or wide spectrum width. Although the cases presented in this section illustrate the performance of the HSE for typical VCPs, we tested the technique with all operational VCPs yielding similar performance improvements. This is because the HSE improvement is mostly realized by the larger number of samples in the CD data. Herein, we present the results for four relevant cases: two widespread stratiform precipitation cases and two severe convective storm cases. These were chosen to exemplify different aspects of the performance of the HSE.
Figure 6 shows fields of reflectivity and Doppler velocity at an elevation angle of 0.5° collected during a snow event with the KDLH radar in Duluth, Minnesota, on 16 March 2013 using VCP 32 (OFCM 2017). For clarity, Fig. 7 contains the corresponding color bars for Fig. 6 and for all subsequent figures with radar-variable fields. Widespread precipitation systems like this are typical during the winter season and can extend beyond the end of the CD-scan maximum unambiguous range (raD ~ 148 km), which results in a large number of CD data with range overlaid echoes. In cases like the one in Fig. 6, the Doppler velocities of stronger first-trip echoes are typically recovered while those of weaker higher-order-trip echoes are unrecoverable. Unrecoverable-overlaid CD data are censored (this is often referred to as “purple haze” due to the color used for these range bins as seen in the right panel of Fig. 6) and cannot be used by the HSE. In these situations, a visual artifact that originates from a sharp data-quality transition at raD is apparent in HSE fields. This occurs because first-trip echoes are mostly recoverable (and HSE can use the better-quality CD data) and second-trip echoes are mostly unrecoverable (and HSE can use only the lower-quality CS data). An example of this can be seen in the center panels of Fig. 8.
Fields of (left) reflectivity and (right) Doppler velocity from data collected with KDLH (Duluth, MN) at 0112 UTC 16 Mar 2013 and an elevation angle of 0.5° using VCP 32. The maximum unambiguous range for the CD data is ~148 km, and range bins with unrecoverable overlaid echoes are purple.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
Color bars corresponding to the fields in Figs. 6 and 8–12.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
Fields of (top) ZDR, (middle) ΦDP, and (bottom) ρHV corresponding to the case in Fig. 6 using the (left) conventional estimators, (center) HSE, and (right) modified HSE. The color bar number 1 (see Fig. 7) is used for ΦDP.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
Figure 8 shows fields of ZDR (top), ΦDP (middle), and ρHV (bottom) corresponding to the case in Fig. 6 and obtained using conventional estimators (left), HSE (center), and modified HSE (right). The center panels in this figure illustrate the visual artifact created by the large differences between the quality of the CS and CD data (best observed in ρHV panels), which, as seen in the corresponding right panels, is mitigated when using the modified HSE. As expected, because ρHV is relatively high and the SNR is generally low, the fields produced with both versions of the HSE exhibit significant data-quality improvements. Compared to the conventional estimators on the left panels, weather features in the HSE data are more clearly defined. This example also illustrates the improvement in ground-clutter-filter performance, which is more apparent in the fields of ρHV. Figure 9 shows zoomed-in fields of ρHV from Fig. 8 to highlight the region close to the radar where enhanced clutter filter performance of the HSE over the conventional ρHV can be observed. That is, the CD data are less biased (increased ρHV values) because the ground-clutter filter does a better job removing the clutter contamination with the larger number of samples per dwell and larger Nyquist cointerval associated with the CD scan. Moreover, there is a higher number of valid ρHV estimates (i.e., ρHV ≤ 1), which is consistent with the selection criteria defined in section 2c when the conventional ρHV estimate exceeds 1. Out of 521 307 range bins with significant returns, HSE chose the CD data for 40.9%, 35.2%, and 36.65% of the bins for ZDR, ΦDP, and ρHV, respectively. As expected, the modified HSE chose the CD data slightly less frequently (37.3%, 32.1%, and 33.5%, respectively). The percentage of invalid conventional ρHV estimates that turned into valid HSE ρHV estimates is 31.91% and 24.16% for the modified HSE.
Zoomed-in fields of ρHV from Fig. 8 to highlight a region with enhanced clutter filter performance in (right) the ρHV HSE data over (left) the conventional ρHV data.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
Figure 10 shows an example of a tornadic isolated supercell storm as observed with the KCRI radar in Norman, Oklahoma, on 19 May 2013 and at an elevation angle of 0.9° using VCP 212 (OFCM 2017). A comparison between the center and right panels also reveals significant data-quality improvements with more clearly defined features in the HSE data. In particular, the typical correlation coefficient depression (ρHV < 0.8) associated with the tornadic debris signature (TDS) as defined by Ryzhkov et al. (2005) becomes more evident. Circles in the ρHV panels of Fig. 10 emphasize the improvement of HSE over the conventional estimator for the identification of the TDS. That is, regions of ρHV < 0.8 are much better defined in the HSE data. This is significant because ρHV has the best discriminating power for polarimetric tornado detection. The HSE data could make analysis and interpretation of this crucial signature easier for forecasters and automatic algorithms. In this case, the improvement in the HSE data is due to high values of συ despite the very high SNR and low values of ρHV. This is consistent with the results in Fig. 4 and the selection criteria defined in section 2c when the conventional συ estimate exceeds 6 m s−1. Out of 108 379 range bins with significant returns in the entire plan position indicator (PPI) display, HSE chose the CD data for 56.4%, 56.3%, and 41.2% of the bins for ZDR, ΦDP, and ρHV, respectively. The percentage of invalid conventional ρHV estimates (ρHV > 1) that turned into valid HSE ρHV estimates is 20.2%.
Zoomed-in fields of (top left) Z, (middle left) υ, and (bottom left) συ, plus (center) conventional and (right) HSE fields of (top) ZDR, (middle) ΦDP, and (bottom) ρHV for data collected with KCRI (Norman, OK) at 2203 UTC 19 May 2013 and an elevation angle of 0.9° using VCP 212. Circles in the ρHV panels highlight a region where the improvement of ρHV HSE helps to more clearly identify the tornadic debris signature. The color bar number 2 (see Fig. 7) is used for ΦDP.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
Figure 11 shows another example of convective storms along a squall line (nontornadic) as observed with the KOUN radar in Norman, Oklahoma, on 4 April 2019 and at an elevation angle of 0.5° using VCP 212. The expected significant data-quality improvements in the HSE data are observed in all dual-polarization variables. In particular, regions of high ZDR values are better defined in the HSE data. In addition, this example also illustrates the improvement in ground-clutter-filter performance, which is more apparent in the fields of ρHV at close ranges. Out of 151 997 range bins with significant returns in the entire PPI, HSE chose the CD data for 41.1%, 39.3%, and 27.7% of the bins for ZDR, ΦDP, and ρHV, respectively. The percentage of invalid conventional ρHV estimates that turned into valid HSE ρHV estimates is 16.1%.
As in Fig. 10, but for data collected with KOUN (Norman, OK) at 0553 UTC 4 Apr 2019 at an elevation angle of 0.5° using VCP 212. Black arrows in the ρHV panels correspond to a region where the enhanced clutter filter performance in the HSE data can be observed. The color bar number 2 (see Fig. 7) is used for ΦDP.
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
The final example demonstrates the performance improvement of a WSR-88D algorithm due to better-quality input data obtained with the HSE. Figure 12 shows another example of a snow event as observed with the KOUN radar on 13 February 2012 at an elevation angle of 0.5° using VCP 221. The top plots show zoomed-in fields of Z (left) and HSE ρHV (right). The bottom plots are the outputs of hydrometeor classification algorithm (HCA; Park et al. 2009) using conventional (left) and HSE (right) data as inputs. These were obtained by reprocessing the two streams of base data (conventional and HSE) through the operational WSR-88D Radar Product Generator. The regions between the dotted arcs in the two HCA plots illustrate the downstream improvements provided by HSE data. In a background of range bins classified as dry snow (light blue), the HCA using conventional data shows speckles of ice crystals (pink) and biological scatterers (gray). The presence of biological scatterers in this region is extremely unlikely given the height of the radar beam (~3.1 km), the freezing temperatures, and the sparse spatial distribution of this class. Conversely, the HCA using HSE data only shows the presence of dry snow and ice crystals. As expected, improving the quality of the input base data with the HSE may result in more reliable algorithm outputs.
Zoomed-in PPI displays of (top left) Z, (top right) HSE ρHV, and hydrometeor classification (HCA) produced using (bottom left) conventional estimators and (bottom right) HSE. Data were collected with KOUN (Norman, OK) at 0124 UTC 13 Feb 2012 and an elevation angle of 0.5° using VCP 221. In the regions between the two dotted arcs, the HCA using conventional data indicated the (very unlikely) presence of biological scatterers (gray) while the HCA using HSE data resulted in a (more likely) classification of dry snow (light blue).
Citation: Journal of Atmospheric and Oceanic Technology 37, 2; 10.1175/JTECH-D-19-0071.1
5. Conclusions
In this paper, we introduced the hybrid-scan estimators (HSE) as a means to improve the quality of dual-polarization data on the WSR-88D. In the low-level split cuts, the same elevation angle is scanned twice using two pulse repetition times, and estimates from each of the two scans in a split cut (referred to as the CS and CD data) may exhibit different statistical performance. Based on the expected bias and standard deviation of the CS and CD estimates, the HSE choose the ones with better quality. That is, in the absence of unrecoverable overlaid echoes, HSE choose the CD data if both their expected bias and standard deviation are lower than those of the CS counterparts. This decision can be made based on theoretical bias and standard-deviation expressions or using lookup tables generated through a one-time simulation process. The use of theoretical expressions is straightforward; however, closed-form expressions may not always be available requiring the use of lookup tables. For the traditional polarimetric-variable estimators, we showed that the decisions with either method are similar.
Through simulations, we showed that, by carefully choosing the CD data, the HSE lead to improved quality of dual-polarization data for low to medium SNRs and high ρHV or wide συ. This also leads to fewer invalid ρHV estimates and enhanced ground-clutter mitigation. We also verified that using estimates of signal characteristics as inputs to the HSE decision process does not lead to significant performance degradation.
The HSE exploit redundant and available data from the split cuts on operational VCPs on the WSR-88D to improve the quality of the polarimetric variables, which benefits all consumers of these data. The performance of the HSE was illustrated with data collected with WSR-88Ds. The data were collected with VCPs including normal split cuts (VCP 32) and those that use systematic phase coding (referred to as SZ-2; Sachidananda and Zrnić 1999) on the CD scan (VCP 212 and 221). As expected, the use of the HSE led to data-quality improvements in all cases. The data-quality improvement was evident in the smoothness of the fields, the clearer definition of meaningful weather features, the increase in the number of valid ρHV estimates, and the more effective mitigation of ground-clutter contamination. Although we only included a few cases to illustrate the performance of the HSE, we tested the technique on over 300 cases covering a wide range of operational VCPs and have verified that HSE always results in data-quality improvements. In addition, we corroborated the performance improvement in the operational hydrometeor-classification algorithm when using HSE data. In general, all consumers of dual-polarization radar data (humans and algorithms) should benefit from the data-quality improvement provided by the HSE.
The analysis of a widespread winter precipitation case illustrated a visual artifact caused by the absence of valid CD data beyond its maximum unambiguous range. Abrupt data-quality transitions can occur across the maximum unambiguous range of the CD scan, since the HSE can only choose the (lower-quality) CS data beyond it. To mitigate a potentially distracting artifact, we proposed a modification of the HSE that artificially shifts the decision boundaries to gradually favor choosing the CS data close to the CD-scan maximum unambiguous range. This produces fields with blended data quality at the price of suboptimal decisions (i.e., choosing lower-quality CS data). In convective precipitation cases, both versions of the HSE result in almost identical performance.
As with other operational techniques that use the data from split cuts, the HSE assume negligible storm evolution in the time between the CS and CD scans, which is less than 20 s in VCPs typically used with convective storms. If there were enough storm evolution in the short time between the CS and CD scans, it may be possible for the CS and CD estimates for a given range bin to not correspond to the same underlying distribution of hydrometeors, invalidating a fundamental assumption in the HSE. This is more likely to occur with rapidly moving storms that are close to the radar and in the presence of strong spatial gradients. For our study, we processed several cases with these characteristics and did not observe any artifacts in the HSE fields due to storm evolution. This is likely due to a combination of a few factors. The first one is the nonhomogeneous spatial blending of the CS and CD data in HSE fields. That is, the HSE decision does not typically result in spatially continuous regions with data from a single scan (i.e., there are no well-defined areas with only CS or only CD data). The second one is that the biases that occur from a combination of rapid storm evolution and strong spatial gradients are likely masked by the underlying errors of dual-polarization estimates. That is, at the small spatial scales where any impacts would be noticeable, the noisiness of the fields is the main contributor to the spatial variability of the data. The final factor is that we only analyzed a limited number of cases. However, we do not expect rapid storm evolution to limit the operational applicability of HSE. In fact, the time difference between the CS and CD data at the same location is similar to the time difference between the first and last radials in a PPI of CS data. Nevertheless, we do not usually see artifacts at these boundaries in the conventional CS-only data. In addition, forecasters and algorithms are used to dealing with fields of reflectivity and Doppler velocity that are obtained from the CS and CD scans, respectively (OFCM 2006).
The HSE choose one of two polarimetric-variable estimates available in split cuts. Thus, a valid question is whether the two estimates can be combined to produce a new estimate with better quality. For unbiased estimates, a common way to do this is through a weighted average, where the weights are inversely proportional to the variance of the estimators. In principle, because we have knowledge of the variance of each estimator, we could optimally combine the CS and CD estimates to produce another estimate with better quality. However, two of the three traditional polarimetric-variable estimators are inherently biased, introducing significant additional complexity to the problem. Also, it is likely that combining correlation estimates (instead of radar-variable estimates) would work best, thus requiring access to intermediate data that may not be readily available operationally. While combining the CS and CD estimates might be possible, doing so would probably introduce more complexity, which could only be justified if accompanied by additional data-quality improvements. Such a determination is left as future work.
In summary, the proposed HSE technique is computationally simple and uses data that are readily available in split cuts of operational VCPs on the WSR-88D. Also, the conservative decision criterion ensures that the HSE preserve or improve the data quality compared to the conventional estimators. Since all current operational VCPs use split cuts at two or three elevations per volume scan, the use of the HSE can lead to operational benefits. In addition to downstream algorithms receiving inputs with better quality, forecasters can gain confidence in the radar data that support their forecast and warning decision process. Individual radar images are examined fleetingly in real time operations, and the higher clarity can make radar signatures easier to recognize thus supporting a more compelling conceptual-model recognition. In other words, through a relatively simple implementation, the HSE are likely to bring considerable operational benefits.
Acknowledgments
The authors thank Jami Boettcher, Donald Burgess, Igor Ivić, and anonymous reviewers for providing comments that improved the manuscript. We would also like to thank Clark Payne for reprocessing the base data through the WSR-88D Radar Product Generator, and Jami Boettcher for helping with data analysis and interpretation. Funding was provided by NOAA/Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA11OAR4320072, U.S. Department of Commerce.
REFERENCES
Crum, T. D., and R. Alberty, 1993: The WSR-88D and the WSR-88D operational support facility. Bull. Amer. Meteor. Soc., 74, 1669–1687, https://doi.org/10.1175/1520-0477(1993)074<1669:TWATWO>2.0.CO;2.
Curtis, C. D., 2018: Weather radar time series simulation: Improving accuracy and performance. J. Atmos. Oceanic Technol., 35, 2169–2187, https://doi.org/10.1175/JTECH-D-17-0215.1.
Fang, M., R. J. Doviak, and V. Melnikov, 2004: Spectrum width measured by WSR-88D: Error sources and statistics of various weather phenomena. J. Atmos. Oceanic Technol., 21, 888–904, https://doi.org/10.1175/1520-0426(2004)021<0888:SWMBWE>2.0.CO;2.
Friedrich, K., U. Germann, and P. Tabary, 2009: Influence of ground clutter contamination on polarimetric radar parameters. J. Atmos. Oceanic Technol., 26, 251–269, https://doi.org/10.1175/2008JTECHA1092.1.
Isom, B., 2015: Improving the quality of dual polarization estimates with multiscan data hybridization. 31st Conf. on Environmental Information Processing Technologies, Phoenix, AZ, Amer. Meteor. Soc., 11.3, https://ams.confex.com/ams/95Annual/webprogram/Manuscript/Paper265145/MultiScans_AMS2015.pdf.
Ivić, I. R., 2014: On the use of a radial-based noise power estimation technique to improve estimates of the correlation coefficient on dual-polarization weather radars. J. Atmos. Oceanic Technol., 31, 1867–1880, https://doi.org/10.1175/JTECH-D-14-00052.1.
Ivić, I. R., 2016: A technique to improve copolar correlation coefficient estimation. IEEE Trans. Geosci. Remote Sens., 54, 5776–5800, https://doi.org/10.1109/TGRS.2016.2572185.
Ivić, I. R., and B. Isom, 2014: Methods to improve fields of correlation coefficient estimates. Proc. Eighth European Conf. on Radar in Meteorology and Hydrology, Garmisch-Partenkirchen, Germany, TEC.P03, http://www.pa.op.dlr.de/erad2014/programme/ShortAbstracts/071_short.pdf.
Ivić, I. R., C. Curtis, and S. M. Torres, 2013: Radial-based noise power estimation for weather radars. J. Atmos. Oceanic Technol., 30, 2737–2753, https://doi.org/10.1175/JTECH-D-13-00008.1.
Lei, L., G. Zhang, R. J. Doviak, R. Palmer, B. L. Cheong, M. Xue, Q. Cao, and Y. Li, 2012: Multilag correlation estimators for polarimetric radar measurements in the presence of noise. J. Atmos. Oceanic Technol., 29, 772–795, https://doi.org/10.1175/JTECH-D-11-00010.1.
Melnikov, V., and D. S. Zrnić, 2004: Simultaneous transmission mode for the polarimetric WSR-88D: Statistical biases and standard deviations of polarimetric variables. NOAA/NSSL Interim Rep., 84 pp, http://www.nssl.noaa.gov/publications/wsr88d_reports/SHV_statistics.pdf.
OFCM, 2006: WSR-88D unit description and operational applications. Part D, Doppler Radar Meteorological Observations, Federal Meteorological Handbook No. 11, U.S. Department of Commerce/NOAA, FCM-H11D-2006, 218 pp., https://www.ofcm.gov/publications/fmh/FMH11/FMH11D-2006.pdf.
OFCM, 2017: WSR-88D products and algorithms. Part C, WSR-88D Meteorological Observations, Federal Meteorological Handbook No. 11, U.S. Department of Commerce/NOAA, FCM-H11C-2017, 394 pp., https://www.ofcm.gov/publications/fmh/FMH11/fmh11partC.pdf.
Park, H., A. V. Ryzhkov, D. S. Zrnić, and K. Kim, 2009: The hydrometeor classification algorithm for the polarimetric WSR-88D: Description and application to an MCS. Wea. Forecasting, 24, 730–748, https://doi.org/10.1175/2008WAF2222205.1.
Ryzhkov, A. V., T. J. Schuur, D. W. Burgess, and D. S. Zrnic, 2005: Polarimetric tornado detection. J. Appl. Meteor., 44, 557–570, https://doi.org/10.1175/JAM2235.1.
Sachidananda, M., and D. S. Zrnić, 1999: Systematic phase codes for resolving range overlaid signals in a Doppler weather radar. J. Atmos. Oceanic Technol., 16, 1351–1363, https://doi.org/10.1175/1520-0426(1999)016<1351:SPCFRR>2.0.CO;2.
Schvartzman, D., D. Warde, and S. Torres, 2017: Hybrid-scan estimator: Using split cut data to improve the quality of polarimetric variables. 33rd Conf. on Environmental Information Processing Technologies, Seattle, WA, Amer. Meteor. Soc., 8A.2, https://ams.confex.com/ams/97Annual/webprogram/Paper301298.html.
Zrnić, D., and A. Ryzhkov, 1999: Polarimetry for weather surveillance radars. Bull. Amer. Meteor. Soc., 80, 389–406, https://doi.org/10.1175/1520-0477(1999)080<0389:PFWSR>2.0.CO;2.
Zrnić, D., V. M. Melnikov, and I. Ivic, 2008: Processing to obtain polarimetric variables on the ORDA. NOAA/NSSL Rep., 55 pp., http://www.nssl.noaa.gov/publications/wsr88d_reports/RPT_DoNotDestroy.doc.
