• View in gallery

    Scattering-center weights used to simulate a pattern such that the main lobe is pointing at 0° azimuth and 2.4° elevation. The axes correspond to azimuth and elevation angles, and the color indicates the magnitude of the weight for each scattering center in logarithmic units. The scattering center in the direction of interest receives unit weight. The other scattering centers within 2° azimuth and 1° elevation of the direction of interest (inside the box outlined by white dashes) receive zero weight (shown as −50 dB here). The weights for all other scattering centers are based on their angular distance from the direction of interest.

  • View in gallery

    A one-dimensional cut of the normalized two-way antenna radiation pattern measured for KOUN (blue curve) and a simulated pattern (red curve) with its associated scattering-center weights (green dots). The WSR-88D specifications (solid black line) and the SENSR sidelobe requirements (dashed black line) are shown as references.

  • View in gallery

    (a) Reflectivity (top row), storm-relative mean radial velocity (middle row), and spectrum width (bottom row) at 0.5° elevation (left) for a storm observed by the KLRX radar at 0445 UTC 22 Jul 2014, along with simulated data for this case (center) without and (right) with moderate sidelobe degradation. (b) As in (a), but showing differential reflectivity in the top row, specific differential phase in the middle row, and correlation coefficient in the bottom row.

  • View in gallery

    Enlarged color bars used to display, from top to bottom, reflectivity, storm-relative mean radial velocity, spectrum width, differential reflectivity, specific differential phase, and correlation coefficient. These color scales are based on the standard color scales used by NWS forecasters in their official software.

  • View in gallery

    (top left) Reflectivity, (top right) storm-relative mean radial velocity, (bottom left) spectrum width, and (bottom right) correlation coefficient at 0.5° elevation for a storm observed by the KDFX radar at 2302 UTC 15 May 2014.

  • View in gallery

    (a) As in Fig. 3a and (b) as in Fig. 3b, but for a storm observed by the KGLD radar at 0039 UTC 23 May 2015.

  • View in gallery

    As in Fig. 5, but for a storm observed by the KMRX radar at 2005 UTC 7 Jul 2014.

  • View in gallery

    As in Fig. 5, but for a storm observed by the KLRX radar at 0445 UTC 22 Jul 2014.

  • View in gallery

    (left) Reflectivity and (right) storm-relative mean radial velocity of case 11 at 0.5° elevation with scores of (top) 1, (top middle) 2, (middle) 3, (bottom middle) 4, and (bottom) 5. The simulated patterns corresponding to the five rows from top to bottom are patterns A, B, D, G, and J. The data-quality impacts caused by increasing sidelobe levels can be clearly seen in the storm-relative mean radial velocity field in the circled area.

  • View in gallery

    (top) Scores for the simulated patterns (blue stars) and the fitted curve for case 7 (blue line). (bottom) The fitted curves for all 12 nonoutlier cases and the average best-fit curve (black line). The x axes are ΔISL in linear units (center column of Table 1). Note that the curves are fitted to minimize the mean square error and capture the trend of the scores. The average best-fit curve uses parameters that are the average of all of the nonoutlier cases.

  • View in gallery

    Model for the relationship between relative antenna sidelobe degradation and impact on forecasters. On the basis of the impact score, there are three regions: acceptable, marginal, and unacceptable. Within the acceptable region, the model curve falls into two separate regions: low sensitivity and high sensitivity. Sensitivity describes how quickly the impact score changes when the relative antenna sidelobe degradation changes.

All Time Past Year Past 30 Days
Abstract Views 176 176 0
Full Text Views 82 82 42
PDF Downloads 89 89 45

The Impact of Elevation Sidelobe Contamination on Radar Data Quality for Operational Interpretation

View More View Less
  • 1 Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, and NOAA/OAR/National Severe Storms Laboratory, Norman, Oklahoma
© Get Permissions
Free access

Abstract

To fulfill the evolving observational needs of the National Weather Service (NWS), future weather radar systems will have to meet demanding requirements. Designing such systems will likely involve trade-offs between system cost and operational performance. A potential cost driver for future weather radars that could cause significant data-quality impacts on forecasters is the required angular resolution and sidelobe performance, which are mainly dictated by the antenna radiation pattern. Typical antenna radiation patterns can be characterized by the width of the main lobe and their sidelobe levels, which are traditionally measured across the azimuthal and elevation dimensions. In this work, we study the impact of increasing sidelobe levels on NWS forecasters’ data interpretation during warning operations. The resulting impact model can be used by decision-makers to better understand the cost–benefit trade-offs inherent in any radar system design.

© 2020 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Feng Nai, feng.nai@noaa.gov

Abstract

To fulfill the evolving observational needs of the National Weather Service (NWS), future weather radar systems will have to meet demanding requirements. Designing such systems will likely involve trade-offs between system cost and operational performance. A potential cost driver for future weather radars that could cause significant data-quality impacts on forecasters is the required angular resolution and sidelobe performance, which are mainly dictated by the antenna radiation pattern. Typical antenna radiation patterns can be characterized by the width of the main lobe and their sidelobe levels, which are traditionally measured across the azimuthal and elevation dimensions. In this work, we study the impact of increasing sidelobe levels on NWS forecasters’ data interpretation during warning operations. The resulting impact model can be used by decision-makers to better understand the cost–benefit trade-offs inherent in any radar system design.

© 2020 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Feng Nai, feng.nai@noaa.gov

1. Introduction

By means of the Service Life Extension Program (SLEP), the current Weather Surveillance Radar-1988 Doppler (WSR-88D) network is expected to meet the National Weather Service’s (NWS) operational needs until 2040 (Radar Operations Center 2019). After that, it is anticipated that a replacement radar system will be required. The functional requirements for a replacement radar system comprise threshold requirements, which must be met by the replacement radars, and objective requirements, which the replacement radars should try to approach without compromising any threshold requirements (NOAA/NWS 2015). Most threshold requirements are set to match the performance of the WSR-88D and thus can be met with conventional radar technology.

It is likely that cost-efficient solutions to meet functional requirements will involve performance trade-offs, which should be evaluated in a manner that can lead to a preferred solution. For example, a dish-antenna system could replicate the performance of the WSR-88D but would have difficulties in meeting the objective one-minute-or-less volume-update-time requirement. On the other hand, a four-face phased-array system could achieve the desired volume update time but would require a larger (and more expensive) aperture to match the performance of the WSR-88D antenna in terms of angular resolution (Doviak 2017). Additionally, complex calibration would be needed to ensure that the quality of the polarimetric data from a phased-array system is comparable to that of the WSR-88D (Lei et al. 2015). To properly balance system cost and performance, it is important for decision-makers to understand the impact of not meeting one or more cost-driving threshold requirements on forecasters’ interpretation of radar data.

One potential cost-driving requirement that has significant operational impact is the spatial resolution. The spatial resolution (measured in the azimuth, elevation, and range dimensions) is tied to the radar’s ability to resolve small-scale weather features and to produce accurate observations in the presence of strong reflectivity gradients. Although both angular (azimuth and elevation) resolution and range resolution are important for weather radars, this study will focus on angular resolution. The angular resolution is mainly determined by the effective antenna radiation pattern, which includes the intrinsic antenna pattern and any antenna motion and signal processing effects. The effective antenna radiation pattern consists of a main lobe, which determines the transmitted and received power in the direction of interest, and sidelobes, which determine the transmitted and received power from directions away from the direction of interest. Hereinafter, the effective antenna radiation pattern will be referred to simply as the pattern. Since any realistic pattern has sidelobes, there is always some received power from directions other than the direction of interest; this is referred to as sidelobe contamination. In general, sidelobe contamination can be reduced if the sidelobes are reduced, but there are usually important trade-offs associated with modifying a phased-array antenna radiation pattern. For example, pattern sidelobe levels for a phased-array antenna can be reduced using tapering (i.e., an uneven weighting of the signals transmitted and/or received by each antenna element). In this case, if the beamwidth is to be preserved, the size of the antenna must be increased, which is more costly to build and operate (Doviak and Zrnić 2006).

Under most circumstances, the sidelobe contamination is much weaker than the received power from the main lobe. However, if the sidelobe contamination is comparable to or greater than the received power from the main lobe at a particular range location, the estimated radar variables are no longer an accurate representation of the atmosphere at the assumed location and can negatively impact forecasters’ interpretation and consequent assessment of possible threats. In the United States, the NWS is tasked with providing weather, water, and climate data; forecasts and warnings for the protection of life and property; and enhancement of the national economy (NOAA/NWS 2019). Therefore, the impacts on forecasters’ base data (i.e., the spectral moments and polarimetric variables) interpretation due to potential data-quality changes caused by sidelobe degradations is a critical component of the evaluation of any replacement system. While the sidelobe problem is well understood by radar engineers, the breadth of ways that sidelobe contamination manifests in the data is not well understood within the NWS operational community. This study presents a novel approach that directly connects antenna sidelobe levels to contamination that affects data interpretation and thus NWS operations.

Whereas sidelobe contamination is the sum of contributions from all directions other than the direction of interest, the way it manifests in the data can be separated into two categories. If the primary cause (e.g., a strong reflectivity gradient) of the sidelobe contamination in a plan position indicator (PPI) image is readily apparent (i.e., can be identified on the PPI of the same elevation), we refer to this type of contamination as azimuthal sidelobe contamination. On the other hand, if the primary cause of the sidelobe contamination in a PPI image is not readily apparent (i.e., the primary cause is from other elevations), we refer to this type of contamination as elevation sidelobe contamination. In the warning operations domain, azimuthal sidelobe contamination is easier to identify in the radar data because the primary cause is in the plane of collection. Because its presence can be readily verified using radar data from a single elevation angle, azimuthal sidelobe contamination has little impact on NWS forecasters’ data interpretation. However, identification of elevation sidelobe contamination requires interrogation of multiple radar variables (usually reflectivity, Doppler velocity, spectrum width, and correlation coefficient) from multiple elevations. Typically, forecasters analyzing and interpreting radar data in real time find it more challenging to identify elevation sidelobe contamination compared to its azimuthal counterpart. Furthermore, the data-quality impacts of elevation sidelobes on forecasters’ interpretation of radar data may have more critical implications because this type of contamination is more likely to occur when observing intense convective storms that must be interrogated for a potential severe thunderstorm warning and/or tornado warning. The storm type used in this study is the supercell, which has high potential for elevation sidelobe contamination, requires intense NWS forecaster cognitive resources for base-data interrogation, and presents one of the NWS warning domains where lives are at stake. In this warning domain, algorithms function primarily as a “safety net” (Andra et al. 2002) to possibly direct forecaster attention to a particular storm. While other applications that use radar data are also impacted by elevation sidelobe contamination, we chose to focus on its impact on forecasters’ interpretation of radar base data as a first step.

A practical way to study the data-quality impact of different elevation sidelobe levels on forecasters’ interpretation of radar data is through realistic simulations. The Signal Processing and Radar Characteristic (SPARC) simulator offers a flexible framework for studying the impact of signal-processing techniques and radar-system characteristics on radar-variable estimates required to support the NOAA/NWS weather-surveillance mission (Schvartzman and Curtis 2019). The SPARC simulator is a versatile, two-dimensional weather radar time series scenario simulator able to ingest archived fields of radar base data and produce time series [in-phase and quadrature (IQ)] data as would be observed by a given radar system. In contrast with other simulators, the SPARC simulator allows for an end-to-end evaluation that considers the interactions between radar subsystems (e.g., the transmitter, the antenna, the receiver), scanning strategies (e.g., the pulse repetition time, the number of samples per dwell), and the signal processing. This makes the SPARC simulator a suitable tool to study and better understand the implications of adopting different technologies (e.g., phased-array antennas) on the quality of radar data. However, the SPARC simulator as described in Schvartzman and Curtis (2019) can only simulate azimuthal sidelobe contamination; this is a significant limitation since certain severe weather events can introduce considerable elevation sidelobe contamination. For this work, we extend the SPARC simulator to produce radar data corresponding to radar systems with systematically larger sidelobe levels (in all directions) than the WSR-88D. Since our simulations do not assume a particular radar design, we adopt a reasonable sidelobe structure and degrade the sidelobes systematically. The degraded sidelobes can be the result of physical obstructions (struts, radome, lightning protection, etc.), the antenna type (e.g., reflector vs phased array), or other antenna characteristics.

The increased elevation sidelobe contamination may result in compromised data in the simulations, which are also observed in WSR-88D data. In this work, the term “compromised data” refers to any data-quality degradation that requires a forecaster to expend cognitive resources toward diagnosing the validity of the data itself. To quantify the impacts of data-quality degradation on forecasters, we developed a scoring system based on the extent of distractions (due to compromised data) to NWS forecasters’ interpretation of radar data during warning operations. We simulated 13 weather cases using antenna patterns with varying sidelobe levels, and the simulated data were analyzed and scored by an experienced forecaster. We used the analysis results to develop a model for the relationship between sidelobe degradation and the impact on forecasters’ interpretation of radar data. This model could be used as a part of the decision-makers’ evaluation process to identify a range of acceptable sidelobe levels and to improve their understanding of the cost–benefit trade-offs involving changing sidelobe levels for future radar designs.

The rest of the paper is organized as follows. Section 2 describes the extension of the SPARC simulator and its validation. Section 3 presents the case selection process, the scoring system we used to quantify the impacts of data-quality degradation on forecasters, and examples of our simulation results. Section 4 focuses on the model we developed to help decision-makers better understand the cost–benefit trade-offs involving increasing sidelobe levels. Section 5 concludes with a summary of our findings and discusses potential solutions to meet angular-resolution requirements more effectively with a WSR-88D replacement radar system.

2. Simulation method

The SPARC simulator provides a way to simulate weather data as observed by radars with varying azimuthal sidelobe levels (Schvartzman and Curtis 2019). Since the original SPARC simulator cannot simulate elevation sidelobe contamination, that capability was added for this study. The input data (i.e., archived WSR-88D data) to the simulator already contain effects from the intrinsic WSR-88D pattern, which are difficult to remove and must be accounted for in our simulations. Without precise knowledge of the intrinsic WSR-88D pattern in the input data, we could not quantify the sidelobe degradation of our simulated patterns using metrics such as peak sidelobe level. Instead, we quantified the relative sidelobe degradation of the simulated patterns with respect to the intrinsic WSR-88D pattern. To ensure our simulations are realistic, we compared simulated data to WSR-88D data and found similar compromised data due to sidelobes in both. The rest of this section is organized as follows. First, we provide an overview of the SPARC simulator. Only the most relevant steps are included, and the reader is referred to Schvartzman and Curtis (2019) for more details. Second, we describe how we modified the SPARC simulator to emulate the effects of 2D antenna patterns. Third, we introduce a metric to quantify sidelobe degradations relative to the sidelobes of the WSR-88D antenna pattern. Fourth, we compare the results obtained with the modified SPARC simulator with real WSR-88D data to validate the realism of the simulation results.

a. SPARC simulator overview

The SPARC simulator ingests archived WSR-88D base data (i.e., fields of spectral moments and polarimetric variables) and produces dual-polarization time series data corresponding to the same event but observed with differing radar designs and/or different scan strategies. The simulation process consists of four basic steps. First, archived WSR-88D data are processed to correct errors (e.g., velocity aliasing) and fill in missing data due to censoring, which ensures that all six radar variables are available at each range location that has significant weather signals. Then, these so-called conditioned fields of radar variables are interpolated to a fine azimuth-by-range grid, where the spacing of this grid depends on the scan strategy being simulated. For this work, since the sampling spacing of our simulated data are identical to the WSR-88D data, azimuthal interpolation is not carried out and data as sampled azimuthally by the WSR-88D are used. Each point in the “conditioned” field is referred to as a “scattering center.” Next, for each scattering center, the SPARC simulator uses the corresponding radar variables as inputs to a time series data simulator based on widely used spectral-shaping methods (e.g., Zrnić 1975; Galati and Pavan 1995) whose output is a dwell of simulated polarimetric time series data with the desired Doppler and polarimetric characteristics (i.e., SNR, velocity, spectrum width, differential reflectivity, differential phase, and correlation coefficient). The time series data for each scattering center represent the weighted sum of backscattered signals from many hydrometeors within a radar resolution volume determined by the WSR-88D pattern (in the azimuth-elevation plane) and range weighting function (in the range dimension). Thus, the simulated time series data for each scattering center already include the effects of the WSR-88D pattern. These effects are intrinsic to the simulated time series data and must be accounted for when attempting to simulate data that correspond to a different antenna pattern. The last step in the simulation process is weighting and summing the time series data from a neighborhood of scattering centers to generate the time series data for a range cell corresponding to the radar and scan strategy being simulated. It is clear that by choosing the scattering-center weights to be a delta function (i.e., a weight of 1 for the scattering center with the same azimuth, elevation, and range as the desired range cell and 0 for all other scattering centers), the SPARC simulator can produce time series data that correspond to the data originally collected by the WSR-88D. However, even in this case, the radar data obtained after processing the time series data generated by the SPARC simulator are not an exact match to the input WSR-88D base data due to the inherent randomness in the simulated IQ data for each scattering center. In general, by adjusting the scattering-center weights, effects of degraded antenna patterns (both copolar and cross-polar patterns) and range-weighting functions can be studied. For a more complete description of the SPARC simulator, we refer the reader to Schvartzman and Curtis (2019).

b. Simulator modifications

To simulate the effects of degraded antenna pattern sidelobes, time series data for scattering centers from multiple azimuths and elevations must be weighted and summed. The scattering-center weights are the only user-controllable parameters that can be adjusted to construct simulated patterns with varying sidelobe levels, but they alone do not determine the sidelobe levels of the simulated patterns. As mentioned before, the time series data for each scattering center already include effects from the intrinsic pattern of the WSR-88D. Therefore, the simulated pattern is the convolution of the scattering-center weights and the WSR-88D pattern. Our limited control of the simulated pattern means that we cannot simulate any arbitrary pattern. Particularly, our methodology cannot be used to simulate an antenna with a pattern that has a narrower beamwidth or lower sidelobe levels than the antenna pattern of the radar system that was used to collect the input data for our simulator. Because we are using archived WSR-88D data as input to our simulator, we can only simulate full 2D antennas patterns with the same or degraded characteristics as the WSR-88D pattern. Note that all sidelobes are increased in our simulations not only those in the principal horizontal and vertical planes.

To properly study the impact of degrading sidelobe levels on forecasters’ interpretation of radar data, we must isolate the effects of sidelobe levels from other sources of data-quality changes (e.g., beamwidth increases). That is, we simulate patterns from different antennas having the same beamwidth but different sidelobe levels. To simulate realistic patterns where the main lobe width is equal to that of the WSR-88D pattern but the sidelobes are degraded, the scattering-center weights must satisfy some restrictions. The first restriction is that the weight for the scattering center in the direction of interest must be 1 while the weights for scattering centers in directions other than the one of interest must be much smaller than 1. This restriction makes sense because the scattering center located in the direction of interest should be most heavily weighted so that the main lobe of the simulated pattern matches the main lobe of the WSR-88D pattern. In addition, scattering centers within a neighborhood (2° in azimuth and 1° in elevation) of the direction of interest must have zero weights. This is done to prevent broadening of the main lobe in the simulated patterns so that any data-quality changes are purely from increased sidelobe levels. The 2° buffer zone in azimuth is chosen because antenna sidelobe specifications (NOAA/NWS 2015) begin at 2°. The reason for the 1° buffer zone in elevation compared to that of 2° in azimuth is due to the limited sampling in elevation by the WSR-88D; 1° is also the minimum separation that ensures that the resolution volumes represented by adjacent scattering centers do not overlap given that the intrinsic WSR-88D pattern has a 1° beamwidth. The second restriction is that the nonzero scattering-center weights outside of the main lobe are monotonically decreasing as the angular distance between the scattering centers and the direction of interest increases. This restriction on the scattering-center weights is reasonable because it allows us to simulate antenna patterns with sidelobe characteristics that are similar to real antenna patterns. That is, sidelobes generally become smaller as the angular distance increases from the main lobe, and sidelobes that are close to each other are designed to not have large fluctuations in amplitude. The last restriction is that the nonzero weights are coarsely spaced so that the simulated patterns have a realistic sidelobe structure. If the scattering-center weights were too finely spaced, the sidelobes of the simulated pattern would resemble a flat envelope instead of the lobed structure seen in actual antenna patterns. The coarser spacing also prevents artificially inflated sidelobe contamination.

With the stated restrictions in mind, we generated the initial set of scattering-center weights based on the Spectrum Efficient National Surveillance Radar (SENSR) Preliminary Performance Requirements (Federal Aviation Administration 2018) and systematically increased the nonzero weights to simulate full 2D patterns with progressively higher sidelobe levels. We chose the SENSR requirements as the starting point because they are the latest requirements provided by NOAA and the most realistic requirements that describe the sidelobe levels of the WSR-88D antenna. The angular distance between the direction of a scattering center and the direction of interest was used to determine the weight for that scattering center based on the sidelobe envelope specified in the SENSR Preliminary Performance Requirements. For each weather case, time series data for scattering centers covering the entire 360° in azimuth and 0.5° to 8.0° in elevation with acquisition parameters (i.e., number of pulses and pulse repetition time) identical to those used in NEXRAD’s volume coverage pattern (VCP) 12 were weighted and summed to simulate data from elevations 0.5° through 5.1°. Data for elevations higher than 5.1° were not simulated because they were not needed for the storm interrogation process for the cases we used.

Figure 1 shows a section of the scattering-center weights used to simulate a pattern that is pointing at 0° azimuth and 2.4° elevation. The x axis and y axis correspond to azimuth and elevation angles, respectively. The color of each pixel represents the scattering-center weight for a scattering center located in that direction (note that the pixels have different sizes in elevation to match the irregular vertical sampling of the input data). The scattering-center weights are roughly circularly symmetric and depend on the angular distance between the direction of the scattering centers and the direction of interest. As the angular distance between the direction of a scattering center and the direction of interest increases, the weight linearly decreases from −15 dB at 1° separation to −50 dB for separation ≥ 10°. It is also clear that these scattering-center weights satisfy the restrictions discussed above. That is, the scattering center in the direction of interest receives unit weight while neighboring scattering centers receive zero weights; the nonzero scattering-center weights outside of the main lobe are monotonically decreasing; and the azimuthal gap between nonzero weights outside of the main lobe is 2°. Note that scattering centers located off the principal horizontal and vertical planes have nonzero weight, meaning that effects of sidelobes in all directions are simulated.

Fig. 1.
Fig. 1.

Scattering-center weights used to simulate a pattern such that the main lobe is pointing at 0° azimuth and 2.4° elevation. The axes correspond to azimuth and elevation angles, and the color indicates the magnitude of the weight for each scattering center in logarithmic units. The scattering center in the direction of interest receives unit weight. The other scattering centers within 2° azimuth and 1° elevation of the direction of interest (inside the box outlined by white dashes) receive zero weight (shown as −50 dB here). The weights for all other scattering centers are based on their angular distance from the direction of interest.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

Figure 2 shows an illustration of the process we used to modify sidelobe levels in our simulations. In this illustration, we utilized a measured KOUN pattern between ±13° (blue line) to represent a cut of the intrinsic WSR-88D pattern for each scattering center. Note that the available measurement limited the KOUN pattern to be within ±13°, but the intrinsic WSR-88D patterns in our simulations are full 2D patterns. The WSR-88D specifications (solid black line) and the SENSR sidelobe requirements (dashed black line) are shown as references. A cut of a simulated pattern (red curve) and the associated scattering-center weights in the same dimension (green dots) are shown. The sidelobe levels of the simulated pattern are mainly determined by the scattering-center weights when they are much larger than the sidelobe levels of the intrinsic WSR-88D pattern (e.g., at ±2°, ±4°, and ±6° in the example shown in Fig. 2). Therefore, by uniformly increasing the scattering-center weights, we can simulate full 2D antenna patterns with uniformly higher sidelobes.

Fig. 2.
Fig. 2.

A one-dimensional cut of the normalized two-way antenna radiation pattern measured for KOUN (blue curve) and a simulated pattern (red curve) with its associated scattering-center weights (green dots). The WSR-88D specifications (solid black line) and the SENSR sidelobe requirements (dashed black line) are shown as references.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

c. Quantification of sidelobe degradation

The straightforward way to quantify sidelobe levels is to measure them from the simulated patterns. Since the simulated pattern is the convolution of the scattering-center weights with the intrinsic WSR-88D pattern, we need precise knowledge of the WSR-88D pattern in addition to the known weights to be able to calculate the sidelobe levels of the simulated patterns. Unfortunately, only certain cuts of the WSR-88D pattern are available (Doviak and Zrnić 1998, 2006), so the simulated 2D patterns cannot be obtained precisely. An alternative approach to quantify sidelobes is through measuring the relative sidelobe degradations compared to the WSR-88D pattern, which eliminates the need to build representative models for WSR-88D patterns at different sites. This can be done without precise knowledge of the intrinsic WSR-88D pattern by assuming that its sidelobe levels are much lower than its main lobe and that the scattering-center weights satisfy the restrictions stated in the previous subsection. Under these conditions, the sidelobes of the simulated pattern are formed by the main lobe of the intrinsic WSR-88D pattern for each scattering center away from the direction of interest. One measure to quantify sidelobe performance for weather radars is the integrated sidelobe level (ISL), which is the ratio of the power received through the sidelobes to the power received through the main lobe. Since the main lobe of the simulated pattern is the same as that of the WSR-88D pattern by design, the difference (in linear units) between the ISL of each simulated pattern and the WSR-88D pattern can be estimated as the sum of the weights that were applied to scattering centers outside of the direction of interest.

For this study, we used the difference in ISL (referred to as “delta” ISL and denoted by ΔISL) to quantify the relative sidelobe degradation of each simulated pattern compared to the WSR-88D pattern. Since the ISL of the simulated pattern is the sum of its ΔISL and the ISL of the intrinsic WSR-88D pattern, the ΔISL of the simulated patterns must be comparable to or larger than the ISL of the intrinsic WSR-88D pattern to prevent the effects of the intrinsic WSR-88D pattern from dominating the quality of the simulated data. In our simulations, we chose the scattering-center weights to be systematically larger than the requirement envelope (see Fig. 2). This assumes that the sidelobes of the intrinsic pattern are less than or equal to the SENSR requirement envelope, which is known to be true based on measurements (Doviak 2017). Selecting the scattering-center weights in this manner guarantees that the resulting ΔISL is greater than or equal to the intrinsic ISL (i.e., the simulated patterns have sidelobes that are at least 3-dB larger than those of the intrinsic pattern). From the example shown in Fig. 2, it can be seen that by uniformly increasing the scattering-center weights, the ΔISL of the simulated patterns will be similar to or surpass the ISL of the intrinsic pattern. For each weather case, 10 different patterns with varying ΔISL were simulated by uniformly increasing the scattering-center weights in directions away from the direction of interest. The simulated patterns and their associated ΔISL (in both linear and dB units) are listed in Table 1 in order of increasing sidelobe degradation. The range of ΔISL is also shown to be reasonable because it roughly corresponds to data-quality impacts on forecasters ranging from minimal to significant.

Table 1.

Simulated patterns and their associated ΔISL in linear and decibel units. The intrinsic WSR-88D pattern is used as a reference to calculate ΔISL and has a ΔISL of 0 (in linear units).

Table 1.

d. Simulation validation

To demonstrate the fidelity of our simulations, we compared simulated data to WSR-88D data, and we found two types of compromised data due to elevation sidelobe contamination. Hereafter, compromised data due to elevation sidelobe contamination will be referred to simply as compromised data. These two types of compromised data seen in the WSR-88D are questionable low-level circulations and noisy velocity data in the storm inflow region.

For the low-level circulation type of compromised data, the sidelobe contamination dominates the received power from the main lobe, resulting in Doppler velocity estimates that appear to be valid. Since this signature may or may not be a valid circulation, forecasters’ cognitive resources are diverted toward diagnosing the validity of this signature (i.e., they must interrogate fields of spectrum width and correlation coefficient).

To demonstrate this type of compromised data, Figs. 3a and 3b show WSR-88D data collected by the KLRX radar at 0445 UTC 22 July 2014 at 0.5° elevation (left column) and simulated data for this case both without (center column) and with moderate sidelobe degradation (right column). Figure 3a displays fields of reflectivity (Z; top row), storm-relative mean radial velocity (SRM; middle row) and spectrum width (W; bottom row), where storm-relative mean radial velocity is simply Doppler velocity with storm motion subtracted. Figure 3b displays differential reflectivity (ZDR; top row), specific differential phase (KDP; middle row), and correlation coefficient (CC; bottom row). The color bars we used to display our data are enlarged in Fig. 4 for easier reference, and the same color bars are used for the remainder of this paper. Comparing the left columns to the middle columns of Figs. 3a and 3b shows that the SPARC simulator can produce data that are well matched to WSR-88D data for all radar variables. The increase in the number of 0 m s−1 spectrum-width estimates is due to our signal processor using a different spectrum-width estimator as compared with the signal processor of the WSR-88D. A comparison of the center columns with the right columns of Figs. 3a and 3b clearly shows the impact of increasing sidelobe levels. This is especially apparent in storm-relative mean radial velocity where higher sidelobe levels result in an expanded footprint of inbound velocities (in the circled region) that corresponds to low reflectivity, large spectrum width, and reduced correlation coefficient.

Fig. 3.
Fig. 3.

(a) Reflectivity (top row), storm-relative mean radial velocity (middle row), and spectrum width (bottom row) at 0.5° elevation (left) for a storm observed by the KLRX radar at 0445 UTC 22 Jul 2014, along with simulated data for this case (center) without and (right) with moderate sidelobe degradation. (b) As in (a), but showing differential reflectivity in the top row, specific differential phase in the middle row, and correlation coefficient in the bottom row.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

Fig. 4.
Fig. 4.

Enlarged color bars used to display, from top to bottom, reflectivity, storm-relative mean radial velocity, spectrum width, differential reflectivity, specific differential phase, and correlation coefficient. These color scales are based on the standard color scales used by NWS forecasters in their official software.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

To verify that our simulation of this type of compromised data is realistic, we compared our simulated data to WSR-88D data from other sites that contained similar questionable circulations. Figure 5 shows the fields of reflectivity (top left), storm-relative mean radial velocity (top right), spectrum width (bottom left), and correlation coefficient (bottom right) for a storm observed by the KDFX radar at 2302 UTC 15 May 2014 at 0.5° elevations. In the real data, we see an expanded footprint of inbound velocities (in the circled region) that corresponds to low reflectivity, large spectrum width, and reduced correlation coefficient. These features match well with our simulated data for this type of compromised data (i.e., compare the corresponding images in the right column of Figs. 3a and 3b with Fig. 5). Note that in Figs. 3a and 3b and subsequent figures, radar features are circled in reflectivity and storm-relative mean radial velocity, while dashed circles are used for other radar variables. This is to reinforce the significantly greater salience of reflectivity and storm-relative mean radial velocity for observing this feature during real-time warning operations. Also note that, for storm interpretation purposes, the collocation of important features, especially a possible circulation in reflectivity and storm-relative mean radial velocity (as well as any other radar variables), must be assessed.

Fig. 5.
Fig. 5.

(top left) Reflectivity, (top right) storm-relative mean radial velocity, (bottom left) spectrum width, and (bottom right) correlation coefficient at 0.5° elevation for a storm observed by the KDFX radar at 2302 UTC 15 May 2014.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

The second type of compromised data consists of noisy velocity data (both base velocities and storm-relative mean radial velocities) in the low-level storm inflow region. For this type of compromised data, the sidelobe contamination is more comparable to the received power from the main lobe, resulting in less reliable velocity estimates that appear to be invalid. Based on personal communications with multiple NWS operational forecasters, our understanding is that this type of compromised data is not typically being identified as caused by sidelobe contamination. This makes sense: a questionable circulation is visually compelling while noisy velocity data in weak signal areas are visually filtered as nonreliable.

Figures 6a and 6b show a comparison of WSR-88D data collected by the KGLD radar at 0039 UTC 23 May 2015 at 0.5° elevation (left column) and simulated data for this case without (center column) and with moderate sidelobe degradation (right column) to demonstrate this type of compromised data. An example of this type of contamination in real data is shown in Fig. 7 from a storm observed by the KMRX radar at 2050 UTC 7 July 2014 at 0.5° elevation. The circled storm inflow region in both real and simulated data contains numerous gates with noisy storm-relative mean radial velocity, large spectrum width, and reduced correlation coefficient (i.e., compare the corresponding images in the right columns of Figs. 6a and 6b with Fig. 7). The examples in Figs. 3 and 6 demonstrate that our simulator can reproduce the same types of compromised data as seen in WSR-88D data.

Fig. 6.
Fig. 6.

(a) As in Fig. 3a and (b) as in Fig. 3b, but for a storm observed by the KGLD radar at 0039 UTC 23 May 2015.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

Fig. 7.
Fig. 7.

As in Fig. 5, but for a storm observed by the KMRX radar at 2005 UTC 7 Jul 2014.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

3. Data analysis method

To improve our understanding of the relationship between relative sidelobe degradation and impact on NWS forecasters’ interpretation of radar data, we selected cases that are most likely to show increased data-quality degradation due to elevation sidelobe contamination as sidelobe levels are increased. The storm type that is most prone to produce elevation sidelobe contamination is the supercell with large hail, which is also one of the most difficult storm types to analyze during NWS warning operations. We developed a scoring system based on the extent of data-quality “distractions” to quantify the impact on forecasters. The case-selection process and the scoring system are discussed next, and examples are shown to demonstrate the analysis process used to arrive at the score for each simulated pattern and to illustrate the data-quality differences among the scoring levels.

a. Case selection

The case-selection process for this work focused on a specific storm type that is most likely to present the necessary strong vertical reflectivity gradient that can lead to significantly compromised data: supercells, particularly those producing significant hail. The storm type used for this study typically has high reflectivity aloft that “overhangs” the low-level clear-air inflow region. Supercells are one of many types of hazardous convection, and convection is one of many threats (e.g., flooding, winter, or fire) addressed by the NWS mission of protecting lives and property. Supercells often warrant a severe thunderstorm warning because of large hail [equal to or in excess of 1 in. (2.54 cm)] or strong straight-line winds [equal to or in excess of 58 mi h−1 (26 m s−1)] (NOAA/NWS 2018). Supercells are also potentially tornadic and require close monitoring of the storm structure and its local environment because a tornado warning may also be warranted. Since these warnings are so critical to fulfilling the NWS mission, it is important to understand the impact of data-quality degradation on forecasters’ interpretation of radar data that could affect these warning decisions.

For our study, we wanted the different cases to have similar impacts on forecasters given comparable elevation sidelobe degradations. Therefore, it was important for the cases we selected to be consistent in the sense that they had similar storm morphology that could cause increasingly compromised data for antenna patterns with higher sidelobe levels. The initial set of candidate cases contains supercells from 2014 to 2015 and was provided to author Boettcher by the experts at the NWS Storm Prediction Center (SPC) from their database of severe convective storms. From this database of candidates, isolated supercells at midrange (i.e., less than 120 km) from the radar were chosen from differing geographical locations (i.e., not limited to the Great Plains) to capture the varying depths of mesocyclones. The selection of a particular time step within the long life cycle of a supercell storm was focused on finding storm structures that are prone to produce elevation sidelobe contamination. We used two methods to identify suitable time steps. First, we found two examples of supercell storms that resulted in significantly compromised WSR-88D data, which presented as a questionable low-level circulation. For these storms, we chose a time step without the questionable low-level circulation that is close to the time in which a questionable low-level circulation was observed in the WSR-88D data. This method to select cases is reasonable because we can expect that any degradation to the pattern sidelobes should result in comparable data-quality degradation for these storms. In addition to the cases selected using the first method, we also identified multiple supercell storms with the characteristic significant reflectivity overhang structure that did not cause a questionable low-level circulation in the WSR-88D data. These marginal cases allowed us to better understand the different levels of impacts on forecasters as different patterns were simulated. That is, they are the most likely candidates to display the full transition from no impact to significant impact as the sidelobe levels are increased. In total, we selected 13 cases for this study; these are listed in Table 2.

Table 2.

Selected weather cases to evaluate impacts of degraded sidelobes on forecasters’ data interpretation process.

Table 2.

b. Assessment of data-quality impacts

To assess the data-quality impacts on forecasters, we simulated data as though collected by radar systems with progressively higher sidelobes and analyzed each set of simulated data independently to assess potential “distractions” to forecasters’ interpretation process caused by degraded data quality. Here, a distraction is defined as a misrepresentation of an important weather feature or a lowering of confidence in the validity of the data. Distractions could result in forecasters expending additional attention and working memory toward confirming the validity of the signature, interrogating other radar variables, waiting for additional scans, or issuing a false alarm warning. These actions are undesirable given the time constraint of NWS warning operations. For each case, simulated data for elevations 0.5°–5.1° were analyzed.

To assess the impacts on NWS forecasters accurately, our data-analysis process was very similar to the current vertical analysis process for storm interrogation commonly used in warning operations. That is, we viewed PPIs at different elevations in sequence up and down as an NWS forecaster would do in real time. Factors such as storm history and environment were not part of the analysis, as the scans were analyzed for the level of distractions caused by compromised data. The radar variables used for the analysis were reflectivity, storm-relative mean radial velocity, spectrum width, and correlation coefficient. While sidelobe contamination can definitely manifest in the reflectivity field, its manifestation in the Doppler velocity field is the most critical to the forecasters in the severe-convection warning domain. In addition, storm-relative mean radial velocity is the most critical radar product for the NWS forecasters’ up-and-down-PPI-viewing process while interrogating storm structure and tornado-production potential. Diagnosing the validity of a circulation involves other radar products such as reflectivity (e.g., the circulation is in the correct location), and possibly spectrum width (e.g., magnitude makes sense) and correlation coefficient (e.g., returns are from precipitation). For the critical time-constrained domain of supercell interrogation, diagnosing elevation sidelobe contamination is best done using correlation coefficient among the polarimetric variables. While compromised data also appear in fields of differential reflectivity (e.g., see Figs. 3b and 6b), they were not used in this study because spectrum width and correlation coefficient typically provide much stronger evidence.

Using these radar products, the analysis was based on the presence and extent of the two types of compromised data identified previously: questionable low-level circulations and noisy velocity in the storm inflow region. Reflectivity and storm-relative mean radial velocity were given greater weight in the analysis as they are the foundational radar variables for storm interrogation. For example, the location of the center of an apparent circulation in storm-relative mean radial velocity was compared to the storm structure in the reflectivity data to diagnose its validity. The storm inflow region at the lower elevations is the key area of interest for either of the types of compromised data, which often manifests over one or two of the lowest elevations (0.5° and 0.9°).

The analysis was done by one of the authors (Boettcher), who has over 20 years of experience in training NWS forecasters as well as 10 years of operational forecasting experience. Thus, she has vast experience in interacting with a variety of NWS forecasters from novices to experts. This makes her uniquely qualified as a representative for the broader NWS forecaster population. To support her goal of seeing the data through the eyes of a broad population of NWS forecasters, she had numerous discussions with operational forecasters throughout case selection and analysis. In addition to gaining a better knowledge of the varying understanding of elevation sidelobe contamination among forecasters, she also learned that there is wide variation in real-time methods when analyzing this particularly difficult type of compromised data (e.g., differing usage of spectrum width and/or correlation coefficient for data-quality analysis). For each case, the data analysis was performed without her knowledge of the underlying parameters of the simulated pattern. This was important to prevent expectation biases in her analysis process. Although any individual forecaster may arrive at somewhat different analysis results, we are confident that our results characterize the impacts of compromised data from the perspective of the NWS forecaster population as a group, which is ideally represented by author Boettcher.

To quantify the impact on forecasters’ interpretation of radar data, a five-level scoring scale was developed as shown in Table 3. The scale was based on how much distraction the compromised data would cause. A score of 1 indicates fully acceptable data quality with no distractions, a score of 2 indicates acceptable data quality that contains minimal distractions, a score of 3 indicates ambiguous data quality where the impact of the distraction would vary significantly from forecaster to forecaster, a score of 4 indicates unacceptable data quality where the distractions would affect nearly all forecasters, and a score of 5 indicates completely unacceptable data quality where distractions would significantly impact data interpretation. We decided on using five levels for the scoring system because they can better capture the relationship between elevation sidelobe degradation and data-quality impact on forecasters compared to a binary acceptable/unacceptable categorization. At the same time, the five levels are distinct enough that the observed data-quality differences can be meaningfully categorized. Examples of simulated data with different scores are shown next.

Table 3.

Scoring system used to codify impacts on forecasters’ interpretation of radar data.

Table 3.

c. Case examples

We present case 11 as an example to demonstrate the analysis process used to arrive at the score for each simulated pattern and to illustrate the data-quality differences among the scoring levels. Figure 8 shows the reflectivity (top left), storm-relative mean radial velocity (top right), spectrum width (bottom left), and correlation coefficient (bottom right) from the KLRX radar at 0.5° elevation at 0445 UTC on 22 July 2014. This storm had elevation sidelobe contamination that resulted in a questionable circulation in scans between 0413 and 0440 UTC, similar to those shown in both the simulated data (right columns of Fig. 3) and WSR-88D data (Fig. 5). Figure 9 shows reflectivity and storm-relative mean radial velocity from simulated data with ascending scores from the top row (score of 1) to the bottom row (score of 5). The simulated patterns for each row from top to bottom correspond to patterns A, B, D, G, and J. The following analysis description was based on elevations 0.5°–5.1°, with 0.5° shown in Fig. 9 for brevity and because the greatest impacts occur on the lowest elevations. In comparing the top row of Fig. 9 with Fig. 8, we see trivial differences along the eastern storm edge as a result of the simulation process. There are no distractions to forecasters’ interpretation of radar data. Therefore, for this case, pattern-A data received a score of 1. In the second row of Fig. 9, we can see a slightly increased footprint of the inbound velocity in the circled area compared to the top row (and Fig. 8) as a result of increased sidelobe levels. However, this slight increase in footprint is not a major distraction to forecasters’ interpretation of radar data; therefore, pattern-B data received a score of 2 in this case. In the third row of Fig. 9, we see more inbound velocities in the circled area mixed with some outbound velocities. Reactions to these noisy velocities can vary significantly among forecasters. On the basis of author Boettcher’s experience, it is concluded that the number of forecasters that would be distracted by the data presentation is roughly even with the number of forecasters that would not be significantly affected. Therefore, pattern-D data received a score of 3 in this case. The fourth row of Fig. 9 shows a further expansion of the footprint of mostly inbound velocity, which is a significant data-quality distraction to almost all forecasters. This resulted in pattern-G data receiving a score of 4. In the last row in Fig. 9, the footprint of inbound velocity grows further, and there is also an expansion of precipitable values in the inflow region due to sidelobe contamination in the reflectivity image. This extensively compromised data caused the pattern-J data to receive a score of 5 in this case.

Fig. 8.
Fig. 8.

As in Fig. 5, but for a storm observed by the KLRX radar at 0445 UTC 22 Jul 2014.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

Fig. 9.
Fig. 9.

(left) Reflectivity and (right) storm-relative mean radial velocity of case 11 at 0.5° elevation with scores of (top) 1, (top middle) 2, (middle) 3, (bottom middle) 4, and (bottom) 5. The simulated patterns corresponding to the five rows from top to bottom are patterns A, B, D, G, and J. The data-quality impacts caused by increasing sidelobe levels can be clearly seen in the storm-relative mean radial velocity field in the circled area.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

4. Impact model

To better understand the relationship between relative sidelobe degradation and impact on forecasters’ interpretation of radar data, we fit a sigmoidal curve to our score data for each of the 13 cases. The sigmoidal curve was chosen because the function asymptotically approaches the maximum score as the sidelobe degradation is increased to infinity and asymptotically approaches the minimum score as the sidelobe degradation is decreased to zero. This captures the fact that once the score reaches 5, continuing to increase sidelobe levels will not change the score. Conversely, once the score reaches 1, continuing to decrease sidelobe levels will not change the score. That is, once the validity of the data is in question, forecasters would need to spend additional cognitive resource to evaluate the data, and higher sidelobe levels with more significant contamination would still result in the same action by forecasters. Similarly, if sidelobe contamination has negligible impact on forecasters, reducing the sidelobe levels further would still result in sidelobe contamination that has negligible impact on forecasters. The fitted curve has the form
f(ΔISL)=1+41+exp[k(ΔISLxo)],
where ΔISL is in linear units (center column of Table 1), and k and xo are unknown parameters (to be determined) that control the steepness and the position of the center of the curve. We used a grid search that minimized the mean square error between the fitted curve and our scoring data to determine the value of k and xo for each case. The top panel of Fig. 10 shows the fitted curve for case 11 (blue line) as an example for which k is 10 and xo is 0.069. The values of xo for the fitted curves vary from case to case because there are variations among supercells and the cases we selected have a different amount of tolerance to sidelobe degradation before the impact on forecasters’ interpretation of radar data reaches the marginal level. However, this does not prevent us from attempting to quantify the relationship between sidelobe degradation and impact on forecasters’ interpretation process since that relationship corresponds to the shape and steepness of the curves. From the 13 cases we simulated, 12 cases have k between 7 and 37, with an average of 20 and a standard deviation of 8.5. The one outlier (case 9) has a k of 87. The reason for this outlier is that its score was dominated by artifacts caused by azimuthal sidelobe contamination. Many of the cases presented some data-quality degradation caused by azimuthal sidelobe contamination as well as elevation sidelobe contamination, but case 9 was significantly affected as a result of much stronger azimuthal reflectivity gradients relative to the others. The bottom panel of Fig. 10 shows the fitted curves (gray lines) for the nonoutlier cases and a curve (black line) with k and xo equal to the average of the nonoutlier cases.
Fig. 10.
Fig. 10.

(top) Scores for the simulated patterns (blue stars) and the fitted curve for case 7 (blue line). (bottom) The fitted curves for all 12 nonoutlier cases and the average best-fit curve (black line). The x axes are ΔISL in linear units (center column of Table 1). Note that the curves are fitted to minimize the mean square error and capture the trend of the scores. The average best-fit curve uses parameters that are the average of all of the nonoutlier cases.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

On the basis of the average steepness of the remaining 12 cases, we developed the model shown in Fig. 11 for the relationship between relative sidelobe degradation with respect to the WSR-88D pattern and impact on forecasters’ interpretation of radar data. This model will be referred to as the “impact model” for simplicity. With the scoring system defined previously, there are three regions of impact: acceptable, marginal (shaded in yellow), and unacceptable (shaded in red). These regions are separated by the horizontal dashed lines. Within the acceptable region, the model curve falls into two separate subregions: low sensitivity (shaded in green) and high sensitivity (shaded in cyan), separated by the first vertical dashed line. Sensitivity here means how much the impact score changes when the relative sidelobe degradation changes (i.e., the slope of the curve). Lower sensitivity is better than higher sensitivity because the impact on forecasters would be less for a similar amount of sidelobe degradation. The value that separates the two regions is determined by the slope of the curve when it first reaches 25% of the maximum slope. Note that, due to our case selection process, the average best-fit curve (black line) has a score of 1.1 for patterns with zero sidelobe degradation (i.e., the WSR-88D pattern) in Fig. 10, which places the WSR-88D in the acceptable with low sensitivity region of the model.

Fig. 11.
Fig. 11.

Model for the relationship between relative antenna sidelobe degradation and impact on forecasters. On the basis of the impact score, there are three regions: acceptable, marginal, and unacceptable. Within the acceptable region, the model curve falls into two separate regions: low sensitivity and high sensitivity. Sensitivity describes how quickly the impact score changes when the relative antenna sidelobe degradation changes.

Citation: Journal of Applied Meteorology and Climatology 59, 4; 10.1175/JAMC-D-19-0092.1

The impact model is a function of the reflectivity gradients in the storm under analysis. For the cases that we chose, the reflectivity gradients are close to the maximum that the WSR-88D can observe without producing compromised data. We refer to these cases as “upper bound” WSR-88D storms. That is, storms more demanding than the WSR-88D upper-bound storms would cause compromised data in the WSR-88D. In general, a future radar system could be designed to observe different upper-bound storms. That is, a future radar system may be required to produce uncompromised data for storms with different maximum reflectivity gradients, and this upper-bound storm would drive the sidelobe requirement. If the upper-bound storm is less demanding than the upper-bound WSR-88D storm, the model curve must be shifted to the right. That is, for the same sidelobes, the impact score for the new upper-bound storm is lower. Conversely, if the upper-bound storm is more demanding, the model curve must be shifted to the left. That is, for the same sidelobes, the impact score for the new upper-bound storm is higher. Therefore, the impact model could be used by decision-makers in two ways. The first use is to estimate the consequences of degradation for a given radar design. In other words, the model can be used to evaluate how much impact would occur if a system’s ISL failed to meet requirements by a certain amount. The second use is to evaluate the robustness of a given design to storms more demanding than the upper-bound storm.

To use the impact model for these applications, we would first need to convert the x axis into the absolute ISL, which is usually inversely related to system cost. This step could be done by measuring the antenna pattern of the WSR-88D precisely or by constructing an accurate model of the WSR-88D pattern at different sites using available measurements. With this conversion of the x axis, any potential radar design could be placed directly on it using its absolute ISL. Moreover, as we argued above, we also need to shift the model curve on the x axis to match the required upper-bound storm. This could be accomplished by averaging the scores of WSR-88D data for a large number of storms with these required upper-bound reflectivity gradients that are particular to each radar design. Then, a sigmoidal curve with the same steepness as the model in Fig. 11 can be fit to this average score. Doing these two things would allow decision-makers to evaluate the operational impact of potential radar designs depending on where they land on the model curve.

After making the necessary changes described before, different radar designs could be evaluated using the impact model in Fig. 11. If a radar design results in an ISL that falls in the marginal or unacceptable region, then the data-quality impacts for such a design are unlikely to be tolerable to forecasters. If a radar design results in an ISL that falls in the acceptable region, then decision-makers can use the model to balance trade-offs between system cost and potential operational impacts. Radar designs with ISLs that fall in the acceptable with low sensitivity region can tolerate more degradation and are more robust to worst-case-scenario storms, but they likely have higher system cost. In contrast, radar designs with ISLs that fall in the acceptable with high sensitivity region likely have lower system cost, but any increase in sidelobe levels can quickly lead to rapidly increasing impacts on forecasters. Such systems are also less robust to storms more demanding than the upper-bound storm. Similar analyses can be performed to determine sidelobe requirements that balance system cost and operational impact on forecasters since each set of sidelobe requirements can be placed on to the impact curve using its ISL. Conversely, once an acceptable ISL is determined (i.e., the x coordinate of a point on the impact curve), it can be used to define an envelope similar to those in Fig. 2 by assuming any realistic pattern structure (e.g., sidelobes that gradually decay away from broadside). Exceptions similar to those in the WSR-88D system specifications can be made for isolated sidelobes caused, for example, by struts.

5. Conclusions

In this work, we extended the SPARC simulator to include the 2D effects of sidelobes, and we used it to simulate data as observed by radars with varying sidelobe levels that are higher than those in the WSR-88D. Moreover, we identified the necessary restrictions on the scattering-center weights to simulate realistic patterns, and we used ΔISL to quantify the relative sidelobe degradation of the simulated 2D patterns with respect to the WSR-88D pattern.

In our simulated data, we identified two types of compromised data: questionable low-level circulations and noisy velocity data in the low-level inflow region. In the course of this study, we learned that noisy velocity data in the low-level inflow are not well known to forecasters to be the result of sidelobe contamination. Since the storms that present these types of compromised data are potential tornado producers, a questionable low-level circulation is a high-stakes data artifact for NWS warning decision-making. Both types of compromised data were also observed by the WSR-88D, which validated the fidelity of our simulator.

To quantify the data-quality impact on forecasters’ interpretation of radar data, we developed a scoring system based on the extent of data-quality distractions. We identified 13 cases and simulated 10 antenna patterns with different sidelobe levels for each case to quantify the impact caused by increases in sidelobe levels. Using the scores of the simulated data, we built a model for the relationship between antenna sidelobe degradation and operational impact on forecasters.

Our impact model provides a curve of acceptable sidelobe degradation, where each point on that curve has a different associated system cost and robustness to further degradation. This model is a first step to improve our understanding of the trade-offs between spatial-resolution requirements and operational impacts. Whereas only a limited number of cases were analyzed, the precision of the model could be improved by simulating and analyzing additional cases. A more robust impact model could be used by decision-makers as part of their evaluation process for radar designs and to determine sidelobe requirements while balancing system costs and operational impacts. This study focused on the impact of elevation sidelobe contamination on operational interpretation by NWS forecasters in the domain of warning operations. While this type of contamination also affects other applications that use radar data, these impacts are outside the scope of this paper and will be investigated in future studies. Therefore, it is important to note that our results should not be interpreted as providing recommendations for changing sidelobe requirements for a replacement system.

As shown in Figs. 5 and 7, the WSR-88D pattern also results in compromised data in some cases, which suggests that sidelobe performance better than the current WSR-88D pattern would be needed to mitigate all possible instances of these two types of compromised data. However, more stringent sidelobe requirements would lead to increased system cost for any radar design. An alternative approach to a fixed antenna pattern is to utilize the unique capability of a phased-array antenna to modify its pattern adaptively to minimize sidelobe contamination when observing storms that can generate elevation sidelobe contamination (Nai 2017). For example, when ground clutter or interference are present, the antenna pattern can be modified to have near-zero sidelobes in the directions of the unwanted signals. When a strong reflectivity gradient is present, the sidelobes in the direction of the strong reflectivity core can be lowered at the expense of wider main lobe and higher sidelobes elsewhere. Trade-offs between angular resolution and sidelobe levels for an adaptive antenna pattern strongly depend on the constraints used, the array architecture, and the storm under observation. By exploiting the phased-array antenna capability to lower the sidelobes adaptively in certain directions (e.g., with strong reflectivity gradients), one could reduce the cost of the antenna system by allowing for increased intrinsic sidelobe levels in all directions. In other words, compared to a nonadaptive phased-array antenna system that meets the functional requirements at all times, a smaller (and therefore less costly) phased-array antenna employing adaptive pattern synthesis could result in similar performance. By having the flexibility to adjust the sidelobe levels automatically in response to different storm types, a phased-array radar could trade higher sidelobe levels for improved performance in other areas (e.g., beamwidth and sensitivity) as the need arises. While there are still many technical challenges, this adaptive approach could be an alternative method to achieve the functional requirements while not meeting system specifications at all times.

Acknowledgments

The authors thank Mark Weber, Vivek Mahale, and anonymous reviewers for providing comments to improve the paper. We also thank the numerous NWS and SPC forecasters who generously provided their time, data, references, and other resources to support this study. Funding was provided by NOAA/Office of Oceanic and Atmospheric Research under NOAA–University of Oklahoma Cooperative Agreement NA11OAR4320072 (U.S. Department of Commerce).

REFERENCES

Save