1. Introduction
The implementation of real-time data quality control is of fundamental importance for observations that are assimilated into operational numerical weather prediction models. One of the most vexing quality-control problems affecting radar wind profilers has been signal contamination from nocturnally migrating birds (Wilczak et al. 1995). Although techniques have been developed that helped reduce the level of contamination (Wilczak et al. 1995; Merritt 1995), these were unable to completely remove the interference during periods of very dense bird migration. For this reason, data from operational wind profiling radar networks, such as the National Oceanic and Atmospheric Administration (NOAA) National Profiler Network (NPN), apply additional simple quality-control procedures to eliminate bird-contaminated data based on time of day, wind direction, season, signal-to-noise ratio (SNR), and spectral width thresholds. This procedure effectively eliminates all data that have characteristics of bird contamination, but it can at times mistakenly flag and eliminate real atmospheric signal. In addition, although the intermittent clutter rejection algorithm (ICRA) technique developed by Merritt (1995) effectively removes interference during periods of light bird contamination, for moderate bird densities it often reduces the magnitude of the SNR and spectral width signal to levels below those associated with birds, but it does not entirely eliminate the velocity contamination. This technique results in mildly contaminated winds that could potentially pass the additional simple quality-control procedure applied to the NPN profiler data.
Given a highly nonstationary intermittent clutter signal—for example, one that is generated by flying birds—it is necessary to deal with the problem before the Doppler spectrum is estimated, as Fourier methods can spread the effect of bird contamination into a broad range of frequencies.
Recently, a new approach to identifying and rejecting wind profiler data contaminated by birds was developed by Lehmann and Teschke (2008). Their technique is based on a Gabor frame expansion of the complex time series of the demodulated receiver voltage. The technique makes use of a discrete Gabor frame expansion of the coherently averaged time series data in combination with a statistical filtering approach to exploit the different signal characteristics between atmospheric and intermittent clutter. The main difference between data containing intermittent clutter components and either clear-air or ground clutter signals is the transient (or nonstationary) character of the intermittent clutter signal component. Specifically, a transient signal is a signal whose duration is short compared to the observation interval—in our case, the dwell time. For detailed information about the method, we refer to Lehmann and Teschke (2008) and Lehmann (2012). Lehmann and Teschke (2008) tested this technique on a limited amount of data of an operational 482-MHz wind profiling radar, and found that it effectively identified and eliminated the bird-contaminated data while most often finding the true atmospheric signal. In the current study, we apply the same algorithm to data from 915-MHz wind profiling radars. The sensitivity of a wind profiler to migrating birds depends on a number of profiler characteristics, including its beamwidth, range resolution, and wavelength (Wilczak et al. 1995; Merritt 1995). Referring to our case, contamination from birds is more severe at higher radar frequencies (915 versus 482 MHz) because the reflectivity of a bird is proportional to λ−4 (where λ is the radar wavelength, ~32 cm for a 915-MHz wind profiling radar and ~62 cm for a 482-MHz radar) for objects that are small compared with the radar wavelength. Also, contamination is more serious for radars with smaller antennas (a 915-MHz radar wind profiler in our case) because their larger beamwidths allow birds to be observed for longer periods. Because of these considerations, the radar used in this study will have a sensitivity to birds relative to the clear-air signal considerably larger compared to the one used in Lehmann and Teschke (2008), making it a greater challenge for any algorithm to eliminate the bird contamination while simultaneously identifying the true atmospheric signal. Because of this, a different tuning of the Gabor scheme is found to be required, as well as additional levels of thresholding of moment data. In addition, we show that a state-of-the-art multipeak picking (MPP) algorithm (Griesser and Richner 1998) can be implemented in parallel with the Gabor processing. Finally, we also show the additional value that can be obtained by applying a moment-level pattern recognition scheme (Weber et al. 1993) to the data after the Gabor and MPP processing.
In the next section, we introduce the dataset used and the various steps taken to remove bird contamination from the data. Conclusions will be discussed in the last section.
2. Dataset and data analysis
A dataset collected during 2010 by three 915-MHz wind profilers located in the Central Valley of California, specifically at Chico [site name: cco, latitude: 39.69°N, longitude: 121.91°W, altitude: 41 m above ground level (AGL)], Chowchilla (site name: ccl, latitude: 37.10°N, longitude: 120.24°W, altitude: 76 m AGL), and Oakhurst (site name: oht, latitude: 37.38°N, longitude: 119.63°W, altitude: 955 m AGL) are used to evaluate and tune the Gabor-based algorithm. For these sites local standard time (LST) = UTC − 8 h. All three sites are impacted heavily by migrating birds following the Pacific Flyway (northwest to southeast and vice versa, http://www.birdnature.com/pacific.html).
For consistency, all of the processing tests performed in this study used an identically configured MPP procedure to derive the spectral moments from the transformed time series data. The processing steps within MPP include multiple spectral peak identification, ground clutter and radio frequency interference (RFI) identification and reduction, and fuzzy-weighted signal selection. Thus, differences in the test results are mostly attributed to the different methods by which the time series and resultant-wind data are processed.
An example of a day with clean data (with no bird contamination) and a day with contaminated data (with many migrating birds detected at nighttime) is presented in Fig. 1 for the Chico site. For both days, the three-panel plots from top to bottom are the range-corrected SNR, radial velocity, and spectral width measured on a radial antenna beam. For the day on the left (14 March 2010), the nighttime is characterized by a stable layer with convection starting after sunrise (1423 UTC). The planetary boundary layer (PBL) height grows to its maximum, reached a few hours after solar noon (2019 UTC), with the inversion layer revealed by a layer of enhanced SNR. Convective plumes are revealed during daytime in the radial velocity panel, while turbulence (proportional to the spectral width of the radial velocity) is very weak at nighttime and confined to the lower part of the atmosphere and residual layers aloft, but it is more pronounced within the convective boundary layer. For the day on the right (9 October 2010), the nighttime is characterized by very strong bird contamination revealed in all three panels. The contamination produces large values of SNR, large values of spectral width (and therefore apparent turbulence), and it appears that the targets are moving away from the receiving antenna in the right-middle panel (negative velocities, dark blue colors). For this day we show the three-panel plots for the antenna beam pointing to the southeast at 138°, 15° from zenith, because we want to verify that the direction of the moving targets is consistent with an autumn migration from northwest to southeast in the Pacific Flyway path. The PBL starts to develop after sunrise (1415 UTC), as revealed from the enhanced values of SNR; convective plumes and turbulence increase within the convective boundary layer as well.
The 915-MHz wind profiler time–height cross sections of the vertical and oblique beam moments for (left) 14 Mar and (right) 9 Oct 2010 at Chico. (top) Range-corrected SNR, (middle) radial velocity, and (bottom) spectral width of the radial velocity. For both datasets, time is in UTC.
Citation: Journal of Atmospheric and Oceanic Technology 30, 12; 10.1175/JTECH-D-13-00089.1
If we do not remove the contamination from the data, then hourly winds computed with a standard consensus procedure (Strauch et al. 1984) show very large northwesterly values at nighttime, largely caused by the migrating birds, as revealed in Fig. 2. Assimilating data such as these into numerical weather prediction models can result in erroneous forecasts, and therefore it is important to remove the contamination, when present.
The 915-MHz wind profiler time–height cross sections of hourly winds computed by a standard consensus procedure for 9 Oct 2010 at Chico.
Citation: Journal of Atmospheric and Oceanic Technology 30, 12; 10.1175/JTECH-D-13-00089.1
In Fig. 3 we present the same data as those in the right panel of Fig. 1 but after the ICRA developed by Merritt (1995) was applied to the time series data. Some of the high range-corrected SNR values present at nighttime have been smoothed, but obviously the contamination is still present. The bottom panel of Fig. 3 presents the hourly winds computed with a standard consensus procedure applied on the ICRA computed moments. These winds still show very large northwesterly values at nighttime, as the data at the moment level were still contaminated by migrating birds.
(top three panels) As in right side of Fig. 1, but for the data after they have been processed by ICRA. (bottom) Hourly winds computed by a standard consensus procedure on the data in the top three panels.
Citation: Journal of Atmospheric and Oceanic Technology 30, 12; 10.1175/JTECH-D-13-00089.1
In an effort to remove this contamination from the data, we tested the Gabor module extensively on a total of 18 days (6 for each site, with bird-contamination levels that varied from very strong to moderate to light).
However, the Gabor frame expansion for a given dataset is adjustable, with the main parameters selectable by the operator. Our tuning of the algorithm depends on two main parameters:
Percentage threshold: a range gate for a single dwell is flagged as suspect if any Gabor frequency has contamination that persists longer than the percentage threshold of the dwell time.
Maximum (max) spectral width: those range gates identified as suspect based on the percentage threshold will be rejected if their spectral width is larger than the selected maximum value of spectral width.
We tuned these two parameters differently for nighttime and daytime hours. After extensive testing, a percentage threshold of 10% and a maximum spectral width of 2 m s−1 were selected for nighttime, while a percentage threshold of 60% and a maximum spectral width of 2 m s−1 were used for daytime. The reason why a larger percentage threshold value was used for the daytime is because the true atmospheric signal in the highly convective boundary layer can have intermittent characteristics in the Gabor domain similar to those of birds at night (e.g., during hours 20–24 below 1 km AGL shown in Fig. 1, right panels). Using the less restrictive threshold values during the daytime allowed for truer atmospheric signal to be accepted. The daytime intermittency typically persists for more than 10% of the dwell, but it rarely persists for more than 60% of the dwell time. Since real spectral widths associated with PBL convection are often larger than 2 m s−1, the choice of percentage threshold principally determines whether a daytime moment is rejected. After the data were quality controlled at the time series level with the above-mentioned procedure, the data are much cleaner and the atmospheric signal is well revealed, particularly during daytime hours. However, even after applying the optimized Gabor module at the time series level, some contamination from migrating birds remained at nighttime, particularly during heavily contaminated hours, as visible in the top three panels of Fig. 4. The bottom panel in Fig. 4 presents the hourly winds computed with a standard consensus procedure applied on the Gabor-computed moments (top three panels of Fig. 4). These hourly winds still show a few very large northwesterly values at nighttime, as the data at the moment level were still contaminated by migrating birds. For this reason we apply another level of thresholding to the Gabor module. Only for nighttime hours, at the moment level, we reject any moment value that has a range-corrected SNR larger than 55 dB and simultaneously a spectral width larger than 1.2 m s−1.
(top three panels) As in right side of Fig. 1, but for the data after they have been processed by Gabor. (bottom) Hourly winds computed by a standard consensus procedure on the data in the top three panels.
Citation: Journal of Atmospheric and Oceanic Technology 30, 12; 10.1175/JTECH-D-13-00089.1
After the additional SNR-plus-spectral-width thresholding is applied, the remaining moments are shown in the top three panels of Fig. 5. Comparing moments in Fig. 5 with those in Fig. 1 (right side), it is apparent that most of the contaminated moments associated with contamination have been removed. The bottom panel of Fig. 5 also presents the hourly winds computed with a standard consensus procedure applied on the moments presented in the top three panels. These hourly winds still show several gates with very large northwesterly values at nighttime. However, we have found that application of a commonly used real-time moment-level pattern recognition processing scheme that is a modified version of the Weber–Wuertz algorithm of Weber et al. (1993) eliminates these few remaining outliers. The original Weber–Wuertz quality-control module implements a method to evaluate the wind (and virtual temperature when used on the radio acoustic sounding system measurements) data using time and height continuity of consensus data samples. The modified version performs time–height continuity checks on the moment-level radial velocities, derives meteorological-component velocities using the continuity-controlled radial velocities, and then applies final time–height continuity checks on the meteorological-component velocities to derive the resultant wind. The hourly winds computed by this procedure (top panel of Fig. 6) show a wind pattern smoother than the one in the bottom panel of Fig. 5, with several gaps filled with reasonable values of wind speed and some more winds recovered, especially at high altitudes when comparing with the bottom panel of Fig. 5.
(top three panels) As in right side of Fig. 1, but for the data after they have been processed by Gabor (plus additional thresholding on range-corrected SNR and relative spectral width). (bottom) Hourly winds computed by a standard consensus procedure on the data in the top three panels.
Citation: Journal of Atmospheric and Oceanic Technology 30, 12; 10.1175/JTECH-D-13-00089.1
(top) Hourly winds computed by the Weber–Wuertz procedure on the data in the top three panels of Fig. 5. (bottom) Hourly winds computed by the Weber–Wuertz procedure on the data in the top three panels of Fig. 5 after a 30% minimum percentage requirement has been added for the hourly wind computation.
Citation: Journal of Atmospheric and Oceanic Technology 30, 12; 10.1175/JTECH-D-13-00089.1
As a final optional step, we added a successive test during nighttime hours restricting the Weber–Wuertz algorithm to compute winds only if at least 30% of the quality-controlled moments were available for a given hour. With this constraint the hourly winds computed by the Weber–Wuertz algorithm applied to the moments of the top three panels of Fig. 5 are now presented in the bottom panel of Fig. 6. In this panel we notice that some potentially questionable winds were eliminated during nighttime. Clearly, the wind pattern is smoother in the bottom panel of Fig. 6, with several gaps filled with reasonable values of wind speed, enhancing the potential of radar wind profiling radars to provide good-quality winds for assimilation into numerical models. The particular day illustrated here is one with very strong bird contamination. For the other 17 days in our analysis, periods with strong contamination gave similar results, while on days with moderate or light contamination, birds were removed and much of the true clear-air signal was retained.
3. Conclusions
A challenging task is to produce high-quality wind profiler data for assimilation into numerical weather prediction models. One of the main problems is to eliminate contamination from migrating birds during the spring and fall seasons. In this study we applied a combination of multiple algorithms to this purpose, acting at the time series level with a Gabor frame expansion method; at the moments’ computational level with a state-of-the-art multipeak picking algorithm that includes the identification and removal of ground clutter and radio frequency interference; and ending at the moment-level pattern recognition and hourly wind computation with a modified version of the Weber–Wuertz processing algorithm. We tested and tuned our combination of methods on a dataset collected in California over three sites heavily contaminated by migrating birds. Although a few isolated erroneous winds were still present on some days, a vast majority of the winds demonstrated the high-quality levels required for assimilation into numerical models. The values of the algorithm parameters used in this study will be dependent on radar system parameters and may require further testing to find the best combination of parameters for different locations and different types of birds. Based on this sample dataset, it appears that the use of the Gabor transform, along with additional thresholding and pattern recognition processing, can explicitly eliminate bird contamination from UHF wind profiler radars. Further, for times when the bird contamination is moderate or light, most of the true atmospheric signal can be retained.
Acknowledgments
This study is sponsored by the U.S. Department of Energy. The authors wish to acknowledge all the engineers of the NOAA/ESRL Physical Sciences Division who maintained and operated the radars used in this study, and Timothy Coleman for promptly ensuring the availability of the data collected by them. We also wish to acknowledge three anonymous reviewers for their constructive comments.
REFERENCES
Griesser, T., and Richner H. , 1998: Multiple peak processing algorithm for identification of atmospheric signal in Doppler radar wind profiler spectra. Meteor. Z., 7, 292–302.
Lehmann, V., 2012: Optimal Gabor-frame-expansion-based intermittent-clutter-filtering method for radar wind profiler. J. Atmos. Oceanic Technol., 29, 141–158.
Lehmann, V., and Teschke G. , 2008: Advanced intermittent clutter filtering for radar wind profiler: Signal separation through a Gabor frame expansion and its statistics. Ann. Geophys., 26, 759–783.
Merritt, D. A., 1995: A statistical averaging method for wind profiler Doppler spectra. J. Atmos. Oceanic Technol., 12, 985–995.
Strauch, R. G., Merritt D. A. , Moran K. P. , Earnshaw K. B. , and Van De Kamp D. , 1984: The Colorado Wind-Profiling Network. J. Atmos. Oceanic Technol., 1, 37–49.
Weber, B. L., Wuertz D. B. , Welsh D. C. , and McPeek R. , 1993: Quality controls for profiler measurements of winds and RASS temperature. J. Atmos. Oceanic Technol., 10, 452–464.
Wilczak, J. M., and Coauthors, 1995: Contamination of wind profiler data by migrating birds: Characteristics of corrupted data and potential solutions. J. Atmos. Oceanic Technol., 12, 449–467.
