• Angevine, W. M., S. K. Avery, W. L. Ecklund, and D. A. Carter, 1993: Fluxes of heat and momentum measured with a boundary layer wind profiler radar–radio acoustic sounding system. J. Appl. Meteor.,32, 73–80.

    • Crossref
    • Export Citation
  • Barth, M. F., R. B. Chadwick, and D. W. van de Kamp, 1994: Data processing algorithms used by NOAA’s wind profiler demonstration network. Ann. Geophys.,12, 518–528.

    • Crossref
    • Export Citation
  • Beran, D. W., and T. L. Wilfong, 1998: U.S. wind profilers: A review. Office of the Federal Coordinator for Meteorological Services and Supporting Research (OFCM) Rep. FCM-R14-1998, 56 pp. [Available from OFCM, 8455 Colesville Road, Suite 1500, Silver Spring, MD 20910.].

  • Chadwick, R. B., 1986: Wind profiler demonstration network. Handbook for MAP, Vol. 20, S.A. Bowhill and B. Edwards, Eds., URSI/SCOSTEP, 336–337.

  • Doviak, R. J., and D. S. Zrnić, 1993: Doppler Radar and Weather Observations. Academic Press, 562 pp.

  • Fischler, M. A., and J. C. Bowles, 1981: A random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography. Commun. Assoc. Comput. Mach.,224, 381–395.

    • Crossref
    • Export Citation
  • ——, D. E. Wolfe, K. P. Moran, R. A. Paulus, K. D. Anderson, and L. T. Rogers, 1998: Measurements of clear-air gradients and turbulence properties with radar wind profilers. J. Atmos. Oceanic Technol.,15, 321–342.

    • Crossref
    • Export Citation
  • Harris, F. J., 1978: On the use of windows for harmonic analysis with the discrete Fourier transform. Proc. IEEE,66, 51–83.

    • Crossref
    • Export Citation
  • Hildebrand, P. H., and R. S. Sekhon, 1974: Objective determination of the noise level in Doppler spectra. J. Appl. Meteor.,13, 808–811.

    • Crossref
    • Export Citation
  • Hocking, W. K., 1997: System design, signal processing, and preliminary results for the Canadian (London, Ontario) VHF atmospheric radar. Radio Sci.,32, 687–706.

    • Crossref
    • Export Citation
  • ——, 1998: Recent advances in radar instrumentation and techniques for studies of the mesosphere, stratosphere, and troposphere. Radio Sci.,32, 2241–2270.

    • Crossref
    • Export Citation
  • May, P. T., and R. G. Strauch, 1989: An examination of wind profiler signal processing algorithms. J. Atmos. Oceanic Technol.,6, 731–735.

    • Crossref
    • Export Citation
  • ——, and ——, 1998: The effect of ground clutter on wind profiler velocity measurements. J. Atmos. Oceanic Technol.,15, 579–586.

    • Crossref
    • Export Citation
  • ——, ——, K. P. Moran, and W. L. Ecklund, 1990: Temperature sounding by RASS with wind profiler radars: A preliminary study. IEEE Trans. Geosci. Remote Sens.,28, 11–20.

    • Crossref
    • Export Citation
  • Merritt, D. A., 1995: A statistical averaging method for wind profiler Doppler spectra. J. Atmos. Oceanic Technol.,12, 985–995.

    • Crossref
    • Export Citation
  • ——, T. L. Wilfong, A. J. Francavilla, D. B. Wuertz, M. K. Simon, and B. L. Weber, 1997: Application of the prototype control, acquisition and signal processing engine for radars (CASPER) to wind profilers and RASS. Proc. 28th Conf. on Radar Meteorology, Austin, TX, Amer. Meteor. Soc. 244–245.

  • Petididier, M., A. Sy, A. Garrouste, and J. Delcourt, 1997: Statistical characteristics of the noise power spectral density in UHF and VHF wind profilers. Radio Sci.,32, 1229–1247.

    • Crossref
    • Export Citation
  • Röttger, J., and M. F. Larsen, 1990: UHF/VHF radar techniques for atmospheric research and wind profiler applications. Radar in Meteorology, D. Atlas, Ed., Amer. Meteor. Soc., 235–281.

    • Crossref
    • Export Citation
  • Sato, T., and R. F. Woodman, 1982: Spectral parameter estimation of CAT radar echoes in the presence of fading clutter. Radio Sci.,17, 817–826.

    • Crossref
    • Export Citation
  • Urkowitz, H., and J. P. Nespor, 1992: Obtaining spectral moments by discrete Fourier transform with noise removal in radar meteorology. Int. Geoscience and Remote Sensing Symp., Houston, TX, IGRASS, 125–127.

  • White, A. B., 1997: Radar remote sensing of scalar and velocity microturbulence in the convective boundary layer. NOAA Tech. Memo. ERL ETL-276, NOAA Environmental Laboratory, Boulder, 127 pp. [Available from NOAA/ETL/ET-5, 325 Broadway, Boulder, CO 80303.].

  • Wilczak, J. M., and Coauthors, 1995: Contamination of wind profiler data by migrating birds: Characteristics of corrupted data and potential solutions. J. Atmos. Oceanic Technol.,12, 449–467.

    • Crossref
    • Export Citation
  • View in gallery

    Steps in the CASPER spectrum generation process. The first step involves the acquisition of long time series and the application of a moving time-domain average. The digital Fourier transform of the averaged time series is computed next. Finally, a low-pass filter is applied to the spectral estimates before statistical averaging to produce a mean spectrum.

  • View in gallery

    Frequency aliasing regions. Because time series are sampled at discrete intervals, τ, signals with periods shorter than 2τ are aliased with longer period signals. For example, it is not possible to differentiate a constant or DC signal from one with a period of τ. Thus, letting 1/τb be the cutoff frequency of an ideal bandpass filter, the shaded areas represent the frequencies aliased into the region of interest (i.e., ±1/τb).

  • View in gallery

    Frequency response of a simple block or “boxcar” average.

  • View in gallery

    Aliasing region and frequency response of a simple boxcar average of length p. Applying a simple block average of length p to a time series with a sampling interval of τ (i.e., windowing p points at a time and shifting p points) causes frequencies shorter than the Nyquist frequency 1/2 to be aliased into the region of interest according to the filter response function. The aliased frequency region is shaded.

  • View in gallery

    The frequency response of a four-term Blackman–Harris filter.

  • View in gallery

    Aliasing region and frequency response of a four-term Blackman–Harris filter. Applying a four-term Blackman–Harris filter of length p (i.e., windowing p points at a time and shifting p points) to a time series with a sampling interval of τ causes frequencies shorter than the Nyquist frequency 1/2 to be aliased into the region of interest according to the filter response function. The aliased frequency region is shaded.

  • View in gallery

    Aliasing region and frequency response of a four-term Blackman–Harris filter followed by a frequency-domain low-pass filter. Here, the length of the time-domain filter is four times the compression factor p. Furthermore, p was chosen such that the Nyquist frequency 1/2 was 16 times larger than the atmospheric region of interest (i.e., ±1/τb). Thus, only 6.25% of the spectral coefficients were retained by the frequency-domain band pass filter.

  • View in gallery

    Aliasing region and frequency response of a simple block average followed by a frequency-domain low-pass filter. Here, the length of the time-domain filter is four times the compression factor p. Furthermore, p was chosen such that the Nyquist frequency 1/2 was 16 times larger than the atmospheric region of interest (i.e., ±1/τb). Thus only 6.25% of the spectral coefficients were retained by the frequency-domain bandpass filter.

  • View in gallery

    Full averaged spectrum produced by a statistical averaging procedure applied to 15 raw spectra. The data correspond to measurements taken on an oblique beam of a 449-MHz wind profiler. The individual raw spectra were computed from a 32 768-point time series spanning an observation time of ≈2 min.

  • View in gallery

    Same as Fig. 9, but showing only the area of interest between ±17.3 m s−1.

  • View in gallery

    Averaged spectrum corresponding to conventional signal processing. A 256-point block average was applied to the individual time series, followed by a 128-point FFT and averaging using a statistical averaging method.

  • View in gallery

    Averaged spectrum corresponding to unconventional signal processing. A 64-point, four-term Blackman–Harris filter with a 16-point shift was applied to the individual time series, followed by a 2048-point FFT and averaging using a statistical averaging method.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 197 194 8
PDF Downloads 50 49 6

Optimal Generation of Radar Wind Profiler Spectra

View More View Less
  • 1 Science and Technology Corporation, Hampton, Virginia
  • | 2 NOAA/ERL/Environmental Technology Laboratory, Boulder, Colorado
  • | 3 Cooperative Institute for Research in Environmental Sciences, University of Colorado, Boulder, Colorado
© Get Permissions
Full access

Abstract

Radar wind profilers (RWPs) sense the mean and turbulent motion of the clear air through Doppler shifts induced along several (3–5) upward-looking beams. RWP signals, like all radars signals, are often contaminated. The contamination is clearly evident in the associated Doppler spectra, and automatic routines designed to extract meteorological quantities from these spectra often yield inaccurate results. Much of the observed contamination is due to an aliasing of higher frequency signals into the clear-air portion of the spectrum and a broadening of the spectral contaminants caused by the relatively short time series used to generate Doppler spectra. In the past, this source of contamination could not be avoided because of limitations on the size and speed of RWP processing computers. Today’s computers, however, are able to process larger amounts of data at greatly increased speeds. Here it is shown how standard and well-known spectral processing methods can be applied to significantly longer time series to reduce contamination in the radar spectra and thereby improve the accuracy and the reliability of meteorological products derived from RWP systems. In particular, spectral processing methods to identify and remove contamination that is often aliased into the clear-air portion of the spectrum are considered. Optimal techniques for combining multiple spectra to produce averaged spectra are also discussed.

Corresponding author address: Mr. Timothy L. Wilfong, NOAA/ERL/ET4, 325 Broadway, Boulder, CO 80303-3328.

Email: twilfong@etl.noaa.gov

Abstract

Radar wind profilers (RWPs) sense the mean and turbulent motion of the clear air through Doppler shifts induced along several (3–5) upward-looking beams. RWP signals, like all radars signals, are often contaminated. The contamination is clearly evident in the associated Doppler spectra, and automatic routines designed to extract meteorological quantities from these spectra often yield inaccurate results. Much of the observed contamination is due to an aliasing of higher frequency signals into the clear-air portion of the spectrum and a broadening of the spectral contaminants caused by the relatively short time series used to generate Doppler spectra. In the past, this source of contamination could not be avoided because of limitations on the size and speed of RWP processing computers. Today’s computers, however, are able to process larger amounts of data at greatly increased speeds. Here it is shown how standard and well-known spectral processing methods can be applied to significantly longer time series to reduce contamination in the radar spectra and thereby improve the accuracy and the reliability of meteorological products derived from RWP systems. In particular, spectral processing methods to identify and remove contamination that is often aliased into the clear-air portion of the spectrum are considered. Optimal techniques for combining multiple spectra to produce averaged spectra are also discussed.

Corresponding author address: Mr. Timothy L. Wilfong, NOAA/ERL/ET4, 325 Broadway, Boulder, CO 80303-3328.

Email: twilfong@etl.noaa.gov

1. Introduction

The radar wind profile (RWP) technique has become widely accepted as an effective tool for probing the clear atmosphere, yielding useful meteorological information for both routine applications and atmospheric research. Although the technique is sound, there are many instances when algorithms developed to estimate Doppler spectral moments associated with clear-air returns produce inaccurate results because of the presence of strong contamination. The problem can be particularly significant for continuous real-time applications where postprocessing of the data is not an option (e.g., Barth et al. 1994). Profilers are increasingly used for boundary layer studies (e.g., Angevine et al. 1993), where contamination from ground clutter is substantial and the high spatial and temporal resolutions used preclude significant signal averaging. Finally, in many applications (e.g., White 1997; Gossard et al. 1998), the desire for quality estimates of all three profiler moments heightens the need for better automated processing algorithms.

The RWP technique depends on scatter from clear-air turbulence and stratified refractive layers to a degree that depends on the atmospheric conditions and radar frequency (e.g., see Röttger and Larsen 1990). RWP radars, like all radars, also are sensitive to noise, both internal (system noise) and external (noise from lightning and extraterrestrial sources), radio frequency interference (RFI), ground and sea clutter, and intermittent returns from aircraft and birds, all of which can produce signals much stronger than the relatively weak clear-air return. While the radar receiver and antenna are designed to minimize contributions from undesirable sources, contamination can never be entirely eliminated, and, therefore, signal processing must be robust in order to deal with such contamination.

In the past, online (i.e., near real time) spectral processing implemented on RWP systems has been very limited, using relatively simple time-domain averaging (TDA) techniques for high-frequency noise suppression, and mean signal removal for ground clutter rejection. The subsequent signal identification function generally made the naive assumption that the atmospheric return was the only signal present in the resulting spectra, which then led to inaccurate moment estimates.

One of the earliest quality-control techniques focused on the elimination of spurious wind estimates from an extended height–time record. It was based on the optimistic belief that erroneous measurements would be few in number and easily identified by comparison with an overwhelming number of accurate measurements. Traditionally, radial velocities submitted to such a consensus-type filtering (Fischler and Bowles 1981) required relatively long (typically hourly) averages in order to eliminate spurious measurements. While the consensus technique eliminates the occasional outlier, it is ineffective in the presence of persistent contamination. An example is the National Oceanic and Atmospheric Administration’s (NOAA) Wind Profiler Demonstration Network (WPDN) (Chadwick 1986). Although current algorithms (Barth et al. 1994) do a good job of retrieving hourly averaged horizontal winds over a wide range of heights, occasionally severe biases can be introduced by migrating birds (Wilczak et al. 1995) and by ground clutter at the lowest heights (May and Strauch 1989, 1998). In addition, the recent decision by the National Telecommunications and Information Administration to shift the WPDN frequency allocation from 404 to 449 MHz has increased the likelihood of RFI contamination from amateur radio operators, who have secondary access in a band that includes 449 MHz.

Several research organizations, including the NOAA Environmental Research Laboratories, have been for some time addressing the recognized limitations of signal processing, and developing and maintaining their own systems of postprocessing and quality control. With the increasing emphasis upon routine operations in both research and nonresearch applications, it is not cost effective for all organizations inside and outside of government to support the staff and computer infrastructure necessary for postprocessing. Furthermore, postprocessing would not be necessary if adequate and complete signal processing could be implemented online with the real-time data acquisition component. Therefore, in 1996, the NOAA Environmental Technology Laboratory established a program to 1) develop a modern software architecture that facilitates efficient implementation of new signal processing algorithms, while allowing replacement of underlying radar hardware components with only minor software modifications; 2) implement a new signal processing system that produces reliable and accurate meteorological products at the radar in near real time; and 3) facilitate the implementation of new signal processing on existing and new RWP systems. The program is called CASPER [Control, Acquisition, and Signal Processing Engine for Radar (Merritt et al. 1997)]. Its goal is to develop a complete signal processing scheme, beginning with radar control and acquisition of time series and ending with useful meteorological products containing error estimates.

For purposes of this discussion and for the goals of the CASPER program, signal processing is separated into four major components, with each further divided into subcomponents. The first major component, and the subject of detailed treatment in this paper, is the optimal generation of Doppler spectra, which starts with raw time series data generated at the output of the radar receiver and ends with the “averaged” spectral estimates. The second major component involves the detection of all significant signals present and the calculation of the corresponding spectral moments. The third major function is the identification of the clear-air echo using techniques that include height–time pattern recognition algorithms applied to the significant signal moments. The fourth and final major component involves the derivation of meteorological products from the radial velocity estimates and from the other spectral moments associated with the clear-air echo.

A great deal of effort has been devoted to reducing the impact of contaminating signals on the estimation of clear-air spectral moments. Several past efforts have addressed processing in the spectral domain, including the work of Sato and Woodman (1982), Merritt (1995), Hocking (1997) and May and Strauch (1998). A recent report highlights the status of RWP development in the United States (Beran and Wilfong 1997), including recent signal processing work. In general, the situation is often complicated by the broadening of spectral contaminants and the aliasing of unwanted signals into the clear-air band due to the use of relatively short time series required by past computational constraints. Recent advances in computer technology have enabled the online spectral processing of longer time series, which, with appropriate filtering, significantly improves the quality of the spectra. The advantages of processing longer time series were pointed out by Hocking (1997, 1998), and are elaborated here with special emphasis on network-scale profilers operating near 449 MHz.

2. RWP signal processing overview

RWP spectra are computed independently on each of many data channels (one for each range gate and antenna beam) starting with time series of radar signals sampled using analog-to-digital converters on the in-phase (I) and quadrature (Q) channels at the video output of the radar receiver. Each time series can be represented by the complex numbers f(tn), where the real part corresponds to the I channel and the imaginary part corresponds to the Q channel. The sample times for this time series are tn = t0 + (n = 0, 1, . . . , N − 1), where t0 is an arbitrary time reference (unique for each data channel) that indicates the start of the time series, τ is the uniform time interval between samples (i.e., the interpulse period between consecutive transmitted pulses), and N is a prescribed number of points in the time series.

The interpulse period τ is an important parameter both for radar operations and for spectral processing. It determines the maximum unambiguous range of the radar Rmax = /2, where c is the speed of light. Since all beams are directed nearly vertically, Rmax also defines the approximate altitude coverage. Normally, Rmax is set just beyond the last range where any detectable signal is expected from the atmosphere. Typical values for τ are in the range 20–120 μs, giving typical values for Rmax in the range 3–18 km. The maximum altitude of useful signal is determined by the radar frequency fr and sensitivity as well as by the atmosphere itself. Thus, some care is necessary in the selection of τ to avoid contamination from sources at ranges beyond Rmax.

The interpulse period τ also determines the maximum unambiguous radial velocity or Nyquist velocity υN = c/4frτ. For example, a sample interval τ = 60 μs at a frequency fr = 449 MHz yields a Nyquist velocity υN = 2784 m s−1, which is much larger than any expected radial velocity for atmospheric winds. The total velocity interval 2υτ is much wider than the clear-air portion of the spectrum, which is a few tens of meters per second about zero velocity for winds, and a few meters per second about the mean acoustic velocity (≈320 m s−1) for Radio Acoustic Sounding System (RASS) echoes (e.g., May et al. 1990). One of the important goals of spectral analysis is to minimize contamination in the useful portion of the spectrum from contamination, such as noise and RFI in other portions of the spectral range ±υN.

Whereas the sample interval τ is restricted by Rmax, the duration T of any given time series is determined by the time required for estimating a single spectrum. Then the number points in a time series is given by N = T/τ. While each time series is composed of a contiguous series of data samples, many such time series, not necessarily contiguous in time, may be required to produce an averaged or smoothed spectral estimate. Smooth spectra are desirable in order to improve detectability of signals in the presence of noise, especially at low signal-to-noise ratios (SNRs). For example, if the sample interval is τ = 60 μs and the number of time series points is N = 32 768, then the time series duration T is almost 2 s to estimate one spectrum. As a result, it will take a total dwell of about 1 min in order to collect 30 spectra for the purpose of smoothing, even if all 30 spectra are not estimated from contiguous time series.

The time series duration T is also important because it determines the velocity resolution Δυ = c/2frT = 2υN/N achieved in a spectral estimate. For T = 1 s and fr = 449 MHz, the velocity resolution is about 0.3 m s−1. The velocity resolution for atmospheric signals is generally determined by the number of points needed to adequately resolve the clear-air peak, which depends on wind shear and turbulent motion within the radar pulse volume. Even higher resolution may be needed to resolve multiple signals falling within the clear air of the spectral band.

The time series for a single range gate typically consist of data samples spaced some tens of microseconds apart in time, while a spectral estimate is generated at intervals of some tens of seconds. Since one useful radial velocity measurement is expected from each spectral estimate, data reduction of about a million to one is accomplished in spectral analysis. For practical reasons, especially with existing systems, time series and the individual spectra are rarely recorded even if it is possible. Therefore, a very large amount of potentially useful information may be lost in the process of deriving radial velocities. Hence, it is important to apply the spectral analysis correctly. If spectral analysis is effective, contamination from noise and unwanted signals will be minimized and whatever atmospheric signals are present will be detected.

Conventional signal processing techniques for network-scale RWPs are generally very similar to those used in the NOAA WPDN (Barth et al. 1994). We summarize briefly here. After removal of the signal mean (i.e., “DC removal”), low-pass filtering and data compression of the time series are achieved in one step by averaging large blocks (100–200 points or so) of time series data using a simple “boxcar” technique with equally weighted samples to produce one filtered point. In each data channel, resulting in filtered segments of lengths of typically 64, 128, or 256 points, are used to compute power spectra. Then to improve detectability by reducing spectral variance, a number of spectra (50 or so) are averaged, again using a simple average. Of course, this step also achieves some data compression. Signal detection based upon the strongest signal present will oftentimes mistake ground clutter for the atmospheric signal. Therefore, before proceeding to the signal identification step, removal of ground clutter is attempted by replacing up to nine spectral bins centered on the zero velocity bin with an interpolation from end points. These traditional techniques are very effective at compression of the data but are certainly not optimal filters and can allow contamination to pass into the final, meteorological products.

3. CASPER spectra generation

The spectral processing approach in CASPER is fundamentally different from the more conventional processing described in section 2. The first few processing steps are illustrated in Fig. 1 and focus on the generation of Doppler spectra in a way that minimizes the amount of contamination that enters into the clear-air portion of the averaged spectrum. The I and Q channels are sampled with sufficient dynamic range that DC removal is not necessary. Long time series are collected and then subjected to TDA using sliding windows before being Fourier transformed. TDA acts as a low-pass filter suppressing high-frequency noise and compressing the data. The application of TDA assumes the signals of interest are baseband (i.e., near zero Doppler shift). Although clear-air turbulence signals satisfy this constraint, RASS echoes, which contain Doppler shifts on the order of 320 m s−1 (∼1 kHz for fr = 449 MHz), do not. When RASS is operating, these signals are mixed to baseband prior to TDA. When data compression is not an issue, the ideal strategy is to neglect TDA entirely and compute a digital Fourier transform (DFT) over the longest time series possible. Contaminating signals could be removed simply by zeroing or discarding spectral coefficients outside the area of interest, that is, by applying a low-pass filter in the frequency domain.

Current processing constraints, however, often require some degree of time-domain data compression, and CASPER allows for a flexible, two-step filtering process comprising the low-pass filtering associated with TDA and low-pass filtering applied in the spectral domain. In a final step, many raw spectra, not necessarily from a single contiguous dwell, are combined (i.e., averaged) in an intelligent way to eliminate contamination while smoothing the spectra to improve signal detectability (e.g., Merritt 1995).

a. Spectral filtering

The CASPER low-pass filter associated with TDA is designed toward the ideal low-pass response function H(υr) = 1, for |υr| < υb, and H(υr) = 0, otherwise, where υr is the radial velocity measurement and υb is the desired velocity bandwidth. The velocity bandwidth is constrained such that υbυN. Only timescales longer than c/2frυb are resolved by this ideal low-pass filter. A very long DFT over the duration of the time series (e.g., 6 s), where only the coefficients in the band ±υb are retained, approaches the ideal. Because of discrete sampling at intervals of τ, however, signals falling in the velocity bands nc/2frτ ± υb, where n ≠ 0 is an integer, will be aliased onto the desired baseband signals. Figure 2 depicts the situation. Any signals in the shaded frequency bands centered about τ−1 are aliased into the atmospheric region at baseband.

The introduction of aliasing at regular frequency intervals over the radar bandwidth has important implications to the rejection of RFI. In the presence of interference at a known frequency (e.g., amateur radio operations near 449 MHz), τ can be chosen such that the interference does not fall at a harmonic of the sampling frequency (i.e., n/τ), thereby reducing the probability of RFI appearing in the band of interest caused by aliasing from sampling.

Computing numerous, very long DFTs (on the order iof 104–105 points) required for real-time operation is computationally intensive. TDA using a simple boxcar average is easy to implement. Thus, traditionally, boxcar TDA has been used for low-pass filtering and data compression. The signal-to-noise-ratio (SNR) is increased by TDA because the sampling interval τ ≈ 50 μs is typically much smaller than the coherence time τc ≈ 0.1 s of the atmospheric signal [i.e., τc ≈ 0.1λ/;συ ≈ 0.1 s for a radar wavelength λ ≈ 1 m and a wind variance συ ≈ 1 m s−1 ∝ Doppler spectrum width (e.g., Doviak and Zrnić 1993)]. Although the noise from sample to sample is uncorrelated, the atmospheric return is highly correlated, and TDA using a window of length p time samples increases the SNR by a factor of p provided τc. The corresponding data compression factor is N/p.

The boxcar average, however, does not have the optimal filter response. That is, signals at different frequencies within the pass band are not passed with uniform energy, and signals outside the frequency bandpass are not completely suppressed. Figure 3 shows a sample filter response function for the boxcar average. The first sidelobe level is approximately 13 dB down from the main lobe peak, and the sidelobes fall off at 6 dB per octave (e.g., see Harris 1978). The number of points in the fast Fourier transform (FFT) is N/p, and is typically 64, 128, or 256. Although the velocity resolution, which depends on the length of the entire time series, is unchanged, the new Nyquist velocity is υN = c/4pfrτ. The shaded region in Fig. 4 shows the frequency range over which signals are aliased into the baseband with a magnitude determined by the filter response function (i.e., the region spanned by the individual spectral bands n/ ± 1/2τ).

Harris (1978) describes a number of windowing functions that can be used in TDA to both compress time series data and serve as a low-pass filter. We have chosen a four-term Blackman–Harris (BH) window where the peak sidelobe level is down approximately 92 dB over the broadened main lobe. Of course, we must trade main lobe width for this superior sidelobe performance. Figure 5 shows a sample BH response function. Again, if we apply the filter in the same way as the boxcar average (windowing p points at a time and shifting p points), the Nyquist velocity is the same as for a boxcar average, and frequencies outside the Nyquist are aliased, as shown in Fig. 6. Note in this configuration, where the length of the filter and compression factor are the same, the first null is at 4/pτ. Thus, the main lobe is a factor of 4 wider than the boxcar average, but the suppression of interference outside this main lobe region is substantially reduced by the extremely low sidelobes of the BH window. By allowing the filter length to be controlled independently of the number of points the filter is moved, the position of the first null can be varied. Using the BH filter as an example, we may choose the filter length l to be four times the data compression factor p, effectively placing the first null at 1/ rather than at 4/pτ.

In CASPER, the low-pass filtering in the time domain (i.e., the TDA) is followed by a DFT, and then by a low-pass filtering (i.e., frequency clipping) of the spectral estimates. The time domain filter may employ a boxcar, BH, or other filters. The second stage is also a low-pass filter but is applied to the spectral estimates. We define the spectral coefficients m after low-pass filtering as m = fm, for |m| < mb, and m = 0, otherwise, where fm is the spectral estimate corresponding to index m and mb = υbυ. Note, the total number of time series points that must be acquired is then N = l + p(NDFT − 1), where NDFT is the number of points in the DFT.

We now consider the combined effect of a low-pass filtering in the time domain and a low-pass filtering in the spectral domain. By choosing p such that υbυN (i.e., pc/4υNfrτ), we can minimize aliasing in the band ±υb and approach the response of an ideal low-pass filter. Suppose, for example, we select a compression factor p, such that the resulting Nyquist velocity υN = 16υb. By choosing l = 4p, the first aliasing region lies in the null of the main lobe (see Fig. 7). The maximum out-of-band response in this region is approximately −81 dB. Since we chose the filter length to be an integral multiple of the compression factor, all higher frequency aliasing regions also lie in filter nulls. Further, the band of interest (±υb) lies in the very flat region of the main BH lobe (minimum baseband response >−0.01 dB). If other constraints require the use of a conventional boxcar average, good performance can still be achieved by choosing p = l = 16. As shown in Fig. 8, the maximum out-of-band response is only down approximately −30 dB but is still significantly better than standard processing.

b. Spectral “averaging”

In standard practice, the dwell time chosen is long enough to permit the estimation of several independent spectra. Those spectra are averaged in order to produce a smooth spectrum, in which the atmospheric signal is more reliably detected. This averaging does not increase the SNR, but it does improve the detectability of the atmospheric signal by reducing the variance in the spectral estimates. Signal detectability is directly proportional to the square root of the number of averages. Thus, detectability slowly improves with increasing numbers of spectra averaged and, thus, for longer overall dwell periods. On the other hand, if the SNR is sufficiently large, then spectral averaging may be unnecessary for signal detection (White 1997). However, the clear-air return is frequently so weak that a certain amount of spectral averaging is usually required. Let Ŝm = |m|2 be the spectral energy density estimate. Over the period of a dwell or several dwells, we produce a number of spectral estimates Ŝm(tk), where tk = t0 + NKτ (k = 0, 1, . . . , K − 1) and K is the number of spectral estimates made. The total dwell time, not necessarily contiguous, is then given by T = NKτ. Traditionally, a mean of the spectral estimates within a single dwell would be generated. This arithmetic mean, however, does not reject transient signals. For example, a single strong signal from transient interference such as an airplane, a bird, or RFI in one of the spectra will contaminate the mean. One option is to choose the median of the spectral estimates rather than the mean. Such median filtering would reduce the impact of a few strong transient signals on the filtered spectrum.

Atmospheric signals, as well as signals from precipitation, clutter, RFI, etc., can be distinguished from noise by using the very different statistical characteristics for radar returns for such signals and those for noise. Atmospheric and other signals are limited to a narrow frequency band, while noise is usually “white” or uniformly distributed across the Doppler spectrum. The detection of atmospheric signals depends upon the ability to make this distinction. Individual Doppler spectra consist of large numbers of data points that can be used in statistical tests to objectively determine the noise level (Hildebrand and Sekhon 1974; Urkowitz and Nespor 1992; Petididier et al. 1997). These tests attempt to determine whether a spectrum consists of noise alone or noise plus signal. When a signal is present, these tests then attempt to separate the signal from the noise for purposes of estimating the spectral moments of the signal, including the Doppler velocity that is used to derive the atmospheric winds and temperatures. Hildebrand and Sekhon (1974) compared statistical moments estimated from data with their expected values based upon an assumed Gaussian statistical distribution and an assumed flat spectrum for noise. Urkowitz and Nespor (1992) used the Kolmogorov–Smirnov test to determine the flatness of the radar Doppler spectrum, which is assumed to be flat or white for noise only. The Kolmogorov–Smirnov test is a sampling distribution approach of statistical inference that distinguishes different statistical distributions for noise and for a signal. Petididier et al. (1997) studied the statistics of profiler noise as an indicator of the signal quality and accuracy of mean noise estimation algorithms.

Such statistical methods for separating and identifying noise and atmospheric signals are examples of baseband filtering. Obviously, multiple signals that occupy the spectral band may be separated and identified based upon differences in their frequency content. After all, that is the motivation for spectral processing. However, some signals (e.g., atmospheric signals and returns from birds) overlap in frequency and cannot always be distinguished on the basis of their frequency content alone. When multiple signals overlap and when those multiple signals must be identified in order for signal processing to proceed, statistical or physical models may be used. Then, differences in the statistical or temporal behavior of the different signals may be used to sometimes separate those signals. The aforementioned statistical methods for determining noise illustrate the use of statistical models.

In addition to the arithmetic mean, which is always computed, CASPER processing currently offers median filtering or the algorithm introduced by Merritt (1995) to reject intermittent returns from such sources as aircraft, birds, and RFI. The Merritt (1995) technique uses a combination statistical–dynamical model. Normally, the algorithm is applied to individual spectra obtained over a single dwell; however, it could be applied to spectra obtained over a number of dwells. The inherent assumption is that the radar averaging time is long enough to reveal the transient behavior of targets like aircraft and birds, but short enough that the atmospheric signal is stationary. Further, since aircraft, birds, and transient RFI can present radar returns much stronger than the clear-air return from the atmosphere, there must be some period during the averaging time in which the atmospheric signal is detectable. In any case, in addition to the averaged spectral values, sufficient statistical information is kept for use in the signal detection and identification processes. Specifically, the information retained for each spectral point is the average value, the number of points comprising the average, the standard deviation of those values, and the threshold value above which data were rejected.

4. An example: RFI rejection at 449 MHz

Since RFI poses a serious potential problem for developing RWP systems operating at the new standard frequency of 449 MHz, time series were collected using CASPER to operate a 449-MHz RWP just east of the Continental Divide at Erie, Colorado, during the month of March 1997. This 449-MHz RWP is a quarter-scale prototype of the next-generation NOAA network systems. This quarter-scale system operates at low power and has a power–aperture product of approximately 16 W m2. The time series were spectrally analyzed with CASPER using both traditional and nontraditional algorithms. The data selected for analysis were gathered on 10 March 1997 at Erie, Colorado. The system was configured to operate in the standard five-beam mode with an interpulse period τ = 37.6 μs. Here we show data from the east oblique beam gathered over a 2-min period beginning at 2310 UTC. During this 2-min period, 15 sets of raw time series of length approximately 33 000 points each were recorded.

The 15 sets of time series were first processed using long (32 768) point FFTs. The resulting 15 spectra were then averaged using a statistical averaging method (SAM) (Merritt 1995). Figure 9 shows a sample of the SAM average to heights of 3500 m. As shown by Fig. 9, there is considerable persistent RFI present, but none in the atmospheric region close to zero velocity. Figure 10 shows the same data where only the velocity bins between ±17.3 m s−1 are kept. To demonstrate the results of conventional processing these same data were first processed through a 256-point boxcar TDA, followed by a 128-point FFT. The resulting 15 spectra were averaged using the same SAM technique. As Fig. 11 shows, some of the RFI is aliased to very near the Nyquist velocity of ±17.3 m s−1. Instead of using the conventional time-domain boxcar averaging filter, these same time series were processed with the four-term BH filter. The filter length was chosen to be four times the data compression factor. In this case the data compression factor was 16. The time-domain filter was followed by a 2048-point FFT clipped (i.e., low-pass filtered) to retain only velocities between ±17.3 m s−1. Again, the SAM average was applied to the resulting raw spectra. As seen in Fig. 12, this technique has effectively rejected the RFI. In this case, very similar results are obtained by replacing the BH filter with a conventional time-domain boxcar average with p = l = 16, followed again by the 2048-point FFT clipping. Since the resulting spectrum is nearly identical to Fig. 12, the results are not shown here.

5. Discussion and conclusions

We have demonstrated that the application of simple proven filtering techniques in the time and frequency domains can greatly improve the ability to reject contamination. Optimal low-pass filtering can be accomplished by forgoing any time-domain averaging and executing a very long digital Fourier transform to produce a spectral estimate where coefficients outside the band of interest are discarded. While such computationally intensive processes were not practical just a few years ago, advances in general-purpose computing hardware now make such a design practical to implement. Still, cost and performance issues may dictate that some time-domain integration be done. A combination of minimal time-domain averaging and the spectral low-pass filter can still achieve good contamination rejection, especially if a weighted averaging technique is used.

Traditionally, numerous spectra collected over a minute or so have been simply averaged incoherently to reduce variability and thereby increase signal detectability. When some of the spectra are contaminated (e.g., by radio frequency interference, bird echoes, etc.), a simple average can produce a contaminated mean spectrum in which the atmospheric signal is obscured by the contamination. Intermittent contamination is reduced or eliminated using other smoothing techniques such as a median filter or a statistical averaging method (Merritt 1995).

These filtering techniques have the potential to greatly improve the quality of the average spectra used for signal detection and identification. However, the performance of these techniques will depend heavily on the situation. While it is highly probable that out-of-band contamination will be rejected, persistent baseband contamination, such as ground clutter, sea clutter, and radio frequency interference, will likely be present in the averaged spectra. Later stages of processing must distinguish the atmospheric signal, if present, from among several other signals.

Acknowledgments

We thank two anonymous reviewers whose suggestions greatly helped to improve the presentation of this material.

REFERENCES

  • Angevine, W. M., S. K. Avery, W. L. Ecklund, and D. A. Carter, 1993: Fluxes of heat and momentum measured with a boundary layer wind profiler radar–radio acoustic sounding system. J. Appl. Meteor.,32, 73–80.

    • Crossref
    • Export Citation
  • Barth, M. F., R. B. Chadwick, and D. W. van de Kamp, 1994: Data processing algorithms used by NOAA’s wind profiler demonstration network. Ann. Geophys.,12, 518–528.

    • Crossref
    • Export Citation
  • Beran, D. W., and T. L. Wilfong, 1998: U.S. wind profilers: A review. Office of the Federal Coordinator for Meteorological Services and Supporting Research (OFCM) Rep. FCM-R14-1998, 56 pp. [Available from OFCM, 8455 Colesville Road, Suite 1500, Silver Spring, MD 20910.].

  • Chadwick, R. B., 1986: Wind profiler demonstration network. Handbook for MAP, Vol. 20, S.A. Bowhill and B. Edwards, Eds., URSI/SCOSTEP, 336–337.

  • Doviak, R. J., and D. S. Zrnić, 1993: Doppler Radar and Weather Observations. Academic Press, 562 pp.

  • Fischler, M. A., and J. C. Bowles, 1981: A random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography. Commun. Assoc. Comput. Mach.,224, 381–395.

    • Crossref
    • Export Citation
  • ——, D. E. Wolfe, K. P. Moran, R. A. Paulus, K. D. Anderson, and L. T. Rogers, 1998: Measurements of clear-air gradients and turbulence properties with radar wind profilers. J. Atmos. Oceanic Technol.,15, 321–342.

    • Crossref
    • Export Citation
  • Harris, F. J., 1978: On the use of windows for harmonic analysis with the discrete Fourier transform. Proc. IEEE,66, 51–83.

    • Crossref
    • Export Citation
  • Hildebrand, P. H., and R. S. Sekhon, 1974: Objective determination of the noise level in Doppler spectra. J. Appl. Meteor.,13, 808–811.

    • Crossref
    • Export Citation
  • Hocking, W. K., 1997: System design, signal processing, and preliminary results for the Canadian (London, Ontario) VHF atmospheric radar. Radio Sci.,32, 687–706.

    • Crossref
    • Export Citation
  • ——, 1998: Recent advances in radar instrumentation and techniques for studies of the mesosphere, stratosphere, and troposphere. Radio Sci.,32, 2241–2270.

    • Crossref
    • Export Citation
  • May, P. T., and R. G. Strauch, 1989: An examination of wind profiler signal processing algorithms. J. Atmos. Oceanic Technol.,6, 731–735.

    • Crossref
    • Export Citation
  • ——, and ——, 1998: The effect of ground clutter on wind profiler velocity measurements. J. Atmos. Oceanic Technol.,15, 579–586.

    • Crossref
    • Export Citation
  • ——, ——, K. P. Moran, and W. L. Ecklund, 1990: Temperature sounding by RASS with wind profiler radars: A preliminary study. IEEE Trans. Geosci. Remote Sens.,28, 11–20.

    • Crossref
    • Export Citation
  • Merritt, D. A., 1995: A statistical averaging method for wind profiler Doppler spectra. J. Atmos. Oceanic Technol.,12, 985–995.

    • Crossref
    • Export Citation
  • ——, T. L. Wilfong, A. J. Francavilla, D. B. Wuertz, M. K. Simon, and B. L. Weber, 1997: Application of the prototype control, acquisition and signal processing engine for radars (CASPER) to wind profilers and RASS. Proc. 28th Conf. on Radar Meteorology, Austin, TX, Amer. Meteor. Soc. 244–245.

  • Petididier, M., A. Sy, A. Garrouste, and J. Delcourt, 1997: Statistical characteristics of the noise power spectral density in UHF and VHF wind profilers. Radio Sci.,32, 1229–1247.

    • Crossref
    • Export Citation
  • Röttger, J., and M. F. Larsen, 1990: UHF/VHF radar techniques for atmospheric research and wind profiler applications. Radar in Meteorology, D. Atlas, Ed., Amer. Meteor. Soc., 235–281.

    • Crossref
    • Export Citation
  • Sato, T., and R. F. Woodman, 1982: Spectral parameter estimation of CAT radar echoes in the presence of fading clutter. Radio Sci.,17, 817–826.

    • Crossref
    • Export Citation
  • Urkowitz, H., and J. P. Nespor, 1992: Obtaining spectral moments by discrete Fourier transform with noise removal in radar meteorology. Int. Geoscience and Remote Sensing Symp., Houston, TX, IGRASS, 125–127.

  • White, A. B., 1997: Radar remote sensing of scalar and velocity microturbulence in the convective boundary layer. NOAA Tech. Memo. ERL ETL-276, NOAA Environmental Laboratory, Boulder, 127 pp. [Available from NOAA/ETL/ET-5, 325 Broadway, Boulder, CO 80303.].

  • Wilczak, J. M., and Coauthors, 1995: Contamination of wind profiler data by migrating birds: Characteristics of corrupted data and potential solutions. J. Atmos. Oceanic Technol.,12, 449–467.

    • Crossref
    • Export Citation

Fig. 1.
Fig. 1.

Steps in the CASPER spectrum generation process. The first step involves the acquisition of long time series and the application of a moving time-domain average. The digital Fourier transform of the averaged time series is computed next. Finally, a low-pass filter is applied to the spectral estimates before statistical averaging to produce a mean spectrum.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

Fig. 2.
Fig. 2.

Frequency aliasing regions. Because time series are sampled at discrete intervals, τ, signals with periods shorter than 2τ are aliased with longer period signals. For example, it is not possible to differentiate a constant or DC signal from one with a period of τ. Thus, letting 1/τb be the cutoff frequency of an ideal bandpass filter, the shaded areas represent the frequencies aliased into the region of interest (i.e., ±1/τb).

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

Fig. 3.
Fig. 3.

Frequency response of a simple block or “boxcar” average.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

Fig. 4.
Fig. 4.

Aliasing region and frequency response of a simple boxcar average of length p. Applying a simple block average of length p to a time series with a sampling interval of τ (i.e., windowing p points at a time and shifting p points) causes frequencies shorter than the Nyquist frequency 1/2 to be aliased into the region of interest according to the filter response function. The aliased frequency region is shaded.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

Fig. 5.
Fig. 5.

The frequency response of a four-term Blackman–Harris filter.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

Fig. 6.
Fig. 6.

Aliasing region and frequency response of a four-term Blackman–Harris filter. Applying a four-term Blackman–Harris filter of length p (i.e., windowing p points at a time and shifting p points) to a time series with a sampling interval of τ causes frequencies shorter than the Nyquist frequency 1/2 to be aliased into the region of interest according to the filter response function. The aliased frequency region is shaded.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

Fig. 7.
Fig. 7.

Aliasing region and frequency response of a four-term Blackman–Harris filter followed by a frequency-domain low-pass filter. Here, the length of the time-domain filter is four times the compression factor p. Furthermore, p was chosen such that the Nyquist frequency 1/2 was 16 times larger than the atmospheric region of interest (i.e., ±1/τb). Thus, only 6.25% of the spectral coefficients were retained by the frequency-domain band pass filter.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

Fig. 8.
Fig. 8.

Aliasing region and frequency response of a simple block average followed by a frequency-domain low-pass filter. Here, the length of the time-domain filter is four times the compression factor p. Furthermore, p was chosen such that the Nyquist frequency 1/2 was 16 times larger than the atmospheric region of interest (i.e., ±1/τb). Thus only 6.25% of the spectral coefficients were retained by the frequency-domain bandpass filter.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

 Fig. 9.
Fig. 9.

Full averaged spectrum produced by a statistical averaging procedure applied to 15 raw spectra. The data correspond to measurements taken on an oblique beam of a 449-MHz wind profiler. The individual raw spectra were computed from a 32 768-point time series spanning an observation time of ≈2 min.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

 Fig. 10.
Fig. 10.

Same as Fig. 9, but showing only the area of interest between ±17.3 m s−1.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

 Fig. 11.
Fig. 11.

Averaged spectrum corresponding to conventional signal processing. A 256-point block average was applied to the individual time series, followed by a 128-point FFT and averaging using a statistical averaging method.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

 Fig. 12.
Fig. 12.

Averaged spectrum corresponding to unconventional signal processing. A 64-point, four-term Blackman–Harris filter with a 16-point shift was applied to the individual time series, followed by a 2048-point FFT and averaging using a statistical averaging method.

Citation: Journal of Atmospheric and Oceanic Technology 16, 6; 10.1175/1520-0426(1999)016<0723:OGORWP>2.0.CO;2

Save