1. Introduction
The U.S. Air Force operates a network of five Radian LAP-3000 915-MHz Doppler radar wind profilers (DRWPs) with radio acoustic sounding systems (RASS) in the Kennedy Space Center/Cape Canaveral Air Force Station (KSC/CCAFS) area (hereafter referred to as the Eastern Range or ER) (Heckman et al. 1996). These profilers provide high-resolution wind estimates between the top of the KSC/CCAFS wind tower network at 150 m and the lowest gate of the NASA 50-MHz DRWP at 2 km. The data are used in support of launch, landing, and ground operations on the ER.
DRWP wind estimates can be affected by main or sidelobe contamination due to reflections from nearby traffic, trees, ocean waves, sea spray, aircraft, birds, or precipitation, and by velocity folding. Each of the ER profilers is located in an area prone to these effects. In addition, one is next to a small airport with both heavy air and ground traffic. Another is next to a road with heavy morning and afternoon automobile traffic. The remaining three are near dense vegetation and trees.
Inaccurate data are not acceptable for operational use, and real-time manual quality control (QC) of the data is not practical. As a result, the Applied Meteorology Unit was tasked to determine what quality control routines would be appropriate for the data collected by the ER DRWP network. In all, five algorithms were developed or acquired and modified for use with this specific network: a consensus time check, a precipitation contamination check, a median filter (Carr et al. 1995), the Weber–Wuertz pattern recognition algorithm (Weber and Wuertz 1991; Miller et al. 1994), and a signal-to-noise ratio (SNR) threshold. These algorithms were compared separately and in combination by Lambert and Taylor (1998, hereafter LT), who determined that using a subset of them in combination produced more accurate results than using them individually.
Recent work to study the statistical properties of boundary layer wind changes using the ER 915-MHz DRWP data prompted the use of the QC routines described in LT. The data quality and sample size requirements were so stringent that automated QC was essential to the feasibility of the project. The recommendations of LT were adopted with minor modifications to produce the composite QC algorithm described herein. It was much simpler to implement than Weber–Wuertz and was expected to perform as well or better based on the results in LT. Twenty-three months of data from the five profilers were processed using the composite algorithm. The processed data were examined manually to ensure that no erroneous data remained before the statistical analysis began.
This paper describes the algorithms and discusses the performance of both the instruments and the algorithms. The DRWP network and dataset are described in section 2. Section 3 contains the details of the automated and manual QC processes, section 4 discusses the performance of the instruments, and section 5 discusses the performance of the composite algorithm. A summary and conclusions are provided in section 6.
2. 915-MHz profiler network data
The five profilers are arranged in a diamond-like pattern over the KSC/CCAFS area with an average spacing of 10–15 km, as shown in Fig. 1. They are operated in a configuration with one vertical beam and two orthogonal oblique beams. The beam directions at each individual site were chosen to minimize potential interference from ground clutter. A 12-min wind consensus period is followed by a 3-min RASS consensus period. As a result, horizontal and vertical wind profile estimates are generated every 15 min.
Each instrument was configured to make wind estimates beginning at an altitude of 130 m with a gate spacing of 101 m (Radian International 2001). The number of range gates varied among instruments and was changed in some cases during the data collection period. In all cases, the instruments were configured to reach at least 4-km altitude and in some cases the configuration could reach 6 km. Each wind profile usually contained less than the maximum number of possible gates since the instrument would not report an estimate at a gate if the signal-to-noise ratio was too low, or the internal wind-finding algorithm failed to reach a consensus.
Data were provided by the ER Technical Services Contractor, Computer Sciences Raytheon, for November 1999–August 2001, for a total of 647 calendar days. Due to a variety of reasons including downtime scheduled for range modernization, maintenance problems, and administrative issues, data were not available for the full period. Data availability for each individual profiler ranged from as many as 484 days to as few as 279 days. In each 24-h period, the maximum number of profiles is 96 (1 profile per 15 min). The profilers also did not generate a full set of profiles on every day of operation. This occurred if an instrument failed or was taken out of service midday, or placed in service midday. Even with these missing profiles, more than 163 000 individual estimates were available for the study.
3. Quality assessment routines
This section contains a description of the elements of the automated QC process developed for the study, and of the manual QC process used to verify it. The automated QC process has three classes of components: DRWP criteria, single-gate atmospheric criteria, and multiple-gate atmospheric criteria. Within each class there are several tests. In the final dataset, each estimate has a 15-bit QC flag associated with it. Each test was assigned one bit in the flag. If an estimate failed a specific test, the flag bit assigned to that test was set to one. These tests, with one exception described later, are independent and a single data point may fail more than one of them.
There was also a bit assigned to manual QC. That bit was set when appropriate by the person performing the manual QC.
a. DRWP criteria
There are three criteria that are based strictly on the design of the instrument and its signal processing algorithms. They do not take the value of the wind estimate into consideration.
1) Averaging period
This algorithm is required by a system check that will reset a profiler if its computer time is more than 5 s off the time on the central computer located in the Range Operations Control Center. The radial velocities are consensus-averaged over a 12-min period to remove outliers at each range gate. The consensus average is calculated by averaging the values of the largest subset of radial velocity estimates that fall within 2 m s−1 of each other. When a radar is reset during a consensus period all data collected up to the time of the reset is erased. However, the profiler will continue to collect data through the end of the allotted period and calculate a consensus wind. If the reset occurs toward the end of the period, a horizontal wind estimate is calculated from data collected over a very short time as long as the consensus criteria are met. Profiles calculated from these shortened time periods are unreliable. This reset procedure rarely occurs, but it must be checked. The algorithm flags data if its consensus period is less than 6 min.
2) Number of profiles in the consensus
Each wind estimate is derived from the average radial velocity components in the vertical beam and each of the two oblique beams. The radial velocity component in each beam at each range gate is measured up to 10 times during the 12-min period over which a wind profile is generated. At each gate in each beam, the consensus average radial velocity is computed by taking the arithmetic average of the values of the largest subset of radial velocity measurements that fall within a defined interval of each other during the 12-min period. If the number of measurements in the consensus average is less than six in the vertical beam, the vertical consensus QC bit is set. Similarly, if the number of measurements in either oblique consensus is less than six, the oblique consensus bit is set.
3) Signal-to-noise ratio
As with the profile number check, the SNR in the oblique and vertical beams are tested separately. If the values fall below −20 dB, the instrument is not receiving a signal strong enough to reliably measure the Doppler shift and the radial velocity is considered not trustworthy. A separate QC bit is set for violations of the vertical and oblique SNR criterion.
b. Single-gate atmospheric criteria
There are three criteria that use the value of an individual wind estimate to determine if it is erroneous. No consideration is given to adjacent observations in space or time.
1) Wind speed and direction limits
If the wind direction is less than 0 or greater than 360°, the data are clearly erroneous and a “no data” bit is set. If the wind speed is less than 0 or inconsistent with the radar Nyquist velocity, the “no data” bit is set.
2) Vertical speed threshold
The magnitude of the vertical wind in the boundary layer does not exceed 10 m s−1 except in extreme events such as tornadoes and microbursts. In those events, the homogeneity assumptions inherent in the beam geometry of the profilers are violated. Thus vertical wind magnitudes exceeding 10 m s−1 were assumed to indicate an erroneous measurement and a vertical speed flag bit is set.
3) Rain contamination
The rain contamination algorithm developed by LT proved extremely useful and reliable in the study. Since this algorithm is currently described only in a project report, some detail is provided here to make it available to a wider audience. During heavy rain, wind profiling radars may track the falling hydrometeors rather than the air, resulting in large downward velocities in the vertical beam and inward components along the oblique beams. This can lead to large errors in the derived vector winds, especially where the rainfall is inhomogeneous over the sampling volume of the radar (Ralph 1995; Williams et al. 1995; Ralph et al. 1996). Showers occur frequently in central Florida and are a major concern for 915-MHz DRWP quality since the KSC/CCAFS area receives almost 1300 mm (∼50 in.) of rain annually. Ralph (1995) concluded that precipitation produces a relatively unambiguous signature in the vertical velocity. In addition, a stronger return signal can be used as an additional signature to identify precipitation in the profiles. High SNRs are regularly associated with these events because the effective scattering cross section of hydrometeors is larger than that for index of refraction fluctuations in the atmosphere.
c. Multiple-gate atmospheric criteria
These tests compare individual wind estimates with spatially and temporally adjacent estimates. The success of these algorithms depends on the quality of the surrounding data. The results from LT showed that the algorithms must be used in a certain order. The DRWP and single-gate atmospheric criteria algorithms must be used first to eliminate erroneous data. This ensures that the algorithms described below will not use bad data in the temporal or spatial continuity checks.
1) Vertical shear threshold
If the magnitude of the vector difference between adjacent vertical gates exceeds 10 m s−1, then the shear exceeds 0.1 s−1. Shears of that magnitude are usually not encountered except in extreme weather events. Such events violate the radar beam volume homogeneity assumptions necessary for estimating the vertical wind profile. In these cases, either the data are incorrect or the environment is unacceptable and the vertical shear QC bit is set.
2) Small median test
The small median test uses equations adapted from those developed by Carr et al. (1995) and used successfully on an extensive 50-MHz profiler dataset by Merceret (1997). It ensures the temporal and spatial consistency of the u and υ components of the calculated horizontal wind. The test compares the u- and υ-component values of a horizontal wind estimate to the median value of the u and υ components of the surrounding estimates in space and time. If the difference between the observed wind component (ui, υi) and the median of the neighboring observations (um, υm) exceeds a critical threshold (Tu, Tυ), then the whole wind observation (ui and υi) is flagged as erroneous. The values immediately above and below and immediately before and after the target observation were examined. If at least three unflagged values were available, then this set was used for the test. If less than three values were available, the range was expanded to two levels above and below and two profiles before and after. If this did not yield at least three unflagged points for comparison, the isolated datum flag was set and the median test was not run.
3) Isolated datum
If the small median test could not be run because there were not enough unflagged surrounding points, the median test flag was not set since the test did not fail. On the other hand, the test was not satisfied either. In this case, an “isolated datum” bit is set to alert the analyst that the median test was not conducted.
d. Manual QC
The data in each file, comprising up to 96 profiles from a single instrument from 0000 to 2345 UTC for a given day, are displayed simultaneously in two windows as shown in Fig. 3. In both windows at the top of the figure, time (UTC) increases from 0 to 24 h along the horizontal axis and height increases from 0 to 6000 m along the vertical axis. The wind estimates at each gate are presented as color-coded pixels, each color representing a range of values. The legends for these ranges are shown below the first window in Fig. 3. Data that have any of the QC flag bits set are shown in black. There are three choices for the contents of the windows: horizontal wind speed and direction, u and υ components, or horizontal wind speed and w (vertical) component. Clicking on a data point in either of the windows will bring up a text listing of the values at the target point and spatially neighboring points in the same profile. Scroll bars below and to the right of the data windows allow a user to scan the text listing vertically in space or horizontally in time. If any datum appears to be invalid, the operator may click on a button to set the manual QC bit for that measurement.
Before making a final decision whether to manually flag a measurement, data from all of the functioning profilers are examined. Consistency among the profilers is an important element of data quality evaluation. Signal contaminants, except velocity folding and rain, are unlikely to affect more than one profiler the same way at the same time. If a specific wind estimate that appears questionable is consistent with the general wind patterns evidenced by the entire set of data from all of the profilers, it is not flagged.
Since the development of this QC software was prompted by research to study the statistical properties of temporal boundary layer wind changes, it was necessary to create wind-change files containing the vector wind changes between adjacent profiles. These change files also proved to be extremely useful for manual QC of the data. After examining a data file for erroneous or suspect data, the change file can be made by clicking the “Make Change File” control (see Fig. 3). The change file is then examined in the same manner as the original data file. If the change file data suggest that one or more measurements are suspect, the original data file should be reexamined. If any additional estimates require that their QC bits be set, the change file is remade and the process repeated.
The process of examining and manually quality controlling the data in each file, intercomparing the data between profilers for spatial continuity, and then examining the change file for temporal continuity proved extremely robust and surprisingly objective. There was far too much data to examine each file independently, but several dozen files were examined independently by two of the authors. There were no disagreements concerning the quality of any of the data in those files.
4. The performance of the profilers
One raw data file was generated for each profiler each day (UTC) unless the instrument was not operating. Each of these files was processed by the automated QC software, and an entry was generated in a log indicating how many times each QC bit was set in that file. The maximum count for each bit is the number of profiles (maximum 96 profiles per day) multiplied by the number of gates per profile (maximum 44 or 60, depending on profiler settings). The log entries were used to generate cumulative statistics for each profiler for each year in the database in order to determine if there were systematic differences among the instruments or identifiable trends. These statistics are the basis for the assessment of the performance of the profilers. The results showed neither a systematic difference between profilers nor any long-term trends, so the results presented here aggregate the statistics for all years and instruments.
The log file (not the data files) for profiler 3 for the year 2000 was lost, so the aggregate statistics do not contain these data except for the file count. Examination of the data files for profiler 3 for 1999 and 2001 indicated that this unit performed similarly to the others, thus unavailability of this information should affect only the sample size, and not the conclusions.
a. Data availability
Data were supplied for 647 days in the period from November 1999 to August 2001. The number of days in this period on which each profiler was operating ranged from 279 to 484 due to planned and unplanned outages. Data files were obtained for 2032 of the 3235 (647 × 5) possible opportunities. Thus, the average combined availability of the instruments was 63%. This may seem low, but for reasons beyond the scope of this paper, the profilers were not maintained to the level normally applicable to ER systems. Under normal operation and maintenance conditions, the availability of the instruments should be much higher.
A file was generated for each profiler on each day on which any data were taken, even if only a single profile was recorded. There were 1818 data files recorded in the logs (excluding the lost profiler 3 logs). These files contained 162 990 of a possible 174 528 (1818 × 96) profiles yielding an average 93% availability of profiles. Nearly all of the missing profiles were due to profilers being started up or shut down during the day resulting in a partial file. Missing profiles in the midst of active data collection were rare.
These 162 990 profiles contained 8 615 405 gates, where a gate is defined as a potential for a wind estimate at a specific time and height. Only 5 083 744 of these or about 59% of them produced wind data. This was almost entirely due to failure to reach consensus at higher altitudes where the SNR was often too low. This was weather dependent since higher humidities and stronger turbulence usually found in the lower altitudes produce higher SNR than the converse conditions typically found at the higher altitudes.
b. Data quality
Of the 5.08 million gates with a consensus wind, 3.79 million or about 75% passed the automated QC tests, while 25% were flagged. The two major reasons consensus data were flagged by the automated QC algorithm were rain contamination and low SNR.
Thirty-five percent of the 8.6 million gates examined failed the SNR test, but this includes the 3.6 million for which no consensus was reached as well as those for which consensus was reached. The way that the log entries were generated did not permit separating these cases, and the vast majority of the SNR flags set by the automated QC were probably redundant with the “no data” QC bit set for gates in which consensus was not reached. This conclusion is supported by the agreement between the fraction of completely unflagged data and the fraction with no rain flag.
The rain flag is unique to the automated QC and was only applied to data where a consensus wind estimate was obtained. It was set in 1 265 358 (24.9%) of these cases.
The remaining algorithms in the automated QC identified less than 0.5% of the consensus measurements as bad. The shear and median tests each rejected 0.2% of the data and these are correlated, generally identifying the same outlier. These results suggest that when these profilers produce a wind estimate, the estimate is generally correct except in the presence of significant rain.
5. The performance of the automated QC algorithm
During the analysis, the data in every file were quality controlled and examined using the methods described in section 3. The subjective impression of the automated QC performance was very positive, but an objective measure of its performance was also generated.
a. Objective measure
Only 0.03% of the 3.79 million gates that had passed the automated QC were flagged manually excluding data contaminated by radio frequency interference (RFI). RFI contamination is discussed separately in the next section.
b. Radio frequency interference
There are two characteristics of RFI that make it easy to identify: the indicated radial velocity in each beam and at each gate is the same and the magnitude usually exceeds reasonable vertical speeds. During the period from December 2000 through April 2001, the Merritt Island profiler (see Fig. 1) was affected by RFI for a few hours per day on 19 days. In some cases these data were manually flagged. On days where the number of affected data were too numerous for manual flagging of each individual point in space and time because of the labor involved, the entire day for that profiler was discarded and listed as missing. At the time, RFI was not recognized as the cause and the statistics in the log files cannot differentiate these cases. They appear in the statistics as missing files or data with the manual QC bit set. There were no other RFI events in the entire dataset.
A simple RFI test could be incorporated easily into the automated algorithm. This test would first examine the vertical speed. If it exceeded a specified threshold (e.g., 2 m s−1), then the radial velocity (including sign) in each of the three beams would be compared. If all three agreed within a threshold of error (e.g., 0.5 m s−1), then the RFI QC bit would be set.
c. Subjective impression
The automated QC algorithm missed a miniscule percentage of defective data. Initially, it appeared that the rain-contamination algorithm did not flag some rain events and set flags in some cases where there was no rain. One-quarter of the data were identified as rain contaminated, which seemed too high. More careful examination showed that when the rain-contamination algorithm did not flag data during a rain event, the rain was light and uniform enough that the wind measurements were actually valid. Examination of archived weather radar and rain gauge data, and hourly weather observations at the Shuttle Landing Facility near the center of the profiler network confirmed that when the algorithm set the rain QC bit, it was raining.
Examination of the data for outliers or unrealistic values indicated that the median test and the checks on vertical speed and shear were adequate to pick up errors due to sidelobe contamination and false targets like birds and aircraft, except for RFI.
6. Summary and conclusions
There are two main conclusions from this work.
Based on a large sample of data that encompassed both the warm and cool seasons, the ER boundary layer profilers produce high quality estimates when estimates are produced. Maintenance and SNR issues limit data availability but do not compromise data quality.
The automated QC algorithm described here flags essentially all erroneous data from the profiler network. Unflagged data contain less than 1 erroneous estimate in 3000. The vast majority of flagged data are due to rain, and an independent verification indicated that rain was almost always present when data were flagged by the algorithm.
These results suggest that boundary layer profiler network data may be safely and confidently used for the initialization of numerical weather prediction systems and data visualization tools as long as automated QC of the kind described here is applied first.
Acknowledgments
The authors thank Paul Wahner of Computer Sciences Raytheon, the Eastern Range Technical Services Contractor, for supplying the raw data used in this analysis. Mention of a proprietary product or service does not constitute an endorsement thereof by the authors or their employers.
REFERENCES
Carr, F. H., Spencer P. L. , Doswell C. A. III, and Powell J. D. , 1995: A comparison of two objective analysis techniques for profiler time–height data. Mon. Wea. Rev., 123 , 2165–2180.
Heckman, S. T., Maier M. W. , Roeder W. P. , Lorens J. B. , and Boyd B. F. , 1996: The operational use of a boundary layer profiler network at the Eastern Range and Kennedy Space Center. Preprints, 27th Conf. on Radar Meteorology, Vail, CO, Amer. Meteor. Soc., 346–348.
Lambert, W. C., and Taylor G. E. , 1998: Data quality assessment methods for the Eastern Range 915 MHz wind profiler network. NASA Contractor Report CR-1998-207906, Kennedy Space Center, FL, 49 pp. [Available from ENSCO, Inc., 1980 N. Atlantic Ave., Suite 230, Cocoa Beach, FL, 32931.].
Merceret, F. J., 1997: Rapid temporal changes of midtropospheric winds. J. Appl. Meteor., 36 , 1567–1575.
Miller, P. A., Barth M. F. , van de Kamp D. W. , Schlatter T. W. , Weber B. L. , Wuertz D. B. , and Brewster K. A. , 1994: An evolution of two automated quality control methods designed for use with hourly wind profiler data. Ann. Geophys., 12 , 711–724.
Panofsky, H. A., and Brier G. W. , 1958: Some Applications of Statistics to Meteorology. The Pennsylvania State University, 224 pp.
Radian International, 2001: LAP®-3000 operation and maintenance manual. Document Control 80018201, Revision F, 376 pp.
Ralph, F. M., 1995: Using radar-measured radial vertical velocities to distinguish precipitation scattering from clear-air scattering. J. Atmos. Oceanic Technol., 12 , 257–267.
Ralph, F. M., Neiman P. J. , and Ruffieux D. , 1996: Precipitation identification from radar wind profiler spectral moment data: Vertical velocity histograms, velocity variance, and signal power-vertical velocity correlations. J. Atmos. Oceanic Technol., 13 , 545–559.
Weber, B. L., and Wuertz D. B. , 1991: Quality control algorithm for profiler measurements of winds and temperatures. NOAA Tech. Memo. ERL WPL-212, 32 pp.
Williams, C. R., Ecklund W. L. , and Gage K. S. , 1995: Classification of precipitating clouds in the Tropics using 915-MHz wind profilers. J. Atmos. Oceanic Technol., 12 , 996–1012.
Map of the KSC/CCAFS area. The 915-MHz DRWP locations are indicated by a solid circle, and the names of the locations are printed next to the sites. A scale is provided in the upper-right corner
Citation: Journal of Atmospheric and Oceanic Technology 20, 11; 10.1175/1520-0426(2003)020<1488:POFMWP>2.0.CO;2
Plot of consensus SNR (dB) vs consensus vertical velocity (kt) for known rain contaminated and clear-air data on 17 Jun 1997 at the False Cape profiler. This dataset was used in the development of the discriminant function, which is represented by the thick line. The lines at SNR = 0 and VV = 0 are drawn for reference
Citation: Journal of Atmospheric and Oceanic Technology 20, 11; 10.1175/1520-0426(2003)020<1488:POFMWP>2.0.CO;2
Manual quality control screen showing displays and controls. The left and right graphics windows may be selected to display, respectively, u and υ, WS and WD, or WS and w, where u, υ, and w are the standard components of vector wind velocity. The wind speed is WS and WD is wind direction. Each window displays the value of the variable as a function of height and time (UTC) using a color code that is displayed in the legend. The text boxes present quantitative listings of certain quantities for a specific height and time selected by placing the mouse cursor on the appropriate pixel in either graphics window and clicking
Citation: Journal of Atmospheric and Oceanic Technology 20, 11; 10.1175/1520-0426(2003)020<1488:POFMWP>2.0.CO;2
