Implementing Quality Control of High-Frequency Radar Estimates and Application to Gulf Stream Surface Currents
1. Introduction
Land-based high-frequency (HF) radar is commonly used to remotely measure speed and direction of ocean surface currents in the near-coastal environment. Networks of HF radars with operating frequencies of 3–42 MHz are capable of mapping surface currents over ranges of 20–200 km every hour with a horizontal resolution ranging from 0.5 to several kilometers. Since 2006, HF networks have continued to expand their coverage. Over 100 radar sites operated by 30 different institutions contribute radial surface current measurements to the Integrated Ocean Observing System (IOOS) data servers (Harlan et al. 2010; Terrill et al. 2007). As the reliance on HF radars to map ocean surface currents increases (Berkson et al. 2010), as their utility spreads (Paduan and Washburn 2013) and as HF radar networks continue to grow (IOOS 2015a), providing high-quality data while understanding and quantifying the observed variance and standard error is as important as ever.
The main goals of this work are to implement a quality control (QC) method that uses nonvelocity metrics of the quality of signal and solutions in direction-finding (DF) radar systems based on the previous works of Kirincich et al. (2012) and de Paolo et al. (2015) and to apply the QC methodology to DF systems that remotely sense the dynamic and unique coastal setting off the coast of North Carolina (NC), where the Gulf Stream (GS) flows within the HF radar footprint (Fig. 1). The analysis of surface current maps that encompass the GS provides a unique perspective of HF radar systems challenged by the high currents (>2 m s−1) that occur where velocity-based QC methods have not been practical.
(a) Satellite SST daily composite for 1800 UTC 25 Oct 2014 (MARACOOS 2013) of the Gulf Stream flowing northeast along the North Carolina coast. (b) Maximum radial coverage for each radar is shown (light gray patches), along with total vector coverage (darker gray patch) where radials from the three stations overlap. Radar sea surface velocity field (duplicated, but with black and colored arrows, respectively) for the same time and date, combined from multiple radar site stations. North arrow shows the velocity scale for the vector lengths. Also shown are the names and locations of three radar installations (teal circles), the ADCPs (magenta squares), and NDBC stations (red triangles). Thin black contour lines are isobaths (m). The 100-m isobath is thick and approximates the continental shelf break.
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
The GS flows over the upper continental slope and close to the shelf break along the southeastern seaboard of the United States. Its path is easily seen in infrared sea surface temperature (SST) imagery, and its shoreward SST front is typically located between the 100- and 600-m isobaths (Miller 1994). The resolution of microwave SST (~50 km) is generally too coarse for this purpose. The GS is prone to instability, in the form of meanders and accompanying frontal eddies, that perturbs its cross-slope position by tens of kilometers (Bane and Dewar 1988; Lee et al. 1991). Warm filaments can accompany the frontal eddies and intrude onto the shelf (Glenn and Ebbesmeyer 1994a,b). Meander size varies downstream of the Charleston bump. These meanders lead to lateral variations of GS position at a given location with periods of 4–5 days to 2 weeks. Identifying mesoscale, lateral shifts of the GS path has long been a difficult task because of frequency of cloud cover (Miller 1994). HF radar provides an essential tool to observe the rich spatial and temporal variability of the GS path, frontal eddies, and meanders of medium scale, O(10–100) km. More recently, measuring the GS position off Cape Hatteras has become essential for understanding the variability in the marine hydrokinetic energy resource from the stream for possible ocean energy extraction. However, this requires understanding and addressing the issues that govern the quality of HF radar–derived measurements in this complicated environment.
The most commonly used HF radar system is the DF SeaSonde system made by CODAR Ocean Sensors. Each CODAR site produces estimates of the current component either away from or toward the site in a radial pattern by measuring the range, Doppler frequency shift or “Doppler velocity,” and angular direction of the backscattered signal. Where they overlap, radial velocities from two or more sites are used to form total vector currents. More than 90% of the IOOS network sites use the CODAR SeaSonde (IOOS 2015a). One of the desirable features of this sensor is its small antenna footprint, which is ideal for shore locations with minimal space, making it suitable for rugged urban or natural terrains. Proper site selection, setup, and configuration along with antenna calibration and hardware maintenance address most quality assurance issues (COS 2012; Roarty et al. 2010).
The biggest accuracy problem for any HF radar system is the bearing determination (Barrick and Lipa 1997; Hubbard et al. 2013). The CODAR system uses complex signal voltages from three collocated antennas as input into the multiple signal classification (MUSIC) algorithm (Schmidt 1986) to determine bearing or to find the direction of arrival (DOA) of each observed velocity from a given range. With the release of CODAR SeaSonde, version 7, software, radial metric output of the DOA function and signal characteristics, used as quality controls for radial currents independent of estimated velocity, is now provided. Kirincich et al. (2012) demonstrated that reduced scatter and biases of HF radar surface currents compared with in situ current sensors can be achieved by removing estimated velocities with poor characteristics and poor DOA metrics. They also found that using weighted averaging of good velocity estimates based on DOA power within the same range and azimuthal bins further reduced observed biases. This two-step methodology provides a big advantage over velocity-based QC for areas with high horizontal shear in surface current fronts like the GS, where the range of velocities observed over short horizontal distances is very large.
A set of three long-range (5 MHz) CODAR SeaSonde sites overlooking the GS have been installed along the NC coast. Figure 1 shows the location of the three sites along the Outer Banks of NC and the coverage of radial and total vector maps. An SST snapshot (Fig. 1a) shows the warmer waters of the GS as it flows along the coast within the HF radar footprint. The NC CODAR systems offer a depiction of hourly surface current measurements (e.g., Fig. 1) that provide insights into the spatial and temporal variability of the GS. However, a particular issue with the NC installations has been errant speed estimates (Lipa et al. 2006). We have implemented a QC methodology based on recommendations by Kirincich et al. (2012) in an effort to improve surface current estimates.
This paper is arranged as follows. First, the details of the HF radar systems overlooking the GS, and in situ measurements used in the comparisons, are provided. Next, the basics of CODAR SeaSonde data processing are described. This provides context for the quality controlled methods based on using radial metric data (QCD) and helps explain how they were applied within the standard CODAR processing. Description and use of regression analysis and statistical difference plots known as Bland–Altman diagrams follow. Results of data processed using both the QCD and CODAR standard processing are next compared with independent velocity measurements from bottom-mounted acoustic Doppler current profilers (ADCPs). The results are then discussed with a focus on 1) changes after eliminating poor-quality velocities and 2) differences between HF radar data and in situ measurements that are not resolved in the QCD process. Finally, a summary is presented.
2. Data
The NC HF radar network consists of three long-range 5-MHz SeaSonde systems. Figure 1 shows the antenna locations of the three sites from north to south: 1) DUCK at the U.S. Army Corps of Engineers Duck Field Research Facility in Duck, NC; 2) HATY at the National Park Service’s (NPS) Cape Hatteras National Seashore in Buxton, NC; and 3) CORE at the NPS’s Cape Lookout National Seashore at the Great Island fish camp along Core Banks. Both HATY and DUCK were first established in 2003 as part of an Office of Naval Research project (Seim et al. 2003; Shay et al. 2008). CORE was added in 2013 with funding from the state, providing substantial improvement of surface current coverage of the GS. Figure 1b shows the maximum coverage for each NC radar site and coverage where two or more sites overlap. A 4-month record of HF radar data from 1 September 2014 to 31 December 2014 were analyzed in this study. The period was chosen when all three sites were operational with minimal to no downtimes and independent ADCP velocity measurements were available. All radials were processed with an ideal antenna pattern after assuring that appropriate phases were set for each site using the measured phases during the 4-month period (COS 2012). Measured beam patterns were intentionally not used to avoid further complicating the interpretation of the findings, principally because of the uncertainty in defining the appropriate level of smoothing to apply in the measured pattern creation. The impact of use of measured patterns will be assessed in future work.
Collocated within the HF radar footprint were three bottom frames (B1, B2, and CH3), each equipped with upward-looking ADCPs and conductivity–temperature–depth sensors (CTDs) (Fig. 1). Each ADCP measured vertical profiles of current velocity while simultaneous bottom CTD measurements were made every 6 min, subsequently averaged over 1 h. The uppermost bin of current observations with consistent high-quality observations is utilized in this comparison study. B1 and B2 were deployed from June 2014 through early January 2015 on the continental shelf in water depths of 33 and 29 m, respectively. The uppermost ADCP bins ranged from 4 to 5 m below the surface (Table 1). The CH3 mooring, located in approximately 250-m water depth just seaward of the shelf break, has been maintained since August 2013 as part of the North Carolina Renewable Ocean Energy Program. Because of degraded signal strength during long deployments, the larger bin size for the ADCP (4 m), and tilt imposed on the ADCP because the bottom frame was resting on a slope, the uppermost bin with the most complete dataset was at 40 m below the surface (Table 1) for the period corresponding to this analysis (September–December 2014). Prior to 1-h averaging, each ADCP profile sample was subject to several QC tests to ensure good echo intensity, correlation magnitude, and proper surface masking at each bin in the profile following IOOS QC standards (IOOS 2015b). Tables 1 and 2 summarize the sample areas corresponding to each ADCP and HF radar sector used to make the comparisons discussed below.
ADCP sample areas and depths.
HFR radial sample areas at each ADCP location. Degrees bearing (°) are clockwise (CW) from true north (TN).
For the 4-month analysis period, surface wave, meteorological, and in situ density data were available. Measured wave height, direction, and period were obtained from the National Data Buoy Center (NDBC) for buoy 41025 located on Diamond Shoals and wave buoy 44095 (Fig. 1), the latter owned and operated by the University of North Carolina (UNC) Coastal Studies Institute (CSI). Wind measurements were obtained from NDBC 41025 until 8 December 2014 when a nor’easter destroyed that station’s anemometers and communication. Near-surface CTD data collected at B1 and B2 combined with bottom frame CTDs provided density stratification at these two shelf locations.
3. CODAR SeaSonde processing and quality control
The standard processing of current velocities for the CODAR long-range 5-MHz SeaSonde system consists of four main steps. Averaging and QC tests are applied at several of these steps within the system software. The following set of processing steps previously documented by Lipa et al. (2006) and the standard QC procedures (Table 3) are collectively referred to as CODAR standard processing.
Summary table of QC tests for standard CODAR processing used with NC HFR sites.
The complex signal voltages of the backscattered HF radio wave received by the three collocated antennas are combined into cross spectra representing reflected power at each range (5.8 km for 5 MHz) and Doppler frequency. One-hour ensemble averages of several cross spectra are produced every 30 min. The Doppler frequency shifts identified within first-order limits surrounding the ideal Bragg frequency can be translated directly to radial velocity estimates or “Doppler velocities.” A radial velocity is rejected if QC tests fail for 1) maximum velocity, 2) minimum signal-to-noise ratio (SNR), and/or 3) ionospheric noise check. Table 3 lists the threshold settings for these CODAR QC checks optimized for the 5-MHz NC GS sites. Note the maximum velocity test requires a setting of 2.5 m s−1 in order to retain valid data.
The left side of Fig. 2 (CODAR) shows the four main processing steps from cross spectra to hourly maps of surface current speed and direction (steps 1–4). Every 30 min, bearing information is derived for each identified Doppler velocity at each range, by using complex signal voltages from each of three antenna elements input into the MUSIC algorithm (step 1). Paraphrasing de Paolo and Terrill (2007a), the MUSIC algorithm determines the bearing(s) of the signal by comparing the received antenna voltages to expected antenna voltages, the latter defining by an antenna manifold. The selected bearing(s) best match(es) the antenna manifold. As outlined by Lipa et al. (2006), a maximum of two bearing estimates allowed by the three-element antenna are tested for validity based on the MUSIC algorithm output. If the dual-angle solution is rejected, the single-angle solution is accepted (Table 3). Also, a bearing solution is removed if it falls outside the angular limits for the site. A DOA function is used to characterize consistency of the measured signal with the antenna manifold and exhibits peaks in response at best-fit bearings. See de Paolo and Terrill (2007a,b) for more complete descriptions.
Left side of the flow diagram shows the major processing steps in CODAR SeaSonde HF radar systems to derive vector surface currents from backscatter. Right side shows quality controlled (QCD) processing steps where quality control (QC) tests and 2D weighted averaging are applied within the major steps. Black arrows indicate steps common to both processes. Spatial and temporal resolution of each output step is noted. Adapted from Kirincich, et al. (2012).
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
Single- or dual-angle bearings, signal characteristics, and details of the DOA calculation are made available in the radial metric output along with the velocity and range of each estimate as of SeaSonde, version 7. The specific outputs from the MUSIC algorithm include the maximum value (dB) of the peak response (MR), the width of the MUSIC DOA function (MW) 3 dB down from the MR, the MUSIC signal power (dB; MP), the SNR for each antenna, the MUSIC eigenvalues, and values used in the CODAR dual-angle rejection QC tests. These parameters are the nonvelocity metrics used to implement QC tests by Kirincich et al. (2012), described below.
At the same time interval (30 min), all Doppler velocities falling within a 1° bearing in the same range bin are then averaged (step 2), regardless of any DOA characteristics, and made available by SeaSonde software as “short-term radials” in SeaSonde terminology. Every hour, the short-term radial velocity estimates in a given range bin within a 5° angle and from a sequence of five 30-min (over 150 min) short-term radial maps are spatially and temporally averaged to produce a radial surface current map (step 3). Finally, every hour, radial currents from two or more HF radar sites within a specified distance to predefined Cartesian grid points are combined to form orthogonal vector velocity estimates or “total vectors” in SeaSonde terminology (step 4). We use this same terminology to distinguish between “radial” and “total” vector currents.
4. QCD radials
The QCD methodology is applied early within the CODAR processing steps just after the direction of each Doppler velocity is determined by MUSIC but before any averaging is performed (step 2, Fig. 2). This methodology is based on the work of Kirincich et al. (2012) and de Paolo et al. (2015), which uses the nonvelocity parameters provided in the radial metric output. Kirincich et al. (2012) uses a two-step approach: 1) eliminate radial velocities that have poor metrics of signal characteristics and/or poor solutions of the DOA function and 2) weight the “good” velocities by the strength of the DOA metrics found within a given range and bearing before averaging. The QCD methodology follows this two-step approach with one notable expansion to increase the number of Doppler velocities available for the weighted average and hence the number of degrees of freedom. At each time, range, and bearing, a two-dimensional (2D) filter is added that includes good velocity estimates from adjacent bearings (1°) and 30-min time samples in the weighted average. It also requires a minimum number of estimates to produce an output, which was found to help reject outliers that may otherwise have gone undetected. The output from the 2D weighted average is CODAR short-term radial data that fully integrate with CODAR utilities and downstream processing. Subsequent processing spatially (over 5°) and temporally (over 150 min) averages to form hourly radial data and combines these to form total vectors. The right-hand side of Fig. 2 depicts how the QCD two-step (steps 2a and 2b) procedure fits within the CODAR schema.
Thresholds for QC tests were established after examining distributions of the radial metric output for NC 5-MHz SeaSonde systems. The thresholds for QC tests used in this study are the same as those found to be effective by Kirincich et al. (2012) with medium-range 25-MHz SeaSonde systems. It is encouraging and somewhat surprising that the same thresholds apply to a radar system operating at a different frequency and observing a very different flow field. The QC thresholds of MR less than 5 dB or an MW greater than 50° (Table 4) were used to eliminate radial velocities in the raw radial metric output. Both Kirincich et al. (2012) and de Paolo et al. (2015) showed that these nonvelocity QC tests alone improved bias and reduced noise. However, even greater improvement came by applying a weighted function to the raw radial data passing the abovementioned threshold tests. They found that while SNR is an effective weighting parameter, MP had the largest impact on the radial averages. In this study, the MP parameter was found to produce superior results and was used as the weighting parameter. The size of the 2D filter and the minimum number of points for the MP weighted average are provided in Table 5, along with all QCD settings for the NC radar sites. A graphical user interface (GUI) was built to evaluate and visualize the effect of changing each of the parameters listed in Table 4. The settings listed in Table 5 were optimized by assessing several radial metric files from each site using this GUI. The best results and thresholds for QCD tests were visually confirmed.
Settings for QCD thresholds used with NC HFR sites.
Settings for QCD weighted averages used with NC HFR sites.
5. HF radar and ADCP comparison methods
This section describes the procedures used to evaluate both the CODAR and QCD methodologies with independent velocity measurements. Comparisons of both CODAR and QCD radial and total output were made with the concurrent bottom-mounted in situ ADCP measurements at the B1, B2, and CH3 locations.
a. Radial current measurements
Statistical comparisons were made using both ordinary least squares regression and newer Bland–Altman analyses (Bland and Altman 1999) to summarize the effect of QCD methods on the 4-monthlong dataset.
For the HF radar–ADCP radial comparison, the mean radial velocity (
b. Bland–Altman diagrams and analysis
The agreement between two measurements of radial velocity for each of the two different means, 
The Bland–Altman diagram shows how large the differences are and whether they vary systematically over the range of average values. In addition, any variation of the differences with increasing magnitude of the values is easily identified. Following Bland and Altman (1999), the limits of agreement (LoA) can be computed, provided that the differences are constant or vary linearly over the range of averages and that the variability of the differences remains the same over all the averages. LoA represent the bounds where most (95%) of the differences between two methods occur assuming a normal distribution, and this can be shown visually on the Bland–Altman plot.
Following this procedure the differences in the velocities between the two measurements each hour, 
Quantitative comparisons of surface current radial velocities from standard processing (CODAR) with ADCPs (green) and quality controlled processing with ADCPs (blue) for the entire 4 months of the study period. Comparison of each radial site–ADCP using Bland–Altman diagrams is shown for (a) CORE–B2, (b) HATY–B1, (c) DUCK–B1, (e) HATY–CH3, (f) CORE–CH3. Solid lines denote the regression line of radial differences (
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
Comparison statistics of HFR to ADCP hourly radial currents for the entire 4 months for the CODAR and QCD processing methods.
c. Total current measurements
For comparing the effect of QCD methodology on HF radar total vector output, radial data from the three sites were combined using the SeaSonde Combine Suite software (version 7) for both CODAR- and QCD-processed radials. The area where the three radial grids overlap and an example map are shown in Fig. 1b. Time series of horizontal velocity components from HF radar total output, 
For total vector comparisons, the slope (a1) and y intercept (a0) of the regression line and the RMSD values between east–west and north–south horizontal velocity components, 
Comparison statistics of HFR to ADCP orthogonal currents for the entire 4 months for the CODAR and QCD processing methods.
As in Fig. 3, but for comparison of 
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
d. Bearing offset
Radial bearing offset between CODAR and QCD processing (Fig. 5) was examined following Emery et al. (2004). The r2, RMSD, and Bland–Altman sR were evaluated between 
(a)–(e) Comparison of radial velocities as a function of bearing for the entire 4 months of the study period. Bearing offset (
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
6. Results
The application of the QCD methodology to the NC HF radar sites resulted in reduced variability and increased correlation in the comparisons with ADCP velocity measurements. However, the magnitude of the change differs substantially between the shelf ADCPs (B1 and B2) and the shelfbreak ADCP (CH3). The change also varies spatially over the NC HF radar footprint, often being greater where current velocities are largest. The following subsections describe the results in more detail for each of these three cases.
a. Shelf moorings
Shelf radial velocity observations are illustrated in Figs. 6c,d. Tidal currents dominate short-term variability with synoptic wind-forced events modulating longer-term variability in all three records—from the ADCP and the CODAR and QCD velocities. Velocities are at times highly correlated, essentially overlying each other, but there are also occasions when the ADCP velocities are offset from the radar-derived velocities. There are 10%–20% fewer QCD velocities (Table 6) and they exhibit less scatter than the CODAR velocities.
September 2014 time series comparing the HFR–ADCP radial velocities with ancillary data. (a) Wind speed and direction (north upward), and wind gust from NDBC 41025. (b) Wave height and direction (toward, north upward) from NDBC 44095 in situ density difference (bottom near surface). (c) HF radar radials (
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
Statistics of the variability between CODAR and QCD in radial comparisons with ADCPs located on the shelf are close in range for CORE–B2, HATY–B1, and DUCK–B1 (Table 6). The effect of applying quality controls at these locations decreases sR and RMSD by 0.01–0.03 m s−1. The lowest values of sR and RMSD occur for the pairing at the smallest range, CORE–B2. The reduction in RMSD is the same found by Kirincich et al. (2012). Note the RMSD values are slightly more than sR values but with a similar decrease between CODAR and QCD processing. Finally, overall mean radial velocity (
The linear regression of radial difference (
Similar reductions in variability by applying QCD are observed when comparing total velocities at B1. The total velocities visually compare well, all displaying similar tidal and lower-frequency variability (Fig. 6e). Both sR and RMSD are lowered by 0.01–0.02 m s−1 (Table 7). The TOT-B1 Bland–Altman slopes (b1) between CODAR and QCD were close with −0.21 and −0.24, respectively, for the east velocity component over a narrower range of velocities and 0.06 and 0.13, respectively for the north velocity component over a larger range (Figs. 4a,b). This suggests a tendency to overestimate northward (alongshelf) flow and underestimate eastward (cross shelf) flow. Likewise, both a1 and r2 increased as a result of applying QCD. The value of a1 for the 
The radial bearing analysis shows 
While the application of QCD reduces departures of HF radar from ADCP measurements, there are differences associated with geophysical processes that contribute to differences between the radar-observed surface currents and ADCP-observed near-surface currents. Increased wind speeds and wave heights in the latter half of September, when the water column was still stratified, correlate with the larger differences that occur during the month (Fig. 6). The southward winds during northward currents are consistent with the sense of shear seen between HF radar (both CODAR and QCD) and ADCP observations, suggesting the importance of shear in the surface Ekman layer, as observed in other studies (e.g., Graber et al. 1997; Emery et al. 2004).
b. Shelfbreak mooring
Shelfbreak radial currents (Figs. 7b–d) are qualitatively different from currents on the shelf. Flow is largely unidirectional and subject to subtidal variability. Tidal currents there make a minor contribution to the variability. QCD data scatter is reduced compared to CODAR data; the latter is especially noisy for CORE, which at 126 km is the most distant station used for comparison (Fig. 7c). Radar data are at times consistent with the ADCP but are often significantly smaller in magnitude. The bearing in the HATY–CH3 comparison is at times perpendicular to the GS currents and CORE–CH3 comparison is more often parallel, and this explains the large difference between the magnitudes of the currents. In addition to reducing noise, QCD increases the magnitude of many of the hourly estimates relative to CODAR (e.g., Fig. 7c) in the shelfbreak comparison.
(a) As in Fig. 6a, but for wave height and direction from NDBC 41025. (b)–(d) As in Figs. 6c–e, but comparing radials from (b) HATY with CH3, (c) CORE with CH3, and (d) HFR totals with CH3. Also, note velocity scale change from Figs. 6c–e.
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
Statistics of CODAR and QCD radial comparisons with the shelfbreak ADCP (CH3) vary strongly with range. Considering the CODAR data, the sR value for HATY–CH3, separated by 40 km, is 0.17 m s−1, whereas the value for CORE–CH3, separated by 126 km, is 0.57 m s−1, both greater than for the shelf moorings (Table 6). The sR values are 0.15 and 0.46 m s−1 for HATY–CH3 and CORE–CH3, respectively, for comparisons of QCD data, a reduction in sR of 0.02 and 0.11 m s−1 between CODAR and QCD data. Similar results for RMSD are observed, with 0.19 (HATY–CH3) and 0.62 m s−1 (CORE–CH3) from CODAR data being decreased to 0.17 and 0.47 m s−1, respectively, by QCD.
Application of QCD to the CORE–CH3 data noticeably increased the overall mean radial velocity (
Another difference in radial currents between shelfbreak and shelf moorings is seen in the slope (b1) of Bland–Altman plots. Figures 3e,f show negative slopes (b1) of −0.16 to −0.24 (see Table 6) over a very large ranges of velocity (−0.5 to +2.0 m s−1), indicating that subsurface ADCP currents (at 40-m depth at CH3) strongly exceed the surface currents, especially when the average is more than 1.5 m s−1. The values of a1 and r2 for the traditional regression analysis are increased by applying QCD but fail to rise to the values seen on the shelf in the CORE–CH3 comparison.
Total currents at CH3 are clearly of a different character than those on the shelf (Fig. 7). Unlike the shelf, it is not obvious that the CODAR or QCD data capture the subtidal variability in the ADCP data at CH3. Reductions in variability as a result of applying QCD are almost an order of magnitude larger than those seen at the shelf moorings. Both sR and RMSD are lowered by 0.04–0.12 m s−1, with sR values from 0.23 to 0.51 m s−1 and RMSD values from 0.28 to 0.67 m s−1, while sR values for north velocity are roughly twice those for eastward velocity (Table 7). The TOT-CH3 Bland–Altman slopes (b1) between CODAR and QCD were similar but large and negative, being −0.43 and −0.43, respectively, for the eastward velocity component and −0.44 and −0.45, respectively, for the northward velocity (Figs. 4d,e). As with the radials, the total comparisons result in both a1 and r2 increases when QCD is applied, but the correlation remains poor. Finally, the TOT-CH3 phase angle (α) between VHF and VADCP for complex vector correlation is only slightly rotated from −22.2° (CODAR) to −23.5° (QCD).
The radial bearing analysis shows 
The cause of periods of obvious offset between the ADCP and CODAR or QCD radial velocities (Figs. 7b,c) at the shelf break is unclear. There, the ADCP subsurface currents exceed HF radar surface currents. Periods in September 2014 of the largest offset in the CORE–CH3 comparison (Fig. 7) do not align consistently with specific periods of winds or waves; thus, the physical mechanism seen on the shelf apparently does not apply. Radar velocities do not exceed 1.5 m s−1, which could represent a threshold within the processing that has not yet been identified or understood.
c. Spatial variation
Differences between QCD and CODAR velocities are not spatially uniform (Fig. 8). Greater differences occur where current velocities are large, most notably along the GS path. The largest differences are seen at range limits and/or angular limits for a given radial site when these lie along the GS path. As with the shelfbreak (CORE–CH3) comparison, the QCD velocity estimates are increased due to elimination of low velocities that had poor radial metrics. When average speeds exceed 0.2 m s−1, the QCD velocity estimates are 15%–20% higher than the CODAR velocity estimates (Figs. 8e–g).
Maps of point-by-point difference (QCD − CODAR) of mean values for September 2014 processed using CODAR standard processing and QCD processing for radials at (a) HATY, (b) DUCK, and (c) CORE, and (d) the resulting combined totals. (a)–(d) Magnitude of each vector is represented on the color scale (c). Bold indicates the 100-m isobath. (e)–(h) Corresponding scatterplots of the velocity ratio (QCD/CODAR) vs average velocity (QCD + CODAR/2). Average ratio (blue horizontal line) where the average velocity >0.20 m s−1 (black vertical line). Average ratio represents the relative increase of QCD mean velocities over CODAR mean velocities.
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
Maps of the point-by-point difference between mean radial and total vectors obtained using CODAR and QCD are presented in Figs. 8a–d. For HATY and CORE, large differences (>0.15 m s−1) in the mean flow field are seen along radial bearings that parallel GS currents, while smaller differences (<0.1 m s−1) are confined to the shelf, offshore from the GS, and along radials perpendicular to GS currents. Along the bearings that are close to paralleling the GS, the differences increase to over 0.3 m s−1 at large ranges (120–180 km), where the quality of Doppler signals can degrade and surface areas of individual sectors increase. For DUCK, differences of ~0.2 m s−1 occur in the GS at large ranges (100–200 km) off the shelf and nearshore parallel to the coastline. The largest radial velocity increases between QCD and CODAR (>0.3–0.4 m s−1) are seen with DUCK and CORE along radials at ranges influenced by the GS and where radial estimates can be effected by headlands (e.g., Figs. 8b,c) at the site’s angular limit.The impact of QCD when totals are formed clearly illustrates that speed changes are greatest in the GS (Fig. 8d). The impact may seem greater for the totals than for the radials, but this is due to the denser Cartesian grid and interpolation in formation of the total vectors (Lipa et al. 2006). The largest total vector differences between QCD and CODAR occur spatially where each of the radial sites shows an increase. The difference varies spatially along the length of the GS offshore of NC and results from the tendency of radial quality to be poorest at the extremes of azimuthal coverage. Mean total vector maps for CODAR and QCD (Figs. 9a,b) also show this change. Importantly, the QCD depiction of the mean currents removes a sudden transition in speed seen in the CODAR data (Fig. 9a) where coverage changes from CORE to DUCK (see Figs. 1, 8b,d). The consistency of the flow field in the QCD data suggests that the quality control procedures produce uniform quality and bias across the coverage area, as evidenced by removal of the transition at a coverage boundary. In particular, implementing QCD improves the identification of the GS path. Radar GS edge detection methods implemented by Muglia et al. (2015) rely on identifying maxima in radial velocity differences, and relative vorticity maxima in the total currents apparent at the landward edge of the GS. The enhanced radial and total current velocities produced by the QCD processing within the GS coverage region improves these edge detection algorithms by making surface current velocity differences and relative vorticities at the edge of the GS appear larger and thus more pronounced relative to those on the shelf.
Map of mean combined total vectors for September 2014 processed using (a) CODAR and (b) QCD. These data form the differences (QCD − CODAR) plotted in Fig. 8d. Bold indicates the 100-m isobath. Color scale for speed is the same for both maps.
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
To quantify the relative changes in magnitudes of the current, the average of the velocity ratio (QCD/CODAR) for each grid point in each map is plotted against the average of the two measures in Figs. 8e–h. The average ratio when average velocity exceeds 0.2 m s−1 shows QCD increases the velocity radial estimates by 15% (HATY), 15% (DUCK), and 22% (CORE). Further, the total estimates increase by 19% for the month of September 2014 (Figs. 8e–h). Below 0.2 m s−1 the ratio is more variable, reflecting shelf and offshore locations away from the GS, where application of QCD does not appear to impact mean speeds.
7. Discussion
QCD data differ from the CODAR data in several ways: in the abovementioned comparisons, variability and scatter are reduced; in all radial and total comparisons, the QCD processing relative to CODAR processing increased r2 in scatter regression, and sR is reduced by approximately 20% and RMSD by 10%–20%, quite similar to that seen by Kirincich et al. (2012). QCD data are better correlated with independent observations, suggesting the method leads to a true reduction in noise, as previously suggested by Kirincich et al. (2012). It is encouraging that with little modification the methodology suggested by Kirincich et al. (2012) worked equally well when applied to 5-MHz long-range observations. The one major change implemented in this study includes good neighboring Doppler velocities in time and space when performing the MP weighted average. This provides improved coverage and a higher level of statistical significance, despite a 2%–19% reduction in the number of solutions (Table 6).
More importantly, QCD was also found to noticeably increase mean velocity estimates of surface currents where the GS influences the surface flow field, suggesting it corrects a significant bias. At CORE–CH3, QCD increased the long-term arithmetic mean by 0.17 m s−1, and the Bland–Altman plots suggest removal of a ~0.25 m s−1 bias. Maps of monthly mean difference display an increase of 15%–22% where velocity estimates fall within the GS and suggest that the bias correction is limited to high-velocity areas. Maps of differences in radial velocity also show that the largest increases (0.3–0.4 m s−1) occur at the coverage limits, in both range and angular extents. The total vectors formed by combining QCD radials from the three sites also reflect these higher average estimates, reduced variability, and a more consistent flow field.
An important issue is to understand why the QCD methodology has the most impact in the GS. One important factor is that the GS lies within the outer ranges of the radar footprint. In addition to increased surface area for each sector as range increases, the strength of the returned HF signal falls off with range and DOA solutions often degrade due to ionospheric interference with increasing range. The thresholding and weighted averaging based on radial metrics implemented in this study have a greater impact at larger range and along the GS path. One possible explanation is that the poor-quality solutions being rejected by QCD are random noise with near-zero mean. If true, a difference between the distribution of QCD and CODAR velocities would be expected. The difference of the distributions should reflect the distribution of the rejected information; however, because of the various averaging steps in the process, there is not a simple way to form the distribution of the rejected data in a way that is comparable to the distribution of the velocities that are accepted. A signature of the effect should persist in the processed data and is explored next.
Radial velocity distributions for the 4-month period, including the ADCP velocity distributions, were formed to assess potential shifts. For the shelf sites, there is little difference among the three distributions. The QCD distribution tends to have fewer points near its mode, and since the mode is close to zero, this is consistent with the hypothesis of random noise being removed. The ADCP distribution for HATY–CH3 has a roughly 50% greater standard deviation than on the shelf and a median value of 0.33 m s−1. The QCD and CODAR distributions are similar but more peaked about the median with less spread. The standard deviation of the ADCP distribution at CORE–CH3 is more than twice those on the shelf and includes speeds greater than 2 m s−1. The QCD distribution for this comparison is shifted to more positive values than the CODAR distribution, and more like the ADCP distribution for speeds less than 1 m s−1; above this speed neither radar distribution captures the highest values. Assuming a correlation scale of one day, 95% confidence limits of the medians were estimated to assess significant differences in average values (Fig. 10). Only for the CORE–CH3 site is the change in the median of the QCD data significant (an increase of 0.17 m s−1 relative to CODAR). This change supports the hypothesis that the QC process eliminates noise with a zero mean, but because this is true only for the CORE–CH3 comparison, it suggests it is a major impact only for ranges greater than 70 km.
Median radial velocities with 95% standard errors for the 4-month study period: 
Citation: Journal of Atmospheric and Oceanic Technology 34, 6; 10.1175/JTECH-D-16-0203.1
For the shelf moorings, the impact of QCD is largely a noise reduction rather than a bias correction. The same is true for HATY–CH3, where a large mean exists, suggesting random noise is not a dominant process. However, for CORE–CH3, located in the GS and at the edge of range coverage, removing the zero-mean noise shifts the distribution to higher values that are more consistent with the observations. Noise is a bigger issue at CORE–CH3 due to distance from the radar site, and clearly there is a vast discrepancy in sampling volume (cf. Tables 1 and 2). Unfortunately, even after applying QCD there is still an inability to measure the highest current speeds, suggesting another limitation of the radar velocity estimates that has yet to be identified.
8. Summary
In this study, the nonvelocity-based QC methodology was applied to CODAR SeaSonde DF systems that remotely sense the highly dynamic, high-energy environment and strong currents of the GS off the coast of North Carolina. This methodology uses metrics of quality of signal and DOA solutions. This study evaluated how the QC methodology changes the radial and total surface current measurements in comparison with ADCP subsurface current measurements. The methodology reduces variability by 0.02–0.03 m s−1 on the shelf and up to (and sometimes over) 0.10 m s−1 at the shelf break. Bearing offsets do not seem to be greatly affected by the QC steps. The impact of QC when totals are formed clearly illustrates that speed increases of 15%–20% are greatest in the GS and have the greatest impact at site angular and range limits.
Differences between HF radar–measured surface currents and ADCP-measured subsurface currents on the shelf are attributed to natural variation in differences between the surface and subsurface. Despite removing ~0.25 m s−1 bias at ranges greater than 70 km and along the GS path, differences are not well understood at the shelf break and in proximity to the GS. The Bland–Altman plots were found to be useful in assessing trends as a function of average speed and quantifying these differences.
Acknowledgments
We thank SECOORA for ongoing operational support, the North Carolina Renewable Ocean Energy Program for funding the CORE site, and UNC IMS staff, T. Whipple and R. Neve, for installing and maintaining the CORE site. Prof. John Bane provided valuable input. The Office of Naval Research Grant N00014-02-1-0972 funded the initial acquisition and operation of the radars (SEACOOS Project).
REFERENCES
Armbrecht, L. H., S. W. Wright, P. Petocz, and L. K. Armand, 2015: A new approach to testing the agreement of two phytoplankton quantification techniques: Microscopy and CHEMTAX. Limnol. Oceanogr. Methods, 13, 425–437, doi:10.1002/lom3.10037.
Bane, J. J. M., and W. K. Dewar, 1988: Gulf Stream bimodality and variability downstream of the Charleston bump. J. Geophys. Res., 9, 6695–6710, doi:10.1029/JC093iC06p06695.
Barrick, D. E., and B. J. Lipa, 1997: Evolution of bearing determination in HF current mapping radars. Oceanography, 10 (2), 72–75, doi:10.5670/oceanog.1997.27.
Berkson, J. M., A. Allen, D. Murphy, and K. Boda, 2010: Integrated Ocean Observing System (IOOS) supports marine operations: A look at the U.S. Coast Guard. Mar. Technol. Soc. J., 44, 156–165, doi:10.4031/MTSJ.44.6.22.
Bland, J. M., and D. G. Altman, 1999: Measuring agreement in method comparison studies. Stat. Methods Med. Res., 8, 135–160, doi:10.1191/096228099673819272.
COS, 2012: SeaSonde Radial Suite software documentation, release 7. CODAR Ocean Systems, 1 pp. [Available online at http://www.codar.com/images/news/press/CODAR_News_03_2012_4.pdf.]
de Paolo, T., and E. Terrill, 2007a: Skill assessment of resolving ocean surface current structure using compact-antenna-style HF radar and the MUSIC direction-finding algorithm. J. Atmos. Oceanic Technol., 24, 1277–1300, doi:10.1175/JTECH2040.1.
de Paolo, T., and E. Terrill, 2007b: Properties of HF radar compact antenna arrays and their effect on the MUSIC algorithm. Scripps Institution of Oceanography Tech Rep. 71, 41 pp. [Available online at http://escholarship.org/uc/item/5bw303tj.]
de Paolo, T., E. Terrill, and A. Kirincich, 2015: Improving SeaSonde radial velocity accuracy and variance using radial metrics. Proc. OCEANS 2015, Genoa, Italy, MTS and IEEE, 9 pp., doi:10.1109/OCEANS-Genova.2015.7271360.
Emery, B., L. Washburn, and J. Harlan, 2004: Evaluating radial current measurements from CODAR high-frequency radars with moored current meters. J. Atmos. Oceanic Technol., 21, 1259–1271, doi:10.1175/1520-0426(2004)021<1259:ERCMFC>2.0.CO;2.
Glenn, S. M., and C. C. Ebbesmeyer, 1994a: The structure and propagation of a Gulf Stream frontal eddy along the North Carolina shelf break. J. Geophys. Res., 99, 5029–5046, doi:10.1029/93JC02786.
Glenn, S. M., and C. C. Ebbesmeyer, 1994b: Observations of Gulf Stream frontal eddies in the vicinity of Cape Hatteras. J. Geophys. Res., 99, 5047–5055, doi:10.1029/93JC02787.
Graber, H. C., B. K. Haus, R. D. Chapman, and L. K. Shay, 1997: HF radar comparisons with moored estimates of current speed and direction: Expected differences and implications. J. Geophys. Res., 102, 18 749–18 766, doi:10.1029/97JC01190.
Harlan, J., E. Terrill, L. Hazard, C. Keen, D. Barrick, C. Whelan, S. Howden, and J. Kohut, 2010: The Integrated Ocean Observing System high-frequency radar network: Status and local, regional, and national applications. Mar. Technol. Soc. J., 44, 122–132, doi:10.4031/MTSJ.44.6.6.
Hubbard, M., D. Barrick, N. Garfield, J. Pettigrew, C. Ohlmann, and M. Gough, 2013: A new method for estimating high-frequency radar error using data from Central San Francisco Bay. Ocean Sci. J., 48, 105–116, doi:10.1007/s12601-013-0009-y.
IOOS, 2015a: A plan to meet the nation’s needs for surface current mapping. Accessed 21 April 2016, 62 pp. [Available online at https://ioos.noaa.gov/wp-content/uploads/2015/09/national_surface_current_plan.pdf.]
IOOS, 2015b: Manual for real-time quality controls of in-situ current observations: A guide to quality control and quality assurance of acoustic Doppler current profiler observations. Version 2.0, accessed 18 July 2016, 51 pp. [Available online at https://www.ioos.noaa.gov/wp-content/uploads/2016/04/qartod_currents_manual.pdf.]
Kirincich, A. R., T. de Paolo, and E. Terrill, 2012: Improving HF radar estimates of surface currents using signal quality metrics, with application to the MVCO high-resolution radar system. J. Atmos. Oceanic Technol., 29, 1377–1390, doi:10.1175/JTECH-D-11-00160.1.
Kundu, P. K., 1976: Ekman veering observed near the ocean bottom. J. Phys. Oceanogr., 6, 238–242, doi:10.1175/1520-0485(1976)006<0238:EVONTO>2.0.CO;2.
Lee, T. N., J. A. Yoder, and L. P. Atkinson, 1991: Gulf Stream frontal eddy influence on productivity of the southeast U.S. continental shelf. J. Geophys. Res., 96, 22 191–22 205, doi:10.1029/91JC02450.
Lipa, B., B. Nyden, D. S. Ullman, and E. Terrill, 2006: SeaSonde radial velocities: Derivation and internal consistency. IEEE J. Oceanic Eng., 31, 850–860, doi:10.1109/JOE.2006.886104.
Liu, Y., R. Weisberg, C. Merz, S. Lichtenwalner, and G. Kirkpatrick, 2010: HF radar performance in a low-energy environment: CODAR SeaSonde experience on the West Florida Shelf. J. Atmos. Oceanic Technol., 27, 1689–1710, doi:10.1175/2010JTECHO720.1.
MARACOOS, 2013: AVHRR sea surface temperature for MARACOOS. Mid-Atlantic Coastal Ocean Observing System. Subset used: Daily composite for 1753 UTC 25 October 2014, accessed 14 April 2016. [Available online at http://tds.maracoos.org/thredds/catalog.html.]
Miller, J. L., 1994: Fluctuations of Gulf Stream frontal position between Cape Hatteras and the Straits of Florida. J. Geophys. Res., 99, 5057–5064, doi:10.1029/93JC03484.
Muglia, M., H. Seim, and S. Haines, 2015: Measuring the landward Gulf Stream front variability off Cape Hatteras with HF radar. 2015 IEEE/OES Eleventh Current, Waves and Turbulence Measurement (CWTM 2015), IEEE, 337–340, doi:10.1109/CWTM.2015.7098152.
Paduan, J. D., and L. Washburn, 2013: High-frequency radar observations of ocean surface currents. Annu. Rev. Mar. Sci., 5, 115–136, doi:10.1146/annurev-marine-121211-172315.
Paduan, J. D., K. Kim, M. Cook, and F. Chavez, 2006: Calibration and validation of direction-finding high-frequency radar ocean surface current observations. IEEE J. Oceanic Eng., 31, 862–875, doi:10.1109/JOE.2006.886195.
Roarty, H., and Coauthors, 2010: Operation and application of a regional high-frequency radar network in the mid-Atlantic Bight. Mar. Technol. Soc. J., 44, 133–145, doi:10.4031/MTSJ.44.6.5.
Schmidt, R., 1986: Multiple emitter location and signal parameter estimation. IEEE Trans. Antennas Propag., 34, 276–280, doi:10.1109/TAP.1986.1143830.
Seim, H., and Coauthors, 2003: SEA-COOS: A model for a multi-state, multi-institutional regional observation system. Mar. Technol. Soc. J., 37, 92–101, doi:10.4031/002533203787537249.
Shay, L. K., H. Seim, D. Savidge, R. Styles, and R. H. Weisberg, 2008: High frequency radar observing systems in SEACOOS. Mar. Technol. Soc. J., 42, 55–67, doi:10.4031/002533208786842435.
Terrill, E. J., M. Orteo, and L. Hazard, 2007: Mapping surface currents around U.S. coasts: A network of high-frequency radar for the Integrated Ocean Observing System. Sea Technol., 48, 29–33.
Voulgaris, G., N. Kumar, K.-W. Gurgel, J. C. Warner, and J. H. List, 2011: 2-D inner-shelf current observations from a single VHF WEllen RAdar (WERA) station. 2011 IEEE/OES 10th Current, Waves and Turbulence Measurements (CWTM), IEEE, 57–65, doi:10.1109/CWTM.2011.5759525.
Zaki, R., A. Bulgiba, R. Ismail, and N. A. Ismail, 2012: Statistical methods used to test for agreement of medical instruments measuring continuous variables in method comparison studies: A systematic review. PLoS One, 7, e37908, doi:10.1371/journal.pone.0037908.
