• Fabry, F., 2004: Meteorological value of ground target measurements by radar. J. Atmos. Oceanic Technol., 21 , 560573.

  • Fabry, F., 2006: The spatial structure of moisture near the surface: Project-long characterization. Mon. Wea. Rev., 134 , 7991.

  • Fabry, F., , Frush C. , , Zawadski I. , , and Kilambi A. , 1997: On the extraction of near-surface index of refraction using radar phase measurements from ground targets. J. Atmos. Oceanic Technol., 14 , 978986.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hao, Y., , Goeckel D. , , Janaswamy R. , , and Frasier S. , 2006: Surface refractive index (RI) field estimation from multiple radars. Radio Sci., 41 , RS3002. doi:10.1029/2005RS003288.

    • Search Google Scholar
    • Export Citation
  • Roberts, R. D., and Coauthors, 2008: REFRACTT-2006: Real-time retrieval of high-resolution, low-level moisture fields from operational NEXRAD and research radars. Bull. Amer. Meteor. Soc., 89 , 15351548.

    • Search Google Scholar
    • Export Citation
  • Weckwerth, T., , Pettet C. , , Fabry F. , , Park S. , , LeMone M. , , and Wilson J. , 2005: Radar refractivity retrieval: Validation and application to short-term forecasting. J. Appl. Meteor., 44 , 285300.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    The REFRACTT domain with gray levels indicating elevation from 1100 to 3600 m MSL. The circles represent 50-km range rings for the (top)–(bottom) Pawnee, CSU–CHILL, S-Pol, and KFTG radars. Surface weather stations are also marked.

  • View in gallery

    Target reliability index maps (RI ≥ 0.8) from (left) CSU–CHILL and (right) Pawnee.

  • View in gallery

    The (top) average and (bottom) standard deviation of the N difference between CSU–CHILL and Pawnee from 2200 to 2240 UTC 14 Aug 2006. The units are ppm of RI change (i.e., N).

  • View in gallery

    Solution to simulated refractivity field: a slanted plane increasing to the northeast. (a) LS solution without smoothness constraint. (b) Constrained LS solution. The green marker indicates the radar location and the black dots (only shown on the CLS result) indicate CSU–CHILL target locations out to 15 km used for both cases.

  • View in gallery

    Plot of the singular value curves of 𝗛u for (top) differential refractivity and (bottom) reference refractivity from data between 0000 and 0100 UTC 26 Jul 2006. The first part of the singular value curves up to the discontinuity are as a result of the ambiguity mapping matrix 𝗔, whose width is the number of groups formed [see (9)].

  • View in gallery

    CLS ΔN field on 26 Jul 2006 from 0012 to 0016 UTC over a 20 km2 area with CSU–CHILL targets out to 18 km at 300-m resolution. CLS results from (left) unfiltered and (right) filtered differential phases. The RMSE between the two is about 11 ppm. Targets are marked in white in the right panel and the radar is at the center of the polar grid.

  • View in gallery

    Phase and refractivity at 0016 UTC 26 Jul 2006 over a 40 km × 35 km area around CSU–CHILL as a moisture gradient was passing through the region. Although the QRPG result looks similar and the changes correspond to the phase map, it tends to underestimate the refractivity gradients. The QRPG result is at a resolution of 1° by 250 m, whereas the CLS resolution is 300 m × 300 m. (a) The filtered 4-min Δϕ in degrees, (b) the CLS 4-min ΔN field, (c) the corresponding 4-min ΔN field from the QRPG, (d) the filtered Δϕ from the reference phase in degrees, (e) the CLS absolute refractivity field (Nref = 280), and (f) the corresponding refractivity field from the QRPG. (a)–(f) cover the exact same geographic area.

  • View in gallery

    Comparison of the average N value with (top) standard deviation and (bottom) average ΔN for the region shown in Fig. 7 from 0000 to 0100 UTC. Here, it is more obvious that the CLS underestimates the refractivity gradient relative to the QRPG, but it does track the change fairly well.

  • View in gallery

    The change in refractivity from 2204 to 2240 UTC 14 Aug 2006 created by adding all scan-to-scan changes. (a) CLS on a 600-m resolution grid; (b) QRPG output on a 300-m grid, merged using the weighted mean discussed in section 2b; and (c) QRPG output on a 300-m grid, merged using the maximum. (top right) The effect of a storm moving through. Overall, the CLS underestimates the refractivity gradients. The white areas indicate no data in at least one of the input fields and the radar locations are marked with white (CHILL) and green (Pawnee) stars with the corresponding 50-km range ring in the same color. The data colors are clipped at ±50 for a better comparison. Areas of very low ΔN have high variance and are likely a result of noise and not isolated pockets of extreme drying.

  • View in gallery

    Comparison of the standard deviation and average N value change from (top) reference and (bottom) average ΔN for the region and time shown in Fig. 9. The N means are relative to the first scan. Here, the CLS underestimates the refractivity gradient even more, which is partly because of the decreased resolution.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 19 19 5
PDF Downloads 9 9 0

Implementation and Analysis of Networked Radar Refractivity Retrieval

View More View Less
  • 1 Colorado State University, Fort Collins, Colorado
© Get Permissions
Full access

Abstract

The surface-layer moisture field can be obtained by estimating the refractive index of air, measured in parts per million, and is referred to as refractivity. A technique to estimate the refractivity by using radar has been demonstrated experimentally using the measured change in phase from stationary ground targets. Recently, a new network-based algorithm was proposed within the Center for Collaborative Adaptive Sensing of the Atmosphere (CASA) as an alternative approach, especially when dealing with multiple radars. That work presented the algorithm and applied it to purely simulated data. The research presented here provides more detail and takes the new networked radar approach to the next level by independently validating and demonstrating the output with data collected during a refractivity field experiment in Colorado during the summer of 2006. The practical aspects of implementing the network-based algorithm are presented along with a more complete mathematical representation. The results are then compared with the previously fielded technique starting from the same filtered phase data. From this comparison, the authors conclude that the networked algorithm has potential for providing a good refractivity estimate from a radar network once some of its own shortcomings are addressed.

Corresponding author address: Jason Fritz, 1373 Campus Delivery, Colorado State University, Fort Collins, CO 80523-1373. Email: jpfritz@ieee.org

Abstract

The surface-layer moisture field can be obtained by estimating the refractive index of air, measured in parts per million, and is referred to as refractivity. A technique to estimate the refractivity by using radar has been demonstrated experimentally using the measured change in phase from stationary ground targets. Recently, a new network-based algorithm was proposed within the Center for Collaborative Adaptive Sensing of the Atmosphere (CASA) as an alternative approach, especially when dealing with multiple radars. That work presented the algorithm and applied it to purely simulated data. The research presented here provides more detail and takes the new networked radar approach to the next level by independently validating and demonstrating the output with data collected during a refractivity field experiment in Colorado during the summer of 2006. The practical aspects of implementing the network-based algorithm are presented along with a more complete mathematical representation. The results are then compared with the previously fielded technique starting from the same filtered phase data. From this comparison, the authors conclude that the networked algorithm has potential for providing a good refractivity estimate from a radar network once some of its own shortcomings are addressed.

Corresponding author address: Jason Fritz, 1373 Campus Delivery, Colorado State University, Fort Collins, CO 80523-1373. Email: jpfritz@ieee.org

1. Introduction

Having an accurate measure of the near-surface moisture field provides beneficial data to numerous applications such as boundary layer research, convection initiation, and quantitative precipitation forecasts (QPF). To estimate this moisture field with conventional Doppler weather radars, a technique developed in Fabry et al. (1997) and Fabry (2004) has been implemented and validated in the field (Weckwerth et al. 2005). This method estimates the index of refraction of air n, which is typically expressed as refractivity N, where N = (n − 1) × 106. The dimensionless (N units or ppm change from n of free space) quantity is a function of air and dewpoint temperature and pressure. This relationship is expressed as
i1520-0426-26-10-2123-e1
where T is air temperature in kelvins, P is pressure in kilopascals, and e is water vapor pressure in kilopascals (derived from dewpoint temperature). A more detailed description of boundary layer moisture variability and its effect on convection is given in Fabry (2006).
Estimation of the refractivity field is made possible by detecting the changes in round-trip phase of stationary ground targets over time. In an unchanging atmosphere, the phase of fixed objects will remain constant, barring any changes to the object. In free space, where n = 1 exactly, the phase is related to the two-way travel time and radar frequency f as ϕ = 2πfttravel. Deviations of refractivity resulting from a nonhomogeneous water vapor field cause fluctuations in the wave speed and path, which results in the phase becoming a path-integrated quantity. The phase of a target at a distance R from the radar is expressed in terms of range and three-dimensional refractivity change along the path as
i1520-0426-26-10-2123-e2
where λ is the radar wavelength. For simpler notation, ϕ = ϕ(r, n), because it is always a function of range r and the refractive index n (RI). Now consider the change in phase for a fixed target as the moisture field changes between time t1 and a reference time t0 given by
i1520-0426-26-10-2123-e3
Estimating the refractivity then involves accurately measuring the phase change over time, inverting (3), and accumulating the result for N. Over the domain of a radar, the phase difference of each resolution cell between any two times (scans) can be calculated as a zero-baseline interferogram, which is achieved from the complex conjugate multiplication of the received signal at every resolution cell. So, for a given range–azimuth gate (resolution cell) g,
i1520-0426-26-10-2123-e4
where Vng is the complex radar voltage from the nth scan.

This procedure was field tested again in 2006 but with multiple radars and additional sensors. From June through August 2006, the Refractivity Experiment for H2O Research And Collaborative operational Technology Transfer (REFRACTT06) in northeast Colorado (Roberts et al. 2008) was conducted using the refractivity retrieval software developed at McGill University (Fabry 2004), referred to in this paper as the quality-guided radial phase gradient (QRPG) algorithm because it uses data quality to weight the estimates of the phase gradient in the radial direction. For the REFRACTT experiment, refractivity fields were estimated using four S-band radars: the National Center for Atmospheric Research (NCAR) S-band dual-polarization Doppler radar (S-Pol), Colorado State University–University of Chicago–Illinois State Water Survey (CSU–CHILL), Pawnee, and the KFTG Weather Surveillance Radar-1988 Doppler (WSR-88D) located east of Denver, as seen in the domain map in Fig. 1. However, the Pawnee radar only came online toward the very end of the experiment.

Phase data collected from the CSU–CHILL and Pawnee radars during REFRACTT provide the basis to compare the results from the QRPG method to the network retrieval algorithm based on a constrained least squares (CLS) approach (Hao et al. 2006). Although Hao et al. (2006) achieved success with the CLS algorithm, they only went as far as to demonstrate the outcome via simulation with uniformly distributed targets and statistically well-defined phase behavior. The challenge is in implementing this novel technique with real data given the high-level description in Hao et al. (2006), which is the primary focus of this research. Real target locations also provide the means to demonstrate the impact of sparsely populated regions and those with a high variance in phase differentials. Section 2 first summarizes the QRPG algorithm, providing some detail on the phase filtering and error measures used. The CLS algorithm is then described in the context of simulated data sampled at real target locations from CSU–CHILL, providing supplemental details required to implement it. Section 3 analyzes the results of applying the CLS algorithm to REFRACTT data, first from one radar followed by two radars. The results are summarized in section 4.

2. Refractivity estimation algorithms

There are many challenges associated with estimating N from phase change that are addressed here. The moisture field changes in all three dimensions, but the current algorithms do not effectively account for the vertical change in refractivity or targets being at different elevations. This becomes more of a problem when attempting to merge the results of multiple radars and targets at different elevations, which is discussed in section 3. Another challenge is the fact that the phase wraps modulo 2π, which is exacerbated at higher radar frequencies such as the X band (8–12.5 GHz). If the sample times at subsequent radar scans are too large relative to the temporal moisture field change, aliasing occurs. Perhaps the biggest challenge, however, is dealing with a lack of stable ground targets that have coherent phase.

a. Stationary target identification and reference phase

Regardless of what algorithm is used to estimate the refractivity field, the ground targets must be identified. By observing the phase history of ground reflections during a period when the atmosphere is relatively free from weather and anomalous propagation, range resolution cells with the highest phase coherence and SNR with low SNR variance are identified as potential targets. Ideal targets are areas with man-made objects, such as towers and buildings, or perhaps bare earth and rock, like many of the targets observed from the Pawnee radar. Following the target identification procedure in Fabry (2004), a target reliability index RI ∈ [0, 1] (where 0 is worst and 1 is ideal) was assigned to each resolution cell of the REFRACTT radars from data collected at various times. The CSU–CHILL and Pawnee RI maps with an index value greater than 0.8 used for REFRACTT postprocessing are depicted in Fig. 2.

Because the important quantity to analyze is phase change, the phase of a scan must be compared to another phase. Any radar can compare a scan with the previous one, but longer wavelengths, such as S band, allow the use of a reference phase field collected at a completely different time. Ground weather stations placed near the radars during this data collection provide the parameters of (1) used to associate a reference refractivity Nref to the reference phase. For CSU–CHILL, Nref = 280, whereas Nref = 272 for Pawnee, despite the fact that the data for both were collected at the same time and with considerable overlap. This 8-ppm discrepancy highlights another difficulty in estimating an absolute refractivity from multiple radars. For a single radar, the ΔN calculated from a refractivity estimation algorithm is simply added to Nref, but an open question remains. Which Nref is used in the overlapping radar coverage regions or how are they combined to resolve the difference? Perhaps a nonuniform Nref must be defined using available, reliable meteorological ground stations. Overall, this procedure of identifying targets and a reference phase is considered as a calibration.

b. Quality-guided radial phase gradient

The technique proposed in Fabry (2004) retrieves the moisture field by essentially differentiating the phase difference field [from (4)] in range to estimate the spatial phase gradients corresponding to moisture gradients. To combat the phase wrapping, aliasing, and overall noise problem, several steps are taken. First, the phase difference between two collinear targets is used where the path is much shorter than the full radar range. This, of course, has problems in regions with a sparse distribution of targets. In general, this is accomplished by dividing the phase change between range gates given by the range difference Δr. Second, the velocity, spectral width, and SNR are used to create a quality index to reduce the impact of weather contamination. Together with the reliability index described above, a weighting mask is calculated to improve the estimate. Third, significant filtering is applied to smooth the sparsely populated phase field. Gaps will appear in the phase because of poor-quality data (e.g., in regions without coherent ground targets) that exacerbate the inversion problem to estimate refractivity but effectively remove bad data. The steps required to implement this phase filtering are presented here to provide a concise, but complete, summary:

  1. Average the I and Q samples comprising an integration cycle to filter higher velocity returns.
  2. Compute the average radial phase gradient over the whole field from the spatial correlation dϕ]/dr = arg{[1]}, where . Here, G is the total number of range gates being considered for each estimate. This is very similar to the pulse pair algorithm used in Doppler velocity estimation, except that it is in range instead of time.
  3. Subtract the gradient found above to leave residuals.
  4. Filter residuals using a weighted average using the aforementioned quality and reliability indices and a pyramidal or Gaussian 2D weighting function with a base on the order of 4 km.
  5. Add back the previously removed spatial gradient.
  6. Compute a local radial phase gradient over a smaller distance, say 2 km, from the spatial correlation. Using this lag 1 correlation, along with the quality and reliability indices, an error is assigned to each measurement.

Unfortunately, there is also no straightforward method to combine the results from different, overlapping radars beyond a mosaic-type procedure. However, we can take advantage of the error calculated in step 6. Here, we use an alternate method of mosaicking refractivity fields from multiple radars after transforming the estimates to a rectangular grid using a weighted average rather than simply taking the maximum value that was used during REFRACTT. The weighted average for a grid cell of refractivity estimates from L radars is calculated as Ncomp = aTw. The N values from each radar are contained in a and the weight vector w is comprised of
i1520-0426-26-10-2123-e5
where εi is the error from the ith radar. In regions where the error is undefined for one radar because of lack of targets, a large error (e.g., the maximum error of all radars) is assigned to limit discontinuities. This is particularly important when data from a radar contain high amounts of noise. An example is displayed in Fig. 3, which shows the average and standard deviation of NPawneeNCSU–CHILL over 40 min. The result of combining the two fields using (5) is discussed in section 3c (and also depicted in Fig. 9b). In some regions, the difference and variance are quite high, but the noise is reduced if at least one of the radars has a consistently low error signal.

c. Constrained least squares

The refractivity estimation algorithm proposed by Hao et al. (2006) addresses some of the challenges mentioned above, especially the multiradar problem. Rather than approach the problem with a radar-centric model, the CLS method uses a linear model for a solution from all radars simultaneously. A low resolution rectangular grid is defined over the domain of the radars. The line-of-site path from a radar to a target consists of segments through the grid cells such that the sum of the segments equals the total pathlength; that is, if the gridcell size were to approach zero, then the sum will approach the integral in (2). Then, for the K targets from multiple radars, (3) can be expressed as
i1520-0426-26-10-2123-e6
where ψ is the K × 1 length vector of phase changes from the reference or previous scan phase for each target observed at each radar, η is the P × 1 vector of refractivity change in each grid cell, and e is error resulting from noise, aliasing, etc. (P = MN for a generalized M × N grid). Arrays from each radar are vertically concatenated, and 𝗛 is K × P. Each row of 𝗛 represents one target, and each column contains the length of the radar-to-target path segment within that cell. A vast majority of the elements are 0, therefore making the extremely large 𝗛 very sparse and rank deficient. Consequently, an estimate is found by using singular value decomposition (SVD) to provide a solution space in conjunction with a smoothness constraint to select a solution within this space to minimize ‖ψ − 𝗛2. Using the conventional SVD solution to (6) can be expressed as
i1520-0426-26-10-2123-e7
where Σt represents the diagonal matrix of monotonically decreasing t singular values, and t is a chosen threshold. The subscripts of 𝗩 and 𝗨 indicate the partitioning to match the matrices with Σt. Thus, 𝗩 = [𝗩1𝗩2] and 𝗨 = [𝗨1𝗨2]. Without the smoothness constraint, however, this solution can lead to errors such as the result shown in Fig. 4a from a simulated differential refractivity field that increases linearly toward the northeast. In this simulation, ∼2300 real target locations were used from the CSU–CHILL radar, located in the bottom right, and limited to a 15-km radius to show an effect of regions with no targets. Each grid cell is 500 m2 and an S-band wavelength of 0.11 m was used.
At the spatial resolution of weather radars, it is convenient to consider water vapor and refractivity fields as being smooth. Taking advantage of this assertion, Hao et al. (2006) used a quadratic cost function D2 to define adjacent grid-value relationships resulting in the following problem statement: minimize D2 subject to minimizing ‖ψ − 𝗛2. The final constrained solution, not explicitly defined in Hao et al. (2006), is then expressed as
i1520-0426-26-10-2123-e8
where 𝗜t is the t × t identity matrix which shows that (8) becomes (7) if 𝗩2 or 𝗤 is 0 (no constraint). A more detailed derivation is given in the appendix that also demonstrates the need for the full 𝗩 matrix versus the standard SVD approximation that stops the calculation once the singular value threshold has been reached. Figure 4b depicts the solution to the constrained model when 𝗩2 is calculated for the same input and target locations as Fig. 4a. Using the RMS performance metric defined in (Hao et al. 2006), the RMS error (rmse) of the CLS solution is 0.3528 (N units). For comparison, when 356 of the 2300 real targets were randomly selected, the RMS error rose to 0.5931.
Phase unwrapping is not directly handled by (8), so additional steps are required. Hao et al. (2006) suggest a solution to this problem. Targets that are in close proximity and have a similar phase change, say within π/4 modulo 2π, are grouped together and locally unwrapped with respect to other targets within the group. So, when a wrap is detected, the phase is adjusted by ±2π. Within a group, however, the span of ψ is no more than the designated threshold, for example, ψ ∈ (−5π/4, 5π/4]. This procedure modifies (6) to produce
i1520-0426-26-10-2123-e9
where 𝗔 is binary ambiguity mapping matrix and g is a vector of group ambiguity numbers (see the appendix for details). A family of singular value curves calculated from a CSU–CHILL test case discussed in section 3b is shown in Fig. 5. These are all the scan-to-scan (top) and reference-to-scan (bottom) singular values over a 1-h period with a scan repeat interval of 4 min. The curves may look unusual because they are the singular values of 𝗛u, so the first part is actually from −2π𝗔. From these curves, a common threshold of 10−2 was selected for processing real data, because this is approximately the level prior to the significant decrease in singular values and will meet the criterion, where σi is the ith singular value and ɛ is some small number.

3. Implementation with real data

The primary test of any radar processing algorithm is to implement it using real data because this is usually quite challenging and the CLS algorithm is no exception. Data collected from the CSU–CHILL and Pawnee S-band radars during the REFRACTT experiment were used to implement and compare the QRPG and CLS refractivity retrieval methods. Specifically, the two cases used are a case with only CSU–CHILL that has a passing storm front and another less-dramatic case involving both CSU–CHILL and Pawnee. The CSU–CHILL and Pawnee radars are situated at elevations of 1432 and 1688 m MSL, respectively, and are approximately 50 km apart. Volume scans were synchronized with the National Weather Service (NWS) KFTG operational radar, so 0° elevation scans were repeated every 4 min. Coherent ground targets were identified using the procedure described in section 2a (Fig. 2), and the REFRACTT reference phase was also used. The two CSU radars were set up to collect data in indexed beam mode to facilitate scan-to-scan comparison (i.e., spatial alignment versus using a constant number of pulses for each sample average). CSU–CHILL data were collected at an azimuth resolution of 0.72°, so they were resampled to 1° to reduce parameter estimation variance and align with Pawnee data. Then, the phase difference is computed using (4). The challenges and results are discussed below.

a. Challenges

By far the most prominent challenge to estimating refractivity is the quality of the phase data, which is considerably more complex than simply adding Gaussian noise to an ideal phase field. Unfortunately, the real data are very noisy and frequently exhibit aliasing, especially at the 4-min scan cycle. Calculating the phase difference between successive scans (differential phase) versus the reference field helps to reduce the noise, but it only provides relatively short time-scale information, potentially resulting in error accumulation. To make matters worse, what are usually coherent targets can become incoherent because of passing weather and other atmospheric effects, such as anomalous propagation. Therefore, the quality index and phase filtering, such as those described in section 2b, are essential. Because that phase field filter has shown success in the field, the same implementation was used to preprocess the data for the CLS algorithm. Using the same filter also aids in comparing the results of the two algorithms because the phase input is then identical for both. In regions of undefined phase, unreliable (poor quality) target phases are removed, which means that the number of targets used for solving (8) changes for each scan.

b. Single radar results

Computing the results using a single radar validated the approach to implementing the CLS retrieval from real data. Around 0000 UTC 27 July 2006 (1800 LT July 26), a storm passed through the CSU–CHILL domain resulting in a noticeable refractivity gradient. The first test was to determine the performance of the CLS algorithm to an unprocessed differential phase field. The same northwest quadrant of the CSU–CHILL scan was used but with a 20 km × 20 km grid at 300-m resolution. For scan-to-scan change, Fig. 6a shows the result of using unprocessed data as compared with the filtered phase difference in Fig. 6b from 0012 to 0016 UTC. The figure also shows the target and radar locations to demonstrate that the CLS algorithm can vary drastically between targets at this resolution if the phase is not smoothed. The RMSE between the two is a very substantial 11 ppm, and the display values in Fig. 6a were clipped at ±18 to show some features in the filtered version. Filtering the differential phase clearly keeps the solution to reasonable values.

Further investigation of this 26 July test case reveals the capability of the CLS algorithm to retrieve refractivity estimates from a single radar. Figure 7 depicts the results from filtering the phase differential (left column), the CLS algorithm (middle column), and the QRPG (right column) at 0016 over a much larger 40 km × 35 km area with a grid resolution of 300 m using targets with an RI above 0.6. The top row of Fig. 7 shows the change from the previous scan, whereas the bottom row shows the phase change from the reference and absolute refractivity. A polar grid with its center at the radar is overlaid on the CLS solution for reference and all panels show the same geographic area. Looking along radials in the smoothed phase images, more folds will indicate a larger gradient in refractivity. Both algorithms show this effect, although it is more prevalent in the QRPG output. The increasing refractivity gradient to the west of the radar is particularly visible in the absolute N fields shown in Figs. 7e,f. Although the CLS results appear to have more “texture” due in part to the resolution, Figs. 7b,e show that the CLS method fills in the gaps of data reasonably well, at least in terms of being relatively smooth. However, regions with a sparse target population can exhibit the striped pattern seen in Figs. 7b,e. A trade-off exists between this effect and a higher level of noise when the target RI threshold is decreased. Regardless, it is dependent on target distribution and grid resolution.

An RMS error between the two methods will include errors because of quantization and resampling differences, so a better method for comparison is the spatial statistics. The top plot in Fig. 8 depicts the average absolute N and the standard deviation for all scans from 0004 to 0100 UTC over the region shown. The mean N from CLS follows that of the QRPG, but it is clear that the CLS method underestimates the gradient. However, the standard deviation is very similar between the two, with a peak around the time of the maximum refractivity gradient. Similarly, the bottom plot of Fig. 8 shows that the mean scan-to-scan refractivity over the same time period is also underestimated by the CLS method (the standard deviations are very similar and not shown). This underestimation is possibly due to the system being overconstrained and is a problem for future investigation.

c. Multiple radar results

Given the moderate success with implementing the CLS algorithm on one radar, it was then applied to two radars: CSU–CHILL and Pawnee. Because there is no reasonable way to account for different Nref values in an overlap region, as discussed in section 2a, the comparison between algorithms is made by accumulating the scan-to-scan ΔN on a 90 km × 38 km grid at a 600-m resolution to reduce the computation. Only targets with an RI above 0.8 are used for the same purpose (see Fig. 2). Figure 9 displays the results of the CLS and QRPG algorithms [merged using (5) in Fig. 9b and the maximum in Fig. 9c]. White and green star icons indicate the radar locations at roughly (30, 0) and (35, 48) on the grid along with 50-km range rings of the same color. This case was from 2200 to 2240 UTC 14 August 2006, and a small rain storm passed by the Pawnee radar during this time. The increase in refractivity because of the storm is indicated in the region just south of the Pawnee radar. The CLS and weighted sum QRPG methods yield similar results here, but again the CLS algorithm is slightly underestimated. The maximum merge result shown in Fig. 9c provides more evidence for the weighted average described by (5). The average value in Fig. 9b is −1 ppm, whereas it is 6.6 ppm in Fig. 9c. The boundary of the Pawnee values is clearly visible at approximately 70 km from Pawnee because of the sharp gradient.

Along the western third of the gridded domain, the reliable target density is light and significant noise manifested in almost all QRPG ΔN fields resulting in the high variance seen on the left side of Fig. 9b. Meanwhile, the CLS solution produced relatively flat responses to those regions, which is probably closer to the truth because drops of more than 50 ppm over 40 min in small pockets while a storm is nearby are unlikely.

A more quantitative analysis to compare the two algorithm results again uses spatial statistics. This result is shown in Fig. 10, which is similar to Fig. 8, except that instead of absolute N the means were referenced to the first scan. Because of the noise and lack of global change in the southern section, the statistics were only computed down to 50 km along the y axis. Again, the mean curves in Fig. 10a show that the CLS follows the QRPG but underestimates it. At the same time, however, the standard deviation was also lower by 1–2 ppm. Similarly, the average scan-to-scan change plotted in Fig. 10b shows the CLS following but underestimating the QRPG result, which fluctuated around zero.

A secondary challenge for the CLS algorithm involves the computational resource usage. The two cases implemented with real data were computed on a dedicated workstation with 2 quad core 2.83-GHz Intel Xeon processors with 8 GB RAM, and the SVD calculation utilizes the Linear Algebra Package (LAPACK) Fortran library. For the case with only CSU–CHILL, the 𝗛u matrix to compute the SVD on was 8200 × 16 800, on average, for ΔN and about 700 columns wider for N because of formation of more groups. The SVD took 95–110 min each for ΔN and N, but an additional 135 min were required for the inversion in (8). This time is about 4 times longer than the time required for only using targets with an RI > 0.8, even though there are only 50% more targets. The dual radar estimation, using only the targets with RI above 0.8, took about 80 min for the SVD and less than 2 min for the inversion. This result is due to the fact that the 𝗛u matrix is roughly 11 000 × 12 000 in this case; the fast inversion provides additional evidence of the benefit of multiple radars. Although the software was optimized to use minimal memory, the memory usage just for the SVD and calculation of (8) ranged from 1200 to 3200 MB, making the estimation of refractivity using CLS over the whole REFRACTT domain impractical at resolutions high enough to capture the gradients to the extent that it can.

4. Summary and conclusions

A number of conclusions can be made from the implementation and analysis of the CLS refractivity retrieval algorithm presented here. The algorithm was independently verified through our own simulations by using real target locations, which shows the necessity of the smoothing constraint. Implementing the CLS method on real data was the primary challenge, which demonstrated the need for filtering the differential phase to reduce noise and aliasing. This was accomplished by applying the same smoothing filter used in the existing refractivity estimation technique (QRGP), which also made comparison between the two methods more direct. For a single radar, the CLS approach is similar to the QRPG result, although slightly underestimated, which is perhaps caused by too much constraint. When applied to multiple radars, the CLS was not affected by the high level of noise; on the other hand, it may not have captured more subtle changes in refractivity that the QRPG picked up. Parameters such as grid resolution and the RI threshold for target selection do effect the result and need to be thoroughly investigated. In general, this algorithm shows great potential, especially when applied to the multiradar estimation problem, but there is room for optimization and improvements. Among the possibilities to investigate are a target weighting scheme to reduce noise, relaxing the constraint, optimizing the various thresholds used, and finding a good approximation to the full SVD and inversion computations.

Acknowledgments

This work is supported by the Engineering Research Center program of the National Science Foundation (NSF) under the Award 0313747. The authors would also like to thank Rita Roberts for her efforts in leading and managing REFRACTT. In addition, the authors acknowledge discussion with Frédéric Fabry from McGill University.

REFERENCES

  • Fabry, F., 2004: Meteorological value of ground target measurements by radar. J. Atmos. Oceanic Technol., 21 , 560573.

  • Fabry, F., 2006: The spatial structure of moisture near the surface: Project-long characterization. Mon. Wea. Rev., 134 , 7991.

  • Fabry, F., , Frush C. , , Zawadski I. , , and Kilambi A. , 1997: On the extraction of near-surface index of refraction using radar phase measurements from ground targets. J. Atmos. Oceanic Technol., 14 , 978986.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hao, Y., , Goeckel D. , , Janaswamy R. , , and Frasier S. , 2006: Surface refractive index (RI) field estimation from multiple radars. Radio Sci., 41 , RS3002. doi:10.1029/2005RS003288.

    • Search Google Scholar
    • Export Citation
  • Roberts, R. D., and Coauthors, 2008: REFRACTT-2006: Real-time retrieval of high-resolution, low-level moisture fields from operational NEXRAD and research radars. Bull. Amer. Meteor. Soc., 89 , 15351548.

    • Search Google Scholar
    • Export Citation
  • Weckwerth, T., , Pettet C. , , Fabry F. , , Park S. , , LeMone M. , , and Wilson J. , 2005: Radar refractivity retrieval: Validation and application to short-term forecasting. J. Appl. Meteor., 44 , 285300.

    • Crossref
    • Search Google Scholar
    • Export Citation

APPENDIX

The Constrained Least Squares Solution

Hao et al. (2006) presented the various components of the CLS algorithm, but here we derive an expression for the complete solution. Using the truncated matrix of singular values, an approximation 𝗛t to 𝗛 can be used to express the model as
i1520-0426-26-10-2123-ea1
Now, let
i1520-0426-26-10-2123-eqa1
where z1 is a t × 1 vector and z2 is Pt × 1. In the conventional SVD solution, z2 is set to zero without significant consequences to error minimization, but here we leave it to be calculated. Therefore,
i1520-0426-26-10-2123-ea2
The smoothing constraint is expressed as a quadratic cost function of the grid, correctly written as D2 = ½ηT𝗤η [see Hao et al. (2006) for more details on defining D]. Substituting (A2) into this quadratic and expanding, considering 𝗤 is symmetric, results in
i1520-0426-26-10-2123-eqa2
The global minima is found by setting D2/dz2 to zero, yielding
i1520-0426-26-10-2123-ea3
If 𝗩2 was left unknown by only using the truncated SVD, z2 would be undefined and would not result in a solution to (A2). An expression for z1 can be found by writing (A1) as
i1520-0426-26-10-2123-eqa3
This leads to
i1520-0426-26-10-2123-ea4
Substituting (A4) and (A3) into (A2) and simplifying results in the estimate of η as
i1520-0426-26-10-2123-ea5
Incorporating the grouping and local phase unwrapping discussed in section 2c into (A5) for multiple radars is fairly straightforward. The binary ambiguity mapping matrix 𝗔 and the vector of group ambiguity numbers g are combined with the locally unwrapped phase observation vector ψu as
i1520-0426-26-10-2123-ea6
i1520-0426-26-10-2123-eqa4
i1520-0426-26-10-2123-ea7
When combining data from multiple radars, 𝗛 matrices from each radar are concatenated vertically; however, the ith radar grouping matrix 𝗔i and the constraint quadratic 𝗤 must also be sized correctly. The grouping matrices for J radars are combined along a diagonal, as in
i1520-0426-26-10-2123-ea8
The constraint matrix is then expanded as
i1520-0426-26-10-2123-ea9
where 𝗜G is the identity matrix with dimensions equal to the total number of groups formed, which is also the width of 𝗔comb. Then, (A8) and (A9) are substituted into (A6) and (A5), respectively, for 𝗔 and 𝗤 for the generalized solution.

Fig. 1.
Fig. 1.

The REFRACTT domain with gray levels indicating elevation from 1100 to 3600 m MSL. The circles represent 50-km range rings for the (top)–(bottom) Pawnee, CSU–CHILL, S-Pol, and KFTG radars. Surface weather stations are also marked.

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 2.
Fig. 2.

Target reliability index maps (RI ≥ 0.8) from (left) CSU–CHILL and (right) Pawnee.

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 3.
Fig. 3.

The (top) average and (bottom) standard deviation of the N difference between CSU–CHILL and Pawnee from 2200 to 2240 UTC 14 Aug 2006. The units are ppm of RI change (i.e., N).

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 4.
Fig. 4.

Solution to simulated refractivity field: a slanted plane increasing to the northeast. (a) LS solution without smoothness constraint. (b) Constrained LS solution. The green marker indicates the radar location and the black dots (only shown on the CLS result) indicate CSU–CHILL target locations out to 15 km used for both cases.

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 5.
Fig. 5.

Plot of the singular value curves of 𝗛u for (top) differential refractivity and (bottom) reference refractivity from data between 0000 and 0100 UTC 26 Jul 2006. The first part of the singular value curves up to the discontinuity are as a result of the ambiguity mapping matrix 𝗔, whose width is the number of groups formed [see (9)].

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 6.
Fig. 6.

CLS ΔN field on 26 Jul 2006 from 0012 to 0016 UTC over a 20 km2 area with CSU–CHILL targets out to 18 km at 300-m resolution. CLS results from (left) unfiltered and (right) filtered differential phases. The RMSE between the two is about 11 ppm. Targets are marked in white in the right panel and the radar is at the center of the polar grid.

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 7.
Fig. 7.

Phase and refractivity at 0016 UTC 26 Jul 2006 over a 40 km × 35 km area around CSU–CHILL as a moisture gradient was passing through the region. Although the QRPG result looks similar and the changes correspond to the phase map, it tends to underestimate the refractivity gradients. The QRPG result is at a resolution of 1° by 250 m, whereas the CLS resolution is 300 m × 300 m. (a) The filtered 4-min Δϕ in degrees, (b) the CLS 4-min ΔN field, (c) the corresponding 4-min ΔN field from the QRPG, (d) the filtered Δϕ from the reference phase in degrees, (e) the CLS absolute refractivity field (Nref = 280), and (f) the corresponding refractivity field from the QRPG. (a)–(f) cover the exact same geographic area.

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 8.
Fig. 8.

Comparison of the average N value with (top) standard deviation and (bottom) average ΔN for the region shown in Fig. 7 from 0000 to 0100 UTC. Here, it is more obvious that the CLS underestimates the refractivity gradient relative to the QRPG, but it does track the change fairly well.

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 9.
Fig. 9.

The change in refractivity from 2204 to 2240 UTC 14 Aug 2006 created by adding all scan-to-scan changes. (a) CLS on a 600-m resolution grid; (b) QRPG output on a 300-m grid, merged using the weighted mean discussed in section 2b; and (c) QRPG output on a 300-m grid, merged using the maximum. (top right) The effect of a storm moving through. Overall, the CLS underestimates the refractivity gradients. The white areas indicate no data in at least one of the input fields and the radar locations are marked with white (CHILL) and green (Pawnee) stars with the corresponding 50-km range ring in the same color. The data colors are clipped at ±50 for a better comparison. Areas of very low ΔN have high variance and are likely a result of noise and not isolated pockets of extreme drying.

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Fig. 10.
Fig. 10.

Comparison of the standard deviation and average N value change from (top) reference and (bottom) average ΔN for the region and time shown in Fig. 9. The N means are relative to the first scan. Here, the CLS underestimates the refractivity gradient even more, which is partly because of the decreased resolution.

Citation: Journal of Atmospheric and Oceanic Technology 26, 10; 10.1175/2009JTECHA1182.1

Save