1. Introduction
The Engineering Research Center for Collaborative Adaptive Sensing of the Atmosphere (CASA ERC) has been established to create the underlying scientific and engineering basis for a new paradigm of the Distributed Collaborative Adaptive Sensing (DCAS) radars applied to hazardous localized weather detection, tracking, and predicting. The DCAS is a new approach to radar sensing of the atmosphere being investigated to overcome the coverage limitations inherent in long-range radar networks. To be able to provide economically viable solutions to this approach, meteorological radar operation must change from the “preferred” S-band operation to higher frequencies such as X band so that physically smaller antennas can be deployed (McLaughlin et al. 2009). However, at higher frequencies, the impact of attenuation due to precipitation need to be resolved for successful implementation.
Since Hitschfeld and Bordan (1954) proposed the attenuation correction technique based on the empirical relationship of reflectivity versus specific attenuation, many attenuation correction algorithms have been developed. For ground radars with polarimetric capability, a simple attenuation correction method using differential phase was discussed in Bringi et al. (1990). Subsequently, a constrained solution for path-integrated attenuation derived from the differential propagation phase shift (ϕdp) was proposed (Testud et al. 2000). Both algorithms are sensitive to the specific attenuation versus specific differential phase parameterization. To eliminate this problem, Bringi and Chandrasekar (2001) suggested a self-consistent algorithm combining a differential phase shift and a differential reflectivity constraint. Gorgucci and Chandrasekar (2005) evaluated the various algorithms for their error structure. In addition, Gorgucci et al. (2006) developed an attenuation correction procedure using the self-consistency principle. The concept of a dual-radar method for reflectivity and specific attenuation retrieval was studied by Srivastava and Tian (1996). They used an analytical method to retrieve specific attenuation and reflectivity. For networked radar systems, Chandrasekar and Lim (2008) proposed a methodology for reflectivity retrieval in a networked radar environment. A solution for the specific attenuation distribution can be provided by solving the integral equation for reflectivities from networked radars. The general concept of arrangement for networked environments is shown in Fig. 1.
The real-time network-based reflectivity retrieval system (NBRR) involves collecting data from multiple remote radars and performing digital signal processing, and those properties of the applications present unique demands on system infrastructure and software. In this paper we describe an architectural framework for multisensor data integration to provide an efficient system for the real-time implementation of NBRR. The architecture provides a radar data managing and searching mechanism for integrating data and a multiprocess coordinate mechanism to exploit parallel processing for the computationally intensive reflectivity retrieval algorithm. This paper presents the real-time performance metrics as well as the attenuation correction capability of the NBRR using CASA Integrated Project 1 (IP1) data during 2007–09 field experiments. To evaluate the real-time performance for NBRR, the execution run times are measured and compared with radar data generation time. The evaluation for attenuation correction capability is also carried out in two different ways. A theoretical way is to compare retrieved results with the self-consistency relation expected between intrinsic measurements. This can provide fundamental validity for the retrieval algorithm. Comparison with the dual-polarization attenuation correction method and with the observations of Weather Surveillance Radar-1988 Dopplers (WSR-88Ds) is also done. Comparison with WSR-88D observations is based on the principle that the intrinsic reflectivities at S and X are similar.
The paper is organized as follows. Section 2 describes briefly the concept of a network-based reflectivity retrieval algorithm and the CASA IP1 system. The design and real-time implementation of an architectural framework for a network-based reflectivity retrieval system is presented in section 3. Section 4 presents the real-time performance and attenuation correction capability of the NBRR system at the CASA radar network. The important results of this paper are summarized in section 5.
2. Background
a. Network-based reflectivity retrieval
b. CASA IP1
The first-generation test bed of the CASA IP1 is currently operational in Oklahoma. The IP1 system is a radar network that can observe a weather event by simultaneously using multiple radars. The IP1 network is composed of four X-band radars located over a grid given by 13.1, −13.8, 15.5, −15.7 km in the east–west direction and 22.6, 5.1, −1.6, −22.6 km in the north–south direction from the center of the network [Cyril (KCYR), Chickasha (KSAO), Lawton (KLWE), and Rush Springs (KRSP)], respectively. Figure 2 depicts the configuration of the CASA IP1 network.
3. Real-time implementation of the network-based reflectivity retrieval system
The IP1 system implements a closed-loop control system to provide the functionality required for data collection, computation, and radar actuation. A control loop begins with a continuous data ingestion of using radars, identifying meteorological features in this data, and ends with determining each radar’s future scan strategy based on detected meteorological features and end-user requirements (Zink et al. 2005). The NBRR involves collecting data from multiple remote radars and performing computations of intensive digital signal processing, and those applications present unique demands on the system infrastructure and software development. The software architecture for the real-time implementation of the NBRR system is shown in Fig. 3. The NBRR system mainly consists of three modules, such as the database management module, the common volume finding module, and the algorithm processing module. The database management module is for obtaining, preprocessing, and storing data and managing the database. The Local Data Manager (LDM; information online at www.unidata.ucar.edu/software/ldm/) is used as the communication software to obtain real-time data from the radars. The common volume finding module selects a set of data (a common volume) collected from overlapping coverage areas by multiple radars. The algorithm processing module performs a reflectivity retrieval algorithm. This module runs multiple threads simultaneously to process multiple rays at the same time, so it reduces the execution run time for the NBRR. Using this framework, we seek to provide efficient system support for the real-time implementation of NBRR. The framework provides a radar data managing and searching mechanism for finding a set of data to be integrated and a multiprocess coordinate mechanism to exploit parallel processing for the computationally intensive reflectivity retrieval algorithm. The system consists of a sensor database that contains a list of radars and related information such as their locations and radar measurements, a data management and searching module, a common volume finding module, and a fusion algorithm processing module. Obtaining the desired accuracy of the end result of the algorithm requires the use of a set of data to be integrated from multiple radars considering time gaps and the physical distances of the sampled locations. Because each of the radars can be assigned to a certain sensing task with a different scanning mode and interval in the CASA radar network, it can be challenging to synchronize the data generation time and the number of radar data in a given time period. In addition, each of the radars generates tens of megabytes of data per minute and the data from multiple radars have to be assembled for the NBRR system due to the need to merge the data. Because of the challenges of the radar data generation pattern and the large amount of data, managing the data and finding a set of correlated data items for multiradar data fusion applications are considerable tasks. Considering all these aspects, we implement a sensor database, a data management module, and a common volume finding module, which provide an efficient managing and searching mechanism for the NBRR. Furthermore, the NBRR requires substantial computation due to extensive data processing and has a real-time processing requirement. The NBRR is implemented in a multiprocessor environment, where multiple threads operate simultaneously and collaboratively to meet the real-time requirement of the algorithm. Figure 4 illustrates the simultaneous operation of the multiple threads. The algorithm-processing module runs multiple threads simultaneously to process multiple radar beams at the same time, so it reduces the execution run time for the NBRR.
4. Performance of the real-time network-based reflectivity retrieval system
a. Performance of aspects of the real-time implementation
In this section, the empirical measurements of the execution run times of the network-based reflectivity algorithm are presented. The machine used in our experiments was equipped with two quad-core Intel Xeon processors, each running at 2.86 GHz. We use existing CASA radar data as inputs into the NBRR program. The data were collected from the CASA IP-1 network during a period of time when storms were moving through the network coverage area. The data generated from all four IP1 nodes were processed by the machine simultaneously. The scan strategy of CASA radars is controlled by the end-user requirements and the meteorological features present in the radar coverage area. The radar scans that are used in CASA IP1 are mainly sector plan position indicator (PPI) scans at multiple elevations and full 360° surveillance scans at 2° elevation angles. Table 1 describes the radar data that are used for this performance evaluation. The data in cases 1 and 2 were collected from all four IP1 sites. Each volume scan of the cases consists of a full 360° scan at a 2° elevation angle, and sector scans collected at 1.00°, 3.86°, 6.70°, 9.50°, 12.26°, 14.96°, 17.60°, 20.47°, 23.72°, 27.34°, and 31.31° elevation angles, and a volume was taken every 170 s. The data in cases 3–5 are also collected from all four radars. In these cases, the radar scans were performed with a 60-s (20 + 40 s) heartbeat interval where 20 s is used for the full 360° scan at a 2° elevation angle while the rest of the 40 s is used for sector scans at 1°, 3°, 5°, 7°, 9°, 11°, and 14° elevation angles.
Figure 5 shows the histogram of per-sweep execution run times for the case 1, 3, and 5 datasets. As seen in Fig. 5, 69% of the sweeps in case 1 and 62% of the sweeps in case 5 were executed in subseconds. More than 98% of the sweeps in both cases were processed in 5 s. In case 3, only 40% of the sweeps were processed in subseconds, and the execution run times for approximately 5% of the sweeps are larger than 5 s. The software takes a longer time to process the case 3 data. This execution run-time difference results from the fact that the sizes of the meteorological objects in the radar field were different among the cases. In cases 1 and 2, the events were isolated storms with relatively small coverage areas, whereas in case 3, several storm cells covered most of the IP1 network area, and the storms continued to intensify. Each CASA radar had a maximum range of 30 km in cases 1 and 2, but the maximum range increased to 40 km in cases 3–5. Because the radars collected larger amounts of data with the increasing range, the NBRR system took more time for processing.
To prevent overflow beyond the system’s limited capacity, it is crucial to maintain the execution run times for the NBRR faster than the radar sweep generation interval that is the consumed time performing one sector scan or one full scan. Table 2 shows the average sweep generation intervals and the average per-sweep execution run times for the test cases. Our measurements show that approximately 1 and 2 s per-sweep processing times are needed on average for isolated storm cases (cases 1, 2, and 5) and clusters of storms cases (cases 3 and 4). The average execution run times are relatively small with respect to the data generation intervals for all of the test cases.
b. Performance of aspects of the algorithm
The network-based reflectivity retrieval technique is evaluated extensively using CASA IP1 data during 2007–09 operations. The retrieval results of the algorithm are evaluated in two ways. One way is to check for consistency with an intrinsic relationship, such as nonattenuated reflectivity [Zh (dB)] versus specific differential phase [Kdp (deg km−1)]. This method can provide comparison validity by self-consistency. The other way is by comparison with single radar attenuation correction methods and against nonattenuated measurements such as S-band radar observations. S-band radar measurements do not suffer much from attenuation and can provide a basis for comparing attenuation corrected reflectivities. This method can provide external validity. For external validity, three WSR-88Ds (KFDR, KTLX, and KOUN) are used, which are located over a grid at coordinates −81.3, 74.8, and 58.0 km in the east–west direction and −52.4, 56.4, and 45.5 km in the north–south direction from the center of the CASA network, as shown in Fig. 2. In this paper the results of the reflectivity retrieval system have also been compared with the conventional single radar rain profiling algorithm (CRP; Bringi and Chandrasekar 2001) operating in CASA IP1 systems. Based on internal and external evaluations, the network-based retrieval approach can be evaluated.
Evaluation has been carried out with 3 yr of data collected from CASA IP1 systems. For brevity, four cases from 2007 and 2009 are shown here. The four cases are 10 June 2007, and 24 May, 2 June, and 10 June 2009. The events on 10 June 2007, 24 May 2009, and 10 June 2009 were severe convective storms. The event on 2 June 2009 is a moderate rain storm. The first case data were collected from all CASA-IP1 radars on 10 June 2007. This event was an isolated convective storm. The storm moved through the center of the IP1 network. Figures 6 and 7 show the comparison of the horizontal and vertical structures of the attenuation-corrected reflectivity with the observations from the WSR-88Ds. Figures 6a and 6b show the composite of the observed and retrieved reflectivities by NBRR from CASA-IP1 radars at 2 km MSL, respectively, where the retrieved reflectivity by CRP is shown in Fig. 6c. The composites of the observed reflectivities by WSR-88Ds are shown in Figs. 6d–f. Note that the NBRR is applied only to areas that have overlapped by at least two radars and Fig. 6 shows only the overlapped region. The vertical structure of the retrieved reflectivity by NBRR is shown in Fig. 7a; the vertical structures of the observed reflectivities by WSR-88Ds are shown in Figs. 7b–d. This case study is mainly focused on experimental validation. The agreement between the retrieved reflectivity from X-band IP1 radars and the observed reflectivity from S-band WSR-88Ds can provide validation of the NBRR.
For quantitative comparison, the difference between the retrieved reflectivities by NBRR from CASA IP1 data and the observed reflectivities from WSR-88Ds (KOUN, KTLX, KFDR) are shown in Table 3. In spite of the widely differing geometrics of the radars, Figs. 6 and 7 and Table 3 show remarkably good agreement in general. The comparison with the KOUN observation provides a good reference for validation of NBRR. The comparison shows little bias and about 5-dBZ standard deviation. For comparison between two radars with different beamwidths, resolution volumes, and frequencies, this level of agreement is very good. Quantitative analyses for before and after network-based attenuation correction are also performed and are listed in Table 3. The results show about a 4-dBZ bias improvement for this case.
The other three case studies focus on comparison validation by self-consistency. Retrieved results by NBRR are compared with the reflectivity [Zh (dB)] versus specific differential phase [Kdp (deg km−1)] relationship. Figures 8a–d show the observed reflectivities from all four radars on 10 June 2009 [in order (a) KCYR, (b) KSAO, (c) KLWE, and (d) KRSP]. Evaluation results of 10 June 2009 are shown in Fig. 9. Figure 9a shows the composite of the observed reflectivity; Figs. 9b and 9c show the composite of attenuation corrected reflectivity using NBRR and CRP, respectively. Figures 9d and 9e show the Kdp versus Zh scatterplots before and after attenuation correction where they are compared against a theoretical mean relation. The theoretical mean relation is based on data obtained by a scattering simulation using the shape model proposed by Brandes et al. (2002) and Bringi et al. (2003) for widely varying drop size distributions (0.5 ≤ D0 ≤ 3.5 mm, 3 ≤ log10Nw ≤ 5 and −1 ≤ μ ≤ 4 for R < 300 mm h−1) at a temperature of 0°–20°C. A similar Kdp versus Zh scatterplot is also given in Bringi and Chandrasekar (2001). It can be seen in Fig. 9e that the attenuation-corrected reflectivities follow the mean theoretical relation very well. Figure 10 shows evaluation results of 24 May 2009, and the case of 2 June 2009 is shown in Fig. 11. Figures 10a and 11a show the composite of the observed reflectivity where Figs. 10b,c and 11b,c show the composite of the attenuation-corrected reflectivity using NBRR and CRP, respectively. Figures 10d,e and 11d,e show the Kdp versus Zh scatterplots before and after attenuation correction. These evaluations for consistency with an intrinsic relationship such as retrieved reflectivity versus specific differential phase can give validation for the NBRR.
For a self-consistent quantitative comparison, the difference between the retrieved reflectivities by NBRR and CRP are analyzed and described in Table 4. The conventional single-radar rain-profiling algorithm (Park et al. 2005) has been used and validated in various studies. The results in Table 4 show that the bias and the standard deviation against CRP are within ±1 and 2.5 dBZ, respectively.
From the results of these evaluations, we can see that the network-based reflectivity retrieval algorithm works well. Comparison of retrieved results of the network approach with conventional attenuation correction and the observations from WSR-88Ds shows good agreement for operational applications. Comparing the retrieved results with a theoretical relationship also offers validation for the network-based attenuation correction system.
5. Summary
The implementation of an architectural framework for timely and accurate processing of radar data fusion algorithms in CASA IP1 has been presented. The architectural framework provides a data managing and searching mechanism and a common volume identification system. In addition, it accommodated simultaneous and collaborative operation of multiple threads to meet the execution time requirements of the CASA multiradar data integration application. Through the execution run-time measurements of NBRR over several storm cases, we have shown that the framework could meet the real-time requirements for CASA application. The network-based retrieval algorithm has been evaluated by comparisons against WSR-88D observations during targets of opportunity as well as using self-consistency for the CASA IP1 data during 2007–09 field experiments. By comparing the retrieved results with a theoretical relationship, the performance of the network approach is validated. The retrieval results of the network approach show good agreement with the results of the conventional attenuation correction method and the observations of the WSR-88Ds. Based on the test results for our 3-yr dataset, we can conclude that the network-based reflectivity retrieval algorithm performs well in a variety of environment. It should be noted here that the network-based retrieval does not use dual-polarization observations.
Acknowledgments
This research is supported by the National Science Foundation (ERC-0313747).
REFERENCES
Brandes, E., Zhang G. , and Vivekanandan J. , 2002: Experiments in rainfall estimation with a polarimetric radar in a subtropical environment. J. Appl. Meteor., 41 , 674–685.
Bringi, V. N., and Chandrasekar V. , 2001: Polarimetric Doppler Weather Radar: Principles and Applications. Cambridge University Press, 636 pp.
Bringi, V. N., Chandrasekar V. , Balakrishnan N. , and Zrnić D. S. , 1990: An examination of propagation effects in rainfall on radar measurements at microwave frequencies. J. Atmos. Oceanic Technol., 7 , 829–840.
Bringi, V. N., Chandrasekar V. , Hubbert J. , Gorgucci E. , Randeu W. L. , and Schoenhuber M. , 2003: Raindrop size distribution in different climatic regimes from disdrometer and dual-polarized radar analysis. J. Atmos. Sci., 60 , 354–365.
Chandrasekar, V., and Lim S. , 2008: Retrieval of reflectivity in a networked radar environment. J. Atmos. Oceanic Technol., 25 , 1755–1767.
Gorgucci, E., and Chandrasekar V. , 2005: Evaluation of attenuation correction methodology for dual-polarization radars: Application to X-band systems. J. Atmos. Oceanic Technol., 22 , 1195–1206.
Gorgucci, E., Chandrasekar V. , and Baldini L. , 2006: Correction of X-band radar observation for propagation effects based on the self-consistency principle. J. Atmos. Oceanic Technol., 23 , 1668–1681.
Hitschfeld, W., and Bordan J. , 1954: Errors inherent in the radar measurement of rainfall at attenuating wavelengths. J. Meteor., 11 , 58–67.
McLaughlin, D., and Coauthors, 2009: Short-wavelength technology and the potential for distributed networks of small radar systems. Bull. Amer. Meteor. Soc., 90 , 1797–1817.
Park, S. G., Bringi V. N. , Chandrasekar V. , Maki M. , and Iwanami K. , 2005: Correction of radar reflectivity and differential reflectivity for rain attenuation at X-band. Part I: Theoretical and empirical basis. J. Atmos. Oceanic Technol., 22 , 1621–1632.
Srivastava, R. C., and Tian L. , 1996: Measurement of attenuation by a dual-radar method: Concept and error analysis. J. Atmos. Oceanic Technol., 13 , 937–947.
Testud, J., Amayenc P. , and Marzoug M. , 2000: The rain profiling algorithm applied to polarimetric weather radar. J. Atmos. Oceanic Technol., 17 , 322–356.
Zink, M., and Coauthors, 2005: Meteorological command and control: An end-to-end architecture for a hazardous weather detection sensor network. Proc. Workshop on End-to-End, Sense-and-Respond Systems, Applications, and Services, USENIX and ACM, 37–42.
Conceptual arrangement of a network radar environment.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
The configuration of the CASA IP1 network (KSAO, KCYR, KLWE, and KRSP) and WSR-88Ds (KTLX, KOUN, and KFDR). Here filled circles (•) indicate the locations of radars and the small gray and large gray circles depict 40- and 80-km ranges for CASA IP1 radars and WSR-88Ds, respectively.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
An architectural framework of real-time implementation of a network-based reflectivity retrieval system.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Illustration of data integration simultaneously from multiple radars.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Histogram of per-sweep execution run times for the case 1, 3, and 5 datasets.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Case of 10 Jun 2007. Composite of (a) observed reflectivity, (b) retrieved reflectivity by NBRR, (c) by CRP from CASA-IP1 radars (KCYR, KSAO, KRSP, and KLWE), and reflectivities from (d) KFDR, (e) KTLX, and (f) KOUN at 2 km MSL. KCYR, KLWE, KRSP, KSAO, KFDR, KTLX, and KOUN are located over a grid given by −13.8, −15.5, 15.5, 13.2, −80.5, 74.8, and 58.0 km in the east–west (x axis) direction and 5.2, −22.7, −1.6, 22.7, −51.3, 56.4, and 45.5 km in the north–south (y axis) direction from the center of the CASA network.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Case of 10 Jun 2007. Vertical structure of (a) retrieved reflectivity by NBRR and reflectivities from (b) KFDR, (c) KTLX, and (d) KOUN at x = −0.5 km from the center of the IP1 network.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Case of 10 Jun 2009 for the observed reflectivities from the following CASA IP1 radars: (a) KCYR, (b) KSAO, (c) KLWE, and (d) KRSP.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Case of 10 Jun 2009. Composite of (a) observed reflectivity, (b) retrieved reflectivity by NBRR, (c) retrieved reflectivity by CRP, and scatterplots of (d) measured reflectivity (Zh) vs specific differential phase (Kdp) and (e) retrieved reflectivity (Zh) by network-based retrieval algorithm vs specific differential phase (Kdp). The solid line shows the theoretical relationship between intrinsic Zh and Kdp.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Case of 24 May 2009. Composite of (a) observed reflectivity, (b) retrieved reflectivity by NBRR, (c) retrieved reflectivity by CRP, and scatterplots of (d) measured reflectivity (Zh) vs specific differential phase (Kdp) and (e) retrieved reflectivity (Zh) by network-based retrieval algorithm vs specific differential phase (Kdp). The solid line shows the theoretical relationship between intrinsic Zh and Kdp.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Case of 2 Jun 2009. Composite of (a) observed reflectivity, (b) retrieved reflectivity by NBRR, (c) retrieved reflectivity by CRP, and scatterplots of (d) measured reflectivity (Zh) vs specific differential phase (Kdp) and (e) retrieved reflectivity (Zh) by network-based retrieval algorithm vs specific differential phase (Kdp). The solid line shows the theoretical relationship between intrinsic Zh and Kdp.
Citation: Journal of Atmospheric and Oceanic Technology 28, 2; 10.1175/2010JTECHA1441.1
Description of case study datasets.
Average sweep generation intervals and the average per-sweep execution run times for the test cases.
Quantitative comparison of retrieved reflectivities by NBRR from CASA IP1 data and observed reflectivities from WSR-88Ds (KOUN, KTLX, and KFDR). Here, ΔZh (dBZ) is the difference between the retrieved reflectivity by NBRR and the observed reflectivities from WSR-88Ds [Zh(WSR − 88D) − Zh(NBRR) for Zh(NBRR) and Zh(WSR − 88D) > 20 dBZ].
Quantitative comparison of retrieved reflectivities by NBRR and CRP. Here, ΔZh is the difference between the retrieved reflectivity by NBRR and CRP [Zh(CPR) − Zh(NBRR)].
