1. Introduction
Quantitative precipitation estimation (QPE) using weather radar has become common and important for many meteorological and hydrological applications such as severe weather warnings and flood forecasting and management (e.g., Zhang et al. 2016; Krajewski et al. 2017). Since its initial deployment in the early 1990s, the QPE algorithm for the U.S. Weather Surveillance Radar-1988 Doppler (WSR-88D) network has evolved according to its hardware and polarimetric upgrades (e.g., Fulton et al. 1998; Istok et al. 2009; Wang et al. 2019). Numerous studies that utilized the dual-polarization (DP) capability have tested a variety of QPE algorithms based on different DP variables and demonstrated the improved accuracy of polarimetric estimates in different conditions and cases (e.g., Ryzhkov et al. 2005a; Wang and Chandrasekar 2010; Cifelli et al. 2011). However, there is still no assurance that such polarimetric estimation methods are consistently superior to the one based on the conventional reflectivity–rain rate relation (see, e.g., Seo et al. 2018).
Attenuation of a radar beam’s power along a propagation path is commonly considered a shortcoming for shorter wavelength radars (e.g., X band), necessitating a correction in rainfall estimation algorithms (e.g., Park et al. 2005; Gorgucci and Chandrasekar 2005). While the attenuation effect at longer wavelengths has been assumed to be negligible, Ryzhkov et al. (2014) demonstrated that rainfall estimation using specific attenuation at S band is less sensitive than that using other radar observables to the variability of drop size distributions (DSDs). Specific attenuation A, unlike radar reflectivity Z, is also immune to radar miscalibration and partial beam blockage (PBB). Because of these benefits, several recent studies (e.g., Cocks et al. 2019; Wang et al. 2019; Wolff et al. 2019; Zhang et al. 2020) have documented development and evaluation of the A-based QPE techniques, R(A), for S-band radar.
The Iowa Flood Center (IFC) generates statewide streamflow predictions based on a distributed hydrologic model known as the Hillslope Link Model, driven by a real-time radar-based precipitation product (see Krajewski et al. 2017). This product is a composite of seven WSR-88D radars covering Iowa and has time and space resolutions of five minutes and approximately 0.5 km, respectively. The product is not adjusted with rain gauge observations. Major QPE challenges that affect IFC’s streamflow prediction include 1) relative biases among different radars that occasionally lead to a noticeable discontinuity at the overlapping areas in the composite product (e.g., Seo et al. 2013) and 2) inconsistent rainfall estimation and its seasonal effect in a region covered by one of the radars arising from PBB at that location (e.g., Keem et al. 2019). To address these challenges, we have adapted and tested the S-band prototype R(A) algorithm documented in Ryzhkov et al. (2014) and Wang et al. (2019). In this paper, we describe the implementation of R(A) within our current real-time QPE framework based on R(Z) and comparatively assess the performance of the adapted R(A) and current R(Z) algorithms.
In section 2, we provide information on the spatial domain and data for WSR-88D radars, temperature sounding retrieved from the numerical weather prediction model analysis, and ground reference data used in this study. Section 3 describes our R(A) implementation and adaptation, as well as the strategies for its performance evaluation at multiple temporal scales. In section 4, we present the evaluation results on the accuracy improvements of R(A) and the uncertainty metrics of the algorithm. In section 5, we summarize our main findings, discuss remaining challenges, and present ideas on how to further improve the performance of R(A).
2. Study domain and data
We apply the R(A) algorithm to the data collected from the WSR-88D radars covering the Iowa domain where the IFC performs extensive flood-related research activities including operational weather monitoring and streamflow forecasting (see Krajewski et al. 2017). As illustrated in Fig. 1, we selected three radars located in Des Moines, Iowa (KDMX); Sioux Falls, South Dakota (KFSD); and La Crosse, Wisconsin (KARX) and processed their observations for the domain within 200 km to produce hourly rainfall estimates. The U.S. National Weather Service (NWS) completed the DP upgrade of the WSR-88D network in 2014, and the DP variables for the three radars shown in Fig. 1 are available from 2013.
The locations of three WSR-88D radars and rain gauges consisting of the COOP and the IFC networks. The circular areas centered on the radar sites (gray dots) indicate the 200-km domain of individual radars. The white solid line represents the boundary of the Turkey River basin in Iowa.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
The locations of three WSR-88D radars and rain gauges consisting of the COOP and the IFC networks. The circular areas centered on the radar sites (gray dots) indicate the 200-km domain of individual radars. The white solid line represents the boundary of the Turkey River basin in Iowa.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
The locations of three WSR-88D radars and rain gauges consisting of the COOP and the IFC networks. The circular areas centered on the radar sites (gray dots) indicate the 200-km domain of individual radars. The white solid line represents the boundary of the Turkey River basin in Iowa.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
a. WSR-88D data
We collected radar Level II volume data (e.g., Kelleher et al. 2007) from Amazon’s big data archive (Ansari et al. 2018) using a data acquisition tool implemented in a system that provides a radar QPE product for a customized space–time domain (Seo et al. 2019). We acquired the KARX data for 1 May–15 June 2013 because of the presence of reliable hourly rain gauge data within the Turkey River basin (see Fig. 1) during this period, which allows us to scrutinize the performance of R(A). The data collection from the KDMX and KFSD radars covers the months of April–October for 2015–17. In this 3-yr period, we excluded cold months because the R(A) method is valid only for liquid precipitation, and the low-altitude freezing level often present in cold months significantly limits the application range of R(A).
The Level II data contain three DP variables: differential reflectivity Zdr, copolar correlation coefficient ρhv, and differential phase ϕdp. Because unprocessed ϕdp in Level II data is noisy and is wrapped to values ranging from 0° to 360°, a procedure to unwrap and smooth these observations is required (e.g., Wang and Chandrasekar 2009). We unwrapped the phase data to values ranging from 0° to 1080° (360° × 3) and then applied smoothing filters. We used the unwrapping (unfolding) and smoothing routines described in the WSR-88D Common Operations and Development Environment (CODE) public package (https://www.weather.gov/code88d/). These routines are commonly used to derive specific differential phase Kdp from ϕdp. We also applied an averaging filter along a radial direction for Z and other DP fields with a specific size of averaging window represented by the number of gates.
b. Temperature sounding data
To define a maximum range to which R(A) is applicable, we retrieved the vertical temperature profiles from the Rapid Refresh (RAP) model analysis (Benjamin et al. 2016). The RAP is a continental-scale, hourly-updated assimilation and model forecast system based on a 13-km resolution horizontal grid. We created a lookup table that defines a corresponding horizontal RAP grid to each radar polar pixel (0.5° by 250 m) to construct three-dimensional temperature information within a radar domain. We then determined the melting layer (ML) altitude and valid ranges for the R(A) application (e.g., below the ML) by checking if the temperature at the top of a radar beam at a given location represented by radar range, azimuth, and elevation angle is greater than 5°C. We applied this procedure to all elevation angles in the Level II data, which enabled us to combine the R(A) estimates from multiple elevation angles.
c. Rain gauge data
We collected rain gauge observations from two different ground networks to evaluate the R(A) estimates from the three radars shown in Fig. 1. The IFC has operated a dense network consisting of 20 double tipping-bucket rain gauges within the Turkey River basin in Iowa (see Fig. 1). This network was deployed as part of the NASA field campaign known as Iowa Flood Studies (e.g., Seo et al. 2018) for the period of 1–15 June 2013. Because the IFC staff and students regularly visited and maintained the gauges during the campaign period to assure the quality of the collected data, we used their hourly observations to assess detailed aspects of R(A) performance. The data quality control procedure of these gauges is documented in Seo et al. (2015a).
For a spatially extensive evaluation of the capability of R(A) (e.g., for radar miscalibration and PBB effects), we also acquired the NWS Cooperative Observer Program (COOP; Mosbacher et al. 1989) rain gauge data for 2015–17. These gauges are well distributed over the study domain as shown in Fig. 1, and there are about 100 gauges within each individual domain of the KDMX and KFSD radars. Because of the well-known issue with these daily gauge totals [see, e.g., Seo et al. (2013) for the report timing error], we accumulated the collected data over time for analyses at longer time scales (e.g., monthly and yearly).
3. Methodology
In this section, we briefly describe estimation procedures associated with the R(A) approach and provide relevant formulas to derive the central elements and parameters of the approach. Our R(A) implementation is based on the S-band prototype algorithm documented in Ryzhkov et al. (2014) and Wang et al. (2019), and we discuss the practical adaptation of the prototype algorithm within our existing QPE framework (e.g., Krajewski et al. 2017). This section also outlines testing and evaluation strategies for the estimation of a key parameter in the R(A) procedures and for the performance of R(A) compared to that of the Z-based conventional estimation.
a. Specific attenuation estimation
b. Precipitation estimation
After applying Eqs. (9)–(11) to the observed data and generating rain rate maps at the several lowest elevation angles, we combine these rain rate estimates using a linear weighting scheme. A nonparametric kernel function (e.g., lognormal) assigns a greater weight to the elevation angle data closer to a predefined constant altitude (e.g., 1.5 km). This hybrid scan approach tends to mitigate significant ground clutter effects at the ranges close to the radar site (e.g., Fulton et al. 1998; Seo et al. 2011). We eliminated the nonmeteorological radar echoes (e.g., ground clutter, anomalous propagation, and wind farm effects) using a combination of data quality control methods described in Seo et al. (2015b, 2020).
c. Performance evaluation
We evaluate the R(A) performance in two different ways using different datasets: 1) the KARX radar and hourly rain gauge data collected over the Turkey River basin and 2) the KDMX and KFSD radar data and daily reports from the COOP rain gauges extensively distributed over the Iowa domain. The first evaluation aims to determine an effective α estimation method by testing Eqs. (6)–(8) and a suitable Zdr–Z sample size for the Zdr slope calculation that can lead to robust α and following rainfall estimation. To minimize the effect of solid and mixed precipitation, we included liquid precipitation estimates only in the evaluation using the retrieved ML information. In other words, we excluded hourly radar–gauge (RG) pairs from the analysis if radar estimates at any time stamps within an hourly span are contaminated by the ML. This hourly assessment also inspects the detailed performance of R(A) with respect to many different rainfall events during the 6-week period, which reveals event-dependent performances. For some event cases that show relatively weak performance (over- or underestimation), we examine a probable uncertainty factor associated with the Zdr–Z relation and compare the relation with the one corresponding to the cases with good performance.
The second evaluation, using multiple-year data based on the sample size and the α estimation equation selected from the first evaluation, provides comprehensive analyses to demonstrate the improvements and practical challenges of R(A). Unlike the first evaluation, these analyses include estimates from solid and mixed precipitation. As such, there should be clear effects of those precipitation estimates on the performance of our R(A) algorithm selectively combined with R(Z) in Eq. (10). Because these effects can be revealed well in climatological tendency at different ranges, we compare the range-dependent feature from our R(A) algorithm with that of conventional R(Z) which typically exposes a climatological pattern associated with the ML (e.g., Krajewski et al. 2011). We also assess the well-known improvements achievable by R(A) concerning the radar miscalibration and PBB effects and discuss the practical limitation for this improvement.
4. Results
In this section, we use hourly radar estimates and gauge observations to test and find the optimal α estimation equation and Zdr–Z sample size for robust α estimation using the data from the KARX radar and the IFC rain gauges shown in Fig. 1. Using these decisions regarding the α estimation, we then evaluate the performance of R(A) estimates generated at multiple temporal scales. For a shorter-scale evaluation, we use the same hourly data as used in the determination procedure for α estimation. On the other hand, a longer-scale analysis evaluates the overall and range-dependent aspects of R(A) performance using the multiple-year data collected from the KDMX and KFSD radars and daily COOP rain gauges.
a. Determination for α estimation
Figure 2 shows the RG comparison results regarding R(Z) and three R(A) estimates using the different α equations provided in Eqs. (6)–(8). The dots in Fig. 2 indicate hourly RG pairs for the 20 rain gauge locations together, and Fig. 2 does not include the pairs affected by the ML during the period of 1 May–15 June 2013. In Fig. 2, the R(Z) estimates show highly scattered distribution while the R(A) estimates demonstrate significant improvements in terms of scatter and agreement with gauge observations. Of the three α estimation equations, the result using Eq. (6) seems slightly better than the one using Eq. (7) based on the statistical metrics, and the Eq. (8) based estimation yielded considerable systematic overestimations. The observed tendency of about 25% overestimation of the two R(A) estimates using Eqs. (6) and (7) tends to depend on the representativeness of estimated α with respect to precipitation systems that passed over the rain gauge locations. We will discuss this overestimation issue further in this section employing two example events.
Hourly RG comparison results for R(Z) and three R(A) estimates using the data from the KARX radar and rain gauges within the Turkey River basin in Iowa. The three R(A) estimates were generated using different α estimation equations provided in Eqs. (6)–(8). The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, MAE, and RMSE. The number of nonzero RG pairs in the comparisons is about 1030 for the period of 1 May–15 June 2013.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Hourly RG comparison results for R(Z) and three R(A) estimates using the data from the KARX radar and rain gauges within the Turkey River basin in Iowa. The three R(A) estimates were generated using different α estimation equations provided in Eqs. (6)–(8). The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, MAE, and RMSE. The number of nonzero RG pairs in the comparisons is about 1030 for the period of 1 May–15 June 2013.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Hourly RG comparison results for R(Z) and three R(A) estimates using the data from the KARX radar and rain gauges within the Turkey River basin in Iowa. The three R(A) estimates were generated using different α estimation equations provided in Eqs. (6)–(8). The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, MAE, and RMSE. The number of nonzero RG pairs in the comparisons is about 1030 for the period of 1 May–15 June 2013.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Based on the results shown in Fig. 2, we decided to use Eq. (6) in estimating α for the rest of the analyses in this study. To test the effect of the Zdr–Z sample size on the α estimation and following QPE accuracy, we performed a simple sensitivity analysis using a variety of sample sizes. The sensitivity analysis aims to reveal how the accuracy of R(A) estimates changes with respect to α derived from the collected Zdr–Z samples that meet the requirement for the minimum number of samples ranging from 3000 to 50 000. If the number of samples did not meet this requirement at a certain (radar volume) observation time, we excluded the corresponding hour from the RG analysis. In the sensitivity analysis, the RG analysis hours decrease (see Table 1) as the minimum number of samples increases (e.g., the requirement becomes stricter). This is because of the increasing hours containing a smaller number of Zdr–Z pairs than the required sample size, which are excluded from the metrics calculation. The analysis results are presented in Table 1, and Fig. 3 illustrates the change of calculated statistical metrics and RG analysis hours for different Zdr–Z sample sizes. In Fig. 3, the change of bias and correlation does not seem to be sensitive to the number of Zdr–Z samples, at least for the tested sample sizes in Table 1. Therefore, we determined the required number of Zdr–Z samples in our algorithm implementation to be the smallest sample size used in Table 1 (i.e., 3000). We note that MAE/RMSE and their normalized values in Table 1 behave oppositely with respect to the sample size, and this is because of the increasing gauge mean as the required sample size becomes greater.
Sensitivity analysis results of R(A) estimates with respect to the number of Zdr–Z samples. The sensitivity was analyzed based on the hourly RG evaluation. The statistical metrics provided in the table are the multiplicative bias B, correlation coefficient r, MAE, and RMSE. NMAE and NRMSE indicate normalized percentage (%) using the gauge mean value. The definitions of these metrics are presented at the end of section 3.
The change of RG bias, correlation, and sample hours with respect to the minimum number of Zdr–Z samples required for α estimation.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
The change of RG bias, correlation, and sample hours with respect to the minimum number of Zdr–Z samples required for α estimation.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
The change of RG bias, correlation, and sample hours with respect to the minimum number of Zdr–Z samples required for α estimation.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
b. Shorter-scale evaluation
Over the Iowa domain shown in Fig. 1, the selected period from 1 May through 15 June 2013 was fairly wet, containing many different rain events characterized by convective/stratiform or heavy/light rain. By inspecting rain gauge records and radar maps acquired from the field experiment (e.g., Seo et al. 2018), we selected 12 rain events that were not affected by the low-level ML and snow cases. Table 2 lists their individual periods, maximum rainfall intensity, and RG evaluation metrics for R(A) and R(Z). The performance metrics associated with the maximum intensity indicate that there is still a challenge for both R(A) and R(Z) in relatively light rain cases (e.g., maximum intensity < 10.0 mm h−1). Because our R(A) implementation applies R(Z) to the rain segments where the total ϕdp change along a radial direction is not sufficient (<3°), the performance of R(A) does not seem to be better than that of R(Z) for the light rain cases in Table 2. Figure 2 demonstrates the overall improvements of R(A) during the entire 6-week period, and the performance metrics for most intense rain cases in Table 2 confirm the superiority of R(A) to R(Z).
List of selected rain events during the period of 1 May–15 June 2013 and hourly RG evaluation results quantified by the statistical metrics.
We chose four intense rain events from Table 2 as representative cases and illustrate their scatterplots in Fig. 4 to scrutinize the detailed aspects of case-dependent R(A) performance. As shown in Fig. 4, the agreement between the hourly RG pairs seems much better for R(A) than for R(Z), and the statistical metrics (e.g., r, MAE, and RMSE) from all four cases verify that R(A) significantly outperforms our current R(Z) QPE except for the overestimation tendency observed, particularly from event 3. To investigate this overestimation tendency, we present the temporal variation of estimated α and bias, two instantaneous base reflectivity maps within the hours of interest (e.g., based on bias), and the estimated Zdr slope and α at the two instances in Fig. 5. As shown in Fig. 5, the bias change implies consistent overestimations over the entire event, and we selected two times indicated by the red and blue dots that recorded the highest and lowest biases. The reflectivity maps in Fig. 5 show the Zdr–Z sampling range represented by the black circle and the spatial pattern of precipitation within the Turkey River basin boundaries (white solid line) where the rain gauges are located. From these reflectivity maps, we can observe the differences of precipitation patterns between the sampling range and the basin area, which primarily cause the estimation uncertainty of α. Because we assume that an estimated α value from the sampling range is homogenous, the heterogeneity of precipitation systems within a radar domain often leads to under- or overestimation depending on the situation (Wang et al. 2019; Cocks et al. 2019). In the reflectivity map at 1413 UTC, the estimated α value (0.0212 dB per degree) was not close to the one for intense rain (e.g., typically 0.0150 dB per degree for convective rain), although several convective systems passed over the basin between 1400 and 1500 UTC. Therefore, the use of greater α yielded the significant overestimation observed for this hour. We confirmed this overestimation tendency from the hourly RG values (i.e., 11.6 versus 6.6 mm) at one of the rain gauge locations on the path of intense rainfall systems. The second reflectivity map at 1639 UTC shows that the estimated α (0.0174 dB per degree) is closer to the convective one, which mitigated the degree of overestimation for the intense rain area that is relatively smaller compared to the case at 1413 UTC. We also confirmed that there were some underestimations at the rain gauge locations indicated by yellow and green areas because the α value used was smaller than the typical one for light rain (e.g., 0.0350 dB per degree for stratiform rain). However, we found that the degree of underestimation tended to be much smaller than that of overestimation, which often hides (or dominates) the underestimation tendency in mesoscale convective systems (MCS) where the convective and stratiform systems exist together. This is somewhat consistent with the fact that the bias ratios of R(A) shown in Tables 1 and 2 and Figs. 2 and 4 are all greater than unity.
Scatterplots of hourly RG comparisons for four intense rain cases in Table 2 to evaluate the improvements of R(A) vs R(Z). The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, MAE, and RMSE. The numbers of nonzero RG pairs included in the comparisons are 131, 83, 135, and 84 for events 1, 3, 5, and 11, respectively.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Scatterplots of hourly RG comparisons for four intense rain cases in Table 2 to evaluate the improvements of R(A) vs R(Z). The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, MAE, and RMSE. The numbers of nonzero RG pairs included in the comparisons are 131, 83, 135, and 84 for events 1, 3, 5, and 11, respectively.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Scatterplots of hourly RG comparisons for four intense rain cases in Table 2 to evaluate the improvements of R(A) vs R(Z). The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, MAE, and RMSE. The numbers of nonzero RG pairs included in the comparisons are 131, 83, 135, and 84 for events 1, 3, 5, and 11, respectively.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Temporal variation of estimated α and bias, two instantaneous base reflectivity maps within the hours of interest, and the estimated Zdr slope and α at the two instances for event 3. The black circle in the reflectivity maps indicates the Zdr–Z sampling range, and the white solid line represents the boundary of the Turkey River basin where the rain gauges are located.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Temporal variation of estimated α and bias, two instantaneous base reflectivity maps within the hours of interest, and the estimated Zdr slope and α at the two instances for event 3. The black circle in the reflectivity maps indicates the Zdr–Z sampling range, and the white solid line represents the boundary of the Turkey River basin where the rain gauges are located.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Temporal variation of estimated α and bias, two instantaneous base reflectivity maps within the hours of interest, and the estimated Zdr slope and α at the two instances for event 3. The black circle in the reflectivity maps indicates the Zdr–Z sampling range, and the white solid line represents the boundary of the Turkey River basin where the rain gauges are located.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Figure 6 illustrates the same analysis as Fig. 5 for event 5. We selected this case because it illustrates the greatest improvement of the R(A) estimator. The event is characterized by MCS where a widespread stratiform system follows a strong convective line as shown in the reflectivity maps of Fig. 6. The first reflectivity map at 2024 UTC 29 May demonstrates that there is no contribution of stratiform rain in both the sampling range and the basin area. Therefore, the estimated α value (0.0146 dB per degree) was close to that of a convective α value, which led to a quite accurate rainfall estimation (the bias ratio for this hourly window was about 1.0) for the basin area. On the other hand, the α value (0.0190 dB per degree) for the second map at 0048 UTC 30 May was greater than the α value at 2024 UTC 29 May due to the contribution of a substantial portion of stratiform rain in the sampling range. This caused similar overestimations to the cases shown in Fig. 5 for the convective areas within the basin in Fig. 6. The RG values between 0000 and 0100 UTC 30 May at two locations in the middle of basin where convective systems passed were 12.4 versus 7.0 and 18.6 versus 8.4 mm, respectively. We also inspected other events (e.g., events 8 and 10) in which R(A) showed worse performance than R(Z) and confirmed that the observed overestimations of R(A) were caused by the use of uniform α as illustrated in Figs. 5 and 6.
As in Fig. 5, but for event 5.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
As in Fig. 5, but for event 5.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
As in Fig. 5, but for event 5.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
c. Longer-scale evaluation
The evaluation in this section focuses on the overall performance of R(A) at longer time scales (e.g., yearly rain totals from April to October) and its well-known immunity to radar miscalibration and partial beam blockage effects (e.g., Ryzhkov et al. 2014). We clarify that the R(A) estimates presented in this section were generated using a combination of R(A) for liquid precipitation and multiple R(Z)s for solid/mixed precipitation. We used 3-yr data for the KDMX and KFSD radars and the daily COOP rain gauges for the evaluation. The coverage of the two radars and the spatial distribution of the COOP gauges within each radar domain are shown in Fig. 1. As we stated that our current R(Z) approach for KFSD has yielded significant underestimations due to the PBB effect, we demonstrate the capability of R(A) regarding PBB and some practical challenges through this evaluation.
We generated KDMX and KFSD radar estimates for the years of 2015–17 using Eq. (9) for liquid precipitation and Eq. (10) for solid and mixed precipitation. The RG evaluation results for R(Z) and R(A) at rain total scale accumulated for April through October are compared in Fig. 7. In Fig. 7, the gray and orange dots denote R(Z) and R(A) estimates, respectively. We recognize from Fig. 7 that the utilization of R(A) results in the mitigation of the opposite tendencies of overestimation and underestimation exhibited by the R(Z) algorithm both for the KDMX and KFSD radars. Moreover, the scatters of R(A) are considerably reduced compared to those of R(Z), and the statistical metrics presented in the scatterplots confirm better agreement of the R(A) estimates with the ground reference data. Figure 8 shows the same analysis as Fig. 7, but with a different representation to expose range-dependent systematic tendencies of both estimates. The R(Z) for KDMX in Fig. 8 shows a typical signature of reflectivity-based estimates affected by the radar miscalibration and ML effects. The shift of gray dots above unity (red solid line) within 100 km for KDMX indicates systematic overestimation caused by radar miscalibration, and the bump observed at the 100–150-km range exhibits the impact of the ML (e.g., Krajewski et al. 2011). On the other hand, R(A) estimates for both radars in Fig. 8 are much closer to and better aligned along the unity line, indicating that the effect of radar miscalibration observed in R(Z) is almost eliminated. The statistical metrics provided in Fig. 8 were calculated for the range within 100 km to exclude the ML effect and attest to the superiority of R(A) with respect to R(Z). It becomes clear from Fig. 8 that quite a few of R(A)’s overestimations (KDMX) and underestimations (KFSD) shown in Fig. 7 were from far ranges for both radars. This represents a practical limitation of R(A) application because the R(A) algorithm is valid only for pure rain for the region below the ML.
Three-year RG comparison for the KDMX and KFSD radars at rain total scale over the period of April–October each year. The gray and orange dots denote R(Z) and R(A) estimates, respectively. The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, and NMAE. The same color code was applied to the statistical metrics provided in each scatterplot.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Three-year RG comparison for the KDMX and KFSD radars at rain total scale over the period of April–October each year. The gray and orange dots denote R(Z) and R(A) estimates, respectively. The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, and NMAE. The same color code was applied to the statistical metrics provided in each scatterplot.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Three-year RG comparison for the KDMX and KFSD radars at rain total scale over the period of April–October each year. The gray and orange dots denote R(Z) and R(A) estimates, respectively. The statistical metrics provided in the figure are the multiplicative bias B, correlation coefficient r, and NMAE. The same color code was applied to the statistical metrics provided in each scatterplot.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Range-dependent bias feature for the KDMX and KFSD radars. The color code is the same as in Fig. 7. The statistical metrics provided in each scatterplot were calculated for the range within 100 km to exclude the ML effect.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Range-dependent bias feature for the KDMX and KFSD radars. The color code is the same as in Fig. 7. The statistical metrics provided in each scatterplot were calculated for the range within 100 km to exclude the ML effect.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Range-dependent bias feature for the KDMX and KFSD radars. The color code is the same as in Fig. 7. The statistical metrics provided in each scatterplot were calculated for the range within 100 km to exclude the ML effect.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
To examine KFSD’s PBB effect, we present the rainfall accumulation maps for the entire month of September 2016 in Fig. 9. In Fig. 9, we can detect clear PBB effect in the two inset areas labeled (a) and (b) of the R(Z) accumulation map, whereas the R(A) map shows much alleviated PBB patterns for the two areas. In the R(A) map of Fig. 9, the weak PBB effect still appears at the range between 100 and 150 km in both (a) and (b) areas. This can occur when the ML is located lower than an altitude corresponding to 150-km range, or when the total span of ϕdp over a rain segment at this range is smaller than 3° (e.g., light rain). We quantified the PBB effect in Fig. 10 based on RG comparison for the rain gauge locations in part of the Iowa domain where the PBB effect is significant. The selected rain gauges (black dots) for the PBB evaluation are shown in Fig. 9. Figure 10 demonstrates that the significant underestimations by PBB in R(Z) were improved in R(A), and the R(A) estimates within 150 km seem to agree with ground observations. The R(A) estimates over 150 km show some consistency with the ones at similar ranges in Fig. 8 because of the ML effect. We note that the example case for September 2016 is a good instance to demonstrate the immunity of R(A) to the PBB effect, and many other months when light rain occurred frequently might not show this improvement for PBB due to the requirement of ϕdp change.
Rainfall accumulation maps generated using R(Z) and R(A) algorithms for the entire month of September 2016. Two inset areas (a) and (b) indicate the regions where the PBB effects are prominent in R(Z). The range rings in the maps demarcate every 50 km from the radar site. The black dots represent the rain gauge locations used for the quantitative evaluation of PBB effect in Fig. 10.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Rainfall accumulation maps generated using R(Z) and R(A) algorithms for the entire month of September 2016. Two inset areas (a) and (b) indicate the regions where the PBB effects are prominent in R(Z). The range rings in the maps demarcate every 50 km from the radar site. The black dots represent the rain gauge locations used for the quantitative evaluation of PBB effect in Fig. 10.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Rainfall accumulation maps generated using R(Z) and R(A) algorithms for the entire month of September 2016. Two inset areas (a) and (b) indicate the regions where the PBB effects are prominent in R(Z). The range rings in the maps demarcate every 50 km from the radar site. The black dots represent the rain gauge locations used for the quantitative evaluation of PBB effect in Fig. 10.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
RG comparison (September 2016) for the PBB area in the KFSD domain. The assessment locations are indicated as black dots in Fig. 9.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
RG comparison (September 2016) for the PBB area in the KFSD domain. The assessment locations are indicated as black dots in Fig. 9.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
RG comparison (September 2016) for the PBB area in the KFSD domain. The assessment locations are indicated as black dots in Fig. 9.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
5. Summary and discussion
We examined the performance of the S-band prototype specific attenuation algorithm (e.g., Wang et al. 2019) implemented in the IFC’s existing QPE framework (e.g., Krajewski et al. 2017) with some practical adaptations of the prototype algorithm. The study aims to improve understanding of the new algorithm’s strengths and weaknesses in its practical (e.g., operational) implementation motivated by real-time streamflow forecasting. There are several novel aspects of this study compared to the earlier ones (e.g., Wang et al. 2019; Cocks et al. 2019; Zhang et al. 2020). These include 1) the evaluation of multiple α estimation equations; 2) using actual example cases to demonstrate how the use of uniform α generates QPE errors with their temporal aspects (i.e., from hour to hour); and 3) investigating the impact of reduced number of Zdr–Z samples (e.g., 3000 versus 30 000) on the QPE accuracy for future improvement of α estimation.
We comparatively evaluated the performance of R(A) with that of our conventional estimation based on R(Z) at multiple temporal scales. For a shorter-scale evaluation, we used a high-density, high-quality rain gauge network deployed within the Turkey River basin in Iowa (see Fig. 1) as ground reference. Using this hourly dataset, we tested three α estimation equations offered in Eqs. (6)–(8) and selected Eq. (6), a quadratic function of the estimated Zdr slope, as our estimation method. We note that Eq. (8) represented as a function of the Zdr slope and maximum reflectivity led to systematic QPE overestimations, although its RG agreement and scatter (Fig. 2) look better than those of R(Z). We also inspected the impact of Zdr–Z sample size on the α estimation and subsequent QPE accuracy because it is often hard to ensure the huge number of Zdr–Z samples (e.g., 30 000) recommended in Wang et al. (2019) depending on the spatial coverage of precipitation in the radar domain. The analysis results presented in Table 1 and Fig. 3 indicated that the QPE accuracy does not seem to be sensitive to the number of samples tested (ranging from 3000 to 50 000). Therefore, we selected 3000 as a requirement for our R(A) algorithm to avoid the use of default (fixed) α as much as possible.
Based on the determination of factors regarding α estimation, we performed an hourly RG analysis for the 12 selected rain events (Table 2) that occurred during the period of 1 May–15 June 2013. The results revealed significant accuracy improvements in the R(A) estimates compared to R(Z) for intense rain cases, while several light rain cases remain a challenge for both R(A) and R(Z). [Our R(A) algorithm applies R(Z) for light rain cases where the total span of ϕdp along a radial direction is smaller than 3°.] Despite R(A)’s superiority in comparison with R(Z), we found that R(A) consistently leads to some degree of overestimation, for example, about 10%–30% based on the cases presented in Fig. 4. This overestimation was due to the heterogeneity of precipitation systems between the sampling and evaluation regions for α estimation and application, respectively. For instance, the use of α (>0.0150 dB per degree) estimated from mixed precipitation systems (convective + stratiform) in a whole radar coverage domain tends to overestimate convective rain with higher reflectivity in the evaluation area for which α should be close to the typical convective value (0.0150 dB per degree). Although we also observed the opposite tendency (e.g., underestimation of rainfall in the stratiform parts of radar echo), the degree of underestimation was not as large as that of overestimation in the mixed precipitation systems (e.g., MCS).
Our multiyear evaluation employed daily rain gauges (COOP) extensively distributed over the Iowa domain, which allowed us to investigate the range-dependent bias pattern in the KDMX and KFSD radar estimates. The results also confirm the advantages of R(A): R(A) from both radars better agreed with ground observations without radar miscalibration and PBB effects, at least within their 100-km range, whereas R(Z) showed clear PBB and climatological ML effects as well as a systematic shift. With regard to the range over 100 km, the QPE accuracy of R(A) seems to decrease because our R(A) algorithm selectively uses R(Z) provided in Eq. (10) for solid and mixed precipitation (depending on the ML location). This decrease is larger for KFSD (Fig. 8) due to the use of R(Z) at this range and a more pronounced PBB effect. However, the low accuracy at this range could be improved by combining estimates from multiple radars with overlapping coverage areas in the PBB region. Figure 11 demonstrates the potential accuracy improvements by the composite R(A) product using adjacent radars. We note the shift of underestimated values toward the unity line as a result of merging multiple radar data. The composite product in Fig. 11 was generated for the IFC’s Iowa domain using seven WSR-88D radars (see Krajewski et al. 2017). The composite product analysis used rain gauges located in each intersection area between the Iowa and individual radar domains (see Fig. 1), which led to the different numbers of dots for the KDMX and KFSD analyses presented in Fig. 11. Since we have recently implemented the R(A) algorithm in our real-time operation to generate a statewide QPE product, we will perform more a comprehensive evaluation and document the results soon. The evaluation will include a comparison with the gauge-corrected product, the Multi-Radar Multi-Sensor (MRMS) estimates (Zhang et al. 2016), although our operation uses a radar-only product.
Comparison of the range-dependent biases between individual and composite R(A) estimates. The number of dots (rain gauge locations) shown in the KFSD analysis is smaller than that in Fig. 8 because the composite product covers the Iowa domain only.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Comparison of the range-dependent biases between individual and composite R(A) estimates. The number of dots (rain gauge locations) shown in the KFSD analysis is smaller than that in Fig. 8 because the composite product covers the Iowa domain only.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
Comparison of the range-dependent biases between individual and composite R(A) estimates. The number of dots (rain gauge locations) shown in the KFSD analysis is smaller than that in Fig. 8 because the composite product covers the Iowa domain only.
Citation: Journal of Hydrometeorology 21, 6; 10.1175/JHM-D-20-0030.1
We note that the most crucial element in the R(A) procedure is the factor α. As demonstrated in Figs. 5 and 6, the uncertainties in α affect QPE accuracy significantly, arising from the use of uniform α over the entire radar domain. Perhaps a simple, but effective way to reduce this uncertainty might be to split rainfall regions into two regimes (convective and stratiform) and estimate α separately for each. This approach would be feasible with a relatively smaller number of Zdr–Z samples as revealed in our sensitivity analysis (Fig. 3). In the near future, we hope to report the result of the suggested approach in conjunction with our precipitation classification scheme documented in Seo et al. (2020). We also note that the requirement of the ϕdp change (e.g., >3°) is a challenge for R(A) application in light rain cases, as the performance of other DP methods (e.g., Cifelli et al. 2011; Chen and Chandrasekar 2015) demonstrated for those cases.
The R(A) implementation in the IFC operation and its evaluation were motivated by improved streamflow forecasting for Iowa. A comprehensive hydrologic evaluation should include a wide range of basin scales because the errors in QPE may propagate differently at different basin scales. This requires more extensive data processing (e.g., generation of a composite product) for a larger spatial domain than the ones used in this study (e.g., individual radars) and wide-ranging streamflow simulations and analyses. Therefore, we do not include those efforts in our current work but plan to conduct a large-scale hydrologic evaluation by inspecting complicated aspects associated with the spatial and temporal structure of the QPE error at numerous basin scales. We also hope to report the evaluation results soon.
Acknowledgments
This study was supported by the Iowa Flood Center at the University of Iowa. We thank Ricard Murnan at the NWS Radar Operations Center for providing guidance to process the WSR-88D’s differential phase data. We are grateful to Amazon Web Services for allowing us to access the historic archive of WSR-88D Level II data used for rainfall product generation in this study.
REFERENCES
Ansari, S., and Coauthors, 2018: Unlocking the potential of NEXRAD data through NOAA’s big data partnership. Bull. Amer. Meteor. Soc., 99, 189–204, https://doi.org/10.1175/BAMS-D-16-0021.1.
Benjamin, S. G., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.
Chen, H., and V. Chandrasekar, 2015: Estimation of light rainfall using Ku-band dual-polarization radar. IEEE Trans. Geosci. Remote Sens., 53, 5197–5208, https://doi.org/10.1109/TGRS.2015.2419212.
Cifelli, R., V. Chandrasekar, S. Lim, P. C. Kennedy, Y. Wang, and S. A. Rutledge, 2011: A new dual-polarization radar rainfall algorithm: Application in Colorado precipitation events. J. Atmos. Oceanic Technol., 28, 352–364, https://doi.org/10.1175/2010JTECHA1488.1.
Cocks, S. B., and Coauthors, 2019: A prototype quantitative precipitation estimation algorithm for operational S-band polarimetric radar utilizing specific attenuation and specific differential phase. Part II: Performance verification and case study analysis. J. Hydrometeor., 20, 999–1014, https://doi.org/10.1175/JHM-D-18-0070.1.
Fulton, R. A., J. P. Breidenbach, D.-J. Seo, D. A. Miller, and T. O’Bannon, 1998: The WSR-88D rainfall algorithm. Wea. Forecasting, 13, 377–395, https://doi.org/10.1175/1520-0434(1998)013<0377:TWRA>2.0.CO;2.
Gorgucci, E., and V. Chandrasekar, 2005: Evaluation of attenuation correction methodology for dual-polarization radars: Application to X-band systems. J. Atmos. Oceanic Technol., 22, 1195–1206, https://doi.org/10.1175/JTECH1763.1.
Hubbert, J. C., M. Dixon, and S. Ellis, 2009: Weather radar ground clutter. Part II: Real-time identification and filtering. J. Atmos. Oceanic Technol., 26, 1181–1197, https://doi.org/10.1175/2009JTECHA1160.1.
Istok, M., and Coauthors, 2009: WSR-88D dual polarization initial operational capabilities. Preprints, 25th Conf. on International Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Phoenix, AZ, Amer. Meteor. Soc., 15.5, https://ams.confex.com/ams/89annual/techprogram/paper_148927.htm.
Keem, M., B.-C. Seo, W. F. Krajewski, and K. R. Morris, 2019: Inter-comparison of reflectivity measurements between GPM DPR and NEXRAD radars. Atmos. Res., 226, 49–65, https://doi.org/10.1016/j.atmosres.2019.04.010.
Kelleher, K. E., and Coauthors, 2007: A real-time delivery system for NEXRAD Level II data via the internet. Bull. Amer. Meteor. Soc., 88, 1045–1058, https://doi.org/10.1175/BAMS-88-7-1045.
Krajewski, W. F., B. Vignal, B.-C. Seo, and G. Villarini, 2011: Statistical model of the range-dependent error in radar-rainfall estimates due to the vertical profile of reflectivity. J. Hydrol., 402, 306–316, https://doi.org/10.1016/j.jhydrol.2011.03.024.
Krajewski, W. F., and Coauthors, 2017: Real-time flood forecasting and information system for the State of Iowa. Bull. Amer. Meteor. Soc., 98, 539–554, https://doi.org/10.1175/BAMS-D-15-00243.1.
Morin, E., and M. Gabella, 2007: Radar-based quantitative precipitation estimation over Mediterranean and dry climate regimes. J. Geophys. Res., 112, D20108, https://doi.org/10.1029/2006JD008206.
Mosbacher, R., W. E. Evans, and E. W. Friday Jr., 1989: Cooperative station observations. National Weather Service Observing Handbook No. 2, NWS, 83 pp.
Park, S.-G., M. Maki, K. Iwanami, V. N. Bringi, and V. Chandrasekar, 2005: Correction of radar reflectivity and differential reflectivity for rain attenuation at X band. Part II: Evaluation and application. J. Atmos. Oceanic Technol., 22, 1633–1655, https://doi.org/10.1175/JTECH1804.1.
Ryzhkov, A. V., M. Diederich, P. Zhang, and C. Simmer, 2014: Potential utilization of specific attenuation for rainfall estimation, mitigation of partial beam blockage, and radar networking. J. Atmos. Oceanic Technol., 31, 599–619, https://doi.org/10.1175/JTECH-D-13-00038.1.
Ryzhkov, A. V., and D. S. Zrnic, 2019: Radar Polarimetry for Weather Observations. Springer, 486 pp.
Ryzhkov, A. V., S. E. Giangrande, and T. J. Schuur, 2005a: Rainfall estimation with a polarimetric prototype of WSR-88D. J. Appl. Meteor., 44, 502–515, https://doi.org/10.1175/JAM2213.1.
Ryzhkov, A. V., T. J. Schuur, D. W. Burgess, P. L. Heinselman, S. E. Giangrande, and D. S. Zrnic, 2005b: The Joint Polarization Experiment: Polarimetric rainfall measurements and hydrometeor classification. Bull. Amer. Meteor. Soc., 86, 809–824, https://doi.org/10.1175/BAMS-86-6-809.
Seo, B.-C., W. F. Krajewski, A. Kruger, P. Domaszczynski, J. A. Smith, and M. Steiner, 2011: Radar-rainfall estimation algorithms of Hydro-NEXRAD. J. Hydroinform., 13, 277–291, https://doi.org/10.2166/hydro.2010.003.
Seo, B.-C., L. K. Cunha, and W. F. Krajewski, 2013: Uncertainty in radar-rainfall composite and its impact on hydrologic prediction for the eastern Iowa flood of 2008. Water Resour. Res., 49, 2747–2764, https://doi.org/10.1002/wrcr.20244.
Seo, B.-C., B. Dolan, W. F. Krajewski, S. Rutledge, and W. Petersen, 2015a: Comparison of single and dual polarization based rainfall estimates using NEXRAD data for the NASA Iowa Flood Studies Project. J. Hydrometeor., 16, 1658–1675, https://doi.org/10.1175/JHM-D-14-0169.1.
Seo, B.-C., W. F. Krajewski, and K. V. Mishra, 2015b: Using the new dual-polarimetric capability of WSR-88D to eliminate anomalous propagation and wind turbine effects in radar-rainfall. Atmos. Res., 153, 296–309, https://doi.org/10.1016/j.atmosres.2014.09.004.
Seo, B.-C., and Coauthors, 2018: Comprehensive evaluation of the IFloodS radar-rainfall products for hydrologic applications. J. Hydrometeor., 19, 1793–1813, https://doi.org/10.1175/JHM-D-18-0080.1.
Seo, B.-C., M. Keem, R. Hammond, I. Demir, and W. F. Krajewski, 2019: A pilot infrastructure for searching rainfall metadata and generating rainfall product using the big data of NEXRAD. Environ. Modell. Software, 117, 69–75, https://doi.org/10.1016/j.envsoft.2019.03.008.
Seo, B.-C., W. F. Krajewski, and Y. Qi, 2020: Utility of vertically integrated liquid water content for radar-rainfall estimation: Quality control and precipitation type classification. Atmos. Res., 236, 104800, https://doi.org/10.1016/j.atmosres.2019.104800.
Straka, J. M., D. S. Zrnić, and A. V. Ryzhkov, 2000: Bulk hydrometeor classification and quantification using polarimetric radar data: Synthesis of relations. J. Appl. Meteor., 39, 1341–1372, https://doi.org/10.1175/1520-0450(2000)039<1341:BHCAQU>2.0.CO;2.
Testud, J., E. Le Bouar, E. Obligis, and M. Ali-Mehenni, 2000: The rain profiling algorithm applied to polarimetric weather radar. J. Atmos. Oceanic Technol., 17, 332–356, https://doi.org/10.1175/1520-0426(2000)017<0332:TRPAAT>2.0.CO;2.
Wang, Y., and V. Chandrasekar, 2009: Algorithm for estimation of the specific differential phase. J. Atmos. Oceanic Technol., 26, 2565–2578, https://doi.org/10.1175/2009JTECHA1358.1.
Wang, Y., and V. Chandrasekar, 2010: Quantitative precipitation estimation in the CASA X-band dual-polarization radar network. J. Atmos. Oceanic Technol., 27, 1665–1676, https://doi.org/10.1175/2010JTECHA1419.1.
Wang, Y. D., S. Cocks, L. Tang, A. Ryzhkov, P. Zhang, J. Zhang, and K. Howard, 2019: A prototype quantitative precipitation estimation algorithm for operational S-band polarimetric radar utilizing specific attenuation and specific differential phase. Part I: Algorithm description. J. Hydrometeor., 20, 985–997, https://doi.org/10.1175/JHM-D-18-0071.1.
Wolff, D. B., W. A. Petersen, A. Tokay, D. A. Marks, and J. L. Pippitt, 2019: Assessing dual-polarization radar estimates of extreme rainfall during Hurricane Harvey. J. Atmos. Oceanic Technol., 36, 2501–2520, https://doi.org/10.1175/JTECH-D-19-0081.1.
Zhang, J., and Coauthors, 2016: Multi-Radar Multi-Sensor (MRMS) quantitative precipitation estimation: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 621–638, https://doi.org/10.1175/BAMS-D-14-00174.1.
Zhang, J., L. Tang, S. Cocks, P. Zhang, A. Ryzhkov, K. Howard, C. Langston, and B. Kaney, 2020: A dual-polarization radar synthetic QPE for operations. J. Hydrometeor., https://doi.org/10.1175/JHM-D-19-0194.1, in press.
