Introduction
The main goal of the Tropical Rainfall Measuring Mission (TRMM) is to provide accurate estimates of global tropical rainfall (Simpson et al. 1996; Kummerow et al. 2000). Since its launch in 1997, TRMM's sensors, which include the precipitation radar and the TRMM Microwave Imager, have provided a wealth of information about rainfall in the Tropics. However, calibration and validation of the TRMM algorithms and their products remains a crucial step toward the full use of this data in water-cycle studies. This can be accomplished by comparing TRMM products with independent observations available from surface sensors, especially radars and rain gauges, among others. The Ground Validation Program (GVP) has been designed to provide ground-based radar-rainfall products of sufficient quality from a number of sites. A main product is the TRMM Standard Product Number (TSPN) product 2A-53, which contains surface rain-rate maps with a resolution of 2 × 2 km2 (Marks et al. 2000). These maps are to be used as a validation tool of the TRMM satellite-rainfall products (e.g., Kummerow et al. 2000; Adler et al. 2000; Schumacher and Houze 2000). However, it is well recognized that rainfall estimates from ground-based radar have their own uncertainties and it is imperative to quantify them before a meaningful comparison with the space-based TRMM products can be made.
The task of quantifying radar-rainfall uncertainty has proven to be difficult. The most fundamental obstacle has been the lack of an adequate standard reference. Traditionally, surface rainfall measurements available from rain gauges are used as a standard reference. Numerous studies compared radar and rain gauge data and showed significant disagreement between the two sensors. Austin (1987) found that for individual storms, radar–gauge differences of a factor of 2 or more were not unusual. Only when she lumped together 20 storms, was the difference between gauges and radar reduced to about 10%. Woodley et al. (1975) and Wilson and Brandes (1979) reported similar results of radar–gauge comparisons. In a recent study, Brandes et al. (1999) found that radar-to-gauge ratios of storm totals were in the range of about 0.7–1.9.
Despite the large number of studies that reported significant discrepancy between radar and gauge observations, only a few attempted to explain or analyze such differences. In an early study, Zawadzki (1975) argued that the radar–gauge measurements being compared are of different character: gauges give an almost near-point rainfall rate while radar estimates correspond to a volume-averaged rainfall rate. He suggested searching for an optimal averaging time of gauge measurements that corresponds to spatial radar smoothing so that radar–gauge differences are minimized. Since radar measures rainfall at an appreciable height above the ground, Zawadzki (1975) also showed that a time delay between radar and gauge observations might improve the comparison. Austin (1987) acknowledged the differences in sampling properties of radars and rain gauges. She indicated that spatial rainfall intensity gradients and the inadequate spatial sampling of rain gauges are responsible for the random disagreement between radar and gauges.
Following a different approach, Kitchen and Blackall (1992) focused on understanding the differences between point and areal rainfall and their relevance to radar–gauge comparisons. They defined the so-called representativeness error as the combination of two sources of errors: the spatial representativeness error associated with comparison between a point and an areal average, and the temporal representativeness error associated with the comparison between an accumulation and an integration of a set of instantaneous measurements. Recently, Ciach and Krajewski (1999) revisited the issue and formulated a statistical procedure—the error variance separation method (EVSM)—that allows practical assessment of the radar-rainfall error characteristics. To address the implications of issues such as radar–gauge scale differences and the small-scale variability of rainfall, one needs information that characterizes the complex structure of rainfall at scales smaller than those of the grid of radar-rainfall products. Such subgrid information is now available from a series of field experiments designed to support the validation efforts of TRMM.
Our purpose in the current study is to characterize the uncertainty levels of the TRMM 2A-53 ground-based radar-rainfall maps. We use surface rainfall observations collected during the Texas and Florida Underflights Experiment (TEFLUN-B), which was conducted during August–September 1998 in central Florida. We use data from several instruments, including rain gauges and a vertical 915-MHz Doppler profiler in addition to radar observations. Since our main focus is on quantification of radar-rainfall uncertainty, we first present exploratory analysis of the observations to shed light on the observed rainfall variability. Then, we apply the EVSM to the radar maps using independent gauge observations from a dense network deployed during the same experiment. We conclude by discussing difficulties encountered in the analysis and offering suggestions to improve the estimation of the product errors.
Experimental setup and data
From the numerous surface instruments deployed during the TEFLUN-B campaign, we used observations from rain gauges, a vertical profiler, and the ground-based Weather Surveillance Radar-1988 Doppler (WSR-88D) radar located in Melbourne, Florida. The National Aeronautics and Space Administration (NASA) installed a dense network of rain gauges (DRGN) about 40 km west of Melbourne. Fourteen gauges were arranged with separation distances ranging from a few meters to about 8 km. Within the network, we also installed three dual-gauge platforms to provide rainfall observations separated by distances as small as 1 m. Figure 1 shows the layout of the network. The gauges were tipping-bucket (TB) type with volume resolution of 0.254 mm (0.01 in.), and were connected to data loggers that were set to sampling resolutions of 5–10 s. The gauges are manufactured by Qualimetrics (Model 6011-A) and dataloggers are manufactured by Unidata. After downloading the recorded tips, we interpolated them to obtain time series of rainfall rates for temporal scales as short as 1 min. However, as we showed in a separate study (Habib et al. 2001b), TB data result in significant sampling errors for rainfall rates at scales smaller than about 5 min. TB gauges also suffer from other operational problems; however, the closely located gauges and the dual-gauge setup at some sites enabled us to perform a quality control (QC) of the collected gauge observations. Quality-control analysis included plotting double-mass curves of rainfall amounts of dual gauges and pairs of gauges that were close to each other. Our confidence in the data quality was lower for some of the gauges that did not have other gauges nearby (e.g., 110 and 116). In addition, we performed analysis of some spatial statistics such as variation of correlation coefficients with intergauge distances. We identified possible malfunctioning periods of gauge observations by detecting correlation values that were inconsistent with the behavior of the spatial correlation pattern of the rest of the network.
Besides rain gauges, we also used rainfall observations from a vertical pointing profiler that was located within the dense gauge network (see Fig. 1). The profiler measures the Doppler velocity of hydrometeors directly overhead and provides estimates of the equivalent reflectivity at different altitudes ranging from 380 m to a few kilometers with a resolution as high as 105 m.
The radar-rainfall products evaluated in this study are the standard 2A-53 products of the TRMM-GVP. These products have undergone several stages of development and improvement (Robinson et al. 2000) and we use the latest released version. The 2A-53 represent “instantaneous” rain-rate maps constructed from radar volume scans collected at the WSR-88D radar at Melbourne. The rain maps are available every about 5 min for the 2-month period with only few missing days. These products are developed entirely by the GVP and here we briefly describe the procedure. The maps are 2 × 2 km2 Cartesian rainfall fields estimated at constant altitude of 1.5 km. They were created using a monthly based reflectivity–rain rate (Z–R) relation of the form Z = ARb where A and b are empirically fitted coefficients. GVP used surface gauge observations from several operational networks for the adjustment analysis of the Z–R relationship. While a value of 1.4 is always used for the exponent b, the multiplier A was selected in such a way to adjust the monthly accumulation of the radar pixels above each gauge to the 7-min gauge accumulation. This is done separately for two classes of rainfall: convective and stratiform, which are determined according to a classification scheme developed by Steiner et al. (1995). GVP also applies an automated quality control procedure using both radar and gauge data. Further details about the development of the 2A-53 maps can be found in Marks et al. (2000) and Robinson et al. (2000). The GVP did not use data from the dense rain gauge network we described earlier in developing the radar-rainfall products. Thus, the current rain gauge data can be used as an independent validation dataset.
Based on the measurements of the rain gauges, a total rainfall depth of the 2-month experiment period ranged from 350 to 450 mm within the DRGN. For illustration, Table 1 gives a summary of 35 events recorded by one of the gauges (gauge 101); for convenience, we defined the start of a new event as occurring after any period of at least 30 min without rain. The table shows that most of the events had duration of less than 1 h, and only a few lasted for about 2–4 h. Similarly, most events had total rainfall accumulations less than 10 mm with only five events exceeding 20 mm accumulation. Higher accumulations were reported at other gauges within the network. In general, the experimental period was characterized by localized and short-lived intensive storms, typical for summertime Florida rainfall. Visual inspection of observations from the profiler, which was located close to gauge 101, enabled us to classify the rain type as shown in Table 1. Most events were convective, a few were stratiform, and the rest were mixed rain, where strong localized convective cells were embedded within or followed by widespread light rainy areas. For illustration, Fig. 2 shows histograms of rain rates at three integration timescales: 5, 15, and 60 min. The histograms are constructed using observations from all gauges. We used the 5 mm h−1 bin and plotted the results in a semilogarithmic scale for clarity; we point out that the large mode at 0–5 mm h−1 is affected by the TB-gauge uncertainty in estimating low rain rates. A characteristic of the plotted histograms is the occurrence of extreme rain rates, which are persistent even with high integration timescales of 15 and 60 min.
Small-scale variability
Before analyzing the radar-rainfall products, we perform exploratory investigation of the observations provided by some of instruments deployed during the TEFLUN-B experiment such as the dense rain gauge network and the vertical profiler. This will provide insight into the relevant temporal and spatial scales over which rainfall exhibits significant variability.
The dense rain gauge network with its high temporal sampling resolution provides an opportunity to examine rainfall variability over spatial and temporal scales as small as a few meters and a few minutes. We compared observations of three pairs of gauges separated by distances of 1 m, 0.85 km, and 2.88 km, for three different timescales of 5, 15, and 60 min. We illustrate this in Fig. 3. The selected scales are relevant to the resolution of the 2A-53 maps: for example, the diagonal of the 2A-53 grid cell is about 2.8 km. The scatterplots show significant differences among the observations of such closely located gauges. Consider a 1-m separation distance: the two gauges still show instances of differences, especially at small timescales. This is most likely a reflection of the random sampling and measurement error of the TB gauges (Habib et al. 2001b). Even with larger integration scales (e.g., 1 h), significant scatter still exists between gauges that are located within a typical radar pixel.
An interesting and important characteristic of rainfall is intermittence in space and time (Barancourt and Creutin 1992; Georgakakos et al. 1994; Kumar and Foufoula-Georgiou 1994). Nonrainy areas may be embedded in larger rainy areas of a size similar to a single radar pixel, leading to problems in evaluating radar-rainfall estimates. It is often observed that gauges, with their near-point sampling volume, indicate zero rainfall while radar, which has a much larger sampling volume, reports nonzero estimates. There is difficulty in defining “no rain” in both gauge and radar observations. Gauges usually measure rainfall in discrete quantities that are determined by the gauge volume resolution (e.g., bucket size in the case of tipping-bucket gauges). Also, minimum rainfall intensities provided by radar (0.1 mm h−1 in the 2A-53 radar maps) are usually determined by an arbitrary threshold set to the measured reflectivity.
We restrict our intermittence analysis to factors that are related to the spatial variability of rainfall across the radar pixel. Since there is more than one gauge inside a single radar pixel (see Fig. 1), we performed a small-scale intermittence analysis using only gauge observations. Consider comparing one gauge versus N gauges. For a certain area size and time interval, we consider conditional probability that, given a nonzero rainfall observation at any of the N gauges, one gauge among the N gauges observed no rainfall. This is an approximation of the probability of zero point-rainfall conditional on nonzero area rainfall—the true value of such probability could be obtained only if there were an infinite number of gauges within the area. Given the arrangement of the gauges within the network layout, we chose areas that are almost uniformly covered with more than one gauge. Accordingly, we considered three areas of about 1 × 1 km2 each and we used three triangles of gauges to provide an approximate representation of rainfall occurrence (see Fig. 1). We also considered another area of about 3 × 3 km2 with seven gauges. We computed the conditional probability of zero-point rain for the two area sizes and for three temporal scales: 5, 15, and 60 min. The results are shown in Fig. 4 where each point represents the probability at a certain gauge computed for each area size and timescale. Despite the dispersion of the plotted points—caused by factors such as the sampling variability and the gauge measurement errors—we can make some interesting observations. In general, the intermittence levels become lower for larger timescales and smaller spatial scales. The computed values indicate the significance of the intermittence for radar–gauge comparison studies. Consider a timescale of 5 min, which is commensurate with the radar temporal sampling (scanning) frequency: there is about a 30% probability that a single gauge will not observe rain that falls within an area of about 1 km2. This probability increases to as high as 50% if the area increases to 9 km2. One should keep in mind that these probability values are likely conservative in the sense that higher values may result if more gauges were available within the areas under consideration. Nevertheless, the computed values indicate the significance of rainfall intermittence for analyses that involve radar–gauge comparisons where a statistic of interest is the probability of zero gauge-rainfall conditioned on a nonzero radar estimate.
Another statistical characteristic of rainfall that can be obtained from our dense gauge observations is the spatial correlation function. We will discuss it in section 4c.
Besides surface variability, we also examine variability in the vertical direction. Several studies (e.g., Williams et al. 1995; Gage et al. 1994) analyzed measurements of vertical profiles of reflectivity and Doppler velocity signals and demonstrated the existence of significant vertical variability. Herein, we only present an illustrative example using the 915-MHz profiler deployed during the TEFLUN-B campaign. Figure 5 shows a comparison of reflectivity measurements at two altitudes: 380 m, which is the lowest reliable profiler gate, and ∼1.5 km, which corresponds to the height of the 2A-53 maps. These elevations are also well below the freezing level so that enhancement in the reflectivity within the melting layer (brightband effect) is avoided. The scatter indicates the significant variations in the reflectivity that occur between the radar elevations and the ground. As discussed in earlier studies (e.g., Austin 1987; Wilson and Brandes 1979; Joss and Waldvogel 1990) such variability at altitudes below typical radar beam height is expected to introduce uncertainty into the radar-rainfall estimates. However, quantifying the contribution of the vertical variability to radar-rainfall errors is beyond the scope of this study.
Uncertainty analysis of the 2A-53 radar-rainfall maps
Basic radar–gauge comparison
As we explained in section 2, TRMM's GVP adjusted the radar maps to a set of operational gauges in such a way that the long-term (monthly) bias—defined as the ratio between radar and gauge total accumulations—is removed. We used the TEFLUN-B gauges, which were not included in producing the radar maps, to independently evaluate the bias since it is a key factor that determines the overall quality of any radar estimates. We accumulated gauge observations over the 2-month experiment period, whereas for the radar we first assigned “instantaneous” rain rates to 5-min intervals (which correspond to the radar temporal sampling resolution), which we then accumulated. The choice of a 5-min interval, or an alternative one, will be discussed later in section 4d. We show the results in Table 2. As discussed earlier, we performed data quality control by flagging and removing suspect data from each gauge separately. This explains why some of the closely located gauges in Table 2 report different accumulations (e.g., gauge 103 vs gauge 102). Since the GVP team already inspected the radar products during the product development stage, our QC analysis focused mainly on the gauge data. However, we point out that the QC-flagged periods of gauge observations were also flagged in the radar data. This resulted in different radar accumulations at the same pixel when compared with different gauges within that pixel (e.g., gauges 101, 101b, 102, and 103). In general, there is good agreement between radar and gauges; most of the differences in the total accumulations did not exceed a few millimeters over the 2-month period. Based on the analyzed radar–gauge pairs, the overall bias over the 2-month period did not exceed 2%. This indicates that the long-term bias is not a significant source of uncertainty for the examined 2A-53 rain maps. We point out that further analysis concerning the error variance statistic implicitly removes the overall bias.
To further examine the differences between gauge and radar estimates, we considered pairs of concurrent radar–gauge observations and plotted them in a scatter form in Fig. 6. The figure is based on instantaneous radar rates from the 2A-53 maps and average 5-min gauge rate centered at the time stamp of the radar maps. Significant scatter characterizes the plotted pairs both at low and high rain rates. Despite the careful quality control, the scatterplot shows some instances where the two estimates differ by as high as 50–100 mm h−1. The correlation coefficient between radar and gauge pairs for all the gauges is about 0.78; however, as argued by Kessler and Neas (1994), this value may be inflated due to the wide range of the values of the radar–gauge pairs. The standard deviation of the radar–gauge differences—conditional on nonzero rain at either—is about 5.9 mm h−1. If normalized by the mean of about 2.7 mm h−1, coefficient of variation of the differences becomes about 2.14 indicating significant difference between concurrent radar and gauge estimates.
Error variance separation method
As we discussed earlier, direct comparisons of gauge and radar estimates are problematic because of the huge difference in their sampling volumes. With a 20-cm-diameter orifice, a gauge samples a near-point fraction of the radar pixel area, which is 2 × 2 km2 in the case of 2A-53 maps. Because of this scale mismatch, one should expect that the high spatial variability of rainfall, which we discussed in section 3, would contribute to the radar–gauge discrepancy. Therefore, since gauge data are the basis for analysis of radar uncertainties, we use a procedure that allows filtering out the uncertainty of gauge observations that is due to their lack of areal representativeness. This general concept was first suggested by Kitchen and Blackall (1992) and was fully formulated as a statistical methodology in a recent study by Ciach and Krajewski (1999). The key concept of this technique, the EVSM, is based on the partitioning of the radar–gauge difference variance into the radar-rainfall estimation error variance and the gauge representativeness error variance.
Correlation estimation
A key factor in implementing the EVSM is the estimation of the correlation function over the radar pixel scale. Correlation has been widely used to characterize spatial variability of rainfall (e.g., Huff and Shipp 1969; Huff 1970; Sharon 1972; see Krajewski et al. 2001, manuscript submitted to Hydrol. Sci. J., for more references). The dense arrangement of the DRGN gauges allows estimating correlation levels at scales smaller than the 2-km radar resolution. Based on the assumption of second-order stationarity, and assuming an isotropic correlation field, we obtain an approximation of the correlation function. For each pair of gauges, we computed the correlation coefficient using the standard Pearson product-moment formula. The results are shown in the left panels of Fig. 7, in which each point represents the correlation coefficient between time series of rainfall rates observed by two gauges separated by a certain distance. We computed the correlation coefficients for accumulation timescales ranging from 1 min up to 1 h.
A clear feature in Fig. 7 is the significant scatter of the correlation coefficients computed for a certain separation distance. Despite the extensive use of the correlation coefficient in rainfall research, problems associated with its estimation are still not fully resolved. For example, Berndtsson (1987) used 5 yr of daily rainfall to show the sensitivity of correlation estimation to the sample size. Young et al. (2000) showed that confidence intervals constructed using normal sampling theory could not explain the observed scatter of hourly rainfall. Kessler and Neas (1994) showed that the correlation coefficient tends to increase with the range over which the variables are measured; this is relevant to rainfall data that include low rain rates associated with infrequent but extreme rates. Besides the sample size effect, it has also been argued that the estimation of population correlation coefficient using Pearson's formula is nonrobust in cases of nonnormal distributions (e.g., Kowalski 1972). This is relevant since rainfall data are far from normally distributed especially at small timescales (Kedem and Chiu 1987; Shimizu 1993). In a recent study, Lai et al. (1999) showed that for lognormal distributions, the correlation coefficient estimation using Pearson's formula suffers from severe overestimation bias that depends on the data's degree of skewness. Stedinger (1981) showed that estimating the correlation using logarithms of the observations results in better estimates of the true correlations than does the Pearson's formula. Habib et al. (2001a) further investigated this concept and its suitability for the analysis of rainfall spatial correlation. In their investigation, rainfall observations at two gauges are fitted to a bivariate intermittent lognormal distribution developed by Shimizu (1993). This distribution accounts for the intermittent behavior of rainfall since it allows representing cases with zero rain at either or both of the two gauges. Using the parameters of the fitted distribution, a sample version of the population correlation coefficient can be obtained. Further details about this transformation-based estimation procedure and its implementation using both simulations and actual rainfall data can be found in the references mentioned above.
Using the transformation-based estimation approach, Habib et al. (2001a) reported correlation estimates with negligible bias and smaller mean square errors when compared to those obtained using the traditional Pearson's formula. Therefore, we decided to implement this approach to obtain more robust estimates of the correlation function. Accordingly, we recomputed the correlation coefficients and plotted them in the right panels of Fig. 7. As expected, the modified correlation coefficients corrected for the bias of the corresponding Pearson's estimates by as much as 0.1–0.15 depending on the timescale. In addition, our procedure provides correlation coefficients and their associated uncertainty bounds. In Fig. 7 we show these as vertical bars denoting ±2 standard deviations. The plotted bounds and larger ones (e.g., 10% and 90% of the distributions, not plotted for clarity) explain most of the scatter of the computed coefficients.
An interesting observation from the correlation plots is the value of the correlation coefficient at near-zero separation distances represented in the exponential model by the parameter ρ0. A value of ρ0 less than unity may arise due to microscale variability, but also due to gauge measurement errors. If we consider timescales of 15 min or longer, where random measurement errors become less significant (Habib et al. 2001b), the estimated ρ0 attains a value of about 0.95–0.97: this represents the natural variability at these very small scales. Note that the use of the Pearson's correlation estimates results in a value of ρ0 = 1. An accurate estimate of ρ0 is crucial for reliable evaluation of uncertainty levels of radar-rainfall estimates (Young et al. 2000; Anagnostou et al. 1999).
Implementation and discussion
With the present formulation of the area–point variance of (3), in which ρ(xg, x) is gauge-location dependent, we perform the variance partitioning analysis on a gauge-by-gauge basis. However, before we proceed with the evaluation of the different variance components, we have to decide on the timescale at which to analyze the radar–gauge observations. The 2A-53 radar rain maps provided by the TRMM GVP are instantaneous estimates sampled about every 5–6 min. Since our main interest is in evaluating the current TRMM products with their original resolution, we limit our analysis to the resolution of the radar maps. The question then is, What is the timescale for accumulation of the gauge observations corresponding to radar-rainfall map products? Since radar-derived estimates correspond to a volume-averaged rainfall, gauge point observations need to be accumulated over a certain timescale. Based on a Taylor's hypothesis argument, Zawadzki (1975) suggested that an optimal value for the gauge accumulation timescale should be related to the size of the radar domain under consideration. A second issue regarding the comparison framework is whether a time shift between radar and gauge observations is needed. This arises because radar measures rainfall at a certain height above the ground where the gauges are located. The time shift accounts for travel time of raindrops and/or drifting of rainfall caused by low-level winds.
Because of the difficulty associated with defining the integration timescale ΔT and the time shift τ, we decided to perform an extensive error analysis across the broad range of both parameters. Specifically, we explored the variance partitioning (2) across a range of 1 min to 1 h for ΔT, and −20 to 20 min for τ. This implies that we evaluated each variance component in (3) using gauge data processed for every possible combination of ΔT and τ and radar products at their original resolution. We also computed the correlation coefficient between radar and gauge pairs, ρ[Rr, Rg(ΔT, τ)], across the full ΔT–τ domain. This correlation coefficient, in addition to the variance component var(Rr − Rg), will indicate the presence of an optimal region for ΔT–τ, if there is any. As we mentioned earlier, the variance component, var(Rr − Rg), can be easily computed by evaluating the radar–gauge differences for each ΔT and τ. The area-point variance component, var(Rg − R), was also computed by substituting the exponential approximation (4)—evaluated for each value of ΔT—into (3) and numerically performing the required integrations. Clearly, this variance component depends on both the relative gauge location inside the radar pixel and the gauge integration timescale but does not depend on the selected time shift of gauge observations. Last, the radar error variance, var(Rr − R), could be obtained as the difference between the two computed variance components according to (2). The results are shown in Fig. 8 in the form of contour plots of each variance component in addition to the correlation coefficient of radar–gauge pairs. For illustration, we also show the radar error variance as a ratio of the total radar–gauge difference variance.
As we indicated before, the plots in Fig. 8 correspond to radar maps evaluated at a certain gauge; results for other gauges showed similar features. Consider the first two panels of the figure, which show the variance of radar–gauge differences and the correlation coefficient of radar–gauge pairs; these two quantities can be considered as objective criteria for evaluation of radar–gauge agreement. The behavior of both quantities is similar and shows an optimal region of ΔT and τ where radar–gauge observations exhibit their lowest levels of disagreement. However, the optimal region of ΔT is relatively wider than that of τ, which is more defined for both panels. In general, the results indicate that, based on variance and correlation measures, instantaneous radar estimates show better agreement with gauge observations if the latter were averaged over time intervals ranging from 5 to 15 min, and time delayed by about 2–5 min. The optimal range of ΔT is in accordance with the gauge-averaging time of 7 min that was applied by the GVP team in constructing the 2A-53 maps (Marks et al. 2000); note that they did not apply any time shift. We point out that the determined optimal values of ΔT and τ, as well as other radar–gauge statistics in our study, are likely to vary with factors such as the size of the radar pixel (2 km in this study) and the range of gauges from the radar (∼40 km in this study). For example, with the increase of pixel sizes, larger integration timescales may become needed while introducing a time shift becomes less important. Similar argument can be made regarding the range of the gauges from the radar—the height of the radar beam is low for gauges with short ranges, making the time shift less significant. The other two panels in Fig. 8 describe the radar error expressed by its variance and variance ratio. The results indicate that, within the predefined region of best agreement, the radar error variance is about 30% of the total variance of radar–gauge differences. Results associated with the other gauges showed the corresponding values ranging from 20% to 60%. In other words, the areal representativeness error of the gauge rainfall observations is responsible for about 40%–80% of the total disagreement between the radar and gauge estimates. We point out that the variation in the results from one gauge to another is caused by 1) sampling variability (2-month field experiments are not long enough!), and 2) the variation in the relative location of the gauge within the radar pixel (gauges located near the center “represent” the true areal rainfall across the pixel size better than do those near the corners or edges).
Uncertainty according to rainfall type
The results we discussed above characterize the overall uncertainty levels of the radar products averaged over the entire sample. However, it is expected that radar errors depend on the magnitude of rainfall. One way to address this issue would be a conditional EVSM. However, implementing such conditioning is not possible with TEFLUN-B data because of their limited sample size. Instead, we used a rather simple approach, dividing the data into two types of rain: light and heavy. We classified each radar pixel, with its concurrent gauge observation, either as heavy rain if the radar rain rate is greater than 10 mm h−1, or as light rain if 0 < Rr < 10 mm h−1. With this classification, the mean of radar rain rates at each gauge location ranged from 2.5 to 3.1 mm h−1 for light rain and from 26.1 to 30.6 mm h−1 for heavy rain; see Tables 3a and 3b for a summary of our results. Next, we estimated the correlation function, using the procedure described earlier, for each rainfall regime separately. In Fig. 9 we show an example of the computed conditional correlation for an accumulation timescale ΔT of 15 min. As expected, the two rainfall types showed quite different behavior in the decay of correlation. For separation distances smaller than the radar pixel size (∼1 km), heavy rain shows higher correlation levels than those of light rain. With slight increase in distance, correlation of heavy rain drops at a much faster rate when compared with light rain that maintained significant correlation levels at large separation distances. This is reflected in the parameters of the approximate formula (4) for which we obtained ρ0 of 0.90 and 0.97, and d0 of 4.6 and 2.5 km, for light and heavy rain, respectively. Using these conditional correlation functions, we performed the variance analysis of (2) to evaluate the radar uncertainty for each rainfall regime separately; the results are summarized in Tables 3a and 3b for each gauge within the network. As before, we explored the full domain of both ΔT and τ parameters; however, in the tables we show results only for ΔT = 15 min and for τ at which highest correlation between radar and gauge pairs was obtained. The statistics reported in Tables 3a and 3b were described earlier, except the last column referred to as the coefficient of variation CV of radar error and defined as the ratio of standard deviation of the estimated radar error to the mean of the radar estimates. The tables also show the sample sizes that were used in the analysis.
Let us make several observations about these statistics. Within each rainfall class, the statistics, such as radar mean and gauge standard deviation, showed some variation from gauge to gauge. The variation is more significant for heavy rain type where the sample is relatively small and characterized by higher natural variability. Few gauges (e.g., 102 and 116 with light rain; 109, 115, and 116 with heavy rain) showed significantly different results in comparison with the rest of gauges and we do not consider their statistics to be representative. Consider the comparison between radar and gauge pairs. The standard deviation of the differences, [var(Rr − Rg)]1/2, is higher for heavy rain than for light rain, which is understandable. However, normalized by the mean, the coefficient of variation of the differences for the light-rain subsample becomes more than twice as much as that of the heavy rain. This is also manifested by the cross-correlation coefficient ρ[Rr, Rg(ΔT, τ)], which is relatively lower for light than heavy rain. As we pointed out earlier, the correlation coefficient could be overestimated if the radar–gauge pairs have a relatively wide range of values, which is more pronounced in the sample of heavy rain. The variance of point–area differences with respect to point variance, which is expressed in the table by VRF, is 10%–30% higher for heavy rain as compared with light rain. This is due to the significant spatial variability that characterizes heavy rain as described by the faster decay of its spatial correlation as shown in Fig. 9. Accordingly, it is expected that the spatial variability, characterized by point–area difference, will have more significant influence in the case of heavy rain than in light rain. The contribution of natural spatial variability to the radar–gauge discrepancy, expressed by the variance ratio var(Rg − R)/var(Rr − Rg), is larger for heavy rain (∼40%–75%) than for light rain (∼30%–45%). Consider now the radar error, described by its standard deviation [var(Rr − R)]1/2. Excluding the few gauges that resulted in significantly different results from the rest, this statistic ranges from about 2.4 to 3.1 mm h−1 for light rain that has a mean of 2.5–3.1 mm h−1, and from 6.4 to 15 mm h−1 for heavy rain that has a mean of 26.1–30.6 mm h−1. In normalized measures, CV varied from 0.95 to 1.10 for light rain and from 0.20 to 0.40 for heavy rain.
Summary and discussion
We presented an exploratory analysis of uncertainty estimates of the TRMM ground-validation standard radar-rainfall maps. In comparison with gauge observations, our results showed that despite low levels of long-term bias, significant random errors of the instantaneous radar-rainfall estimates exist. In contrast to earlier studies (see introduction for numerous examples), our approach distinguishes the radar error from the radar–gauge differences. In this sense, to the best of our knowledge, this work is the first study that in a rigorous way attempts to quantify random error of radar-rainfall products. The approach we followed in the analysis allowed us to account for the effect of the spatial natural variability of surface rainfall. The variability, expressed as variance ratio, contributed about 30%–45% for light rain and 40%–75% for heavy rain to the differences of radar–gauge pairs. Equivalently, the radar error, expressed by the ratio of its standard deviation to its mean, was as high as 90%–120% for light rain and 20%–40% for heavy rain. We point out that our results are based on summer rainfall events in Florida. Different rainfall regimes that are characterized by different levels of small-scale variability may yield different radar error statistics.
Let us discuss the current results with respect to some previous relevant studies. Kitchen and Blackall (1992) and Anagnostou et al. (1999) quantified the contribution of gauge representativeness error; however, their analyses were based on logarithmic transformations of ratios of radar–gauge pairs, which makes it difficult to compare their results with the ones of our study. As compared with our results, Ciach and Krajewski (1999) reported relatively lower levels of radar error. For accumulation times of 5–30 min, the ratio of radar error variance to the total radar–gauge difference variance was in the order of 30%–40% (20%–60% in the current results). However, because of the lack of gauge observations on the scale of radar subpixel, Ciach and Krajewski (1999) used a rather crude approximation of the correlation function, resulting in overestimation of the point–area variance component. Young et al. (2000) investigated the sensitivity of the results to the correlation parameters, especially the parameter ρ0. They showed that the estimated radar error variance is highly sensitive to the value of ρ0. Such sensitivity is less relevant to our results since we used more robust and accurate correlation estimates, especially at small separation distances. In a recent study, Adler et al. (2000) compared some TRMM space-based estimates with ground-validation radar maps from the Melbourne radar and reported significant biases and scatter levels. We were unable to assess the impact of the current estimated uncertainties on their results because of differences in the temporal and spatial scales used. However, given the significant radar uncertainties indicated above, we recommend taking these into account in future radar-based validation studies.
When using the results of the current study, one should keep in mind some of the issues and difficulties that affected the implementation of the uncertainty estimation methodology. The dense rain gauge network used in the validation of radar maps was only available for the 2-month period of the experiment; to our knowledge, the network is not in full operation anymore. In addition, most of the other operational gauge networks that have sufficient sampling requirements and quality were used in the development stage of the radar maps, leaving few independent gauge observations for validation studies. The current results also show the need for longer deployment of experimental gauge networks of the kind used in this study. The limited sample size of concurrent radar and gauge observations made it difficult to perform a more extensive validation investigation. The statistics needed for the error variance separation showed some instability as reflected in their variability from one gauge to another. The expected dependence of the radar–gauge difference and radar error on the gauge location inside the radar pixel is masked by the small sample size and by possible navigation uncertainty in locating the gauges. In addition, we were able to estimate the radar errors by conditioning on only two classes of rain. A more useful and practical analysis where radar errors are described for each radar rain rate necessitates much larger samples. In addition, the current study characterized the radar uncertainty in terms of its variance only; further analysis is under way to develop parallel approaches where the full distribution of radar error can be described.
Last, the current analysis has accounted for natural variability only at the surface. Preliminary analysis of other ground-based observations collected during the experiment (e.g., profiler and disdrometer) showed significant variability in the vertical direction. Profiler observations showed both systematic-like and random differences in reflectivity profiles at altitudes close to the surface (about few hundred meters), which is different from other commonly observed and accounted-for features such as the brightband effect. Investigating the significance of this kind of variability may help to better explain the radar–gauge differences and provide reliable quantification of the uncertainty of radar estimates of surface rainfall. We hope that our results will contribute to the objectives of TRMM by providing a means for more meaningful evaluation of its space-based products.
Acknowledgments
This study was supported by NASA Grant NAG5-2774. We thank all participants of the TEFLUN-B experiment who collected the data we used. In particular, we thank Anton Kruger, Paul Kucera, Doug Houser, Brad Fisher, Ali Tokay, Christopher Williams, and the team of the University of Central Florida led by Linwood Jones. We also thank Eyal Amitai for helpful information about the 2A-53 radar products. Special thanks go to Grzegorz Ciach. Our discussions with him stimulated much of this research.
REFERENCES
Adler, R. F., G. J. Huffman, D. T. Bolvin, S. C. Urtis, and E. J. Nelkin. 2000. Tropical rainfall distributions determined using TRMM combined with other satellite and rain gauge information. J. Appl. Meteor. 39:2007–2023.
Anagnostou, E. N., W. F. Krajewski, and J. A. Smith. 1999. Uncertainty quantification of mean-areal radar-rainfall estimates. J. Atmos. Oceanic Technol. 16:206–215.
Austin, P. M. 1987. Relation between measured radar reflectivity and surface rainfall. Mon. Wea. Rev. 115:1053–1070.
Barancourt, C. and J-D. Creutin. 1992. A method for delineating and estimating rainfall fields. Water Resour. Res. 28:1133–1144.
Brandes, E. A., J. Vivekanandan, and J. W. Wilson. 1999. A comparison of radar reflectivity estimates of rainfall from collocated radars. J. Atmos. Oceanic Technol. 16:1264–1272.
Berndtsson, R. 1987. On the use of cross-correlation analysis in studies of patterns of rainfall variability. J. Hydrol. 93:113–134.
Bras, R. L. and I. Rodriguez-Iturbe. 1993. Random Functions and Hydrology. Dover, 559 pp.
Ciach, J. G. and W. F. Krajewski. 1999. On the estimation of radar rainfall error variance. Adv. Water Resour. 22:585–595.
Gage, K. S., C. R. Williams, and W. L. Ecklund. 1994. UHF wind profilers: A new tool for diagnosing tropical convective cloud systems. Bull. Amer. Meteor. Soc. 75:2289–2294.
Georgakakos, K. P., A. A. Carsteanu, P. L. Sturdevant, and J. A. Cramer. 1994. Observation and analysis of Midwestern rain rates. J. Appl. Meteor. 33:1433–1444.
Habib, E., W. F. Krajewski, and G. J. Ciach. 2001a. Estimation of rainfall interstation correlation. J. Hydrometeor. 2:621–629.
Habib, E., W. F. Krajewski, and A. Kruger. 2001b. Sampling errors of tipping-bucket rain gauge measurements. J. Hydrol. Eng. 6:159–166.
Huff, F. A. 1970. Spatial distribution of rainfall rates. Water Resour. Res 6:254–259.
Huff, F. A. and W. L. Shipp. 1969. Spatial correlations of storm, monthly and seasonal precipitation. J. Appl. Meteor. 8:542–550.
Joss, J. and A. Waldvogel. 1990. Precipitation measurement and hydrology. Radar in Meteorology, Battan Memorial 40th Anniversary Radar Meteorology Conference, D. Atlas, Ed., American Meteorological Society, 577–606.
Kedem, B. and L. S. Chiu. 1987. On the lognormality of rain rate. Proc. Nat. Acad. Sci. U. S. A. 84:901–905.
Kessler, E. and B. Neas. 1994. On correlation, with applications to the radar and raingage measurement of rainfall. Atmos. Res. 34:217–229.
Kitchen, M. and R. M. Blackall. 1992. Representativeness errors in comparisons between radar and gauge measurements of rainfall. J. Hydrol. 134:13–33.
Kowalski, C. J. 1972. On the effect of non-normality on the distribution of the sample product-moment correlation coefficient. Appl. Stat. 27:1–12.
Kumar, P. and E. Foufoula-Georgiou. 1994. Characterizing multiscale variability of zero intermittency in spatial rainfall. J. Appl. Meteor. 33:1516–1525.
Kummerow, C. Coauthors,. 2000. The status of the Tropical Rainfall Measuring Mission (TRMM) after two years in orbit. J. Appl. Meteor. 39:1965–1982.
Lai, C. D., J. C. W. Rayner, and T. P. Hutchinson. 1999. Robustness of the sample correlation—The bivariate lognormal case. J. Appl. Math. Decision Sci. 3:7–19.
Marks, D. A. Coauthors,. 2000. Climatological processing and product development for the TRMM ground validation program. Phys. Chem. Earth, Part B: Hydrol. 25:871–875.
Press, W. H., B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling. 1988. Numerical Recipes: The Art of Scientific Computing,. Cambridge University Press, 818 pp.
Robinson, M. Coauthors,. 2000. Evolving improvements to TRMM ground validation rainfall estimates. Phys. Chem. Earth, Part B: Hydrol. 25:971–976.
Sharon, D. 1972. The spottiness of rainfall in a desert area. J. Hydrol. 17:161–176.
Schumacher, C. and R. A. Houze Jr. 2000. Comparison of radar data from the TRMM satellite and Kwajalein oceanic validation site. J. Appl. Meteor. 39:2151–2164.
Shimizu, K. 1993. A bivariate mixed lognormal distribution with an analysis of rainfall data. J. Appl. Meteor. 32:161–171.
Simpson, J. S., C. Kummerow, W-K. Tao, and R. F. Adler. 1996. On the Tropical Rainfall Measuring Mission (TRMM). Meteor. Atmos. Phys. 60:19–36.
Stedinger, J. R. 1981. Estimating correlations in multivariate streamflow models. Water Resour. Res. 17:200–208.
Steiner, M., R. A. Houze Jr., and S. E. Yuter. 1995. Climatological characterization of three-dimensional storm structure from operational radar and rain gauge data. J. Appl. Meteor. 34:1978–2007.
Williams, C. R., W. L. Ecklund, and K. S. Gage. 1995. Classification of precipitation clouds in the Tropics using 915-MHz wind profilers. J. Atmos. Oceanic Technol. 12:966–1012.
Wilson, J. W. and E. A. Brandes. 1979. Radar measurement of rainfall—a summary. Bull. Amer. Meteor. Soc. 60:1048–1058.
Woodley, W. L., A. R. Olsen, A. Herndon, and V. Wiggert. 1975. Comparison of gage and radar methods of convective rain measurement. J. Appl. Meteor. 14:909–928.
Young, C. B., A. A. Bradley, W. F. Krajewski, A. Kruger, and M. L. Morrissey. 2000. Evaluating NEXRAD multisensor precipitation estimates for operational hydrologic forecasting. J. Hydrometeor. 1:241–254.
Zawadzki, I. 1975. On radar-raingage comparison. J. Appl. Meteor. 14:1430–1436.
Layout of NASA's DRGN cluster deployed during the TEFLUN-B field experiment. The inset in the figure shows the University of Iowa's three dual-gauge platforms, the 2D video disdrometer, and the 915-MHz Doppler profiler, as set up at the Tripple Ranch site in Melbourne, FL. The 2 × 2 km2 grid (solid lines) shows the Cartesian pixels of the 2A-53 radar rain maps. The dashed boxes show areas used for intermittence analysis
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Histograms of rainfall rates at three timescales of 5, 15, and 60 min. Histograms are constructed using observations from all gauges within the DRGN
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Scatterplots of rain rates for three pairs of rain gauges separated by distances of (left to right) 1 m, 0.85 km, and 2.86 km for three timescales of (top to bottom) 5, 15, and 60 min
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Description of rainfall intermittence as observed by closely located gauges. Three gauges within 1 × 1 km2 (open circles) and seven gauges within 3 × 3 km2 (filled circles) are considered. Gauge zero-rain probability conditional on rain existence at any other gauge within the area is plotted for different timescales
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Scatterplot of 1-min reflectivity observations obtained by the 915-MHz profiler at two elevations, 380 m and 1.5 km
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Scatterplots of instantaneous radar rain rates and 5-min gauge observations centered at the time stamp of the radar map
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Spatial correlation function estimated from the rain gauge cluster for 5-, 15-, and 60-min timescales. (left) Correlation estimates using the standard Pearson's formula. (right) Adjusted estimates based on a transformation-based procedure
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Results of the various statistics used in the error separation analysis. (top left) Variance of radar–gauge difference (mm h−1)2. (top right) Correlation coefficient of radar and gauge pairs. (bottom left) Radar error (mm h−1)2. (bottom right) Radar error as a percentage of the variance of radar–gauge difference (i.e., percentage of lower left panel to upper left panel)
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Spatial correlation for light and heavy rain samples. Transformation-based adjusted correlation estimates are shown for the 15-min timescale
Citation: Journal of Applied Meteorology 41, 5; 10.1175/1520-0450(2002)041<0558:UAOTTG>2.0.CO;2
Summary of rain events observed by gauge 101 during the TEFLUN-B period of Aug–Sep 1998. Event type, indicated as convective C, stratiform S, and mixed M, was done through visual inspection of the images of a profiler located close to the gauge
Total rainfall 2-month accumulations as estimated by radar Rr and gauges Rg
Table 3a. Summary of radar error analysis for heavy rainfall. Instantaneous radar maps and 15-min gauge rain rates are used. Statistics at each gauge location are as follows: radar mean, std dev of gauge observations, std dev of radar–gauge differences, correlation coefficient of radar–gauge pairs, variance reduction factor, variance of point (gauge) error as a percentage of variance of radar–gauge differences, std dev, and coefficient of variation of radar error
Table 3b. Same as in Table 3a but for the case of light rain
