1. Introduction
Reliable quantitative information on the spatial distribution of rainfall is essential for hydrologic and climatic applications, which range from real-time flood forecasting to evaluation of regional and global atmospheric model simulations. Given their quasi-global coverage, satellite-based quantitative rainfall estimates are becoming widely used for such purposes. Converting satellite measurements into quantitative precipitation estimates poses challenges. The link between the observations and surface rain rates depends on the calibration and operating protocol of the instrument itself, the spatial heterogeneity of the rain fields (e.g., coexistence of convective and stratiform precipitation within a single instrumental field of view and vertical heterogeneity of rainfall), the indirect nature of the measurement, and the retrieval algorithm used. As underlined by the Program to Evaluate High Resolution Precipitation Products (Turk et al. 2008) led by the International Precipitation Working Group (IPWG; see http://www.isac.cnr.it/~ipwg/), characterizing the error structure of satellite rainfall products is recognized as a major issue for the usefulness of the estimates (Yang et al. 2006; Zeweldi and Gebremichael 2009; Sapiano and Arkin 2009; Wolff and Fisher 2009). The error characterization is needed for data assimilation and climate analysis (Stephens and Kummerow 2007) and more specifically over land in hydrological modeling of natural hazards and budgeting water resources (Grimes and Diop 2003; Lebel et al. 2009).
In this study, we focus primarily on the Tropical Rainfall Measurement Mission (TRMM) Precipitation Radar (PR) quantitative precipitation estimation (QPE) at ground. The methodology presented herein would equally apply to all satellite precipitation products—in particular those on board low-earth-orbiting satellites. The TRMM PR is currently the only active instrument measuring rainfall from a satellite platform conjointly with a radiometer [TRMM Microwave Imager (TMI)]. PR rainfall estimates are often considered as a reference for TMI-based rainfall estimates (e.g., Yang et al. 2006; Wolff and Fisher 2008). It impacts rain estimates from polar-orbiting passive microwave measurements and a number of satellite-based high-resolution precipitation products (Ebert 2007; Bergès et al. 2010; Ushio et al. 2006). Given the variety of potential sources of error in PR-based QPE and the impact of correction algorithms, the only practical solution is to evaluate PR QPE with respect to an external, independent reference rainfall dataset. The reference is derived from high-resolution ground validation measurements using NOAA/National Severe Storms Laboratory (NSSL) ground radar–based National Mosaic and QPE system (NMQ; Zhang et al. 2011). These products yield instantaneous rainfall-rate products over vast regions including regions of the conterminous United States (CONUS) covered by Next Generation Weather Radar (NEXRAD) data. While a number of studies have investigated the quality of PR estimates in various regions of the world (e.g., Adeyewa and Nakamura 2003; Wolff and Fisher 2008; 2009; Amitai et al. 2009, 2012), our aim is to perform a systematic and comprehensive evaluation for regions over the southern CONUS. We will characterize errors in PR estimates at the pixel measurement scale in order to minimize additional uncertainties caused by resampling. Systematic and stochastic errors of PR estimates will be documented in terms of bias and spatial structure.
One should note that it is not possible to “validate” the PR estimates in a strict sense because independent rainfall estimates with no uncertainty do not exist. Many errors affect the estimation of rainfall from ground-based radars, like nonuniform beam filling, conversion of reflectivity to rain intensity, and calibration. While we do not know the truth at ground, the available independent measurements do provide a useful reference to help identify possible biases and the general levels of uncertainty associated with PR estimates. The reference rainfall accuracy issue will be investigated by systematically comparing PR estimates with different references at ground. Three levels of processing to remove biases characterize these references.
Rainfall estimates from low-earth-orbiting satellites suffer from their poor temporal sampling (Wolff and Fisher 2008; 2009; Lin and Hou 2008). Hence, representative samples of direct comparisons between instantaneous coincident measurements from ground and space are difficult to achieve without a sufficient number of overpasses. This study uses three months (March–May 2011) of satellite overpasses over the lower CONUS. The data are pixel matched in both time and space, and statistics are provided for comparing reference rain intensities to satellite-based estimates. The quasi-instantaneous matching is performed at the scale of the PR measurement scale (4.5 × 4.5 km2).
The PR data and steps required to refine the Q2 ground-based rainfall to arrive at the reference rainfall used for comparisons are presented in section 2. Section 3 assesses the ability of PR rain retrievals to represent the rainfall variability derived from the reference data in terms of rainfall detectability, sensitivity, and spatial structure. Section 4 provides an empirical error model of the PR estimates versus reference rainfall and segregates systematic and random error. The paper is closed with concluding remarks in section 5.
2. Data sources
a. Original ground-based products
The NOAA/NSSL NMQ/Q2 (http://nmq.ou.edu; Zhang et al. 2011) is a set of experimental radar products comprising high-resolution (0.01°, 5 min) instantaneous rainfall-rate mosaics available over CONUS. The NMQ system combines information from all ground-based radars comprising the Weather Surveillance Radar-1988 Doppler (WSR-88D) network (NEXRAD), mosaics reflectivity data onto a common 3D grid, and estimates surface rainfall accumulations and types to arrive at accurate ground-based estimates of rainfall (Zhang et al. 2005; Lakshmanan et al. 2007; Vasiloff et al. 2007; Kitzmiller et al. 2011). Figure 1 shows an example of CONUS coverage of Q2 rainfall at 0725 UTC on 11 April 2011 highlighting several rainy systems associated with orography in the west and a wide frontal system in the central part of the domain.
Map of CONUS area with NMQ/Q2 instantaneous rain rates at 0725 UTC on 7 Apr 2011. The red area shows the good quality radar coverage corresponding to radar quality index equal to 1. The shaded area is not sampled by the TRMM PR.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
At hourly time step, Q2 adjusts radar estimates with automated rain gauge networks using a spatially variable bias multiplicative factor. A radar quality index (RQI) is produced at the (0.01°, 5 min) resolution. While the true quality of the Q2 QPEs varies in space and time because of a number of complicating factors [e.g., measurements errors, nonprecipitation echoes, uncertainties in Z–R relationships, and variability in the vertical profile of reflectivity (VPR)], the RQI represents the radar QPE uncertainty associated with reflectivity changes with height and near the melting layer (Zhang et al. 2011). It applies to the radar beam used for QPE—that is, hybrid scan reflectivity comprising elevation angles closest to the surface. The RQI field is composed of a static part relative to the radar beam sampling characteristics such as percent blockage, beam height and width, and a dynamic part accounting for the freezing level height. The static part is illustrated in Fig. 1, where the reduced radar coverage in the western part of the United States results in lower RQI values. The dynamic part causes the RQI values to decrease in cool season months when the freezing level is lower and the radar samples the melting layer and the ice phase at closer range, and to increase in the warm season when the freezing level is at higher altitudes. This is illustrated in Fig. 1, where the freezing level is lower behind a cold frontal system, which deteriorates the already limited coverage in the western part of CONUS.
The original Q2 products utilized in this study are (i) the radar-only instantaneous rain-rate National Mosaic updated every 5 min, (ii) the radar-only rain-rate National Mosaic at hourly time step, (iii) the hourly rain gauge–corrected National Mosaic product, and (iv) the RQI. The primary Q2 product used for comparison with PR is the radar-only instantaneous rain-rate mosaic. Current Q2 radar products do not include an instantaneous gauge-adjusted rain-rate mosaic. For this study and similarly to Amitai et al. (2009, 2012), a second reference rainfall was derived from an instantaneous bias-corrected Q2 product. Pixel-by-pixel ratios between the hourly gauge-adjusted and the hourly radar-only products are calculated. These hourly ratios are then applied as multiplicative adjustment factors to the radar-only 5-min product. Extreme adjustment factors [outside the (0.1–10) range] are discarded and no comparison is performed with PR for the corresponding Q2 values. Thus, the gauge adjustment also serves as a data quality control procedure. A subsequent reference is derived from the bias-corrected Q2 product filtered using the RQI index. Only the rain rates associated with the best RQI values (i.e., equal to 1) were retained. This selection ensures that only Q2 estimates representing the best measurements conditions (i.e., no beam blockage and radar beam below the melting level of rainfall) are retained.
As radar-based only, the first reference may have issues like nonuniform beamfilling because of VPR effects, inaccurate conversion from reflectivity to rain intensity, and calibration errors. The blending with rain gauge data in the second reference should significantly mitigate these biases and provide a more accurate reference. The filtering through the radar quality index for the last level of processing eliminates a large part of the impact of the VPR. One should note these incremental improvements of the Q2 products may not screen out all possible errors in ground-based radar estimates. In particular, the gauge adjustment may suffer from representativeness errors from scarce rain gauge network density and from the difference of the temporal resolution between the hourly adjustment factors applied downscale to 5-min Q2 rain rates. Nevertheless, they provide the best possible reference at the scale of PR in terms of sampling conditions and unbiased estimates.
b. Q2-based reference rainfall
In the current study, all significant rain fields observed coincidentally by TRMM overpasses and the NEXRAD radar network from March to May 2011 are collected. The Q2 products closest in time to the TRMM satellite local overpass schedule time are used. To compute the reference rainfall, a block-Q2 rainfall pixel is computed to match each PR pixel in case of TRMM overpasses in a similar manner to Kirstetter et al. (2010, 2012).
Matched PR and Rref(A) estimates only exist at locations where both the PR and ground radars have taken actual observations. This technique averages the minimum number of Q2 meshes needed to produce spatially coincident sample Rref(A) estimates. The advantages of the current technique over gridded approaches are that there is no interpolation, extrapolation, smoothing, or oversampling of PR data. The PR rainfall statistical characteristics are preserved because the product remains untouched: the total rainfall amount, the total rainy area, and the probability distribution function (PDF) shapes. All of these properties may therefore be compared to the reference at once.
Figure 2 shows an example of continuous mapping of the weighted mean estimator for the reference rainfall Rref(A). The estimator is a smoother of the original Q2 rain field. The maximum of the rainfall rate decreases from 145 to 130 mm h−1. The total rainfall area increases, mainly at the edges of the rain field. To avoid a contamination of the PR–reference comparison by the uncertainty on the ground reference, the reference pixels were segregated into “robust” [Rref(A) > σfootprint] and “nonrobust” [Rref(A) < σfootprint] estimators. This procedure illustrated in Fig. 2 filters out the reference values at the edges of the rain fields. Nonrobust reference values are discarded for quantitative comparison. The robustness check is applied to the three Q2 products considered for reference (native Q2, bias-corrected Q2, and RQI+bias-corrected Q2). As an example for the “RQI+bias-corrected Q2” the averaged relative error (σref/Rref) of the reference decreases from 832% to 16%. The ratio of the mean error to the standard deviation of the reference [σref/σ(Rref)] decreases slightly from 5.6% to 5.4%. This method of reference selection therefore increases the reliability and representativeness of the block-Q2 values that constitute our ground reference.
Maps of instantaneous rain rates at 0725 UTC on 7 Apr 2011: (top left) the NMQ/Q2 product, (top right) the equivalent reference rainfall Rref, (bottom left) the robust reference set, and (bottom right) the nonrobust reference set.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
c. PR-based rainfall
The PR measures reflectivity profiles at Ku band. Surface rain rates are estimated over the southern United States up to a latitude of 37°N (Fig. 1). Artifacts such as contamination by surface backscatter, attenuation and extinction of the signal, nonuniform beam filling, brightband effects and accuracy of the Z–R relationship (Wolff and Fisher 2008) must be accounted for. In the present study, the surface rain rate at each PR pixel location is a standard TRMM product (2A25 v6) described in Iguchi et al. (2000). The scan geometry and sampling rate of the PR lead to pixels spaced approximately 5.1 km cross and along track over a 245-km-wide swath. The minimum theoretical detectable rain rate by the PR is fixed by its sensitivity and is about 17 dBZ, or ~0.5 mm h−1.
d. Comparison samples
Several factors—including rainfall intermittency, discrete temporal sampling of TRMM, and censoring of reference values for required quality—reduce the number of comparison samples for reference and PR estimates over the comparison period. Table 1 provides the number of these samples for the reference values, inclusive and exclusive of nonrainy pixels. The comparison sample sizes in Table 1 are primarily driven by the number of rain events and the overpass frequency of TRMM, then by the censoring of reference values. The quality control in the bias adjustment discarded 26% of original Q2 values and an additional 34% were filtered using RQI. Note that after two levels of processing and censoring, the comparison sample size for the RQI+bias-corrected Q2 remains significant at 393 347. This is credited to the large number of samples offered by the high-resolution, gridded Q2 product.
Comparison samples for different reference datasets.
To assess the representativeness of our spatially and temporally limited samples, we compared the statistics of the reference rainfall resampled to the PR pixel resolution with respect to the whole reference dataset (CONUS-wide below 38°N, which do not necessarily match a TRMM overpass). Figure 3 shows quantile–quantile plots between (i) the whole reference dataset (x axis) and (ii) the subset of pixels that matched to PR pixel resolution for the different reference datasets. Table 2 provides values of the conditional mean and standard deviation. The PR-resampled reference rainfall distribution does not show a clear deviation from the 1:1 degree line compared to the whole distribution. The reference distributions are fairly stable given the different censoring levels with the mean of the PR-resampled distributions being within 7% of the one for the whole dataset. We may therefore consider each reference dataset to be quite representative of the corresponding whole rainfall distribution.
Quantile–quantile plots for reference PR-sampled and whole rainfall distribution comparison for (a) native, (b) bias-corrected, and (c) bias+RQI-corrected references. The positions of 10, 50, and 95 percentiles are showed for each distribution.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
Conditional mean and standard deviation of whole- and “PR-resampled” references datasets.
Similarly, we compared the different PR datasets to assess the impact of Q2 censoring on their representativeness. Figure 4 shows quantile–quantile plots between (i) the complete (“native”) PR dataset (x axis) and (ii) the censored subsets according to the bias-corrected and bias+RQI-corrected Q2 samples. Table 3 provides values of the conditional mean and standard deviation for each set. The different PR rainfall distributions do not show a clear deviation from the 1:1 degree line compared to the native PR rainfall distribution. The means and standard deviations of the bias-corrected-censored and bias+RQI-corrected-censored distributions are less than 1% and around 10% higher, respectively. We may therefore consider the representativeness of each PR dataset, following censoring steps, to be quite comparable to each other.
Quantile–quantile plots for PR “native–reference-sampled,” “bias-corrected–reference-sampled,” and “bias+RQI-corrected–reference-sampled” rainfall distribution comparison. The positions of 10, 50, and 95 percentiles are shown for each distribution.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
Conditional mean and standard deviation of PR estimates for different references.
3. Rainfall data analysis
This section reviews the ability of PR rain retrievals to represent the rainfall variability derived from the Q2 data. First, contingency tables provide information on the reference rainfall reliability and on the influence of PR sensitivity to detect rainfall occurrence. The PDF of rainfall estimates provide in-depth information on the sensor’s global ability to capture rain regimes given the influence of its sensitivity and the several factors (attenuation of the radar signal, nonuniform beam filling, and accuracy of the Z–R relationship). Another feature to compare is the spatial structure of rainfall fields.
a. Contingency tables
Table 4 shows the contingency tables for PR rain/no-rain occurrence relative to the references with percentile of hits (H; both Q2 and PR detect rain), misses (M; PR does not detect rain while Q2 does), false alarms (F; PR detects rain while Q2 does not), and correct rejections (C; both Q2 and PR do not detect rain). The reference data are separated into three subsamples: the nonrobust set (Rref < σfootprint; see section 2b), the robust set (Rref > σfootprint) and the “whole” Q2 set. Reference null values are considered as robust. All coincident and collocated PR values are considered and sorted according to the reference samples. Table 5 provides the mean rainfall values according to the same contingency tables with PR on the left-hand side of the “/” sign and the reference on the right-hand side.
Contingency table for PR relative to the three references. The results are provided for robust/non robust reference data according to a criterion based on the variability of the Q2 rainfall (at native resolution) inside the PR pixel (Rref > σfootprint).
Mean rainfall values associated to the contingency table for PR/references.
The false detections (M + F) of PR are mainly associated with the nonrobust reference data, with a rate of more than 80% for all nonrobust sets, while around 50% are improperly classified when using the robust reference dataset. The misses (M) are the main contributors to the false detection population (i.e., approximately 85% for the whole dataset). These misses of PR are coincident with low reference values (less than 0.15 mm h−1 for the nonrobust set for all references; see Table 5). By comparison, the correct detections (H + C) of PR are mainly associated with the robust reference set from 45% to 52%. For the same robust reference sets, the hits of PR are coincident with the higher reference values with mean rainfall rates more than 6 mm h−1. One should note for all references that (i) the mean PR (F) values are significantly lower than the PR (H) values and (ii) the mean reference (M) values are significantly lower than the mean reference (H) values. Finally, both mean reference and PR values are higher for the robust Q2 set than for the nonrobust Q2 set. Table 6 shows the discarded rain volumes in question; the misses of PR represent less than 12% of the reference rainfall volume, while false alarms represent less than 16% of the PR rainfall volume. Note the lowest values (less than 8%) are obtained with the bias+RQI-corrected Q2 reference.
Discarded rain volumes from PR due to misses relative to references and rain volume implied in the false alarms relative to robust references.
The impact of the reference rainfall on the contingency scores is shown in Fig. 5. Contingency values are used to compute probability of detection (POD), false alarm rates (FAR), and critical success index (CSI). Scores are generally better for the robust reference than for the nonrobust one. Within this category, CSI shows a general increase with sequential Q2 data quality steps, while the FAR shows the lowest values with additional processing of the Q2 reference. A general convergence between the Q2 reference and PR estimates is therefore acknowledged as a function of the reference accuracy.
CSI, POD, and FAR, and for the three references and partitioned as a function of robustness.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
Considering that 80% of the whole reference rain-rate dataset that are not detected by the PR are lower than 0.3 mm h−1, the sensitivity of PR is close to this value. The misses are likely associated with high intermittency and/or the “rain/no-rain” limits of rain fields. These features are missed by the PR because the rain rates are close to the detection threshold. Further, we calculated PR’s POD for different rainfall-rate thresholds (not shown) using the robust, bias+RQI-corrected Q2 reference. The POD increased from 56% at the rain/no-rain detection level to 71% using a threshold of 0.5 mm h−1 and leveled off to 76% for 1.0 mm h−1. This suggests that the PR can indeed capture the main rain regions but loses the weaker echoes (Schumacher and Houze 2000), probably because of its sensitivity. The false alarms may be due to shallow rain not detected by ground-based radars when occurring at significant distance (greater than 100 km). This is supported by the positive impact of the RQI on the false alarm rate, which as seen on Fig. 1 limits the range of ground radar data selection.
b. Probability distributions by occurrence and rain volume
Hereafter, the PR rain estimates are the conditional ones (nonzero rainfall) coincident and collocated with nonzero reference estimates. The robust reference rain-rate datasets are used. Two PDFs for PR versus reference rainfall are computed and shown in Fig. 6: (i) the PDF by occurrence (PDFc) and (ii) the PDF by rain volume (PDFv) (Wolff and Fisher 2009; Amitai et al. 2009, 2012). The PDFc provides statistical information on the rain-rate distribution and highlights the estimate’s sensitivity as a function of rain rate; it is computed as a ratio between the number of the rain rates inside each bin and the total number of rain rates. The PDFv represents the relative contribution of each rain-rate bin to the total rainfall volume; it is computed as a ratio between the sum of the rain rates inside each bin and the total sum of rain rates. It is therefore an important characteristic of the instantaneous products from the perspective of building merged rainfall accumulations; it enables a comparison of PDFs based on estimates derived from instruments characterized by different detection limits (in particular at weak intensities).
Probability distributions of rain rates for the PR rainfall and for the (a) native-, (b) bias-corrected-, and (c) bias+RQI-corrected–reference rainfall. The robust reference rain rates are used. The solid and dashed–dotted lines represent the distribution by volume PDFv and the distribution by occurrence PDFc, respectively, while the gray and black lines represent the distributions for references and PR, respectively. Note that the x axis is in log scale.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
The rain rates of PR exhibit similar PDFc for all references. Compared to references’ PDFc, PR tends to overestimate light rain rates (in the interval [0.3–0.5] mm h−1). But, PR demonstrates poor detection of the lightest rain rates (below ~0.3 mm h−1) compared to the two bias-corrected references. This is consistent with the concept of rain area “edges” that might be only partially detected by PR, resulting in misses associated with low rain rates (see previous section). PR PDFc presents similar features with references for rain rates >~1 mm h−1. One may note the improved convergence between PR and reference rainfall PDFc in the rain-rate interval [0.5–1.0] mm h−1 with the sequential Q2 data quality steps.
Despite the low occurrence of relatively high rain rates (>10 mm h−1), their contribution to the total rainfall volume is significant (greater than 60%). As a consequence, the mode of PDFv for PR is shifted toward lower rain rates (~18 mm h−1) compared to the reference’s mode (~60 mm h−1), which is in agreement with the results found in Amitai et al. (2006, 2009). This is attributed to high rainfall rates (>10 mm h−1), which are underestimated by PR because insufficient correction due to attenuation losses for the 2A25 version 6 (as suggested by Wolff and Fisher (2008)), nonuniform beam filling effects, and/or inaccurate conversion from reflectivity to rain intensity. Note that it is difficult to distinguish between these different influences by comparing solely rain rates at ground.
c. Spatial structure of estimated rainfall fields
Spatially normalized variograms of references and PR estimates are displayed in Fig. 7. Table 7 summarizes the parameters of these variograms. The variogram ranges of PR are quite similar to the three references’ (approximately 18 km). The nugget values, however, are more distinct. While it is ~32% for the Q2 references, it is significantly higher for PR (approximately 45% of the sill). These decorrelations of spatial structure at short interdistances suggest the resolution of the PR measurements may be limited when sampling the variability of small, disorganized rainfall structures associated with localized convection. The smaller reference nugget is an indication of the better sampling of the rain field by the reference rainfall, which is an issue previously discussed in section 2b. It must also be noted that the upscaling of the reference estimates from their original resolution to the PR resolution tends to smooth the original Q2 rain field. The comparatively higher nugget with PR may be caused by the rain intermittency, contamination by surface backscatter, attenuation of the signal, brightband effects, or inaccuracy of the Z–R relationship. An interesting feature is that both sensors present a slightly decreasing nugget with the sequential Q2 data quality steps. This feature could be attributed to the censoring of the reference, which filters out complicated sampling situations for the ground-based radars.
Spatial variograms for (a),(c),(e) reference and (b),(d),(f) PR for the (a),(b) native, (c),(d) bias-corrected, and (e),(f) bias+RQI-corrected reference. The empirical variograms are plotted with crosses, and the models fitted are represented by the thick black lines.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
Parameters of the normalized variograms (exponential model) for references and PR. The effective range values are indicated. The nugget is expressed as a percentage of the normalized sill.
4. Quantitative error modeling
a. Correlations and biases
Scatterplots of PR versus reference rainfall are presented for the three sets of Q2 reference in Fig. 8. Classical performance criteria of satellite-based rainfall estimation compared to reference values are listed in Table 8: correlation coefficient and mean relative error (MRE), expressed in percentage and defined as MRE = (PRmean − Refmean)/Refmean. The comparisons between the PR and reference estimates are assessed on a point-to-point basis. A rainy pixel is included in the statistics if both PR and the reference are nonzero to emphasize the PR ability to quantify precipitation when it is raining (the case of PR having zero rainfall when it is raining has been addressed in section 3a). This is particularly significant given the significant misses of PR.
Scattergraphs of PR vs (a) native-, (b) bias-corrected-, and (c) bias+RQI-corrected–reference rainfall (mm h−1). The first bisectors (solid lines) are displayed.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
Performance criteria values for PR estimates: mean, standard deviation, MRE, and correlation (R) with respect to references. Only the reliable Q2 data are kept (see section 2b) for references.
The two sensors present coherent mean and standard deviation values as long as the representativeness of the comparison samples are kept in mind. As expected, the means of the three PR sets are quite similar. In all cases the PR underestimates the reference mean values by ~17%. This is once again attributed to the significant underestimation of the higher rain rates in the 2A25-v6 products, presumably because of a combination of several factors like attenuation losses, inaccurate Z−R relationship, and/or nonuniform beam filling. The variations of the reference mean for the three sets explain in large part the variations in the apparent bias of PR relative to the reference. The native reference set is affected by (i) a global overestimation of rain rates, which could be due to the inaccuracy of the Z−R relationship, and (ii) an underestimation of rain rates linked to partial beam blockage and VPR effects (i.e., overshooting above the melting layer by the radar beam far from the radar). The gauge-based bias correction of the native Q2 product decreases the mean reference values, so the negative bias of PR is apparently improved. The additional RQI filtering removes the underestimation of Q2 at far range so the bias of PR is degraded. The reference shows higher standard deviation than the PR in coherence with the PDF features presented in section 3b.
The correlation coefficients between PR and Q2 reference estimates are moderate (around 0.6). One could note the best correlation between the two sensors is achieved with the bias+RQI-corrected reference. The differences between the two products on a point-to-point comparison basis can be attributed to sample volume discrepancies, timing and navigation mismatches, and the uncertainties in the respective rainfall estimates. The significantly greater nugget in the PR variogram than in the reference variogram is also an indication of the greater level of noise in the PR rain field spatial structure, which may limit the correlation between the two series on a point-to-point comparison.
b. Error model
The departures of PR estimates from the references are analyzed in this section on a point-to-point basis. The uncertainties associated with satellite estimates of rainfall include systematic errors as well as random effects from several sources (Yang et al. 2006; Kirstetter et al. 2012). There is a fundamental issue in segregating the proportion of the scatter due to purely random error and the proportion due to conditional biases of the PR estimates that may be either positive or negative, producing additional scatter.
With the true rainfall being unknown, the residuals are defined as the difference ɛ = (R − Rref) between the reference rainfall (Rref) and the satellite estimates (R). Only pairs for which Rref and R are both nonzero are considered in the calculations in order to emphasize the PR ability to quantify precipitation where it is raining. The sets of ɛ distributions are studied using the generalized additive models for location, scale, and shape (GAMLSS; Rigby and Stasinopoulos 2005) technique. As a preliminary step, Rref is considered as the main driving (explanatory) variable conditioning the departures of PR estimates from references.
Generalized linear models for location, scale, and shape aim at modeling the parameters of a response variable’s distribution. Two main assumptions are made: 1) the response variable ɛ is a random variable following a known parametric distribution with density f(ɛ | μ, σ) conditional on the parameters (μ, σ), and 2) the observations ɛ are mutually independent given the parameter vectors (μ, σ). Each parameter is modeled as a function of Rref (the explanatory variable) using monotonic (linear/nonlinear or smooth) link functions. More details are provided by Rigby and Stasinopoulos (2001, 2005), Akantziliotou Rigby and Stasinopoulos (2002), and Stasinopoulos and Rigby (2007). A wide variety of distributional forms are available within GAMLSS. To simplify and distinguish between systematic and random errors, a number of conditional densities with the first two moments as parameters are considered here: the location μ (mean) describing systematic errors and the scale σ (standard deviation) representative of random errors. For a given conditional distribution of the response variable, the conditional quantiles can be expressed as a function of the location and scale. GAMLSS is best fitted using the algorithm GAMLSS in the R software package (Stasinopoulos and Rigby 2007). The rainfall trends for each parameter are fitted using locally weighted scatterplot smoothing (LOESS), which are more flexible than polynomials or fractional polynomials for modeling complex nonlinear relationships. It is a polynomial curve determined by Rref, which is fitted locally by weighted polynomial regression, giving more weight to points near the point whose response is being estimated and less weight to points farther away (see Cleveland et al. 1991).
Several two-parameter density functions (lognormal, normal, reverse gumbel, logistic, gamma, etc.) have been tested to fit the data. The distributions of residuals (not shown here) were generally found to be unimodal and asymmetric. The goodness of fit on the whole dataset has been checked by investigating the Akaike information criteria (AIC) for each of the semiparametric density fits. The reverse Gumbel distribution 
(left) PR residuals represented vs bias+RQI-corrected reference and (right) the GAM model fitted is represented by [5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 95] conditional quantile lines.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
Standard deviation of PR–reference residuals (dashed line), estimated standard deviation of the reference rainfall (dotted line), and standard deviation of PR–true rainfall residuals (solid line) as functions of the bias+RQI-corrected reference. The vertical line (50 mm h−1) indicates the limit of the good sampling conditions.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
Figure 11 shows the conditional biases and random errors of PR relative to the three Q2 references. The global bias (see previous section and Table 8) of PR results from a balance between overestimation of light rain rates and underestimation of high rain rates. The underestimation is more frequent, inducing a global negative bias. The conditional biases of PR relative to the references are quite similar. Note the bias-corrected conditional bias is shifted to the right compared to the native one, so overestimation of light rain rates is more significant and the underestimation of higher rain rates less pronounced, which is consistent with the reduced negative global bias for this specific reference (see Table 8). Note also the negative slope of the bias+RQI-corrected conditional bias is lower than for the two other (the conditional bias is less significant), which could be seen as a sign of a better convergence between PR estimates and this Q2 reference. This is confirmed when considering the random part of error. The bias+RQI-corrected curve shows the lowest random errors up to Rref = 4 mm h−1 (more than 65% of the reference rain rates are under this value). The random error increases consistently with Rref. It is systematically higher for the bias-corrected than for the native reference—a result consistent when applying a bias correction (Ciach et al. 2000). It represents a significant part of error, suggesting that other factors than Rref could be considered to evaluate the error of PR rain-rate estimates at ground.
Conditional bias (median) of (left) residuals and (right) conditional random error (interquantile 90%–10%) of residuals for PR as a function of native (dotted line), bias-corrected (dotted line), and bias+RQI-corrected references.
Citation: Journal of Hydrometeorology 13, 4; 10.1175/JHM-D-11-0139.1
5. Conclusions
In preparation for National Aeronautics and Space Administration (NASA)’s future Global Precipitation Measurement (GPM) mission, a 3-month data sample of TRMM PR–based rainfall products have been compared to surface rainfall derived from Q2 over the lower conterminous United States. The major advantage of the Q2 ground-based reference dataset is its resolution in both time and space commensurate with rainfall estimates derived from sensors on board low-earth-orbiting satellites. The comparisons have been performed at the PR pixel resolution. A framework is proposed herein to address methodological issues so as to provide a preliminary version of an error model for satellite QPEs. The error model is empirically derived and is thus prone to be specific to the dataset considered and the PR/Q2 data processing implemented. However, the results show similarities with previous rainfall comparisons over West Africa and thus give credence to the developed framework (Kirstetter et al. 2012). Results from the error model presented herein provide insights into the most significant characteristics of PR rainfall retrieval errors that need to be taken into account when such data are used in applications.
A consistent result noted throughout each analysis was the increased consistency between PR and the Q2 reference following sequential data quality control steps, including bias correction using rain gauges and filtering using the radar quality index (RQI) product. This finding, alone, highlights the importance of matching the scales and refining the accuracy of the reference dataset as much as possible before reaching meaningful conclusions about the PR accuracy.
Different error sources were identified and quantified for PR rain-rate estimates. The most significant error is most likely due to a combination of inaccurate Z−R relationship, nonuniform beam filling, and/or attenuation of the PR radar signal. It is difficult to distinguish between these different influences by comparing solely rain rates at ground. Segregating rain from no-rain boundaries is also a driving contributor to the PR rain-rate errors, probably linked to the lack of sensitivity in the most inhomogeneous and light parts of the edges of rainy regions. Nevertheless, the variogram analysis showed that the PR adequately represents the spatial structure of the rain fields. The scatterplots revealed PR-estimated rain rates are only moderately correlated (Pearson correlation coefficient of 0.6) to the best reference rainfall on a point-to-point basis.
The statistical model developed here quantifies the relation between instantaneous PR rainfall and the corresponding reference rainfall. It consists of a deterministic additive function and a random uncertainty component, both conditioned on given reference values. The contribution of systematic PR errors is confirmed to be quite large because of the aforementioned signal attenuation issue.
In terms of perspectives, the relative contributions of errors linked to rainfall type and off-nadir angle need to be evaluated, as well as influence of the underlying terrain. The same framework and reference rainfall datasets can be readily applied to rainfall retrievals from other sensors on board low-earth-orbiting satellites [i.e., TMI, Advanced Microwave Scanning Radiometer for Earth Observing System (EOS) (AMSR-E), Special Sensor Microwave Imager (SSM/I), and Microwave Analysis and Detection of Rain and Atmospheric Structures (MADRAS)]. This framework will also be applied to GPM rainfall estimates following its launch in 2013. Another important issue to study is how the various error sources in PR, which is often used as a calibrator, propagate when merging with geostationary infrared data for a number of satellite-based, high-resolution precipitation products. Finally, the error model discussed in this study would be useful to generate rainfall ensembles and in hydrologic error propagation studies of satellites estimates.
Acknowledgments
We are very much indebted to the team responsible for the NMQ/Q2 products, especially Carrie Langston. We want to thank two anonymous reviewers whose comments were very useful for improving the manuscript. This work was funded by a postdoctoral grant from the NASA Global Precipitation Measurement Mission Ground Validation Management.
REFERENCES
Adeyewa, Z. D., and Nakamura K. , 2003: Validation of TRMM radar rainfall data over major climatic regions in Africa. J. Appl. Meteor., 42, 331–347.
Akantziliotou, K., Rigby R. A. , and Stasinopoulos D. M. , 2002: The R implementation of generalized additive models for location, scale and shape. Proc. 17th Int. Workshop on Statistical Modelling, Chania, Greece, Statistical Modelling Society, 75–83.
Amitai, E., Llort X. , and Sempere-Torres D. , 2006: Opportunities and challenges for evaluating precipitation estimates during GPM mission. Meteor. Z., 15, 551–557.
Amitai, E., Llort X. , and Sempere-Torres D. , 2009: Comparison of TRMM radar rainfall estimates with NOAA next-generation QPE. J. Meteor. Soc. Japan, 87, 109–118.
Amitai, E., Petersen W. , Llort X. , and Vasiloff S. , 2012: Multi-platform comparisons of rain intensity for extreme precipitation events. IEEE Trans. Geosci. Remote Sens., 50, 675–686.
Bergès, J. C., Chopin F. , Jobard I. , and Roca R. , 2010: EPSAT-SG: A satellite method for precipitation estimation; its concept and implementation for AMMA experiment. Ann. Geophys., 28, 289–308.
Ciach, G. J., and Krajewski W. F. , 1999: On the estimation of rainfall error variance. Adv. Water Resour., 2, 585–595.
Ciach, G. J., Morrissey M. L. , and Krajewski W. F. , 2000: Conditional bias in radar rainfall estimation. J. Appl. Meteor., 39, 1941–1946.
Cleveland, W. S., Grosse E. , and Shyu W. M. , 1991: Local regression models. Statistical Models in S, J. M. Chambers and T. J. Hastie, Eds., Wadsworth and Brooks/Cole, 309–376.
Ebert, E., 2007: Methods for verifying satellite precipitation estimates. Measuring Precipitation from Space: EURAINSAT and the Future, V. Levizzani, P. Bauer, and F. J. Turk, Eds., Springer, 345–356.
Grimes, D. I. F., and Diop M. , 2003: Satellite-based rainfall estimation for river flow forecasting in Africa. I: Rainfall estimates and hydrological forecasts. Hydrol. Sci. J., 48, 567–584.
Iguchi, T., Kozu T. , Meneghini R. , Awaka J. , and Okamoto K. , 2000: Rain-profiling algorithm for the TRMM precipitation radar. J. Appl. Meteor., 39, 2038–2052.
Journel, A., and Huijbregts C. , 1978: Mining Geostatistics. London Academic Press, 131 pp.
Kirstetter, P. E., Delrieu G. , Boudevillain B. , and Obled C. , 2010: Toward an error model for radar quantitative precipitation estimation in the Cévennes–Vivarais region, France. J. Hydrol., 394 (1–2), 28–41, doi:10.1016/j.jhydrol.2010.01.009.
Kirstetter, P. E., Viltard N. , and Gosset M. , 2012: An error model for instantaneous satellite rainfall estimates: Evaluation of BRAIN-TMI over West Africa. Quart. J. Roy. Meteor. Soc., doi:10.1002/qj.1964, in press.
Kitzmiller, K., and Coauthors, 2011: Evolving multisensor precipitation estimation methods: Their impacts on flow prediction using a distributed hydrologic model. J. Hydrometeor., 12, 1414–1431.
Lakshmanan, V., Fritz A. , Smith T. , Hondl K. , and Stumpf G. , 2007: An automated technique to quality control radar reflectivity data. J. Appl. Meteor. Climatol., 46, 288–305.
Lebel, T., Bastin G. , Obled C. , and Creutin J. D. , 1987: On the accuracy of areal rainfall estimation: A case study. Water Resour. Res., 23, 2123–2134.
Lebel, T., and Coauthors, 2009: AMMA-CATCH studies in the Sahelian region of West-Africa: An overview. J. Hydrol., 375 (1–2), 3–13, doi:10.1016/j.jhydrol.2009.03.020.
Lin, X., and Hou A. Y. , 2008: Evaluation of coincident passive microwave rainfall estimates using TRMM PR and ground measurements as references. J. Appl. Meteor. Climatol., 47, 3170–3187.
Rigby, R. A., and Stasinopoulos D. M. , 2001: The GAMLSS project: A flexible approach to statistical modelling. Proc. 16th Int. Workshop on Statistical Modelling, Odense, Denmark, University of Southern Denmark and Danish Natural Science Research Council, 249–256.
Rigby, R. A., and Stasinopoulos D. M. , 2005: Generalized additive models for location, scale and shape (with discussion). Appl. Stat., 54, 507–554.
Sapiano, M. R. P., and Arkin P. A. , 2009: An intercomparison and validation of high-resolution satellite precipitation estimates with 3-hourly gauge data. J. Hydrometeor., 10, 149–166.
Schumacher, C., and Houze R. A. Jr., 2000: Comparison of radar data from the TRMM satellite and Kwajalein oceanic validation site. J. Appl. Meteor., 39, 2151–2164.
Stasinopoulos, D. M., and Rigby R. A. , 2007: Generalized additive models for location scale and shape (GAMLSS) in R. J. Stat. Software, 10, 1–64.
Stephens, G. L., and Kummerow C. D. , 2007: The remote sensing of clouds and precipitation from space: A review. J. Atmos. Sci., 64, 3742–3765.
Takahashi, N., Hanado H. , and Iguchi T. , 2006: Estimation of path-integrated attenuation and its nonuniformity from TRMM/PR range profile data. IEEE Trans. Geosci. Remote Sens., 44, 3276–3283.
Teo, C. K., and Grimes D. I. F. , 2007: Stochastic modelling of rainfall from satellite data. J. Hydrol., 346 (1–2), 33–50.
Turk, F. J., Arkin P. , Ebert E. E. , and Sapiano M. R. P. , 2008: Evaluating high-resolution precipitation products. Bull. Amer. Meteor. Soc., 89, 1911–1916.
Ushio, T., and Coauthors, cited 2006: A combined microwave and infrared radiometer approach for a high resolution global precipitation mapping in the GSMAP project Japan. Third Int. Precipitation Working Group Workshop on Precipitation Measurements, Melbourne, Australia, Bureau of Meteorology. [Available online at http://www.isac.cnr.it/~ipwg/meetings/melbourne-2006/melbourne2006-pres.html.]
Vasiloff, S., and Coauthors, 2007: Improving QPE and very short term QPF: An initiative for a community-wide integrated approach. Bull. Amer. Meteor. Soc., 88, 1899–1911.
Wolff, D. B., and Fisher B. L. , 2008: Comparisons of instantaneous TRMM ground validation and satellite rain-rate estimates at different spatial scales. J. Appl. Meteor. Climatol., 47, 2215–2237.
Wolff, D. B., and Fisher B. L. , 2009: Assessing the relative performance of microwave-based satellite rain-rate retrievals using TRMM ground validation data. J. Appl. Meteor. Climatol., 48, 1069–1099.
Yang, S., Olson W. S. , Wang J. J. , Bell T. L. , Smith E. A. , and Kummerow C. D. , 2006: Precipitation and latent heating distributions from satellite passive microwave radiometry. Part II: Evaluation of estimates using independent data. J. Appl. Meteor. Climatol., 45, 721–739.
Zeweldi, D. A., and Gebremichael M. , 2009: Sub-daily scale validation of satellite-based high-resolution rainfall products. Atmos. Res., 92, 427–433, doi:10.1016/j.atmosres.2009.01.001.
Zhang, J., Howard K. , and Gourley J. J. , 2005: Constructing three-dimensional multiple radar reflectivity mosaics: Examples of convective storms and stratiform rain echoes. J. Atmos. Oceanic Technol., 22, 30–42.
Zhang, J., Qi Y. , Howard K. , Langston C. , and Kaney B. , 2011: Radar Quality Index (RQI)—A combined measure of beam blockage and VPR effects in a national network. Proc. Eighth Int. Symp. on Weather Radar and Hydrology, Exeter, United Kingdom, Royal Meteorological Society.
