• Battan, L. J., 1973: Radar Observation of the Atmosphere. University of Chicago Press, 324 pp.

  • Bickel, P. J., and K. A. Doksum, 1977: Mathematical Statistics: Basic Ideas and Selected Topics. Holden–Day, 492 pp.

  • Carroll, R. J., D. Ruppert, and L. A. Stefanski, 1995: Measurement Error in Nonlinear Models. Chapman and Hall, 305 pp.

  • Ciach, G. J., and W. F. Krajewski, 1999: Radar–rain gauge comparisons under observational uncertainties. J. Appl. Meteor.,38, 1519–1525.

  • ——, ——, E. N. Anagnostou, M. L. Baeck, J. A. Smith, J. R. McCollum, and A. Kruger, 1997: Radar rainfall estimation for ground validation studies of the Tropical Rainfall Measuring Mission. J. Appl. Meteor.,36, 735–747.

  • Fuller, W. A., 1987: Measurement Error Models. John Wiley and Sons, 440 pp.

  • Rosenfeld, D., and E. Amitai, 1998: Comparison of WPMM versus regression for evaluating Z–R relationships. J. Appl. Meteor.,37, 1241–1249.

  • Seber, G. A. F., 1989: Nonlinear Regression. John Wiley and Sons, 768 pp.

  • View in gallery

    Multiplicative conditional bias as a function of given true rain rate. Two cases of true-rain-rate variability coefficients are shown, CVr = 2 (left panel) and CVr = 4 (right panel). In each panel, conditional biases for three values of the exponent ratios γ = b/β are presented (descriptions are at the corresponding curves).

  • View in gallery

    Behavior of the total conditional bias and MSE as a function of the exponent ratio γ. The CBtot is represented by the dotted curves. The solid curves show MSE for two values of the variation coefficient CVez of the reflectivity measurement errors (descriptions are at the corresponding curves). Two cases of true-rain-rate variability coefficients are considered, CVr = 2 (left panel) and CVr = 4 (right panel).

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 270 270 24
PDF Downloads 164 164 17

Conditional Bias in Radar Rainfall Estimation

View More View Less
  • a Environmental Verification and Analysis Center, University of Oklahoma, Norman, Oklahoma
  • | b Iowa Institute of Hydraulic Research, The University of Iowa, Iowa City, Iowa
© Get Permissions
Full access

Abstract

The goal of this study is to improve understanding of the optimization criteria for radar rainfall (RR) products. Conditional bias (CB) is formally defined and discussed. The CB is defined as the difference between a given rain rate and the conditional average of its estimates. A simple analytical model is used to study the behavior of CB and its effect on the relationship between the estimates and the truth. This study shows the measurement errors of near-surface radar reflectivity and the natural reflectivity–rainfall rate variability can affect CB. This RR estimation error component is also compared with the commonly used mean-square error (MSE). A dilemma between the minimization of these two errors is demonstrated. Removing CB from the estimates significantly increases MSE, but minimizing MSE results in a large CB that manifests itself in underestimation of strong rainfalls.

Corresponding author address: Grzegorz J. Ciach, Environmental Verification and Analysis Center, University of Oklahoma, Norman, OK 73069.

gciach@ou.edu

Abstract

The goal of this study is to improve understanding of the optimization criteria for radar rainfall (RR) products. Conditional bias (CB) is formally defined and discussed. The CB is defined as the difference between a given rain rate and the conditional average of its estimates. A simple analytical model is used to study the behavior of CB and its effect on the relationship between the estimates and the truth. This study shows the measurement errors of near-surface radar reflectivity and the natural reflectivity–rainfall rate variability can affect CB. This RR estimation error component is also compared with the commonly used mean-square error (MSE). A dilemma between the minimization of these two errors is demonstrated. Removing CB from the estimates significantly increases MSE, but minimizing MSE results in a large CB that manifests itself in underestimation of strong rainfalls.

Corresponding author address: Grzegorz J. Ciach, Environmental Verification and Analysis Center, University of Oklahoma, Norman, OK 73069.

gciach@ou.edu

Introduction

The goal of radar rainfall (RR) estimation is to provide approximate rainfall values that are as close to the physical truth as possible. Here, the true rainfall is defined as the amount of rainwater that falls on a specified area of the earth’s surface in a specified time interval. The question is how to formalize rigorously the requirement of “closeness to the truth.” The abundance of the existing reflectivity–rainfall rate (Z–R) relationships (Battan 1973) suggests that an unambiguous way to fulfill this apparently obvious demand might not exist. In this paper, it is shown that indeed there are at least two RR estimation performance criteria that are in opposition to each other. One is the commonly used mean-square error (MSE), and the other is a conditional bias (CB).

Generally, CB (it can also be called conditional mean error) can be described as a quantity that shows how the radar rainfall estimates differ, on the average, from a given true rainfall. Formally, we define it as a statistic that quantifies discrepancies between a true rain rate and the conditional average of its estimates, where the averaging is done only for the cases close to the given true rain rate (the conditioning). Large CB can manifest itself in the severe underestimation of strong rainfalls. Demonstrating the behavior of this new RR error component and the role it plays in RR estimation is the main purpose of this study. We apply a conceptual model proposed by Ciach and Krajewski (1999) to illustrate how CB affects the relationship between the RR estimates and the truth and how CB and MSE compete with each other.

The CB problem is fairly well recognized in the statistical literature that discusses the “error-in-variable” effects (Carroll et al. 1995; Seber 1989; Fuller 1987). It appears when the least square regression is used directly to build a predictive relationship in a situation where the predictors are measured with substantial errors. The resulting predictions (by definition) minimize MSE but also systematically underestimate the outcomes in their higher range and overestimate them in their lower range. This systematic discrepancy is known in the statistical literature as an “attenuation” effect, because it can be viewed as significant reduction of the dynamic range of the response. Recently, Rosenfeld and Amitai (1998) encountered the conditional bias problem in the RR context. In their paper, they demonstrate the effect using disdrometer data and argue that their “window probability matching method” removes it. In our study, we attempt to formalize the CB description and to develop an analytical insight into the problem.

This paper is organized as follows. In section 2, we briefly define and discuss the model and the RR estimation formulas used in this study. Next, we define CB and demonstrate its effect on the RR estimates. In section 4, we compare and discuss the CB and MSE errors. A summary and conclusion section ends the study.

The model framework

The conceptual model of radar reflectivity and rain-rate measurements that we use here is a simplified version of the model described and discussed in detail in Ciach and Krajewski (1999). Let us consider a precipitation system in which the relationship between the true near-surface rain rate Rt and the true near-surface radar reflectivity Zt can be described by a one-to-one functional dependency of power-law type:
ZtARbt
where the coefficients A and b are called the relationship multiplier and exponent, respectively. The true reflectivity factor Zt is a physical descriptor of the precipitation volume, as defined by Battan (1973). Both the true rain rate Rt and reflectivity Zt are averaged over the same area over which the radar reflectivity is measured. Also, let the measured radar reflectivity Zm, corrupted with errors, be associated with Zt through the following equation:
ZmZtEz
where Ez is the reflectivity measurement error. From Eqs. (1) and (2), we can obtain the “observation equation” of the modeled system,
ZmARbtEz
which binds the measured Zm to the true rain rates and the errors. We define Rt and Ez as the following independent and lognormally distributed random variables:
i1520-0450-39-11-1941-e4a
where CVr are CVez are the coefficients of variation of the rain-rate process and the reflectivity measurement errors, respectively. Note that, in the above definitions, to simplify further notation and without any significant loss of generality, we assume that the ensemble means of the two random variables are equal to 1. For the Rt values, it is equivalent to converting them into dimensionless quantities by scaling the rain rates with their climatological average. This idealized model is designed to be analytically tractable and to describe qualitatively only those elements of reality that are crucial for the conditional bias question. As a consequence, its solutions can be obtained in a synthetic form of closed mathematical expressions. For a discussion of the model assumptions, see Ciach and Krajewski (1999).
One of the assumptions of the model used here is a fixed power-law ZtRt relationship such as Eq. (1) between the true reflectivity and rain rate. However, our analytical framework is, in fact, more flexible, and this assumption can be relaxed. We can incorporate a simple Z–R variability model into the multiplicative factor A in Eq. (1), assuming that the A parameter is also a lognormal random variable:
i1520-0450-39-11-1941-e5a
where Ao is the average value of the ZtRt relationship multiplier A, and the independent lognormally distributed random variable Ea describes the variability of this multiplier in a real precipitation system. Substituting definition (5a) into Eq. (3), one obtains the modified observation equation
ZmAoRbtEaEz
where the natural Z–R variability factor Ea and the reflectivity measurement error Ez are mixed in a single error factor. The product of Ea and Ez is also a lognormally distributed random variable, independent of Rt, describing the combined effect of the two uncertainty sources in our rainfall observation system. It suggests that the natural Z–R variability can be considered in a simplified way as an increment of the radar reflectivity measurement error, with all its consequences for the CB error discussed in the next sections.
The RR estimation method that we model here is based on a conversion of the radar-measured reflectivities Zm of a precipitation system into radar-estimated rain rates Rr. For this conversion, we need to choose a ZmRr relationship adjusted to a specific situation. The Rr values obtained this way are approximations of the true rain rates Rt averaged over the area over which the radar reflectivities are measured. Within our model framework, this predictive ZmRr relationship is also a power-law function preserving the lognormality of the system. It can be represented in a traditional way as a Z–R relationship:
ZmαRβr
which, for the RR estimation purpose, is rearranged to a mathematically equivalent form,
Rrα−1/βZ1/βm
that is applied to perform the necessary Zm-to-Rr conversion.
We can substitute the observation equation [Eq. (3)] into this predictive equation to express the dependence of the RR estimate on the true rainfall and the errors:
RrAα1/βRγtE1/βzcRγtE1/βz
where the denotations for the exponent ratio γ = b/β and multiplier c are introduced to simplify further derivations. This equation is a basis for our further analysis of the behavior of the RR estimator [Eq. (8)]. Our focus is on the exponent parameters b and β that, as Eq. (9) shows, govern the way the radar estimates Rr are related to different intensities Rt of the true rain rate. First, however, let us consider the multiplier α. For any fixed β, α can be adjusted so that the overall bias of the RR estimation is removed, and the climatological averages of Rr and Rt are equal (Ciach et al. 1997). Representing these averages in our model with statistical expectations, we obtain:
ERrERt
which leads to the following expression for the multiplier in Eq. (9):
i1520-0450-39-11-1941-e11
significantly simplifying all further derivations.

Definition of CB and its effect on RR estimates

The CB can be formally defined with either of the following two expressions:
i1520-0450-39-11-1941-e12a
where rt is a given Rt value (the conditioning), CBm is the multiplicative conditional bias, and CBa is its additive equivalence. Both expressions tell us how the average RR for a given Rt differs from this fixed true rain rate. Equation (12a) expresses it as a dimensionless coefficient, and Eq. (12b) expresses it in absolute rain-rate units. Both statistics can be useful in different situations, and we will utilize them in our further analysis. The CBa averaged over all the rt values is equal to zero. This fact is due to the unbiasedness condition [Eq. (10)] and the “double expectation theorem” [see, e.g., p. 6 in Bickel and Doksum (1977)]. Note, however, that the respective average CBm is not equal to 1, because the expectation of a ratio is not the same as the ratio of expectations.
In the previous section, we already have established the components necessary to analyze the properties of the CB within our model. They can be applied to express it explicitly as a function of given true rain rate and the model parameters. For the multiplicative version, after substituting Eqs. (9) and (11), and after several rearrangements, we obtain
i1520-0450-39-11-1941-e13
Note that for the ZmRr conversion exponent β equal to the model assumed b, that is, for γ = 1, the exponents in Eq. (13) vanish and CBm(rt) = 1 for all rt values. In this, and only this, case we obtain conditionally unbiased rain-rate estimates
RrcRtE1/bz
that relate to the true rainfall in a linear way. For all other values of β, the average RR estimate conditioned on a specified rain rate rt differs from this truth. This nonlinear departure is a function of rt and can be described by CBm(rt) not only within this model but also in analysis of results based on real data, provided that we can estimate it.

Equation (9) and the overall nonbias condition Eq. (10) imply that, if we choose β > b (i.e., γ < 1), then the Rr values obtained based on Eq. (8) underestimate high rain rates and overestimate low rain rates. Statistical literature on the error-in-variable problem (e.g., Fuller 1987) calls this effect attenuation. Note that statistical meaning of the term attenuation (pertaining to the excessive decrease of the estimation’s dynamic range) must not be confused with its use in radar meteorology, where it means the extinction of electromagnetic waves in heavy precipitation. For β < b it works in the opposite direction, causing on the average an “enhancement” of the dynamic range of RR estimates in comparison with the truth. Equation (13) shows that the magnitude of the effect depends on the actual rain-rate intensity rt, the climatological rain-rate variability CVr, and the difference between the predictive and the physical Z–R exponents.

In Fig. 1, we demonstrate the behavior of the multiplicative conditional bias defined by Eq. (12a), as a function of the true rain rate, for a few cases of the assumed model parameters. One can see that using β, which is significantly larger than the physical Z–R exponent b (the case of γ = 0.6), leads, on the average, to underestimation of the high rainfalls by a factor of about 2. To maintain zero overall bias, this underestimation is compensated by the 2 to 3 times overestimation of the weak rainfalls. As a result, the dynamic range of the intensities is much smaller for the RR estimates than for the true rain rates. A comparison of the left and right panels in Fig. 1 shows that this result is not affected much by the climatological variability of the rain rates. If the predictive exponent is smaller than b (e.g., γ = 1.2), the opposite effect occurs.

Comparison of CB and MSE

As we have already stated, we want the predictive ZmRr relationship [Eq. (7)] to result in RR estimates that are as close to the true rain rates as possible. This goal can be achieved through an appropriate adjustment of its parameters α and β. An obvious component of this closeness is the overall nonbias requirement [Eq. (10)] that can be fulfilled by an adjustment of the α multiplier, as shown in section 2. The remaining differences between Rr and Rt can be evaluated in different ways. One meaningful criterion is the multiplicative conditional bias CBm(rt) defined by Eq. (12a). We used it to describe the systematic differences between the RR estimators for low versus high rainfalls. Using Eq. (12b), one can construct a statistic that can be applied to compare CB with MSE. The MSE is a traditional statistic, often used to characterize the random estimation errors. Within our model framework, MSE can be simply defined as follows.
RrRtERrRt2
where the variance Var is equal to the mean square of the RR–truth difference from the unbiasedness condition [Eq. (10)].
To compare MSE in a meaningful way with the CB effect, we must first express the conditional bias using a statistic that is compatible with MSE. The multiplicative and additive CBs discussed before are functions of given true rain rate. On the other hand, the MSE for given parameters of the system is a number and cannot be compared directly with a function. Thus, we need to reduce the function to a measure that characterizes globally its magnitude. The appropriate measure is a quantity that we can call a “total additive conditional bias,” given by
totaRtEaRt2
where the definition of the additive CB is given by Eq. (12b). Again, the variance is equal to the mean square because the expectation of the CBa(Rt) is equal to zero, as stated in the previous section. The CBtot measures the conditional bias effect synthetically for all of the rt values, is an analytically tractable quantity, and, being also a variance, is the same kind of statistic as MSE. Formally, it is possible to define many global measures of the CB error. However, the above advantages make the CBtot a good choice for our purposes.
To compare the behavior of MSE and CBtot, we first need to derive explicit expressions for the two statistics as functions of the ZmRr exponent β and other parameters of our model. For MSE, we expand Eq. (15) and substitute Eq. (9) to obtain
MSE = c2E{R2γtE2/βz}−2cE{Rγ+1tE1/βz}+E{R2t}.
This equation, after replacing c with Eq. (11) and several rearrangements, yields
γ[(CV2r + 1)(CV2ez + 1)1/b2]γ2 − 2(CV2r + 1)γ+ CV2r + 1,
which, for any coefficients of variation CVr and CVez and exponent b, is a function of the argument γ = b/β.
We now proceed with the total conditional bias defined with Eq. (16). The additive bias definition Eq. (12a), after substituting Eqs. (9) and (11), can be expanded as follows:
i1520-0450-39-11-1941-e19
This equation, after substituting to Eq. (16) and several rearrangements, leads to the following result:
totγ(CV2r + 1)γ2CV2rγCV2r
which is similar to Eq. (18), except that it is free of the error term.
Analysis of functions MSE(γ) and CBtot(γ) shows that they are both concave and have single minima. For γ = 0 (i.e., in the limit of large β values), they have the same value and MSE(0) = CBtot(0) = CV2r. This result is natural because, in this case, the RR estimates are equal to the climatological average rain rate, and the error of such predictions is the same as the rainfall variability. For large values of γ, both functions go to infinity. The argument values minimizing MSE(γ) and CBtot(γ), however, differ significantly. They can be found by differentiating Eqs. (18) and (20) and setting the derivatives equal to zero. The MSE has its minimum for the following value of the β parameter:
i1520-0450-39-11-1941-e21
which agrees with the result in Ciach and Krajewski (1999) obtained based on different considerations. On the other hand, minimum of the total CB error [Eq. (20)] is for βcb = b(γ = 1), and, at this point, the conditional bias simply vanishes, that is, CBtot(1) = 0. The minimum for the total conditional bias function is, of course, in perfect agreement with the results in the previous section regarding the multiplicative bias.

Note that the βms value that optimizes MSE is always bigger than b. For given physical parameters b and CVr of the modeled precipitation system, the difference between βms and βcb = b minimizing the two error variances depends on the level of the reflectivity measurement errors described by CVez. Figure 2 demonstrates the relative behavior of the MSE and CB error expressions derived above, as a function of the γ parameter, for a few cases of the model parameters. In the left panel, one can see that, for moderate reflectivity measurement errors (CVez = 1), βms is about 22% larger than the βcb exponent value, whereas, for large errors (CVez = 2), the difference is almost 40%. For the model with larger climatological rain-rate variability (CVr = 4 in the right panel), the effect is slightly smaller.

From Fig. 2 one can immediately see the dilemma that arises when the β exponent of the predictive ZmRr conversion is tuned to make the RR estimates as close as possible to the truth. Inherently, one cannot simultaneously minimize both the MSE and CBtot errors. Minimization of MSE (the most common statistical optimization method) leads to substantial conditional bias distortions. On the other hand, choosing an RR estimator that has no conditional bias (CBtot = 0 for γ = 1) can only be done at the cost of a significant increase of MSE. For moderate reflectivity measurement errors, MSE is larger than its minimum by a factor of about 1.7. However, for higher measurement errors, MSE is about 3 times larger for the conditionally unbiased estimates than at the minimum point and is even bigger than MSE at γ = 0 (climatological average prediction).

Note that, in this framework, there is no conflict of the overall unbiasedness condition and CBtot (or MSE). This fact is most likely also true in real data setups. Not all performance criteria compete, even if they pertain to different aspects of a product. The knowledge of which criteria are in conflict and which can be adjusted independently is necessary to control the optimization process properly.

Summary and conclusions

We investigated a new radar rainfall estimation performance criterion, called the conditional bias error, using a simple analytical model. After a short presentation of the model, we formally defined two useful descriptors of CB. We then demonstrated the behavior of this quantity and showed how it affects the relationship between the RR estimates and the true rain rates. We also compared the conditional bias with the commonly used mean square estimation error, pointing out an acute dilemma between reduction of CB and minimization of MSE. We showed that the natural variability of the physical Z–R relationship can add to the CB effect in the same way as the reflectivity measurement errors.

Our study shows that nonzero CB in the RR products indicates that the RR values are not linearly related to the corresponding true rainfall values and that CBtot is a general and convenient global measure of the magnitude of this distortion that can also be used for real data analysis. It is an undesirable systematic effect that one would like to remove from the RR products. However, the cost of removing CB from the RR estimates can be high in terms of the MSE increase. On the other hand, minimizing MSE can result in strong underestimation of heavy rainfalls and overestimation of weak rain rates. This dilemma is an inevitable consequence of the uncertainties involved in the RR estimation process and cannot be solved with the uncertainties present. Finding a compromise between the two extreme RR estimation strategies most likely requires careful recognition of the prevailing priorities in a specific application of the hydrological radar–rainfall products. Such a discussion is beyond the scope of this paper, and the tradeoff between MSE and CB is considered here only in a descriptive way as a fact of nature. However, in our opinion, one should be aware of the conditional bias effects and be able to quantify them in order to make conscious decisions about the RR estimation schemes used in practice. Practical problems of efficient estimation of the CB errors using uncertain reference data will be one of the topics of our future research.

We hope that this study will promote better understanding and more rigorous definitions of the optimality criteria for the RR estimation/validation procedures. It may also be viewed as the next step toward a parsimonious model of the RR error structure and the efficient estimation of its parameters. Pursuing these difficult goals will also be a subject of our future research effort.

Acknowledgments

This work was supported by Oklahoma EPSCoR Grant NCC 5-171, and NASA Grant NAG 6-2084. This support is gratefully acknowledged. We also acknowledge an anonymous reviewer of the paper by Ciach and Krajewski (1999) whose remarks inspired this study.

REFERENCES

  • Battan, L. J., 1973: Radar Observation of the Atmosphere. University of Chicago Press, 324 pp.

  • Bickel, P. J., and K. A. Doksum, 1977: Mathematical Statistics: Basic Ideas and Selected Topics. Holden–Day, 492 pp.

  • Carroll, R. J., D. Ruppert, and L. A. Stefanski, 1995: Measurement Error in Nonlinear Models. Chapman and Hall, 305 pp.

  • Ciach, G. J., and W. F. Krajewski, 1999: Radar–rain gauge comparisons under observational uncertainties. J. Appl. Meteor.,38, 1519–1525.

  • ——, ——, E. N. Anagnostou, M. L. Baeck, J. A. Smith, J. R. McCollum, and A. Kruger, 1997: Radar rainfall estimation for ground validation studies of the Tropical Rainfall Measuring Mission. J. Appl. Meteor.,36, 735–747.

  • Fuller, W. A., 1987: Measurement Error Models. John Wiley and Sons, 440 pp.

  • Rosenfeld, D., and E. Amitai, 1998: Comparison of WPMM versus regression for evaluating Z–R relationships. J. Appl. Meteor.,37, 1241–1249.

  • Seber, G. A. F., 1989: Nonlinear Regression. John Wiley and Sons, 768 pp.

Fig. 1.
Fig. 1.

Multiplicative conditional bias as a function of given true rain rate. Two cases of true-rain-rate variability coefficients are shown, CVr = 2 (left panel) and CVr = 4 (right panel). In each panel, conditional biases for three values of the exponent ratios γ = b/β are presented (descriptions are at the corresponding curves).

Citation: Journal of Applied Meteorology 39, 11; 10.1175/1520-0450(2000)039<1941:CBIRRE>2.0.CO;2

Fig. 2.
Fig. 2.

Behavior of the total conditional bias and MSE as a function of the exponent ratio γ. The CBtot is represented by the dotted curves. The solid curves show MSE for two values of the variation coefficient CVez of the reflectivity measurement errors (descriptions are at the corresponding curves). Two cases of true-rain-rate variability coefficients are considered, CVr = 2 (left panel) and CVr = 4 (right panel).

Citation: Journal of Applied Meteorology 39, 11; 10.1175/1520-0450(2000)039<1941:CBIRRE>2.0.CO;2

Save