1. Introduction
Several satellite programs are under way to measure precipitation especially in remote areas such as the tropical oceans (see Simpson et al. 1988; Wilheit et al. 1991;Theon et al. 1992). This paper concerns itself with the ground validation program of the Tropical Rainfall Measuring Mission (TRMM), which was launched in November 1997 (Simpson et al. 1988) and has now generated over a year of data from several sensors. One problem inherent in all such missions is that of ground validation. We present here an analysis of validating the satellite estimations with point gauge measurements. This is a complex comparison because the two sensors are measuring different quantities: 1) the point gauge measures precipitation nearly continuously in time at a point on the surface, while 2) the satellite measures a snapshot in time (actually about a 5–10-min average) of an area average over its field of view (FOV) at the surface (typically 20 km across). In comparing two simultaneous measurements the point gauge may be located at some random position within the FOV. While in the long run these two measurements should agree, there is likely to be a large random (zero mean) difference between the two because of the different space–time sampling configurations. It should be possible, however, by taking enough simultaneous pairs of measurements to compare them and check for bias in the satellite retrieval algorithm.
There are various forms of ground validation for precipitation. One is comparing horizontal precipitation patterns from ground-based or airplane-based radars. This method is very useful in judging the qualitative patterns of the precipitation, but because of the uncertainty in the conversion of radar reflectivity to surface rain rate, it cannot alone answer the question of whether rain rates derived from the satellite are biased. Hence, we turn to a more certain estimate of the rain rate at the ground (the point gauge) but surrender the certainty of matching sampling designs.
We proceed in our purely theoretical study by asking about the size of the random errors incurred in comparing the two designs, satellite and point gauge. We make the null hypothesis that both measurements are correct and see what the distribution of differences is due solely to the differences in sampling design. In principle, we can reduce these kinds of random errors by collecting more and more pairs and averaging. We seek the number of such pairs that would allow us to see a real bias in the satellite measurement if it should occur. The question is will a few such pairs suffice to reveal a significant bias or not. As we shall see in this somewhat idealized study it takes many months of data to reduce the random error enough to detect a bias of 10%. This figure comes from assuming an FOV of about 20 km and an autocorrelation distance of a few kilometers. Larger autocorrelation lengths will lead to smoother fields and therefore a smaller number of pairs to detect bias at the 10% level.
Our approach is to take a hypothetical FOV and subdivide it into M subareas, which we call tiles. We then construct a random field for the rain rate, which is somewhat like real rain. The random field is constructed by drawings from a Bernoulli distribution where values in neighboring tiles are independent. This effectively means that the horizontal correlation length is about half a tile width. This model of rain rates is of course highly simplified but provides some insight into the problem since it allows mostly analytical results that can be interpreted easily.
In this paper, the point gauge is used as the ground-truth measurement to validate satellite precipitation retrieval algorithms at the FOV spatial level (typically about 20 km).
Because the probability distribution of real rain has a large nonzero contribution at zero rain rate (usually greater than 90%), many of the visits will lead to (no rain, no rain) measurement pairs or perhaps (no rain, rain) pairs, where the second entry is the FOV average. For this reason, we can consider the following three ground-truth designs based on the point gauge measurement.
Design 1 uses all visits even though either of the two measurements (gauge, satellite) may have no rain.
Design 2 throws out all the visits when the FOV average has no rain. Note that when the FOV has no rain, the gauge also has no rain.
Design 3 throws out all the visits when the gauge has no rain. Note that the FOV average can have rain even though the gauge has no rain.
We use a spatial white noise Bernoulli random field as the rain-rate model. This means that for each tile within the FOV it is either raining at (fixed) rate r or it is not raining at all. For a particular overpass of the satellite and for a given tile the probability of rain being nonzero is p. While this is a fairly crude as a model of rain fields, simplicity is important here to establish the main principles involved in the problem. We derive the probability density function, ensemble mean, and the variance of gauge measurements for each design. At the same time, we examine the relationship between ground-truth designs proposed here and evaluate each design.
To keep the numbers specific and roughly relevant to TRMM we choose some parameters to have nominal values such as the FOV width to be 20 km. The finest resolution taken for the tiles within the FOV used in simulating the random field is 4 km, which is the same resolution as Global Atmospheric Research Program Atlantic Tropical Experiment (GATE) radar data. This resolution is also consistent with the fact that the satellite actually measures 5–10-min averages of the rain rate. Such time averaging is thought to be roughly equivalent to spatial averaging to about 4 km.
2. Definitions
The ground-truth design must satisfy two conditions to detect the retrieval bias of the retrieval algorithm we want to check. 1) The error εdi = Ψsi − Ψgi must have no bias, that is, 〈εdi〉 = 〈Ψsi − Ψgi〉 = 0. If in addition to a retrieval bias, there is a bias due to the design itself, which we refer to as design bias, we could mistakenly attribute an error to the retrieval algorithm that is actually inherent in the design. (In reality both types of bias are likely to be present). 2) The mean-square error 〈
3. Ground-truth designs for Bernoulli random field
We use the Bernoulli distribution as the distribution of rain rate to compare the ground-truth designs described above because this distribution describes the no-rain phenomenon of real rain in its simplest possible form. Let x of the M tiles be raining with rain rate r, and in the rest of the M − x tiles there is no rain. We assume that the probability of rain rate in an individual tile is p and one tile is independent of the other. In this section, we present some numerical values for the 4-km resolution (as in GATE radar data). This means that when we increase the number of tiles M, the FOV increases. In order to change the tile size for fixed FOV size and to see the effect of the tile size using the results in this section, one needs to adjust the probability of rain p and rain rate r because it is dependent on the resolution.
a. The error distributions
b. Mean of the error
c. Mean-square error for a single visit
In section 3b, we showed that design 3 cannot be used as a ground-truth design, so we henceforth only consider design 1 and design 2 as ground-truth designs and compute the mean-square error for each.
The values of rmse(d1) and rmse(d2), which are the square root-mean-square errors for design 1 and design 2, are given in Fig. 3. Note that rmse(d1) and rmse(d2) increase if the probability of rain p increases where 0 < p < 0.5. When the FOV increases, rmse(d1) increases but rmse(d2) decreases. This result derives from the probability that the satellite measurement, if it has rain, will increase if the FOV size increases. When the rain rate r increases, rmse(d1) and rmse(d2) increase. In Fig. 4, note that rmse(1) and rmse(2) are almost the same for a rain rate r = 4, the probability of rain p = 0.1, and typical FOV size (20 km across). The values r = 4 and p = 0.1 are chosen to match rain-rate statistics from GATE data.
d. Mean-square error for N visits
When the probability of rain p is small (usually less than 10%), the number of visits N1 and N2 are almost the same. Remember that N1 is the number of all visits and N2 is the number of visits when the satellite has rain.
We have already found that W2(d1) and W2(d2) are almost the same for the values p = 0.1, r = 4, and FOV size 20 km. For these values, the number of visits to achieve 10% of the standard deviation of the gauge measurement is N1 = 96 for design 1 and is N2 = 97 for design 2. Since the probability that the satellite measurement has rain PM = 1 − (1 − p)M is almost 1 for p = 0.1, we obtained almost the same number of visits to achieve 10% tolerance level.
4. Summary and conclusions
In this paper we have considered three ground-truth designs based on the point gauge measurement to validate the satellite measurement. Design 1 uses data pairs from all visits. Design 2 uses visits only when the FOV average has rain. Design 3 uses the visits only when the gauge has rain. The ground-truth designs we proposed in this paper are based upon properties encountered with real rain.
We modeled the rain-rate field by subdividing the FOV into M tiles and took the random rain variable for each tile to be rain at a fixed rate r with probability p, or no rain to be 1 − p. The number of tiles in a fixed FOV size corresponds in some sense to the autocorrelation length of the rain field. Once this model is adopted all comparisons between the point gauge (randomly located in the FOV) and the FOV average corresponding to the satellite measurement can be calculated.
We found that the error distribution is bimodal for design 1 and design 2 but not for design 3. We have shown that the satellite measurement is an unbiased estimator of the gauge measurement for design 1 and design 2. However, design 3 has a serious disadvantage as the ground-truth design because it exhibits a large design bias.
The efficiencies of design 1 and design 2 are indexed by the mean-square error (difference) between the satellite and gauge estimates. We have shown that the mean-square error of design 2 is equal to the mean-square error of design 1 divided by the probability that the satellite measurement has rain inside the FOV. This fact gives us a way to compute the mean-square error of design 2 without using the conditional random field. The dimensionless mean-square errors of design 1 and design 2 are almost the same for the typical rain statistics.
Our major finding is that for an FOV width of 20 km and autocorrelation length of a few kilometers, the number of measurement pairs (containing rain) necessary to distinguish a bias of 10% is of the order of 100. This means many months of data will need to be taken for a single-point gauge to detect such a bias with any confidence.
Our model of rain is too simplified and needs to be made more realistic at the cost of clarity. The Bernoulli field is too uniformly “speckly” compared to real rain, which consists of “patches” of speckly areas. We are presently looking at fields that have this more realistic property.
Acknowledgments
The first author (E.H.) wishes to thank the Korea Research Foundation for its support. The second author (G.R.N.) thanks the NASA TRMM program for its support.
REFERENCES
Bell, T. L., A. Abdullah, R. L. Martin, and G. R. North, 1990: Sampling error for satellite-derived tropical rainfall: Monte Carlo study using a space–time stochastic model. J. Geophys. Res.,95, 2195–2205.
Ha, E., and G. R. North, 1994: Use of multiple gauges and microwave attenuation of precipitation for satellite verification. J. Atmos. Oceanic Technol.,11, 629–636.
North, G. R., J. B. Valdes, E. Ha, and S. P. Shen, 1994: The ground-truth problem for satellite estimates of rain rate. J. Atmos. Oceanic Technol.,11, 1035–1041.
Simpson, J., R. F. Adler, and G. R. North, 1988: A proposed rainfall measuring mission (TRMM) satellite. Bull. Amer. Meter. Soc.,69, 278–295.
Theon, J. S., T. Matsuno, T. Sakata, and N. Fugono, Eds., 1992: The Global Role of Tropical Rainfall. A. Deepak, 280 pp.
Thiele, O. W., 1992: Ground truth for rain measurement from space. The Global Role of Tropical Rainfall, J. S. Theon, T. Matsuno, T. Sakata, and N. Fugono, Eds., A. Deepak, 245–260.
Wilheit, T. T., and L. S. Chiu, 1991: Retrieval of monthly rainfall indices from microwave radiometric measurements using probability distribution functions. J. Atmos. Oceanic Technol.,8, 118–136.
Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 623 pp.
Schematic diagram for FOV of L km on a side.
Citation: Journal of Atmospheric and Oceanic Technology 16, 12; 10.1175/1520-0426(1999)016<1949:EAFSGV>2.0.CO;2
The probability density function of the error (=satellite measurement − gauge measurement) for Bernoulli random field. A rain rate of 4 mm h−1 and FOV size 20 km × 20 km (resolution is 4 km) are used. The probability of rain is p = 0.1 and p = 0.5.
Citation: Journal of Atmospheric and Oceanic Technology 16, 12; 10.1175/1520-0426(1999)016<1949:EAFSGV>2.0.CO;2
The absolute value of the bias of the error (=satellite measurement − gauge measurement) for Bernoulli random field when the resolution is 4 km.
Citation: Journal of Atmospheric and Oceanic Technology 16, 12; 10.1175/1520-0426(1999)016<1949:EAFSGV>2.0.CO;2
The root-mean-square errors for design 1 and design 2 of Bernoulli random field when the resolution is 4 km.
Citation: Journal of Atmospheric and Oceanic Technology 16, 12; 10.1175/1520-0426(1999)016<1949:EAFSGV>2.0.CO;2
The dimensionless root-mean-square errors for design 1 and design 2 of the Bernoulli random field when the resolution is 4 km. Key: design 1, solid line; design 2, dotted line.
Citation: Journal of Atmospheric and Oceanic Technology 16, 12; 10.1175/1520-0426(1999)016<1949:EAFSGV>2.0.CO;2
The schematic distribution of the error for N visits. The Bernoulli random field with rain rate r = 4 mm h−1, the probability of rain p = 0.1, and the FOV size 20 km × 20 km (resolution is 4 km) are used. Key: N = 30 visits, solid line; N = 100 visits, dotted line.
Citation: Journal of Atmospheric and Oceanic Technology 16, 12; 10.1175/1520-0426(1999)016<1949:EAFSGV>2.0.CO;2