## 1. Introduction

The Special Sensor Microwave/Imager (SSM/I), which was initially launched onboard a Defense Meteorological Satellite Program (DMSP) spacecraft in June of 1987, was the first operational sensor for rainfall retrieval using passive microwave technology. SSM/I was also the first sensor with channels at 85 GHz, providing scattering information for land-based retrievals. Rain rate is one of several parameters identified as essential to the mission of the Department of Defense. Prior to the launch of the first SSM/I in June of 1987, the United States Navy contracted the Hughes Aircraft Company to develop a number of geophysical retrieval algorithms from the microwave data gathered by the new sensor (Hollinger et al. 1987). The original algorithms, referred to as the D-Matrix algorithms after the approach used for their development, suffered from a number of deficiencies but provided many insights into the information content and usefulness of the new microwave sensor.

The SSM/I sensor evolved from the early success of the passive microwave remote sensors on the *Nimbus-5* and *Nimbus-6* satellites in retrieving rainfall from space. The Electronically Scanning Microwave Radiometer (ESMR) 5 was limited to rainfall estimation over ocean since it only measured emission at 19 GHz. The later ESMR-6 showed some potential in discriminating rain over land at 37 GHz, using the contrast between the land surface and the combined emission and scattering due to raindrops. Radiative transfer modeling by Paris (1971), Savage and Weinman (1975), and Savage (1976, 1978) predicted that scattering by large raindrops and hail would produce cold brightness temperature anomalies over land. The modeling results from Savage (1976) also suggested that the discrimination of rainfall fields over land could be improved with the addition of a channel near 90 GHz. Wilheit et al. (1982) independently provided observational evidence for the significant scattering due to ice in rain clouds at 92 GHz. While the D-Matrix algorithm being produced for the SSM/I rainfall retrieval did not incorporate the effects of scattering due to ice, these effects were addressed in subsequent SSM/I rainfall algorithms.

In subsequent calibration and validation efforts for the SSM/I sensor and the resulting D-Matrix retrieval products, the navy contracted the Calibration/Validation (Cal/Val) group (Hollinger 1991) to develop new and improved retrieval algorithms for its operational use. The resulting Cal/Val rainfall algorithm replaced the D-Matrix algorithm for operational use until it too was replaced in July 1995 by a third-generation algorithm developed by the National Oceanic and Atmospheric Administration/National Environmental Satellite, Data and Information System (NOAA/NESDIS) (Ferraro and Marks 1995). A further improvement, including emission over the ocean, is being considered for implementation in the current operational scheme.

Throughout the early development of SSM/I rainfall-retrieval algorithms, the operational navy algorithms have been used by many researchers as a baseline against which subsequent algorithm development efforts have been measured. Results from the first- and second-generation operational navy rainfall algorithms were submitted to a number of intercomparison projects including the current PIP-2 project. Because of the widespread use and availability of these algorithms, their continued use has provided perhaps the best indication of the progress that has been made in the development of algorithms for the retrieval of rainfall from SSM/I. This paper details the original specifications for both the D-Matrix and Cal/Val algorithms, discusses their revision histories, and evaluates the performance of these algorithms with respect to each other and to other algorithms throughout a series of five intercomparison projects.

## 2. The Navy/Hughes D-Matrix algorithm

The original navy operational rainfall-retrieval algorithm, developed prior to the launch of the first SSM/I,is referred to as the D-Matrix algorithm. A detailed explanation of the development and implementation of the D-Matrix algorithm is given in the SSM/I user’s guide (Hollinger et al. 1987). This original algorithm was in fact a combination of several different algorithms covering both land and ocean as well as various seasons and locations.

Due to seasonal and latitudinal variations in both the ocean background and rainfall characteristics, the D-Matrix algorithm uses different linear regression coefficients depending on the region and time of year. A total of 11 climate codes were designated for the D-Matrix algorithm to account for differences in storm system characteristics, as well as atmospheric conditions and changes in surface emissivity.

^{−1}, where

*j*is the climate code, and the

*T*

_{B}are the observed brightness temperatures for the indicated frequencies and polarizations. The D-Matrix algorithm is an emission-based algorithm over the ocean (OCN), using a linear combination of the lower frequency channels, as shown in Eq. (1). The D-Matrix algorithm over land (LND) uses the 37V and 85V channels and is given by Eq. (2):

The algorithm coefficients were originally determined using a combination of radiative transfer and geophysical models as well as climatology. A screening test to determine the presence of rain is made by checking for values of *T*_{B19H} > *R*_{0} and (*T*_{B37V} − *T*_{B37H}) < *R*_{1}. The coefficient values as well as the designation for each of the 11 climate codes are given in Tables 1 and 2.

### a. Problems with the D-Matrix algorithm

The original D-Matrix rainfall algorithm was found to have a number of serious problems. The most significant of these was unrealistic rainfall estimates over land, which were the result of neglecting the effects of scattering by ice particles in the radiative transfer simulations. In addition, large discontinuities between climate zones made it impossible to intercompare results outside of the Tropics. While the D-Matrix algorithm for the Tropics appeared to produce realistic rainfall estimates, biases at higher latitude and discontinuities in the transition zones between the climate regimes defined in Table 1 limited its usefulness.

Figure 1a shows the distribution of the mean rain rate calculated from the original D-Matrix algorithm withlatitude, averaged for all nonzero rainfall estimates. As shown in the figure, the average estimated rain rates increase dramatically with latitude. In addition, sharp discontinuities are evident at the climate zone boundaries located at 25° and 35° latitude.

Figure 1b shows the distribution of average rain rates using only the tropical summer coefficients from the D-Matrix algorithm. The strong latitudinal bias evident in Fig. 1a is even more pronounced at higher latitudes when the tropical coefficients are applied to the midlatitude regions. It is evident from these figures that the D-Matrix algorithm does not properly account for atmospheric and surface contributions to the observed brightness temperatures. Simulated retrievals with the D-Matrix algorithm using a radiative transfer model suggest that this latitudinal bias is due to changes in both sea surface temperatures (SSTs) and water vapor (Berg 1993). While the approach taken in the original D-Matrix algorithm using different coefficient values for the various climate codes accounts in part for the latitudinal bias, the resulting discontinuities shown in Fig. 1a indicate that this approach is not satisfactory.

A comparison of results from the D-Matrix algorithm with ocean coverage radar data from both Darwin and Kwajalein was performed for the Tropics as part of the SSM/I Cal/Val effort (Hollinger 1991). These results aresummarized along with the Cal/Val algorithm statistics in Table 3. The results indicate a bias in the D-Matrix algorithm of 0.64 mm h^{−1} and a correlation with the radar data of 0.63 for the Tropics. The correlation value is much lower, however, for rain rates greater than 0.5 mm h^{−1}, with a correlation value of only 0.224. This low correlation for significant rain rates suggests serious problems with the retrieval of rainfall intensities. Specifics of the radars and further details of this comparison are given in the discussion of the Cal/Val algorithm development effort in section 3. The problem of identifying the structure of precipitating systems is discussed in more detail in the result summaries of the intercomparison projects given in sections 4 and 5.

### b. A modified version of the D-Matrix algorithm

In order to correct for discontinuities between different climate codes, as well as latitudinal and seasonal biases, modifications were made to the original D-Matrix algorithm. Although changes in SST and water vapor are highly variable, it was determined that corrections to the latitudinally and seasonally biased D-Matrix estimates could be made for monthly or long-term averaged estimates, at least over the tropical regions. From an analysis of hourly rain gauge data over the Tropics,an average rain rate of 4 mm h^{−1} was found with no significant latitudinal bias over the Tropics (Berg 1993). Based on this observation, an empirically derived cosine weighting function was computed from 3.5 yr of the D-Matrix monthly rainfall estimates for each of the 12 calendar months.

*C*(

*i, m*) are the polynomial coefficients where

*m*is the season,

*l*is the latitude, and

*W*

_{m}(

*l, m*) is the resulting weighting function. The original D-Matrix rain-rate estimate is RR and the modified values is RR*. As Fig. 1b indicates, this weighting scheme has little effect on estimates in the Tropics (20°S–20°N) but provides a significant adjustment to midlatitude estimateswithout introducing the discontinuities seen in Fig. 1a. Although the weighted rain-rate adjustment is made for instantaneous estimates, this scheme was developed for the purpose of computing monthly averages over the Tropics. Instantaneous estimates were submitted to several of the intercomparison projects, however, and a discussion of those results is given in sections 3 and 4. A more detailed discussion of the modifications made to the D-Matrix algorithm along with the weighting coefficient values is given by Berg (1993).

Although the weighting scheme that was adopted to correct for latitudinal biases was empirically derived, the modified D-Matrix algorithm provided more realistic rainfall estimates than most of the other SSM/I rain-retrieval algorithms available at the time. It was especially valuable for producing monthly or time-averaged estimates (Berg and Avery 1995). While better algorithms have been developed subsequently, as is discussed in sections 4 and 5, the problems and limitations of the D-Matrix algorithm provided valuable information for the development of the Cal/Val and other SSM/I rain-retrieval algorithms and established a baseline for measuring subsequent improvements.

## 3. The navy Cal/Val algorithm

During the SSM/I Cal/Val program, the original D-Matrix rain-rate retrieval algorithm was replaced by a revised algorithm, which was based upon statistical regression of SSM/I brightness temperatures against collocated surface radar measurements of rain rate. *DMSP-F8* SSM/I data were obtained from archives maintained by the Naval Research Laboratory. Radar data were collected from the NOAA/TOGA (Tropical Ocean Global Atmosphere) radar at Darwin, Australia, and the Kwajalein radar at the Tropical Rainfall Measuring Mission (TRMM) Ground Validation site in the Marshall Islands.

Operating at 5.3-cm wavelength (C-band) with a 1.7° beamwidth, the NOAA/TOGA radar was located on the northern coast of Australia at 12.5°S, 130.9°E. The radar provided coverage over both the Australian continent and the Timor and Arufura Seas during special observing periods from 1988 to 1991. A coincident network of 23 tipping bucket rain gauges provided an independent test of the radar calibration at the surface.

The Kwajalein radar site was maintained as a forerunner to the TRMM surface validation radar network. This radar operated at 10.2 cm (S-band), with a 2.2° beamwidth. Located on the Kwajalein atoll at 8.7°N, 167.7°E, the radar provided data over the surrounding Pacific Ocean from 1988 to 1992. Ten rain gauges in the vicinity of the radar were available for calibration studies.

### a. SSM/I and radar data geolocation

Since rainfall is highly variable in both space and time, accurate coregistration of both the satellite andradar data is required for proper statistical comparisons. Initial operational satellite navigation procedures resulted in errors as great as 30 km in the observed position of land–ocean boundaries in the *DMSP-F8* SSM/I imagery. The geolocation procedure utilized an optimization routine to search for corrections in the spacecraft effective pitch and yaw angles that maximized the correlation between coastal discontinuities and the 85.5-GHz horizontally polarized SSM/I brightness temperature imagery and the known location of coastal boundaries as specified in the World Data Base II coastline map. Upon review of 20 relocated SSM/I images, the automated procedure located the satellite data to within 6 km of the World Data Base II coastline, which has an uncertainty of approximately 3 km.

### b. Processing of radar data and collocation with SSM/I

The sampling interval for all-channel SSM/I measurements is 25 km both along and cross track. Since the highest-resolution channels have footprint dimensions of only ∼15 km (see Hollinger et al. 1987), a comparative rain-rate measurement consistent with the sampling limitations of the SSM/I was chosen for statistical analysis. The area-average rain rate within a 625 km^{2} (25 km × 25 km) circular area centered on each SSM/I all-channel measurement was used as a standard for comparison in this study. Area-average rain rates were derived from the Darwin and Kwajalein radar observations.

In order to maximize the correlation between the radar rain estimates and surface rainfall amounts, only low antenna elevation angle (⩽1.1°) plan-position indicator radar scans were used in the Cal/Val study. Also, since the height at which the radar beam intercepts the cloud and the sample volume increases with range, no radar measurements beyond a range of 220 km were considered. Ground clutter and obvious radar artifacts were also screened. The remaining bin reflectivities from the Darwin and Kwajalein radars were converted to rain rate using the Marshall and Palmer relationship (*Z* = 200*R*^{1.6}: Marshall and Palmer 1948) and then interpolated to a 5-km Cartesian grid.

Typically three 5-km rain-rate grids bracketing a given *DMSP-F8* overpass time were utilized in the final ground truth processing. All gridded rain rates falling within a 625 km^{2} circular area of a given SSM/I seven-channel scene were time-interpolated to the SSM/I measurement time. The time-interpolated, gridded rain rates were then area-averaged and stored along with the corresponding SSM/I sensor data record brightness temperatures (Hollinger et al. 1987).

Data from 11 *DMSP-F8* SSM/I overpasses of Darwin and nine overpasses of Kwajalein were collocated with the averaged radar rain data to produce the ground truth dataset. The Darwin overpasses occurred during February and March 1988, while the Kwajelein overpasses spanned the months of August–November 1988.

### c. Regression-based algorithms

*R*as a function of the collocated SSM/I brightness temperature measurements

*T*

_{Bi}were obtained by regressing the brightness temperatures against rain rates in a stepwise multiple linear regression procedure. From limited cloud radiative transfer modeling experiments, it was determined that the nonlinear relationship between brightness temperature and rain rate could best be expressed by regressing brightness temperatures against ln(

*R*+

*c*), where

*c*is a constant. This produces a rain rate retrieval formula of the typeThis regression formula generally allowed for a better fit to the data than a simple linear fit. The reason for this is that the response of the SSM/I channels to changes in rain rate generally diminishes with increasing rain rate. Separate formulas were developed for data over land and ocean surfaces (see Table 4), since the response of the SSM/I channels to rain over land and ocean differs substantially. The difference in response is due to the much greater microwave emissivity of land surfaces compared to ocean surfaces.

### d. Regression statistics

Various constants *c* were tested in each regression formula; the statistically optimal formulas for land and ocean backgrounds, both with and without the 85.5-GHz SSM/I data, are listed in Table 3, along with the statistics of the D-Matrix algorithm applicable to tropical data. The statistics were stratified in each regression formula test. The first row statistics from each test are based upon all validation data that were sufficiently removed from coastal boundaries and for which both the D-Matrix and radar produced rain rates between 0 and 25 mm h^{−1}. The second row statistics are for only those collocated data corresponding to D-Matrix and radar rain rates greater than or equal to 0.5 mm h^{−1}. The statistics are stratified to compensate for the naturally skewed rain-rate distribution, which is dominated by light rainfall events.

It should be noted that the revised regression formulas were based upon a less restricted dataset than the one used in Table 3. This set included 342 collocated observations over land and 1365 observations over ocean. The data in Table 3 do not include observations that were flagged as nonprecipitating or not retrievable (i.e., coastal) by the D-Matrix algorithm screening logic. It should also be noted that nearly all of the radar-derived area-averaged rain rates were less than 6 mm h^{−1}. As aresult, no high-intensity rain rates were included in the collocated dataset. This fact more than any other may help to explain why in all of the intercomparisons, which are discussed in sections 4 and 5, it was observed that the Cal/Val algorithm appears to underestimate rain rates, especially for intense storm systems. Finally, it is important to recognize that a regression-based retrieval formula is expected to yield the mean of the dependent data (in this case, the mean of the collocated radar rainfall rates) given a set of observed brightness temperatures. Therefore, a set of “high” rain rates that exceeds the mean but that corresponds to the same set of observed brightness temperatures will be underestimated by the regression formula.

As indicated by Table 3, the revised regression formulas yield estimates of rain rate over both land and ocean that are superior to the D-Matrix estimates, even if the 85.5-GHz data are eliminated from the analysis. It is curious to note that, over land, the regressions without the 85.5-GHz data appear to yield slightly moreaccurate rain-rate estimates than the “all-channel” regressions. This result can be attributed to the stepwise regression channel selection procedure, which adds or removes channels based upon the partial correlation of available channel data with the rain-rate data. Since the 85.5-GHz data are typically most highly correlated with rain rate over land, these channels were selected first in the all-channel formula; however, competitive regression formulas based upon the 37-GHz and lower-frequency data may also be realized.

During the Cal/Val effort, the 85-V channel began to fail in December 1987 and eventually became unusable by May 1988. To meet the operational requirements of the U.S. Department of Defense, the entire Cal/Val team was ordered to develop alternative algorithms in a timely fashion, which could be used with the *F8* sensor. An intermediate algorithm for rain rate was developed, which appears in the Cal/Val report volume I (Hollinger 1989). Although this algorithm was never implemented operationally, it is worth mentioning for historical purposes. Eventually, near the end of the Cal/Val effort, the 85-H channel became noisy and, subsequently, algorithms for use without any 85-GHz measurements were delivered. These appear in volume II of the Cal/Val report (Hollinger 1991) and are described in Table 4 of this paper. They were also used in the AIP-2 intercomparison (see section 4b).

### e. Screening logic

In addition to the exponential regression formulas represented by Eq. (5), the revised Cal/Val Navy algorithm was fitted with improved logic for the screening of nonraining regions over land and ocean based upon SSM/I brightness temperatures. Over land, the Hughes negative polarization test for bad data was augmented by screening tests suggested by M. J. McFarland and C. Neale (1991, personal communication) for the Cal/Val program. The land screening is applied for the purpose of identifying and removing the effects of snow, cold land surfaces, deserts, etc. (see Ferraro et al. 1998). Over ocean, a discriminant function developed by Olson et al. (see Hollinger 1991) was applied to eliminate false rain signatures near coasts. Coastal SSM/I data were not processed. The revised navy algorithm recommended at the end of the Cal/Val program is presented in Table 4.

Although the revised rain algorithm was developed based upon collocated SSM/I and radar data from the Tropics, due to the lack of well-calibrated, continuously archived radar data at higher latitudes, the algorithm was recommended for use at midlatitudes until improvements could be implemented.

## 4. Results from the GPCP algorithm intercomparisons

A series of three algorithm intercomparisons have been undertaken as part of the Global Precipitation Climatology Project (GPCP), which was established by the World Climate Research Programme to provide climate researchers with global precipitation statistics for the period from 1986 to 1995. Rainfall estimates from visible, infrared, and passive microwave satellite measurements, as well as combinations of these spectral bands, were included in the intercomparisons. Results from a modified version of the D-Matrix algorithm were submitted for all three AIPs, and results from different versions of the Cal/Val algorithm were submitted for the second and third AIPs. A summary of the results is given by Ebert et al. (1996), while the results of the individual intercomparison projects are provided in the reports by Lee et al. (1991), Allam et al. (1993), and Ebert (1996). The following discussion focuses specifically on the performance of the D-Matrix and Cal/Val algorithms.

### a. AIP-1 (Japan)

The first AIP was conducted over Japan and the surrounding ocean regions during June–August 1989. Results from a total of 27 algorithms were submitted, including eight that used only passive microwave observations from SSM/I. The D-Matrix algorithm is referred to as the Ferriday algorithm in the AIP-1 report (Lee et al. 1991). Because this intercomparison took place prior to the development of the Cal/Val algorithm, it was not included.

A composite rainfall dataset produced from over 1300 automated rain gauges in the Automated Meteorological Data Acquisition System network along with approximately 20 precipitation radars was produced by the Japan Meteorological Agency for validation of the AIP-1 satellite estimates. Comparisons of the microwave retrieval algorithms were made over monthly (1–30 June and 15 July–15 August) and instantaneous timescales. The monthly estimates were averaged over 1.25° latitude–longitude boxes, while the instantaneous estimates were binned into 0.125° longitude by 0.1° latitude boxes for comparison with the validation data.

The complete summary of results for AIP-1 is given by Lee et al. (1991). A more comprehensive overview of the GPCP project and AIP-1 in particular is also given by Arkin and Xie (1994). A summary of the statistics resulting from a comparison of the D-Matrix estimates with the validation data for the two monthly periods is given in Table 5. Because estimates were submitted from the ocean algorithm only and limited validation data were available over the ocean, the statistics are based on a small sample of points (only 25 bins).

The correlation for the June monthly total was the second highest among the SSM/I algorithms, while both the mean and rms errors were the lowest of the algorithms submitted. In general, all of the algorithms had much lower correlations for the July–August period (several were negative). The large difference in the statistics between the two periods may be the result of differences in the rainfall systems or other meteorological or surface conditions. The magnitude of this difference, however, suggests that either poor sampling by the satellite or problems with the validation data might also have been contributing factors.

For the instantaneous comparisons, a total of 10 SSM/I swaths were selected. The instantaneous statistics are also given in Table 5; however, these numbers were taken from a plot in the AIP-1 report and may not be exact. Over these 10 swaths, the D-Matrix algorithm had a correlation of around 0.85 and, in general, had among the highest correlation and lowest errors of all the microwave algorithms submitted. Qualitative comparisons of the D-Matrix swath values with the other algorithms indicated generally reasonable rainfall patterns, although because few significant rain events occurred over the ocean for these 10 cases, the results were not very meaningful.

AIP-1 provided the first real intercomparison of both infrared and microwave satellite rainfall-retrieval techniques, and the results clearly demonstrated the value of passive microwave for estimating rainfall. This was especially evident for instantaneous snapshots where the SSM/I retrievals performed significantly better than any of the infrared-based techniques. However, because no distinction was made between those algorithms that provided estimates over ocean only and those providing estimates over land and ocean, the validation statistics did not provide a fair intercomparison of the algorithms submitted. While the D-Matrix algorithm appeared to perform as well as any of the others over the ocean, limitations of the validation data and the statistical analysis made it impossible to make any meaningful conclusions as to the relative performance of the algorithms.The lessons learned from this initial project, however, proved to be very valuable for the subsequent intercomparisons.

### b. AIP-2 (The United Kingdom and western Europe)

The second AIP was conducted over the United Kingdom and a large region of western Europe during February–April 1991. Results from 19 SSM/I retrieval algorithms were submitted, including results from both the modified D-Matrix algorithm and two versions of the Cal/Val algorithm (one version using the 85-GHz data and one without). A summary of the results is given in the AIP-2 workshop report (Allam et al. 1993). In the report, the D-Matrix algorithm is referred to as the Berg algorithm, while the Cal/Val algorithms are referred to as Ferraro Cal/Val 1 and 2.

Validation data for AIP-2 were provided from a network of precipitation radars augmented by a numerical weather prediction (NWP) model. Due to poor quality of rainfall data from the European radar network, the NWP model data was used as ground truth. A comparison of rain gauge data with the radar totals indicated serious underestimation of rainfall by the radars, which could not be attributed to uniformly poor calibration. Although corrected monthly radar totals were produced based on the gauge data, NWP estimates were used to supplement the radar observations.

Statistics were computed for the SSM/I estimates versus the validation data for an integrated set of 20 cases. A summary of the statistics for the three algorithms versus both the radar and NWP model validation data are given for ocean only and land–coast–ocean coverage in Table 6. For the combined land–coast–ocean scenes correlations between the SSM/I estimates and the radar validation data were extremely low, ranging from −0.02 to 0.55. The correlations with the NWP model data were similarly poor, and although coast contamination was determined to be responsible for some of the low values, the ocean-only cases faired just as poorly. Since almost none of the correlations for any of the algorithms had any statistical significance, the value of the comparison in determining the relative performance of the algorithms was rather limited. The Cal/Val algorithm with the 85-GHz channel had among the best correlations of all the algorithms, however, as shown in Table 6.

### c. AIP-3 (TOGA COARE)

The last in the series of AIPs coincided with the intensive observing period of the Tropical Ocean GlobalAtmosphere Coupled Ocean–Atmosphere Response Experiment (TOGA COARE), which took place from November 1992 to February 1993 over the western tropical Pacific. A total of 55 satellite rainfall algorithms were submitted including 29 algorithms for retrieving rainfall from SSM/I. Data from two shipboard 5.37-cm Doppler radars deployed to measure rainfall during the IOP were used as validation for the third AIP. Estimates from both the modified D-Matrix and the Cal/Val algorithms were submitted for AIP-3. In the AIP-3 report (Ebert 1995), the modified D-Matrix algorithm is referred to as the BE1 algorithm, while the Cal/Val algorithm is referred to as the BA0 algorithm. The results from the Cal/Val algorithm submitted to AIP-3 were multiplied by two as a result of the findings of PIP-1, which is discussed in the next section.

The satellite estimates were averaged over 0.5° latitude–longitude bins and compared over both monthly and instantaneous timescales. In addition, a number of case studies focusing on various types of rain systems were investigated in more detail. A summary of the statistical results from AIP-3 for the D-Matrix and the Cal/Val algorithms is given in Table 7. The rain estimates from all algorithms submitted to the tropical AIP-3 intercomparison showed much higher correlations with the validation data than those that were obtained from either the first AIP over Japan or the second AIP over western Europe. This was likely due to a combination of better validation data and a higher sensitivity of the microwave brightness temperatures to the type of convective systems found in the western Pacific during TOGA COARE. It was found, however, that almost all of the satellite estimates were a factor of 1.5–2.0 higher than the radar-derived rain rates. Further calibration of the radar values has led to an increase in the radar-derived rain rates, although the magnitude of most of the satellite-derived rain rates is still higher than the radar values.

The statistics of the comparison were much more useful for determining the relative performance of the algorithms and for intercomparisons with algorithms based on geostationary infrared observations and blended infrared–microwave techniques. It became apparent from AIP-3 that the microwave algorithms were clearly superior to the infrared techniques for instantaneous estimates, but for monthly or long-term averaged estimates, the higher temporal sampling provided by the geostationary satellite provided better results.

Based upon the three cruise average statistics versus radar, all of the SSM/I retrievals had high correlations (0.75–0.85), with both the D-Matrix and Cal/Val algorithms performing very well. The Cal/Val algorithm had a correlation of 0.849 with the radar-derived values compared to a correlation of 0.816 for the D-Matrix algorithm, although the D-Matrix algorithm had a lower bias and rms error. The factor of 2 correction applied to the Cal/Val estimates appeared to improve the time-averaged performance of this algorithm, at least with respect to its performance in the PIP-1 intercomparison.

For the instantaneous comparisons and case studies, the Cal/Val algorithm clearly provided better estimates than the D-Matrix algorithm. As shown in Table 7, the D-Matrix algorithm had a low probability of detection. In the instantaneous comparisons, the D-Matrix algorithm detected 13% of the pixels as raining as opposed to 69% determined from the radar data. This suggests that this algorithm had difficulty detecting low rain rates. The D-Matrix algorithm also did not perform as well as many of the other algorithms in the case studies. It appeared that the D-Matrix algorithm provided reasonable time-averaged estimates but suffered from a thresholding problem and therefore did not perform nearly as well for instantaneous retrievals. The Cal/Val algorithm, on the other hand, performed very well for both time-averaged and instantaneous cases, although it had higher biases and rms errors for several of the comparisons.

## 5. Results from the WetNet precipitation intercomparisons

### a. PIP-1 (Global monthly estimates)

The first WetNet precipitation intercomparison project (PIP-1) concentrated on intercomparing a short 4-month (July–November 1987) global rainfall climatology. The results of this intercomparison are documented in a PIP-1 special issue of *Remote Sensing Reviews* (Barrett et al. 1994). Results from a total of 15 SSM/I algorithms were submitted to PIP-1, including both the Cal/Val algorithm (referred to as SJG) and the modified D-Matrix algorithm (referred to as BER). Aswith previous intercomparison projects, estimates from the D-Matrix algorithm were provided over ocean only.

Quantitative results from PIP-1 were based on comparisons with Pacific atoll rain gauge data and the Global Precipitation Climatology Centre (GPCC) data (Morrissey et al. 1994) and are given in Table 8. The atoll data were limited to islands in the western Pacific. The correlations for the monthly totals were modest for the D-Matrix and Cal/Val algorithms at 0.70 and 0.69, respectively. In this statistical comparison both algorithms performed reasonably well with respect to the others. It should be noted that the D-Matrix and the MSU estimates submitted to PIP-1 were calibrated from the atoll data, while the GPI estimates were calibrated from other rain gauge data. As a result, the rms errors were among the lowest of all the SSM/I algorithms.

Qualitative comparisons of the global estimates gridded over 0.5° latitude–longitude boxes over monthly and 4-month periods were also performed. In addition, several time–latitude and time–longitude sections were selected for comparison. While the resulting rainfall distribution from the D-Matrix algorithm appeared reasonable, the estimates from the Cal/Val algorithm are significantly lower than those from most of the other algorithms. In addition, the frequency of precipitation from the Cal/Val algorithm was determined to be much higher than the rain frequency indicated by many of the other algorithms. The suggestion that the Cal/Val estimates are around a factor of 2 too low led to a doubling of the Cal/Val estimates submitted for the AIP-3 intercomparison.

An additional comparison of the time–latitude and time–longitude estimates from 10 of the algorithms submitted to PIP-1, including the D-Matrix and Cal/Val algorithms, with shipboard observations has been conducted by Petty (1997). Petty’s results indicated excellent agreement between the frequency of precipitation in the Tropics from the D-Matrix results and the shipboard observations. In the extratropics, however, the ship observations indicated a significantly higher frequency of precipitation than the D-Matrix estimates, although the D-Matrix algorithm appears to generally reproduce the frequency of heavy precipitation observed by the ships. This same result was found for most of the other SSM/I algorithms in Petty’s comparison. The frequency of precipitation from the Cal/Val algorithm was much higher, however, and, while it showed betteragreement with the ship data at high latitudes, it also appeared to overestimate the frequency of precipitation in the intertropical convergence zone (ITCZ) by a factor of 2–3.

The results from Petty (1997) generally support observations from the intercomparisons, at least in the Tropics, suggesting a low rainfall threshold in the Cal/Val algorithm, which results in false alarm rainfall signatures for nonprecipitating clouds. Because no validation of the modified D-Matrix algorithm was performed outside of the Tropics, these findings support the conclusion that estimates from the D-Matrix algorithm cannot be relied upon outside of the Tropics. Overall, the D-Matrix estimates compared favorably in the Tropics both quantitatively with the validation data and qualitatively with respect to the other algorithms. The Cal/Val estimates did not fare as well for producing climatological rainfall totals, as it appeared to overestimate the rain area and underestimate the rain amount over the Tropics.

### b. PIP-2 (Case studies)

The second WetNet precipitation intercomparison project detailed in this paper provides by far the best opportunity for intercomparing instantaneous rainfall estimates for a wide variety of storm types and regions. Of the 28 cases containing 123 overpasses, a subset of eight overpasses from eight different cases were selected for further qualitative comparison of the D-Matrix and Cal/Val algorithms. The selected cases include two from each of the four rain system types defined by Smith et al. (1998), including tropical cyclones, midlatitude cyclones, convective systems, and stratiform systems. Although only the modified D-Matrix estimates were submitted to PIP-2, estimates from both the original D-Matrix algorithm and the modified version are shown in Figs. 2–5 along with the Cal/Val estimates. Estimates from Ferraro’s algorithm (Ferraro and Marks 1995), which has been adopted as the third-generation navy algorithm, are also included. For PIP-2 the Cal/Val implementation used the common screen coastal detection scheme instead of the wide coastal mask used in the estimates for PIP-1.

Statistics from a comparison of the D-Matrix, Cal/Val, and Ferraro algorithm estimates with the available validation data, from 36 of the 123 PIP-2 cases, aregiven in Table 9. As indicated in the table, both algorithms had relatively low correlations over ocean regions. The D-Matrix algorithm had a correlation of 0.30, while the Cal/Val correlations were 0.44 over ocean and 0.47 over land, compared to slightly lower correlations of 0.41 over ocean and 0.45 over land for the Ferraro algorithm. Average correlations with the ocean validation data from all the PIP-2 algorithms were slightly higher at 0.51. As indicated by these results, however, the relatively poor correlations with the validation data suggests limited value of the quantitative analysis. Additional quantitative results from PIP-2 as well as a discussion of these results is presented by Smith et al. (1998).

A qualitative comparison of the PIP-2 results indicates a number of basic differences between the D-Matrix and Cal/Val algorithms, which are illustrated by the eight PIP-2 cases presented in this study and shown in Figs. 2–5. First, the areal coverage of rainfall retrieved by the Cal/Val algorithm is consistently greater than the areal coverage of rain from the D-Matrix code. This difference can be attributed to the less restrictive screening logic in the Cal/Val algorithm, especially over ocean areas. Over ocean, only coastal data that may be contaminated by nearby land surface emission are screened in the Cal/Val. Also, over both land and ocean, there is no minimum rain-rate threshold imposed on retrievals from the Cal/Val algorithm. These factors result in fairly widespread areas of light rain retrieved by the Cal/Val, in which rain rates of 1 mm h^{−1} or less cover a fairly large percentage of the raining area. Screening of the D-Matrix retrievals results in significantly smaller rain areas, with minimum rain rates typically around 2 mm h^{−1}.

It is also apparent from the PIP-2 cases that the maximum intensities of rain retrieved by the Cal/Val algorithm are less than those obtained from the D-Matrix and significantly less than estimates from the Ferraro algorithm. For example, in Hurricane Andrew maximum rain rates approximately 7 mm h^{−1} southwest of the Florida peninsula are retrieved using D-Matrix, whereas a maximum of around 5 mm h^{−1} is produced by the Cal/Val algorithm at roughly the same location. An even more dramatic difference is seen in the southern United States squall line case shown in Fig. 5. In comparisonto the Ferraro algorithm, the original D-Matrix algorithm produces a much larger area of rainfall with maximum rain rates >15 mm h^{−1} over Louisiana and Mississippi. Maximum rain rates retrieved by the Cal/Val code are ∼10 mm h^{−1} just south of the Louisiana coast.

The differences in retrieved rain-rate magnitudes may be attributed to differences in the algorithm formulations. The D-Matrix algorithm was based upon statistical model fits to radiative transfer calculations. The main deficiency in this formulation was that no ice-phase precipitation (i.e., snow, graupel, or hail) was included in radiative transfer calculations through the assumed precipitating clouds. Ice-phase precipitation is nearly always present in deep-convective clouds, such as those found in the rainbands of tropical cyclones and squall lines (see Houze 1989). The presence of ice-phase precipitation causes brightness temperatures to decrease with rain rate at 85.5 GHz over land, due to microwave scattering by these particles. Since ice-phase precipitation was not included in the D-Matrix algorithm formulation, decreases in brightness temperature at 85.5 GHz over land are attributed to excessive amounts of rainfall, rather than much smaller increases in ice precipitation water content. As a result, the D-Matrix algorithm tends to overestimate rain rates in convective situations over land, as shown in Figs. 3 and 5 and confirmed by the Cal/Val tropical zone radar validation (Olson et al., in Hollinger 1991). Over ocean, the 85.5-GHz data are not utilized by the D-Matrix algorithm, and the scattering effect of ice precipitation is much smaller at 37 GHz and the lower SSM/I frequencies. Therefore, a smaller positive bias in rain estimates due to the ice effect is expected.

In addition to the intense rainfall estimates provided by the original D-Matrix algorithm over land, the original formulation also suffers from difficulties over coastal boundaries indicating a strong sensitivity to changes in the background emissivity. This conclusion is supported by the large increases in mean rain rates at higher latitudes, as shown in Fig. 1. Both the original and modified versions of the D-Matrix algorithm poorly resolve the spatial structure within severe storm events, although the modified version appears to wash out these variations even more so at higher latitudes as shown in Figs. 3 and 5. In the intense cyclones, the rain ratesdrop from significant values to zero at the boundaries showing no low-intensity outer boundary as would be expected in most systems. It is apparent that the D-Matrix algorithms provide more reasonable estimates in the tropical regions shown in Figs. 2 and 4 with extreme rain rates over land being a more significant problem at higher latitudes.

The Cal/Val algorithm displays the same tendencies of much broader coverage of precipitation and lower intensities for all of the four storm system types. The Ferraro algorithm produces significantly higher rain intensities and a smaller spatial extent of precipitation over all types of storm systems than either the Cal/Val or D-Matrix algorithms. Although the Cal/Val algorithm shows lower rain rates, even in the intense hurricanes, the internal structure of the storms appears to agree reasonably well with the estimates from the Ferraro algorithm.

The Cal/Val algorithm was based upon statistical regression analysis of SSM/I brightness temperatures and collocated radar rain rates. The primary limitation in this approach was the lack of higher radar rain rates (only about 10 data points greater than 5 mm h^{−1}) in the regression database. Therefore, higher rain rates associated with convective rainfall had negligible influence on the derived regression relationships. This factor is responsible for the underestimates of rain rates by the Cal/Val algorithm in convective (or higher rain rate) regions. Another difficulty with this approach is that a slight spatial misregistration of the SSM/I and radar datasets tends to decorrelate the brightness temperatures and radar rain rates, especially if the features to be collocated are small scale. Since convective rain regions typically have scales ∼10 km or less, and since collocation errors are of the same order, the Cal/Val regression relationships possibly do not reflect the full sensitivity of brightness temperatures to rain rates in convective regions.

## 6. Operational considerations

As discussed in section 5b, the D-Matrix algorithm suffers from a number of difficulties. The most serious of these are artificially high rainfall estimates and poor screening over land. As a result, the land-retrieval algorithm is inadequate for operational use. In addition, the discontinuities between climate code boundaries and poor screening for surface ice at high latitudes severely limits the value of the ocean algorithm. Finally, the lack of structure observed in the PIP-2 cases suggests that the D-Matrix algorithm is clearly inadequate for producing operational estimates, especially given the recent improvements made in SSM/I rainfall retrieval algorithms.

For operational use, the Cal/Val algorithm appears to be a good rain–no rain discriminator, although it provides poor estimates in high-intensity rain cells. Because it uses an emission-based approach over ocean and ascattering approach over land, it cannot retrieve rain rates over coastal regions or over sea ice. In order to determine land or sea-ice contamination, however, the Cal/Val algorithm requires surface-type flags.

## 7. Climatological applications

As demonstrated by PIP-2 and several of the other intercomparison projects, the D-Matrix algorithm in both its original and modified forms appears to do a poor job of capturing the structure of many types of precipitation systems. The PIP-1 results show, however, that the D-Matrix algorithm still provides reasonable monthly estimates over the Tropics and, to a limited extent, even over the midlatitudes. The ability of the simple D-Matrix algorithm to provide reasonable time-averaged estimates of precipitation when the instantaneous estimates suffer from a variety of problems and errors demonstrates an important result for satellite rainfall estimation. Many of the significant sources of error for the instantaneous estimates cancel or average out, leaving a bias in time-averaged estimates, which can be removed by subsequent calibration with rain gauge or other validation data. As the results from AIP-3 demonstrated, although estimation techniques based on passive microwave data from SSM/I produce significantly better instantaneous rainfall estimates, due to temporal sampling considerations the geostationary infrared-based techniques produced better time-averaged estimates (Ebert 1996).

Climatological applications of the Cal/Val algorithm have been minimal as the focus of the Cal/Val development effort was with operational retrievals in mind. As discussed in the PIP-1 results section, the monthly estimates produced by the Cal/Val algorithm had significantly lower magnitudes than many of the other algorithms, as well as larger regions of low precipitation amounts resulting from a lower rain–no rain threshold value. As a result, the D-Matrix algorithm appears to provide better climatological estimates, at least over the tropics.

## 8. Summary and conclusions

The contribution of rainfall estimates from the D-Matrix and Cal/Val rainfall algorithms to the three AIPs and two PIPs to date has served to stimulate subsequent algorithm development as well as provide a baseline measure for assessing the progress of SSM/I rainfall algorithm development. The inclusion of estimates from the D-Matrix and Cal/Val algorithms to the intercomparisons to date have clearly shown the improvements made in SSM/I rainfall retrieval algorithms since the initial launch of the instrument. A clear indicator of this improvement is the progression from favorable results, albeit for rather limited ocean coverage, in the first AIP to the more recent AIP-3 and PIP-2 projects. The more recent intercomparisons have revealed the limited abilityof the D-Matrix algorithm to capture the spatial structure of precipitating systems and to recover lower rain rates, a common problem with earlier algorithms that has been addressed in subsequent development efforts.

Through the series of intercomparisons a number of deficiencies in the D-Matrix algorithm have become evident. As discussed in section 2, serious problems in the land rainfall-retrieval algorithm were found as a result of neglecting the effect of scattering by ice particles in the radiative transfer model used for the development of the D-Matrix algorithm. Screening problems in the original algorithm have also served to identify this as a significant issue for rainfall retrievals (Ferraro et al. 1997). Over ocean regions, the D-Matrix algorithm suffers from discontinuous features in time-averaged estimates between climate zones and poor representation of the structure within storms.

Modifications made to account for the discontinuities between climate zones have seemingly improved the performance of the algorithm for time-averaged estimates over the Tropics, but for instantaneous retrieval applications, the ability of the algorithm to retrieve the storm system structure is still inadequate. This is especially true in extratropical regions where it is suggested that the frequency of precipitation detected by the D-Matrix algorithm is significantly lower than truth (Petty 1997). The ability of the D-Matrix algorithm to provide reasonable estimates of monthly precipitation, however, underscores the relative dependence of various estimation errors on the subsequent application of the rainfall estimates.

The second generation Cal/Val algorithm has been shown to provide a significant improvement in the retrieval of instantaneous estimates of rainfall over both land and ocean from the first-generation D-Matrix algorithm. Although the algorithm was calibrated from observations of tropical storms at Kwajalein and Darwin, it does surprisingly well over land in the extratropics where the cloud microphysics is decidedly different. Results from PIP-2 and other more recent intercomparison projects, however, indicate an underestimation of rain rates in most cases, as well as the detection of overly broad regions of precipitation. It is somewhat surprising that the algorithm does reasonably well statistically in the detection of rain–no rain when compared to limited validation cases, while at the same time it appears to overestimate the precipitation coverage in the large-scale integrated totals. This is possibly due to the character of the validation datasets and the comparative underestimation of rain areas by many of the other SSM/I retrieval algorithms, such as the D-Matrix. If the validation datasets contain many cases of shallow “warm rain” (i.e., no scattering), this may explain the good performance of the algorithm in mid- and upper-latitude oceans, as well as the gross underestimation of rain in deep convection over ocean and land.

## Acknowledgments

This work is dedicated to Dr. James Hollinger, leader of the navy’s SSM/I Cal/Val program. The authors wish to thank all of the members of the SSM/I Cal/Val team for their contributions to the development of the algorithm, especially Mark Goodberlet for his help with SSM/I data geolocation. David Wolff, Lynn Rose, Brian Morrison, and David Brown provided invaluable validation radar data for this study.

## REFERENCES

Allam, R., G. Holpin, P. Jackson, and G. L. Liberti, 1993: Second Algorithm Intercomparison Project of the Global Precipitation Climatology Project (AIP-2): U.K. and North-West Europe February–April 1991. Satellite Image Applications Group, U.K. Meteorological Office, Bracknell, Berkshire, United Kingdom, 136 pp.

Arkin, P. A., and P. Xie, 1994: The Global Precipitation Climatology Project: First algorithm intercomparison project.

*Bull. Amer. Meteor. Soc.,***75,**401–419.Barrett, E. C., and Coauthors, 1994: The first WetNet precipitation intercomparison project (PIP-1): Interpretation of results.

*Remote Sens. Rev.,***11,**303–373.Berg, W., 1993: Estimation and analysis of climate-scale rainfall over the tropical Pacific. Ph.D. dissertation, University of Colorado, 202 pp. [Available from University of Colorado, Boulder, CO 80309.].

——, and S. K. Avery, 1995: Evaluation of monthly rainfall estimates derived from the special sensor microwave/imager (SSM/I) over the tropical Pacific.

*J. Geophys. Res.,***100,**1295–1315.Ebert, E. E., 1996: Results of the 3rd Algorithm Intercomparison Project (AIP-3) of the Global Precipitation Climatology Project (GPCP). Bureau of Meteorology Research Centre, Rep. 55, Melbourne, Australia, 199 pp. [Available from BMRC, GPO Box 1289K, Melbourne, Victoria 3001, Australia.].

——, M. J. Manton, P. A. Arkin, R. E. Allam, and A. Gruber, 1996:Results from the GPCP algorithm intercomparison program.

*Bull. Amer. Meteor. Soc.,***77,**2875–2887.Ferraro, R. R., and G. F. Marks, 1995: The development of SSM/I rain-rate retrieval algorithms using ground-based radar measurements.

*J. Atmos. Oceanic Technol.,***12,**755.——, E. A. Smith, W. Berg, and G. J. Huffman, 1998: A screening methodology for passive microwave precipitation retrieval algorithms.

*J. Atmos. Sci.,***55,**1583–1600.Hollinger, J., 1989: DMSP Special Sensor Microwave/Imager Calibration/Validation. Vol. 1. Naval Research Laboratory Final Rep. Washington, DC, 186 pp. [Available from Naval Research Laboratory, Washington, DC 20375.].

——, 1991: DMSP Special Sensor Microwave/Imager Calibration/Validation. Vol. 2. Naval Research Laboratory, Final Rep. Washington, DC, 305 pp. [Available from Naval Research Laboratory, Washington, DC 20375.].

——, R. Lo, and G. Poe, 1987: Special Sensor Microwave/Imager user’s guide. Naval Research Laboratory, Washington, DC, 120 pp. [Available from Naval Research Laboratory, Washington, DC 20375.].

Houze, R. A., Jr., 1989: Observed structure of mesoscale convective systems and implications for large-scale heating.

*Quart. J. Roy. Meteor. Soc.,***115,**425–461.Lee, T. H., J. E. Janowiak, and P. A. Arkin, 1991:

*Atlas of Products from the Algorithm Intercomparison Project 1: Japan and Surrounding Oceanic Regions June–August 1989.*University Corporation for Atmospheric Research, 131 pp. [Available from Climate Analysis Center, NOAA, Washington, DC 20233.].Marshall, J. S., and W. M. Palmer, 1948: The distribution of raindrops with size.

*J. Meteor.*,**5,**165–166.Morrissey, M., M. A. Shafer, H. Hauschild, M. Reiss, B. Rudolf, W. Reuth, and U. Schneider, 1994: Surface data sets used inWetNet’s PIP-1 from the Comprehensive Pacific Rainfall Data Base and the Global Precipitation Climatology Centre.

*Remote Sens. Rev.,***11,**61–91.Paris, J. F., 1971: Transfer of thermal microwaves in the atmosphere. Ph.D. dissertation, Texas A&M University, 210 pp.

Petty, G. W., 1997: An intercomparison of oceanic precipitation frequencies from 10 SSM/I rain rate algorithms and shipboard present-weather reports.

*J. Geophys. Res.,***102,**1757–1777.Savage, R. C., 1976: The transfer of thermal microwaves through hydrometeors. Ph.D. dissertation, University of Wisconsin—Madison, 144 pp.

——, 1978: The radiative properties of hydrometeors at microwave frequencies.

*J. Appl. Meteor.,***17,**904–911.——, and J. A. Weinman, 1975: Preliminary calculations of the upwelling radiance from rain clouds at 37.0 and 19.35 GHz.

*Bull. Amer. Meteor. Soc.,***56,**1272–1274.Smith, E., and Coauthors, 1998: Results of WetNet PIP-2 Project.

*J. Atmos. Sci.***55,**1483–1536.Wilheit, T. T., A. T. C. Chang, J. L. King, and E. B. Rodgers, 1982:Microwave radiometric observations near 19.35, 92, and 183 GHz of precipitation in tropical storm Cora.

*J. Appl. Meteor.,***21,**1137–1145.

The definition and limits of the D-Matrix climate zones as well as the rainfall criteria limits over both ocean and land.

Coefficients for the D-Matrix algorithm.

Multichannel algorithm statistics for the D-Matrix and Cal/Val algorithms over tropical land and ocean. Listed for each regression formula test are the number of data in the sample, the mean rain rate of the sample, the bias of the rain-rate estimate with respect to radar, the error standard deviation of the rain-rate estimate, and the correlation coefficient. The optimal values for the coefficient *c,* from Eq. (5), are also given for the different formulations. For each test, statistics based upon a second data sample, which includes only collocated data for which the D-Matrix (*E*) and radar rain rates (*R*) are greater than or equal to 0.5 mm h^{−1}, are presented to indicate errors in the rain-rate estimates at higher rainfall rates. Note that all negative regression estimates of rainfall rate are set equal to 0 mm h^{−1} in the analysis, which is consistent with the D-Matrix application.

Screening logic and rain rate formulas for the Cal/Val algorithm.

Statistics for the D-Matrix algorithm vs validation data for AIP-1.

Comparison statistics between the D-Matrix and Cal/Val algorithms vs radar validation data for AIP-2.

Comparison statistics for the D-Matrix and Cal/Val algorithms vs the updated radar validation data for AIP-3. Statistics for both instantaneous estimates and averaged over all three cruises are given. In addition the correlation, bias, and rms error, the probability of detection (POD) and the false alarm ratio (FAR) are given. The units on the mean, bias, and rms errors are in mm for the three cruise average statistics and in mm h^{−1} for the instantaneous statistics.

Comparison statistics for monthly rainfall composites from the D-Matrix and Cal/Val algorithms vs the validation data for PIP-1. The validation data consisted of Pacific atoll rain gauge measurements and GPCC data, which is a dataset of mainly continental monthly rainfall observations generated by the GPCC. The units for all the statistics are given in mm month^{−1}, with the exception of the normalized correlation coefficient and the bias. The bias is defined as the estimated mean divided by the observed mean.

Comparison statistics for the D-Matrix and Cal/Val instantaneous rainfall retrievals vs the available validation data for PIP-2 using the original algorithm screening techniques and not the PIP-2 common screen. Values are also given for the Ferraro algorithm for purposes of comparison. The units for all the statistics are given in millimeters per hour with the exception of the normalized correlation coefficient. In this case, the bias is defined as the difference between the observed mean and the estimated mean.