Abstract

Spaceborne microwave sensors provide critical rain information used in several global multisatellite rain products, which in turn are used for a variety of important studies, including landslide forecasting, flash flood warning, data assimilation, climate studies, and validation of model forecasts of precipitation. This study employs 4 yr (2003–06) of satellite data to assess the relative performance and skill of the Special Sensor Microwave Imager [SSM/I (F13, F14, and F15], Advanced Microwave Sounding Unit [AMSU-B (N15, N16, and N17)], Advanced Microwave Scanning Radiometer for Earth Observing System [AMSR-E (Aqua)], and the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI) in estimating surface rainfall based on direct instantaneous comparisons with ground-based rain estimates from the TRMM Ground Validation (GV) sites at Kwajalein, Republic of the Marshall Islands (KWAJ), and Melbourne, Florida (MELB). The relative performance of each of these satellite estimates is examined via comparisons with space- and time-coincident GV radar-based rain-rate estimates. Because underlying surface terrain is known to affect the relative performance of the satellite algorithms, the data for MELB were further stratified into ocean, land, and coast categories using a 0.25° terrain mask. Of all the satellite estimates compared in this study, TMI and AMSR-E exhibited considerably higher correlations and skills in estimating–observing surface precipitation. While SSM/I and AMSU-B exhibited lower correlations and skills for each of the different terrain categories, the SSM/I absolute biases trended slightly lower than AMSR-E over ocean, where the observations from both emission and scattering channels were used in the retrievals. AMSU-B exhibited the least skill relative to GV in all of the relevant statistical categories, and an anomalous spike was observed in the probability distribution functions near 1.0 mm h−1. This statistical artifact appears to be related to attempts by algorithm developers to include some lighter rain rates, not easily detectable by its scatter-only frequencies. AMSU-B, however, agreed well with GV when the matching data were analyzed on monthly scales. These results signal to developers of global rainfall products, such as the TRMM Multisatellite Precipitation Analysis (TMPA) and the Climate Data Center’s Morphing (CMORPH) technique, that care must be taken when incorporating data from these input satellite estimates to provide the highest-quality estimates in their products.

1. Introduction

Precipitation is a key component of the earth’s hydrological cycle and an important factor in climate change because of its association with the global transport and distribution of water and latent heat. Until the advent of meteorological satellites and the development of remote sensing techniques for measuring precipitation from space, there was no observational system capable of providing accurate estimates of precipitation at global scales. Precipitation, moreover, is one of the most difficult physical quantities to measure accurately because of extreme variability both temporally and spatially, and insufficient observations over the planet’s oceans.

Since the early 1970s, satellites have been used to quantitatively estimate precipitation by observing the emission and scattering processes associated with clouds and precipitation in the atmosphere. Multichannel passive microwave remote sensing techniques hold the most promise because these instruments sample rain systems at different depths, and subsequently can be used to physically probe precipitation in its different thermodynamic states. Since the launch of the single-channel Electrically Scanning Microwave Radiometer (ESMR) aboard the Nimbus-5 in 1972, many other multichannel microwave platforms have been deployed in space and used to estimate surface rainfall over a significant fraction of the earth’s surface.

However, a single orbiting platform does not collect enough rain information to accurately estimate precipitation everywhere on the earth’s surface. The Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), U.S. Department of Defense Special Sensor Microwave Imager (SSM/I), National Oceanic and Atmospheric Administration (NOAA) Advanced Microwave Scanning Radiometer for Earth Observing System (EOS) (AMSR-E), and Advanced Microwave Sounding Unit (AMSU), for example, each provide about two samples per day from any localized region of the earth’s surface. Statistical sampling on regional scales can thus be significantly improved by combining the rain information from a multitude of satellites. These passive microwave satellite platforms are currently being used to develop high-resolution global estimates of precipitation using varying types of merging of the various satellite observations, including the TRMM Multisatellite Precipitation Analysis (TMPA) (Huffman et al. 2007), the Climate Data Center’s Morphed (CMORPH) analysis (Joyce et al. 2004), and the Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN) (Hsu et al. 1997; Sorooshian et al. 2000), all of which provide near-global 3-hourly estimates at 0.25° resolution. Identifying the strengths and weaknesses of the satellite measurements associated with each sensor class is critical to understanding the measurement and error characteristics of the multisatellite products used in various applications, including climate modeling.

Rain estimates determined from ground-based sensors provide an independent source of validation for inferring the error characteristics of estimates derived from spaceborne remote sensors. However, to eliminate the temporal sampling component of the error, it is necessary to match both the satellite and Ground Validation (GV) estimates in time and space. In previous studies, Ferraro and Marks (1995) estimated errors in SSM/I-inferred rain rates (RRs) by matching SSM/I data obtained from F8 and F11 to radar estimates collected in the United States, United Kingdom, and Japan. Lin and Hou (2008) validated rain estimates from eight microwave sensors (TMI, F13, F14, F15, AMSR-E, NOAA-15, NOAA-16, and NOAA-17) by comparing near-coincident measurements from the TRMM precipitation radar (PR).

The Tropical Rainfall Measuring Mission satellite program (Simpson et al. 1988, 1996; Kummerow et al. 2000) has been operational since its launch in November 1997. As part of the TRMM program, a Ground Validation program was established to provide validation of the TRMM satellite estimates. The GV program established four principal sites that use both ground-based radars and networks of rain gauges and disdrometers to provide high-resolution rain-rate measurements, both temporally and spatially, for such efforts. Wolff et al. (2005) provide an overview of the TRMM GV program. Wolff and Fisher (2008) used TRMM GV data to show the relative accuracy of the three principal TRMM satellite retrievals derived from the PR and the TMI and combined (COM) algorithms. Significant effort has been made toward improving the quality of the TRMM GV rain-rate estimates. In particular, success in postcalibrating the radar data from Kwajalein, Republic of the Marshall Islands (KWAJ; Silberstein et al. 2009), has provided a high-quality, open ocean dataset for comparison with TRMM and other satellite precipitation retrievals (Marks et al. 2009).

In this study, we use 4 yr (2003–06) of GV data from Kwajalein and Melbourne, Florida (MELB), to assess the relative performance of satellite precipitation estimates from seven polar-orbiting satellites and the TRMM Microwave Imager, which flies in sun-asynchronous orbit. The comparisons are based on instantaneous estimates, all averaged to 0.25°. Using GV as a reference, we will show both the similarities of the various satellite estimates, as well as some fundamental differences, due to algorithmic and instrumental issues, as well as beam-filling issues, given the different footprint sizes of the various satellite platforms. Although considerable effort has been made to produce the highest-quality and most accurate rain-rate data from the GV observations, no claim as to the absolute accuracy of the GV precipitation estimates can be made, but rather GV data is offered as a reference, from which the relative performance of the satellite estimates can be made.

2. Satellite microwave radiometers: SSM/I, AMSU-B, AMSR-E, and TMI

a. Data sources: General

This study analyzes data collected from multichannel microwave rain sensors onboard eight spaceborne satellites. The rain sensors involved in this study include AMSU-B on N15, N16, and N17 satellites, SSM/I on F13, F14, and F15 satellites, AMSR-E on the Aqua satellite, and the TMI on the TRMM satellite. Each platform samples precipitating cloud structure across a broad range of discrete frequency bands in the microwave spectrum. Table 1 shows the frequency, polarization, and beamwidth of the various channels for each sensor class.

Table 1.

List of variables (var) of available frequencies (ν), polarizations (σ), and beamwidths (θ) of the various channels for the satellites assessed in this study, where V and H represent vertical and horizontal polarizations, respectively. Column 1 provides the name of the respective satellite platform.

List of variables (var) of available frequencies (ν), polarizations (σ), and beamwidths (θ) of the various channels for the satellites assessed in this study, where V and H represent vertical and horizontal polarizations, respectively. Column 1 provides the name of the respective satellite platform.
List of variables (var) of available frequencies (ν), polarizations (σ), and beamwidths (θ) of the various channels for the satellites assessed in this study, where V and H represent vertical and horizontal polarizations, respectively. Column 1 provides the name of the respective satellite platform.

Level-II AMSR-E data and AMSU-B data were acquired from the National Snow and Ice Data Center (NSIDC) in their native resolution. These data were then gridded to a resolution of 0.25° using software developed by E. Nelkin at the Goddard Space Flight Center. SSM/I data were obtained from the Goddard Earth Science Distributed Active Archive Center (DAAC), which was already gridded at 0.25°. The TMI data were gridded to 0.25° by the TRMM Science Data and Information System (TSDIS) for use in this study, at the special request of the authors (E. Stocker 2008, personal communication).

b. SSM/I, TMI, and AMSR-E

The SSM/I, TMI, and AMSR-E are conically scanning microwave radiometers with similar physical characteristics. The SSM/I was designed for the U.S. Air Force Defense Meteorological Satellite Program (DMSP), which began in June 1987 with the successful launch of F8. SSM/I data for this study were obtained from F13, F14, and F15, which were launched in 1995, 1997, and 1999, respectively. The DMSP satellites fly in a near-circular polar orbit at a nominal altitude of 865 km and an orbital period of about 102 min.

The SSM/I is equipped with seven linearly polarized channels, which sample microwave radiances at four different frequencies over a swath of about 1400 km. Dually polarized measurements—horizontal and vertical—are collected at 19.35, 37.0, and 85.5 GHz, while the channel centered on the water vapor line at 22.24 GHz measures only vertically polarized radiances (Prabhakara et al. 1992; Kummerow and Giglio 1994).

The TRMM satellite was launched in November 1997 with a payload that included both passive and active rain sensors: a microwave sensor (TMI) and the PR. The TMI was based on the original design of the SSM/I, with a few additions and modifications: the TMI added two dually polarized emission channels at 10.7 GHz, while the 21.3-GHz channel on the TMI was shifted slightly off center of the water vapor line to avoid saturation (Kummerow et al. 1998). Both the TMI and SSM/I only allocate one vertically polarized channel to the water vapor band at 22.3 GHz.

TRMM flies in a sun-asynchronous, low-earth orbit, with an inclination angle of 35° and a mean altitude of 402 km. Because TRMM flies in a lower earth orbit than the other satellites, the TMI covers a smaller swath of 878 km and has a smaller footprint at each frequency. The TRMM satellite, unlike the polar orbiters that collect two samples per day at about the same time, precesses through the entire diurnal cycle over a period of about 46 days.

AMSR-E is flown on board the Aqua satellite and is part of the multisatellite Earth Observing System. Aqua was launched in May 2002 and orbits at an altitude of 705 km with an orbital period of about 99 min. AMSR-E has 12 dually polarized channels that sample microwave radiances at six different frequencies ranging from 6.9 to 89 GHz across a swath of about 1445 km (see Tables 1 and 2). AMSR-E is similar in design to the TMI, but includes two additional channels at 6.9 GHz; however, poor spatial resolution limits the usefulness of these channels in rainfall estimation. AMSR-E also includes a horizontally polarized channel at 23.8 GHz, with the off-center shift from the water vapor line toward a slightly higher frequency.

Table 2.

Average altitudes and footprint sizes for various channels of the satellite platforms utilized in this study.

Average altitudes and footprint sizes for various channels of the satellite platforms utilized in this study.
Average altitudes and footprint sizes for various channels of the satellite platforms utilized in this study.

Rain-rate retrievals for the SSM/I, TMI, and AMSR-E were carried out using Version 6 of the Goddard Profiling (GPROF) algorithm described by Kummerow et al. (1996, 2001, 2006), Wilheit et al. (2003), and others, although it should be noted that minor modifications of the basic Version 6 algorithm are employed by the algorithm developers for each platform. However, in general, the GPROF algorithm retrieves both instantaneous rainfall and the vertical latent heating structure by constructing a vertical profile from the observed brightness temperatures in each channel. Rain rates are calculated using a Bayesian inversion procedure, which relates the retrieved profile, TB, to a simulated rain profile R:

 
formula

Given the observed profile TB, Eq. (1) relates the conditional probability of observing a particular profile R to the product of the two probabilities on the right-hand side. The first term is derived from cloud-resolving models and describes the probability that a certain rain-rate profile R will be observed. The second term is obtained by matching radiative transfer calculations (corresponding to the cloud-resolving model rainfall profiles) and describes the Bayesian probability of observing the brightness temperature TB given the rain-rate profile R (Kummerow et al. 2001; Wilheit et al. 2003).

The estimation of surface rain rates from spaceborne microwave sensors depends on the ability to separate emission and scattering radiances due to rain and clouds from the radiative upwelling emanating from the earth’s surface (Weinman and Guetter 1977). The task is complicated by the fact that radiative properties of the land and ocean surfaces differ significantly. Whereas the ocean surface is radiometrically cold and homogeneous, the land surface is radiometrically warm, and emissivities over land can be highly variable in time and space. Coastal regions, which include radiative contributions from both ocean and land, require even more complicated “physics” and assumptions, and yield the most uncertain results.

Consequently, GPROF handles the calculation of rain rates differently depending on the underlying terrain. First, each pixel sampled is classified as ocean, land, or coast using a fixed surface-terrain mask that was originally developed at 1/6° resolution for the TRMM TMI; the surface-terrain masks used in this study for KWAJ and MELB are also shown (Figs. 2a,b, respectively). The “ocean” algorithm derives a vertical profile of precipitation using all of the brightness temperature information collected from the available channels, and is thus more physically based because the low-frequency emission channels more directly probe the precipitable water at the cloud base. The land and coast algorithms, on the other hand, are empirically based because they rely on the high-frequency scattering channels (85.5 GHz for SSM/I and TMI and 89.0 GHz for AMSR-E), which are correlated with ice scattering processes above the freezing level in the cloud. Over land and coast the emission spectrum from the surface is too warm and highly variable to distinguish from the rain signal in the atmosphere.

Fig. 2.

(a) Terrain masks for the KWAJ GV site. Each 0.25° grid box is designated as ocean (3), land (0), coastal water (4), or coastal land (5). Both coastal land and coastal water are treated together as “coast” in this study. The respective GV radar is located at the center of each image and range rings at 50, 100, and 150 km are also shown. (b) As in (a), but for MELB.

Fig. 2.

(a) Terrain masks for the KWAJ GV site. Each 0.25° grid box is designated as ocean (3), land (0), coastal water (4), or coastal land (5). Both coastal land and coastal water are treated together as “coast” in this study. The respective GV radar is located at the center of each image and range rings at 50, 100, and 150 km are also shown. (b) As in (a), but for MELB.

The GPROF land algorithm is based on the National Environmental Satellite Data and Information Service (NESDIS) approach. A more complete description of the land algorithm can be found in Ferraro (1997) and McCollum and Ferraro (2003). The operational coast algorithm was originally developed from the Goddard Scattering Algorithm (GSCAT) and is described in McCollum and Ferraro (2005). Again, minor differences in the various GPROF algorithms employed by the various platforms are common.

c. AMSU

The Advanced Microwave Sounding Unit is a cross-track, line-scanning observational system consisting of two multichannel radiometers: AMSU-A and AMSU-B. The AMSU-B sensor package was designed for probing the temperature and moisture structure of the atmosphere. It is also being used in the estimation of surface rainfall. This sensor package is flown aboard the N15, N16, and N17 satellites at an altitude of 850 km (swath 1650 km).

AMSU-A has 15 channels that range from 23.8 to 89.0 GHz and AMSU-B has 5 channels ranging from 89.0 to 183.31 GHz. The AMSU-B rain-rate algorithm is fundamentally based on the indirect correlation between surface rain rate and ice scattering aloft as observed by the high-frequency channels (Spencer et al. 1989; Weng et al. 2003; Qiu et al. 2005). The algorithm simultaneously derives the ice water path (IWP) and the effective particle diameter De. Like GPROF, the AMSU algorithm discriminates between ocean, land, and coastal terrains. In each case, the IWP is obtained using a two-stream approximation to the radiative transfer equation that uses the 89 and 150 channels (Kongoli et al. 2007; Vila et al. 2007).

Recent algorithmic improvements use information in some of the lower-frequency channels, namely, 23.8, 31.4, 53.8, and 183 GHz, to refine the rain estimate for a particular case (ocean, land, or coast). The 23.8- and 31.4-GHz channels are used to increase the areal coverage over the oceans, while the 53.8-GHz channel is used to identify falling snow. The three channels at 183 GHz, centered on the strongly absorbing water vapor band, are used to compute a convective index to discriminate between convective and nonconvective rain systems (Kongoli et al. 2007; Vila et al. 2007).

Rain rates (RR) are determined based on a nonlinear relationship between RR and IWP computed using the fifth-generation Pennsylvania State University–National Center for Atmospheric Research Mesoscale Model (MM5). The mean IWP–RR relation can be statistically determined and assumes the form

 
formula

where a0, a1, and a2 are estimated constants.

Multiple scattering channels are also needed to simultaneously determine both IWP and De without ambiguity (Weng et al. 2003). Although AMSU has some distinct weaknesses due to its underutilization of the lower-frequency channels, its use of the high-frequency channels offers a smaller field of view for resolving, while the additional 150-GHz channel allows for the detection of lighter rain rates.

d. Intersensor sources of uncertainty

Several different sources of uncertainty need to be accounted for in explaining the intersensor differences in the passive microwave rain retrievals examined in this study. Variations in the rain retrievals can be attributed to differences in 1) precipitation algorithms, 2) field-of-view (FOV), 3) instrument characteristics, 4) sampling, and 5) calibration. These sources of uncertainty not only explain some of the observed differences between the rain sensors but also address the absolute uncertainties as well. Here we are more concerned with relative intersensor differences, both with respect to the spaceborne sensors and GV. Moreover, the relative contribution of these uncertainties in explaining the observed intersensor differences depends on rudimentary factors such as the underlying surface (ocean, land, or coast) and the type of rain system.

For example, over the oceans the large FOV in the low-frequency emission channels leads to a so-called beam-filling problem, resulting from an inhomogeneous distribution of rain information across the FOV in association with the existence of strong rainfall gradients, and/or regions of rain and no rain within a single footprint (Joyce et al. 2004). If not corrected for, smearing of the signal across the FOV simply produces a homogenous FOV, leading to an underestimate of the rainfall due to the nonlinear response of brightness temperature to rain rate (Kummerow et al. 1998). A larger FOV both exacerbates the beam-filling problem and also reduces the number of FOVs (i.e., samples) in a given 0.25° grid box. The beam-filling problem tends to represent the most significant source of error for the TMI, AMSR-E, and SSM/I over the oceans. Of course, AMSU faces a different problem over the oceans, where rain rate is inferred entirely from an ice scattering algorithm that relies solely on the precipitation signal in the high-frequency channels. The AMSU algorithm thus has difficulty observing low rain rates (<1 mm h−1) and warm rain processes, where no ice scattering is present.

The FOV depends on the altitude of satellite, the observing frequency, and the beamwidth of the antenna. Table 1 provides the beamwidth of the various channels for the different satellite platforms used in this study. Table 2 shows the altitude and FOV area sizes for three different frequencies for each of the satellite platforms. Although the TMI and AMSR-E fly at different altitudes, they have about the same FOV because the antenna beamwidth on AMSR-E is significantly narrower than that of TMI. For example, the TMI beamwidth is 0.42 and 0.43 for the vertical and horizontal 85.5-GHz channels, respectively, while the AMSR-E beamwidth is 0.18 for both the vertical and horizontal 89.0-GHz channels.

Over land and coast, GPROF is limited to the high-frequency channels because of ground contamination in the low-frequency emission channels. The low-frequency emission channels are subsequently not utilized. As shown in Table 2, the high-frequency channels have a significantly smaller FOV; however, the ice scattering processes responsible for producing a rain signature occur at higher elevations and therefore are less correlated with surface rainfall. It is also important to note that even though the GPROF rain algorithm generates rain rates for the SSM/I, TMI, and AMSR-E retrievals, as was already shown, each of these sensors has different instrument characteristics related to the number, frequency, and polarization of the different channels. Consequently, though all three sensors are similar in design, they are not entirely equivalent regarding their respective measurements.

3. GV data

The GV data for this study used the official TRMM 2A53 rain products for KWAJ and MELB produced by the TRMM Satellite Validation Office and available from the Goddard Earth Sciences (GES) Data and Information Services Center (DISC). The 2A53 product provides instantaneous rain rates at 2 km × 2 km horizontal resolution, extending 150 km from the respective GV radar. The GV radar data were processed using the official Version 5 (MELB) and Version 7 (KWAJ). The algorithm applies the Window Probability Matching Method (WPMM; Rosenfeld and Wolff 1995) to the statistical determination of rain rates from radar reflectivities. The GV processing system and descriptions of the GV algorithms are detailed in Wolff et al. (2005). We note that the KWAJ data, now Version 7, have undergone significant improvement from previous versions, after the application of the Relative Calibration Adjustment (RCA) technique to correct calibration and pointing angle errors in the raw reflectivity data (Silberstein et al. 2009). The RCA method and its application to the KWAJ reflectivity data are described in Marks et al. (2009). Marks et al. (2009) convincingly demonstrate that the year-to-year radar–rainfall relationships converge, as was to be expected from long-term analysis of drop size distributions. Further, Marks et al. (2009) show the mean difference between the KWAJ reflectivity estimates to be within ±1 dB of the well-calibrated and stable TRMM Precipitation Radar (Takahashi et al. 2003).

4. Data analysis: Description

This study assesses the relative performance of the eight microwave sensors for the GV sites at KWAJ and MELB over a 4-yr period from 2003 to 2006. The validation strategy was to intercompare ground- and space-based measurements by matching the retrieved instantaneous rain rates from each sensor to the GV rain rates during satellite overpasses of the GV site. The matched rain rates were averaged at a grid resolution of 0.25° × 0.25° (latitude × longitude). The key advantage of matching the satellite and GV datasets at instantaneous scales is that the temporal sampling error associated with noncontinuous regional sampling by the orbiting satellites can be eliminated as a major source of uncertainty. Although there are myriad problems with estimating precipitation with radar data (such as anomalous propagation, ground clutter, beam filling, range effects, improper rain–reflectivity relationships, etc.), the authors of this study suggest not relying too heavily on simple statistics such as biases, which might occur from over- or underestimation by the radar, the satellite, or both. Rather, one should make these comparisons using a variety of statistical tests to determine how the relative performances of the various satellite estimates vary. This validation strategy uses the ground data as an empirical reference for assessing the relative performance of passive microwave-based rain retrievals at instantaneous scales.

The WPMM used to generate radar-derived rain rates from measured reflectivity depends on the instantaneous rain-rate information from an extended network of gauges as a function of range. Figures 1a and 1b show the radar and gauge networks for KWAJ and MELB, respectively. The gauge network at KWAJ only extends out to about the 100-km range ring shown in Fig. 1a, whereas in Fig. 1b, it can be seen that the gauge network at MELB extends well beyond the outer 150-km range ring. In the case of KWAJ, satellite rain rates were only matched out to 100 km but were extended out to 150 km in the case of MELB.

Fig. 1.

(a) Illustration of the gauge and radar networks for the GV network at KWAJ. Range rings at 50, 100, 150, and 200 km are also shown. Note that for this study all usable GV data were restricted to the first 100 km from each respective radar site. KWAJ has only seven gauge sites; however, each site contains two or more gauges to improve reliability and uncertainty of the measured rain rates. (b) As in (a), but for MELB. There are three gauge networks in FL: St. John’s River Water Management District (STJ) and the South Florida Water Management District (SFL) are operated by the state of FL, and a NASA-owned network is located on Cape Canaveral at NASA Kennedy Space Center.

Fig. 1.

(a) Illustration of the gauge and radar networks for the GV network at KWAJ. Range rings at 50, 100, 150, and 200 km are also shown. Note that for this study all usable GV data were restricted to the first 100 km from each respective radar site. KWAJ has only seven gauge sites; however, each site contains two or more gauges to improve reliability and uncertainty of the measured rain rates. (b) As in (a), but for MELB. There are three gauge networks in FL: St. John’s River Water Management District (STJ) and the South Florida Water Management District (SFL) are operated by the state of FL, and a NASA-owned network is located on Cape Canaveral at NASA Kennedy Space Center.

Each 0.25° grid box, as mentioned in the previous section, was classified as land, coast, or ocean according to a surface terrain mask. The original TRMM terrain map was produced at a scale of 1/6°, whereas the matching in the study was at 0.25°. For classification purposes, the mask was interpolated to 0.25°. For cases where more than one terrain type existed, the classification was determined based on a “majority rules” criterion. The 0.25° classifications for KWAJ and MELB are shown in Figs. 2a and 2b, respectively. KWAJ is a pure, open ocean site and as seen in Fig. 2a, every 0.25° grid box is classified as ocean. MELB, on the other hand, contains all three classifications, with about 1/3 of the boxes classified as coast.

The seven polar-orbiting satellites and TRMM differ fundamentally in their temporal sampling of the atmosphere. For a given grid box, each of the polar orbiters collect two samples per day at about the same nominal times, and thus mean rain rate inferred from these observations incurs an intrinsic diurnal bias. TRMM, on the other hand, reduces the diurnal bias by precessing through the diurnal cycle in a period of about 46 days. Thus, over the 4-yr period employed in this study, TRMM is able to capture the diurnal cycle well at both KWAJ and MELB. To illustrate the potential impact of the diurnal cycle on longer-term rainfall estimation, estimated diurnal cycles were generated for KWAJ and MELB based on mean hourly conditional mean (R|R > 0) rain rate computed from 5 yr (2000–04) of radar data. These results are shown in Figs. 3a,b. The filled circles displayed in each profile designate the nominal times when each satellite flies over the site. Figure 3b for MELB reveals a strong diurnal cycle, characterized by its high-amplitude maximum in the late afternoon (1500 LST), and a relative minimum occurring between 2300 and 0600. KWAJ, on the other hand, exhibits a much weaker diurnal cycle, with a nocturnal maximum occurring at around 0100–0200 LST and a minimum during the daytime hours (Wolff et al. 1995), typical of a tropical oceanic environment.

Fig. 3.

(a) Diurnal cycle of hourly conditional mean rain rate, derived from 7 yr (2000–06) of GV data at MELB. Superimposed are colored symbols showing the local overpass times of each of the polar-orbiting satellites assessed in this study. (b) As in (a), but for MELB.

Fig. 3.

(a) Diurnal cycle of hourly conditional mean rain rate, derived from 7 yr (2000–06) of GV data at MELB. Superimposed are colored symbols showing the local overpass times of each of the polar-orbiting satellites assessed in this study. (b) As in (a), but for MELB.

a. Probability distributions by occurrence and rain volume

This section analyzes the probability distribution functions (PDFs) of rain rates for the satellite and coincident GV data during the study period. Two PDF were computed for each satellite: 1) PDF by occurrence (PDFc) and 2) PDF by rain volume (PDFυ). The PDFc provides statistical information on the breadth and shape of the rain-rate distribution and highlights differences in the respective estimate’s sensitivity as a function of rain rate. The PDFυ is a normalized distribution obtained by multiplying the PDFc (bin) by the rain rate assigned to each bin and represents the percentage of rain accumulation that each rain-rate interval contributes to the total rain volume. We note that these distributions are computed using collocated data in both space and time. In other words, the GV estimates that go into the distributions are only those that are sampled during a given satellite overpass.

PDFc and PDFυ for KWAJ are shown in Fig. 4. All of the satellite rain rates determined from the GPROF algorithm (AMSR-E, F13, F14, F15, and TMI) exhibit similar PDFυ. AMSR-E tends to slightly underestimate heavier rain rates, relative to GV. Both TMI and AMSR-E distributions agree quite well, especially in their ability to detect lighter rain rates, which all of the SSM/I and AMSU-B estimates fail to detect. Furthermore, the PDFυ for SSM/I also shows decreased performance in detecting moderate rain rates (above about 0.5 mm h−1) and a tendency to overestimate the highest rain rates. The AMSU-B rain distributions are highly peaked, around 1 mm h−1, which is due to ad hoc adjustments to the product algorithm by its developers in an effort to increase the sensitivity of the AMSU-B estimates in light rain, which is significantly diminished by the lack of emission channels available on the platforms (R. Ferraro 2008, personal communication). The result shows a large peak in rates at about 1 mm h−1, which we suggest is not physically plausible and should be considered when using AMSU-B data over oceanic areas.

Fig. 4.

PDF of rain rates for TRMM GV and the various satellite retrievals at KWAJ. The dashed and dotted lines represent the PDF by occurrence: PDFc = PDF(R) for GV and satellite, respectively, while the solid and dashed–dotted lines represent the PDF by volume: PDFυ = [R × PDF(R)] for GV and satellite, respectively.

Fig. 4.

PDF of rain rates for TRMM GV and the various satellite retrievals at KWAJ. The dashed and dotted lines represent the PDF by occurrence: PDFc = PDF(R) for GV and satellite, respectively, while the solid and dashed–dotted lines represent the PDF by volume: PDFυ = [R × PDF(R)] for GV and satellite, respectively.

Figures 5a–d show the PDFc and PDFv for GV and satellite estimates at MELB over ocean, land, coast, and all terrain types (see Fig. 2b for reference), respectively. Figure 5a shows the respective PDFs over ocean at MELB. It is obvious that these PDFs are quite similar to the KWAJ PDFs, indicating consistency in our results. Again the AMSR-E distributions are quite similar to the GV estimates, while both SSM/I and AMSU-B estimates tend to undersample the lightest rain rates. The peaked AMSU-B distributions are again evident, at about 1 mm h−1, again due to the lack of emission channels for the platform.

Fig. 5.

As in Fig. 4, but for MELB over (a) ocean areas only, (b) land areas only, (c) coastal areas only, and (d) the entire GV domain.

Fig. 5.

As in Fig. 4, but for MELB over (a) ocean areas only, (b) land areas only, (c) coastal areas only, and (d) the entire GV domain.

Figure 5b shows the respective PDFs over land areas at MELB. Both TMI and AMSR-E show some diminished performance in detecting the lightest rain rates, relative to GV. At the highest rain rates, AMSR-E and SSM/I (except F15) all agree fairly well with GV estimates, while TMI slightly overestimates the heaviest rain rates. Not surprisingly, the AMSU-B estimates over land are significantly better than over ocean. Indeed, the AMSU-B estimates are in line with the TMI and AMSR estimates, and somewhat better than the SSM/I estimates, which is likely due to the additional scattering channels available for the AMSU-B platform. Figure 5c shows the respective PDFs over coastal areas at MELB. Both TMI and AMSR-E show similar PDFs relative to GV, which is somewhat surprising in that Wolff and Fisher (2008) demonstrated significant differences between the GV and TMI distributions, at the scale of the TMI footprint (approximately 154 km2). The AMSU-B estimates also perform quite well relative to GV over coastal areas. Figure 5d shows the respective PDFs over the entire GV domain at MELB. Once again, the AMSU-B distributions most closely match the GV distributions and show increased skill in detecting the lighter rain rates. The SSM/I distributions show a large peak at the more moderate rain rates, not seen in GV but consistent with coastal influences shown previously by Wolff and Fisher (2008). The AMSU-B estimates over the entire domain perform relatively well, with a strong peak in the distributions introduced by the erroneous ocean-area estimates.

b. Monthly mean rain rates

In this section, instantaneous satellite rain rates were validated on monthly scales by computing monthly rainfall estimates using the matched datasets from KWAJ and MELB at 0.25°. The results for KWAJ and MELB are shown in Figs. 6 and 7, respectively. Also, Table 3 and 4 provide summary statistics of our comparisons for KWAJ and MELB, respectively. The mean monthly rain estimates for KWAJ and MELB show good agreement with measurements made from earth-based and space-based microwave sensors, but there are also some important differences.

Fig. 6.

Monthly rain computed estimates for KWAJ for each of the eight spaceborne microwave estimates. Rain estimates only integrated matched GV–satellite rain rates during satellite overpasses.

Fig. 6.

Monthly rain computed estimates for KWAJ for each of the eight spaceborne microwave estimates. Rain estimates only integrated matched GV–satellite rain rates during satellite overpasses.

Fig. 7.

As in Fig. 6, but for MELB over (a) ocean areas only, (b) land areas only, (c) coastal areas only, and (d) the entire GV domain.

Fig. 7.

As in Fig. 6, but for MELB over (a) ocean areas only, (b) land areas only, (c) coastal areas only, and (d) the entire GV domain.

Table 3.

General statistics derived from the period 2003–06, showing comparisons between TRMM GV estimates at KWAJ vs the various satellites estimates assessed in this study. The columns, from left to right, correspond to site, satellite, GV mean rain rate (mm h−1), satellite mean rain rate (mm h−1), regression equation intercept, and slope, correlation, and bias. The bias, expressed in percentage, is defined as bias = 100% × (Sat_mean − GV_mean)/GV_mean, so that a negative bias indicates a satellite underestimate relative to GV.

General statistics derived from the period 2003–06, showing comparisons between TRMM GV estimates at KWAJ vs the various satellites estimates assessed in this study. The columns, from left to right, correspond to site, satellite, GV mean rain rate (mm h−1), satellite mean rain rate (mm h−1), regression equation intercept, and slope, correlation, and bias. The bias, expressed in percentage, is defined as bias = 100% × (Sat_mean − GV_mean)/GV_mean, so that a negative bias indicates a satellite underestimate relative to GV.
General statistics derived from the period 2003–06, showing comparisons between TRMM GV estimates at KWAJ vs the various satellites estimates assessed in this study. The columns, from left to right, correspond to site, satellite, GV mean rain rate (mm h−1), satellite mean rain rate (mm h−1), regression equation intercept, and slope, correlation, and bias. The bias, expressed in percentage, is defined as bias = 100% × (Sat_mean − GV_mean)/GV_mean, so that a negative bias indicates a satellite underestimate relative to GV.
Table 4.

As in Table 3, but for MELB. Also, the statistics are subclassified by terrain type (ocean, land, coast, and all).

As in Table 3, but for MELB. Also, the statistics are subclassified by terrain type (ocean, land, coast, and all).
As in Table 3, but for MELB. Also, the statistics are subclassified by terrain type (ocean, land, coast, and all).

In Fig. 6 for KWAJ, all eight satellites are highly correlated with GV month to month and demonstrate good skill at estimating surface rainfall on monthly time scales. However, as displayed in Table 3, all eight satellites show an overall negative bias relative to GV. Similar results are observed in the MELB ocean case shown in Fig. 7a. With the exception of F15 (+0.5%), all of the other sensors exhibited a negative bias. The GV–satellite comparisons over MELB land exhibit a greater tendency for the satellite sensor to overestimate the rainfall during the summer months. This tendency is especially prevalent in the SSM/I group, which resulted in overall biases ranging between 13.1% and 25.5%. Furthermore, the bias exhibits oscillatory behavior, overestimating convective summertime rainfall while underestimating wintertime rainfall. The MELB coast case displays similar features as observed in the land case, with SSM/I exhibiting the highest summertime biases. For MELB coast, the SSM/I biases ranged from 44.4% to 71.1%.

The AMSU-B sensors showed less consistency as a group. For MELB coast, N15 tends to overestimate summertime rainfall over land and coast, resulting in an overall positive bias in both cases. The N15 bias in the coast case is only 0.5% because of the tendency for N15 to underestimate surface rainfall during the transition months associated with spring and fall. N17, on the other hand, was negatively biased for both land (−18.3%) and coast (−42.9%). Note that in Fig. 3b, N17 straddles the beginning and end of the convective period, characterized by the large amplitude that peaks in the midafternoon.

The results for MELB considered alongside the results for KWAJ illustrate some of the issues over land and coast, and the importance of the lower-frequency channels in accurately determining instantaneous surface precipitation from convective systems. During the summer months, central Florida is dominated by smaller-scale sea-breeze convection, which can produce large anvils. The GPROF land and coast algorithm must first screen the FOV to determine whether or not rain exists before attempting to quantify the rainfall based on the 85.5/89.0-GHz rain signature. Nonprecipitating anvils can easily be interpreted as rain because of their ice content. Moreover, in the land case, the rain algorithm has been “engineered” to match cloud profiles to brightness temperature based only on the high-frequency channel (Wang and Wolff 2009). Consequently, the algorithm does not yet address the actual microphysics, whereas in the ocean case, the complex microphysics is treated more realistically by constructing a TB vector from all of the different emission and scattering channels. The cloud profiles in the ocean case can therefore more adequately represent the actual microphysics based on a radiative sounding through the column. Since biases over land and coast can depend on regional factors, more studies are needed to better probe and understand this physical nature of the bias.

c. Correlations and regressions

Scatterplots of the various satellite estimates versus GV estimates are shown in Figs. 8 and 9 for KWAJ and MELB, respectively. These plots provide the regression equations and correlations. The slope of each regression equation provides an indication of the relative biases between the various estimates. Figure 8 shows the scatterplots for KWAJ. The AMSR-E estimates exhibit the highest correlation of 0.89, slightly greater than the TMI, which shows a correlation of 0.86. However, with a slope of 0.79, the TMI exhibits a slightly lower bias than AMSR-E, which had a slope of 0.75. The SSM/I and AMSU-B correlations are significantly lower, ranging from 0.62 to 0.67. The slopes of the SSM/I range from 0.68 to 0.78, while the AMSU-B slopes range from 0.49 to 0.65.

Fig. 8.

Scatterplots of instantaneous satellite and GV rain rates for KWAJ ocean for the period 2003–06.

Fig. 8.

Scatterplots of instantaneous satellite and GV rain rates for KWAJ ocean for the period 2003–06.

Fig. 9.

Scatterplots of instantaneous satellite and GV rain rates for MELB over (a) ocean areas only, (b) land areas only, (c) coastal areas only, and (d) the entire GV domain for the period 2003–06.

Fig. 9.

Scatterplots of instantaneous satellite and GV rain rates for MELB over (a) ocean areas only, (b) land areas only, (c) coastal areas only, and (d) the entire GV domain for the period 2003–06.

Figure 9a shows the scatterplots for MELB over ocean areas only. AMSR-E estimates are better matched with GV estimates with a correlation of 0.86 and a slope of 0.74, consistent with our KWAJ results. TMI correlations are 0.84 with a slope of 0.64. The correlations for the SSM/I rain rates fell within a tight range between 0.66 and 0.72, and though lower than TMI and AMSR-E, were significantly greater than the AMSU-B, whose correlations fell within a range between 0.52 and 0.54. These results for MELB ocean are again consistent with the results for KWAJ.

Figure 9b shows the scatterplots for MELB over land areas only. Here, TMI shows the highest correlation (0.74) with a slope of 0.86. AMSR-E estimates show a correlation of 0.72, with a decreased slope of 0.62, indicating an increased bias (AMSR-E lower than GV) over land areas. However, it is important to note that neither SSM/I nor AMSU-B estimates provided significantly improved skill over land areas.

Figure 9c shows the scatterplots for MELB over coastal areas only. Although none of the estimates provide correlations greater than 0.74 (TMI), AMSR-E and SSM/I/F15 both have correlations of 0.69. Somewhat surprisingly, the slopes of the SSM/I regression lines are all near unity (0.91–1.05). AMSU-B fares worse than either AMSR-E or SSM/I, with slopes ranging from 0.34 to 0.48 and correlations on the order of 0.5.

Figure 9d shows the scatterplots over the entire GV domain at MELB. TMI shows the highest correlation of 0.74, with a slope of 0.71, while AMSR-E shows a correlation of 0.72 and a slope of 0.67. SSM/I estimates show correlations ranging from 0.64 to 0.68, with slopes ranging from 0.81 to 0.93. AMSU-B estimates show the lowest correlations, ranging from 0.50 to 0.55, with slopes ranging from 0.37 to 0.50.

d. Rain-rate profiles

Mean instantaneous satellite rain-rate profiles were constructed for KWAJ and MELB by sorting the satellite rain rates along the GV rain-rate continuum. For this analysis, the matched GV–satellite rain rates were averaged at logarithmic binning intervals between 0 and 12 mm h−1 according to the value of the GV rain rate. Wolff and Fisher (2008) previously applied this analysis scheme to the validation of TRMM level-II rain rates. In that study, GV rain rates were matched to rain rates obtained from the TMI, PR, and combined products at the scale of the TMI footprint. The GV–satellite data were then binned and averaged on a linear scale at 1 mm h−1 intervals.

When analyzed on a linear scale the highest rain rates are undersampled, while the lowest rain rates are oversampled. The logarithmic scale, applied to this case, has the advantage of naturally partitioning the data into sampling bins of near equal size. The logarithmic scaling provides considerably more resolution at the low end of the spectrum where the sampling was best. How each sensor performs at the low and high ends of the rain-rate spectrum is important criteria in evaluating the relative performance and uncertainty of each sensor class. The range of each profile was constrained at 12 mm h−1, which represents about 99% of the total rain volume for each sensor.

The rain-rate profiles for all eight satellites are shown in Figs. 10 and 11. For the case of MELB, the matched data were further stratified into ocean, land, and coast regimes. These are displayed in the four panels shown in Fig. 11. In this evaluation, we examine the members of each sensor class as a group (AMSU-B and SSM/I), since each class tends to exhibit structural features characteristic of the instrument and its physical properties.

Fig. 10.

Rain-rate profiles for AMSR, F13, F14, F15, N15, N16, and N17 generated for KWAJ using GV as an empirical reference. Satellite rain rates were binned and sorted along the GV rain-rate continuum.

Fig. 10.

Rain-rate profiles for AMSR, F13, F14, F15, N15, N16, and N17 generated for KWAJ using GV as an empirical reference. Satellite rain rates were binned and sorted along the GV rain-rate continuum.

Fig. 11.

Rain-rate profiles for AMSR, F13, F14, F15, N15, N16, and N17 generated from matched GV–satellite dataset for MELB using GV as an empirical reference. Satellite rain rates were binned and sorted along the GV rain-rate continuum. The matched data were further stratified according to (top left) ocean, (top right) land, (bottom left) coast, and (bottom right) all.

Fig. 11.

Rain-rate profiles for AMSR, F13, F14, F15, N15, N16, and N17 generated from matched GV–satellite dataset for MELB using GV as an empirical reference. Satellite rain rates were binned and sorted along the GV rain-rate continuum. The matched data were further stratified according to (top left) ocean, (top right) land, (bottom left) coast, and (bottom right) all.

Looking first at the general characteristics of the different microwave profiles, we observe that all of the rain-rate profiles for KWAJ and MELB are well correlated and exhibit good agreement with GV across the nominal midrange of each plot (i.e., 1 and 10 mm h−1). The most significant deviations relative to GV occur at the lowest and highest rain rates. The microwave profiles, when considered as a single group, show a distinct tendency to overestimate the low rain rates less than 1.0 mm h−1 and underestimate the high rain rates greater than 10 mm h−1. This pattern results in a crossing of the one-to-one line shown in Figs. 10 and 11. In evaluating this trend, it is important to keep in mind that the seesaw relationship between the lower and upper rain-rate regimes is fundamentally asymmetric in that the total number of samples at the low and high end differ by about three orders of magnitude. There is subsequently considerably more sampling of the lower rain rates, while the higher rain rates above 10 mm h−1 are rarer at the 0.25° scale.

The intersensor dispersion is largest for observed rain rates below 1.0 mm h−1. This region of the profile approaches the sensitivity threshold of each sensor and its capacity to detect and quantitatively measure light rain rates. At the high end of each profile, we attribute the crossing pattern in each profile (relative to the GV rain-rate continuum) to the well-known saturation of the brightness temperatures at high rain rates and beam filling in the low-frequency channels over the oceans.

We expect beam filling to be more prominent at the higher rain rates in association with smaller-scale convection where strong rainfall gradients are more likely to occur. Looking at Table 2, it is seen that TMI and AMSR-E have about the same FOV compared to the SSM/I, which is considerably larger, but that the mean rain rates are commensurate. We consequently do not observe significant intersensor differences that can be attributed to differences in the FOV. However, examining the scatter diagrams in Figs. 8 and 9 for the KWAJ and MELB ocean cases, we clearly observe more scatter at the higher rain rates in the SSM/I case. The TMI and AMSR-E, on the other hand, exhibit a much tighter configuration of points around the one-to-one line. Given the similarities in the instrument characteristics, these differences observed in the instantaneous matched data can be attributed to differences in the FOV between the sensors. Intersensor differences in the FOV subsequently appear to be affecting the variance in the instantaneous estimates, but not the statistical mean, which in this case represents 4 yr of matched data (see Figs. 8 and 9 for sample sizes).

Kummerow et al. (2001) notes that one of the significant improvements to the GPROF rain algorithm in version 6 is that it accounts for the beam filling in the lower-frequency channels by statistically estimating the fraction of convection in the footprint as determined from the high-frequency channels (85.5 for TMI and 89 for AMSR-E). It is expected that this correction should improve the statistical estimate of the mean at the higher rain rates but would not necessarily lesson the random errors (variance) at instantaneous scales, which is what we observe when comparing Figs. 11 and 12 with Figs. 8 and 9.

Fig. 12.

HSS2D plots for the various satellite and GV estimates. The line through the contours represents the maximum HSS for a given GV rain rate.

Fig. 12.

HSS2D plots for the various satellite and GV estimates. The line through the contours represents the maximum HSS for a given GV rain rate.

In the first case of KWAJ shown in Fig. 10, AMSR-E and the TMI show the best agreement with GV at the lowest rates. Each sensor appears highly correlated with GV, even at the lowest rain rates. This observation is further supported by the PDF for TMI and AMSR shown in Fig. 4. Both sensors exhibit impressive statistical agreement with GV at the low rain rates. It should be emphasized, however, that the rain-rate profiles are constructed from one-to-one matching of the satellite with GV in time and space; the PDF, on the other hand, are computed independently for the satellite and GV from the matched data.

TMI and AMSR were the only microwave sensors that exhibited a negative total rainfall difference over the 4-yr period for rain rates less than 1.0 mm h−1. From Table 2 it is seen that at KWAJ, TMI, and AMSR had −14.0% and −16.5% biases relative to GV, respectively, while in the case of MELB (ocean), the biases were only −5.3% for AMSR-E and −11.3% for the TMI. Some of these differences between KWAJ and MELB may be attributable to differences in the diurnal cycles of the two regions. MELB exhibits a modified oceanic diurnal cycle, with small maximum amplitude in the late afternoon. TRMM, as noted earlier, temporally samples the entire diurnal cycle, while AMSR-E flies over each site twice per day at about the same times as shown in Figs. 3a and 3b.

SSM/I sensors F13, F14, and F15 also show good agreement with GV for KWAJ. Overall biases for each of the three SSM/I sensors fall within a tight range and exhibit similar features at the low and high end. SSM/I sensors performed even better for the MELB ocean case with biases ranging between 0.5% and −4.0%. The AMSU-B sensors N15, N16, and N17, in turn, display properties common to the AMSU-B group, overestimating the rainfall at rain rates less than 1 mm h−1 more than the other sensors. This result was expected because the AMSU-B rain rates are inferred from an ice scattering algorithm. Consequently, the observed rain signature is less correlated with surface rainfall (Spencer et al. 1989). At rain rates greater than 10 mm h−1, the N16 and N17 profiles clearly underestimated the rain rates more than the other sensors, resulting in negative biases of −24.5% and −18.4%, respectively. The AMSU-B sensors for MELB ocean show more intragroup variance, with biases ranging between −28.9% and −13.9%.

For MELB, we observe more intersensor dispersion in the land and coast cases. Some of this additional variance is attributable to the smaller sample sizes available for each terrain category, but algorithmic uncertainties are also greater due to the dependence of inferred rain rates on passive scattering signatures. For MELB land, intersensor biases range from −18.3% for N17 to 25.5% for F13, and for MELB coast, the satellite biases range from −42.9% to 48.4%. Intersensor dispersion is greatest at the low end of the rain-rate profile. The land and coast profiles for TMI and AMSR show the best agreement and highest overall correlation with GV. The three AMSU-B sensors reveal the greatest amount of dispersion within the group. In the case of AMSU-B, some of this variability may be partly ascribed to differences in the diurnal sampling. SSM/I sensors are fairly well correlated with GV over land and coast but tend to exceed GV across most of the dynamic range, resulting in land biases of 25.5%, 16.0%, and 13.1% and coast biases of 48.4%, 71.1%, and 44.4% for F13, F14, and F15, respectively.

e. Two-dimensional HSS

A common metric used to assess the skill of a given estimate (observation) to measure a predicted rain rate is the Heidke skill score (HSS). However, as noted by Conner and Petty (1998), “in this context, HSS only yields information concerning the algorithm’s ability to mimic the radar’s delineation of the lightest possible surface precipitation—something that radars themselves are not particularly adept at, especially at greater distances … Furthermore, since area coverage by light precipitation is a strong function of threshold rain rate, minor differences in the minimum rain rate detectable by each data source can severely degrade the apparent skill of an otherwise robust algorithm. We therefore view HSS as a nearly meaningless measure of algorithm performance when used in this way.”

Table 5 shows a generalized 2 × 2 contingency table, where in this study, the satellite represents the observed values while GV estimates serve as the predicted values. The four results illustrate the observed values: hits, misses, false alarms, and correct rejections. These values can then be used to compute a number of useful metrics, including bias, proportion correct, probability of detection, false alarm rates, probability of false detection, and others. We use the same definition for HSS as Conner and Petty (1998), and follow their lead by constructing multiple contingency tables, using separate rain-rate thresholds for both the validation and the satellite estimates. The resultant array is then used to generate the plots shown in Figs. 12 –15. For this study, we chose rain rates for both estimates ranging from 0.1 to 20 mm h−1, in increments of 0.1 mm h−1. Equation (3) provides a definition of HSS:

 
formula

where H (hits), M (misses), F (false alarms), and C (correct rejections) are defined in Table 5.

Table 5.

Contingency table construction used for calculating the HSS and other metrics. In our study, satellite estimates are “observed,” while validation estimates are “predicted.”

Contingency table construction used for calculating the HSS and other metrics. In our study, satellite estimates are “observed,” while validation estimates are “predicted.”
Contingency table construction used for calculating the HSS and other metrics. In our study, satellite estimates are “observed,” while validation estimates are “predicted.”
Fig. 15.

Maximum HSS for a given GV rain rate at MELB for the various satellites over (top left) ocean, (top right) land, (bottom left) coast, and (bottom right) all.

Fig. 15.

Maximum HSS for a given GV rain rate at MELB for the various satellites over (top left) ocean, (top right) land, (bottom left) coast, and (bottom right) all.

These plots provide significant information regarding the bias and correlation of the two estimates, all as a function of rain rate. If the line of maximum HSS is above/along/below the 1:1 line, then the estimate is high/non-/low biased, respectively. Also, the larger the gradient of the HSS values around the line of maximum HSS is, the higher the correlations are.

Figure 12 shows the two-dimensional HSS (HSS2D) for KWAJ for all of the satellite estimates evaluated in this study. TMI and AMSR clearly show the highest skill at all rain rates, with maximum HSS of greater than 0.8 for rain rates up to about 2 mm h−1, and are generally better at all estimates within 0–15 mm h−1, although there is a negative bias that is exacerbated at the higher rain rates and is also shown in Table 3. TMI shows similar skill as AMSR-E at rain rates below about 5 mm h−1. The various SSM/I estimates are similar, but clearly show less skill than the AMSR-E and TRMM estimates. There is a secondary maximum of skill for the SSM/I estimates for rain rates between about 12–20 mm h−1. Of the three AMSU-B estimates, N15 and N17 show the greatest skill at KWAJ, while N16 shows the least skill and strong bias above about 4 mm h−1, illustrated by the axis of maximum HSS below the 1:1 line.

Figure 13 shows, for each validation rain rate, the maximum HSS at KWAJ, again illustrating that AMSR-E shows the best skill at all rain rates, followed by TMI, SSM/I, and AMSU. The maximum HSS values TMI and AMSR are about 0.2 higher than all other estimates, for all rain rates up to about 15 mm h−1. The SSM/I skills are in general higher than the AMSU-B skills by about 0.1, with some increased skill of about +0.2 (relative to AMSU) for rain rates greater than 10 mm h−1. Conner and Petty (1998) noted that these types of plots make it possible to evaluate and objectively intercompare the maximum potential skill of competing algorithms, irrespective of algorithm calibration differences. Further, they correctly state that there is no built-in assumption of linearity in the relationship between the satellite estimate and the validation data, as would be inferred from simple scatterplots.

Fig. 13.

Maximum HSS for a given GV rain rate at KWAJ for each of the various satellites. Line colors specify the particular satellite estimate.

Fig. 13.

Maximum HSS for a given GV rain rate at KWAJ for each of the various satellites. Line colors specify the particular satellite estimate.

Figure 14a shows the HSS2D for MELB over ocean areas only. Once again, the TMI and AMSR skills are quite high, with maximum HSS values of about 0.8 in the 1–2 mm h−1 range. Also, there is a clear negative bias in both TMI and AMSR estimates, which is exacerbated at rain rates greater than 10 mm h−1. Although the biases of the SSM/I estimates tend to be smaller than the AMSR-E biases, its skills are lower. AMSU-B shows the lowest skill, again, over ocean areas at MELB, with little or no skill in observing rain rates beyond about 10 mm h−1.

Fig. 14.

(a) As in Fig. 12, but for MELB over (a) ocean areas, (b) land areas, (c) coastal areas, and (d) the entire GV domain.

Fig. 14.

(a) As in Fig. 12, but for MELB over (a) ocean areas, (b) land areas, (c) coastal areas, and (d) the entire GV domain.

Over land, shown in Fig. 14b, the AMSR-E skills are not as high as over ocean, but are unbiased. TMI actually shows a slight increase in skill, relative to AMSR-E. The SSM/I shows slightly lower skills than AMSR-E, but biases are near zero for rain rates less than 10 mm h−1. AMSU-B shows the lowest skills at all rain rates, with a negative bias (i.e., AMSU-B less than GV). Over coast, the AMSR-E and SSM/I estimates show skills on the order of 0.5–0.8 for rain rates less than 10–15 mm h−1; however, the AMSR-E estimates are negatively biased (−8.2%), while all of the SSM/I estimates are positively biased (+44.4%–71.1%). Given that AMSR-E, TMI, and SSM/I all use the GPROF algorithm, we are not sure how to assess the discrepancy between their performances over coastal areas, but suggest that algorithm developers investigate possible reasons and incorporate the apparent improvements into their algorithms. Finally, over the entire GV domain at MELB, shown in Fig. 14d, AMSR-E shows the highest skills with a slight negative bias, SSM/I shows somewhat reduced skill but either near-unity or slightly positive biases, and AMSU-B shows the lowest skill and is strongly negatively biased.

Figure 15 shows the maximum HSS for each validation rain bin. Over ocean, both TMI and AMSR show significantly better skill at all rain rates than SSM/I and AMSU-B estimates. Over land, differences between the HSS maxima between AMSR-E, TMI, and other estimates are not as large but still show increased skills over all except the highest rain rates, where F15 shows some increased skill at rain rates greater than 17 mm h−1.

5. Summary and conclusions

In this study, 4 yr (2003–06) of instantaneous radar rain estimates obtained from the Tropical Rainfall Measuring Mission Ground Validation sites at Kwajalein, Republic of the Marshall Islands, and Melbourne, Florida, were used to assess the relative performance of satellite precipitation estimates from seven polar-orbiting satellites and the TMI onboard TRMM. Instantaneous rain rates derived from each microwave sensor onboard the different satellites were matched to the GV estimates in time and space at a resolution of 0.25°. The study evaluates the measurement and error characteristics of the various satellite estimates through intercomparisons with GV radar estimates. The GV rain observations provided an empirical ground-based reference for assessing the relative performance of each sensor and sensor class.

All eight satellites compared well with GV at KWAJ and MELB when intercompared on monthly scales, especially over the ocean where all of the available rain information was utilized. The microwave sensors were more prone to overestimates over land and coast, and not unexpectedly, they performed the worst in the case of MELB coast. AMSR-E and the TMI generally performed the best over all terrain types, though their biases over the ocean were commensurate with the other satellites. TMI and AMSR also showed the best correlation with GV and displayed the highest skills in observing GV rain rates over the full dynamic range of the observations. Table 3 illustrates that over ocean at KWAJ, all of the satellite estimates showed negative biases on the order of −10% to −25%.

Table 4 revealed that over ocean areas at MELB, the biases were also generally negative, ranging from +0.5% (F15) to −29% (N17). Over land at MELB, there was considerable more spread in the sign and magnitude of the biases, with a +25.5% positive bias for F13 and −18% bias for N17. Also, over land at MELB, AMSR-E displayed a very small bias of −1.2%, while TMI showed a positive bias of 12.2%. We infer that the differences in these biases are due to the following: 1) the TRMM orbit allowing a full sample of the diurnal cycle, especially over the large temporal scales employed in this study; 2) minor variations in the instrumentation of the different platforms; and 3) variations in the algorithms used to retrieve the rain rates, often referred to as “versionitis.”

Over coastal areas at MELB (Table 4), there was considerable deviation in the biases, with SSM/I showing large positive biases ranging from +44% to +71%, as well as the rather inconsistent AMSU-B estimates, which range from −43% to +0.5%. TMI and AMSR-E both showed relatively small negative biases of −12.9% and −8.2%, respectively. Looking at the entire domain, irrespective of the terrain type, the biases range from −31.8% (N17) to +33.5% (F14), with TMI and AMSR showing remarkably small biases of −4.2% and −5.5%.

It is important to note that assessing the biases alone is not sufficient to determine which satellite estimates fare best, especially since the GV estimates themselves may incur some bias. Further, a simple comparison of the PDF can also be misleading, because the information relating to the time–space matching is lost in the statistical construction of the PDF. Hence, analysis of scatterplots, simple statistics, and the two-dimensional skill scores, as was done in this study, is needed to provide a more complete picture of how a satellite estimate is performing, especially with respect to the observation (i.e., detection) surface precipitation from space.

The rain-rate profiles generated in Figs. 10 and 11 revealed that only TMI and AMSR-E performed well in estimating GV rain rates less than 1.0 mm h−1, while the other satellites tended to overestimate the rain rates in this region. This result is important because the sampling was extremely high in that region of each profile. Two of the AMSU-B estimates (N15 and N16) performed the worst in overestimating the low rain rates, while N17 underestimated the high rain rates by more than any of the other satellites. It is noteworthy that over land and coast there was more overall dispersion between the different satellites. The profiles for each sensor group, nonetheless, tended to display similar characteristics.

Probability distributions for the various satellites indicate that both TMI and AMSR-E are significantly better and more highly correlated with GV estimates than the SSM/I retrievals, and to a larger extent the AMSU-B estimates. Ad hoc adjustment of the AMSU-B estimates of light rain rates (approximately 1 mm h−1) over ocean, in particular, provides for physically implausible distributions, and caution should be taken when these data are used. Two-dimensional Heidke skill score plots show also that both TMI and AMSR-E are significantly better skilled at detecting surface precipitation, providing less biased, more highly correlated, and more linear comparisons than SSM/I and significantly more so than AMSU-B, especially over ocean. Over land areas, the differences between the skills of the various estimates are smaller, but we find the TMI estimates to be superior, followed by AMSR-E, SSM/I, and AMSU-B.

These results signal to developers of global rainfall products, such as the TRMM MultiSatellite Precipitation Analysis (TMPA), that care must be taken when incorporating data from these input satellite estimates to provide the highest-quality estimates when attempting to merge the data from external sources. For example, as is currently done with the TMPA product, if two or more estimates are available in a given 3-h, 0.25° grid, the two estimates would be averaged. We would suggest instead that the contributions be weighted by quality to provide the most robust estimates.

Acknowledgments

National Aeronautics and Space Administration (NASA) Grant NNG07EJ50C funded this study. The authors thank Dr. Ramesh Kakar (NASA Headquarters), Dr. Scott Braun (TRMM Project Scientist), Dr. Arthur Hou (GPM Project Scientist), and Mr. Richard Lawrence (Chief, TRMM Satellite Validation Office) for their support of this effort. We also greatly appreciate the efforts of Dr. Erich Stocker of the TRMM Science and Data Information System (TSDIS) and NASA Precipitation Processing System (PPS) for generating the quarter-degree TRMM data used in this study. We also thank the support staff of the TRMM Ground Validation program at NASA GSFC, especially David Marks, David Silberstein, Jason Pippitt, and David Makofski.

REFERENCES

REFERENCES
Conner
,
M. D.
, and
G. W.
Petty
,
1998
:
Validation and intercomparison of SSM/I rain-rate retrieval methods over the continental United States.
J. Appl. Meteor.
,
37
,
679
700
.
Ferraro
,
R. R.
,
1997
:
Special sensor microwave imager derived global rainfall estimates for climatological applications.
J. Geophys. Res.
,
102
, (
D14
).
16715
16735
.
Ferraro
,
R. R.
, and
G. F.
Marks
,
1995
:
The development of SSM/I rain-rate retrieval algorithms using ground-based radar measurements.
J. Atmos. Oceanic Technol.
,
12
,
755
770
.
Hsu
,
K.
,
X.
Gao
,
S.
Sorooshian
, and
H. V.
Gupta
,
1997
:
Precipitation estimation from remotely sensed information using artificial neural networks.
J. Appl. Meteor.
,
36
,
1176
1190
.
Huffman
,
G. J.
, and
Coauthors
,
2007
:
The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales.
J. Hydrometeor.
,
8
,
38
55
.
Joyce
,
R. J.
,
J. E.
Janowiak
,
P. A.
Arkin
, and
P.
Xie
,
2004
:
CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution.
J. Hydrometeor.
,
5
,
487
503
.
Kongoli
,
C.
,
R. R.
Ferraro
,
P.
Pellegrino
,
H.
Meng
, and
C.
Dean
,
2007
:
The utilization of the AMSU high frequency measurements for improved coastal rain retrievals.
Geophys. Res. Lett.
,
34
,
L17809
.
doi:10.1029/2007GL029940
.
Kummerow
,
C.
, and
L.
Giglio
,
1994
:
A passive microwave technique for estimating rainfall and vertical structure information from space. Part II: Applications to SSM/I data.
J. Appl. Meteor.
,
33
,
19
34
.
Kummerow
,
C.
,
W. S.
Olson
, and
L.
Giglio
,
1996
:
A simplified scheme for obtaining precipitation and vertical hydrometeor profiles from passive microwave sensors.
IEEE Trans. Geosci. Remote Sens.
,
34
,
1213
1232
.
Kummerow
,
C.
,
W.
Barnes
,
T.
Kozu
,
J.
Shiue
, and
J.
Simpson
,
1998
:
The Tropical Rainfall Measuring Mission (TRMM) sensor package.
J. Atmos. Oceanic Technol.
,
15
,
809
817
.
Kummerow
,
C.
, and
Coauthors
,
2000
:
The status of the Tropical Rainfall Measuring Mission (TRMM) after two years in orbit.
J. Appl. Meteor.
,
39
,
1965
1982
.
Kummerow
,
C.
, and
Coauthors
,
2001
:
The evolution of the Goddard profiling algorithm (GPROF) for rainfall estimation from passive microwave sensors.
J. Appl. Meteor.
,
40
,
1801
1820
.
Kummerow
,
C.
,
W.
Berg
,
J.
Thomas-Stahle
, and
H.
Masunaga
,
2006
:
Quantifying global uncertainties in a simple microwave rainfall algorithm.
J. Atmos. Oceanic Technol.
,
23
,
23
37
.
Lin
,
X.
, and
A. Y.
Hou
,
2008
:
Evaluation of coincident passive microwave rainfall estimates using TRMM PR and ground measurements as references.
J. Appl. Meteor. Climatol.
,
47
,
3170
3187
.
Marks
,
D. A.
,
D. B.
Wolff
,
D. S.
Silberstein
,
A.
Tokay
,
J. L.
Pippitt
, and
J.
Wang
,
2009
:
Availability of high-quality TRMM ground validation data from Kwajalein, RMI: A practical application of the relative calibration adjustment technique.
J. Atmos. Oceanic Technol.
,
26
,
413
429
.
McCollum
,
J. R.
, and
R. R.
Ferraro
,
2003
:
Next generation of NOAA/NESDIS TMI, SSM/I, and AMSR-E microwave land rainfall algorithms.
J. Geophys. Res.
,
108
,
8382
.
doi:10.1029/2001JD001512
.
McCollum
,
J. R.
, and
R. R.
Ferraro
,
2005
:
Microwave rainfall estimation over coasts.
J. Atmos. Oceanic Technol.
,
22
,
497
512
.
Prabhakara
,
C.
,
G.
Dalu
,
G. L.
Liberti
,
J. J.
Nucciarone
, and
R.
Suhasini
,
1992
:
Rainfall estimation over oceans from SSMR and SSM/I microwave data.
J. Appl. Meteor.
,
31
,
532
552
.
Qiu
,
S.
,
P.
Pellegrino
,
R.
Ferraro
, and
L.
Zhao
,
2005
:
The improved AMSU rain-rate algorithm and its evaluation for a cool season event in the western United States.
Wea. Forecasting
,
20
,
761
774
.
Rosenfeld
,
D. E. A.
, and
D. B.
Wolff
,
1995
:
Improved accuracy of radar WPMM estimated rainfall upon application of objective classification criteria.
J. Appl. Meteor.
,
34
,
212
223
.
Silberstein
,
D. S.
,
D. B.
Wolff
,
D. A.
Marks
,
D.
Atlas
, and
J. L.
Pippitt
,
2008
:
Ground clutter as a monitor of radar stability at Kwajalein, RMI.
J. Atmos. Oceanic Technol.
,
25
,
2037
2045
.
Simpson
,
J.
,
R. F.
Adler
, and
G. R.
North
,
1988
:
Proposed Tropical Rainfall Measuring Mission (TRMM) satellite.
Bull. Amer. Meteor. Soc.
,
69
,
278
295
.
Simpson
,
J.
,
C.
Kummerow
,
W-K.
Tao
, and
R. F.
Adler
,
1996
:
On the Tropical Rainfall Measuring Mission (TRMM).
Meteor. Atmos. Phys.
,
60
,
19
36
.
Sorooshian
,
S.
,
K-L.
Hsu
,
X.
Gao
,
H. V.
Gupta
,
B.
Imam
, and
D.
Braithwaite
,
2000
:
Evaluation of PERSIANN system satellite-based estimates of tropical rainfall.
Bull. Amer. Meteor. Soc.
,
81
,
2035
2046
.
Spencer
,
R. W.
,
H. M.
Goodman
, and
R. E.
Hood
,
1989
:
Precipitation retrieval over land and ocean with the SSM/I. Part I: Identification and characteristics of the scattering signal.
J. Atmos. Oceanic Technol.
,
6
,
254
273
.
Takahashi
,
N.
,
H.
Kuroiwa
, and
T.
Kawanishi
,
2003
:
Four-year result of external calibration for precipitation radar (PR) of the Tropical Rainfall Measuring Mission (TRMM) satellite.
IEEE Trans. Geosci. Remote Sens.
,
41
,
2398
2403
.
Vila
,
D.
,
R.
Ferraro
, and
R.
Joyce
,
2007
:
Evaluation and improvement of AMSU precipitation retrievals.
J. Geophys. Res.
,
112
,
D20119
.
doi:10.1029/2007JD008617
.
Wang
,
J.
, and
D. B.
Wolff
,
2009
:
Comparisons of reflectivities from TRMM precipitation radar and ground-based radars.
J. Atmos. Oceanic Technol.
,
26
,
857
875
.
Weinman
,
J. A.
, and
P. J.
Guetter
,
1977
:
Determination of rainfall from microwave radiation measured by Nimbus 6 ESMR.
J. Appl. Meteor.
,
16
,
437
442
.
Weng
,
F.
,
L.
Zhao
,
R. R.
Ferraro
,
G.
Poe
,
X.
Li
, and
N. C.
Grody
,
2003
:
Advanced microwave sounding unit cloud and precipitation algorithms.
Radio Sci.
,
38
,
8068
.
doi:10.1029/2002RS002679
.
Wilheit
,
T.
,
C. D.
Kummerow
, and
R.
Ferraro
,
2003
:
NASDA rainfall algorithms for AMSR-E.
IEEE Trans. Geosci. Remote Sens.
,
41
,
204
214
.
Wolff
,
D. B.
, and
B. L.
Fisher
,
2008
:
Comparisons of instantaneous TRMM ground validation and satellite rain-rate estimates at different spatial scales.
J. Appl. Meteor. Climatol.
,
47
,
2215
2237
.
Wolff
,
D. B.
,
B. L.
Fisher
,
D. A.
Short
,
O. W.
Thiele
, and
D. S.
Han
,
1995
:
Diurnal cycle of tropical rainfall based on rain gauge data: Implications for satellite rainfall retrievals.
Preprints, 27th Conf. on Radar Meteorology, Vail, CO, Amer. Meteor. Soc., 743–745
.
Wolff
,
D. B.
,
D. A.
Marks
,
E.
Amitai
,
D. S.
Silberstein
,
B. L.
Fisher
,
A.
Tokay
,
J.
Wang
, and
J. L.
Pippitt
,
2005
:
Ground validation for the Tropical Rainfall Measuring Mission.
J. Atmos. Oceanic Technol.
,
22
,
365
380
.

Footnotes

Corresponding author address: David B. Wolff, NASA/GSFC, Code 613.1, Greenbelt, MD 20771. Email: david.b.wolff@nasa.gov