1. Introduction
Over the past decade, a number of precipitation products with high spatial and temporal resolution and near-global coverage have been developed. These products combine precipitation information from multiple sensors and multiple algorithms to produce estimates of rainfall over the globe at spatial resolutions of 0.25° latitude/longitude (or finer) and 3-h temporal resolution (or less). Because these products are constructed from satellite data, they supply crucial rainfall information over the oceans and parts of the world where conventional surface-based observations of rainfall (rain gauges and radars) are very sparse or nonexistent. These products are similar in that most of them combine data from passive microwave (PM) and thermal infrared (TIR) sensors. The main differences among them are the manner in which the individual data inputs are combined. Other differences may include use of rain gauge observations to reduce bias and the spatial and temporal resolution of the products.
These differences may lead to differences in the accuracy of these estimates over different regions of the world. Thus, the evaluation of the different satellite rainfall estimates over different climatic and geographic regions is very important. This will be useful in identifying specific weaknesses and strengths of the different products under different circumstances. However, the evaluation of these products, particularly over Africa and South America, has been very limited. Yet it may be argued that these regions are where the satellite products are needed most because of the sparse station networks over most parts of the two regions. Lack of ground observations and lack of access to the available observations have been among the major limitations to validations of satellite rainfall estimates over these regions. Though the distribution of gauges is generally sparse over these regions, there are some relatively data rich areas. Dinku et al. (2007, 2008) used a relatively dense station network over East Africa to evaluate a number of satellite rainfall products over parts of this region.
A station network of about 600 gauges is used here to evaluate seven satellite rainfall products over Colombia in South America. The rain gauge data are obtained from the Institute of Meteorology, Hydrology and Environmental Studies (IDEAM) in Colombia. The evaluated satellite rainfall products are the Tropical Rainfall Measuring Mission (TRMM) multisatellite precipitation analysis (TMPA; Huffman et al. 2007) products, the National Oceanic and Atmospheric Administration Climate Prediction Center (NOAA/CPC) morphing technique (CMORPH; Joyce et al. 2004), Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network (PERSIANN; Hsu et al. 1997), the Naval Research Laboratory’s blended product (NRLB; Turk et al. 1999), and the Global Satellite Mapping of Precipitation (GSMaP; Okamoto et al. 2007). These products are evaluated at daily and 10-daily time scales and a spatial resolution of 0.25° latitude/longitude. The next sections describe the validation region, gauge, and satellite data used and present and discuss the results.
2. Validation region and data
a. Validation region
The validation data come from Colombia, which is located over the northwestern part of South America (Fig. 1). The main topographic features are the three mountain ranges that form the northern tip of the Andes Mountains, the two valleys that divide the mountain ranges, and the large plain region that covers the eastern and southern parts of the country. The elevation of the mountainous region can exceed 3000 m while the eastern half of the country is below 500 m. The change from the lowland region to the mountainous region is very dramatic (Fig. 1), which has significant influence on the rainfall climatological pattern (Fig. 2). The western coastal region is the wettest, with mean annual rainfall up to 11 000 mm, and the lowland area in the north is the driest, with mean annual rainfall of less than 500 mm (Hurtado 2005). The eastern plain receives between 2000 and 3000 mm, and the southern lowlands receive from 3000 to 4000 mm of annual rainfall. The intertropical convergence zone, which stays in the vicinity of Colombia throughout the year, is the main synoptic feature.
Validation statistics are computed for the whole country as well as for four different parts of the country separately. The four regions are (referring to Fig. 1)
the highland region, consisting of all areas with elevations that are above 750 m,
the eastern region, consisting of areas east of 74°W and south of 7°N with elevations that are less than or equal to 500 m,
the northern region, consisting of areas north of 7°N with elevations that are less than or equal to 500 m, and
the western region, consisting of areas west of 76°W with elevations that are less than or equal to 500 m.
b. Rain gauge data
Rainfall data from about 900 stations were provided by IDEAM. After a laborious quality check, only about 600 stations were retained and used for the current evaluation. The quality check included removing nonnumerical values (there were many, some representing rainfall values of less than 0.1 mm), removing unreasonably high values (when compared with nearby stations), replacing blanks with missing values, and checking station names, coordinates, and identifiers. This was done manually for each data file. The distribution of the stations used is shown in Fig. 1. The density of the stations is very good over the mountainous region. It is very sparse over the southern and eastern plains, but the spatial variability of rainfall is also low over those regions (Fig. 2). Data from 2003 to 2005 were used. The quality-controlled rain gauge measurements were interpolated into regular grids of 0.05° latitude/longitude using kriging and an approach similar to Barancourt et al. (1992). The interpolated values were then averaged to 0.25° spatial resolution for comparison with the satellite products. All available stations were used for interpolation, but only 0.25° grid boxes with at least one rain gauge were used for comparison with the satellite pixels. There are 274 grid boxes with at least one rain gauge, and there are 87, 38, 9, 1, and 2 grid boxes with two, three, four, five, and six rain gauges, respectively.
c. Satellite data
The following satellite products were evaluated: CMORPH (Joyce et al. 2004), which is produced by NOAA/CPC; TMPA products (Huffman et al. 2007) 3B42 and its near-real-time version 3B42RT, which are produced by the TRMM project at the National Aeronautics and Space Administration; PERSIANN (Hsu et al. 1997) from the University of California, Irvine; NRLB (Turk et al. 1999); and GSMaP from Osaka Prefecture University in Japan (Okamoto et al. 2007). Two of the GSMaP products, GSMaP moving vector with Kalman filter (GSMaP-MVK) and its latest version (GSMaP-MVK+), are evaluated. Table 1 provides a summary of the main characteristics of the above satellite rainfall products.
The CMORPH algorithm uses a relatively new technique to combine the better accuracy of the PM rainfall estimates with better sampling frequency of TIR observations. The PM rainfall estimates produced from different sensors are interpolated (morphed) using motion vectors derived from half-hourly TIR observations (Joyce et al. 2004). The final product is a spatially and temporally complete PM rainfall estimate, which is independent of the TIR rainfall estimates. The TMPA algorithm is used to produce 3B42 and 3B42RT (Huffman et al. 2007). This algorithm combines TIR data from geostationary satellites and PM retrievals from different sources in four steps:
the different PM estimates are adjusted and combined,
TIR precipitation estimates are created using the PM estimates for calibration,
PM and TIR estimates are combined, and
the final product is rescaled to monthly totals whereby gauge observations are used, indirectly, to adjust the combined satellite product.
The PERSIANN algorithm uses a three-layer feed-forward artificial neural network (ANN) technique to estimate rainfall rates from TIR data (Hsu et al. 1997). The features used in the ANN include infrared brightness temperature (Tb) of the pixel, mean Tb of the 3 × 3 and 5 × 5 pixel windows around the pixels of interest, and standard deviations of Tbs in these windows. Initially the ANN was trained using radar data and the input was limited to TIR data. The recent version also uses daytime visible imagery (Hsu et al. 1999) and the TRMM Microwave Imager rainfall estimates (2A12) to adjust the ANN parameters (Sorooshian et al. 2000). The NRLB algorithm (Turk et al. 1999) uses area-dependent statistical relationships between collocated PM rainfall estimates and TIR observations. Here the PM estimates are used only to calibrate the TIR brightness temperature using the probability-matching method. The PM passes that are within 15 min of TIR observations are collocated with TIR geolocations and saved. Then the probability distribution functions of the PM rain estimates are matched with those of TIR brightness temperatures to estimate rainfall rates from the TIR brightness temperatures.
The GSMaP-MVK is one of the latest in this category of algorithms. It starts with CMORPH’s algorithm of using TIR-derived motion vectors for propagating PM estimates in time and space. Whereas CMORPH uses the TIR observations just to extract information about the time evolution of the PM rain rates, GSMaP-MVK (hereinafter referred to as GSMaP) also uses the TIR rainfall estimates, at times when the PM estimates are not present, along with the propagated PM estimates within a Kalman filter framework (Okamoto et al. 2007). Thus, in this case the hourly TIR observations provide more than just the evolution of the PM rain rates. NOAA/CPC is also in the process of developing a Kalman filter approach to combine the various satellite inputs in the CMORPH technique. GSMaP-MVK+ (hereinafter GSMaP+) is the latest version of GSMaP. The main difference is that the earlier version did not use PM data from sounders, mainly the Advanced Microwave Sounding Unit (AMSU) on the NOAA satellites. The comparison of GSMaP and GSMaP+ may thus help to explore the effect of the AMSU data.
3. Validation and intercomparison
a. Validation of daily rainfall
Table 3 presents validation results obtained using daily rainfall data from all of the stations in Fig. 1. Correlation coefficients are low but look reasonable for daily rainfall over such complex terrain. All of the products, except PERSIANN, underestimate the amount (bias < 1) and frequency (FBS < 1) of rainfall. However, the ME is small for most of the products relative to the average rainfall. MAE is comparable to the average rainfall, except for PERSIANN for which it is more than 2 times the average rainfall. Detection probabilities are close to 70% for most of the products, and false-alarm ratios are generally small, ranging from 0.09 for GSMaP to 0.21 for PERSIANN. The HSS statistics show that the satellite estimates do have reasonably good skills in detecting the occurrence of rainfall.
In a comparison of the different satellite products, two products stand out as having the poorest relative performances while two others exhibit the best performance in this group of satellite products. Products with poor performance are PERSIANN, with large overestimation and GSMaP with serious underestimation. GSMaP has also very low probability of detection. Comparison of GSMaP and GSMaP+ shows that the use of AMSU data does make a difference in the accuracy of the estimates. This is consistent with the report by Kubota et al. (2009). The products with the best performance are GSMaP+ and CMORPH. Both products have higher correlation coefficients and HSS values. CMORPH has a slightly better accuracy—its underestimations of rainfall amount and frequency are less than that of GSMaP+. The performance of the other three products, NRLB, 3B42RT, and 3B42, is different for the different statistics. The NRLB product exhibits the lowest bias of all of the products but has slightly lower correlation and HSS values and higher MAE. When 3B42RT and 3B42 are compared, the latter outperforms the former in every statistic except the bias. The rain gauge adjustment involved in 3B42 is supposed to remove the bias; thus, it is not clear why the adjusted product shows more bias than the unadjusted one. PERSIANN, GSMaP, and 3B42RT will not be considered in further analyses because of their relative weak performances.
As shown in Fig. 1, the validation region has very complex terrain. Thus, validation with data from the whole country may not give the whole picture. Validations are therefore performed over different parts the country to investigate the performance of the products over different climatic regimes. The first division is between “lowlands” and “highlands.” Lowlands are defined here as areas having an elevation of less than 500 m, and the highland areas are defined as having an elevation that is above 750 m. This division is arbitrary but intuitive (Figs. 1, 2). Table 4 compares the results for the lowland and highland regions. These results show that the performance of the satellite products is better over the lowlands, with lower bias and better rainfall detection accuracy. However, the correlation coefficients are slightly higher and the MAE is slightly smaller over the highland region. The better correlation might be because of the strong seasonality (two distinct peaks; Fig. 2) of rainfall over the highland areas.
The division of the country into two broad categories is still very coarse. For instance, the areas under 500 m cover regions with very different climatological characteristics (Fig. 2). As a result, the lowland areas are further divided into three subregions as defined in section 2a. The highland area was also divided into two regions (west and east of 75°W), and validation statistics were computed for each region separately (results not presented). The eastern part has a slightly better performance, but the differences were not significant.
Some of the validation statistics (bias, FAR, and HSS) for the three lowland regions are presented in Fig. 3. These statistics are selected because they show some observable differences among the different regions. The main conclusion that can be drawn from these results is that the performance of the satellite products is relatively better over the eastern region: there is considerable overestimation over the northern region and serious underestimation over the western region (Pacific coast). All products underestimate rainfall amounts over the western region; NRLB, CMORPH, and GSMaP+ overestimate rainfall amounts over the northern region. Rainfall amounts are also underestimated over the eastern region, but the underestimation is not as severe as that observed over the Pacific coast. It is interesting to note that 3B42 underestimates rainfall over all three regions, with severe underestimation over the western region. The false-alarm ratios are higher over the northern (and drier) region. The HSS values are significantly higher for the northern and eastern regions. Comparison of FBS and POD values over the western region with values for the other two regions (not shown) confirms the weakness of the satellite products in detecting rainfall over this region. The underestimation of rainfall over the western region may be attributed to the warm-rainfall process associated with orographic uplift caused by onshore flow toward the Andes; the overestimation over the northern region could be a result of subcloud evaporation.
Seasonal variations of the different validation statistics were also investigated. Figures 4a and 4b compare correlation coefficient and HSS values across different months over the eastern and western parts. Both figures show distinct monthly variations. For the eastern region, correlation coefficient is lowest in July and HSS has its the lowest value in June. The overall patterns are similar for the two statistics, however. The seasonal patterns are slightly different for the western region where the monthly variations for correlation coefficient and HSS values are also different. Correlation coefficient peaks in January, April, and September; HSS values are higher during January–March and lower during June–September. The mean monthly rainfall over the two regions is given in Fig. 5. Comparison of Figs. 4 and 5 shows that satellite products have better accuracy during the relatively dry months and perform relatively poorly during the wet months.
b. Validation of 10-daily rainfall
The satellite products are also evaluated at 10-daily (hereinafter called dekadal) accumulations, but at the same spatial resolution as the daily rainfall (0.25°). The resolution is kept the same to explore just the effect of temporal aggregation on the accuracy of the estimates. Evaluations are done for the four regions described in section2a. The results are presented in Tables 5 –8, and in the scatterplots of Figs. 6 and 7. Only some of the statistics used to evaluate daily rainfall estimates are computed, because rainfall detection is not an issue at dekadal time scale. One additional statistic, Eff, is used to evaluate the skill of the satellite estimates with respect to a reference average rainfall. The average is computed over all pixels and all times for a given region. Only pixels containing at least one gauge are used in computing the average values. Eff was not used for daily rainfall because the main interest at that scale is rainfall detection, and the skill of satellite estimates is generally not very good at the daily time scale.
Results for the highland region are presented in Table 5. Comparison of the statistics in Table 5 with the corresponding statistics in Table 3 shows that there is an increase in the correlation coefficient, a decrease in MAE, but no change in the multiplicative bias. The Eff statistics show that the products have some skill except for NRLB. All of the other products underestimate most of the rainfall amounts. As discussed earlier, the underestimation by the other products may be attributed to an orographic warm-rain process, but the source of overestimation by NRLB is not clear. Table 6 and Fig. 6 compare results for the eastern lowlands. Again there are significant improvements over the corresponding statistics computed for daily rainfall. The correlation and the Eff values are better than those for the highland region. Figure 6 shows that there is a good agreement between gauge observation and the satellite estimates over the eastern region. However, GSMaP+ exhibits considerable bias, which is consistent with the bias value in Table 6. Though CC looks reasonable, the skill of the products is not very good for the northern region (Table 7). Estimation skill is particularly bad for NRLB and CMORPH. The overestimation over this region was also observed for the daily rainfall and was attributed to possible evaporation of raindrops in the dry air under the cloud. All products underestimate rainfall amounts over the western (Pacific coast) region as shown in Table 8 and Fig. 7. There is also wider scatter relative to that in Fig. 6. As a result, the skill of the satellite products is not very good (Table 8) for this region. The underestimation observed in Fig. 7 is also reflected in the bias values in Table 8.
4. Summary and conclusions
Seven different satellite rainfall estimates were evaluated using a station network of about 600 rain gauges over Colombia. Evaluation is done at daily and 10-daily time scales and a spatial resolution of 0.25° latitude/longitude. Validation statistics were computed for the whole country as well as different parts of the country. The performance of the satellite rainfall products over Colombia is similar to what has been obtained for other similar regions. Overall, the satellite products are reasonably good in detecting the occurrence of rainfall but are poor in estimating the amount of daily rainfall. The products have good skill in estimating rainfall amounts at the 10-daily time scale. Results are different for the different parts of the country. There are significant overestimations of daily rainfall frequency and amount over the relatively dry northern region, and significant underestimations are observed over the mountainous regions and the Pacific coast. The overestimation over the northern region has been ascribed to possible subcloud evaporation, and the underestimations over the mountainous and coastal regions may be associated with warm-rain processes over the two regions. The best performance is observed for the eastern region. Comparisons of the different satellite products have shown that CMORPH and GSMaP+ are the best products, whereas PERSIANN and GSMap exhibited the poorest performance among the products evaluated here.
Acknowledgments
This work was funded by a grant/cooperative agreement from the National Oceanic and Atmospheric Administration. The views expressed herein are those of the author(s) and do not necessarily reflect the views of NOAA or any of its subagencies. We are very grateful to IDEAM for providing the rain gauge data.
REFERENCES
Barancourt, C., J. D. Creutin, and J. Rivoirard, 1992: A method for delineating and estimating rainfall fields. Water Resour. Res., 28 , 1133–1144.
Dinku, T., P. Ceccato, E. Grover-Kopec, M. Lemma, S. J. Connor, and C. F. Ropelewski, 2007: Validation of satellite rainfall products over East Africa’s complex topography. Int. J. Remote Sens., 28 , 1503–1526.
Dinku, T., S. Chidzambwa, P. Ceccato, S. J. Connor, and C. F. Ropelewski, 2008: Validation of high-resolution satellite rainfall products over complex terrain in Africa. Int. J. Remote Sens., 29 , 4097–4110.
Hsu, K. L., X. Gao, S. Sorooshian, and H. V. Gupta, 1997: Precipitation estimation from remotely sensed information using artificial neural networks. J. Appl. Meteor., 36 , 1176–1190.
Hsu, K. L., H. V. Gupta, X. Gao, and S. Sorooshian, 1999: Estimation of physical variables from multi-channel remotely sensed imagery using a neural network: Application to rainfall estimation. Water Resour. Res., 35 , 1605–1618.
Huffman, G. J., and Coauthors, 2007: The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J. Hydrometeor., 8 , 38–55.
Hurtado, G., 2005: Atlas Climatológico de Colombia. IDEAM, 218 pp.
Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5 , 487–503.
Kubota, T., T. Ushio, S. Shige, S. Kida, M. Kachi, and K. Okamoto, 2009: Validation of high-resolution satellite-based rainfall estimates around Japan using gauge-calibrated ground-radar dataset. J. Meteor. Soc. Japan, 87A , 203–222.
Okamoto, K., T. Iguchi, N. Takahashi, T. Ushio, J. Awaka, S. Shige, and T. Kubota, 2007: High precision and high resolution global precipitation map from satellite data. Proc. ISAP, Kaohsiung, Taiwan, Intelligent System Applications to Power Systems, 506–509.
Sorooshian, S., K. Hsu, X. Gao, H. V. Gupta, B. Imam, and D. Braithwaite, 2000: Evaluation of PERSIANN system satellite-based estimates of tropical rainfall. Bull. Amer. Meteor. Soc., 81 , 2035–2046.
Turk, F. J., G. D. Rohaly, J. Hawkins, E. A. Smith, F. S. Marzano, A. Mugnai, and V. Levizzani, 1999: Meteorological applications of precipitation estimation from combined SSM/I, TRMM and infrared geostationary satellite data. Microwave Radiometry and Remote Sensing of the Earth’s Surface and Atmosphere, P. Pampaloni and S. Paloscia, Eds., VSP International Science Publishers, 353–363.
Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. Academic Press, 627 pp.
Topography and distribution of rain gauge stations (plus signs) used for the validation.
Citation: Journal of Applied Meteorology and Climatology 49, 5; 10.1175/2009JAMC2260.1
Spatiotemporal distribution of precipitation pattern (mm) over Colombia (Hurtado 2005).
Citation: Journal of Applied Meteorology and Climatology 49, 5; 10.1175/2009JAMC2260.1
Histograms comparing the performance of daily satellite rainfall estimates over eastern, northern, and western parts of Colombia.
Citation: Journal of Applied Meteorology and Climatology 49, 5; 10.1175/2009JAMC2260.1
Seasonal variation of validation statistics (CC and HSS) over the (a) eastern region and (b) western region.
Citation: Journal of Applied Meteorology and Climatology 49, 5; 10.1175/2009JAMC2260.1
Monthly mean (1971–2000) rainfall over western and eastern parts of Colombia.
Citation: Journal of Applied Meteorology and Climatology 49, 5; 10.1175/2009JAMC2260.1
Comparison of the performance of satellite rainfall estimates over eastern lowlands at the 10-daily time scale and a spatial resolution of 0.25° lat/lon.
Citation: Journal of Applied Meteorology and Climatology 49, 5; 10.1175/2009JAMC2260.1
As in Fig. 6, but for western (Pacific coast) region.
Citation: Journal of Applied Meteorology and Climatology 49, 5; 10.1175/2009JAMC2260.1
Summary of the different satellite products evaluated here; the PM and gauge columns indicate whether the product includes passive microwave and gauge observations.
Contingency table for comparing rain gauge measurements and satellite rainfall estimates. The rainfall threshold used is 1.0 mm.
Validation statistics comparing the performance of daily satellite rainfall estimates over the whole of Colombia, where N is the number of data pairs compared, which is the count of all pixels containing at least one gauge over the 3-yr period (2003–05). Avg is the average of the N values for the gauge observations (mm day−1).
Validation statistics comparing the performance of daily satellite rainfall estimates over the lowland (elevation under 500 m) and highland parts of Columbia.
Validation statistics comparing the performance of 10-daily satellite rainfall estimates over the highlands (elevation > 750 m).
