1. Introduction
Precipitation estimates from spaceborne sensors have provided, over the last decades, a valuable source of information that has enabled a number of hydrologic and climatic studies at the global scale (Ashouri et al. 2015; Serrat-Capdevila et al. 2014; Zhou et al. 2014). After the launch of the Tropical Rainfall Measuring Mission (TRMM) in 1997, which was the first meteorological satellite dedicated to precipitation, the next step toward the future of satellite precipitation missions involved the Global Precipitation Measurement (GPM) Core Observatory (Hou et al. 2014) that was launched in February 2014. One of the main GPM-related precipitation products is the Integrated Multisatellite Retrievals for GPM (IMERG) that was released in early 2015. IMERG is based on an advanced algorithm that combines all available satellite microwave precipitation estimates with satellite infrared estimates and precipitation gauge analysis [see Huffman et al. (2015) for details] to provide a final satellite precipitation product at high spatial (0.1°) and temporal (0.5 h) resolution.
Based on a large body of literature regarding the evaluation of satellite precipitation products (e.g., Mei et al. 2014; AghaKouchak et al. 2012; Anagnostou et al. 2010; Tian and Peters-Lidard 2010; Turk et al. 2008; Hossain and Huffman 2008; Xie and Arkin 1995), a common conclusion is that error in satellite precipitation products depends on the region and precipitation type/magnitude examined. Therefore, performance evaluation of the newly released IMERG product is necessary to provide feedback to the algorithm developers and to understand what is the right context (e.g., in terms of scales and applications) for using these data.
This work provides a preliminary assessment of the IMERG precipitation product performance over the Blue Nile basin in Ethiopia, which is an area where hydrologic predictions have an important societal and economic impact. Results from error analysis of IMERG are contrasted with a current high-resolution satellite product to assess potential improvement from the new product.
2. Study area and data
The study domain is within the upper Blue Nile basin of Ethiopia (Fig. 1a). Terrain is characterized by steep topographic gradients with elevations ranging from 500 m MSL in the Blue Nile gorge to 4000 m MSL at the mountain peaks. The climate system of the region was described in several studies (Korecha and Sorteberg 2013; Segele et al. 2009; Nicholson 2000). The main rain-producing systems for the season are the global rain-bearing systems, particularly the intertropical convergence zone (ITCZ) where the Congo basin and Indian and Atlantic Oceans are moisture sources for the rainy period. The period from June to September represents the major rainy season of the region; however, significant rain storms can occur during May and October.
(a) Geographical location of the upper Blue Nile basin of Ethiopia with respect to the complex mountain chain in the Horn of Africa. The spatial distribution of tipping gauge rainfall stations over the study domain is shown in the rectangular region. Accumulated rainfall (mm) for the period of May–October 2014 is shown for (b) IMERG-Cal, (c) CMORPH, and (d) IMERG-Uncal.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
(a) Geographical location of the upper Blue Nile basin of Ethiopia with respect to the complex mountain chain in the Horn of Africa. The spatial distribution of tipping gauge rainfall stations over the study domain is shown in the rectangular region. Accumulated rainfall (mm) for the period of May–October 2014 is shown for (b) IMERG-Cal, (c) CMORPH, and (d) IMERG-Uncal.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
(a) Geographical location of the upper Blue Nile basin of Ethiopia with respect to the complex mountain chain in the Horn of Africa. The spatial distribution of tipping gauge rainfall stations over the study domain is shown in the rectangular region. Accumulated rainfall (mm) for the period of May–October 2014 is shown for (b) IMERG-Cal, (c) CMORPH, and (d) IMERG-Uncal.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
This study evaluates three satellite precipitation products against hourly rainfall measurements from an experimental network of 37 tipping-bucket rain gauges that are operated by the Ethiopian Institute of Water Resources, none of which are included in the Global Precipitation Climatology Centre gauge dataset (Fig. 1). The experimental high-temporal-resolution tipping gauges are deployed at 55 locations. From those, only 37 gauges passed through a quality-control (QC) process according to the guidelines of the World Meteorological Organization (WMO 2008), and gauges that did not pass the QC were discarded. The first two satellite products are the uncalibrated (IMERG-Uncal) and gauge-calibrated (IMERG-Cal) versions. Both products are provided at high spatial (0.1°) and temporal (0.5 h) resolution (
3. Methodology
Evaluation of satellite rainfall products is carried out at a satellite pixel scale and using gauge rainfall as reference. A total of 36 satellite pixels that contain gauges were considered in the analysis. Satellite rainfall estimates are evaluated at hourly and daily temporal scales. Although rain gauge rainfall was available at 15-min intervals, we chose the 1-h (and not 0.5-h original resolution of IMERG and CMORPH) scale as the highest temporal scale for comparison, as a way to somewhat compensate for the representativeness error between gauge and satellite (considering that only one gauge was falling within each satellite pixel). The observed hourly and daily gauge rainfall data were interpolated, using ordinary kriging algorithm, to produce rainfall fields at 0.02° grid, which were aggregated to 0.1° estimates (that correspond to the scale of comparison with the high-resolution satellite products) and considered as the reference rainfall estimates. Error statistics were based on those 0.1° interpolated gauge estimates that are represented by at least one gauge within their grid cell. Furthermore, the CMORPH product is regridded to 0.1° resolution to reduce the effect of mismatches due to spatial scale difference.
4. Results and discussion
a. Total rainfall pattern
Spatial pattern of total rainfall for the period of May–October 2014 obtained from three satellite products is shown in Figs. 1b–d. Visual verification and qualitative comparison of the spatial distribution among the three satellite products is described. The precipitation pattern exhibits a decreasing trend from west to east, which is consistent for all products. Regions of relatively higher seasonal rainfall amount, particularly above 2000 mm, are identified by all satellite products. The gauge-calibrated IMERG product (Fig. 1b) exhibits higher seasonal rainfall in the central southern part while the uncalibrated IMERG product (Fig. 1d) exhibits rainfall peaks in the northern and southern central parts of the basin. CMORPH also identifies a large area of the northern part of the basin (Fig. 1c) having higher seasonal rainfall. Other characteristics of the CMORPH product are the relatively higher seasonal rainfall in the lowland (near the northwestern boundary of the basin) and the relatively lower rainfall in the southern part of the basin as compared to IMERG-Cal.
b. Error metrics
The error metrics for the unconditional case (i.e., gauge rainfall threshold ≥0.1) are summarized in Table 1. All three satellite products had a slight underestimation of the total observed rainfall; the bias ratio of IMERG-Cal (~96.9%) and IMERG-Uncal (~96.5%) are shown to be better than CMORPH (~89.2%). Similarly, the linear correlation coefficients of IMERG-Cal (~0.55) and IMERG-Uncal (~0.54) were slightly better than CMORPH (0.44) at the daily time scale. At the hourly time scale, the CC decreases for all products to around ~0.33 (IMERG) and ~0.28 (CMORPH). This could be partly attributed to the inherent sampling uncertainty of point observations by rain gauges at short temporal resolution. The rainfall events might cover part of the grid cell away from the gauge location where the satellite estimate represents an areal average and gauge reports no rainfall event. Gridding sparse gauge rainfall data would not necessarily mean improvement of areal rainfall representation as compared to point observation. Unfortunately, the validity of the latter cannot be tested in our case because of the lack of a high-density network (i.e., only one gauge is available within each satellite pixel).
Unconditional values of error metrics of the satellite products over the study domain.
The result from NRMSE showed that the spread of error of satellite products at daily time scale is in the range from 1.31 to 1.37, while at hourly time scale the values increase to range from 4.78 to 5.43 with CMORPH yielding a lower NRMSE. This indicates that the random error components cancel out significantly through temporal aggregation. Past rainfall error studies in the regions (Dinku et al. 2008, 2015) have reported temporal effects on correlation, while Romilly and Gebremichael (2011) have shown that CMORPH underestimates by 11% over the Ethiopian highlands. Results obtained in this study indicate that there is an improvement in systematic error (bias) and correlation coefficient for both IMERG products.
We further examined the error metrics for different rainfall thresholds as discussed in section 3, and the results are shown in Fig. 2. The magnitude of MRE increases with gauge rainfall threshold categories on both time scales, showing that underestimation of rainfall increases with increasing rainfall magnitude. This pattern is consistent for all products examined, with CMORPH being associated with the highest degree of underestimation.
Satellite error metrics presented for different threshold values of gauge rainfall.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
Satellite error metrics presented for different threshold values of gauge rainfall.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
Satellite error metrics presented for different threshold values of gauge rainfall.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
The MAE was used to assess the average magnitude of the absolute error of satellite retrieval for the various threshold categories of rainfall. Results showed that the three satellite products exhibit an increasing trend in MAE with threshold on both time scales (Figs. 2b,f), and the rate of increase of MAE accelerates nearly exponentially for higher thresholds.
The CC at daily time scale is nearly 0.48 (IMERG) and 0.38 (CMORPH) for the 25th quantile threshold and drops to 0.19 (IMERG) and 0.13 (CMORPH) at the 95th quantile; the corresponding values at the hourly time scale are 0.32 (IMERG) and 0.28 (CMORPH) at the 25th quantile and 0.17 (IMERG) and 0.04 (CMORPH) at the 95th quantile, showing that satellite rainfall cannot adequately represent the hourly variability of rainfall as compared to the daily time scale in the domain during the study period. The NRMSE patterns were consistent both in unconditional case (Table 1) and for different quantile threshold values of gauge rainfall (Figs. 2d,h), which showed that CMORPH yields slightly lower values of the random error component on both time scales. At the 25th quantile threshold, the NRMSE for IMERG is nearly 1.19 and CMORPH is about 1.14 at daily resolution; at the hourly time scale, NRMSE is lower for CMORPH (2.26) and drops to 0.59 at the 95th quantile. Even though the magnitudes of the improvements on the error metrics are marginal, the CC and MRE indicate that both IMERG products exhibit an improvement in quantifying higher rainfall values (Fig. 2).
c. Contingency metrics
Values of contingency metrics are summarized in Table 1. For the daily time scale, the POD statistic is 0.87 for IMERG and 0.88 for CMORPH. On the other hand, FAR values are 0.04 for IMERG and 0.06 for CMORPH. At the hourly time scale, IMERG has a POD of 0.43 and FAR is about 0.38 for IMERG and 0.44 for CMORPH. This indicates that IMERG and CMORPH are equivalent in detecting rainy events, while IMERG is slightly better than CMORPH in avoiding detecting false rainfall events. To assess the significance, in terms of rainfall volume of missed and false detected rainfall, we calculated MRV and FRV. These indices (from Table 1) show that at the daily time scale IMERG has relatively lower MRV (4.2%) while CMORPH (4.8%) has a slightly higher missed rainfall volume. False rainfall volume detection has a slightly higher value for CMORPH (2.2%) than IMERG (1.0%). At the hourly time scale, IMERG also has a lower MRV and FRV than CMORPH. The satellite products against reference gauge rainfall are analyzed using the Wilcoxon rank sum two-tailed statistics to test if there is a real statistical difference. The null hypothesis assumes that satellite and gauge rainfall are drawn from the same distribution with equal median, and the alternative hypothesis is that they do not have equal medians. Results show that the difference between a satellite product and gauge rainfall is statistically significantly different at the 1% level of significance on both time scales.
d. Rainfall quantiles and spatial correlation
Figure 3 shows a quantile–quantile (Q–Q) plot of satellite rainfall products against the corresponding gauge rainfall values, for both daily and hourly scales. To exclude the effect of missed/false detection of satellite rainfall products, calculation of quantiles was based on rainfall values when both gauge and satellite measurements were ≥0.1 mm depth for both time scales. This threshold value was obtained after visual inspection of time series and was used to avoid including very low rainfall values that often manifest as high-frequency noise in rainfall signal.
Q–Q plots of (a) daily and (b) hourly satellite rainfall products vs gauge rainfall.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
Q–Q plots of (a) daily and (b) hourly satellite rainfall products vs gauge rainfall.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
Q–Q plots of (a) daily and (b) hourly satellite rainfall products vs gauge rainfall.
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
Visual inspection of Fig. 3 shows that for both time scales CMORPH exhibits consistent underestimation of rainfall quantiles while IMERG switches from underestimation of lower quantiles to overestimation of the higher quantiles. Both products are equivalent below the 50th quantile (~4 mm day−1) on the daily scale, while CMORPH is consistently closer to reference rainfall on the hourly scale and at both temporal scales better captures the highest rainfall quantiles. The difference between calibrated and uncalibrated IMERG is very small and apparent mainly in the highest rainfall quantiles.
Gauge rainfall stations are unevenly distributed in the domain, and separation distances between gauge rainfall locations range from 3 to 450 km. As a rule of thumb, the average lag interval is estimated by computing the mean of the minimum separation of pairs (~27 km) whereas the maximum lag distance is estimated by taking half of the maximum separation distance between gauging stations. The ratio of maximum lag (~225 km) to lag interval (~27 km) gives the number of distance classes or bins (about eight). Rainfall spatial correlation, expressed as average value of correlation coefficients of pairs of pixels within each distance bin versus lag distance is shown in Fig. 4. Note that the entire time period examined was considered for the calculation of correlation coefficients. On the hourly scale, the vast correlation drops between the first two bins indicate the sharp rainfall gradient characteristics of the mountainous tropical rainfall. On the daily scale, as expected, spatial variability is smoother and correlation reaches the characteristic value of 1/e at approximately 75 km. Overall, results show that the general pattern of rainfall spatial variability is captured adequately well by all satellite products, with IMERG exhibiting a slightly higher degree of agreement with reference rainfall while CMORPH exhibits a somewhat smoother signal (i.e., associated with higher correlation values), particularly on the daily scale. Correlation decay with distance is similar among products, with the exception of CMORPH on the hourly scale in the first distance bins that exhibits a considerably higher decay rate (from ~0.7 to 0.3) than the rest of products (going from ~0.5 to 0.27). Distinct differences between IMERG and CMORPH, such as in first and second distance bin correlation, can potentially be used to highlight the effect of different approaches, in the corresponding algorithms, regarding morphing and estimation techniques applied.
Spatial correlation of satellite and gauge (a) daily and (b) hourly rainfall. Correlation values shown correspond to the average values within a lag distance bin of size (~27 km).
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
Spatial correlation of satellite and gauge (a) daily and (b) hourly rainfall. Correlation values shown correspond to the average values within a lag distance bin of size (~27 km).
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
Spatial correlation of satellite and gauge (a) daily and (b) hourly rainfall. Correlation values shown correspond to the average values within a lag distance bin of size (~27 km).
Citation: Journal of Hydrometeorology 17, 11; 10.1175/JHM-D-15-0230.1
5. Conclusions
The results of this study provided a first evaluation of the new IMERG and KF-based CMORPH satellite products at two temporal resolutions (daily and hourly) for a rainy period (May–October 2014) over the Blue Nile basin, Ethiopia. It is shown that the three satellite products moderately underestimate the rainfall values in the study period. The visual comparison from the Q–Q plots of satellite products versus gauge products indicates that higher reference rainfall quantiles are better represented by CMORPH, which can have an important implication for flood-related applications of this product.
Error metrics showed that the satellite products slightly underestimated the total seasonal rainfall amount. The gauge-calibrated IMERG product exhibited marginal improvements relative to CMORPH in the overall bias. The CC of daily time series showed that the IMERG product is marginally better than CMORPH. The random error (NRMSE) analysis showed that daily satellite products are closer to gauge measurements than hourly estimates; CMORPH yields slightly lower values of the random error component on both time scales. The POD, FAR, MRV, and FRV error metrics showed a slight improvement in the skill of detecting rainfall events in IMERG compared to CMORPH. Results also show a decreasing trend in detection ability of satellite products for increasing threshold values, highlighting the need to further improve detection during heavy precipitation. Overall, the performances of IMERG and CMORPH products are approximately equivalent, with IMERG showing marginally improved comparison statistics. Results from the performance of day-1 IMERG product are quite satisfactory and build up our expectations for the future, more advanced versions of this product.
Acknowledgments
This work was supported by the EU-funded Earth2Observe (ENVE.2013.6.3-3) project. The authors thank the three anonymous reviewers for providing constructive comments that helped us to improve the original version of this manuscript.
REFERENCES
AghaKouchak, A., and Mehran A. , 2013: Extended contingency table: Performance metrics for satellite observations and climate model simulations. Water Resour. Res., 49, 7144–7149, doi:10.1002/wrcr.20498.
AghaKouchak, A., Mehran A. , Norouzi H. , and Behrangi A. , 2012: Systematic and random error components in satellite precipitation data sets. Geophys. Res. Lett., 39, L09406, doi:10.1029/2012GL051592.
Anagnostou, E. N., Maggioni V. , Nikolopoulos E. I. , Meskele T. , Hossain F. , and Papadopoulos A. , 2010: Benchmarking high-resolution global satellite rainfall products to radar and rain-gauge rainfall estimates. IEEE Trans. Geosci. Remote Sens., 48, 1667–1683, doi:10.1109/TGRS.2009.2034736.
Ashouri, H., Hsu K.-L. , Sorooshian S. , Braithwaite D. K. , Knapp K. R. , Cecil L. D. , Nelson B. R. , and Prat O. P. , 2015: PERSIANN-CDR: Daily precipitation climate data record from multisatellite observations for hydrological and climate studies. Bull. Amer. Meteor. Soc., 96, 69–83, doi:10.1175/BAMS-D-13-00068.1.
Dinku, T., Chidzambwa S. , Ceccato P. , Connor S. J. , and Ropelewski C. F. , 2008: Validation of high‐resolution satellite rainfall products over complex terrain. Int. J. Remote Sens., 29, 4097–4110, doi:10.1080/01431160701772526.
Dinku, T., Ruiz F. , Connor S. J. , and Ceccato P. , 2010: Validation and intercomparison of satellite rainfall estimates over Colombia. J. Appl. Meteor. Climatol., 49, 1004–1014, doi:10.1175/2009JAMC2260.1.
Dinku, T., Alessandrini S. , Evangelisti M. , and Rojas O. , 2015: A description and evaluation of FAO satellite rainfall estimation algorithm. Atmos. Res., 163, 48–60, doi:10.1016/j.atmosres.2015.01.020.
Feidas, H., 2010: Validation of satellite rainfall products over Greece. Theor. Appl. Climatol., 99, 193–216, doi:10.1007/s00704-009-0135-8.
Hossain, F., and Huffman G. J. , 2008: Investigating error metrics for satellite rainfall data at hydrologically relevant scales. J. Hydrometeor., 9, 563–575, doi:10.1175/2007JHM925.1.
Hou, A. Y., and Coauthors, 2014: The Global Precipitation Measurement Mission. Bull. Amer. Meteor. Soc., 95, 701–722, doi:10.1175/BAMS-D-13-00164.1.
Huffman, G. J., Bolvin D. T. , Braithwaite D. , Hsu K. , Joyce R. , Kidd C. , Nelkin E. J. , and Xie P. , 2015: NASA Global Precipitation Measurement Integrated Multi-satellitE Retrievals for GPM (IMERG). Algorithm Theoretical Basis Doc., version 4.5, 30 pp. [Available online at http://pmm.nasa.gov/sites/default/files/document_files/IMERG_ATBD_V4.5.pdf.]
Joyce, R. J., and Xie P. , 2011: Kalman filter–based CMORPH. J. Hydrometeor., 12, 1547–1563, doi:10.1175/JHM-D-11-022.1.
Korecha, D., and Sorteberg A. , 2013: Validation of operational seasonal rainfall forecast in Ethiopia. Water Resour. Res., 49, 7681–7697, doi:10.1002/2013WR013760.
Mei, Y., Anagnostou E. N. , Nikolopoulos E. I. , and Borga M. , 2014: Error analysis of satellite precipitation products in mountainous basins. J. Hydrometeor., 15, 1778–1793, doi:10.1175/JHM-D-13-0194.1.
Nicholson, S. E., 2000: The nature of rainfall variability over Africa on time scales of decades to millenia. Global Planet. Change, 26, 137–158, doi:10.1016/S0921-8181(00)00040-0.
Porcù, F., Milani L. , and Petracca M. , 2014: On the uncertainties in validating satellite instantaneous rainfall estimates with raingauge operational network. Atmos. Res., 144, 73–81, doi:10.1016/j.atmosres.2013.12.007.
Romilly, T. G., and Gebremichael M. , 2011: Evaluation of satellite rainfall estimates over Ethiopian river basins. Hydrol. Earth Syst. Sci., 15, 1505–1514, doi:10.5194/hess-15-1505-2011.
Segele, Z. T., Lamb P. J. , and Leslie L. M. , 2009: Large-scale atmospheric circulation and global sea surface temperature associations with Horn of Africa June–September rainfall. Int. J. Climatol., 29, 1075–1100, doi:10.1002/joc.1751.
Serrat-Capdevila, A., Valdes J. B. , and Stakhiv E. Z. , 2014: Water management applications for satellite precipitation products: Synthesis and recommendations. J. Amer. Water Resour. Assoc., 50, 509–525, doi:10.1111/jawr.12140.
Stampoulis, D., Anagnostou E. N. , and Nikolopoulos E. I. , 2013: Assessment of high-resolution satellite-based rainfall estimates over the Mediterranean during heavy precipitation events. J. Hydrometeor., 14, 1500–1514, doi:10.1175/JHM-D-12-0167.1.
Tang, G., Zeng Z. , Long D. , Guo X. , Yong B. , Zhang W. , and Hong Y. , 2016: Statistical and hydrological comparisons between TRMM and GPM level-3 products over a midlatitude basin: Is Day-1 IMERG a good successor for TMPA 3B42V7? J. Hydrometeor., 17, 121–137, doi:10.1175/JHM-D-15-0059.1.
Tian, Y., and Peters-Lidard C. D. , 2010: A global map of uncertainties in satellite-based precipitation measurements. Geophys. Res. Lett., 37, L24407, doi:10.1029/2010GL046008.
Turk, F. J., Arkin P. , Sapiano M. R. P. , and Ebert E. E. , 2008: Evaluating high-resolution precipitation products. Bull. Amer. Meteor. Soc., 89, 1911–1916, doi:10.1175/2008BAMS2652.1.
WMO, 2008: Guide to meteorological instruments and methods of observation. 7th ed. WMO 8, 716 pp. [Available online at http://library.wmo.int/pmb_ged/wmo_8_en-2012.pdf.]
Xie, P., and Arkin P. A. , 1995: An intercomparison of gauge observations and satellite estimates of monthly precipitation. J. Appl. Meteor., 34, 1143–1160, doi:10.1175/1520-0450(1995)034<1143:AIOGOA>2.0.CO;2.
Zhou, T., Nijssen B. , Huffman G. J. , and Lettenmaier D. P. , 2014: Evaluation of real-time satellite precipitation data for global drought monitoring. J. Hydrometeor., 15, 1651–1660, doi:10.1175/JHM-D-13-0128.1.
