• Beck, H. E., and Coauthors, 2017: Global-scale evaluation of 22 precipitation datasets using gauge observations and hydrological modeling. Hydrol. Earth Syst. Sci., 21, 62016217, https://doi.org/10.5194/hess-21-6201-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beck, H. E., and Coauthors, 2019: Daily evaluation of 26 precipitation datasets using Stage-IV gauge-radar data for the CONUS. Hydrol. Earth Syst. Sci., 23, 207224, https://doi.org/10.5194/hess-23-207-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, H., D. Lu, Z. Zhou, Z. Zhu, Y. Ren, and B. Yong, 2019a: An overview of the evaluation of satellite precipitation products for Global Precipitation Measurement (GPM) (in Chinese). Water Resour. Prot., 35, 2734.

    • Search Google Scholar
    • Export Citation
  • Chen, H., B. Yong, J. J. Gourly, J. Liu, L. Ren, W. Wang, Y. Hong, and J. Zhang, 2019b: Impact of the crucial geographical and climatic factors on the input source errors of GPM-based global satellite precipitation estimates. J. Hydrol., 575, 116, https://doi.org/10.1016/j.jhydrol.2019.05.020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, H., B. Yong, Y. Shen, J. Liu, Y. Hong, and J. Zhang, 2020: Comparison analysis of six purely satellite-derived global precipitation estimates. J. Hydrol., 581, 124376, https://doi.org/10.1016/j.jhydrol.2019.124376.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, M., W. Shi, P. Xie, V. B. Silva, V. E. Kousky, R. Wayne Higgins, and J. E. Janowiak, 2008: Assessing objective techniques for gauge-based analyses of global daily precipitation. J. Geophys. Res., 113, D04110, https://doi.org/10.1029/2007JD009132.

    • Search Google Scholar
    • Export Citation
  • Deng, P., M. Zhang, H. Guo, C. Xu, J. Bing, and J. Jia, 2018: Error analysis and correction of the daily GSMaP products over Hanjiang River Basin of China. Atmos. Res., 214, 121134, https://doi.org/10.1016/j.atmosres.2018.07.022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hou, A. Y., and Coauthors, 2014: The Global Precipitation Measurement mission. Bull. Amer. Meteor. Soc., 95, 701722, https://doi.org/10.1175/BAMS-D-13-00164.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., and Coauthors, 2007: The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J. Hydrometeor., 8, 3855, https://doi.org/10.1175/JHM560.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., and Coauthors, 2019: NASA Global Precipitation Measurement (GPM) Integrated Multi-satellitE Retrievals for GPM (IMERG). Algorithm Theoretical Basis Doc., version 06, 38 pp., https://pmm.nasa.gov/sites/default/files/document_files/IMERG_ATBD_V06.pdf.

  • Jin, X., H. Shao, C. Zhang, and Y. Yan, 2016: The applicability evaluation of three satellite products in Tianshan Mountains (in Chinese). J. Nat. Resour., 31, 20742085, https://doi.org/10.11849/zrzyxb.20160057.

    • Search Google Scholar
    • Export Citation
  • Joyce, R. J., and P. Xie, 2011: Kalman filter–based CMORPH. J. Hydrometeor., 12, 15471563, https://doi.org/10.1175/JHM-D-11-022.1.

  • Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kidd, C., and G. Huffman, 2011: Global Precipitation Measurement. Meteor. Appl., 18, 334353, https://doi.org/10.1002/met.284.

  • Kidd, C., and V. Levizzani, 2011: Status of satellite precipitation retrievals. Hydrol. Earth Syst. Sci., 15, 11091116, https://doi.org/10.5194/hess-15-1109-2011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krajewski, W. F., G. J. Ciach, J. R. Mccollum, and C. Bacotiu, 2000: Initial validation of the global precipitation climatology project monthly rainfall over the United States. J. Appl. Meteor., 39, 10711086, https://doi.org/10.1175/1520-0450(2000)039<1071:IVOTGP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kubota, T., and Coauthors, 2007: Global precipitation map using satellite-borne microwave radiometers by the GSMaP project: Production and validation. IEEE Trans. Geosci. Remote Sens., 45, 22592275, https://doi.org/10.1109/TGRS.2007.895337.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., 2016: Comparison of Integrated Multi-satellite Retrievals for GPM (IMERG) and TRMM Multisatellite Precipitation Analysis (TMPA) monthly precipitation products: Initial results. J. Hydrometeor., 17, 777790, https://doi.org/10.1175/JHM-D-15-0068.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lu, D., and B. Yong, 2018: Evaluation and hydrological utility of the latest GPM IMERG V5 and GSMaP V7 precipitation products over the Tibetan Plateau. Remote Sens., 10, 2022, https://doi.org/10.3390/rs10122022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lu, X., G. Tang, X. Wang, Y. Liu, L. Jia, G. Xie, and S. Li, 2019: Correcting GPM IMERG precipitation data over the Tianshan mountains in China. J. Hydrol., 575, 12391252, https://doi.org/10.1016/j.jhydrol.2019.06.019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ma, Y., G. Tang, D. Long, B. Yong, L. Zhong, W. Wan, and Y. Hong, 2016: Similarity and error intercomparison of the GPM and its predecessor-TRMM multisatellite precipitation analysis using the best available hourly gauge network over the Tibetan Plateau. Remote Sens., 8, 569, https://doi.org/10.3390/rs8070569.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Maggioni, V., P. C. Meyers, and M. D. Robinson, 2016: A review of merged high-resolution satellite precipitation product accuracy during the Tropical Rainfall Measuring Mission (TRMM) era. J. Hydrometeor., 17, 11011117, https://doi.org/10.1175/JHM-D-15-0190.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McCollum, J. R., and W. F. Krajewski, 1998: Uncertainty of monthly rainfall estimates from rain gauges in the Global Precipitation Climatology Project. Water Resour. Res., 34, 26472654, https://doi.org/10.1029/98WR02173.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Prakash, S., A. Seshadri, J. Srinivasan, and D. S. Pai, 2019: A new parameter to assess impact of rain gauge density on uncertainty in the estimate of monthly rainfall over India. J. Hydrometeor., 20, 821832, https://doi.org/10.1175/JHM-D-18-0161.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ren, Y., B. Yong, D. Lu, and H. Chen, 2019: Evaluation of the Integrated Multi-satellitE Retrievals (IMERG) for Global Precipitation Measurement (GPM) mission over the mainland China at multiples scales (in Chinese). Hupo Kexue, 31, 560572.

    • Search Google Scholar
    • Export Citation
  • Roca, R., P. Chambon, I. Jobard, P. Kirstetter, M. Gosset, and J. C. Berges, 2010: Comparing satellite and surface rainfall products over west Africa at meteorologically relevant scales during the AMMA campaign using error estimates. J. Appl. Meteor. Climatol., 49, 715731, https://doi.org/10.1175/2009JAMC2318.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rudolf, B., H. Hauschild, W. Rueth, and U. Schneider, 1994: Terrestrial precipitation analysis: Operational method and required density of point measurements. Global Precipitation and Climate Change, M. Desbois and F. Desalmand, Eds., Springer, 173186.

    • Crossref
    • Export Citation
  • Shen, Y., P. Zhao, Y. Pan, and J. Yu, 2014: A high spatiotemporal gauge-satellite merged precipitation analysis over China. J. Geophys. Res. Atmos., 119, 30633075, https://doi.org/10.1002/2013JD020686.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sorooshian, S., K. L. Hsu, X. Gao, H. V. Gupta, B. Imam, and D. Braithwaite, 2000: Evaluation of PERSIANN system satellite-based estimates of tropical rainfall. Bull. Amer. Meteor. Soc., 81, 20352046, https://doi.org/10.1175/1520-0477(2000)081<2035:EOPSSE>2.3.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Su, J., H. , Y. Zhu, X. Wang, and G. Wei, 2018: Component analysis of errors in four GPM-based precipitation estimations over Mainland China. Remote Sens., 10, 1420, https://doi.org/10.3390/rs10091420.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tan, J., W. A. Petersen, P. E. Kirstetter, and Y. Tian, 2017: Performance of IMERG as a function of spatiotemporal scale. J. Hydrometeor., 18, 307319, https://doi.org/10.1175/JHM-D-16-0174.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tang, G., A. Behrangi, D. Long, C. Li, and Y. Hong, 2018: Accounting for spatiotemporal errors of gauges: A critical step to evaluate gridded precipitation products. J. Hydrol., 559, 294306, https://doi.org/10.1016/j.jhydrol.2018.02.057.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tian, F., S. Hou, L. Yang, H. Hu, and A. Hou, 2018: How does the evaluation of the GPM IMERG rainfall product depend on gauge density and rainfall intensity? J. Hydrometeor., 19, 339349, https://doi.org/10.1175/JHM-D-17-0161.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tian, Y., and C. D. Peters-Lidard, 2010: A global map of uncertainties in satellite-based precipitation measurements. Geophys. Res. Lett., 37, L24407, https://doi.org/10.1029/2010GL046008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ushio, T., and Coauthors, 2009: A Kalman filter approach to the Global Satellite Mapping of Precipitation (GSMaP) from combined passive microwave and infrared radiometric data. J. Meteor. Soc. Japan, 87, 137151, https://doi.org/10.2151/JMSJ.87A.137.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Villarini, G., and W. F. Krajewski, 2007: Evaluation of the research-version TMPA three-hourly 0.25° × 0.25° rainfall estimates over Oklahoma. Geophys. Res. Lett., 34, L05402, https://doi.org/10.1029/2006GL029147.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Villarini, G., P. V. Mandapaka, W. F. Krajewski, and R. J. Moore, 2008: Rainfall and sampling uncertainties: A rain gauge perspective. J. Geophys. Res., 113, D11102, https://doi.org/10.1029/2007JD009214.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, C., G. Tang, Z. Han, X. Guo, and Y. Hong, 2018: Global intercomparison and regional evaluation of GPM IMERG Version-03, Version-04 and its latest Version-05 precipitation products: Similarity, difference and improvements. J. Hydrol., 564, 342356, https://doi.org/10.1016/j.jhydrol.2018.06.064.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, Z., R. Zhong, C. Lai, and J. C. Chen, 2017: Evaluation of the GPM IMERG satellite-based precipitation products and the hydrological utility. Atmos. Res., 196, 151163, https://doi.org/10.1016/j.atmosres.2017.06.020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wu, Y., L. Guo, H. Zheng, B. Zhang, and M. Li, 2019: Hydroclimate assessment of gridded precipitation products for the Tibetan Plateau. Sci. Total Environ., 660, 15551564, https://doi.org/10.1016/j.scitotenv.2019.01.119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P., and P. A. Arkin, 1997: Global precipitation: A 17-year monthly analysis based on gauge observations, satellite estimates, and numerical model outputs. Bull. Amer. Meteor. Soc., 78, 25392558, https://doi.org/10.1175/1520-0477(1997)078<2539:GPAYMA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P., M. Chen, S. Yang, A. Yatagai, T. Hayasaka, Y. Fukushima, and C. Liu, 2007: A gauge-based analysis of daily precipitation over East Asia. J. Hydrometeor., 8, 607626, https://doi.org/10.1175/JHM583.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P., M. Chen, and W. Shi, 2010: CPC global unified gauge-based analysis of daily precipitation. 24th Conf. on Hydrology, Atlanta, GA, Amer. Meteor. Soc., 2, https://ams.confex.com/ams/90annual/techprogram/paper_163676.htm.

  • Xu, R., F. Tian, L. Yang, H. Hu, H. Lu, and A. Hou, 2017: Ground validation of GPM IMERG and TRMM 3B42V7 rainfall products over southern Tibetan Plateau based on a high-density rain gauge network. J. Geophys. Res. Atmos., 122, 910924, https://doi.org/10.1002/2016JD025418.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xu, S., Y. Shen, and Z. Du, 2016: Tracing the source of the errors in hourly IMERG using a decomposition evaluation scheme. Atmosphere, 7, 161, https://doi.org/10.3390/atmos7120161.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yang, Y., and Y. Luo, 2014: Evaluating the performance of remote sensing precipitation products CMORPH, PERSIANN, and TMPA, in the arid region of northwest China. Theor. Appl. Climatol., 118, 429445, https://doi.org/10.1007/s00704-013-1072-0.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yong, B., L. Ren, Y. Hong, J. H. Wang, J. J. Gourley, S. Jiang, X. Chen, and W. Wang, 2010: Hydrologic evaluation of multisatellite precipitation analysis standard precipitation products in basins beyond its inclined latitude band: A case study in Laohahe basin, China. Water Resour. Res., 46, W07542, https://doi.org/10.1029/2009WR008965.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yong, B., D. Liu, J. J. Gourley, Y. Tian, G. J. Huffman, L. Ren, and Y. Hong, 2015: Global view of real-time TRMM multisatellite precipitation analysis: Implications for its successor global precipitation measurement mission. Bull. Amer. Meteor. Soc., 96, 283296, https://doi.org/10.1175/BAMS-D-14-00017.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yong, B., B. Chen, Y. Tian, Z. Yu, and Y. Hong, 2016: Error-component analysis of TRMM-based multi-satellite precipitation estimates over mainland China. Remote Sens., 8, 440, https://doi.org/10.3390/rs8050440.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhao, H., B. Yang, S. Yang, Y. Huang, G. Dong, J. Bai, and Z. Wang, 2018: Systematical estimation of GPM-based global satellite mapping of precipitation products over China. Atmos. Res., 201, 206217, https://doi.org/10.1016/j.atmosres.2017.11.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Spatial distribution of the rain gauges used in two ground-based precipitation products over mainland China at 0.5° spatial resolution: (a) CPCU and (b) MPA.

  • View in gallery

    Spatial distribution of (a)–(d) POD and (e)–(h) FAR for IMERG-Late and GSMaP-MVK using MPA and using CPCU as references over mainland China at 0.5° spatial resolution.

  • View in gallery

    As in Fig. 2, but for (a)–(d) RBIAS and (e)–(h) RMSE.

  • View in gallery

    The variations of POD values for IMERG-Late and GSMaP-MVK with precipitation intensity using MPA and CPCU as references at 0.5° spatial resolution. Note that the precipitation intensity categories are from the reference.

  • View in gallery

    As in Fig. 4, but for (a) RBIAS and (b) NRMSE.

  • View in gallery

    The variations of POD values for IMERG-Late and GSMaP-MVK with rain gauge density in different precipitation intensities at 0.5° spatial resolution: (a) IMERG-Late and (b) GSMaP-MVK. Note that the precipitation intensity categories are from the reference.

  • View in gallery

    As in Fig. 6, but for (a),(b) RBIAS and (c),(d) NRMSE.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 31 31 31
PDF Downloads 34 34 34

Investigating the Evaluation Uncertainty for Satellite Precipitation Estimates Based on Two Different Ground Precipitation Observation Products

View More View Less
  • 1 State Key Laboratory of Hydrology-Water Resources and Hydraulic Engineering, Hohai University, Nanjing, China
  • | 2 School of Earth Sciences and Engineering, Hohai University, Nanjing, China
  • | 3 Key Laboratory for Digital Land and Resources of Jiangxi Province, East China University of Technology, Nanchang, China
  • | 4 School of Civil Engineering and Environment Sciences, University of Oklahoma, Norman, Oklahoma
Free access

Abstract

The evaluation uncertainty caused by a standard reference itself is harmful to both algorithm developers and data users in substantially understanding the error features and the performance of satellite precipitation products (SPPs). In this study, the Climate Precipitation Center Unified (CPCU) data and the Merged Precipitation Analysis (MPA) data are used as the benchmark to investigate the evaluation uncertainties of satellite precipitation estimates generated by the reference itself. Two SPPs, IMERG-Late and GSMaP-MVK, are employed here. The results show that the approach using two different ground-based precipitation products as the references can effectively reveal the potential evaluation uncertainties. Interestingly, it is found that the evaluation results are prone to resulting in larger uncertainties over semihumid areas. Furthermore, evaluation uncertainty of statistical metrics is closely related to rainfall intensity in that it has a gradually decreasing tendency with increasing rainfall intensities. Additionally, we also found that the dependency of the false alarm ratio (FAR) and root-mean-square error (RMSE) scores on the spatial density of rain gauges is relatively low. Both relative bias (RBIAS) and normalized root-mean-square error (NRMSE) scores for light precipitation (1–5 mm day−1) increase with the spatial density of the rain gauges, suggesting that the evaluation of light precipitation can easily cause uncertainties relative to medium-to-high rain rates. Finally, the minimum gauge density required for different scores and different rainfall intensities is discussed. This study is expected to provide criteria to investigate the reliability of evaluation results for the satellite quantitative precipitation estimation community.

© 2020 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Bin Yong, yongbin@hhu.edu.cn

Abstract

The evaluation uncertainty caused by a standard reference itself is harmful to both algorithm developers and data users in substantially understanding the error features and the performance of satellite precipitation products (SPPs). In this study, the Climate Precipitation Center Unified (CPCU) data and the Merged Precipitation Analysis (MPA) data are used as the benchmark to investigate the evaluation uncertainties of satellite precipitation estimates generated by the reference itself. Two SPPs, IMERG-Late and GSMaP-MVK, are employed here. The results show that the approach using two different ground-based precipitation products as the references can effectively reveal the potential evaluation uncertainties. Interestingly, it is found that the evaluation results are prone to resulting in larger uncertainties over semihumid areas. Furthermore, evaluation uncertainty of statistical metrics is closely related to rainfall intensity in that it has a gradually decreasing tendency with increasing rainfall intensities. Additionally, we also found that the dependency of the false alarm ratio (FAR) and root-mean-square error (RMSE) scores on the spatial density of rain gauges is relatively low. Both relative bias (RBIAS) and normalized root-mean-square error (NRMSE) scores for light precipitation (1–5 mm day−1) increase with the spatial density of the rain gauges, suggesting that the evaluation of light precipitation can easily cause uncertainties relative to medium-to-high rain rates. Finally, the minimum gauge density required for different scores and different rainfall intensities is discussed. This study is expected to provide criteria to investigate the reliability of evaluation results for the satellite quantitative precipitation estimation community.

© 2020 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Bin Yong, yongbin@hhu.edu.cn

1. Introduction

Accurate estimation of precipitation is essential for climate analysis, hydrological simulation, drought monitoring, flood forecasting, landslide warning, and related emergency management (Kidd and Levizzani 2011; Maggioni et al. 2016). At present, high-quality precipitation estimation mainly depends on rain gauge networks and ground-based radars although satellite technology and satellite-based retrievals have made great progress in recent years. The widely used satellite precipitation products (SPPs), including the TRMM Multisatellite Precipitation Analysis (TMPA; Huffman et al. 2007), the Global Satellite Mapping of Precipitation (GSMaP; Kubota et al. 2007), the Climate Prediction Center morphing technique (CMORPH; Joyce et al. 2004; Joyce and Xie 2011), the Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN; Sorooshian et al. 2000), and the Integrated Multisatellite Retrievals for GPM (IMERG; Huffman et al. 2019), have been released to the public. However, these SPPs are subject to a larger number of random errors and systematic errors, which inevitably limits their potential applications (Chen et al. 2020).

The error evaluation is an effective method to reveal the actual performance and error features for SPPs, which is helpful for algorithm developers to further improve the satellite-based retrievals and for end users to guide their applications (Hou et al. 2014; Maggioni et al. 2016; Chen et al. 2019a). Currently, a considerable number of evaluation studies have been performed at global (e.g., Tian and Peters-Lidard 2010; Yong et al. 2015; Liu 2016; Beck et al. 2017; Wang et al. 2018; Chen et al. 2020), regional (e.g., Yang and Luo 2014; Yong et al. 2010, 2016; Tan et al. 2017; Beck et al. 2019; Ren et al. 2019; Wu et al. 2019; Chen et al. 2019b) and local scales (e.g., Xu et al. 2017; Deng et al. 2018; Lu and Yong 2018). With respect to mainland China, there are many related studies to investigate the spatiotemporal performance of SPPs from different perspectives (e.g., Yang and Luo 2014; Jin et al. 2016; Ma et al. 2016; Xu et al. 2017; Wang et al. 2017; Zhao et al. 2018; Su et al. 2018; Lu and Yong 2018; Wu et al. 2019; Chen et al. 2019b, 2020). Specifically, some of them evaluated the performance of SPPs over those areas with complex topography, such as Tibetan Plateau (e.g., Ma et al. 2016; Xu et al. 2017; Lu and Yong 2018; Wu et al. 2019) and Tian Shan (e.g., Yang and Luo 2014; Jin et al. 2016; Lu et al. 2019). Several studies executed the evaluation by analyzing the components of error (e.g., Xu et al. 2016; Yong et al. 2016; Su et al. 2018; Chen et al. 2020). However, most studies only selected a sole ground precipitation product as a reference to reveal the actual performance and error characteristics for SPPs. Meanwhile, these studies preliminarily assumed that the benchmark used is reliable and the evaluation results reflect the actual performance of SPPs. In practice, the uncertainties caused by the reference itself might have imperceptibly existed in the evaluation results.

Prakash et al. (2019) indicated that the uncertainties of rain gauge measurements increase with a decrease in the number of rainfall gauges, suggesting that the precipitation observations from a sparse rain gauge network might have larger uncertainties. Several previous papers (e.g., Rudolf et al. 1994; Xie and Arkin 1997; McCollum and Krajewski 1998; Krajewski et al. 2000; Villarini and Krajewski 2007; Villarini et al. 2008; Roca et al. 2010) investigated the possible link between the number of rain gauges and the reliability of observation precipitation. Furthermore, several studies (e.g., Tian et al. 2018; Tang et al. 2018) attempted to use different approaches to investigate the evaluation uncertainties and reveal the actual performance for SPPs. They found that the larger uncertainties normally correspond to relatively sparser rain gauge networks. However, these studies cannot identify the regions where the assessment results of SPPs are prone to producing uncertainties. Also, the study aiming at the relationship between evaluation uncertainty and rain gauge density for different precipitation intensities is still lacking. These limitations, which existed in previous studies, are unfavorable for obtaining reliable assessment results.

In this study, we used two types of ground-based precipitation products, i.e., the Climate Precipitation Center Unified (CPCU; Xie et al. 2007, 2010; Chen et al. 2008) and the Merged Precipitation Analysis (MPA; Shen et al. 2014), as the standard benchmark to demonstrate the evaluation uncertainties caused by the reference itself. The performances of two schemes were intercompared and investigated in detail. The main efforts focused on answering the following five questions:

  1. Can evaluation uncertainties caused by the reference itself be revealed by using a performance comparison between using two different ground-based precipitation products (GPPs) as references?

  2. What is the potential link between evaluation uncertainty and rainfall intensity?

  3. Where does the evaluation tend to generate larger uncertainties, especially when using a relatively sparse rain gauge network to assess the performance of SPPs?

  4. Which range of rainfall intensity is prone to generating uncertainties caused by the benchmark itself?

  5. What is the minimum density of rain gauges to consider for different scores and different rainfall intensities?

This paper is structured as follows: section 2 describes the study area, datasets, and used methods. Detailed results are presented in section 3. Discussion follows in section 4. Section 5 summarizes the conclusions.

2. Study area, datasets, and methodology

a. Study area

Our study area is mainland China with complex topography, which primarily includes four types of climate regions (i.e., humid, semihumid, semiarid, and arid; refer to Chen et al. 2019b). Humid regions (area marked as 1 in Fig. 1a), dominated by the subtropical monsoon, have an average annual precipitation (AAP) exceeding 800 mm yr−1. In contrast, arid (area marked as 4 in Fig. 1a) and semiarid areas (area marked as 3 in Fig. 1a) have lower APP, with less than 200 mm yr−1 and 200–400 mm yr−1, respectively. Besides, they have complex topography, such as the Tian Shan and the Tibetan Plateau. While semihumid (area marked as 2 in Fig. 1a) is a typical climate transition zone, with an AAP of 400–800 mm yr−1.

Fig. 1.
Fig. 1.

Spatial distribution of the rain gauges used in two ground-based precipitation products over mainland China at 0.5° spatial resolution: (a) CPCU and (b) MPA.

Citation: Journal of Hydrometeorology 21, 11; 10.1175/JHM-D-20-0103.1

b. Datasets

1) Satellite precipitation products

In our study, two satellite-only precipitation products, i.e., IMERG-Late V6 and GSMaP-MVK V7, are employed. One reason is that IMERG-Late and GSMaP-MVK are the most accurate satellite-only versions of IMERG and GSMaP, respectively (Chen et al. 2020). Besides, these satellite-only precipitation products have not been corrected by ground-based observations relative to ground-adjusted satellite precipitation products (e.g., IMERG-Final and GSMaP-Gauge), avoiding the potential uncertainties caused by the overlap between evaluated SPPs and references. Both IMERG-Late and GSMaP-MVK merged several passive microwave (PMW) and infrared (IR) data for getting high-accuracy global precipitation estimates (Hou et al. 2014; Tan et al. 2017). The detailed description of the two products can be found in our previous study (see section 2.2.1 in Chen et al. 2020). The detailed information of the two satellite-only precipitation products and two ground-based precipitation products are listed in Table 1.

Table 1.

The detailed information of the two satellite precipitation products (i.e., IMERG-Late and GSMaP-MVK) and two ground-based precipitation products (i.e., MPA and CPCU).

Table 1.

2) Ground-based precipitation products

Two ground-based precipitation products with different rain gauge densities are used as the references in our study, i.e., CPCU and MPA.

CPCU, developed by the National Oceanic and Atmospheric Administration (NOAA) Climate Prediction Center, is regarded as one of the references, which combines ground-based rain gauge observations from around 400 gauges over mainland China. The optimal interpolation is applied to produce a real-time gauge-based precipitation product (Xie et al. 2007, 2010; Chen et al. 2008). However, this real-time gauge-based precipitation product exhibits poor performance in areas with sparse rain gauge networks (Chen et al. 2008). The spatial distribution of the rain gauges used in the CPCU system over mainland China is shown in Fig. 1a. In the grid cells with rain gauges in the CPCU system, 98% of these grid cells have only one rain gauge.

MPA is another reference. Note that MPA is a gauge–satellite merged precipitation product, which was developed by the National Meteorological Information Center of the China Meteorological Administration. It combines gauge-based observations from over 30 000 automatic weather stations (AWS) with a satellite-only version of CMORPH (Shen et al. 2014). The spatial distribution of rain gaugess used in the MPA system over mainland China is represented in Fig. 1b. It can be seen that rain gauges used in MPA are mainly concentrated in the humid and semihumid areas. Thus, in spatial analysis, a comparison of the performance between using MPA and using CPCU as references is mainly focused on the humid and semihumid regions. In addition, the spatial density of rain gauges for these two ground precipitation datasets in mainland China differs greatly, which is helpful for identifying the areas where the results are prone to producing larger uncertainties due to the benchmark itself using spatial analysis. Finally, CPCU is a subset of MPA over mainland China, but the number of rain gauges used in MPA is close to 100 times that used in CPCU. Despite that, the results of using different benchmarks may show similarity in the areas where the number of rain gauges used in these two references is similar.

c. Methodology

We need to preprocess the satellite data (i.e., IMERG-Late and GSMaP-MVK) and gauge-satellite merged MPA data before calculating the evaluation scores. First, we used the spatial average method to resample the satellite data into a 0.5° spatial resolution. As for MPA data, we only selected the AWS observations and resampled them into a 0.5° spatial resolution. This method is that the mean value of all AWS’s precipitation for per 0.5° grid cell is paired with that from the grid cell. For example, for 10 rain gauges per grid cell, the mean value of these 10 rain gauges’ precipitation is paired with that from the grid cell. Then, the resampled data were further aggregated into a daily temporal and 0.5° spatial resolution. It can be seen that only the AWS data of MPA was used in both spatial and temporal resampling processes. Consequently, the reprocessed MPA data completely excludes the satellite information, avoiding the possible uncertainties caused by the overlap between MPA data and evaluated satellite data.

The comparisons were executed at a daily temporal and 0.5° spatial resolution by computing, for each grid cell, probability of detection (POD), false alarm ratio (FAR), relative bias (RBIAS), root-mean-square error (RMSE), and normalized root-mean-square error (NRMSE) from January 2015 to September 2019, based on different GPPs as the benchmark. The contingency statistics, including POD and FAR, were used to examine the differences of detection capability using different GPPs as the references. As for RBIAS, RMSE, and NRMSE, they were used to compare the differences of accuracy using different GPPs as the benchmark. RMSE reflects the absolute values of errors. However, RMSE values increase with precipitation intensities (Chen et al. 2020). Thus, we used NRMSE as a supplemental metric because NRMSE avoided increasing values with precipitation intensity compared with RMSE. The detailed calculation formulas of these evaluation metrics are given in Table 2.

Table 2.

The metrics used in the evaluation and their calculated formulas. Notation: H expresses the counts of hit rainfall events for satellite precipitation estimates; M denotes the number of miss rainfall events for satellite precipitation estimates; F indicate the number of false rainfall events for satellite precipitation estimates; G expresses precipitation observations from the gauge; S is precipitation estimates from the satellite; and n is the number of sample sizes for satellite precipitation estimates.

Table 2.

Finally, the analysis is executed by using a comparison of the evaluation results between using reprocessed MPA data and using CPCU data as the references to reveal the areas where the evaluation results are prone to generating larger uncertainties. Then, the relationship between the results (differences) of each metric using CPCU and using MPA as the benchmark and precipitation intensity is utilized to study the potential link between evaluation uncertainty and rainfall intensity. Besides, the evaluation metrics (including POD, RBIAS, and NRMSE) as a function of the gauge density are used to identify the rainfall intensities that are prone to producing larger evaluation uncertainties. We executed the comparisons at a coarse spatial resolution (0.5°); the spatial average may lead to the smoothing of errors for SPPs. Nevertheless, this has little effect on our conclusions because these conclusions are mainly based on the performance comparisons between using two GPPs as the references but not between different SPPs. In addition, it should be noted that only the pixels with at least one CPCU rain gauge stations are considered in this study, and we used 1 mm day−1 as a threshold to judge rainfall/no rainfall.

3. Results

a. General analysis

Table 3 shows the summary of four evaluation metrics for IMERG-Late and GSMaP-MVK over the whole mainland China using two different ground-based products (i.e., MPA and CPCU) as references. In terms of contingency statistics, the values of POD that used MPA as a reference are higher than those that used CPCU as a reference, while all FARs are almost identical in value sizes. Consistent with the POD results, the RBIAS values are prone to being higher when using MPA data as a reference. This result reports IMERG-Late and GSMaP-NRT overestimate the precipitation compared to MPA. Finally, the summary values of RMSE for IMERG-Late and GSMaP-MVK using these two GPPs as the references are presented. On the basis of same benchmark, the RMSE values of IMERG-Late are all lower than those of GSMaP-MVK, indicating that the accuracy of IMERG-Late outperforms that of GSMaP-MVK. Overall, the results of using different GPPs as the references show apparent differences. For example, the RBIAS value for IMERG-Late using MPA as the benchmark is 20.45%, however, its RBIAS is a lower value of 6.92% when using CPCU as the reference. These can be due to the differences between these two references used. It means that a comparison of the performance between using two different GPPs as the references can effectively find out the potential evaluation uncertainty issues due to the benchmark itself. This conclusion implies that the evaluation results are unreliable when using a relatively sparse rain gauge density network to assess the performance of SPPs.

Table 3.

Summary of four evaluation metrics (i.e., POD, FAR, RBIAS, and RMSE) for IMERG-Late and GSMaP-MVK over the whole mainland China at 0.5° spatial resolution using MPA and using CPCU as references.

Table 3.

b. Spatial analysis

Figure 2 represents the spatial distribution of contingency statistics (i.e., POD and FAR) for IMERG-Late and GSMaP-MVK using MPA and using CPCU as the benchmark. There are obvious differences in the spatial distribution of POD for each SPP using different GPPs as references. For example, the POD values of using MPA as a reference are significantly higher than those of using CPCU as the benchmark over most semihumid regions, especially over the areas with denser rain gauge networks used in MPA system. It can be concluded that the evaluation uncertainties caused by the benchmark itself are prone to appearing in semihumid regions compared to humid areas. The cause may be that semihumid is a typical climate transition zone with strong spatiotemporal heterogeneity of precipitation and has complex terrain over most areas, which enables locally sparse ground observations having uncertainties.

Fig. 2.
Fig. 2.

Spatial distribution of (a)–(d) POD and (e)–(h) FAR for IMERG-Late and GSMaP-MVK using MPA and using CPCU as references over mainland China at 0.5° spatial resolution.

Citation: Journal of Hydrometeorology 21, 11; 10.1175/JHM-D-20-0103.1

The spatial distribution of RBIAS and RMSE for IMERG-Late and GSMaP-MVK using MPA and using CPCU as the references is shown in Fig. 3. The positive and negative values of RBIAS for IMERG-Late and GSMaP-MVK are scattered over the humid areas of mainland China (see Figs. 3a–d). However, these two SPPs overestimate the precipitation over most residual regions of mainland China. In particular, GSMaP-MVK obviously overestimates the precipitation with the RBIAS values exceeding 80% over ones. Like the differences of spatial distribution of POD for IMERG-Late/GSMaP-MVK between using MPA and using CPCU as references over semihumid regions (see Fig. 2), the similar differences between using MPA and using CPCU as the benchmark also exist in the spatial distribution of RBIAS for each SPP. This is further confirmed a performance comparison between using two GPPs with different rain gauge densities as the references can effectively identify the regions where the evaluation results are prone to generating larger uncertainties. On the basis of our findings, it is suggested that more rain gauge observations should be used to evaluate the performance of SPPs over regions with strong spatiotemporal heterogeneity of precipitation and complex terrain, avoiding the potential evaluation uncertainties due to the reference having uncertainties.

Fig. 3.
Fig. 3.

As in Fig. 2, but for (a)–(d) RBIAS and (e)–(h) RMSE.

Citation: Journal of Hydrometeorology 21, 11; 10.1175/JHM-D-20-0103.1

On the other hand, for each SPP, RMSE values calculated by using MPA as the benchmark are highly consistent with those calculated by using CPCU as the reference over most regions of mainland China. This is because RMSE value mainly relies on precipitation intensity. The higher RMSE values (>6.5 mm) of IMERG-Late are mainly distributed in the southeast of humid regions, while those of GSMaP-MVK are primarily distributed in most humid and semihumid regions. Overall, IMERG-Late shows a relatively high accuracy relative to GSMaP-MVK. The similar results were found in previous studies (e.g., Chen et al. 2020).

c. The relationship between evaluation uncertainty and precipitation intensity

Figure 4 represents the variations of POD values for IMERG-Late and GSMaP-MVK with precipitation intensity using MPA and using CPCU as references. It is seen that some similar features exist between IMERG-Late and GSMaP-MVK. One is that the PODs of each SPP using different GPPs as references show high similarity. Another is that their detection capability increases with precipitation intensities. It means that the satellite sensors tend to capture the heavy rainfall events but are relatively insensitive for light precipitation. Evidently, the differences of POD using different GPPs as the benchmark do not remarkably change with increasing precipitation intensities, indicating that the evaluation uncertainty of contingency statistics is not strongly related to rainfall intensity.

Fig. 4.
Fig. 4.

The variations of POD values for IMERG-Late and GSMaP-MVK with precipitation intensity using MPA and CPCU as references at 0.5° spatial resolution. Note that the precipitation intensity categories are from the reference.

Citation: Journal of Hydrometeorology 21, 11; 10.1175/JHM-D-20-0103.1

The variations of RBIAS and NRMSE values for IMERG-Late and GSMaP-MVK with precipitation intensities using MPA and using CPCU as the benchmark are shown in Fig. 5 for further studying the relationship between evaluation uncertainty and precipitation intensity. Both RBIAS and NRMSE have an apparent decreasing tendency with increasing precipitation intensities in all four cases. Contrary to the variations of the difference for POD, the differences of RBIAS/NRMSE using different GPPs as the references decrease quickly with increasing rainfall intensities, which implies that the evaluation uncertainties of accuracy scores are associated with precipitation intensity and uncertainties decrease with increasing precipitation intensities.

Fig. 5.
Fig. 5.

As in Fig. 4, but for (a) RBIAS and (b) NRMSE.

Citation: Journal of Hydrometeorology 21, 11; 10.1175/JHM-D-20-0103.1

d. The relationship between evaluation uncertainty and rain gauge density for different precipitation intensities

In this section, we further investigate the relationship between the evaluation uncertainty and rain gauge density for different precipitation intensities. Besides, the rainfall intensities that are more prone to producing larger evaluation uncertainties are revealed. Note that we only used the MPA data as the benchmark in this section because CPCU has only one rain gauge within 98% of evaluated grid cells over mainland China.

Figure 6 shows the variations of POD for IMERG-Late and GSMaP-MVK with rain gauge density for different precipitation intensities. We found that the PODs of SPPs for each rainfall intensity are all stable near a certain value in all rain gauge densities. Evidently, the detection capability of SPPs is related to rainfall intensity and increases with increasing precipitation intensities. This result is consistent with the that of Fig. 4.

Fig. 6.
Fig. 6.

The variations of POD values for IMERG-Late and GSMaP-MVK with rain gauge density in different precipitation intensities at 0.5° spatial resolution: (a) IMERG-Late and (b) GSMaP-MVK. Note that the precipitation intensity categories are from the reference.

Citation: Journal of Hydrometeorology 21, 11; 10.1175/JHM-D-20-0103.1

Figure 7 represents the evaluation results of the two accuracy metrics (i.e., RBIAS and NRMSE) as a function of the rain gauge density for six different precipitation intensities. The RBIAS values of these light rainfall events (1–5 mm day−1) generally increase with rain gauge densities, suggesting that the performance of SPPs might be overestimated in light rainfall events in previous studies when using a relatively sparse rain gauge network to evaluate the performance of SPPs. In addition, SPPs tend to overestimate the precipitation in the light rainfall events, with the RBIAS values over 10%. On the other hand, the rainfall events (>10 mm day−1) are more likely to be accurately retrieved by satellite sensors relative to other rainfall events. This is because their RBIAS values approach to 0 in all rain gauge densities.

Fig. 7.
Fig. 7.

As in Fig. 6, but for (a),(b) RBIAS and (c),(d) NRMSE.

Citation: Journal of Hydrometeorology 21, 11; 10.1175/JHM-D-20-0103.1

For the evaluation results of NRMSE metric it is seen that the NRMSE values of both IMERG-Late and GSMaP-MVK are stable near a certain value in the rainfall events (>10 mm day−1). Conversely, they increase with increasing rain gauge densities in the light rainfall events (<5 mm day−1), especially for light rainfall events in the range of 1–2 mm day−1.

4. Discussion

a. The minimum rain gauge density to consider for different scores and different precipitation intensities

Determining the minimum rain gauge density for different scores and different rainfall intensities is crucial for guiding future evaluation studies. Existing studies (e.g., Rudolf et al. 1994; Xie and Arkin 1997; McCollum and Krajewski 1998; Krajewski et al. 2000; Villarini and Krajewski 2007) investigated the possible link between the number of rain gauges and the reliability of the observations from rain gauges at different spatiotemporal resolutions [i.e., 2.5° spatial and monthly temporal solution for Rudolf et al. (1994), McCollum and Krajewski (1998), Xie and Arkin (2009), and Krajewski et al. (2000); 0.25° spatial and 3-h temporal solution for Villarini and Krajewski (2007)]. Nevertheless, specific indicators in terms of the minimum rain gauge density for different evaluation metrics and different rainfall intensities are not to be solved.

One can see that the spatial distribution of FAR (or RMSE) for using different GPPs as the benchmark is highly similar to one another, shown in Figs. 2e–h and 3e–h. This means that the results of these two scores including FAR and RMSE do not strongly rely on dense rain gauge density. Thus, it can be concluded that a reasonable spatial distribution of RMSE and FAR for SPP over mainland China can be obtained by using a rain gauge network with only one rain gauge within all grid pixels as a reference. For POD metric, it seems that the density of the rain gauge has limited influence on this metric for all rainfall intensities (see Fig. 6). However, the reliability of POD values relies on the density of rain gauges over semihumid areas (see Figs. 2a–d). Also, the values of NRMSE metric are relatively stable in the rainfall events (>10 mm day−1) in all rain gauge densities (see Fig. 7). These results indicate that using the benchmark with five rain gauges within all grid cells could be given a reliable result for the POD metric at all rainfall intensities and for the NRMSE score in the rainfall events (>10 mm day−1). In addition, the rain gauge density of 15 is the minimum density of the rain gauges for the RBIAS score in the rainfall events (>40 mm day−1). However, the results of the accuracy metrics have significant dependency on the density of the rain gauges in the light rainfall events (1–5 mm day−1). Due to the limited rain gauge density used in this study, the minimum density of the rain gauges for light precipitation events (1–5 mm day−1) is still not clear and needs to be resolved using a much denser rain gauge network in future studies.

b. Current performance evaluation for SPP in light rainfall events and its future improvement

Chen et al. (2020) found that SPPs overestimate the proportions of light rainfall events and show larger errors in ones. Our results further reveal that SPPs perform worse in the light rainfall events, with larger RBIAS values (>10%) and NRMSE values (>1.5), meaning that the measurement of light rainfall events is still one of conundrums in the satellite precipitation field. Although DPR data and GMI data were used to improve the retrievals of the light rainfall, the difficulty of retrieving light precipitation over land in IMERG and GSMaP stems from the PMW sensors, which generally rely on the scattering signal from frozen hydrometeors. Since light precipitation may come from shallow (warm) precipitation, the absence of frozen hydrometeors poses challenges to this retrieval approach (Kidd and Huffman 2011; Kidd and Levizzani 2011).

Furthermore, our results confirmed that the assessment results in light rainfall events are more prone to producing larger uncertainties compared to other rainfall events, due to their RBIAS values and NRMSE values increasing with the rain gauge densities. On the other hand, it can be seen that even if the rain gauge density exceeds 25, their values of RBIAS and NRMSE are still increasing. Additionally, the lower rain gauge density, the lower the RBIAS values and the NRMSE values (see Fig. 7). It suggests that gauge networks that are not dense enough may miss precipitation that is light when averaged over the grid box, which can be due to local, isolated showers. If the satellite misses such precipitation as well, this will lead to an artificially lower NRMSE because both the satellite and reference miss the light precipitation. Therefore, using as many rain gauge observations as possible to obtain reliable evaluation results of SPPs in light precipitation is necessary in the future studies.

c. The impact of the study area on the results and the potential for these results extrapolation to elsewhere

The rainfall observations from rain gauges are impacted by the study area to some extent. In the arid and semiarid regions with a similar spatial density of rain gauges for MPA and CPCU, it can be seen that the results of each score based on different benchmarks are almost consistent with one another (see Figs. 2 and 3), which overall smooths the differences between the results of using different references. In addition, the evaluation results are prone to generating larger uncertainties over semihumid areas where the precipitation has an obvious feature of strong spatiotemporal heterogeneity and the topography is complex. This is because the quality of precipitation observations from rain gauges in such areas primarily depends on the high density of the rain gauge network. Consequently, the evaluation results tend to have larger uncertainties over such areas when using the observations from a relatively sparse rain gauge network as the benchmark.

One of the criteria for judging the significance of the work is the transferability of the research results to other areas of the world and whether the results can guide future evaluation studies. In this study, the results considering different regional characteristics and rainfall intensities overall could be extrapolated to other similar regions, especially for the minimum rain gauge density for RMSE and FAR scores. Nevertheless, this statement needs to be verified in future studies. Moreover, the relationship between scores and rain gauge density for six rainfall intensities shown in this study could guide the performance evaluation for SPPs.

5. Conclusions

This study aims to investigate the potential evaluation uncertainties of satellite precipitation estimates by adopting two different ground-based precipitation products (i.e., MPA and CPCU) as the references. Based on our analysis results, we answered five questions concerned by both data users and algorithm developers. The major conclusions of this study are summarized as follows:

  1. Generally, our analyses show that almost all the evaluation indices calculated by using different reference precipitation products as references exhibit significant differences, implying that the evaluation uncertainties from the reference itself can be effectively identified by the approach proposed here. On the other hand, this also suggests that the evaluation results in many previous studies, which selected the sparse ground observations as the benchmark, are likely to be unreliable.

  2. In terms of spatial patterns, the values of each evaluation metric (including POD and RBIAS) show significant differences over most semihumid areas when using different ground-based precipitation products as the benchmark. This implies that evaluation results are prone to producing uncertainties over semihumid areas with strong precipitation heterogeneity and complex terrain condition, especially when using a relatively sparse rain gauge network (only one rain gauge in all grid cells) as a standard benchmark. Therefore, we suggest that one should use as many rain gauges as possible to alleviate the uncertainties of evaluation results in these special areas.

  3. The results of statistical metrics indicate that the evaluation results strongly depend on precipitation intensity and rain gauge density. Among them, the differences of POD using different GPPs as the benchmark have no obvious change with the increasing rainfall intensities, yet the values of RBIAS/NRMSE using different GPPs as the references quickly close with increasing precipitation intensity. This suggested that the evaluation uncertainty of accuracy scores is closely related to rainfall intensity and uncertainties decrease with increasing rainfall intensities. Additionally, the results of NRMSE are relatively stable in the rainfall events (>10 mm day−1) in all rain gauge densities, indicating that the impact of rain gauge density on this score in such rainfall events is relatively low.

  4. The values of the two evaluation metrics (i.e., RBIAS and NRMSE) for IMERG-Late and GSMaP-MVK increase with the decrease in rain gauge densities in the light rainfall events (1–5 mm day−1), meaning that the evaluation results of previous studies based on a relatively sparse rain gauge network might overestimate the performance of SPPs in the light rainfall events. In practice, the performance of SPPs is still worse in light precipitation despite that some advanced instruments (i.e., DPR and GMI) and iterative updating that improved retrieval algorithms have been used to improve the retrievals for light precipitation. Additionally, the evaluation results of SPP in the light rainfall events are prone to producing larger uncertainties relative to medium-to-high rain rates. Although the approach proposed in this study cannot identify the minimum rain gauge density for light rainfall events due to limited density of the rain gauges, we considered that the satellite precipitation estimation community might experience similar issues as well. We expect that the results reported in this paper will provide a better understanding of evaluation uncertainties of satellite precipitation estimates for both algorithm developers and data users.

Acknowledgments

We are grateful to the satellite precipitation products developers and ground-based precipitation products providers. In addition, the IMERG-Late data can be obtained from https://pmm.nasa.gov/data-access/downloads/gpm; the GSMaP-MVK data are obtained from ftp://rainmap:Niskur+1404@hokusai.eorc.jaxa.jp/; the MPA data can be downloaded from http://data.cma.cn/data/cdcdetail/dataCode/SEVP_CLI_CHN_MERGE_CMP_PRE_HOUR_GRID_0.10.html; the CPCU data can be downloaded from ftp://ftp.cpc.ncep.noaa.gov/precip/. This work was supported by National Key Research and Development Program of China (2018YFA0605402) and National Natural Science Foundation of China (51979073). In addition, this work is partially supported by the Fundamental Research Funds for the Central Universities (B200204029) and the Key Laboratory for Digital Land and Resources of Jiangxi Province, East China University of Technology (DLLJ201907). The authors thank the chief editor and two anonymous reviewers, who helped to improve the earlier version of this paper.

REFERENCES

  • Beck, H. E., and Coauthors, 2017: Global-scale evaluation of 22 precipitation datasets using gauge observations and hydrological modeling. Hydrol. Earth Syst. Sci., 21, 62016217, https://doi.org/10.5194/hess-21-6201-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beck, H. E., and Coauthors, 2019: Daily evaluation of 26 precipitation datasets using Stage-IV gauge-radar data for the CONUS. Hydrol. Earth Syst. Sci., 23, 207224, https://doi.org/10.5194/hess-23-207-2019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, H., D. Lu, Z. Zhou, Z. Zhu, Y. Ren, and B. Yong, 2019a: An overview of the evaluation of satellite precipitation products for Global Precipitation Measurement (GPM) (in Chinese). Water Resour. Prot., 35, 2734.

    • Search Google Scholar
    • Export Citation
  • Chen, H., B. Yong, J. J. Gourly, J. Liu, L. Ren, W. Wang, Y. Hong, and J. Zhang, 2019b: Impact of the crucial geographical and climatic factors on the input source errors of GPM-based global satellite precipitation estimates. J. Hydrol., 575, 116, https://doi.org/10.1016/j.jhydrol.2019.05.020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, H., B. Yong, Y. Shen, J. Liu, Y. Hong, and J. Zhang, 2020: Comparison analysis of six purely satellite-derived global precipitation estimates. J. Hydrol., 581, 124376, https://doi.org/10.1016/j.jhydrol.2019.124376.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, M., W. Shi, P. Xie, V. B. Silva, V. E. Kousky, R. Wayne Higgins, and J. E. Janowiak, 2008: Assessing objective techniques for gauge-based analyses of global daily precipitation. J. Geophys. Res., 113, D04110, https://doi.org/10.1029/2007JD009132.

    • Search Google Scholar
    • Export Citation
  • Deng, P., M. Zhang, H. Guo, C. Xu, J. Bing, and J. Jia, 2018: Error analysis and correction of the daily GSMaP products over Hanjiang River Basin of China. Atmos. Res., 214, 121134, https://doi.org/10.1016/j.atmosres.2018.07.022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hou, A. Y., and Coauthors, 2014: The Global Precipitation Measurement mission. Bull. Amer. Meteor. Soc., 95, 701722, https://doi.org/10.1175/BAMS-D-13-00164.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., and Coauthors, 2007: The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J. Hydrometeor., 8, 3855, https://doi.org/10.1175/JHM560.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., and Coauthors, 2019: NASA Global Precipitation Measurement (GPM) Integrated Multi-satellitE Retrievals for GPM (IMERG). Algorithm Theoretical Basis Doc., version 06, 38 pp., https://pmm.nasa.gov/sites/default/files/document_files/IMERG_ATBD_V06.pdf.

  • Jin, X., H. Shao, C. Zhang, and Y. Yan, 2016: The applicability evaluation of three satellite products in Tianshan Mountains (in Chinese). J. Nat. Resour., 31, 20742085, https://doi.org/10.11849/zrzyxb.20160057.

    • Search Google Scholar
    • Export Citation
  • Joyce, R. J., and P. Xie, 2011: Kalman filter–based CMORPH. J. Hydrometeor., 12, 15471563, https://doi.org/10.1175/JHM-D-11-022.1.

  • Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kidd, C., and G. Huffman, 2011: Global Precipitation Measurement. Meteor. Appl., 18, 334353, https://doi.org/10.1002/met.284.

  • Kidd, C., and V. Levizzani, 2011: Status of satellite precipitation retrievals. Hydrol. Earth Syst. Sci., 15, 11091116, https://doi.org/10.5194/hess-15-1109-2011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krajewski, W. F., G. J. Ciach, J. R. Mccollum, and C. Bacotiu, 2000: Initial validation of the global precipitation climatology project monthly rainfall over the United States. J. Appl. Meteor., 39, 10711086, https://doi.org/10.1175/1520-0450(2000)039<1071:IVOTGP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kubota, T., and Coauthors, 2007: Global precipitation map using satellite-borne microwave radiometers by the GSMaP project: Production and validation. IEEE Trans. Geosci. Remote Sens., 45, 22592275, https://doi.org/10.1109/TGRS.2007.895337.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z., 2016: Comparison of Integrated Multi-satellite Retrievals for GPM (IMERG) and TRMM Multisatellite Precipitation Analysis (TMPA) monthly precipitation products: Initial results. J. Hydrometeor., 17, 777790, https://doi.org/10.1175/JHM-D-15-0068.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lu, D., and B. Yong, 2018: Evaluation and hydrological utility of the latest GPM IMERG V5 and GSMaP V7 precipitation products over the Tibetan Plateau. Remote Sens., 10, 2022, https://doi.org/10.3390/rs10122022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lu, X., G. Tang, X. Wang, Y. Liu, L. Jia, G. Xie, and S. Li, 2019: Correcting GPM IMERG precipitation data over the Tianshan mountains in China. J. Hydrol., 575, 12391252, https://doi.org/10.1016/j.jhydrol.2019.06.019.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ma, Y., G. Tang, D. Long, B. Yong, L. Zhong, W. Wan, and Y. Hong, 2016: Similarity and error intercomparison of the GPM and its predecessor-TRMM multisatellite precipitation analysis using the best available hourly gauge network over the Tibetan Plateau. Remote Sens., 8, 569, https://doi.org/10.3390/rs8070569.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Maggioni, V., P. C. Meyers, and M. D. Robinson, 2016: A review of merged high-resolution satellite precipitation product accuracy during the Tropical Rainfall Measuring Mission (TRMM) era. J. Hydrometeor., 17, 11011117, https://doi.org/10.1175/JHM-D-15-0190.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McCollum, J. R., and W. F. Krajewski, 1998: Uncertainty of monthly rainfall estimates from rain gauges in the Global Precipitation Climatology Project. Water Resour. Res., 34, 26472654, https://doi.org/10.1029/98WR02173.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Prakash, S., A. Seshadri, J. Srinivasan, and D. S. Pai, 2019: A new parameter to assess impact of rain gauge density on uncertainty in the estimate of monthly rainfall over India. J. Hydrometeor., 20, 821832, https://doi.org/10.1175/JHM-D-18-0161.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ren, Y., B. Yong, D. Lu, and H. Chen, 2019: Evaluation of the Integrated Multi-satellitE Retrievals (IMERG) for Global Precipitation Measurement (GPM) mission over the mainland China at multiples scales (in Chinese). Hupo Kexue, 31, 560572.

    • Search Google Scholar
    • Export Citation
  • Roca, R., P. Chambon, I. Jobard, P. Kirstetter, M. Gosset, and J. C. Berges, 2010: Comparing satellite and surface rainfall products over west Africa at meteorologically relevant scales during the AMMA campaign using error estimates. J. Appl. Meteor. Climatol., 49, 715731, https://doi.org/10.1175/2009JAMC2318.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rudolf, B., H. Hauschild, W. Rueth, and U. Schneider, 1994: Terrestrial precipitation analysis: Operational method and required density of point measurements. Global Precipitation and Climate Change, M. Desbois and F. Desalmand, Eds., Springer, 173186.

    • Crossref
    • Export Citation
  • Shen, Y., P. Zhao, Y. Pan, and J. Yu, 2014: A high spatiotemporal gauge-satellite merged precipitation analysis over China. J. Geophys. Res. Atmos., 119, 30633075, https://doi.org/10.1002/2013JD020686.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sorooshian, S., K. L. Hsu, X. Gao, H. V. Gupta, B. Imam, and D. Braithwaite, 2000: Evaluation of PERSIANN system satellite-based estimates of tropical rainfall. Bull. Amer. Meteor. Soc., 81, 20352046, https://doi.org/10.1175/1520-0477(2000)081<2035:EOPSSE>2.3.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Su, J., H. , Y. Zhu, X. Wang, and G. Wei, 2018: Component analysis of errors in four GPM-based precipitation estimations over Mainland China. Remote Sens., 10, 1420, https://doi.org/10.3390/rs10091420.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tan, J., W. A. Petersen, P. E. Kirstetter, and Y. Tian, 2017: Performance of IMERG as a function of spatiotemporal scale. J. Hydrometeor., 18, 307319, https://doi.org/10.1175/JHM-D-16-0174.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tang, G., A. Behrangi, D. Long, C. Li, and Y. Hong, 2018: Accounting for spatiotemporal errors of gauges: A critical step to evaluate gridded precipitation products. J. Hydrol., 559, 294306, https://doi.org/10.1016/j.jhydrol.2018.02.057.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tian, F., S. Hou, L. Yang, H. Hu, and A. Hou, 2018: How does the evaluation of the GPM IMERG rainfall product depend on gauge density and rainfall intensity? J. Hydrometeor., 19, 339349, https://doi.org/10.1175/JHM-D-17-0161.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tian, Y., and C. D. Peters-Lidard, 2010: A global map of uncertainties in satellite-based precipitation measurements. Geophys. Res. Lett., 37, L24407, https://doi.org/10.1029/2010GL046008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ushio, T., and Coauthors, 2009: A Kalman filter approach to the Global Satellite Mapping of Precipitation (GSMaP) from combined passive microwave and infrared radiometric data. J. Meteor. Soc. Japan, 87, 137151, https://doi.org/10.2151/JMSJ.87A.137.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Villarini, G., and W. F. Krajewski, 2007: Evaluation of the research-version TMPA three-hourly 0.25° × 0.25° rainfall estimates over Oklahoma. Geophys. Res. Lett., 34, L05402, https://doi.org/10.1029/2006GL029147.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Villarini, G., P. V. Mandapaka, W. F. Krajewski, and R. J. Moore, 2008: Rainfall and sampling uncertainties: A rain gauge perspective. J. Geophys. Res., 113, D11102, https://doi.org/10.1029/2007JD009214.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, C., G. Tang, Z. Han, X. Guo, and Y. Hong, 2018: Global intercomparison and regional evaluation of GPM IMERG Version-03, Version-04 and its latest Version-05 precipitation products: Similarity, difference and improvements. J. Hydrol., 564, 342356, https://doi.org/10.1016/j.jhydrol.2018.06.064.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, Z., R. Zhong, C. Lai, and J. C. Chen, 2017: Evaluation of the GPM IMERG satellite-based precipitation products and the hydrological utility. Atmos. Res., 196, 151163, https://doi.org/10.1016/j.atmosres.2017.06.020.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wu, Y., L. Guo, H. Zheng, B. Zhang, and M. Li, 2019: Hydroclimate assessment of gridded precipitation products for the Tibetan Plateau. Sci. Total Environ., 660, 15551564, https://doi.org/10.1016/j.scitotenv.2019.01.119.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P., and P. A. Arkin, 1997: Global precipitation: A 17-year monthly analysis based on gauge observations, satellite estimates, and numerical model outputs. Bull. Amer. Meteor. Soc., 78, 25392558, https://doi.org/10.1175/1520-0477(1997)078<2539:GPAYMA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P., M. Chen, S. Yang, A. Yatagai, T. Hayasaka, Y. Fukushima, and C. Liu, 2007: A gauge-based analysis of daily precipitation over East Asia. J. Hydrometeor., 8, 607626, https://doi.org/10.1175/JHM583.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P., M. Chen, and W. Shi, 2010: CPC global unified gauge-based analysis of daily precipitation. 24th Conf. on Hydrology, Atlanta, GA, Amer. Meteor. Soc., 2, https://ams.confex.com/ams/90annual/techprogram/paper_163676.htm.

  • Xu, R., F. Tian, L. Yang, H. Hu, H. Lu, and A. Hou, 2017: Ground validation of GPM IMERG and TRMM 3B42V7 rainfall products over southern Tibetan Plateau based on a high-density rain gauge network. J. Geophys. Res. Atmos., 122, 910924, https://doi.org/10.1002/2016JD025418.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xu, S., Y. Shen, and Z. Du, 2016: Tracing the source of the errors in hourly IMERG using a decomposition evaluation scheme. Atmosphere, 7, 161, https://doi.org/10.3390/atmos7120161.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yang, Y., and Y. Luo, 2014: Evaluating the performance of remote sensing precipitation products CMORPH, PERSIANN, and TMPA, in the arid region of northwest China. Theor. Appl. Climatol., 118, 429445, https://doi.org/10.1007/s00704-013-1072-0.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yong, B., L. Ren, Y. Hong, J. H. Wang, J. J. Gourley, S. Jiang, X. Chen, and W. Wang, 2010: Hydrologic evaluation of multisatellite precipitation analysis standard precipitation products in basins beyond its inclined latitude band: A case study in Laohahe basin, China. Water Resour. Res., 46, W07542, https://doi.org/10.1029/2009WR008965.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yong, B., D. Liu, J. J. Gourley, Y. Tian, G. J. Huffman, L. Ren, and Y. Hong, 2015: Global view of real-time TRMM multisatellite precipitation analysis: Implications for its successor global precipitation measurement mission. Bull. Amer. Meteor. Soc., 96, 283296, https://doi.org/10.1175/BAMS-D-14-00017.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yong, B., B. Chen, Y. Tian, Z. Yu, and Y. Hong, 2016: Error-component analysis of TRMM-based multi-satellite precipitation estimates over mainland China. Remote Sens., 8, 440, https://doi.org/10.3390/rs8050440.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhao, H., B. Yang, S. Yang, Y. Huang, G. Dong, J. Bai, and Z. Wang, 2018: Systematical estimation of GPM-based global satellite mapping of precipitation products over China. Atmos. Res., 201, 206217, https://doi.org/10.1016/j.atmosres.2017.11.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
Save