Cross-Validation Methods for Multisource Precipitation Datasets over the Sparse-Gauge Region: Applicability and Uncertainty

Mingze Ding aCollege of Oceanic and Atmospheric Sciences, Ocean University of China, Qingdao, China

Search for other papers by Mingze Ding in
Current site
Google Scholar
PubMed
Close
,
Zhehui Shen bCollege of Civil Engineering, Nanjing Forestry University, Nanjing, China

Search for other papers by Zhehui Shen in
Current site
Google Scholar
PubMed
Close
,
Ruochen Huang cCollege of Hydrology and Water Resources, Hohai University, Nanjing, China

Search for other papers by Ruochen Huang in
Current site
Google Scholar
PubMed
Close
,
Ying Liu dShanghai Maritime Hydrographic Center, Eastern Navigation Service Center, Maritime Safety Administration, Shanghai, China

Search for other papers by Ying Liu in
Current site
Google Scholar
PubMed
Close
, and
Hao Wu eSchool of Geographic Information and Tourism, Chuzhou University, Chuzhou, China

Search for other papers by Hao Wu in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Evaluating the accuracy of various precipitation datasets over ungauged or even sparse-gauge areas is a challenging task. Cross-validation methods can evaluate three or more datasets based on the error independence from input data, without relying on ground observations. Here, the triple collocation (TC) method is employed to evaluate multisource precipitation datasets: China Gauge-based Daily Precipitation Analysis (CGDPA), model-based ERA5, and satellite-derived IMERG-Early, IMERG-Late, GSMaP in near–real time (GSMaP-NRT), and GSMaP moving vector with Kalman filter (GSMaP-MVK) over the Tibetan Plateau (TP). TC-based results show that ERA5 has better performances than satellite-only precipitation products over mountainous regions with complex terrains. For purely satellite-derived products, IMERG products outperform GSMaP products. Considering the potential existence of error dependencies among input datasets, caution should be exercised. Thus, it is necessary to introduce an alternative cross-validation method (generalized three-cornered hat) and explore the applicability of cross validation from the perspective of error independence. We found that cross-validation methods have high applicability in most TP regions with sparse-gauge density (accounting for about 80.1% of the total area). Additionally, we conducted simulation experiments to discuss the applicability and robustness of TC. The simulation results substantiated that cross validation can serve as a robust evaluation method over sparse-gauge regions. Although it is generally known that the cross-validation methods can be served in sparse-gauge regions, the application condition of different evaluation methods for precipitation products is identified quantitatively in TP now.

Significance Statement

Cross validation is a powerful assessment method for multisource precipitation datasets. This method is widely used by hydrologists in sparse-gauge or ungauged regions, like the African continent and the Tibetan Plateau (TP). However, as an indirect assessment method, the inherent uncertainty in cross validation warrants emphasis. Here, two cross-validation methods (the triple collocation method and the generalized three-cornered hat method) are employed to evaluate multisource precipitation datasets: China Gauge-based Daily Precipitation Analysis (CGDPA), model-based ERA5, and satellite-derived IMERG-Early, IMERG-Late, GSMaP in near–real time (GSMaP-NRT), and GSMaP moving vector with Kalman filter (GSMaP-MVK) over the TP. In this study, we not only assessed the current mainstream six precipitation datasets but also analyzed their uncertainties by combining these two cross-validation methods.

© 2024 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Zhehui Shen, shenzhehui@njfu.edu.cn

Abstract

Evaluating the accuracy of various precipitation datasets over ungauged or even sparse-gauge areas is a challenging task. Cross-validation methods can evaluate three or more datasets based on the error independence from input data, without relying on ground observations. Here, the triple collocation (TC) method is employed to evaluate multisource precipitation datasets: China Gauge-based Daily Precipitation Analysis (CGDPA), model-based ERA5, and satellite-derived IMERG-Early, IMERG-Late, GSMaP in near–real time (GSMaP-NRT), and GSMaP moving vector with Kalman filter (GSMaP-MVK) over the Tibetan Plateau (TP). TC-based results show that ERA5 has better performances than satellite-only precipitation products over mountainous regions with complex terrains. For purely satellite-derived products, IMERG products outperform GSMaP products. Considering the potential existence of error dependencies among input datasets, caution should be exercised. Thus, it is necessary to introduce an alternative cross-validation method (generalized three-cornered hat) and explore the applicability of cross validation from the perspective of error independence. We found that cross-validation methods have high applicability in most TP regions with sparse-gauge density (accounting for about 80.1% of the total area). Additionally, we conducted simulation experiments to discuss the applicability and robustness of TC. The simulation results substantiated that cross validation can serve as a robust evaluation method over sparse-gauge regions. Although it is generally known that the cross-validation methods can be served in sparse-gauge regions, the application condition of different evaluation methods for precipitation products is identified quantitatively in TP now.

Significance Statement

Cross validation is a powerful assessment method for multisource precipitation datasets. This method is widely used by hydrologists in sparse-gauge or ungauged regions, like the African continent and the Tibetan Plateau (TP). However, as an indirect assessment method, the inherent uncertainty in cross validation warrants emphasis. Here, two cross-validation methods (the triple collocation method and the generalized three-cornered hat method) are employed to evaluate multisource precipitation datasets: China Gauge-based Daily Precipitation Analysis (CGDPA), model-based ERA5, and satellite-derived IMERG-Early, IMERG-Late, GSMaP in near–real time (GSMaP-NRT), and GSMaP moving vector with Kalman filter (GSMaP-MVK) over the TP. In this study, we not only assessed the current mainstream six precipitation datasets but also analyzed their uncertainties by combining these two cross-validation methods.

© 2024 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Zhehui Shen, shenzhehui@njfu.edu.cn

Supplementary Materials

    • Supplemental Materials (PDF 1.6950 MB)
Save
  • Allen, M. R., and W. J. Ingram, 2002: Constraints on future changes in climate and the hydrologic cycle. Nature, 419, 224232, https://doi.org/10.1038/nature01092.

    • Search Google Scholar
    • Export Citation
  • Awange, J. L., V. G. Ferreira, E. Forootan, Khandu, S. A. Andam-Akorful, N. O. Agutu, and X. F. He, 2016: Uncertainties in remotely sensed precipitation data over Africa. Int. J. Climatol., 36, 303323, https://doi.org/10.1002/joc.4346.

    • Search Google Scholar
    • Export Citation
  • Benoit, L., 2021: Radar and rain gauge data fusion based on disaggregation of radar imagery. Wat. Resour. Res., 57, e2020WR027899, https://doi.org/10.1029/2020WR027899.

    • Search Google Scholar
    • Export Citation
  • Cattani, E., A. Merino, and V. Levizzani, 2016: Evaluation of monthly satellite-derived precipitation products over East Africa. J. Hydrometeor., 17, 25552573, https://doi.org/10.1175/JHM-D-15-0042.1.

    • Search Google Scholar
    • Export Citation
  • Chen, C., M. He, Q. Chen, J. Zhang, Z. Li, Z. Wang, and Z. Duan, 2022: Triple collocation-based error estimation and data fusion of global gridded precipitation products over the Yangtze River basin. J. Hydrol., 605, 127307, https://doi.org/10.1016/j.jhydrol.2021.127307.

    • Search Google Scholar
    • Export Citation
  • Chen, H., V. Chandrasekar, R. Cifelli, P. Xie, and H. Tan, 2017: A data fusion system for accurate precipitation estimation using satellite and ground radar observations: Urban scale application in Dallas-Fort Worth Metroplex. 2017 XXXIInd General Assembly and Scientific Symp. of the International Union of Radio Science (URSI GASS), Montreal, QC, Canada, Institute of Electrical and Electronics Engineers, 1–2, https://doi.org/10.23919/URSIGASS.2017.8105093.

  • Chen, H., B. Yong, P.-E. Kirstetter, L. Wang, and Y. Hong, 2021: Global component analysis of errors in three satellite-only global precipitation estimates. Hydrol. Earth Syst. Sci., 25, 30873104, https://doi.org/10.5194/hess-25-3087-2021.

    • Search Google Scholar
    • Export Citation
  • Duan, Z., E. Duggan, C. Chen, H. Gao, J. Dong, and J. Liu, 2021: Comparison of traditional method and triple collocation analysis for evaluation of multiple gridded precipitation products across Germany. J. Hydrometeor., 22, 29832999, https://doi.org/10.1175/JHM-D-21-0049.1.

    • Search Google Scholar
    • Export Citation
  • Fang, H., S. Wei, C. Jiang, and K. Scipal, 2012: Theoretical uncertainty analysis of global MODIS, CYCLOPES, and GLOBCARBON LAI products using a triple collocation method. Remote Sens. Environ., 124, 610621, https://doi.org/10.1016/j.rse.2012.06.013.

    • Search Google Scholar
    • Export Citation
  • Forootan, E., and Coauthors, 2014: Multivariate prediction of total water storage changes over West Africa from multi-satellite data. Surv. Geophys., 35, 913940, https://doi.org/10.1007/s10712-014-9292-0.

    • Search Google Scholar
    • Export Citation
  • Galindo, F. J., and J. Palacio, 2003: Post-processing ROA data clocks for optimal stability in the ensemble timescale. Metrologia, 40, S237, https://doi.org/10.1088/0026-1394/40/3/301.

    • Search Google Scholar
    • Export Citation
  • Gruber, A., C.-H. Su, S. Zwieback, W. Crow, W. Dorigo, and W. Wagner, 2016: Recent advances in (soil moisture) triple collocation analysis. Int. J. Appl. Earth Obs. Geoinf., 45, 200211, https://doi.org/10.1016/j.jag.2015.09.002.

    • Search Google Scholar
    • Export Citation
  • Hossain, F., and E. N. Anagnostou, 2006: A two-dimensional satellite rainfall error model. IEEE Trans. Geosci. Remote Sens., 44, 15111522, https://doi.org/10.1109/TGRS.2005.863866.

    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., and Coauthors, 2020: Integrated Multi-satellite Retrievals for the Global Precipitation Measurement (GPM) Mission (IMERG). Satellite Precipitation Measurement, V. Levizzani et al., Eds., Advances in Global Change Research, Vol. 67, Springer International Publishing, 343–353.

  • Jafarpour, M., A. Adib, and M. Lotfirad, 2022: Improving the accuracy of satellite and reanalysis precipitation data by their ensemble usage. Appl. Wat. Sci., 12, 232, https://doi.org/10.1007/s13201-022-01750-z.

    • Search Google Scholar
    • Export Citation
  • Jiang, Q., and Coauthors, 2021: Evaluation of the ERA5 reanalysis precipitation dataset over Chinese mainland. J. Hydrol., 595, 125660, https://doi.org/10.1016/j.jhydrol.2020.125660.

    • Search Google Scholar
    • Export Citation
  • Kidd, C., and G. Huffman, 2011: Global precipitation measurement. Meteor. Appl., 18, 334353, https://doi.org/10.1002/met.284.

  • Kidd, C., E. Dawkins, and G. Huffman, 2013: Comparison of precipitation derived from the ECMWF operational forecast model and satellite precipitation datasets. J. Hydrometeor., 14, 14631482, https://doi.org/10.1175/JHM-D-12-0182.1.

    • Search Google Scholar
    • Export Citation
  • Li, C., G. Tang, and Y. Hong, 2018: Cross-evaluation of ground-based, multi-satellite and reanalysis precipitation products: Applicability of the triple collocation method across mainland China. J. Hydrol., 562, 7183, https://doi.org/10.1016/j.jhydrol.2018.04.039.

    • Search Google Scholar
    • Export Citation
  • Li, Z., G. Tang, P. Kirstetter, S. Gao, J.-L. F. Li, Y. Wen, and Y. Hong, 2022: Evaluation of GPM IMERG and its constellations in extreme events over the conterminous United States. J. Hydrol., 606, 127357, https://doi.org/10.1016/j.jhydrol.2021.127357.

    • Search Google Scholar
    • Export Citation
  • Lu, H., L. Ding, Z. Ma, H. Li, T. Lu, M. Su, and J. Xu, 2020: Spatiotemporal assessments on the satellite‐based precipitation products from Fengyun and GPM over the Yunnan‐Kweichow Plateau, China. Earth Space Sci., 7, e2019EA000857, https://doi.org/10.1029/2019EA000857.

    • Search Google Scholar
    • Export Citation
  • Lu, X., G. Tang, X. Liu, X. Wang, Y. Liu, and M. Wei, 2021: The potential and uncertainty of triple collocation in assessing satellite precipitation products in Central Asia. Atmos. Res., 252, 105452, https://doi.org/10.1016/j.atmosres.2021.105452.

    • Search Google Scholar
    • Export Citation
  • Ma, Y., X. Sun, H. Chen, Y. Hong, and Y. Zhang, 2021: A two-stage blending approach for merging multiple satellite precipitation estimates and rain gauge observations: An experiment in the northeastern Tibetan Plateau. Hydrol. Earth Syst. Sci., 25, 359374, https://doi.org/10.5194/hess-25-359-2021.

    • Search Google Scholar
    • Export Citation
  • Ma, Z., J. Xu, S. Zhu, J. Yang, G. Tang, Y. Yang, Z. Shi, and Y. Hong, 2020: AIMERG: A new Asian precipitation dataset (0.1°/half-hourly, 2000–2015) by calibrating the GPM-era IMERG at a daily scale using APHRODITE. Earth Syst. Sci. Data, 12, 15251544, https://doi.org/10.5194/essd-12-1525-2020.

    • Search Google Scholar
    • Export Citation
  • Massari, C., W. Crow, and L. Brocca, 2017: An assessment of the performance of global rainfall estimates without ground-based observations. Hydrol. Earth Syst. Sci., 21, 43474361, https://doi.org/10.5194/hess-21-4347-2017.

    • Search Google Scholar
    • Export Citation
  • Nielsen, J. M., C. Z. R. van de Beek, S. Thorndahl, J. Olsson, C. B. Andersen, J. C. M. Andersson, M. R. Rasmussen, and J. E. Nielsen, 2024: Merging weather radar data and opportunistic rainfall sensor data to enhance rainfall estimates. Atmos. Res., 300, 107228, https://doi.org/10.1016/j.atmosres.2024.107228.

    • Search Google Scholar
    • Export Citation
  • Ochoa-Rodriguez, S., L.-P. Wang, P. Willems, and C. Onof, 2019: A review of radar-rain gauge data merging methods and their potential for urban hydrological applications. Water Resour. Res., 55, 63566391, https://doi.org/10.1029/2018WR023332.

    • Search Google Scholar
    • Export Citation
  • Peng, J., and Coauthors, 2021: Satellite-based precipitation datasets evaluation using gauge observation and hydrological modeling in a typical arid land watershed of central Asia. Remote Sens., 13, 221, https://doi.org/10.3390/rs13020221.

    • Search Google Scholar
    • Export Citation
  • Prat, O. P., B. R. Nelson, E. Nickl, and R. D. Leeper, 2021: Global evaluation of gridded satellite precipitation products from the NOAA Climate Data Record Program. J. Hydrometeor., 22, 22912310, https://doi.org/10.1175/JHM-D-20-0246.1.

    • Search Google Scholar
    • Export Citation
  • Roebeling, R. A., E. L. A. Wolters, J. F. Meirink, and H. Leijnse, 2012: Triple collocation of summer precipitation retrievals from SEVIRI over Europe with gridded rain gauge and weather radar data. J. Hydrometeor., 13, 15521566, https://doi.org/10.1175/JHM-D-11-089.1.

    • Search Google Scholar
    • Export Citation
  • Shen, Y., and A. Xiong, 2016: Validation and comparison of a new gauge-based precipitation analysis over mainland China. Int. J. Climatol., 36, 252265, https://doi.org/10.1002/joc.4341.

    • Search Google Scholar
    • Export Citation
  • Shen, Z., and Coauthors, 2020: Recent global performance of the Climate Hazards group Infrared Precipitation (CHIRP) with Stations (CHIRPS). J. Hydrol., 591, 125284, https://doi.org/10.1016/j.jhydrol.2020.125284.

    • Search Google Scholar
    • Export Citation
  • Shen, Z., B. Yong, J. J. Gourley, and W. Qi, 2021: Real-time bias adjustment for satellite-based precipitation estimates over Mainland China. J. Hydrol., 596, 126133, https://doi.org/10.1016/j.jhydrol.2021.126133.

    • Search Google Scholar
    • Export Citation
  • Silverman, B. W., 2017: Density Estimation for Statistics and Data Analysis. Routledge, 176 pp.

  • Stampoulis, D., and E. N. Anagnostou, 2012: Evaluation of global satellite rainfall products over continental Europe. J. Hydrometeor., 13, 588603, https://doi.org/10.1175/JHM-D-11-086.1.

    • Search Google Scholar
    • Export Citation
  • Stoffelen, A., 1998: Toward the true near-surface wind speed: Error modeling and calibration using triple collocation. J. Geophys. Res., 103, 77557766, https://doi.org/10.1029/97JC03180.

    • Search Google Scholar
    • Export Citation
  • Sun, Z., D. Long, Z. Hong, M. A. Hamouda, M. M. Mohamed, and J. Wang, 2022: How China’s Fengyun satellite precipitation product compares with other mainstream satellite precipitation products. J. Hydrometeor., 23, 785806, https://doi.org/10.1175/JHM-D-21-0179.1.

    • Search Google Scholar
    • Export Citation
  • Tang, G., Y. Ma, D. Long, L. Zhong, and Y. Hong, 2016: Evaluation of GPM day-1 IMERG and TMPA version-7 legacy products over mainland China at multiple spatiotemporal scales. J. Hydrol., 533, 152167, https://doi.org/10.1016/j.jhydrol.2015.12.008.

    • Search Google Scholar
    • Export Citation
  • Tian, Y., G. J. Huffman, R. F. Adler, L. Tang, M. Sapiano, V. Maggioni, and H. Wu, 2013: Modeling errors in daily precipitation measurements: Additive or multiplicative? Geophys. Res. Lett., 40, 20602065, https://doi.org/10.1002/grl.50320.

    • Search Google Scholar
    • Export Citation
  • Ushio, T., and Coauthors, 2009: A Kalman filter approach to the Global Satellite Mapping of Precipitation (GSMaP) from combined passive microwave and infrared radiometric data. J. Meteor. Soc. Japan, 87A, 137151, https://doi.org/10.2151/jmsj.87A.137.

    • Search Google Scholar
    • Export Citation
  • Utsumi, N., F. J. Turk, Z. S. Haddad, P.-E. Kirstetter, and H. Kim, 2021: Evaluation of precipitation vertical profiles estimated by GPM-era satellite-based passive microwave retrievals. J. Hydrometeor., 22, 95112, https://doi.org/10.1175/JHM-D-20-0160.1.

    • Search Google Scholar
    • Export Citation
  • Wilson, A. M., and A. P. Barros, 2017: Orographic land-atmosphere interactions and the diurnal cycle of low-level clouds and fog. J. Hydrometeor., 18, 15131533, https://doi.org/10.1175/JHM-D-16-0186.1.

    • Search Google Scholar
    • Export Citation
  • Xie, P., M. Chen, S. Yang, A. Yatagai, T. Hayasaka, Y. Fukushima, and C. Liu, 2007: A gauge-based analysis of daily precipitation over East Asia. J. Hydrometeor., 8, 607626, https://doi.org/10.1175/JHM583.1.

    • Search Google Scholar
    • Export Citation
  • Yamamoto, M. K., and S. Shige, 2015: Implementation of an orographic/nonorographic rainfall classification scheme in the GSMaP algorithm for microwave radiometers. Atmos. Res., 163, 3647, https://doi.org/10.1016/j.atmosres.2014.07.024.

    • Search Google Scholar
    • Export Citation
  • Yong, B., D. Liu, J. J. Gourley, Y. Tian, G. J. Huffman, L. Ren, and Y. Hong, 2014: Global view of real-time TRMM multisatellite precipitation analysis: Implications for its successor Global Precipitation Measurement mission. Bull. Amer. Meteor. Soc., 96, 283296, https://doi.org/10.1175/BAMS-D-14-00017.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 1993 1711 0
Full Text Views 596 544 142
PDF Downloads 234 206 15