• Alemohammad, S. H., K. A. McColl, A. G. Konings, D. Entekhabi, and A. Stoffelen, 2015: Characterization of precipitation product errors across the United States using multiplicative triple collocation. Hydrol. Earth Syst. Sci., 19, 34893503, https://doi.org/10.5194/hess-19-3489-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ashouri, H., K.-L. Hsu, S. Sorooshian, D. K. Braithwaite, K. R. Knapp, L. D. Cecil, B. R. Nelson, and O. P. Prat, 2015: PERSIANN-CDR daily precipitation climate data record from multisatellite observations for hydrological and climate studies. Bull. Amer. Meteor. Soc., 96, 69–83, https://doi.org/10.1175/BAMS-D-13-00068.1.

    • Crossref
    • Export Citation
  • Bae, D. H., I. W. Jung, and D. P. Lettenmaier, 2011: Hydrologic uncertainties in climate change from IPCC AR4 GCM simulations of the Chungju Basin, Korea. J. Hydrol., 401, 90105, https://doi.org/10.1016/j.jhydrol.2011.02.012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berrisford, P., and Coauthors, 2011: The ERA-Interim Archive, Version 2.0. ERA Rep. Series 1, 23 pp., https://www.ecmwf.int/sites/default/files/elibrary/2011/8174-era-interim-archive-version-20.pdf.

  • Brocca, L., and Coauthors, 2014: Soil as a natural rain gauge: Estimating global rainfall from satellite soil moisture data. J. Geophys. Res. Atmos., 119, 51285141, https://doi.org/10.1002/2014JD021489.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brocca, L., and Coauthors, 2017: A review of the applications of ASCAT soil moisture products. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 10, 22852306, https://doi.org/10.1109/JSTARS.2017.2651140.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cecinati, F., M. A. Rico-Ramirez, G. B. M. Heuvelink, and D. Han, 2017: Representing radar rainfall uncertainty with ensembles based on a time-variant geostatistical error modelling approach. J. Hydrol., 548, 391405, https://doi.org/10.1016/j.jhydrol.2017.02.053.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, C., and Coauthors, 2020: Performance of multiple satellite precipitation estimates over a typical arid mountainous area of China: Spatiotemporal patterns and extremes. J. Hydrometeor., 21, 533550, https://doi.org/10.1175/JHM-D-19-0167.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, F., and Coauthors, 2021: Enhanced large-scale validation of satellite-based land rainfall products. J. Hydrometeor., 22, 245257, https://doi.org/10.1175/JHM-D-20-0056.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ciach, G. J., W. F. Krajewski, and G. Villarini, 2007: Product-error-driven uncertainty model for probabilistic quantitative precipitation estimation with NEXRAD data. J. Hydrometeor., 8, 13251347, https://doi.org/10.1175/2007JHM814.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crow, W. T., and M. J. van den Berg, 2010: An improved approach for estimating observation and model error parameters in soil moisture data assimilation. Water Resour. Res., 46, W12519, https://doi.org/10.1029/2010WR009402.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dankers, R., and Coauthors, 2014: First look at changes in flood hazard in the Inter-Sectoral Impact Model Intercomparison Project ensemble. Proc. Natl. Acad. Sci. USA, 111, 32573261, https://doi.org/10.1073/pnas.1302078110.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, https://doi.org/10.1002/qj.828.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dong, J., W. T. Crow, Z. Duan, L. Wei, and Y. Lu, 2019: A double instrumental variable method for geophysical product error estimation. Remote Sens. Environ., 225, 217228, https://doi.org/10.1016/j.rse.2019.03.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dong, J., L. Wei, X. Chen, Z. Duan, and Y. Lu, 2020a: An instrument variable based algorithm for estimating cross-correlated hydrological remote sensing errors. J. Hydrol., 581, 124413, https://doi.org/10.1016/j.jhydrol.2019.124413.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dong, J., F. Lei, and L. Wei, 2020b: Triple collocation based multi-source precipitation merging. Front. Water, 2, 1, https://doi.org/10.3389/frwa.2020.00001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Duan, Z., J. Z. Liu, Y. Tuo, G. Chiogna, and M. Disse, 2016: Evaluation of eight high spatial resolution gridded precipitation products in Adige Basin (Italy) at multiple temporal and spatial scales. Sci. Total Environ., 573, 15361553, https://doi.org/10.1016/j.scitotenv.2016.08.213.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Duan, Z., and Coauthors, 2019a: Hydrological evaluation of open-access precipitation and air temperature datasets using SWAT in a poorly gauged basin in Ethiopia. J. Hydrol., 569, 612626, https://doi.org/10.1016/j.jhydrol.2018.12.026.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Duan, Z., and Coauthors, 2019b: Spatiotemporal analysis of nonlinear trends in precipitation over Germany during 1951–2013 from multiple observation-based gridded products. Int. J. Climatol., 39, 21202135, https://doi.org/10.1002/joc.5939.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fang, H. L., S. S. Wei, C. Y. Jiang, and K. Scipal, 2012: Theoretical uncertainty analysis of global MODIS, CYCLOPES, and GLOBCARBON LAI products using a triple collocation method. Remote Sens. Environ., 124, 610621, https://doi.org/10.1016/j.rse.2012.06.013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gao, H., H. Cai, and Z. Duan, 2018: Understanding the impacts of catchment characteristics on the shape of the storage capacity curve and its influence on flood flows. Hydrol. Res., 49, 90106, https://doi.org/10.2166/nh.2017.245.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gentemann, C. L., 2014: Three way validation of MODIS and AMSR-E sea surface temperatures. J. Geophys. Res. Oceans, 119, 25832598, https://doi.org/10.1002/2013JC009716.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Germann, U., G. Galli, M. Boscacci, and M. Bolliger, 2006: Radar precipitation measurement in a mountainous region. Quart. J. Roy. Meteor. Soc., 132, 16691692, https://doi.org/10.1256/qj.05.190.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gruber, A., and Coauthors, 2016: Recent advances in (soil moisture) triple collocation analysis. Int. J. Appl. Earth Obs. Geoinf., 45, 200211, https://doi.org/10.1016/j.jag.2015.09.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guzzetti, F., A. C. Mondini, M. Cardinali, F. Fiorucci, M. Santangelo, and K. T. Chang, 2012: Landslide inventory maps: New tools for an old problem. Earth-Sci. Rev., 112, 4266, https://doi.org/10.1016/j.earscirev.2012.02.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hossain, F., and E. N. Anagnostou, 2006: A two-dimensional satellite rainfall error model. IEEE Trans. Geosci. Remote Sens., 44, 15111522, https://doi.org/10.1109/TGRS.2005.863866.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hsu, K., X. Gao, S. Sorooshian, and H. V. Gupta, 1997: Precipitation estimation from remotely sensed information using artificial neural networks. J. Appl. Meteor., 36, 11761190, https://doi.org/10.1175/1520-0450(1997)036<1176:PEFRSI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kidd, C., and G. Huffman, 2011: Global precipitation measurement. Meteor. Appl., 18, 334353, https://doi.org/10.1002/met.284.

  • Li, C., G. Tang, and Y. Hong, 2018: Cross-evaluation of ground-based, multi-satellite and reanalysis precipitation products: Applicability of the Triple Collocation method across Mainland China. J. Hydrol., 562, 7183, https://doi.org/10.1016/j.jhydrol.2018.04.039.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, W., F. Sun, W. H. Lim, J. Zhang, and Y. Zhang, 2018: Global drought and severe drought-affected populations in 1.5 and 2°C warmer worlds. Earth Syst. Dyn., 9, 267283, https://doi.org/10.5194/esd-9-267-2018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Maggioni, V., M. R. P. Sapiano, R. F. Adler, Y. D. Tian, and G. J. Huffman, 2014: An error model for uncertainty quantification in high-time-resolution precipitation products. J. Hydrometeor., 15, 12741292, https://doi.org/10.1175/JHM-D-13-0112.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Massari, C., W. Crow, and L. Brocca, 2017: An assessment of the performance of global rainfall estimates without ground-based observations. Hydrol. Earth Syst. Sci., 21, 43474361, https://doi.org/10.5194/hess-21-4347-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McColl, K. A., and Coauthors, 2014: Extended triple collocation: Estimating errors and correlation coefficients with respect to an unknown target. Geophys. Res. Lett., 41, 62296236, https://doi.org/10.1002/2014GL061322.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mei, Y. W., E. N. Anagnostou, E. I. Nikolopoulos, and M. Borga, 2014: Error analysis of satellite precipitation products in mountainous basins. J. Hydrometeor., 15, 17781793, https://doi.org/10.1175/JHM-D-13-0194.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mei, Y. W., E. I. Nikolopoulos, E. N. Anagnostou, D. Zoccatelli, and M. Borga, 2016: Error analysis of satellite precipitation-driven modeling of flood events in complex alpine terrain. Remote Sens., 8, 293, https://doi.org/10.3390/rs8040293.

    • Crossref
    • Export Citation
  • Mei, Y. W., E. N. Anagnostou, X. Y. Shen, and E. I. Nikolopoulos, 2017: Decomposing the satellite precipitation error propagation through the rainfall-runoff processes. Adv. Water Resour., 109, 253266, https://doi.org/10.1016/j.advwatres.2017.09.012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Oki, T., and S. Kanae, 2006: Global hydrological cycles and world water resources. Science, 313, 10681072, https://doi.org/10.1126/science.1128845.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Paeth, H., and Coauthors, 2017: An efficient model approach for very high resolution orographic precipitation. Quart. J. Roy. Meteor. Soc., 143, 22212234, https://doi.org/10.1002/qj.3080.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pastor, A. V., F. Ludwig, H. Biemans, H. Hoff, and P. Kabat, 2014: Accounting for environmental flow requirements in global water assessments. Hydrol. Earth Syst. Sci., 18, 50415059, https://doi.org/10.5194/hess-18-5041-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peel, M. C., B. L. Finlayson, and T. A. McMahon, 2007: Updated world map of the Koppen-Geiger climate classification. Hydrol. Earth Syst. Sci., 11, 16331644, https://doi.org/10.5194/hess-11-1633-2007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peleg, N., and Coauthors, 2018: Spatial variability of extreme rainfall at radar subpixel scale. J. Hydrol., 556, 922933, https://doi.org/10.1016/j.jhydrol.2016.05.033.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Portabella, M., and A. Stoffelen, 2009: On scatterometer ocean stress. J. Atmos. Oceanic Technol., 26, 368382, https://doi.org/10.1175/2008JTECHO578.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rauthe, M., H. Steiner, U. Riediger, A. Mazurkiewicz, and A. Gratzki, 2013: A Central European precipitation climatology – Part I: Generation and validation of a high-resolution gridded daily data set (HYRAS). Meteor. Z., 22, 235256, https://doi.org/10.1127/0941-2948/2013/0436.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roebeling, R. A., E. L. A. Wolters, J. F. Meirink, and H. Leijnse, 2012: Triple collocation of summer precipitation retrievals from SEVIRI over Europe with gridded rain gauge and weather radar data. J. Hydrometeor., 13, 15521566, https://doi.org/10.1175/JHM-D-11-089.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schyns, J. F., A. Y. Hoekstra, and M. J. Booij, 2015: Review and classification of indicators of green water availability and scarcity. Hydrol. Earth Syst. Sci., 19, 45814608, https://doi.org/10.5194/hess-19-4581-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Scott, K. A., M. Buehner, and T. Carrieres, 2014: An assessment of sea-ice thickness along the Labrador coast from AMSR-E and MODIS data for operational data assimilation. IEEE Trans. Geosci. Remote Sens., 52, 27262737, https://doi.org/10.1109/TGRS.2013.2265091.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stoffelen, A., 1998: Toward the true near-surface wind speed: Error modeling and calibration using triple collocation. J. Geophys. Res., 103, 77557766, https://doi.org/10.1029/97JC03180.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, Q. H., and Coauthors, 2018: A review of global precipitation data sets: Data sources, estimation, and intercomparisons. Rev. Geophys., 56, 79107, https://doi.org/10.1002/2017RG000574.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tang, G. Q., Y. Z. Ma, D. Long, L. Z. Zhong, and Y. Hong, 2016: Evaluation of GPM Day-1 IMERG and TMPA Version-7 legacy products over Mainland China at multiple spatiotemporal scales. J. Hydrol., 533, 152167, https://doi.org/10.1016/j.jhydrol.2015.12.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tang, G. Q., A. Behrangi, D. Long, C. M. Li, and Y. Hong, 2018: Accounting for spatiotemporal errors of gauges: A critical step to evaluate gridded precipitation products. J. Hydrol., 559, 294306, https://doi.org/10.1016/j.jhydrol.2018.02.057.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tang, G. Q., M. P. Clark, S. M. Papalexiou, Z. Ma, and Y. Hong, 2020: Have satellite precipitation products improved over last two decades? A comprehensive comparison of GPM IMERG with nine satellite and reanalysis datasets. Remote Sens. Environ., 240, 111697, https://doi.org/10.1016/j.rse.2020.111697.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tapiador, F. J., and Coauthors, 2012: Global precipitation measurement: Methods, datasets and applications. Atmos. Res., 104–105, 7097, https://doi.org/10.1016/j.atmosres.2011.10.021.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thao, S., L. Eymard, E. Obligis, and B. Picard, 2014: Trend and variability of the atmospheric water vapor: A mean sea level issue. J. Atmos. Oceanic Technol., 31, 18811901, https://doi.org/10.1175/JTECH-D-13-00157.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tian, Y. D., and Coauthors, 2013: Modeling errors in daily precipitation measurements: Additive or multiplicative? Geophys. Res. Lett., 40, 20602065, https://doi.org/10.1002/grl.50320.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Dijk, A. I. J. M., L. J. Renzullo, Y. Wada, and P. Tregoning, 2014: A global water cycle reanalysis (2003-2012) merging satellite gravimetry and altimetry observations with a hydrological multi-model ensemble. Hydrol. Earth Syst. Sci., 18, 29552973, https://doi.org/10.5194/hess-18-2955-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Villarini, G., W. F. Krajewski, G. J. Ciach, and D. L. Zimmerman, 2009: Product-error-driven generator of probable rainfall conditioned on WSR-88D precipitation estimates. Water Resour. Res., 45, W01404, https://doi.org/10.1029/2008WR006946.

    • Search Google Scholar
    • Export Citation
  • Wang, A. H., and X. B. Zeng, 2012: Evaluation of multireanalysis products with in situ observations over the Tibetan Plateau. J. Geophys. Res., 117, D05102, https://doi.org/10.1029/2011JD016553.

    • Search Google Scholar
    • Export Citation
  • Winterrath, T., and Coauthors, 2016: Towards a radar-based high-resolution precipitation climatology for Germany – Methodology and potential areas of application. European Meteorological Society Conf., Trieste, Italy, European Meteorological Society, 24 pp., https://presentations.copernicus.org/EMS2016-375_presentation.pdf.

  • Wu, X., Q. Xiao, J. Wen, and D. You, 2019: Direct comparison and triple collocation: Which is more reliable in the validation of coarse-scale satellite surface albedo products. J. Geophys. Res. Atmos., 124, 51985213, https://doi.org/10.1029/2018JD029937.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P. P., and P. A. Arkin, 1997: Global precipitation: A 17-year monthly analysis based on gauge observations, satellite estimates, and numerical model outputs. Bull. Amer. Meteor. Soc., 78, 25392558, https://doi.org/10.1175/1520-0477(1997)078<2539:GPAYMA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P. P., and A. Y. Xiong, 2011: A conceptual model for constructing high-resolution gauge-satellite merged precipitation analyses. J. Geophys. Res., 116, D21106, https://doi.org/10.1029/2011JD016118.

    • Search Google Scholar
    • Export Citation
  • Yilmaz, M. T., and W. T. Crow, 2014: Evaluation of assumptions in soil moisture triple collocation analysis. J. Hydrometeor., 15, 12931302, https://doi.org/10.1175/JHM-D-13-0158.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zwieback, S., K. Scipal, W. Dorigo, and W. Wagner, 2012: Structural and statistical properties of the collocation technique for error characterization. Nonlinear Processes Geophys., 19, 6980, https://doi.org/10.5194/npg-19-69-2012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    Topography and precipitation climatology of Germany. (a) Digital elevation model (DEM). (b) Mean daily precipitation (2007–17) estimated by REGNIE. Data are shown at the study 0.5° grid scale in the lower left insets.

  • View in gallery
    Fig. 2.

    Results of 12 strategies for managing daily precipitation values equal to zero from the MTC analysis using the triplet B (CMORPH-ERA-SM2RAIN combinations). The four evaluation metrics are detailed in section 4e in the text. They are country-wise mean values for 1) MTC-derived RMSE, 2) MTC-derived CC, 3) the absolute relative difference (ARD) for RMSE between values derived from MTC and the traditional method, and 4) ARD for CC.

  • View in gallery
    Fig. 3.

    Spatial distribution of correlation coefficient for each precipitation product from the traditional method with REGNIE as reference and the MTC analysis using seven different triplet combinations (triplet A–G as detailed in Table 1).

  • View in gallery
    Fig. 4.

    Spatial distribution of root-mean-square error (RMSE; mm day−1) for each product from the traditional method with REGNIE as reference and the MTC analysis using seven different triplet combinations (triplet A–G as detailed in Table 1).

  • View in gallery
    Fig. 5.

    Spatial distribution of correlation coefficient for each precipitation product from the traditional method with REGNIE as reference and the classic triplet collocation (TC) analysis with additive error model using seven different triplet combinations (triplet A–G as detailed in Table 1).

All Time Past Year Past 30 Days
Abstract Views 631 367 0
Full Text Views 234 191 25
PDF Downloads 251 200 35

Comparison of Traditional Method and Triple Collocation Analysis for Evaluation of Multiple Gridded Precipitation Products across Germany

Zheng DuanaDepartment of Physical Geography and Ecosystem Science, Lund University, Lund, Sweden

Search for other papers by Zheng Duan in
Current site
Google Scholar
PubMed
Close
,
Edward DugganbChair of Hydrology and River Basin Management, Technical University of Munich, Munich, Germany

Search for other papers by Edward Duggan in
Current site
Google Scholar
PubMed
Close
,
Cheng ChencState Key Laboratory of Hydrology-Water Resources and Hydraulic Engineering, Nanjing Hydraulic Research Institute, Nanjing, China
dCenter for Eco-Environmental Research, Nanjing Hydraulic Research Institute, Nanjing, China

Search for other papers by Cheng Chen in
Current site
Google Scholar
PubMed
Close
,
Hongkai GaoeKey Laboratory of Geographic Information Science (Ministry of Education of China), East China Normal University, Shanghai, China

Search for other papers by Hongkai Gao in
Current site
Google Scholar
PubMed
Close
,
Jianzhi DongfDepartment of Civil and Environmental Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts

Search for other papers by Jianzhi Dong in
Current site
Google Scholar
PubMed
Close
, and
Junzhi LiugKey Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, China
hJiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, China

Search for other papers by Junzhi Liu in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

Evaluating the accuracy of precipitation products is essential for many applications. The traditional method for evaluation is to calculate error metrics of products with gauge measurements that are considered as ground truth. The multiplicative triple collocation (MTC) method has been demonstrated powerful in error quantification of precipitation products when ground truth is not known. This study applied MTC to evaluate five precipitation products in Germany: two raw satellite-based products (CMORPH and PERSIANN), one reanalysis product (ERA-Interim), one soil moisture–based product (SM2RAIN-ASCAT), and one gauge-based product (REGNIE). Evaluation was performed at the 0.5° daily spatial–temporal scales. MTC involves a log transformation of data, necessitating dealing with zero values in daily precipitation. Effects of 12 different strategies for dealing with zero values on MTC results were investigated. Seven different triplet combinations were tested to evaluate the stability of MTC. Results showed that different strategies for replacing zero values had considerable effects on MTC-derived error metrics, particularly for root-mean-square error (RMSE). MTC with different triplet combinations generated different error metrics for individual products. The MTC-derived correlation coefficient (CC) was more reliable than RMSE. It is more appropriate to use MTC to compare the relative accuracy of different precipitation products. Based on CC with unknown truth, MTC with different triplet combinations produced the same ranking of products as the traditional method. A comparison of results from MTC and the classic TC with additive error model showed the potential limitation of MTC in arid areas or dry time periods with a large ratio of zero daily precipitation.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Hongkai Gao, hkgao@geo.ecnu.edu.cn

Abstract

Evaluating the accuracy of precipitation products is essential for many applications. The traditional method for evaluation is to calculate error metrics of products with gauge measurements that are considered as ground truth. The multiplicative triple collocation (MTC) method has been demonstrated powerful in error quantification of precipitation products when ground truth is not known. This study applied MTC to evaluate five precipitation products in Germany: two raw satellite-based products (CMORPH and PERSIANN), one reanalysis product (ERA-Interim), one soil moisture–based product (SM2RAIN-ASCAT), and one gauge-based product (REGNIE). Evaluation was performed at the 0.5° daily spatial–temporal scales. MTC involves a log transformation of data, necessitating dealing with zero values in daily precipitation. Effects of 12 different strategies for dealing with zero values on MTC results were investigated. Seven different triplet combinations were tested to evaluate the stability of MTC. Results showed that different strategies for replacing zero values had considerable effects on MTC-derived error metrics, particularly for root-mean-square error (RMSE). MTC with different triplet combinations generated different error metrics for individual products. The MTC-derived correlation coefficient (CC) was more reliable than RMSE. It is more appropriate to use MTC to compare the relative accuracy of different precipitation products. Based on CC with unknown truth, MTC with different triplet combinations produced the same ranking of products as the traditional method. A comparison of results from MTC and the classic TC with additive error model showed the potential limitation of MTC in arid areas or dry time periods with a large ratio of zero daily precipitation.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Hongkai Gao, hkgao@geo.ecnu.edu.cn

1. Introduction

Precipitation is a critical component of the water cycle (Oki and Kanae 2006), maintaining terrestrial ecosystem stability and providing essential water resources for human society (Pastor et al. 2014), in terms of “green” water (soil moisture) and “blue” water (streamflow) (Schyns et al. 2015). Precipitation is characterized by significant spatial heterogeneity and temporal variability (Duan et al. 2016), leading to numerous water-related hazards, e.g., flooding, drought, and landslide (Dankers et al. 2014; Gao et al. 2018; Liu et al. 2018; Guzzetti et al. 2012). Moreover, precipitation is also a primary input for hydrological, climatological, and ecological models used to study and forecast numerous geophysical and biogeochemical processes (Bae et al. 2011; Duan et al. 2019a). Research in many scientific disciplines is therefore dependent on the quality of precipitation estimates.

Several established methods exist for estimating large-scale gridded precipitation products, including gauge based, remote sensing based, and model reanalysis. Gauge-based products are derived by interpolating direct measurements from in situ rain gauges. Remote sensing products are based on indirect measurements from ground-based radar or satellite sensors. Model-based reanalysis products combine meteorological model outputs with observations of geophysical parameters. A few state-of-the-art products merge different product types to produce a single, unified product. Comprehensive summary and discussion of different methods for precipitation estimation and available precipitation products can be found in many papers, e.g., Kidd and Huffman (2011), Tapiador et al. (2012), Massari et al. (2017), and Sun et al. (2018).

There is uncertainty associated with each method of estimating precipitation based on a variety of error sources. Numerous studies have investigated the sources and magnitudes of error in precipitation estimates (Cecinati et al. 2017; Chen et al. 2020; Maggioni et al. 2014; Mei et al. 2014, 2016, 2017; Tang et al. 2016). In situ gauge measurements of precipitation are frequently used as a proxy for true precipitation (Peleg et al. 2018; Tang et al. 2018), but the general lack of dense networks of rain gauge stations makes these point observations unable to sufficiently represent rainfall over larger areas. Rain gauges themselves also suffer from errors due to wind, precipitation type, and evaporation, depending on gauge type. Another source of errors in gauge-based products is the specific algorithm used to interpolate gridded values from point measurements. Ground-based radar estimates suffer from relatively high errors, primarily due to obstructions to the line of sight caused by mountains (Germann et al. 2006). Errors in satellite measurements are due to the algorithms that convert remotely measured signals into precipitation rates (Kidd and Huffman 2011; Xie and Arkin 1997), and other factors related to the indirect nature of the measurements. Finally, for model-based reanalysis products, error is dependent on the quality of the model (structure, spatial resolution, and parameterization) and the data assimilation technique, the representativeness of the observations, and the quality of a model’s physical parameterizations (Wang and Zeng 2012).

Considerable and continuous efforts have been made to generate more accurate precipitation products, leading to an increasing number of different products (Sun et al. 2018; Tapiador et al. 2012).Precipitation estimates have limited benefit without an evaluation of their uncertainty. Evaluation or validation activities are valuable for both product developers to identify issues and directions for improvements, and for end-users to understand the quality of products so as to select the suitable product (from the ranking of the errors of the precipitation products) for their specific applications (Duan et al. 2016). Despite several known deficiencies, gauge measurement is still the most direct and most accurate method at the gauge locations (Duan et al. 2016; Xie and Xiong 2011) and is often used with interpolation methods to obtain regional precipitation. Therefore, the most common method (termed as the traditional method in this study) for evaluation of different products is to calculate error metrics of evaluated products with respect to the gauge measurements (either point-based or interpolated) that are considered as ground truth. For gauge-sparse and ungauged regions, which occupies a majority of the planet’s surface, it is difficult or impossible to assess the uncertainty of precipitation products with the traditional method.

Triple collocation (TC) is a mathematical method to evaluate product error statistics without requiring the ground truth. The method was introduced by Stoffelen (1998) to model error associated with wind speed measurements, but it can be applied to the measurement of any geophysical variable using three or more collocated (i.e., target variable measured at the same time and place) datasets. The validity of this technique depends on each dataset being an independent estimate of the truth. Historically, TC has primarily been applied to ocean wind speed and wave height (Stoffelen 1998; Portabella and Stoffelen 2009), and rapidly extended to other variables, such as soil moisture products (Crow and van den Berg 2010; Gruber et al. 2016), leaf area index (LAI; Fang et al. 2012), sea ice thickness (Scott et al. 2014), atmospheric columnar integrated water vapor (Thao et al. 2014), sea surface temperature (Gentemann 2014), sea surface salinity (Rauthe et al. 2013), total water storage (van Dijk et al. 2014), and surface albedo (Wu et al. 2019).

In recent years, TC has been applied to precipitation products and enhanced to improve its applicability. Roebeling et al. (2012) demonstrated the viability of the method by estimating errors for three precipitation products in Europe. McColl et al. (2014) introduced the extended TC (ETC), which uses the same mathematical principles as TC to estimate the Pearson correlation coefficient (CC) between each dataset and the unknown truth. ETC provides a normalized metric to supplement RMSE, which is dependent on the magnitude of the underlying data. Alemohammad et al. (2015) evaluated four precipitation products in the United States using multiplicative triple collocation (MTC), which replaces the additive error model of ETC with a multiplicative model. Previous studies that analyzed uncertainty in precipitation measurements concluded that the multiplicative error model generally provides a more accurate characterization of precipitation error than the additive model (Ciach et al. 2007; Hossain and Anagnostou 2006; Tian et al. 2013; Villarini et al. 2009). Massari et al. (2017) found that TC with the additive model worked well and no advantage of MTC was observed, while Li et al. (2018) found that MTC performed better than TC in China. A few more recent studies have also applied MTC to precipitation products (Tang et al. 2020). MTC requires a log transformation of data, thus precipitation datasets need to be modified to remove or alter zero values before implementing MTC. Several different strategies for dealing with daily precipitation values that equal zero have been proposed and implemented in previous studies. Roebeling et al. (2012) and Li et al. (2018) replaced all zero values in their datasets with a very small (≪1) constant. They chose replacement values of 10−3 and 10−9, respectively. Massari et al. (2017) simply removed daily values of all three collocated products if any individual value equaled zero. Alemohammad et al. (2015) also removed all zero values from their evaluation, but only after aggregating data to a relatively coarse spatial (2° × 2°) and temporal (biweekly) resolution. However, the effects of different strategies for replacing zero values on MTC results have not been sufficiently investigated and compared, which motivated us to fill this research gap. Twelve different strategies were investigated in detail in this study.

A majority of existing studies that evaluate precipitation products were based on the traditional method. Since the MTC method is expected to be increasingly used, it is relevant and timely to compare the two methods to investigate to what extent their evaluation results can agree with each other. Germany was chosen as the focus area in this study because of its varied topography and the availability of rain gauge measurements from dense and well-designed networks. The gauge-based HYRAS-PRE (REGNIE) dataset (Rauthe et al. 2013), developed by the DWD (German Weather Service), was used as a reference to reliably perform the evaluation using the traditional method. Specifically, the objectives of this study were to 1) investigate the effects of different strategies for handling zero values and different triplet combinations on MTC results in a greater detail than existing studies and 2) compare both the traditional method and MTC for evaluation of several widely used precipitation products covering different product types (gauge based, satellite based, reanalysis, and soil moisture based).

2. Study area

The study domain includes the land area of Germany, ranging from 47.5° to 54.5°N latitude and from 6.5° to 15°E longitude. We divided the land area of Germany into spatial grids at 0.5° × 0.5° intervals. Grid cells were only included in the evaluation if less than 20% of their area lies outside Germany’s land border. The 3-arc-s resolution (~30 m) digital elevation model (DEM) of Germany from the Shuttle Radar Topography Mission (SRTM) was downloaded at http://www.opendem.info/download_srtm.html, and used to analyze the country’s varied terrain. Figure 1 shows the topography and precipitation climatology of Germany. Most of northern Germany has generally flat terrain, with elevation less than 100 m. The central part of the country is characterized by alternating hills and valleys as well as a few low mountain ranges. Mountainous terrain exists only in the very southern tip of the country, along the northern edge of the Alps, where elevations exceed 2000 m in some areas. According to the Köppen climate classification, the western half of Germany is mostly classified as temperate oceanic, while the eastern half is mostly humid continental (Peel et al. 2007). Isolated mountainous regions are classified as subarctic. Average precipitation generally decreases from west to east, with a stronger positive correlation with elevation. The northeast of Germany displays the lowest amount of precipitation while the largest amount of precipitation occurs in southern Germany. In terms of annual precipitation, most areas of Germany have small interannual variability with a standard deviation less than 120 mm yr−1, while western and southern Germany present larger interannual variability (Duan et al. 2019b).

Fig. 1.
Fig. 1.

Topography and precipitation climatology of Germany. (a) Digital elevation model (DEM). (b) Mean daily precipitation (2007–17) estimated by REGNIE. Data are shown at the study 0.5° grid scale in the lower left insets.

Citation: Journal of Hydrometeorology 22, 11; 10.1175/JHM-D-21-0049.1

3. Precipitation products

This study used five gridded precipitation products. They include one gauge-based gridded precipitation product, two raw satellite-based precipitation products (CMORPH and PERSIANN), one reanalysis product (ERA-Interim), and the satellite soil moisture–based product (SM2RAIN). The SM2RAIN product is widely considered as a truly independent source of precipitation that facilitates application of TC analysis (Massari et al. 2017). It is worth noting that in this paper the wording “satellite-based precipitation product” is used to refer to the precipitation products based on the top-down approach that generally estimates precipitation through the inversion of atmospheric signals. The SM2RAIN product is also a satellite-based precipitation product but it is based on the bottom-up approach that estimates precipitation from satellite soil moisture observations (Brocca et al. 2017), and thus we use the wording “satellite soil moisture–based precipitation product” to distinguish different products. For satellite-based precipitation products, besides raw data based on purely satellite observations, there are bias-corrected or calibrated products that are based on gauge-based precipitation products. They were not used in this study to maximize the independence of products for MTC analysis. Each of the selected precipitation products is briefly described below.

a. Gauge-based gridded precipitation product: REGNIE

HYRAS Precipitation Climatology (HYRAS-PRE) version 2.0, commonly referred to as REGNIE (Regionalisierte Nieederschläge), is a high-resolution, gridded precipitation dataset developed by DWD and BfG (German Federal Institute of Hydrology). It has a daily temporal resolution and a spatial resolution of 1 km. The dataset covers all of Germany and spans from 1931 to the present. Data generation is based on the REGNIE method, which combines multiple linear regression (MLR) of orographic factors (latitude, longitude, elevation, aspect, slope) with inverse distance weighting interpolation of precipitation anomalies from rain gauges. As of 2015, approximately 2200 DWD rain gauges were in operation, slightly more than half of which are automated hourly stations (Winterrath et al. 2016). This number changes slightly from year to year but has been relatively steady since 2007. The DWD’s network of quality-controlled gauges comprises one of the densest and most well-designed networks in the world (Paeth et al. 2017). Even though spatial coverage is very dense, the resolution of the REGNIE product is much finer than the average distance between gauges. REGNIE also assimilates data from several gauges located in the neighboring countries of Belgium, the Netherlands, Luxembourg, France, Switzerland, Austria, and the Czech Republic. But only a few of these stations lie inside the study area. A complete description and validation of REGNIE is provided by Rauthe et al. (2013). REGNIE data are available at https://opendata.dwd.de/climate_environment/CDC/grids_germany/daily/regnie/.

In this study, REGNIE was used as a reference to evaluate the error of other datasets using standard statistical means. This generally follows the widespread convention of using rain gauge measurements as a proxy for true precipitation. Since it is not possible to directly verify gridscale estimates with point observations, a gauge-based interpolation is a suitable choice of reference. Rauthe et al. (2013) reported catchment-averaged mean absolute errors (MAE) of less than 2.4 mm day−1 for all second-order catchments in Germany.This supports the use of REGNIE as a reference dataset, but spatial variations in error still exist due to inhomogeneous gauge distribution and complex terrain. Following the methodology described by Li et al. (2018), REGNIE is included in some TC triplets. Testing multiple triplet combinations serves to establish the stability of the method and allows for thorough comparison of the performance of all products, including REGNIE itself. Provided that all assumptions are met, a TC error estimate for any one dataset should not change regardless of which other two datasets are included in a triplet. In reality, the composition of a TC triplet does affect the results because the mathematical assumptions of TC are never fully satisfied.

b. Satellite-based precipitation products

1) CMORPH

CMORPH combines passive microwave (PM) observations from low orbiting satellites with infrared observations from geostationary satellites (Joyce et al. 2004). Instantaneous precipitation rates are derived from PM sensors on board three classes of satellites: NOAA polar-orbiting meteorological satellites, United States Defense Department meteorological satellites, and TRMM satellites. Simultaneously, infrared sensors on board GOES-8, GOES-10, Meteosat-8, Meteosat-5, and GMS-5 continuously measure the location and movement of precipitation-producing clouds. The CMORPH algorithm merges the precipitation “snapshot” with cloud propagation vectors to produce 30-min averaged precipitation rates. The resulting dataset has a highest spatial resolution of 8 km × 8 km and a 30-min temporal resolution. Coverage extends from 60°S to 60°N and spans from 1998 to the present. Products at different spatial (8 km and 0.25°) and temporal (30 min, 3 hourly, daily) resolutions are available at http://ftp.cpc.ncep.noaa.gov/precip/CMORPH_V1.0/. CMORPH-RAW daily precipitation product at the spatial resolution of 0.25° was used in this study.

2) PERSIANN

PERSIANN uses an adaptive artificial neural network (ANN) model and satellite infrared measurements to estimate precipitation (Hsu et al. 1997). The ANN allows for improvements based on the incorporation of additional independent precipitation estimates and can theoretically “train” the model based on a variety of input variables. Data are available from March 2000 to the present at 0.25° and hourly resolution. Spatial coverage extends from 60°S to 60°N. PERSIANN products at different temporal resolutions are available at the University of California, Irvine, Center for Hydrometeorology and Remote Sensing (CHRS) Data Portal: http://chrsdata.eng.uci.edu/. The daily product at 0.25° was used in this study.

c. Reanalysis product: ERA-Interim

ERA-Interim (ERA hereafter for conciseness) is a global atmospheric reanalysis product developed by the ECMWF. Atmospheric modeling and data assimilation are performed by the ECMWF Integrated Forecasting System (IFS), cycle 31r2 (Dee et al. 2011). Precipitation estimates are generated in 6-h intervals at the T255 spectral resolution, which corresponds to a spatial resolution at the surface of approximately 80 km (Berrisford et al. 2011). Data at the native resolution are interpolated to different spatial resolutions by the ECMWF. ERA-Int is often used to represent precipitation at the spatial resolution of 0.5° (Dong et al. 2019) and we followed this practice in the current study. The ERA data from 1979 to the present are available at http://apps.ecmwf.int/datasets/data/interim-full-daily/.

d. SM2RAIN product

SM2RAIN is a method of estimating precipitation based on changes in satellite soil moisture (SM) (Brocca et al. 2014). The method is based on the water balance equation, which relates rainfall to changes in soil storage, assuming other processes are neglected. A unique aspect of SM2RAIN derived precipitation estimates is that they are completely independent from other precipitation products, which in theory makes the product ideally suited to TC. Massari et al. (2017) demonstrated the successful application of SM2RAIN to the evaluation of global precipitation products using TC. The SM2RAIN method has been applied to different satellite moisture products to obtain different SM2RAIN precipitation products with varying spatial resolutions. For example, SM2RAIN-ASCAT precipitation product is a global daily precipitation dataset that uses the SM2RAIN algorithm and satellite soil moisture data derived from the Advanced Scatterometer (ASCAT) (Brocca et al. 2017). Various versions of SM2RAIN products are available at http://hydrology.irpi.cnr.it/download-area/sm2rain-data-sets/. The SM2RAIN-ASCAT daily global precipitation product at 0.5° spatial resolution available from January 2007 to June 2015 was used in this study. To be specific, the product used is the version SM2R_ASCAT_halfdegree_DR2015_EXT_02 (DOI: 10.13140/RG.2.1.4434.8563; Brocca et al. 2017). This version involved the calibration of parameters in the SM2RAIN model using partial data from gauge-based product GPCP (Global Precipitation Climatology Centre) at 1° spatial resolution (Brocca et al. 2014). However, Brocca et al. (2017) used GPCP as the benchmark to evaluate this SM2RAIN-ASCAT product, and particularly Fig. 2b of Brocca et al. (2017) showed that the correlation between SM2RAIN-ASCAT and GPCP for 5-day accumulated precipitation at 1° spatial resolution is quite low for Germany (correlation coefficient less than 0.4 for most areas). The product is referred to as SM2RAIN hereafter for conciseness.

4. Methodology

a. Triple collocation formulation

In this section, a simplified mathematical description of TC is first presented, followed by the modifications to the original formulation, which were used in this study. TC can be applied to measurements of geophysical variables to quantify the uncertainty in the measurements without a reference or the need for “ground truth”/true values (Dong et al. 2020a,b). It is based on an error model that relates estimates to true values. The affine error model (Zwieback et al. 2012), which is the most common in the literature, is described by
Ri=αi+βiT+εi,
where R is the estimated value of a variable, T is the true value, α is the additive error, β is the multiplicative error, and ε is the residual error. The subscript i denotes one of three or more collocated datasets that must be used in the calculation. This TC is also referred to as the classic TC with additive error model and more details on it can be found in Massari et al. (2017). The TC was also implemented in this study to compare with the MTC. More recently, the MTC with multiplicative error model was applied for the estimation of precipitation errors (Alemohammad et al. 2015):
Ri=aiTβieεi.
Taking the natural logarithm of Eq. (2) and making the substitutions αi = ln(ai), ri = ln(Ri), and t = ln(T), the equation simplifies to the standard linear form:
ri=αi+βit+εi.
Stoffelen (1998) proposed two mathematically equivalent procedures to solve for RMSE (root-mean-square error) using TC. His preferred method involves averaging the cross-multiplied differences between the datasets. Importantly, the datasets must first be rescaled based on an arbitrary reference dataset. The alternative approach, which was used in this study, is based on combinations of dataset covariances. In this formulation, data rescaling is not required. The procedure is described by the following equations (McColl et al. 2014):
σr12=C11C12C13C23,
σr22=C22C12C23C13,
σr32=C33C13C23C12,
where σri is the RMSE of the ri product, C is the covariance matrix (3 × 3) of the datasets, and the subscripts refer to the row and column numbers of the matrix. Because a log transformation of the data is performed using MTC, the resulting RMSE estimates do not have a physical meaning. To convert the results back to the units of the original data, e.g., millimeters per day (mm day−1), the following equation is used (Alemohammad et al. 2015):
σRi=μRiσri,
where σRi is the RMSE of the original data, σri is the RMSE of the log transformed data, and μRi is the mean of the original data.
RMSE is an absolute metric, which means that its magnitude is affected by the magnitude of the underlying data. Values of RMSE by themselves therefore provide an incomplete picture of the performance of different products. McColl et al. (2014) addressed this shortcoming through the development of extended TC (ETC), which modifies TC to generate Pearson correlation coefficients (CC) between each dataset and the unknown truth. CC is a normalized measure of covariance and in tandem with RMSE improves the utility of uncertainty assessments. The following equations describe ETC:
ρt,12=C12C13C11C23,
ρt,22=C12C23C22C13,
ρt,32=C13C23C33C12,
where ρt,i is the correlation between each dataset and the set of true values in logarithmic scale when MTC is applied.

The validity of TC depends on a series of assumptions (Gruber et al. 2016). First, TC requires a collection of three or more datasets (typically three) that estimate the same variable at the same time and place (collocation). It is assumed that the correlation between product signals is linear (linearity) and the mean and variance of their distributions do not change over time (stationarity). The mean residual error of each product, with respect to the unknown truth, is assumed to equal zero (zero expectation): E(e) = 0. Residual errors of different products should not be correlated (zero error cross correlation): Cov(ei, ej) = 0. Finally, the residual errors are also assumed to be uncorrelated with the unknown truth (error orthogonality): Cov(ei, t) = 0. In principle, if these assumptions are not met, the results of TC are likely to have a bias with respect to the true uncertainty. In practice, these assumptions are violated to some degree in nearly all applications. While this does not invalidate the method, the manner in which the assumptions are violated affects the reliability of results. The quadruple collocation can be used to test the existence of cross-correlated errors among product pairs with high chance of cross correlated errors. More details on this method can be found in a recent paper by Chen et al. (2021).

b. MTC implementation

The evaluated seven precipitation products differed in terms of spatial scale, duration, global coverage, and file format. All products were scaled to the same 0.5° spatial resolution by pixel averaging and to the same daily temporal resolution by aggregation, leading to 18 × 16 × 3103 matrices consisting of collocated precipitation estimates for each product. The matrix size corresponds to longitude (0.5°) × latitude (0.5°) × duration (days). Erroneous (e.g., negative) values were also removed for each product. Finally, using a triplet of three collocated precipitation datasets, the MTC was separately performed for each grid cell. The MTC in MATLAB code developed by Alemohammad et al. (2015) was used in this study. The code is available at https://github.com/HamedAlemo/MTC. The MATLAB code used to implement MTC relies on bootstrapping, i.e., sampling with replacement, to ensure a standardization of comparisons. Bootstrapping can also be used to assess the uncertainty of the results, but this assessment was not made in this study due to the extremely high consistency of repeated simulations. In the bootstrapping procedure, 1000 random bootstrap samples were chosen from the full time series length of collocated data. MTC was then applied to this random subset. This process was simulated 1000 times, with the averaged simulation values reported as results. Increasing both the number of bootstrap samples and the number of simulations was tested. Both changes resulted in increased processing time but only a negligible change in results.

c. Different triplet combinations for MTC

Previous studies have shown that violations of the TC assumptions cause a bias in the resulting error estimates (Yilmaz and Crow 2014; Zwieback et al. 2012). This can happen in a few different ways. Significant overlap exists in the observation techniques and equipment used to generate state-of-the-art precipitation products. Most satellite-based products, for example, use measurements from infrared and/or microwave sensors as the basis for their precipitation estimates. Reanalysis and gauge-based products also incorporate measurements from many of the same observation stations. In addition, many different products use gauge-based data for bias correction. The commonality of retrieval methods introduces a dependence between products and also between the errors of their estimates. This violates the assumption of zero error cross correlation, which causes a negative bias in TC estimates (Gruber et al. 2016; Massari et al. 2017; Yilmaz and Crow 2014). The fact that precipitation is a positive-only statistic introduces another dependence. At true precipitation values of (or near) zero, estimated values can (mostly) only be greater than or equal to the truth; their positive or zero errors are therefore correlated. This violates both the zero error cross correlation and error orthogonality assumptions. Precipitation estimates in many regions also frequently display climatological errors (seasonal systematic errors), which violates the assumption of stationarity. However, if all collocated datasets show the same pattern, which is likely for most precipitation products, the effect of nonstationarity is considered negligible (Gruber et al. 2016).

Recent studies found that violations of the zero error cross-correlation assumption were the dominant sources of bias in TC estimates (Alemohammad et al. 2015; Yilmaz and Crow 2014). A careful selection of datasets is therefore critical to ensure reliable TC results. Our initial analysis showed all triplets that included two satellite-based precipitation products (e.g., CMORPH-RAW and PERSIANN) would generate positive biased MTC results. Similar positive bias can be clearly observed in other studies (Massari et al. 2017). This is because the same type products (satellite-based precipitation products in this case) contain cross correlated errors due to their overlapping use of common input data and processing methods (Massari et al. 2017; Chen et al. 2021). These results reinforce the importance of carefully selecting products when using MTC to minimize violations of the underlying mathematical assumptions. To maximize independence of datasets, it is theorized that triplets containing three different types of precipitation products should yield the most reliable uncertainty estimates. For example, such a triplet could include one satellite-based product, one reanalysis product, and the SM2RAIN product or gauge-based interpolated products (Massari et al. 2017; Li et al. 2018). In this study, we applied MTC to seven different triplet combinations (triplet A–G, as listed in Table 1). For example, triplet A includes satellite-based precipitation product CMORPH, reanalysis product ERA and gauge-based product REGNIE; triplet B is the same as triplet A except REGNIE is replaced with SM2RAIN. Testing a diverse combination of products serves as a basis for evaluating the sensitivity of the MTC method to the composition of the triplet and also testing whether particular triplet combinations can be assumed to have independent errors or not.

Table 1.

Country-wise mean value of correlation coefficient (CC, in bold numeric values) for each product from the traditional method with REGNIE as the reference and the multiplicative triplet collocation (MTC) analysis using seven different triplet combinations (triplet A–G). The numeric values in parentheses are the country-wise mean values of the absolute relative difference (ARD) representing the relative difference in CC between the MTC and traditional method.

Table 1.

d. Different strategies for treatment of zero rainfall

MTC requires a log transformation of data, thus precipitation datasets need to be modified to remove or alter zero values before implementing MTC. This problem is magnified at the daily scale since locations in Germany receive no daily precipitation, on average, approximately one out of every three days according to the REGINE product. Removing all zero values is a quick and simple solution for preprocessing datasets for MTC. However, the downside to this procedure is a significantly reduced sample length and an artificial alteration to the true product signal. To mitigate these negative effects, the daily zero values can be replaced with a small constant value ≪ 1 mm. The final method proposed is to add a constant value to all daily values, not just the zero values. This preserves the shape of the signal and error distributions and maintains sample length. Values are shifted to the right along the natural logarithm curve, where its shape is more linear. This type of strategies allows for strict comparison of error metrics derived from MTC and those from traditional method as the same constant value can be added to the reference dataset. In summary, a total of 12 different strategies for handling zero precipitation values were tested in this study as detailed below.

  • 1)Remove if any daily value (of the three datasets) is zero.

  • 2)–7) Add a constant value to all daily values: the constants of 1, 10−1, 10−2, 10−3, 10−6, 10−9 were tested, respectively.

  • 8)–12) Replace daily zero values with a small constant value ≪ 1 mm: the constants of 10−1, 10−2, 10−3, 10−6, 10−9 were tested, respectively.

e. Evaluation metrics

MTC was performed using three products in various combinations chosen from seven different precipitation datasets. In each case, two statistics were used to characterize the error of precipitation products: RMSE and CC. These statistics were calculated using 1) MTC and 2) the traditional method using REGNIE as the reference or ground truth. To be precise, MTC estimates the absolute root-mean-square error and correlation with the unknown ground truth, whereas the traditional method calculates the root-mean-square difference and the correlation with respect to the reference dataset that is often assumed as ground truth. For the traditional method, RMSE is calculated with
RMSE=1Nn=1N(fnrn)2,
and CC is traditionally calculated with
CC=Nfnrn(fn)(rn)[Nfn2(fn)2][Nrn2(rn)2],
where N represents the number of samples, n represents the sample index, f represents the reference value, and r represents the observed value.

This study compared error results calculated with MTC to the results calculated with the traditional method. The same strategies as mentioned earlier for treatment of zero rainfall were performed before implementing traditional method to make a fair comparison between MTC and the traditional method. The agreement of each pair of results was assessed using several different metrics. Such a comparison was used to validate MTC in previous studies, e.g., Li et al. (2018) in which results from traditional method were considered as the reference, and if MTC results agreed with traditional method then MTC results were considered as reasonable. We followed that study to validate MTC results in the same manner in the comparison. For both RMSE and CC, the arithmetic mean values of all grid cell results were calculated and reported for comparison. The mean values by themselves, however, are not a reliable indicator of well-matched results since high and low deviations could theoretically average themselves out. It is therefore possible to have poorly matched grid cell by grid cell results and perfectly matched mean results. To enhance the comparison and account for this possibility, an assessment was made of the degree of “pattern” matching in the spatial distribution of values. Li et al. (2018) used the goodness of fit (GOF) as the CC between the two sets of results, where a high GOF value would indicate that the MTC method accurately captures regional variations in precipitation uncertainty. However, GOF also depends on the spatial variability, and a low GOF value can be obtained even when there are very small differences in values for two results. After analyzing different metrics, we found that the relative difference (RD) is more informative to characterize the agreement between MTC and traditional method on pixel basis. RD is calculated by dividing the difference between the MTC-derived error estimate (RMSE or CC) and traditional method derived one by the traditional result. A negative value of RD means that MTC underestimates the error compared to traditional method, while a positive RD means overestimation by MTC. The mean value of absolute values of the RD (ARD) from all pixels can well represent the overall agreement between two methods; the lower the ARD value (optimal 0 representing perfect agreement), the better MTC and traditional method agree with each other. During the comparison, we looked at the spatial maps of error metrics (RMSE and CC) values and the ARD values representing the difference between error metrics derived from the traditional method and MTC on the pixel basis. The country-wise mean values were also calculated by averaging all pixel values to facilitate the comparison. Comparison depends on a combined assessment of how well the two mean RMSE and CC values match each other and how close the two ARD metrics are to 0.

5. Results and discussion

a. Effects of the replacement of zero values on MTC results

Twelve different strategies for dealing with daily precipitation values equal to zero were evaluated. Each method was tested with several different triplet combinations to minimize the influence of any one dataset on the outcome. Figure 2 shows the country-wise mean values of the MTC derived metrics (CC and RMSE) and their differences (absolute value of relative difference, ARD) with metrics derived from the traditional method for each case using 12 different strategies for dealing with zero values. Figure 2 shows MTC results using the triplet B consisting of CMORPH-ERA and SM2RAIN for illustrative purpose, which produced results that were representative of the full range of product combinations. Different methods for dealing with zero precipitation values generated different MTC results for each of the triplets as represented by the MTC-derived metrics RMSE and CC. A spatial analysis shows that during the studied 3103 days the number of zero precipitation values varies substantially among different regions and different products: the range and mean of the number of zero precipitation are range 677–1981, average 1801 for CMORPH, range 1515–1824, average 1684 for PERSIANN, range 737–1065, average 903 for ERA, range 526–1062, average 867 for REGNIE, range 29–1754, average 670 for SM2RAIN. On average, ERA and REGNIE are more similar with the number of zero precipitation, while the two satellite-based precipitation products have much more zero values and SM2RAIN has less zero values. As a result, the two satellite-based precipitation products significantly underestimated the precipitation over Germany with mean daily precipitation of 1.0 mm day−1 for CMORPH and 1.4 mm day−1 for PERSIANN, while the mean daily precipitation was 2.1, 2.1, and 1.9 mm day−1 for ERA, REGNIE, and SM2RAIN, respectively. Therefore, the influence of the replacement of zero values on MTC results is magnified in Germany with quite large ratio of zero precipitation values. Such a significant impact of replacement of zero values complicates the application of MTC and makes it difficult to compare results with other studies as different studies tend to use different zero values replacement (Li et al. 2018; Roebeling et al. 2012). Specifically, the replacement values of 10−9 in Li et al. (2018) and 10−3 in Roebeling et al. (2012) did not produce optimal results in our case study, as clearly shown in Fig. 2. Therefore, we call for more studies to test our identified optimal treatment (adding a constant of 10−2 to all values) to check the differences in results in different study areas.

Fig. 2.
Fig. 2.

Results of 12 strategies for managing daily precipitation values equal to zero from the MTC analysis using the triplet B (CMORPH-ERA-SM2RAIN combinations). The four evaluation metrics are detailed in section 4e in the text. They are country-wise mean values for 1) MTC-derived RMSE, 2) MTC-derived CC, 3) the absolute relative difference (ARD) for RMSE between values derived from MTC and the traditional method, and 4) ARD for CC.

Citation: Journal of Hydrometeorology 22, 11; 10.1175/JHM-D-21-0049.1

One practical question arises: which method for replacing zero values should be used that produces the most accurate and consistent results. In previous studies, the performance of TC or MTC has often been evaluated in regions with dense rain gauge networks by comparing the TC-derived or MTC-derived metrics RMSE and CC with the same metrics derived from traditional methods in regions with dense rain gauge networks (e.g., Li et al. 2018). We followed this to compare the results of MTC with results calculated with traditional formulas using the REGNIE dataset as a reference (ground truth), but we also keep in mind that metrics from MTC and the traditional method should not be the same as the traditional method contains the uncertainty of the benchmark (REGNIE in this study). A close look at Fig. 2 shows that replacing daily zero values with 10−2 and adding a constant 10−2 to all values both yielded low ARD values for RMSE and CC, but adding a constant 10−2 to all values performed slightly better when taking the average of ARD values of RMSE and CC. Therefore, this method was chosen as the optimal method. Removing zero values from the datasets yielded very low CC values from the traditional method and thus lead to extremely large ARD values (>1) for CC compared to other strategies for dealing with zero values. Using a smaller value than 10−2 (10−3, 10−6, and 10−9 in this study) to add to all values or replace zero values increased the MTC-derived error estimate (RMSE) as well as the ARD values for both RMSE and CC.

b. Comparison of MTC and the traditional method

The results of MTC were compared to results calculated with the traditional method using the REGNIE dataset as a reference (ground truth), and following Li et al. (2018). This comparison serves as a way for validation of MTC. Adding a constant of 10−2 mm day−1 was applied for all comparisons, based on the analysis detailed in section 5a. Figure 3 shows spatial distribution of the metric CC derived from the traditional method and the MTC with seven different triplets (triplet A–G, detailed in Table 1). The country-wise mean values of the metric CC from spatial maps shown in Fig. 1 are presented in Table 1 to facilitate the comparison. Similarly, Fig. 4 shows the spatial distribution of the metric RMSE and Table 2 presents country-wise mean values of RMSE. Both figures and tables show that for all triplets MTC generally exhibits higher CC estimates than the traditional metric using the REGNIE dataset as the reference. This is consistent with previous studies, and since the reference data used in the traditional method (REGNIE) also contain errors, thus MTC generally would be expected to yield better metrics (Alemohammad et al. 2015; Li et al. 2018; Massari et al. 2017; Yilmaz and Crow 2014). MTC-derived CC values matched those calculated with the traditional method fairly well for all products despite the slight overestimation. In contrast, MTC-derived RMSE values showed relatively larger difference between MTC and the traditional method particularly for the CMORPH and PERSIANN products. For MTC, the conversion of RMSE from logarithmic scale to the unit of millimeters per day is based on the expansion of Taylor series and the mean daily precipitation is used, as described in Eq. (7) earlier in section 4a (Alemohammad et al. 2015). The converted RMSE would be unreliable if the precipitation product is consistently biased toward lower or higher values relative to the unknown truth. These two satellite-based precipitation products had a relatively large number of zero precipitation values and largely underestimated the precipitation across Germany (section 5a) and thus the mean daily precipitation biased toward lower value leading to unreliably low RMSE for MTC results. When the mean daily precipitation value is closer to the ground truth or reference data, the conversion by Eq. (7) would yield more reliable RMSE values. This is true for the ERA and SM2RAIN products for which MTC-derived RMSE values were much closer to the traditional method derived ones regardless of using different triplets. Taken together, MTC-derived CC values for individual products were more reliable than RMSE values considering the aforementioned possible issue with RMSE.

Fig. 3.
Fig. 3.

Spatial distribution of correlation coefficient for each precipitation product from the traditional method with REGNIE as reference and the MTC analysis using seven different triplet combinations (triplet A–G as detailed in Table 1).

Citation: Journal of Hydrometeorology 22, 11; 10.1175/JHM-D-21-0049.1

Fig. 4.
Fig. 4.

Spatial distribution of root-mean-square error (RMSE; mm day−1) for each product from the traditional method with REGNIE as reference and the MTC analysis using seven different triplet combinations (triplet A–G as detailed in Table 1).

Citation: Journal of Hydrometeorology 22, 11; 10.1175/JHM-D-21-0049.1

Table 2.

Mean value of root-mean-square error (RMSE; mm day−1, in bold numeric values) for each product from the traditional method with REGNIE as the reference and the multiplicative triplet collocation (MTC) analysis using seven different triplet combinations (triplet A–G). The numeric values in parentheses are the country-wise mean values of the absolute relative difference (ARD) representing the relative difference in RMSE between the MTC and traditional method.

Table 2.

The effect of seven different triplet combinations (triplet A–G shown in Table 1) on MTC results was further investigated to analyze the stability of the MTC method. Each of the satellite-based precipitation products CMORPH and PERSIANN had three MTC results based on different triplets. The reanalysis product ERA, soil moisture–based product SM2RAIN, and gauge-based product REGNIE had five MTC results based on five different triplet combinations. Figures 3 and 4 and Tables 1 and 2 show that overall MTC results with different triplets showed modest variability for all metrics CC and RMSE for each individual product. The agreement between MTC-derived metrics and the traditional method varied among different triplets, as shown from different values of ARD in Tables 1 and 2. For satellite-based precipitation products, we can observe that MTC yielded more consistent metric CC values for CMORPH than PERSIANN among using different triplets, and the agreement in the CC values between MTC and the traditional method was also higher. A larger effect of different triplet combinations on MTC results can be observed for PERSIANN; MTC with triplet C yielded closer CC values to the traditional method, while using other triplets (triplet D and F) MTC yielded much higher CC values across the entire Germany but the general pattern was similar with the northern area displaying lower CC than the southern area.

c. Ranking of the accuracy of precipitation products

Previous sections show the observable effects of different triplet combinations and different strategies for dealing with zero values on MTC derived RMSE and CC values. This suggests the magnitudes of error estimates from MTC may not always be comparable with the traditional method, and the RMSE is more vulnerable. It is worthwhile noting that MTC is useful for comparatively ranking the accuracy of different precipitation products in Germany. Ranking of products was based on the metric CC since it is a normalized statistic and not dependent on the magnitude of the estimated variable. As shown in Fig. 3 and Table 1, MTC with seven different triplets yielded consistent overall ranking of products. The order was as follows, from the best to worst in terms of mean value of correlation with the unknown truth from all pixels: 1) REGNIE, 2) ERA, 3) CMORPH, 4) PERSIANN, and 5) SM2RAIN. Metrics of traditional method using REGINE as the reference also showed the same ranking of products. It is worth stressing that MTC with triplets not including REGNIE (triplet B and D) can yield the same ranking of products as the traditional method. This suggest that MTC could be used to identify “best” available precipitation products for ungauged or poorly gauged areas where a good reference data like REGNIE is not available and the traditional method cannot be performed. Several other studies have evaluated the error of satellite products and/or reanalysis products (Wang and Zeng 2012; Duan et al. 2016) using traditional methods. Based on these studies, significant variability can be observed in terms of the calculated error of precipitation products, depending on the region of study. However, the relative ranking of precipitation products in previous studies matches the ranked order of products in Germany. It is worth noting that similar findings were reported by Li et al. (2018) in China using MTC method; overall the gauge-based product is the most accurate, the reanalysis product ERA is very close to gauge-based product, and the satellite precipitation product CMORPH is better than PERSIANN although they used the bias corrected version of these two satellite precipitation products. A similar finding was also reported by Massari et al. (2017) which showed the overall ranking with gauge-based product being the best and ERA better than satellite precipitation products for the contiguous United States. The aforementioned consistencies further demonstrate the reliability of MTC for comparatively ranking the accuracy of different precipitation products.

d. Comparison of MTC and the classic TC with additive error model

Considering the relatively large ratio of zero daily precipitation values in Germany and the effects of the replacement of zero values on MTC results, it is interesting to compare the performance of MTC and the classic TC with additive error model that does not need to deal with the zero values. The original daily precipitation from each of the five products were used to perform the classic TC analysis with the same seven different triplets. Similar to MTC, the TC yielded lower RMSE values for all products regardless of different triplets (results are not shown for conciseness), compared to the traditional method. The two satellite precipitation products had consistently higher TC-derived RMSE values than the other three products, and PERSIANN had higher error than CMORPH, which is consistent with the traditional method. For those two satellite precipitation products, the agreement between TC-derived and traditional method derived RMSE values were better compared to the MTC with much biased lower values. The REGNIE generally had lower RMSE values than ERA, similar to the pattern showed by MTC results in Fig. 4. However, for SM2RAIN, it had consistently high RMSE values derived from MTC but low RMSE values derived from TC. For triplet G, the TC yielded lower RMSE values for SM2RAIN than for REGNIE and ERA. The agreement in the derived RMSE values between the traditional method and MTC was much higher than TC for the SM2RAIN. Compared to the traditional method, both MTC and TC could yield different relative ranking of products in terms of derived RMSE values. For example, CMORPH could be identified as the second-best product by the MTC, while SM2RAIN could be even the best product by the TC when using triplet G. Similar finding on inconsistency in ranking of products in terms of the metric RMSE was reported in other studies (Alemohammad et al. 2015; McColl et al. 2014). The metric CC is thus often considered to provide more important information than RMSE (Li et al. 2018). Figure 5 shows the spatial distribution of the metric CC for each precipitation product from the traditional method with REGNIE as reference and the TC. Table 3 presents the country-wise mean value of CC from those spatial maps. It should be noted that we should not compare the absolute values of CC derived from the MTC and TC shown in Figs. 3 and 5, Tables 1 and 3, as the MTC-derived CC values were based on the log-transformed values after adding a constant value of 0.01 mm day−1 to all daily precipitation values. We should focus on comparing the relative similarity and difference in the degree of agreement with the traditional method for MTC-derived and TC-derived metrics. Similar to MTC results, TC generally yielded higher CC values than the traditional method regardless of the composition of the triplet, and there are certain effects of different triplets on the magnitude of TC-derived CC values. Considering the ranking of products, TC with different triplets and the traditional method yielded the same ranking with REGNIE being the best and ERA the second-best product. As shown in Fig. 5, the CC values derived from TC and the traditional method are generally in the range of 0.3–0.6 across Germany; this range is consistent with results from other studies, e.g., Massari et al. (2017) based on the TC with different satellite precipitation product TRMM 3B42RT, and Brocca et al. (2017). The relative lower accuracy of SM2RAIN in Germany is also in line with the finding of worse performance of SM2RAIN in northern high latitudes and continental Europe due to poor quality of ASCAT soil moisture retrievals there (Chen et al. 2021; Massari et al. 2017). By comparison, the ranking of products from MTC and TC was the same for the top three products (REGNIE, ERA and CMORPH), but the two least accurate products (PERSIANN and SM2RAIN) were ranked differently. Comparing the ARD values in Tables 3 and 1 showed that the agreement in the metric CC between the traditional method and the TC varied less for the PERSIANN and SM2RAIN products among different triplets than the MTC. For the ranking, PERSIANN was identified worse than SM2RAIN by the TC while MTC showed the opposite. Similar finding can be found in Table 1 of Massari et al. (2017) for the contiguous United States. Using the same triplet (ERA-SM2RAIN-CMORPH), both TC and MTC ranked ERA as the best product for the contiguous United States. The TC ranked SM2RAIN better than CMORPH, while MTC ranked the opposite (note that they removed all zero precipitation values for the MTC analysis). This suggests that zero treatment and further log transformation could significantly modify the original precipitation data particularly for products with large number of zero values, thereby possibly leading to a different ranking of certain products, compared to the ranking based on the original data. This points to a potential limitation on the application of MTC in evaluating daily precipitation in arid areas or dry time periods with a large number of zero daily precipitation. For those areas or time periods, the classic TC with additive error model can provide additional useful insights.

Fig. 5.
Fig. 5.

Spatial distribution of correlation coefficient for each precipitation product from the traditional method with REGNIE as reference and the classic triplet collocation (TC) analysis with additive error model using seven different triplet combinations (triplet A–G as detailed in Table 1).

Citation: Journal of Hydrometeorology 22, 11; 10.1175/JHM-D-21-0049.1

Table 3.

Mean value of correlation coefficient (CC; in bold numeric values) for each product from the traditional method with REGNIE as the reference and the classic triplet collocation (TC) analysis with additive error model using seven different triplet combinations (triplet A–G). The numeric values in parentheses are the country-wise mean values of the absolute relative difference (ARD) representing the relative difference in CC between the MTC and traditional method.

Table 3.

6. Summary and conclusions

Many different precipitation products have been developed in the past few years. Evaluation of these precipitation products is important for both product developers to identify issues and directions for improvements, and for end-users to understand the quality of products so as to select optimal products for specific applications. The most common method for evaluation (termed as the traditional method in this study) is to calculate error metrics of evaluated products with respect to the gauge measurements (either point-based or interpolated) that are considered as “ground truth.” The triple collocation (TC) method has been increasingly used to assess errors of different geoscience variables without the need for ground truth. The extended MTC method has been particularly identified by many previous studies as a reliable method for evaluation of precipitation products. This study applied MTC to five gridded precipitation products covering all product types (including gauged based, satellite based, reanalysis, soil moisture based) in Germany where there exists a dense rain gauge network. We compared error metrics of five precipitation products based on the traditional method and the MTC method. The influences of several factors on the MTC method were also investigated. Results from the MTC and the classic TC with additive error model were also compared and discussed.

Because MTC involves a log transformation of data, this necessitates a method of dealing with zero values in daily precipitation datasets. This study evaluated a comprehensive number of 12 different strategies for treatment of zero values, which were shown to have considerable effects on MTC derived metrics. Careful handling of zero precipitation values must be ensured to produce realistic metrics particularly RMSE for MTC. Adding a constant value of 10−2 mm to all values was found to produce the best MTC performance in our study area when all metrics were considered together. It is worth noting that different studies could use different zero replacement strategies for MTC, which complicates the application of MTC and direct comparison of results among different studies. Seven different triplet combinations were investigated in this study. Different triplet combinations have certain effects on the magnitude of MTC-derived metrics for the same individual product. MTC tended to yield higher CC estimates, compared to the traditional method. The magnitude of error metric RMSE from MTC should be treated with caution because it is reliable only when the mean daily precipitation of the individual product is close to the ground truth or reference data. It was found more appropriate to use MTC (also TC) to compare the relative accuracy of different precipitation products in terms of the error metric CC rather than RMSE. MTC with different triplets produced the same ranking order of precipitation products as the traditional methods. Overall, REGNIE was the most accurate product, followed by the reanalysis product ERA and then satellite-based precipitation CMORPH. The MTC and the TC with additive error model yielded different ranking of the two least accurate precipitation products (PERSIANN and SM2RAIN), suggesting that zero values treatment and further log transformation could largely modify the original precipitation data particularly for products with large percentage of zero values. This could lead to different ranking of certain products, compared to the ranking based on the original data. Therefore, MTC has potentially limited usefulness when evaluating daily precipitation in arid areas or dry time periods with a large number of zero daily precipitation. For those areas or time periods, the classic TC with additive error model can provide additional useful insights.

Continuous efforts are being made to improve existing precipitation products by using gauge measurements for calibration or bias correction, which will generally improve the accuracy of products but also increase the overlap in data sources. The resulting error cross correlation would lead to a bias in TC results. Several recent studies have attempted to address the problem of limited availability of independent products by developing methods that require only two independent products (Dong et al. 2019, 2020a) to estimate product errors (referred to as two-product approaches). These studies demonstrated that error estimates from the two-product approaches (e.g., the extended double instrumental variable algorithm, denoted as EIVD; Dong et al. 2020a) are only marginally less accurate than error estimates from the TC approach using three products. These two-product approaches should be further investigated as an alternative when three independent products are not available. A comprehensive analysis of effects of different spatial and temporal scales on the performance of TC with more different precipitation products at the global domain is an interesting future study to improve our understanding of influencing factors on the TC method. In addition, TC-based merging of multiple precipitation products provides a potential way to develop better regional and global precipitation products (Dong et al. 2020b). Such potential needs to be further investigated in future studies.

Acknowledgments

We thank the developers and providers of the various precipitation products evaluated in this study for making their products available. The study is funded by the National Natural Science Foundation of China (Grants 42071081; 41801036; 41911530191). Z.D. is grateful for the financial support from The Royal Physiographic Society of Lund, Sweden, and Lund University. We are grateful to the handling editors and reviewers for their constructive comments that greatly improved this manuscript. Author contribution statement: Zheng Duan: conceptualization, methodology, formal analysis, writing, project administration; Edward Duggan: conceptualization, methodology, formal analysis, writing; Cheng Chen: writing—review and editing; Hongkai Gao: conceptualization, writing—review and editing; Jianzhi Dong: writing—review and editing; Junzhi Liu: writing—review and editing. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

REFERENCES

  • Alemohammad, S. H., K. A. McColl, A. G. Konings, D. Entekhabi, and A. Stoffelen, 2015: Characterization of precipitation product errors across the United States using multiplicative triple collocation. Hydrol. Earth Syst. Sci., 19, 34893503, https://doi.org/10.5194/hess-19-3489-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ashouri, H., K.-L. Hsu, S. Sorooshian, D. K. Braithwaite, K. R. Knapp, L. D. Cecil, B. R. Nelson, and O. P. Prat, 2015: PERSIANN-CDR daily precipitation climate data record from multisatellite observations for hydrological and climate studies. Bull. Amer. Meteor. Soc., 96, 69–83, https://doi.org/10.1175/BAMS-D-13-00068.1.

    • Crossref
    • Export Citation
  • Bae, D. H., I. W. Jung, and D. P. Lettenmaier, 2011: Hydrologic uncertainties in climate change from IPCC AR4 GCM simulations of the Chungju Basin, Korea. J. Hydrol., 401, 90105, https://doi.org/10.1016/j.jhydrol.2011.02.012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berrisford, P., and Coauthors, 2011: The ERA-Interim Archive, Version 2.0. ERA Rep. Series 1, 23 pp., https://www.ecmwf.int/sites/default/files/elibrary/2011/8174-era-interim-archive-version-20.pdf.

  • Brocca, L., and Coauthors, 2014: Soil as a natural rain gauge: Estimating global rainfall from satellite soil moisture data. J. Geophys. Res. Atmos., 119, 51285141, https://doi.org/10.1002/2014JD021489.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brocca, L., and Coauthors, 2017: A review of the applications of ASCAT soil moisture products. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 10, 22852306, https://doi.org/10.1109/JSTARS.2017.2651140.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cecinati, F., M. A. Rico-Ramirez, G. B. M. Heuvelink, and D. Han, 2017: Representing radar rainfall uncertainty with ensembles based on a time-variant geostatistical error modelling approach. J. Hydrol., 548, 391405, https://doi.org/10.1016/j.jhydrol.2017.02.053.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, C., and Coauthors, 2020: Performance of multiple satellite precipitation estimates over a typical arid mountainous area of China: Spatiotemporal patterns and extremes. J. Hydrometeor., 21, 533550, https://doi.org/10.1175/JHM-D-19-0167.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, F., and Coauthors, 2021: Enhanced large-scale validation of satellite-based land rainfall products. J. Hydrometeor., 22, 245257, https://doi.org/10.1175/JHM-D-20-0056.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ciach, G. J., W. F. Krajewski, and G. Villarini, 2007: Product-error-driven uncertainty model for probabilistic quantitative precipitation estimation with NEXRAD data. J. Hydrometeor., 8, 13251347, https://doi.org/10.1175/2007JHM814.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crow, W. T., and M. J. van den Berg, 2010: An improved approach for estimating observation and model error parameters in soil moisture data assimilation. Water Resour. Res., 46, W12519, https://doi.org/10.1029/2010WR009402.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dankers, R., and Coauthors, 2014: First look at changes in flood hazard in the Inter-Sectoral Impact Model Intercomparison Project ensemble. Proc. Natl. Acad. Sci. USA, 111, 32573261, https://doi.org/10.1073/pnas.1302078110.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quart. J. Roy. Meteor. Soc., 137, 553597, https://doi.org/10.1002/qj.828.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dong, J., W. T. Crow, Z. Duan, L. Wei, and Y. Lu, 2019: A double instrumental variable method for geophysical product error estimation. Remote Sens. Environ., 225, 217228, https://doi.org/10.1016/j.rse.2019.03.003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dong, J., L. Wei, X. Chen, Z. Duan, and Y. Lu, 2020a: An instrument variable based algorithm for estimating cross-correlated hydrological remote sensing errors. J. Hydrol., 581, 124413, https://doi.org/10.1016/j.jhydrol.2019.124413.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dong, J., F. Lei, and L. Wei, 2020b: Triple collocation based multi-source precipitation merging. Front. Water, 2, 1, https://doi.org/10.3389/frwa.2020.00001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Duan, Z., J. Z. Liu, Y. Tuo, G. Chiogna, and M. Disse, 2016: Evaluation of eight high spatial resolution gridded precipitation products in Adige Basin (Italy) at multiple temporal and spatial scales. Sci. Total Environ., 573, 15361553, https://doi.org/10.1016/j.scitotenv.2016.08.213.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Duan, Z., and Coauthors, 2019a: Hydrological evaluation of open-access precipitation and air temperature datasets using SWAT in a poorly gauged basin in Ethiopia. J. Hydrol., 569, 612626, https://doi.org/10.1016/j.jhydrol.2018.12.026.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Duan, Z., and Coauthors, 2019b: Spatiotemporal analysis of nonlinear trends in precipitation over Germany during 1951–2013 from multiple observation-based gridded products. Int. J. Climatol., 39, 21202135, https://doi.org/10.1002/joc.5939.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fang, H. L., S. S. Wei, C. Y. Jiang, and K. Scipal, 2012: Theoretical uncertainty analysis of global MODIS, CYCLOPES, and GLOBCARBON LAI products using a triple collocation method. Remote Sens. Environ., 124, 610621, https://doi.org/10.1016/j.rse.2012.06.013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gao, H., H. Cai, and Z. Duan, 2018: Understanding the impacts of catchment characteristics on the shape of the storage capacity curve and its influence on flood flows. Hydrol. Res., 49, 90106, https://doi.org/10.2166/nh.2017.245.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gentemann, C. L., 2014: Three way validation of MODIS and AMSR-E sea surface temperatures. J. Geophys. Res. Oceans, 119, 25832598, https://doi.org/10.1002/2013JC009716.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Germann, U., G. Galli, M. Boscacci, and M. Bolliger, 2006: Radar precipitation measurement in a mountainous region. Quart. J. Roy. Meteor. Soc., 132, 16691692, https://doi.org/10.1256/qj.05.190.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gruber, A., and Coauthors, 2016: Recent advances in (soil moisture) triple collocation analysis. Int. J. Appl. Earth Obs. Geoinf., 45, 200211, https://doi.org/10.1016/j.jag.2015.09.002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guzzetti, F., A. C. Mondini, M. Cardinali, F. Fiorucci, M. Santangelo, and K. T. Chang, 2012: Landslide inventory maps: New tools for an old problem. Earth-Sci. Rev., 112, 4266, https://doi.org/10.1016/j.earscirev.2012.02.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hossain, F., and E. N. Anagnostou, 2006: A two-dimensional satellite rainfall error model. IEEE Trans. Geosci. Remote Sens., 44, 15111522, https://doi.org/10.1109/TGRS.2005.863866.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hsu, K., X. Gao, S. Sorooshian, and H. V. Gupta, 1997: Precipitation estimation from remotely sensed information using artificial neural networks. J. Appl. Meteor., 36, 11761190, https://doi.org/10.1175/1520-0450(1997)036<1176:PEFRSI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kidd, C., and G. Huffman, 2011: Global precipitation measurement. Meteor. Appl., 18, 334353, https://doi.org/10.1002/met.284.

  • Li, C., G. Tang, and Y. Hong, 2018: Cross-evaluation of ground-based, multi-satellite and reanalysis precipitation products: Applicability of the Triple Collocation method across Mainland China. J. Hydrol., 562, 7183, https://doi.org/10.1016/j.jhydrol.2018.04.039.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, W., F. Sun, W. H. Lim, J. Zhang, and Y. Zhang, 2018: Global drought and severe drought-affected populations in 1.5 and 2°C warmer worlds. Earth Syst. Dyn., 9, 267283, https://doi.org/10.5194/esd-9-267-2018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Maggioni, V., M. R. P. Sapiano, R. F. Adler, Y. D. Tian, and G. J. Huffman, 2014: An error model for uncertainty quantification in high-time-resolution precipitation products. J. Hydrometeor., 15, 12741292, https://doi.org/10.1175/JHM-D-13-0112.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Massari, C., W. Crow, and L. Brocca, 2017: An assessment of the performance of global rainfall estimates without ground-based observations. Hydrol. Earth Syst. Sci., 21, 43474361, https://doi.org/10.5194/hess-21-4347-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McColl, K. A., and Coauthors, 2014: Extended triple collocation: Estimating errors and correlation coefficients with respect to an unknown target. Geophys. Res. Lett., 41, 62296236, https://doi.org/10.1002/2014GL061322.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mei, Y. W., E. N. Anagnostou, E. I. Nikolopoulos, and M. Borga, 2014: Error analysis of satellite precipitation products in mountainous basins. J. Hydrometeor., 15, 17781793, https://doi.org/10.1175/JHM-D-13-0194.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mei, Y. W., E. I. Nikolopoulos, E. N. Anagnostou, D. Zoccatelli, and M. Borga, 2016: Error analysis of satellite precipitation-driven modeling of flood events in complex alpine terrain. Remote Sens., 8, 293, https://doi.org/10.3390/rs8040293.

    • Crossref
    • Export Citation
  • Mei, Y. W., E. N. Anagnostou, X. Y. Shen, and E. I. Nikolopoulos, 2017: Decomposing the satellite precipitation error propagation through the rainfall-runoff processes. Adv. Water Resour., 109, 253266, https://doi.org/10.1016/j.advwatres.2017.09.012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Oki, T., and S. Kanae, 2006: Global hydrological cycles and world water resources. Science, 313, 10681072, https://doi.org/10.1126/science.1128845.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Paeth, H., and Coauthors, 2017: An efficient model approach for very high resolution orographic precipitation. Quart. J. Roy. Meteor. Soc., 143, 22212234, https://doi.org/10.1002/qj.3080.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pastor, A. V., F. Ludwig, H. Biemans, H. Hoff, and P. Kabat, 2014: Accounting for environmental flow requirements in global water assessments. Hydrol. Earth Syst. Sci., 18, 50415059, https://doi.org/10.5194/hess-18-5041-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peel, M. C., B. L. Finlayson, and T. A. McMahon, 2007: Updated world map of the Koppen-Geiger climate classification. Hydrol. Earth Syst. Sci., 11, 16331644, https://doi.org/10.5194/hess-11-1633-2007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peleg, N., and Coauthors, 2018: Spatial variability of extreme rainfall at radar subpixel scale. J. Hydrol., 556, 922933, https://doi.org/10.1016/j.jhydrol.2016.05.033.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Portabella, M., and A. Stoffelen, 2009: On scatterometer ocean stress. J. Atmos. Oceanic Technol., 26, 368382, https://doi.org/10.1175/2008JTECHO578.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rauthe, M., H. Steiner, U. Riediger, A. Mazurkiewicz, and A. Gratzki, 2013: A Central European precipitation climatology – Part I: Generation and validation of a high-resolution gridded daily data set (HYRAS). Meteor. Z., 22, 235256, https://doi.org/10.1127/0941-2948/2013/0436.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Roebeling, R. A., E. L. A. Wolters, J. F. Meirink, and H. Leijnse, 2012: Triple collocation of summer precipitation retrievals from SEVIRI over Europe with gridded rain gauge and weather radar data. J. Hydrometeor., 13, 15521566, https://doi.org/10.1175/JHM-D-11-089.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schyns, J. F., A. Y. Hoekstra, and M. J. Booij, 2015: Review and classification of indicators of green water availability and scarcity. Hydrol. Earth Syst. Sci., 19, 45814608, https://doi.org/10.5194/hess-19-4581-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Scott, K. A., M. Buehner, and T. Carrieres, 2014: An assessment of sea-ice thickness along the Labrador coast from AMSR-E and MODIS data for operational data assimilation. IEEE Trans. Geosci. Remote Sens., 52, 27262737, https://doi.org/10.1109/TGRS.2013.2265091.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stoffelen, A., 1998: Toward the true near-surface wind speed: Error modeling and calibration using triple collocation. J. Geophys. Res., 103, 77557766, https://doi.org/10.1029/97JC03180.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, Q. H., and Coauthors, 2018: A review of global precipitation data sets: Data sources, estimation, and intercomparisons. Rev. Geophys., 56, 79107, https://doi.org/10.1002/2017RG000574.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tang, G. Q., Y. Z. Ma, D. Long, L. Z. Zhong, and Y. Hong, 2016: Evaluation of GPM Day-1 IMERG and TMPA Version-7 legacy products over Mainland China at multiple spatiotemporal scales. J. Hydrol., 533, 152167, https://doi.org/10.1016/j.jhydrol.2015.12.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tang, G. Q., A. Behrangi, D. Long, C. M. Li, and Y. Hong, 2018: Accounting for spatiotemporal errors of gauges: A critical step to evaluate gridded precipitation products. J. Hydrol., 559, 294306, https://doi.org/10.1016/j.jhydrol.2018.02.057.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tang, G. Q., M. P. Clark, S. M. Papalexiou, Z. Ma, and Y. Hong, 2020: Have satellite precipitation products improved over last two decades? A comprehensive comparison of GPM IMERG with nine satellite and reanalysis datasets. Remote Sens. Environ., 240, 111697, https://doi.org/10.1016/j.rse.2020.111697.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tapiador, F. J., and Coauthors, 2012: Global precipitation measurement: Methods, datasets and applications. Atmos. Res., 104–105, 7097, https://doi.org/10.1016/j.atmosres.2011.10.021.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thao, S., L. Eymard, E. Obligis, and B. Picard, 2014: Trend and variability of the atmospheric water vapor: A mean sea level issue. J. Atmos. Oceanic Technol., 31, 18811901, https://doi.org/10.1175/JTECH-D-13-00157.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tian, Y. D., and Coauthors, 2013: Modeling errors in daily precipitation measurements: Additive or multiplicative? Geophys. Res. Lett., 40, 20602065, https://doi.org/10.1002/grl.50320.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Dijk, A. I. J. M., L. J. Renzullo, Y. Wada, and P. Tregoning, 2014: A global water cycle reanalysis (2003-2012) merging satellite gravimetry and altimetry observations with a hydrological multi-model ensemble. Hydrol. Earth Syst. Sci., 18, 29552973, https://doi.org/10.5194/hess-18-2955-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Villarini, G., W. F. Krajewski, G. J. Ciach, and D. L. Zimmerman, 2009: Product-error-driven generator of probable rainfall conditioned on WSR-88D precipitation estimates. Water Resour. Res., 45, W01404, https://doi.org/10.1029/2008WR006946.

    • Search Google Scholar
    • Export Citation
  • Wang, A. H., and X. B. Zeng, 2012: Evaluation of multireanalysis products with in situ observations over the Tibetan Plateau. J. Geophys. Res., 117, D05102, https://doi.org/10.1029/2011JD016553.

    • Search Google Scholar
    • Export Citation
  • Winterrath, T., and Coauthors, 2016: Towards a radar-based high-resolution precipitation climatology for Germany – Methodology and potential areas of application. European Meteorological Society Conf., Trieste, Italy, European Meteorological Society, 24 pp., https://presentations.copernicus.org/EMS2016-375_presentation.pdf.

  • Wu, X., Q. Xiao, J. Wen, and D. You, 2019: Direct comparison and triple collocation: Which is more reliable in the validation of coarse-scale satellite surface albedo products. J. Geophys. Res. Atmos., 124, 51985213, https://doi.org/10.1029/2018JD029937.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P. P., and P. A. Arkin, 1997: Global precipitation: A 17-year monthly analysis based on gauge observations, satellite estimates, and numerical model outputs. Bull. Amer. Meteor. Soc., 78, 25392558, https://doi.org/10.1175/1520-0477(1997)078<2539:GPAYMA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xie, P. P., and A. Y. Xiong, 2011: A conceptual model for constructing high-resolution gauge-satellite merged precipitation analyses. J. Geophys. Res., 116, D21106, https://doi.org/10.1029/2011JD016118.

    • Search Google Scholar
    • Export Citation
  • Yilmaz, M. T., and W. T. Crow, 2014: Evaluation of assumptions in soil moisture triple collocation analysis. J. Hydrometeor., 15, 12931302, https://doi.org/10.1175/JHM-D-13-0158.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zwieback, S., K. Scipal, W. Dorigo, and W. Wagner, 2012: Structural and statistical properties of the collocation technique for error characterization. Nonlinear Processes Geophys., 19, 6980, https://doi.org/10.5194/npg-19-69-2012.

    • Crossref
    • Search Google Scholar
    • Export Citation
Save