• Aires, F., C. Prigent, F. Bernardo, C. Jiménez, R. Saunders, and P. Brunel, 2011: A Tool to Estimate Land-Surface Emissivities at Microwave frequencies (TELSEM) for use in numerical weather prediction. Quart. J. Roy. Meteor. Soc., 137, 690699, https://doi.org/10.1002/qj.803.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Amitai, E., C. L. Unkrich, D. C. Goodrich, E. Habib, and B. Thill, 2012: Assessing satellite-based rainfall estimates in semiarid watersheds using the USDA-ARS Walnut Gulch gauge network and TRMM PR. J. Hydrometeor., 13, 15791588, https://doi.org/10.1175/JHM-D-12-016.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carr, N., and et al. , 2015: The influence of surface and precipitation characteristics on TRMM Microwave Imager rainfall retrieval uncertainty. J. Hydrometeor., 16, 15961614, https://doi.org/10.1175/JHM-D-14-0194.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Garcia, M., C. D. Peters-Lidard, and D. C. Goodrich, 2008: Spatial interpolation of precipitation in a dense gauge network for monsoon storm events in the southwestern United States. Water Resour. Res., 44, W05S13, https://doi.org/10.1029/2006WR005788.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gebregiorgis, A. S., P.-E. Kirstetter, Y. E. Hong, N. J. Carr, J. J. Gourley, W. Petersen, and Y. Zheng, 2017: Understanding overland multisensor satellite precipitation error in TMPA-RT products. J. Hydrometeor., 18, 285306, https://doi.org/10.1175/JHM-D-15-0207.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goodrich, D. C., T. O. Keefer, C. L. Unkrich, M. H. Nichols, H. B. Osborn, J. J. Stone, and J. R. Smith, 2008: Long-term precipitation database, Walnut Gulch Experimental Watershed, Arizona, United States. Water Resour. Res., 44, W05S04, https://doi.org/10.1029/2006WR005782.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamada, A., and Y. N. Takayabu, 2016: Improvements in detection of light precipitation with the Global Precipitation Measurement Dual-Frequency Precipitation Radar (GPM DPR). J. Atmos. Oceanic Technol., 33, 653667, https://doi.org/10.1175/JTECH-D-15-0097.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hiebl, J., and C. Frei, 2018: Daily precipitation grids for Austria since 1961—Development and evaluation of a spatial dataset for hydroclimatic monitoring and modelling. Theor. Appl. Climatol., https://doi.org/10.1007/s00704-017-2093-x, in press.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hou, A. Y., and et al. , 2014: The Global Precipitation Measurement Mission. Bull. Amer. Meteor. Soc., 95, 701722, https://doi.org/10.1175/BAMS-D-13-00164.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., and et al. , 2017: NASA Global Precipitation Measurement (GPM) Integrated Multi-satellitE Retrievals for GPM (IMERG). Algorithm Theoretical Basis Doc., version 4.6, 28 pp., https://docserver.gesdisc.eosdis.nasa.gov/public/project/GPM/IMERG_ATBD_V4.6.pdf.

  • Iguchi, T., S. Seto, R. Meneghini, N. Yoshida, J. Awaka, M. Le, V. Chandrasekar, and T. Kubota, 2010: GPM/DPR level-2. Algorithm Theoretical Basis Doc., 72 pp., https://pmm.nasa.gov/sites/default/files/document_files/ATBD_GPM_DPR_n3_dec15.pdf.

  • Kann, A., I. Meirold-Mautner, F. Schmid, G. Kirchengast, J. Fuchsberger, V. Meyer, L. Tüchler, and B. Bica, 2015: Evaluation of high-resolution precipitation analyses using a dense station network. Hydrol. Earth Syst. Sci., 19, 15471559, https://doi.org/10.5194/hess-19-1547-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kidd, C., T. Matsui, J. Chern, K. Mohr, C. Kummerow, and D. Randel, 2016: Global precipitation estimates from cross-track passive microwave observations using a physically based retrieval scheme. J. Hydrometeor., 17, 383400, https://doi.org/10.1175/JHM-D-15-0051.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kidd, C., J. Tan, P.-E. Kirstetter, and W. A. Petersen, 2018: Validation of the version 05 level 2 precipitation products from the GPM Core Observatory and constellation satellite sensors. Quart. J. Roy. Meteor. Soc., https://doi.org/10.1002/qj.3175, in press.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kirchengast, G., T. Kabas, A. Leuprecht, C. Bichler, and H. Truhetz, 2014: WegenerNet: A pioneering high-resolution network for monitoring weather and climate. Bull. Amer. Meteor. Soc., 95, 227242, https://doi.org/10.1175/BAMS-D-11-00161.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kirstetter, P.-E., Y. Hong, J. J. Gourley, M. Schwaller, W. Petersen, and J. Zhang, 2013: Comparison of TRMM 2A25 products, version 6 and version 7, with NOAA/NSSL ground radar–based National Mosaic QPE. J. Hydrometeor., 14, 661669, https://doi.org/10.1175/JHM-D-12-030.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kirstetter, P.-E., Y. Hong, J. J. Gourley, Q. Cao, M. Schwaller, and W. Petersen, 2014: Research framework to bridge from the Global Precipitation Measurement Mission core satellite to the constellation sensors using ground-radar-based National Mosaic QPE. Remote Sensing of the Terrestrial Water Cycle, Geophys. Monogr., Amer. Geophys. Union, 61–79, https://doi.org/10.1002/9781118872086.ch4.

    • Crossref
    • Export Citation
  • Kirstetter, P.-E., Y. Hong, J. J. Gourley, M. Schwaller, W. Petersen, and Q. Cao, 2015: Impact of sub-pixel rainfall variability on spaceborne precipitation estimation: Evaluating the TRMM 2A25 product: Impact of sub-pixel rainfall variability on TRMM 2A25. Quart. J. Roy. Meteor. Soc., 141, 953966, https://doi.org/10.1002/qj.2416.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kummerow, C. D., D. L. Randel, M. Kulie, N.-Y. Wang, R. Ferraro, S. J. Munchak, and V. Petkovic, 2015: The evolution of the Goddard profiling algorithm to a fully parametric scheme. J. Atmos. Oceanic Technol., 32, 22652280, https://doi.org/10.1175/JTECH-D-15-0039.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Menne, M. J., I. Durre, R. S. Vose, B. E. Gleason, and T. G. Houston, 2012: An overview of the Global Historical Climatology Network-Daily database. J. Atmos. Oceanic Technol., 29, 897910, https://doi.org/10.1175/JTECH-D-11-00103.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O, S., U. Foelsche, G. Kirchengast, J. Fuchsberger, J. Tan, and W. A. Petersen, 2017: Evaluation of GPM IMERG Early, Late, and Final rainfall estimates using WegenerNet gauge data in southeastern Austria. Hydrol. Earth Syst. Sci., 21, 65596572, https://doi.org/10.5194/hess-21-6559-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O, S., U. Foelsche, G. Kirchengast, and J. Fuchsberger, 2018: Validation and correction of rainfall data from the WegenerNet high density network in southeast Austria. J. Hydrol., 556, 11101122, https://doi.org/10.1016/j.jhydrol.2016.11.049.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Petković, V., and C. D. Kummerow, 2015: Performance of the GPM passive microwave retrieval in the Balkan flood event of 2014. J. Hydrometeor., 16, 25012518, https://doi.org/10.1175/JHM-D-15-0018.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Petković, V., and C. D. Kummerow, 2017: Understanding the sources of satellite passive microwave rainfall retrieval systematic errors over land. J. Appl. Meteor. Climatol., 56, 597614, https://doi.org/10.1175/JAMC-D-16-0174.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Romanov, P., G. Gutman, and I. Csiszar, 2000: Automated monitoring of snow cover over North America with multispectral satellite data. J. Appl. Meteor., 39, 18661880, https://doi.org/10.1175/1520-0450(2000)039<1866:AMOSCO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Skofronick-Jackson, G., and et al. , 2017: The Global Precipitation Measurement (GPM) mission for science and society. Bull. Amer. Meteor. Soc., 98, 16791695, https://doi.org/10.1175/BAMS-D-15-00306.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tan, J., W. A. Petersen, and A. Tokay, 2016: A novel approach to identify sources of errors in IMERG for GPM ground validation. J. Hydrometeor., 17, 24772491, https://doi.org/10.1175/JHM-D-16-0079.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tan, J., W. A. Petersen, P.-E. Kirstetter, and Y. Tian, 2017: Performance of IMERG as a function of spatiotemporal scale. J. Hydrometeor., 18, 307319, https://doi.org/10.1175/JHM-D-16-0174.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Villarini, G., and W. F. Krajewski, 2007: Evaluation of the research version TMPA three-hourly 0.25° × 0.25° rainfall estimates over Oklahoma. Geophys. Res. Lett., 34, L05402, https://doi.org/10.1029/2006GL029147.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. International Geophysics Series, Vol. 100, Academic Press, 704 pp.

  • Wolff, D. B., and B. L. Fisher, 2009: Assessing the relative performance of microwave-based satellite rain-rate retrievals using TRMM ground validation data. J. Appl. Meteor. Climatol., 48, 10691099, https://doi.org/10.1175/2008JAMC2127.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, J., and et al. , 2016: Multi-Radar Multi-Sensor (MRMS) quantitative precipitation estimation: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 621638, https://doi.org/10.1175/BAMS-D-14-00174.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    (a) HSS at 0.2 mm h−1, (b) unconditional bias, (c) unconditional correlation, and (d) unconditional NRMSE of Ku against gauges as a function of lag time. The circles mark where the Ku–gauge comparison is closest to the perfect value according to each metric (section 3b).

  • View in gallery

    (a) HSS at 0.2 mm h−1, (b) unconditional bias, (c) unconditional correlation, and (d) unconditional NRMSE of GPROF against gauges as a function of lag time. The strong lines are for all GPROF values while the faint lines are for each sensor. The circles mark where the GPROF–gauge comparison is closest to the perfect value according to each metric (section 3b).

  • View in gallery

    POD, FAR, and HSS of the satellite estimates against (left) PCMK, (center) WEGN, and (right) WGEW for (a)–(c) a +5-min lag and (d)–(f) no lag at a rain/no-rain threshold of 0.2 mm h−1. The numbers to the right of each subplot indicate the sample size. The light gray bars provide a visual comparison of the magnitudes of the metrics. The GPROF category shows the metrics for the combined data from the five individual sensors.

  • View in gallery

    Conditional bias, conditional NRMSE, and conditional correlation of the satellite estimates against (left) PCMK, (center) WEGN, and (right) WGEW for (a)–(c) a +5-min lag and (d)–(f) no lag, using only hits at a threshold of 0.2 mm h−1. The numbers to the right of each subplot indicate the sample size. The light gray bars provide a visual comparison of the magnitudes and signs of the metrics. The GPROF category shows the scores for the combined data from the five individual sensors.

  • View in gallery

    Scatter diagram comparing rainfall intensities of Ku against (left) PCMK, (center) WEGN, and (right) WGEW for (a)–(c) a +5-min lag and (d)–(f) no lag. Red crosses indicate stratiform precipitation and blue dots indicate convective precipitation as determined by the Ku algorithm. Note the logarithm scales of the axes.

  • View in gallery

    Scatter density diagram comparing rainfall intensities of GPROF against (left) PCMK, (center) WEGN, and (right) WGEW for (a)–(c) a +5-min lag and (d)–(f) no lag. Note the logarithm scales of the axes. See Figs. S1–S3 for individual sensors.

  • View in gallery

    Scatter density diagram comparing the difference in GPROF from (a) PCMK, (b) WEGN, and (c) WGEW against the fraction of gauges that have a nonzero rain accumulation. The green line shows the mean error in each rain fraction bin at intervals of 0.1.

  • View in gallery

    Scatter density diagram comparing the difference in GPROF from (a) PCMK, (b) WEGN, and (c) WGEW against of the coefficient of variation in the gauges. The green line shows the mean error in each bin of coefficient of variation at intervals of 1.0.

  • View in gallery

    HSS as a function of GPROF and gauge thresholds for (a) PCMK, (b) WEGN, and (c) WGEW. The white crosses denote where the highest HSSs occur. Note that the thresholds range from 0.1 to 10 mm h−1 at intervals of 0.1 mm h−1.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 188 188 13
PDF Downloads 187 187 17

Evaluation of Global Precipitation Measurement Rainfall Estimates against Three Dense Gauge Networks

View More View Less
  • 1 Universities Space Research Association, Columbia, and NASA Goddard Space Flight Center, Greenbelt, Maryland
  • | 2 Earth Science Office, NASA Marshall Space Flight Center, Huntsville, Alabama
  • | 3 Wegener Center for Climate and Global Change, and Institute for Geophysics, Astrophysics and Meteorology, Institute of Physics, University of Graz, Graz, Austria
  • | 4 Southwest Watershed Research Center, Agricultural Research Service, USDA, Tucson, Arizona
  • | 5 Earth Sciences Field Support Office, NASA Wallops Flight Facility, Wallops Island, Virginia
© Get Permissions
Full access

Abstract

Precipitation profiles from the Global Precipitation Measurement (GPM) Core Observatory Dual-Frequency Precipitation Radar (DPR; Ku and Ka bands) form part of the a priori database used in the Goddard profiling algorithm (GPROF) for retrievals of precipitation from passive microwave sensors, which are in turn used as high-quality precipitation estimates in gridded products. As GPROF performs precipitation retrievals as a function of surface classes, error characteristics may be dependent on surface types. In this study, the authors evaluate the rainfall estimates from DPR Ku as well as GPROF estimates from passive microwave sensors in the GPM constellation. The evaluation is conducted at the level of individual satellite pixels (5–15 km) against three dense networks of rain gauges, located over contrasting land surface types and rainfall regimes, with multiple gauges per satellite pixel and precise accumulation about overpass time to ensure a representative comparison. As expected, it was found that the active retrievals from DPR Ku generally performed better than the passive retrievals from GPROF. However, both retrievals struggle under coastal and semiarid environments. In particular, virga appears to be a serious challenge for both DPR Ku and GPROF. The authors detected the existence of lag due to the time it takes for satellite-observed precipitation to reach the ground, but the precise delay is difficult to quantify. It was also shown that subpixel variability is a contributor to the errors in GPROF. These results can pinpoint deficiencies in precipitation algorithms that may propagate into widely used gridded products.

Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/JHM-D-17-0174.s1.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

This article is included in the Global Precipitation Measurement (GPM) special collection.

Corresponding author: Jackson Tan, jackson.tan@nasa.gov

Abstract

Precipitation profiles from the Global Precipitation Measurement (GPM) Core Observatory Dual-Frequency Precipitation Radar (DPR; Ku and Ka bands) form part of the a priori database used in the Goddard profiling algorithm (GPROF) for retrievals of precipitation from passive microwave sensors, which are in turn used as high-quality precipitation estimates in gridded products. As GPROF performs precipitation retrievals as a function of surface classes, error characteristics may be dependent on surface types. In this study, the authors evaluate the rainfall estimates from DPR Ku as well as GPROF estimates from passive microwave sensors in the GPM constellation. The evaluation is conducted at the level of individual satellite pixels (5–15 km) against three dense networks of rain gauges, located over contrasting land surface types and rainfall regimes, with multiple gauges per satellite pixel and precise accumulation about overpass time to ensure a representative comparison. As expected, it was found that the active retrievals from DPR Ku generally performed better than the passive retrievals from GPROF. However, both retrievals struggle under coastal and semiarid environments. In particular, virga appears to be a serious challenge for both DPR Ku and GPROF. The authors detected the existence of lag due to the time it takes for satellite-observed precipitation to reach the ground, but the precise delay is difficult to quantify. It was also shown that subpixel variability is a contributor to the errors in GPROF. These results can pinpoint deficiencies in precipitation algorithms that may propagate into widely used gridded products.

Supplemental information related to this paper is available at the Journals Online website: https://doi.org/10.1175/JHM-D-17-0174.s1.

© 2018 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

This article is included in the Global Precipitation Measurement (GPM) special collection.

Corresponding author: Jackson Tan, jackson.tan@nasa.gov

1. Introduction

Global observation of precipitation relies on spaceborne instruments, but high quality estimates from passive microwave (PMW) sensors are only available on low-Earth-orbiting satellites, each of which has limited coverage. To obtain a complete global estimate of precipitation in a timely fashion, observations from different sensors have to be converted to surface precipitation through a well-calibrated, parametric (i.e., not sensor specific) algorithm and stitched together in a consistent manner. In the Global Precipitation Measurement (GPM) mission (Hou et al. 2014; Skofronick-Jackson et al. 2017), the Dual-Frequency Precipitation Radar (DPR) and the GPM Microwave Imager (GMI) serve as the reference for a constellation of passive microwave sensors, from which observations of microwave radiance are converted to precipitation using the Goddard profiling algorithm (GPROF; Kummerow et al. 2015) and are then merged into the gridded global Integrated Multisatellite Retrievals for GPM (IMERG) dataset (Huffman et al. 2017).

Because of the uniformity and completeness of gridded datasets, IMERG is the product from the GPM mission that is of leading interest among users. However, as IMERG ingests estimates from GPROF, it inherits errors from products upstream. Indeed, errors in such merged products can often be traced back to the GPROF estimates from individual PMW sensors (e.g., Tan et al. 2016; Gebregiorgis et al. 2017). Errors in GPROF estimates may arise because of several factors, including the less accurate nature of the PMW observations, errors relating to the Bayesian inversion, inadequate representation of the ancillary conditions such as land surface category and surface temperature, and biases in the a priori database from DPR–GMI estimates. Such issues are a lesser concern for DPR because of the active, three-dimensional nature of its retrievals, but errors may still exist through factors such as an incorrect attenuation correction, an inaccurate reflectivity–precipitation relationship, and a nonuniform beam filling effect.

This study evaluates surface precipitation estimates from the Ku channel of DPR as well as GPROF over three dense gauge networks with highly contrasting surface types and rainfall regimes. The Pocomoke gauge network is deployed in the mid-Atlantic coast of the United States, where the mixture of the cold ocean surface and warm land surface often present a challenge for satellite remote sensing. The WegenerNet gauge network is located in the hilly Feldbach region in southeast Austria, where the orography in the area and dense vegetation are factors that may influence the accuracy of the retrievals. The Walnut Gulch Experimental Watershed (WGEW) gauge network is situated in the semiarid region of southeastern Arizona in the US, where the highly variable rainfall regime and dry conditions may hinder the performance of DPR and GPROF. All three networks have a high density of gauges and a high time resolution, which allow us to perform the evaluation at the level of individual pixels and to accumulate the precipitation precisely during the overpass time, respectively.

Through the juxtaposition of these three sets of results, we aim to elucidate how different environmental conditions can influence the performance of both active and passive satellite remote sensing of precipitation. We show that the coastal environment of the Pocomoke network proved to be a challenge for GPROF, while the semiarid conditions over the WGEW led to a large number of false alarms for both DPR Ku and GPROF, possibly due to the evaporation of falling precipitation. In addition, given that both sensors do not actually observe the signal at the surface, we aim to investigate the possibility of a lag between the satellite observation and the surface measurement. The results suggest that a positive lag likely exists, but a definitive value is elusive. We also aim to examine the effects of subpixel variability in the GPROF estimates and the optimal thresholds for raining events. We demonstrate that GPROF generally underestimated uniform rain in all three networks and overestimated highly variable rain over the Pocomoke and Walnut Gulch networks, with possible issues in retrieving light rain over these two networks.

2. Data

a. GPM DPR Ku and GPROF

The GPM Core Observatory satellite was launched on 27 February 2014. With the DPR in the Ka and Ku bands and the 13-channel conical-scanning GMI, it serves as a reference standard for calibrating the constellation of PMW sensors on board partner satellites in the GPM mission (Hou et al. 2014). Precipitation estimates from these satellites are merged into IMERG, the high-resolution gridded precipitation product from the NASA GPM mission that is expected to be widely used.

In this study, we are focusing on precipitation estimates from the GPM Core Observatory Ku-band radar (“Ku” hereafter) and PMW sensors in the GPM constellation based on the GPROF algorithm for the 3-yr period from April 2014 to March 2017. Both the Ku and GPROF estimates are level 2 datasets in the official NASA GPM products, containing derived geophysical parameters at the full resolution of individual sensor pixels. Both datasets are currently in version 5 (V05). The Ku-band radar is a cross-track scanning active instrument at 13.6 GHz with a swath width of 245 km. It has a surface horizontal resolution of about 5 km at nadir, with a vertical range resolution of 250 m sampled every 125 m. Its minimum detection threshold is 18 dBZ—corresponding to about 0.5 mm h−1—according to its initial design specifications, but postlaunch tests found that it can discriminate precipitation from instrument noise down to about 12 dBZ or 0.2 mm h−1 (Hamada and Takayabu 2016). The measured radar reflectivity is used to determine the three-dimensional structure of precipitation through reflectivity–precipitation rate relationships based on precipitation type (convective/stratiform) and phase (Iguchi et al. 2010). GPROF is a fully parametric, physically based retrieval algorithm that estimates the surface precipitation rate from the brightness temperature observed by passive microwave sensors flying on satellites in the GPM constellation (Kummerow et al. 2015). These constellation sensors are either conical-scanning sensors or cross-track sensors, with horizontal resolutions (instantaneous fields of view) ranging from about 5 to 17 km in the 89-GHz channel (though their effective fields of view may be different because of their scan strategies). This channel is the primary input used in PMW retrieval over land because of its sensitivity to the ice-scattering signal. GPROF performs a conditional Bayesian retrieval of precipitation by comparing the observed brightness temperature to a priori databases according to the surface class, surface temperature, and total precipitable water. Surface classes are defined using self-similar microwave emissivity at a monthly time scale from Special Sensor Microwave Imager observations (Aires et al. 2011), supplemented with daily snow maps from NOAA’s AutoSnow product (Romanov et al. 2000). These a priori databases match brightness temperatures with associated surface precipitation rates and hydrometeor profiles from the DPR–GMI combined product, supplemented with the Cloud Profiling Radar on board CloudSat for light precipitation.

The five passive microwave sensors evaluated in this study are the GMI on board the GPM Core Observatory; the Advanced Microwave Scanning Radiometer 2 (AMSR2) on board the Global Change Observation Mission–Water (GCOM-W1) satellite; the Special Sensor Microwave Imager/Sounder (SSM/IS) on board the DMSP F16, F17, F18, and F19 satellites; the Microwave Humidity Sounder (MHS) on board the NOAA-18, NOAA-19, MetOp-A, and MetOp-B satellites; and the Advanced Technology Microwave Sounder (ATMS) from the Suomi-NPP satellite. While data are available for the TRMM Microwave Imager on board TRMM, they are excluded from this analysis as the GPROF data are currently not in version 5. Our study will examine the GPROF estimates based on the sensor types, with the same sensors on different platforms grouped together (e.g., SSM/IS on F16, F17, F18, and F19).

The GPM Ku and GPROF data can both be downloaded at https://pmm.nasa.gov/data-access/. The Ku dataset used in this study comes from the “2A-Ku” research product while the GPROF dataset comes from the “2A-GPROF-constellation” product. Surface precipitation rates are given by the variables precipRateESurface and surfacePrecipitation, respectively. Additional variables used in this study are precipType (convective/stratiform/other distinction in Ku) and surfaceTypeIndex (surface class in GPROF), as well as standard geolocation variables.

b. Gauge networks

We use ground measurements from three gauge networks, all of which contain a large number of gauges in a small area. This high density of gauges provides us with multiple point measurements of precipitation for each satellite pixel, which enables the evaluation of individual pixels against a spatially representative ground reference. Moreover, the subpixel information allows us to examine spatial variability within the pixel. These gauges also report the precipitation at a very fine time interval (1 s, 1 min, and 5 min), which allow a precise temporal matching between the instantaneous satellite estimates and the gauges at overpass time.

The Pocomoke gauge network (PCMK) is deployed by the NASA GPM Ground Validation (GV) program near the NASA Wallops Flight Facility in the Eastern Shore region of Virginia as part of the Wallops GPM Precipitation Research Facility for the ground validation of satellite precipitation. The WegenerNet gauge network (WEGN) is set up by the Wegener Center of the University of Graz in the Feldbach region of Austria, with a key application being the long-term monitoring and validation of precipitation (Kirchengast et al. 2014; Kann et al. 2015; O et al. 2018; Hiebl and Frei 2018). The WGEW, established in 1953, is part of the USDA Agricultural Research Service (ARS) Long-Term Agroecosystem Research (LTAR) network and is operated by the USDA-ARS Southwest Watershed Research Center in Tucson and Tombstone, Arizona (Goodrich et al. 2008; Garcia et al. 2008).

Table 1 provides the key properties of the three gauge networks. The surface classes are identified by examining all GPROF pixels within the coordinates over the period of study (from April 2014 to March 2017). Note that for WGEW, the surface class indices are 4 and 5, corresponding to “high vegetation” and “moderate vegetation,” respectively1; however, given the shrubland landscape, it is perhaps more appropriate to describe the surface class of WGEW as “low vegetation.” It is unknown if this misclassification is due to inaccurate descriptors for the surface class indices or an erroneous identification of the surface type, though it should also be noted that the WGEW network is situated at about 110°W, at the edge of two 1° surface class grid boxes.

Table 1.

Key properties of the three gauge networks used as ground references in this study.

Table 1.

As the gauges can only measure liquid precipitation, we remove data when there is possible snowfall over the gauges. For removal of possible snow over PCMK, we use the Global Historical Climatology Network (GCHN)-Daily produced by NOAA (Menne et al. 2012) and identified days during which snow was reported (including trace amounts); we exclude data from days with detected snowfall and the day following these days (to account for melting hysteresis in the measurement). For WEGN, a data quality flag allows the removal of detected snow events. For WGEW, snow is a very rare occurrence. Data for PCMK can be accessed at https://gpm-gv.gsfc.nasa.gov/Gauge/index.html. Data for WEGN can be accessed at https://wegenernet.org/. Data for WGEW can be accessed at https://www.tucson.ars.ag.gov/dap/.

3. Approach

a. Matching satellite pixels to gauges

Because of the high density of our gauge networks, we will evaluate the Ku and GPROF estimates at the level of individual satellite pixels. We adopt this approach as opposed to averaging over a larger area (e.g., over the coverage of the gauge network) for three reasons. First, the Ku and GPROF algorithms perform the estimation of precipitation properties from the radar reflectivity and brightness temperature, respectively, at the level of the individual pixels. While surrounding pixel information may be used, for example, to identify the degree of spatial variability, the primary information that affects the precipitation estimate is within the pixel. Hence, a pixel-level comparison is a most direct evaluation of the algorithms. Second, averaging to a larger scale may lead to enhanced performance due to the cancellation of errors (e.g., Tan et al. 2017). Third, GPROF estimates are ingested into the gridded IMERG product by resampling onto the 0.1° grid using nearest neighbor interpolation, so the precipitation estimates in each IMERG grid box correspond to pixel-level precipitation information from GPROF.

The matching between the satellite pixels and the gauges is performed using the following method. Using the coordinates of each pixel, we identify gauges that are within 7.5 km (for GPROF) or 2.5 km (for Ku) of the satellite sensor geolocation point. This effectively assumes a circular footprint of diameter 15 and 5 km, respectively. To ensure area representativeness of these point measurements, we only consider matchups that have at least 12 gauges (for GPROF) or 6 gauges (for Ku) in each pixel. While this may give greater weight toward precipitation within the grid where gauges are more clustered, we expect subpixel spatial sampling biases to be reasonably minimal because of the coverage of the networks and uniformity of the gauges within the networks. Therefore, each matchup consists of a set of satellite variables (e.g., surface precipitation rate) and a set of gauge variables (e.g., precipitation rates of all matched gauges). Each satellite overpass may have multiple matchups, especially for the larger networks of WEGN and WGEW. Unless otherwise stated, we will compare the surface precipitation rate from Ku and GPROF against the average precipitation rates of all matched gauges.

To extract the precipitation rate from the gauges, we will need to select an accumulation time. On one hand, since the satellite retrievals are considered instantaneous, the gauge accumulation time should be as short as possible. On the other hand, gauges have measurement uncertainty, so the gauge accumulation time should be sufficiently long to ensure a “stable” gauge measurement. After some trial and error, we determine 5 min to be a reasonable balance between these two factors.

PCMK reports the time (to the precision of 1 s) of bucket tipping in the gauge with each tip representing 0.254 mm of rain, which we convert to rain rates at 1-min intervals. WGEW report the 1-min rain rates of individual rain events. We take the 5-min accumulation of these rain rates, centered on the overpass time. That is, we identify the minute during which the overpass occurs and average the rain rates at −2, −1, 0, +1, and +2 min. For example, if an overpass occurs at a time of 1211:25 UTC, we are averaging the rain rates from 1209:00 to 1213:59 UTC. For WEGN, the precipitation rates are recorded at 5-min intervals, so we just use the estimate of the 5 min that contains the overpass time. Taking the previous example of an overpass at 1211:25 UTC, we will use the estimate at 1210:00–1214:59 UTC. All rainfall accumulations (mm) are converted to rain rates (mm h−1) by dividing them by the accumulation time.

b. Metrics for evaluation

On top of scatter diagrams comparing rain intensities between the satellite estimate and the ground reference, we will employ a suite of metrics to summarize the performance in terms of the detection of rain and the representation of rain rates.

To evaluate rain detection, we first count the number of hits (both satellite and gauge are above a threshold; denoted as H), misses (satellite is below threshold while gauge passes the threshold; denoted as M), false alarms (satellite passes the threshold while gauge is below threshold; denoted as F), and correct negatives (both satellite and gauge are below threshold; denoted as C). Then, we can calculate the probability of detection (POD; also called hit rate), false alarm ratio (FAR), and Heidke skill score (HSS), defined as,
e1
e2
e3
where
e4
Here, He is the number of correct rain occurrences by chance, and N is the sample size (Wilks 2011). It may help the interpretation of these metrics to recall that is the number of pixels that are raining according to the gauge while is the number of pixels that are raining according to the satellite. POD is the fraction of raining pixels in the ground reference that the satellite estimate correctly identified; a perfect value is one. FAR is the fraction of raining pixels in the estimate that are false alarms; a perfect value is zero. HSS is a generalized skill score that quantifies the performance of rain detection against random chance; a perfect value is one and a value above zero indicates a performance better than random chance. We use a threshold of 0.2 mm h−1 for both Ku and GPROF to delineate between rain and no rain. (In section 4d, we will briefly examine if this threshold is reasonable.)
To evaluate the rain rates, we use bias, normalized root-mean-square error (NRMSE), and the Pearson correlation coefficient, defined as
e5
e6
e7
where and are the gauge precipitation rate and satellite precipitation rate, respectively; the overbar indicates overall mean of the gauges; and n is the number of samples. Bias quantifies, on average across the entire sample, how much higher or lower the estimate is relative to the reference; a perfect value is zero. NRMSE is the average deviation of an estimate from the reference in a relative sense across the sample; a perfect value is zero. Correlation characterizes the linear covariation between the satellite and estimate, that is, how close the scatter points fall along a straight line; a perfect value is one. These three metrics can be computed unconditionally, that is, using all values including zeros, or conditionally, that is, with just the hits ( ≥ threshold and ≥ threshold); we will use both unconditional and conditional metrics depending on the context of the analysis.

4. Results

a. Investigating the effects of lag

Over land, GPROF relies predominantly on the ice scattering signal of the precipitation. Therefore, in theory, there should be a lag from when the signal is observed aloft to when the precipitation corresponding to this signal reaches the surface, to the extent that the surface precipitation rate in the a priori database profiles reflects the column contribution of ice to the surface precipitation rate at the time of the observation. For Ku, while the active nature of the retrieval means that the three-dimensional structure of precipitation can be observed, the lowest useful bin of the signal is actually about 1–2 km above the surface due to ground clutter, with the precipitation rate beneath filled in through extrapolation from the lowest clutter-free gate based on the vertical profile of reflectivity. So Ku too should have a lag, but with a shorter delay. This section investigates the existence of such a lag due to fall delay.

To assess for different lag times, we add offsets, ranging from −10 to +30 min at 1-min intervals, to the overpass time, thus changing the gauge accumulation period, and compute the HSS, bias, correlation, and NRMSE for each lag time. In the section 3a example of a 1211:25 UTC overpass, a +7-min lag means the gauge accumulation period is centered upon 1218:25 UTC, resulting in the averaging of rain rates from 1216:00 to 1220:59 UTC. These four metrics reveal different aspects of the comparison, each with its strengths and drawbacks. HSS quantifies how well the satellite is able to detect rain, but it tells us nothing about the magnitude of the estimated rain beyond the predefined threshold. Bias quantifies the overall systematic error in the satellite estimates, but it contains no information on the random error or its dependence on rain intensity. Note that systematic errors may arise because of inaccurate calibration of the sensors or algorithms, so bias may not be a good candidate for this analysis. Correlation disregards any linear systematic error in the estimates, thus revealing (primarily) the random error. NRMSE is affected by both random and systematic errors. Here, HSS is calculated with a threshold of 0.2 mm h−1 as the rain/no-rain delineation; bias, correlation, and NRMSE are calculated unconditionally using all values, including zeros pairs (i.e., correct negatives). The inclusion of zero values in the computation of bias, correlation, and NRMSE allows these three metrics to account for misses and false alarms.

Figure 1 shows the results for Ku, while Fig. 2 shows the results for GPROF. Contrasting between the two figures, it is clear that the lower sample size in Ku is causing a significant variation in the metrics. Even so, it is worth noting, first of all, that all the optimal lag values, determined by the lag value for which the metrics are closest to their respective perfect values as defined in section 3b, are positive. That is, the best matches between Ku and the gauges according to these four metrics occur when the gauges are accumulated after the overpass has occurred. However, there is no consistency in the value of the optimal lag between the four metrics considered. For example, the optimal lag in PCMK compared to the other two networks is the lowest in HSS but highest in correlation. Furthermore, PCMK has a strong negative bias near 0-min lag, which also affects the NRMSE; this distorts where the optimal lag occurs for these two metrics. For PCMK, the optimal lag occurs at +4, +11, +6, and +13 min for these four metrics (HSS, bias, correlation, and NRMSE, respectively). WEGN has less fluctuation in the metrics because of the larger sample size, but its optimal lag still occurs at a wide range of values: +12, +29 (though above +16 min the bias is fluctuating about zero), +5, and +3 min. While WGEW has a coverage of similar extent to WEGN, the lower occurrence of rain leads to lower sample sizes and thus higher fluctuations in the metrics, such as the oscillations in bias. The optimal lag for WGEW occurs at +24, +11, +3, and +11 min.

Fig. 1.
Fig. 1.

(a) HSS at 0.2 mm h−1, (b) unconditional bias, (c) unconditional correlation, and (d) unconditional NRMSE of Ku against gauges as a function of lag time. The circles mark where the Ku–gauge comparison is closest to the perfect value according to each metric (section 3b).

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

Fig. 2.
Fig. 2.

(a) HSS at 0.2 mm h−1, (b) unconditional bias, (c) unconditional correlation, and (d) unconditional NRMSE of GPROF against gauges as a function of lag time. The strong lines are for all GPROF values while the faint lines are for each sensor. The circles mark where the GPROF–gauge comparison is closest to the perfect value according to each metric (section 3b).

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

For GPROF (Fig. 2), the considerably larger sample sizes suppress the fluctuations seen in Ku, though individual sensors (faint lines in Fig. 2) clearly exhibit strong variations. Notably for WEGN, the optimal lag is negative in three out of four metrics, though it should be pointed out that in some of these cases the metrics are relatively constant in value across different lag times. In particular, the variations in bias over WEGN and in HSS over WGEW are small, thus raising questions about the precision of the optimal lags from these two metrics. It is possible that the lower time resolution of 5 min in WEGN hinders the identification of the optimal lag through this approach. For PCMK, the optimal lag occurs at +6, +16, +26, and +26 min. For WEGN, the optimal lag occurs at +8, −5, −1, and −1 min. For WGEW, the optimal lag occurs at 0, +6, +12, and +11 min.

The absence of a consistent lag value between different metrics prevents us from objectively selecting an optimal lag value for subsequent analysis. Nevertheless, there are two notable points in Figs. 1 and 2: first, most of the optimal lag times are positive, and second, the lag times generally are within 20 min. This is roughly consistent with the lag times found in the TRMM Precipitation Radar (Amitai et al. 2012) as well as gridded satellite precipitation products such as TRMM Multisatellite Precipitation Analysis (Villarini and Krajewski 2007) and IMERG (O et al. 2017). Hence, in subsequent sections, we subjectively choose a nominal lag value of +5 min lag for both Ku and GPROF over all three networks. However, to provide an indication of the effects of lag, we will also include, in the next section, a “naïve” 0-min lag comparison. As we will see, adding a lag value generally changes the absolute but not the relative performance.

b. Evaluation of precipitation

This section evaluates the surface precipitation estimates of the Ku and GPROF sensors over the three networks by examining 1) their rain detection abilities, 2) their errors and correlations from the rain intensities, and 3) the rain intensities of individual pixels. Some of these results are similar to those used in Kidd et al. (2018). However, the focuses are different: in Kidd et al. (2018), the results from the gauges were used to support the results from ground-based radar and compare the overall performance of GPROF and Ku between western Europe and the United States; here, the gauge-based results are contrasted against each other with their specific surface classes playing a central role in their respective results.

Figure 3 shows the metrics for rain detection, POD, FAR, and HSS at a threshold of 0.2 mm h−1. Focusing first on the results with a +5-min lag imposed (Figs. 3a–c), it is clear that Ku generally performs best in terms of rain detection among all sensors. Over PCMK, it has the highest POD and lowest FAR, resulting in an HSS well above all GPROF sensors. Over WEGN, its POD is also high, but its FAR—while lower than most GPROF sensors—is just a bit higher than AMSR2. Nevertheless, it still has the highest HSS. Over WGEW, its POD is, surprisingly, lowest among all sensors, but it also has the lowest FAR, resulting in a high HSS that is comparable with AMSR2. The fact that Ku has the highest HSS across all sensors is a reflection of the strength of active retrievals over passive observations.

Fig. 3.
Fig. 3.

POD, FAR, and HSS of the satellite estimates against (left) PCMK, (center) WEGN, and (right) WGEW for (a)–(c) a +5-min lag and (d)–(f) no lag at a rain/no-rain threshold of 0.2 mm h−1. The numbers to the right of each subplot indicate the sample size. The light gray bars provide a visual comparison of the magnitudes of the metrics. The GPROF category shows the metrics for the combined data from the five individual sensors.

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

For GPROF, the individual passive sensors show a large variation in rain detection performance. In general, GMI has the highest POD over all networks and lowest FAR over PCMK. On the other hand, AMSR2 has the lowest FARs over WEGN and WGEW. As a result, GMI and AMSR2 both have the highest HSSs over the three networks. One possible cause of this superior performance in GMI and AMSR2 is the higher sampling resolution and instantaneous field of view of both sensors (e.g., 6 km for GMI and 5 km for AMSR2 at 89 GHz, compared to about 15 km for the other sensors). More broadly, conical-scanning sensors have a consistent Earth incidence angle, whereas the varying scan angles from cross-track sensors result in different footprint resolutions, atmospheric path, and polarization (Kidd et al. 2016), which may lead to less consistent performance. Indeed, it is for this reason that IMERG prioritizes conical-scanning sensors over cross-track sensors when there is more than one passive microwave observation in a grid box within the half-hour (Huffman et al. 2017). In our results, we see some suggestions of this (e.g., ATMS and MHS have the lowest HSS over PCMK), but it is not consistent throughout all gauge networks. A larger sample size is needed before we can confirm or dispute this statement with greater certainty.

Judging between the three gauge networks on their HSSs, there is a hierarchy of performance across all satellite estimates. Both Ku and GPROF are best over the dense vegetation of WEGN, poorer over the coastal region of PCMK—more so for GPROF than Ku—and worst over the semiarid environment of WGEW. One salient fact when comparing the results between the three networks is the alarmingly high FAR over WGEW (Fig. 3c), ranging between 0.56 and 0.80 for all sensors. This means that in the majority of the events in which the satellite observed at least 0.2 mm h−1 it is actually not raining (gauges < 0.2 mm h−1). In contrast, over PCMK and WEGN, the FAR is about 0.3 for Ku and 0.5 for GPROF. The median rain rates of Ku and GPROF during these false alarms over WGEW are between those of PCMK and WEGN (not shown); that is, the false alarm rain rates are not smaller over WGEW, just that there are more of them. We suspect that the semiarid conditions over WGEW are leading to a high occurrence of virga, in which the falling precipitation—measured by the satellites—evaporates before it can reach the gauges at the surface.

Now, contrasting the +5-min lag (Figs. 3a–c) to the 0-min lag (Figs. 3d–f), we can see that, in general, the performance is lower for the 0-min lag case over PCMK. This is especially true for Ku, which suffers a sharp drop in HSS from 0.70 to 0.55, which is already evident in Fig. 1. For WEGN and WGEW, the differences between +5- and 0-min lag are marginal and mixed for GPROF, but slight improvements can be seen upon the introduction of a lag for Ku. On the whole, introducing a lag time appears to lead to improved scores, implying the measurable existence of an actual lag time (though its magnitude is unclear). It is worth noting that, between the GPROF sensors, the relative performance remains similar; for example, GMI and AMSR2 still have the highest HSSs among all GPROF sensors.

Next, on the evaluation of rain intensity, Fig. 4 shows the biases, NRMSEs, and correlations of the hits, that is, conditional raining events in which both the satellite estimate and ground reference are at least 0.2 mm h−1. Unlike in section 4a, we exclude missed precipitation and false precipitation here because these aspects of the estimate have already been evaluated in Fig. 3. Focusing first on the +5-min lag results, Ku has among the lowest NRMSEs and highest correlations, but it has a strongly negative bias over PCMK and WGEW. Part of the reason, as we shall see, is due to the low sample sizes causing a high sensitivity of these metrics to outliers. (Remember that we are evaluating only the hits, so Fig. 4 has ~5% of the sample size of Fig. 3.) For GPROF, sensors both overestimate and underestimate the rain rates over PCMK, but consistently underestimate rain rates over WEGN and WGEW. The correlations are low over PCMK and WGEW, and even negative for ATMS. Based on supplemental Figs. 1–3 (hereafter Figs. S1–S3) in the online supplemental material, this negative correlation of ATMS is likely due to a low sample size.

Fig. 4.
Fig. 4.

Conditional bias, conditional NRMSE, and conditional correlation of the satellite estimates against (left) PCMK, (center) WEGN, and (right) WGEW for (a)–(c) a +5-min lag and (d)–(f) no lag, using only hits at a threshold of 0.2 mm h−1. The numbers to the right of each subplot indicate the sample size. The light gray bars provide a visual comparison of the magnitudes and signs of the metrics. The GPROF category shows the scores for the combined data from the five individual sensors.

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

Between the gauge networks, the satellite estimates are mostly better over WEGN than the PCMK and WGEW, with lower magnitude of biases, lower NRMSEs, and higher correlations. This result, consistent with that of rain detection, is likely a reflection of the more benign retrieval conditions over WEGN. The estimates over WGEW have larger magnitudes of biases and lower correlations, while the estimates over PCMK have larger magnitudes of biases (but mixed in sign), lower correlations, and higher NRMSEs. As with the case of rain detection, this worse performance over PCMK and WGEW is a result of the more challenging retrieval conditions over coastal and semiarid regions. The difference between PCMK and WGEW—similar magnitude of biases and correlations but not NRMSE—is probably caused by the generally higher intensity of the events over the coastal PCMK.

Contrasting between the +5-min lag and the 0-min lag, the inclusion of a lag time appears to return mixed results. For example, the bias for Ku improved over PCMK and WEGN but worsened over WGEW. On the other hand, all networks have better correlations for Ku once a lag is added, with WGEW showing a dramatic improvement from −0.26 to 0.60. This is consistent with Amitai et al. (2012), who found that the maximum correlation between TRMM Precipitation Radar intensity retrievals and WGEW occurred with a lag from +4 to +8 min. For GPROF as a whole, there is a slight decline in performance over PCMK and WEGN with the introduction of a lag time, though just as with Fig. 3, the relative performance between GPROF sensors is roughly preserved for all three networks. The ambiguous change in the performance on the quantification of rain rates and the increased performance in rain detection (Fig. 3) illustrates the difficulty of determining a lag time through the limited sample sizes of gauges.

The metrics in Fig. 4 summarize aspects of the performance of the estimates in terms of their rain rates. Here, we focus on direct comparisons of the rain rates themselves through scatter diagrams. Beginning with Ku (Fig. 5), over PCMK and WEGN, many of the Ku rain rates lie in the vicinity of the one-to-one line; over WGEW, there is a greater degree of scatter. However, it is also evident that there is a paucity of points over PCMK and WGEW, which can lead to a high sensitivity in the metrics. Indeed, using PCMK as an example, the two convective pixels with the highest intensities in the gauges underestimate the rain rate and contribute disproportionately to the bias (note that the log–log scale used in Fig. 5 suppresses the apparent differences at higher rain rates). In fact, these two points occur during the same overpass of an intense deep convective event (Fig. S4); removing these two points when calculating the bias increased it from −0.43 to 0.20. Hence, because of the low sample sizes, the metrics for Ku should be interpreted with caution. Comparing between the two lag times (Figs. 5a–c vs Figs. 5d–f), the general distribution of points remains similar except for WGEW, which is due to the more variable nature of precipitation over WGEW.

Fig. 5.
Fig. 5.

Scatter diagram comparing rainfall intensities of Ku against (left) PCMK, (center) WEGN, and (right) WGEW for (a)–(c) a +5-min lag and (d)–(f) no lag. Red crosses indicate stratiform precipitation and blue dots indicate convective precipitation as determined by the Ku algorithm. Note the logarithm scales of the axes.

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

Figure 6 shows scatter density diagrams of all GPROF points against the three networks. It is clear that, perhaps with the exception of WGEW, there is a greater spread in the points for GPROF than for Ku. There also appears to be a tendency for GPROF over PCMK and WGEW to “taper” off at low rain rates. Specifically, for gauge values below 1 mm h−1, GPROF estimates do not decline proportionally but remain at about 1 mm h−1. This may be a hint that GPROF has difficulties estimating precipitation rates below 1 mm h−1 over PCMK and WGEW, though it is not clear whether this is because of the surface conditions or the rainfall regimes. Between +5-min lag (Figs. 6a–c) and 0-min lag (Figs. 6d–f), the overall distributions of the points do not change much, which is reflected in the small magnitude of the differences in the metrics.

Fig. 6.
Fig. 6.

Scatter density diagram comparing rainfall intensities of GPROF against (left) PCMK, (center) WEGN, and (right) WGEW for (a)–(c) a +5-min lag and (d)–(f) no lag. Note the logarithm scales of the axes. See Figs. S1–S3 for individual sensors.

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

c. Impact of subpixel variability in GPROF

Different degrees of subpixel variability may affect the accuracy of rainfall retrievals. As such, this section examines the impact of subpixel variability in rainfall on the retrieved precipitation rate of GPROF. We exclude Ku from this analysis, despite the importance of understanding the effects of nonuniform beam filling on radar retrievals (e.g., Kirstetter et al. 2013, 2014, 2015), because of the low sample sizes.

Figure 7 shows the difference in precipitation rate (i.e., GPROF minus gauge) as a function of the fraction of gauges within each matchup that are raining (a lag of +5 min is applied). Here, we consider a gauge to be raining if it has a nonzero precipitation rate, as opposed to the 0.2 mm h−1 threshold used in section 4b. It should be noted that the difference is not independent of fraction; for example, if the gauge fraction is zero, then the average gauge rain rate is of course zero, so the difference cannot be negative. For all three networks, there is a clear and consistent trend between the rainfall error and raining fraction. At low rain fractions (i.e., “patchy” rain), GPROF tends to overestimate the rain rates over PCMK and WGEW, though only minimally over WEGN. At high rain fractions (i.e., uniform rain), GPROF tends to underestimate rain rates for all three networks. The crossover points vary with gauge network: about 0.6 for PCMK and about 0.25 for WEGN and WGEW.

Fig. 7.
Fig. 7.

Scatter density diagram comparing the difference in GPROF from (a) PCMK, (b) WEGN, and (c) WGEW against the fraction of gauges that have a nonzero rain accumulation. The green line shows the mean error in each rain fraction bin at intervals of 0.1.

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

Another way to look at the impact of subpixel variability is to examine the difference as a function of the coefficient of variation of the gauges. The coefficient of variation is simply the ratio of the standard deviation to the mean, and, unlike the rain fraction, it takes into account the variability in the nonzero intensities of the rain rates. At a low coefficient of variation (i.e., low variability), GPROF tends to underestimate the rain rates over all three networks (Fig. 8). At a high coefficient of variation (i.e., highly variable rain), GPROF overestimates the rain rates over PCMK and WGEW, but only marginally over WEGN. These results with the coefficient of variation are consistent with the results from the rain fraction, both demonstrating that GPROF overestimates highly variable rain and underestimates uniform rain over all three networks.

Fig. 8.
Fig. 8.

Scatter density diagram comparing the difference in GPROF from (a) PCMK, (b) WEGN, and (c) WGEW against of the coefficient of variation in the gauges. The green line shows the mean error in each bin of coefficient of variation at intervals of 1.0.

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

d. Rainfall detection at different thresholds

In sections 4a and b, we used a uniform threshold of 0.2 mm h−1 for the rain/no-rain delineation for both the satellite estimates and the ground reference. This effectively divides Figs. 5 and 6 into four quadrants with the horizontal and vertical lines at 0.2 mm h−1; the top-right quadrant contains the hits, the bottom-right quadrant contains the misses, the top-left quadrant contains the false alarms, and the bottom-left quadrant contains the correct negatives (note, however, that Figs. 5 and 6 do not show points below 0.1 mm h−1). Judging from Fig. 6c, it is possible that a higher threshold, or even different thresholds for GPROF/Ku and gauges, will lead to better characterization of rain detection. To investigate this possibility, we follow Wolff and Fisher (2009) and compute HSS as a function of both GPROF and gauge thresholds.

Figure 9 shows the HSS for different GPROF and gauge thresholds. While this is not a comparison of rain intensity, it is clear, as anticipated, that the HSS patterns bear a striking resemblance to Fig. 6. For example, the overestimation of GPROF below 1 mm h−1 over PCMK and WGEW results in a corresponding shift in HSS, with the highest HSS occurring well above the one-to-one line. This asymmetry between the GPROF threshold and gauge threshold means that PCMK and WGEW experience more false alarms than misses, which is reflected in the FAR and POD, respectively (Fig. 3).

Fig. 9.
Fig. 9.

HSS as a function of GPROF and gauge thresholds for (a) PCMK, (b) WEGN, and (c) WGEW. The white crosses denote where the highest HSSs occur. Note that the thresholds range from 0.1 to 10 mm h−1 at intervals of 0.1 mm h−1.

Citation: Journal of Hydrometeorology 19, 3; 10.1175/JHM-D-17-0174.1

One strategy in identifying a suitable threshold for rain is by finding the rain rate at which the HSS is a maximum (Wilks 2011). Based on this approach, the threshold for the gauges should be 0.1–0.2 mm h−1, which is reassuringly close to our choice of 0.2 mm h−1 in previous sections. However, this only applies to the gauges; over PCMK and WGEW, the maximum HSSs occur at a GPROF threshold of 0.7–0.8 mm h−1. This implies that the rainfall between 0.2 mm h−1 and these higher thresholds should be considered as nonraining events in order to achieve the best performance in rainfall detection; this may be a reflection of a potential limitation of the current GPROF algorithm in retrieving light rain over these two regions. In fact, in the case of WGEW, this discrepancy in thresholds between the gauge and GPROF may be an indication of virga: an observed 0.8 mm h−1 at the altitude of the ice-scattering signal is equivalent to a measured 0.2 mm h−1 on the ground, at least for rain detection purposes.

5. Conclusions

In an evaluation of level 2 precipitation datasets, we compared active retrievals from the Ku channel of the GPM DPR and GPROF estimates from the GPM constellation of PMW sensors against three dense gauge networks. These gauge networks have contrasting environments that affect the passive microwave retrieval of precipitation: PCMK is located in the mid-Atlantic coast of the United States; WEGN is located in the vegetated Feldbach region in Austria; and WGEW is located in the semiarid region of southeastern Arizona in the United States. The high density of gauges and the high time resolution of all three networks allow a spatially and temporally representative comparison between individual satellite pixels and the gauges.

We found that, as expected for active retrievals, Ku generally performed better than GPROF, both in terms of rain detection and rain intensity, but the low sample size in Ku resulted in metrics sensitive to outliers. There is a marked contrast in performance between the various networks, with the best performance over the densely vegetated WEGN, a mixed performance over the radiometrically complex coast of PCMK, and the worst performance over the semiarid WGEW. In particular, both Ku and GPROF have an unusually high false alarm ratio over WGEW, which we attribute to the possible occurrence of virga. Another factor that may have contributed to the poorer performance of GPROF over WGEW is the coarse identification of surface type at 1°, in which the semiarid environment of WGEW is assigned to the “high vegetation” and “moderate vegetation” classes. In a lag analysis to detect the delay it takes for satellite-observed precipitation at the altitude of the ice-scattering signal to reach the gauges on the ground, we found a positive lag, consistent with the notion of a delay. Although the lags are mostly positive and generally less than 20 min, there is no consensus on the precise lag value between either the metrics considered or the networks. We also demonstrated a dependence of the errors between the satellite estimates and the ground measurements on the subgrid pixel variability, with an underestimation of uniform precipitation and an overestimation of spatially variable precipitation. An investigation into the threshold for rain uncovered an asymmetry between GPROF and the gauges over PCMK and WGEW—the maximum HSS occurring at about 0.2 mm h−1 for the gauges but at 0.7–0.8 mm h−1 for GPROF—possibly reflecting a limitation in retrieving light rain over these two regions. Note that these conclusions are based on V05 of the datasets and thus may change with future refinements in the algorithm.

The decreased performance of the PMW estimates over the semiarid WGEW is consistent with the results from Carr et al. (2015), who compared GPROF estimates (albeit from an older version) from the TRMM Microwave Imager against ground-based radar. They attributed the increased false alarms over more arid surfaces to the failure of the GPROF algorithm to distinguish between the microwave emission from the surface and the ice-scattering signal of the hydrometeors. While this may play a role in our results, the fact that Ku also suffers the same problem suggests another factor at play, which we conjecture as the presence of virga. Carr et al. (2015) also found that the bias (mean relative error therein) of GPROF is positive when the fraction of the pixel filled is low; this is again consistent with our results in Fig. 7. However, surface type is not the only factor affecting the performance of passive microwave retrieval algorithms. For example, Petković and Kummerow (2015) and Petković and Kummerow (2017) found that GPROF performance is also linked to the precipitation regime, system structure, and level of organization. Unfortunately, the comparatively small coverage of gauge networks is ill suited to examine these aspects of GPROF performance as compared to ground-based radars (see, e.g., Kidd et al. 2018).

The use of gauge networks necessarily constrains the spatial coverage of our study, restricting us to only a few surface types (and none over the ocean) and limiting the sample sizes of our comparisons, the latter of which is a recurring issue. Indeed, the difficulty in identifying a definitive lag value is an indication that this aspect of satellite retrievals is best studied in combination with other instruments. Another limitation of these three networks is the inability (at least for the majority of the gauges) to measure frozen precipitation. Nonetheless, the results herein may be of value to users and developers of these level 2 datasets and may provide directions for future research. For example, our conjecture that virga is responsible for a considerable fraction of false alarms can be investigated through other approaches (e.g., using range-height indicator scans from ground-based radar); if verified, this may be addressed in the algorithm by taking into account variables such as relative humidity. On the other hand, the characterization of lag due to fall delay can be examined with larger datasets (such as the 2-min Multi-Radar Multi-Sensor product; Zhang et al. 2016) and may be corrected for in downstream datasets or taken into account by users of precipitation products.

The vast majority of evaluation studies of satellite precipitation are on gridded datasets such as IMERG. However, as many gridded datasets ingest level 2 PMW estimates from low-Earth-orbit satellites, errors in the level 2 algorithms can propagate into the gridded estimates. Therefore, an evaluation of level 2 datasets can more precisely identify specific deficiencies in the algorithms. For example, we are able to connect the performance of GPROF and Ku directly to different land surfaces, whereas a similar exercise with IMERG produces results that are convoluted by sensor intercalibration, IR inputs, and gauge adjustment. Similarly, the precise observation time in level 2 datasets can, in principle, give a more well-defined quantification of lag associated with fall delay, whereas similar analyses with gridded datasets are limited by their time resolutions and ambiguity due to different sources of estimates. Therefore, while the evaluation of level 2 datasets can be more challenging because of the nonregular, nongridded nature of the pixels, it can provide a more targeted and thus more informative evaluation of satellite precipitation.

Acknowledgments

J.T. is supported by an appointment to the NASA Postdoctoral Program at Goddard Space Flight Center, administered by Universities Space Research Association through a contract with NASA (NNH15CO48B). J.T., W.A.P., and D.B.W. acknowledge support from the PMM Science Team funding (NNH15ZDA001N-PMM). W.A.P. also acknowledges support from the GPM Mission (Project Scientist, Gail S.-Jackson, and GV Systems Manager, Mathew Schwaller). G.K. acknowledges the WegenerNet team, in particular senior engineer Jürgen Fuchsberger and field coordinator Christoph Bichler, for the excellent joint work on the operations, maintenance, and continuous improvement of the network and its data products. D.B.W. acknowledges support from the GPM Mission (GPM Program Scientist, Dr. Ramesh Kakar). DPR and GPROF data were provided by the NASA Goddard Space Flight Center’s PMM and PPS teams, which develop and compute DPR and GPROF as a contribution to GPM, and archived at the NASA GES DISC. We thank Jianxin Wang for the data processing of the PCMK data. The WegenerNet is funded by the Austrian Ministry for Science and Research, the University of Graz, the state of Styria, and the city of Graz; detailed information is found at www.wegcenter.at/wegenernet, and the data are available through the WegenerNet data portal (www.wegenernet.org). The USDA-ARS Southwest Watershed Research Center’s development and maintenance of the Walnut Gulch Experimental Watershed research facilities and diligent long-term collection of high-quality hydrologic data by its staff are gratefully acknowledged. The authors thank three anonymous reviewers whose comments and suggestions have refined the manuscript. The codes used to perform the analysis can be accessed online (at https://github.com/JacksonTanBS/GPROF-gauge/).

REFERENCES

  • Aires, F., C. Prigent, F. Bernardo, C. Jiménez, R. Saunders, and P. Brunel, 2011: A Tool to Estimate Land-Surface Emissivities at Microwave frequencies (TELSEM) for use in numerical weather prediction. Quart. J. Roy. Meteor. Soc., 137, 690699, https://doi.org/10.1002/qj.803.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Amitai, E., C. L. Unkrich, D. C. Goodrich, E. Habib, and B. Thill, 2012: Assessing satellite-based rainfall estimates in semiarid watersheds using the USDA-ARS Walnut Gulch gauge network and TRMM PR. J. Hydrometeor., 13, 15791588, https://doi.org/10.1175/JHM-D-12-016.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carr, N., and et al. , 2015: The influence of surface and precipitation characteristics on TRMM Microwave Imager rainfall retrieval uncertainty. J. Hydrometeor., 16, 15961614, https://doi.org/10.1175/JHM-D-14-0194.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Garcia, M., C. D. Peters-Lidard, and D. C. Goodrich, 2008: Spatial interpolation of precipitation in a dense gauge network for monsoon storm events in the southwestern United States. Water Resour. Res., 44, W05S13, https://doi.org/10.1029/2006WR005788.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gebregiorgis, A. S., P.-E. Kirstetter, Y. E. Hong, N. J. Carr, J. J. Gourley, W. Petersen, and Y. Zheng, 2017: Understanding overland multisensor satellite precipitation error in TMPA-RT products. J. Hydrometeor., 18, 285306, https://doi.org/10.1175/JHM-D-15-0207.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goodrich, D. C., T. O. Keefer, C. L. Unkrich, M. H. Nichols, H. B. Osborn, J. J. Stone, and J. R. Smith, 2008: Long-term precipitation database, Walnut Gulch Experimental Watershed, Arizona, United States. Water Resour. Res., 44, W05S04, https://doi.org/10.1029/2006WR005782.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamada, A., and Y. N. Takayabu, 2016: Improvements in detection of light precipitation with the Global Precipitation Measurement Dual-Frequency Precipitation Radar (GPM DPR). J. Atmos. Oceanic Technol., 33, 653667, https://doi.org/10.1175/JTECH-D-15-0097.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hiebl, J., and C. Frei, 2018: Daily precipitation grids for Austria since 1961—Development and evaluation of a spatial dataset for hydroclimatic monitoring and modelling. Theor. Appl. Climatol., https://doi.org/10.1007/s00704-017-2093-x, in press.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hou, A. Y., and et al. , 2014: The Global Precipitation Measurement Mission. Bull. Amer. Meteor. Soc., 95, 701722, https://doi.org/10.1175/BAMS-D-13-00164.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huffman, G. J., and et al. , 2017: NASA Global Precipitation Measurement (GPM) Integrated Multi-satellitE Retrievals for GPM (IMERG). Algorithm Theoretical Basis Doc., version 4.6, 28 pp., https://docserver.gesdisc.eosdis.nasa.gov/public/project/GPM/IMERG_ATBD_V4.6.pdf.

  • Iguchi, T., S. Seto, R. Meneghini, N. Yoshida, J. Awaka, M. Le, V. Chandrasekar, and T. Kubota, 2010: GPM/DPR level-2. Algorithm Theoretical Basis Doc., 72 pp., https://pmm.nasa.gov/sites/default/files/document_files/ATBD_GPM_DPR_n3_dec15.pdf.

  • Kann, A., I. Meirold-Mautner, F. Schmid, G. Kirchengast, J. Fuchsberger, V. Meyer, L. Tüchler, and B. Bica, 2015: Evaluation of high-resolution precipitation analyses using a dense station network. Hydrol. Earth Syst. Sci., 19, 15471559, https://doi.org/10.5194/hess-19-1547-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kidd, C., T. Matsui, J. Chern, K. Mohr, C. Kummerow, and D. Randel, 2016: Global precipitation estimates from cross-track passive microwave observations using a physically based retrieval scheme. J. Hydrometeor., 17, 383400, https://doi.org/10.1175/JHM-D-15-0051.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kidd, C., J. Tan, P.-E. Kirstetter, and W. A. Petersen, 2018: Validation of the version 05 level 2 precipitation products from the GPM Core Observatory and constellation satellite sensors. Quart. J. Roy. Meteor. Soc., https://doi.org/10.1002/qj.3175, in press.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kirchengast, G., T. Kabas, A. Leuprecht, C. Bichler, and H. Truhetz, 2014: WegenerNet: A pioneering high-resolution network for monitoring weather and climate. Bull. Amer. Meteor. Soc., 95, 227242, https://doi.org/10.1175/BAMS-D-11-00161.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kirstetter, P.-E., Y. Hong, J. J. Gourley, M. Schwaller, W. Petersen, and J. Zhang, 2013: Comparison of TRMM 2A25 products, version 6 and version 7, with NOAA/NSSL ground radar–based National Mosaic QPE. J. Hydrometeor., 14, 661669, https://doi.org/10.1175/JHM-D-12-030.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kirstetter, P.-E., Y. Hong, J. J. Gourley, Q. Cao, M. Schwaller, and W. Petersen, 2014: Research framework to bridge from the Global Precipitation Measurement Mission core satellite to the constellation sensors using ground-radar-based National Mosaic QPE. Remote Sensing of the Terrestrial Water Cycle, Geophys. Monogr., Amer. Geophys. Union, 61–79, https://doi.org/10.1002/9781118872086.ch4.

    • Crossref
    • Export Citation
  • Kirstetter, P.-E., Y. Hong, J. J. Gourley, M. Schwaller, W. Petersen, and Q. Cao, 2015: Impact of sub-pixel rainfall variability on spaceborne precipitation estimation: Evaluating the TRMM 2A25 product: Impact of sub-pixel rainfall variability on TRMM 2A25. Quart. J. Roy. Meteor. Soc., 141, 953966, https://doi.org/10.1002/qj.2416.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kummerow, C. D., D. L. Randel, M. Kulie, N.-Y. Wang, R. Ferraro, S. J. Munchak, and V. Petkovic, 2015: The evolution of the Goddard profiling algorithm to a fully parametric scheme. J. Atmos. Oceanic Technol., 32, 22652280, https://doi.org/10.1175/JTECH-D-15-0039.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Menne, M. J., I. Durre, R. S. Vose, B. E. Gleason, and T. G. Houston, 2012: An overview of the Global Historical Climatology Network-Daily database. J. Atmos. Oceanic Technol., 29, 897910, https://doi.org/10.1175/JTECH-D-11-00103.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O, S., U. Foelsche, G. Kirchengast, J. Fuchsberger, J. Tan, and W. A. Petersen, 2017: Evaluation of GPM IMERG Early, Late, and Final rainfall estimates using WegenerNet gauge data in southeastern Austria. Hydrol. Earth Syst. Sci., 21, 65596572, https://doi.org/10.5194/hess-21-6559-2017.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O, S., U. Foelsche, G. Kirchengast, and J. Fuchsberger, 2018: Validation and correction of rainfall data from the WegenerNet high density network in southeast Austria. J. Hydrol., 556, 11101122, https://doi.org/10.1016/j.jhydrol.2016.11.049.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Petković, V., and C. D. Kummerow, 2015: Performance of the GPM passive microwave retrieval in the Balkan flood event of 2014. J. Hydrometeor., 16, 25012518, https://doi.org/10.1175/JHM-D-15-0018.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Petković, V., and C. D. Kummerow, 2017: Understanding the sources of satellite passive microwave rainfall retrieval systematic errors over land. J. Appl. Meteor. Climatol., 56, 597614, https://doi.org/10.1175/JAMC-D-16-0174.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Romanov, P., G. Gutman, and I. Csiszar, 2000: Automated monitoring of snow cover over North America with multispectral satellite data. J. Appl. Meteor., 39, 18661880, https://doi.org/10.1175/1520-0450(2000)039<1866:AMOSCO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Skofronick-Jackson, G., and et al. , 2017: The Global Precipitation Measurement (GPM) mission for science and society. Bull. Amer. Meteor. Soc., 98, 16791695, https://doi.org/10.1175/BAMS-D-15-00306.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tan, J., W. A. Petersen, and A. Tokay, 2016: A novel approach to identify sources of errors in IMERG for GPM ground validation. J. Hydrometeor., 17, 24772491, https://doi.org/10.1175/JHM-D-16-0079.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tan, J., W. A. Petersen, P.-E. Kirstetter, and Y. Tian, 2017: Performance of IMERG as a function of spatiotemporal scale. J. Hydrometeor., 18, 307319, https://doi.org/10.1175/JHM-D-16-0174.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Villarini, G., and W. F. Krajewski, 2007: Evaluation of the research version TMPA three-hourly 0.25° × 0.25° rainfall estimates over Oklahoma. Geophys. Res. Lett., 34, L05402, https://doi.org/10.1029/2006GL029147.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. International Geophysics Series, Vol. 100, Academic Press, 704 pp.

  • Wolff, D. B., and B. L. Fisher, 2009: Assessing the relative performance of microwave-based satellite rain-rate retrievals using TRMM ground validation data. J. Appl. Meteor. Climatol., 48, 10691099, https://doi.org/10.1175/2008JAMC2127.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, J., and et al. , 2016: Multi-Radar Multi-Sensor (MRMS) quantitative precipitation estimation: Initial operating capabilities. Bull. Amer. Meteor. Soc., 97, 621638, https://doi.org/10.1175/BAMS-D-14-00174.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
1

See file specification for 2AGPROFGMI.

Supplementary Materials

Save