1. Introduction
The past three decades have witnessed the advance of satellite precipitation products (SPPs). The availability of increased resolution (both spatially and temporally) gridded global quantitative precipitation estimates (QPEs) has allowed researchers to overcome the limited spatial coverage of in situ observations. Global or quasi-global satellite precipitation products have become suitable to investigate weather variability and climate trends in support of economic vitality, energy, agriculture, water resources, human health, public safety, community resilience to climate change, and national security (Tapiador et al. 2011; Kucera et al. 2013; Serrat-Capdevila et al. 2014; Yang et al. 2016; Prat and Nelson 2020, among others).
Available SPPs rely on different physical retrieval principles, bias adjustment techniques, and varying reference datasets (Michaelides et al. 2009; Kidd et al. 2010; Kidd and Huffman 2011; Tapiador et al. 2012; Prat and Nelson 2020). Remotely sensed techniques for rainfall measurement are mainly based on three types of sensors: infrared (IR), passive microwave (PMW), and active microwave (AMW) sensors. A fourth method includes multisensor techniques, which combines remotely sensed data from different observation platforms and takes advantage of each technique to provide optimal quantitative precipitation estimates. Detailed descriptions of each measurement method can be found in Kidd et al. (2010) and Prat and Nelson (2020), among others.
Due to their growing popularity and utilization for numerous hydrological and climatological applications, satellite datasets have been the object of evaluations against other satellite, radar, or in situ datasets (see, e.g., Tian et al. 2009; Hirpa et al. 2010; AghaKouchak et al. 2011, 2012; Gebremichael et al. 2014; Miao et al. 2015; Prat and Nelson 2015; Katiraie-Boroujerdy et al. 2017; Beck et al. 2017, 2019; Adler et al. 2018; Ferraro et al. 2018; Nguyen et al. 2018; Tan and Santo 2018; Sadeghi et al. 2019; Prat et al. 2021). Most of the evaluations performed in the aforementioned studies have been limited to a specific geographical area and/or temporal duration. Long-term evaluations are particularly important in the context of SPP applications to hydrological and hydroclimatological events such as extreme precipitation (Prat and Nelson 2020) or drought detection and monitoring (Prat et al. 2018, 2020). Some studies have provided a global assessment and long-term performance evaluation of SPPs (Tian and Peters-Lidard 2010; Prat and Nelson 2015; Beck et al. 2017, 2019; Massari et al. 2017; Nguyen et al. 2018; Sun et al. 2018; Prat et al. 2021, among others). An even smaller number of studies have focused on SPP performance with respect to seasonal precipitation (Tian et al. 2009; Beck et al. 2019).
The performance (in a statistical sense) of SPPs is dependent on the underlying physical principles of retrieval algorithms used for rainfall estimation [see Kidd et al. (2010) for a review]. Recently, an evaluation of three gridded daily satellite precipitation products (SPPs) that are part of the NOAA/NCEI’s Climate Date Record (CDR) portfolio (https://www.ncdc.noaa.gov/cdr) was performed by Prat et al. (2021). The three precipitation CDRs (PERSIANN-CDR, GPCP, CMORPH) were developed for long-term hydrological and climatic analysis and applications. Two of the SPPs—PERSIANN-CDR and GPCP—are based primarily on IR techniques (Huffman et al. 2001; Ashouri et al. 2015; Adler et al. 2017a), while CMORPH is based on PMW techniques (Joyce et al. 2004; Xie et al. 2017). The three SPPs presented comparable annual average precipitation. However, significant differences were found with respect to the daily rainfall distribution, particularly in the low and high rainfall rate ranges. The comparison against in situ measurements from the U.S. Climate Reference Network (USCRN), showed that CMORPH performed consistently better than GPCP and PERSIANN-CDR for an ensemble of usual metrics: bias, correlation, accuracy, probability of detection, and false alarm ratio, among others. As a physically based retrieval, for which the measurement is directly affected by the size and the distribution of hydrometeors, PMW products typically show better performance than IR products which assume a direct relationship between a cloud-top brightness temperature and rain rate (Griffith et al. 1978; Scofield 1987). Conversely, PMW products have well-known limitations due to their difficulty of detecting cold or frozen precipitation and over snow-covered surfaces (Tian et al. 2009; Beck et al. 2019). Prat et al. (2021) provided a long-term, global evaluation of each SPP from annual to daily temporal scales. However, the study did not propose an evaluation at the seasonal scale. The goal of the present study is to fill this void by determining the ability of each dataset to capture cold season rainfall. The SPPs’ performance with respect to cold-season precipitation will be compared to warm-season and full-year analysis for benchmarking purposes. The period of study (2007–18) is similar to that of Prat et al. (2021) for the section related to CONUS and the comparison with in situ USCRN. The collocated measurements of precipitation and air temperature at USCRN stations provide us the opportunity to segment comparisons for freezing and warm temperatures (Nelson et al. 2021a,b).
The paper is organized as follows. In section 2, we present the characteristics and physical principles of the three SPPs along with the USCRN in situ dataset used as reference. The results of the evaluation are presented in sections 3 and 4. Section 3 presents a comparison of the SPPs long-term seasonal precipitation characteristics. Section 4 presents the evaluation of each dataset against USCRN data and focuses on daily statistics. Finally, section 5 summarizes the most important findings.
2. Precipitation datasets and algorithms description
a. GPCP daily
The Global Precipitation Climatology Project (GPCP) suite of products (monthly and daily) combines satellite precipitation estimates from infrared and microwave sensors (Huffman et al. 2001). The GPCP daily product used in this work is available since October 1996. The product has a global coverage (90°S–90°N) with a spatial resolution of 1° × 1° (Adler et al. 2017b). It is updated monthly as an Interim CDR (ICDR) with the final version (CDR) available after three months (Adler et al. 2017a). Both monthly and daily GPCP CDRs are available at the NOAA/National Centers for Environmental Information (NCEI) Climate Data Records portal (https://www.ncei.noaa.gov/products/climate-data-records/precipitation-gpcp-daily). In this work, we use the daily version v01r03. Technical details on the GPCP algorithm can be found in the Climate Algorithm Theoretical Basis Documents (C-ATBD) (Adler et al. 2017a).
b. PERSIANN-CDR
The Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks–Climate Data Record (PERSIANN-CDR) provides quasi-global (60°S–60°N) daily precipitation estimates at a spatial resolution of 0.25° × 0.25°. PERSIANN-CDR combines IR satellite precipitation estimates using a neural network technique along with a monthly bias correction using in situ data through the GPCP monthly product (Hsu et al. 2014; Ashouri et al. 2015). PERSIANN-CDR relies on the fundamental CDR GRIDSAT B1 (i.e., geostationary IR channel brightness temperature; Knapp et al. 2011) as the main input which provides quasi-global (70°S–70°N) IR brightness temperatures every 3 h at an 8 km × 8 km spatial resolution. PERSIANN-CDR is available from 1983 to the delayed present (i.e., 6 months before the present date) with a quarterly update frequency (https://www.ncei.noaa.gov/products/climate-data-records/precipitation-persiann). In that regard, PERSIANN-CDR is different in nature than the other products of the PERSIANN family of products (Nguyen et al. 2018). The PERSIANN-CDR version used is v01r01. More information regarding PERSIANN-CDR can be found in Hsu et al. (2014) (C-ATBD) and in Ashouri et al. (2015).
c. CMORPH
The Climate Prediction Center (CPC) morphing technique (CMORPH) provides daily precipitation estimates at a spatial resolution of 0.25° × 0.25° for the domain 60°S–60°N (Joyce et al. 2004). The daily CMORPH is developed from the 30-min temporal and 8 km × 8 km spatial native resolution products. CMORPH combines precipitation estimates derived from passive microwave sensors through the advection of cloud features from more frequently available infrared measurements (i.e., the morphing technique). The bias-adjusted version of the product uses daily surface gauge analysis from the Climate Prediction Center at 0.5° × 0.5° over land (Xie et al. 2010) and pentad precipitation estimates from GPCP over ocean (Xie et al. 2003). As with PERSIANN-CDR, GPCP is used for bias adjustment. In this case it is the pentad version that is used for bias adjustment over ocean instead of the monthly version. More details regarding the CMORPH algorithm and the precipitation products available can be found in Xie et al. (2017, 2018). CMORPH is available from 1998 to the delayed present. A non-bias-adjusted CMORPH is available as an interim version within 2 days of the present date. The version used in this work is CMORPH V1.0 (https://www.ncei.noaa.gov/products/climate-data-records/precipitation-cmorph).
The main difference between the three SPPs lies in the underlying physical principles. PERSIANN-CDR is primarily an IR product, while CMORPH and GPCP incorporate both PMW and IR sensors, predominantly PMW for CMORPH and IR for GPCP (Prat et al. 2021). The three SPPs are bias-adjusted with in situ datasets that incorporate the gauge information from the Global Historical Climatology Network-Daily (GHCN-D: Menne et al. 2012). A detailed comparison of each SPP algorithm is provided in Prat et al. (2021).
d. In situ quantitative precipitation estimates datasets: The U.S. Climate Reference Network
The U.S. Climate Reference Network is composed of about 140 stations and provides an ensemble of quality-controlled atmospheric parameters (rain rate, relative humidity, air temperature, wind speed at 1.5 m, and soil moisture for instance) at 5-min, 1-h, and 1-day temporal resolutions (Diamond et al. 2013; Leeper et al. 2015; https://www.ncei.noaa.gov/access/crn/qcdatasets.html). About half of those USCRN stations are located at latitudes between 40° and 60°N (70 stations) and elevations above 1500 m (65 stations) that experience annually a mix of warm and cold season precipitation. The USCRN stations are not used in the in situ bias adjustment procedures of the three SPPs. Therefore, they can be used to provide an independent evaluation of precipitation estimates from the three satellite precipitation datasets. All the stations used in this study are presented in Fig. 1a. They include the 114 USCRN stations located over CONUS, one station located in Canada (Egbert, Ontario), and two additional test stations in Goodwell (Oklahoma) and Oak Ridge (Tennessee). The USCRN stations located in Alaska (22 stations) and Hawaii (2 stations) are not included in this study (Fig. 1b). The USCRN stations were augmented by a higher-density network of U.S. Regional Climate Reference Network stations in the Four Corners states (Utah, Colorado, New Mexico, Arizona) for a short period (2009–14), allowing for an analysis of 56 stations in this region over that period (herein after referred to as 4CCRN; Fig. 1c). The regional network similar to USCRN deployed in Alabama (herein after referred to as ALCRN: 17 stations) is also included in the evaluation (Fig. 1d). For each station, the percentage of days in the period 2007–18 with data available is indicated. Although stations from the higher-density network (and to a lesser extent the stations in Alabama as well) operated for a shorter period than the USCRN stations, the statistics from period when all the data were available (i.e., 2009–14) were consistent with the statistics from the entire 2007–18 period. Including the stations from the higher-density network over the Four Corner states allows for a larger range of station elevations and cold season precipitation regimes. From the 1-h rainfall accumulation, we are able to aggregate daily precipitation totals to the same daily accumulation as the satellite datasets (0000–0000 UTC).
(a) Percentage of days in the period 2007–18 with data available for the gauges used in this analysis (ALL). The ensemble includes (b) the stations from the USCRN network (USCRN), (c) the stations from the U.S. Regional Climate Reference Network (4CCRN) located in the Four Corners states (UT, CO, NM, AZ) that were operational for a short period (2009–14), and (d) the U.S. Regional Climate Reference Network over the state of Alabama (ALCRN).
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
3. Seasonal precipitation characteristics
a. Average daily precipitation
Figure 2 displays annual (YEA: Figs. 2a–d) and seasonal for winter (DJF: Figs. 2e–h) and summer (JJA: Figs. 2i–l) average precipitation for PERSIANN-CDR and CMORPH and the differences between them for the period 1998–2018. We also present the differences between PERSIANN-CDR and GPCP. PERSIANN-CDR and CMORPH present similar precipitation patterns over CONUS regardless of the season (annual, winter, summer). Both CMORPH and PERSIANN-CDR capture the seasonal differences characterizing winter and summer: higher rainfall along the Pacific Coast and the Southeast in winter and higher rainfall over Florida and the eastern United States in summer. Annual and seasonal differences between CMORPH and PERSIANN-CDR indicate areas of positive and negative differences over CONUS. The biggest differences are negative (i.e., CMORPH average precipitation lower than PERSIANN-CDR) and localized over the Pacific Northwest, the upper Midwest, and the Northeast in winter (Fig. 2g). Although less important in magnitude and limited to a smaller geographic area, those negative differences persist on an annual scale (Fig. 2c). In summer, differences are more moderate with positive differences (i.e., CMORPH average precipitation greater than PERSIANN-CDR) over the Rockies in particular (Fig. 2k). Negative differences are found for the Florida peninsula (Fig. 2k). As shown elsewhere, the most important differences between CMORPH and PERSIANN-CDR are found at higher latitudes (Prat et al. 2021). Globally, similar negative differences are observed during the cold season at higher latitudes (eastern United States, Canada, Europe, and European Russia) where CMORPH displays lower annual average rainfall when compared to PERSIANN-CDR (not shown). The important differences observed between CMORPH and PERSIANN-CDR during the cold season at higher latitudes is due to the inability for microwave sensors to retrieve precipitation over snow-covered surface (Tian et al. 2009; Beck et al. 2019; Prat et al. 2021).
Average daily precipitation for PERSIANN-CDR, CMORPH, differences between CMORPH and PERSIANN-CDR, and differences between GPCP and PERSIANN-CDR (shown from left to right, respectively). Average daily rainfall and average daily rainfall differences are presented for (a)–(d) the year (YEA), (e)–(h) winter (DJF), and (i)–(l) summer (JJA). Data are for the period 1998–2018.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
The differences between PERSIANN-CDR and GPCP are more moderate and less seasonally dependent (Figs. 2d,h,l). The good agreement between PERSIANN-CDR and GPCP is due to the fact that the former uses the monthly version of GPCP for bias adjustment (Ashouri et al. 2015). The biggest differences are found over the Pacific Northwest coast in winter where PERSIANN-CDR average rainfall is higher than GPCP (Fig. 2h). Those differences observed locally over the Pacific Northwest, a domain characterized by complex topography and associated orographic precipitation in winter, can be explained by difference in spatial resolution between the two SPPs and possibly by the bias adjustment procedure which includes upscaling/downscaling of PERSIANN-CDR in order to maintain consistency between GPCP (GPCP-Monthly v02r02 at 2.5° × 2.5° resolution) and PERSIANN-CDR monthly rainfall accumulation (Ashouri et al. 2015; Prat et al. 2021). Because of the similarity in average annual/monthly global precipitation fields between PERSIANN-CDR and GPCP, differences between CMORPH and GPCP (not shown) are similar to the differences between CMORPH and PERSIANN-CDR (Figs. 2c,g,k).
b. Percentage of rainy days
Figure 3 presents the percentage of rainy days reported by each SPP on an annual and seasonal basis. For all SPPs, rainy days are days with greater than zero rainfall accumulation. On an annual basis, PERSIANN-CDR (Fig. 3a) and GPCP (Fig. 3c) display more rainy days than CMORPH, particularly at higher latitudes (Fig. 3b). The contrast is even more noticeable in winter when CMORPH (Fig. 3e) indicates a much lower percentage of rainy days at higher latitudes over land compared to PERSIANN-CDR (Fig. 3d) and GPCP (Fig. 3f). In summer, the three SPPs display more similar patterns with noticeable differences over Canada where CMORPH (Fig. 3h) presents locally a higher percentage of rainy days than PERSIANN-CDR (Fig. 3g) and GPCP (Fig. 3i). This is also the case in mountainous areas (Rocky Mountains, Sierra Nevada) (Fig. 3g). Despite similar patterns for GPCP and PERSIANN-CDR, we note that orographic features are less detailed when compared to CMORPH. For GPCP, this is due to its coarser resolution (Fig. 3i). For PERSIANN-CDR, it is related to the bias adjustment procedure that uses the coarser GPCP-Monthly at 2.5° × 2.5°, which tends to “wash-out” localized precipitation features (Fig. 3g), while CMORPH uses CPC daily gauge analysis at 0.5° × 0.5° over land. This is noticeable over the western United States but more particularly over the major mountain ranges in the world (not shown since Fig. 3 is limited to CONUS). For CMORPH, we observe localized higher percentage of rainy days (i.e., “bull’s-eye” features) indicating possible mismatches between CMORPH and the daily CPC gauge analysis used for bias adjustment over land. We note that those bull’s-eyes are not visible in the bias-adjusted average daily precipitation (Fig. 2).
Percentage of rainy days for (left) PERSIANN-CDR, (center) CMORPH, and (right) GPCP. The percentage of rainy days are presented for (a)–(c) the year (YEA), (d)–(f) winter (DJF), and (g)–(i) summer (JJA). Data are for the period 1998–2018.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
For ease of comparison, Fig. 4 displays the differences in rainy days between the three SPPs. Unequivocally, CMORPH displays a lower percentage of rainy days for areas and seasons associated with cold precipitation. This is the case on an annual basis at higher latitudes when compared to PERSIANN-CDR (Fig. 4c) and GPCP (Fig. 4b). The most important differences are found between CMORPH and PERSIANN-CDR in winter over the higher latitudes (Fig. 4f). The differences between CMORPH and GPCP display the same spatial patterns as those between CMORPH and PERSIANN-CDR but are significantly smaller in terms of magnitude (Figs. 4b,e,h).
Differences in the percentage of rainy days between (left) PERSIANN-CDR and GPCP, (center) CMORPH and GPCP, and (right) CMORPH and PERSIANN-CDR. Differences are presented for (a)–(c) the year (YEA), (d)–(f) winter (DJF), and (g)–(i) summer (JJA). Data are for the period 1998–2018.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
The differences between PERSIANN-CDR and GPCP are particularly important over CONUS with PERSIANN-CDR displaying a higher percentage of rainy days annually (Fig. 4a). Those differences are even more important in winter with more than 50% more rainy days for PERSIANN-CDR when compared to GPCP over the majority of CONUS (Fig. 4d). As shown elsewhere, PERSIANN-CDR displays a higher percentage of rainy days over land when compared to GPCP globally (Prat et al. 2021). Here, the areas where GPCP presents a slightly higher percentage of rainy day than PERSIANN-CDR are limited to the higher latitudes of eastern Canada, Pacific Northwest, and the central valley in California in summer (Fig. 4g).
c. Maximum daily rainfall
While the three datasets present similar annual precipitation patterns, there are important seasonal differences—particularly for cold season precipitation—between CMORPH and the two other datasets (Fig. 2). Significant differences in the number of rainy days are observed between the IR (PERSIANN-CDR, GPCP) and PMW (CMORPH) precipitation datasets. The differences are also important between PERSIANN-CDR and GPCP even though GPCP is used for PERSIANN-CDR bias adjustment (Figs. 3 and 4). Equally significant are the differences for daily maximum rainfall (Fig. 5). Independently of the season, CMORPH presents the highest maximum daily rainfall (Figs. 5b,e,h) while PERSIANN-CDR displays significantly lower daily maximum rainfall (Figs. 5a,d,g) and GPCP has the lowest daily maximum values (Figs. 5c,f,i). The three datasets present comparable seasonal patterns for extreme precipitation in winter (Figs. 5d–f) and summer (Figs. 5g–i). The seasonality is noticeable for CMORPH with higher daily maximum rainfall found in winter along the Pacific Northwest and over the southern United States (Fig. 5e) and in summer in a less organized fashion over areas of intense convective activity (Midwest and southeastern United States, Gulf of Mexico) (Fig. 5h). Although PERSIANN-CDR displays a similar seasonality, the daily maxima are significantly lower than for CMORPH (Figs. 5d,g).
Maximum daily precipitation for (left) PERSIANN-CDR, (center) CMORPH, and (right) GPCP. The maximum daily rainfall is presented for (a)–(c) the year (YEA), (d)–(f) winter (DJF), and (g)–(i) summer (JJA). Data are for the period 1998–2018.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
Among the three SPPs, while PERSIANN-CDR and GPCP are the closest, they exhibit noticeable differences over land on an annual and seasonal basis. PERSIANN-CDR (Fig. 5a) displays higher daily maxima when compared to GPCP (Fig. 5c). However, GPCP presents more widespread patterns in particular in the southern United States in winter (Fig. 5f). These differences could be explained by the coarser spatial resolution of GPCP when compared to PERSIANN-CDR. For the highest values of the daily maximum rainfall (i.e., 99th–100th percentile), the ranges are 230–397 mm day−1 for CMORPH, 163–273 mm day−1 for PERSIANN-CDR, and 141–158 mm day−1 for GPCP. The values of the daily maximum rainfall for PERSIANN-CDR are about 60%–70% those of CMORPH. Both PERSIANN-CDR and CMORPH share consistency in terms of spatial and temporal resolutions. For GPCP, the values of the daily maximum rainfall are about 40%–60% those of CMORPH. In this case, the intercomparison between CMORPH and GPCP has to be met with caution since GPCP has a coarser spatial resolution than CMORPH (and PERSIANN-CDR). Direct intercomparison of the SPPs would require interpolating CMORPH and PERSIANN-CDR to the same 1° × 1° grid to preserve the spatial consistency. This, however, is not the primary scope of this work that aims to evaluate the three SPPs at their native resolution against USCRN.
d. Daily rainfall distributions
A comparison of the rainfall distributions between SPPs and USCRN stations is provided in Fig. 6.
Cumulative distribution function of daily rainfall for events observed simultaneously by all USCRN stations (black) and SPPs (red), rainfall events observed only at the USCRN stations (gray), and rainfall events observed only by each SPP (blue). (a) PERSIANN-CDR (PERSIANN in the figure), (b) CMORPH, and (c) GPCP for the warm season (JJA). (d)–(f) As in (a)–(c), but for the cold season (DJF). (g)–(i) As in (a)–(c), but for stations where average daily temperature is below 0°C. Only stations that had at least 30 days per year of average daily temperature below 0°C are considered.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
In summer, CMORPH (Fig. 6b, red line) presents a better agreement for events observed simultaneously at USCRN stations (black line), than GPCP (Fig. 6c, red line) and PERSIANN-CDR (Fig. 6a, red line). CMORPH tends to overestimate daily rainfall below the 60th percentile which corresponds to approximatively to 5 mm day−1 and to underestimate rainfall above the 70th percentile which corresponds approximatively to 10 mm day−1 (Fig. 6b).
Likewise, PERSIANN-CDR overestimates rainfall below the 60th percentile (i.e., below 5 mm day−1) and significantly underestimates higher rain rates above the 70th percentile (Fig. 6a). GPCP displays a comparable underestimation as compared to PERSIANN-CDR in the left-hand side of the distribution below the 70th percentile (i.e., for rainfall lower than 10 mm day−1) and underestimates the right side of the spectra above that same value (Fig. 6c). In addition, GPCP presents a noticeable amount of low daily rates up to the 10th percentile that correspond to rain rates under 0.2 mm day−1. Overall, CMORPH presents a relatively good agreement with USCRN stations for daily rain rates ranging from the 50th–80th percentile. For PERSIANN-CDR and GPCP, the best agreement with USCRN is reduced to a narrower range of daily rain rates corresponding to the 55th–65th and 65th–75th percentiles for PERSIANN-CDR and GPCP, respectively.
The distribution of rain events observed only by the satellite (i.e., false alarms: blue line) and those observed only at USCRN stations (i.e., misses: gray line) are also reported for illustration. False alarms and misses are characterized by lower rain rates when compared to hits. Although different in nature, the relative similarity between the distribution of misses and false alarms for PERSIANN-CDR and GPCP indicates that misses and false alarms are independent with respect to the rainfall intensity.
In winter, the distribution of concurrent daily rainfall events observed by CMORPH and GPCP is shifted leftward toward lower rain rates displaying rainfall underestimation throughout the distribution when compared to USCRN stations (Figs. 6e,f). Like for summer, PERSIANN-CDR overestimates low rainfall rates below the 25th percentile (i.e., approximately 1 mm day−1) and underestimates higher rainfall rates above the 40th percentile (i.e., approximately 3 mm day−1). The best agreement between PERSIANN-CDR and the USCRN stations corresponds to the range of 1–3 mm day−1 (Fig. 6d). For negative average daily temperatures (T < 0°C: Figs. 6g–i), the distribution for concurrent events for PERSIANN-CDR is shifted leftward (i.e., toward lower rain rates) when compared to summer and winter (Fig. 6g). A similar behavior, yet less pronounced, is observed for GPCP (Fig. 6i). Conversely, the distributions for misses (USCRN only) and false alarms (SPPs only) are relatively similar when compared to those of summer and winter for PERSIANN-CDR and GPCP. For concurrent daily rainfall events, CMORPH significantly underestimates daily rain rates ranging from the 0th to the 95th percentile (i.e., from 0.1 to 30 mm day−1). Interestingly, CMORPH displays higher daily rainfall as compared to the USCRN stations above the 95th percentile. For the 50th percentile (median value), there is a twofold difference in the nominal rain rate between CMORPH (1.5 mm day−1) and USCRN (3 mm day−1). Like for winter, the distribution of events missed by CMORPH is shifted toward higher rainfall intensity when compared to summer (Fig. 6h).
Figure 7 displays scatterplots for concurrent events. For summer, all SPPs display significant underestimation of the daily rainfall (Figs. 7a–c). The best agreement is obtained for CMORPH with a quantile–quantile (Q–Q) plot (black dots) closer to the 1:1 line (Fig. 7b). PERSIANN-CDR (Fig. 7a) and GPCP (Fig. 7c) present very similar performances and a tendency of underestimation when compared to USCRN. Similar results are obtained for winter.
Scatterplots (gray) and quantile–quantile (black) plots for daily precipitation hits derived from (a) PERSIANN-CDR, (b) CMORPH, and (c) GPCP when compared for the warm season (JJA). (d)–(f) As in (a)–(c), but for the cold season (DJF). (g)–(i) As in (a)–(c), but for stations where average daily temperature is below 0°C. Only stations that had at least 30 days per year of average daily temperature below 0°C are considered. The red line represents the best-fit line.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
Interestingly, we note that CMORPH presents a better agreement for concurrent events in winter (Fig. 7e) than in summer (Fig. 7b). The same is true for PERSIANN-CDR (Figs. 7a,d) and GPCP (Figs. 7c,f). For precipitation events corresponding to negative average daily temperature, the SPPs present a similar underestimation with respect to USCRN stations, with CMORPH, GPCP, and PERSIANN-CDR ranking from best to worst (Figs. 7g–i). CMORPH indicates occurrences of higher daily rainfall when compared to USCRN as indicated by the Q–Q plot (Fig. 7h). This observation is consistent with previous results that showed that CMORPH displayed higher daily rainfall above the 95th percentiles (i.e., rainfall intensities above 30 mm day−1) when compared to USCRN (Fig. 6h).
4. Daily precipitation statistics
a. Contingency analysis
For the contingency analysis we also compute the hits (YgYs), misses (YgNs), and false alarms (NgYs) scaled to the sum of the three (YgYs + YgNs + NgYs) so that their sum is equal to one. Although hits and false alarms are statistical entities close to POD and FAR, respectively, they provide valuable information in support of the conditional/unconditional analysis presented in the next section. The conditional analysis is performed only on the hits (simultaneous rain detection at the gauge and at the satellite pixel), while the unconditional analysis is irrespective of the presence of rain or no rain at the gauge or the satellite pixel. A point-to-pixel comparison is justified assuming that random sampling errors are negligible, which is the case for the spatial resolutions of the three SPPs at the daily scale (Prat and Nelson 2015, 2020; Prat et al. 2021).
Figure 8 presents the results of the contingency analysis (i.e., misses, hits, and false alarm) and associated performance metrics—POD, FAR, and POFD—as a function of the latitude of each USCRN station. Metrics are computed for cold (DJF: blue dots) and warm seasons (JJA: red dots). For most of the metrics, differences between cold and warm seasons appear moderate for PERSIANN-CDR (top row) and GPCP (bottom row) as seen from the red and blue vertical lines that indicate the average values for the warm and cold season, respectively. PERSIANN-CDR displays similar results between warm and cold seasons for misses (YgNs). There is a slight improvement for the warm season when compared to the cold season for the other indicators: hits (YgYs), false alarms (NgYs), POD, FAR, and POFD where average values are closer to the ideal score. GPCP presents similar results with more pronounced differences between the warm and cold seasons. Better results are found for misses (YgNs), hits (YgYs), POD, and FAR during the warm season, while false alarms (NgYs) and POFD are slightly better during the cold season. Moreover, PERSIANN-CDR and GPCP, based on IR sensors, do not show any pronounced dependence with respect to the latitude for any of the metrics. Conversely, CMORPH which is based on passive microwave (PMW) displays significant latitudinal differences between the cold and warm seasons (middle row).
Contingency analysis and performance statistics for (top) PERSIANN-CDR, (middle) CMORPH, and (bottom) GPCP compared to USCRN stations organized with latitude. From left to right, results are presented for the contingency analysis, that is, for rain detected at the USCRN station only (misses: YgNs), rain detected at the USCRN station and by the satellite (hits: YgYs), and rain detected by the satellite only (false alarm: NgYs), probability of detection (POD), false alarm ratio (FAR), and probability of false detection (POFD). The green circles indicate the perfect score. The vertical lines represent the average values over all latitudes for cold (blue) and warm (red) seasons.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
The values for the warm season (JJA) for misses, hits, false alarm, POD, FAR, and POFD for CMORPH are better than those of PERSIANN-CDR and GPCP. For the cold season (DJF), we note for CMORPH an important increase in the number of misses (YgNs) and a strong decrease in the number of hits (YgYs) and in POD. At the same time, the number of false alarms (NgYs) and the POFD remain lower than during the warm season.
On an annual basis (YEA) and during the warm season (JJA), CMORPH almost always exhibits the best results apart from misses and POD (Table 1). PERSIANN-CDR displays the lowest number of misses (7%) and the highest annual POD (85%) when compared with CMORPH and GPCP. We also observe a higher POFD (41%–42%) and a lower accuracy (66%–67%) than for the two other SPPs on an annual basis and during the warm season.
Contingency analysis and performance evaluation for daily rainfall retrieved by the three satellite products (S) with respect to in situ USCRN (G). The contingency analysis presents the normalized occurrences for satellite hits (YgYs), misses (YgNs), and false alarm (NgYs). The metrics for SPP performance assessment are accuracy (ACC), false alarm ratio (FAR), probability of detection (POD), and probability of false detection (POFD). Results are computed on an annual basis (YEA), for summer (JJA), and winter (DJF). The best scores as a function of the season (YEA, JJA, DJF) are indicated in bold while the lowest scores are indicated in italic.
During the cold season (DJF), the SPP displaying the best performance depends on the metric considered. PERSIANN-CDR displays the lowest number of misses (7%) and the highest POD (84%). Because CMORPH does not detect precipitation on snow covered ground, it displays the lowest number of false alarms (32%) and POFD (16%) and the best values for FAR (56%). Coincidentally, the best values obtained for those metrics (false alarms, FAR, POFD) are most likely due to PMW sensor limitations in detecting precipitation over snow covered surfaces than to the superiority of the rainfall retrieval algorithm. Those results illustrate the necessity to select an ensemble of complementary metrics to accurately evaluate the performance of each SPP.
To further refine the analysis, we look at the influence of atmospheric conditions considering positive (T > 0°C: Fig. 9) and negative (T < 0°C: Fig. 10) temperatures measured near the ground at USCRN stations.
Performance comparison for PERSIANN-CDR, CMORPH, and GPCP with respect to USCRN for the years 2007–18. The metrics for performance assessment are (a)–(c) accuracy, (d)–(f) probability of detection (POD), (g)–(i) false alarm ratio (FAR), and (j)–(l) probability of false detection (POFD). Results are provided for average daily temperature above 0°C. The sizes of the circles are related to the values.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
As in Fig. 9, but results are provided for average daily temperature below 0°C. Only stations that had at least 30 days per year of average daily temperature below 0°C are considered for representativeness reason.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
For positive temperatures, CMORPH performs better in terms of accuracy (Fig. 9b), FAR (Fig. 9e), and POFD (Fig. 9h). Although PERSIANN-CDR presents a higher POD (Fig. 9g), it displays higher FAR values (Fig. 9d) and lower accuracy (Fig. 9a) than the two other SPPs. Results are reported at stations that had at least four full years of data available over the period 2007–18. The same criterion is used in the figures and analysis for annual and seasonal analysis. Regardless of the metric, the best values are located along the Sun Belt from the Eastern Seaboard and up to the Pacific Coast and we observe decreasing performances along a northward direction (Fig. 9).
Figure 10 presents the same analysis conducted for negative air temperature recorded at the station. For representativeness purposes, we consider stations with at least 30 days annually of average daily temperature below 0°C. Those are stations located approximatively above 36°N which corresponds to the northern borders of Virginia, Kentucky, Oklahoma, New Mexico, and Arizona. CMORPH displays very low POD values due to PMW limitations in capturing cold season precipitation (Fig. 10h). Over the Rockies and the upper Midwest, CMORPH displays higher FAR values than PERSIANN-CDR and GPCP (Fig. 10e). Counterintuitively, this is due to the combined effect of a very low number of hits (YgYs) and a lower number of false alarms (NgYs). As per the FAR definition [Eq. (3)], the number of hits (YgYs) being smaller relatively to the number of false alarms (NgYs), the sum of hits and false alarms in the denominator (YgYs + NgYs) will be closer to the number of false alarms in the numerator (NgYs). This can also be seen in Fig. 8 for the cold season.
Because CMORPH detects fewer precipitation events, it presents better POFD values (i.e., lower) than the two other SPPs (Fig. 10k). PERSIANN-CDR displays the highest POD (Fig. 10g) along with the highest POFD (Fig. 10j), similar to its performance for positive temperatures. Despite its inability to detect cold precipitation, CMORPH accuracy is comparable (GPCP: Fig. 10c) or better (PERSIANN-CDR: Fig. 10a) than the two other SPPs (Fig. 10b). Overall for negative temperatures, GPCP displays the best performance.
b. Bias and correlation
Bias ratio for daily precipitation with respect to USCRN data for PERSIANN-CDR, CMORPH, and GPCP for the period 2007–18 for the USCRN and ALCRN stations and for 2009–14 for the stations from the U.S. Regional Climate Reference Network over the Four Corners states (4CCRN). Biases are computed unconditionally. Results are presented (a)–(c) on an annual basis, (d)–(f) for the warm season (JJA), (g)–(i) for the cold season (DJF), and (j)–(l) for stations where the average daily temperature is below 0°C. Only stations that had at least 30 days per year of average daily temperature below 0°C are considered. The sizes of the circles are related to the values.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
Biases are computed unconditionally (i.e., including zeros and nonzeros alike for USCRN stations and SPP precipitation). Regardless of the seasonal and atmospheric conditions, all SPPs display higher bias ratios in the western United States and over the Rockies with an overestimation of precipitation by the satellite with respect to USCRN. The higher bias ratios observed over the western United States can be explained by the smaller precipitation amounts when compared to the higher precipitation amounts over the eastern United States (Fig. 2). Therefore it is not surprising that even moderate nominal differences in precipitation amounts between the USCRN gauge and the satellite would lead to higher bias values. For annual (Figs. 11a–c) and summertime (Figs. 11d–f) precipitation, CMORPH presents lower bias ratios than GPCP and PERSIANN-CDR with GPCP presenting the largest rainfall overestimation over the Rockies. During the cold season, bias ratios are the highest over the Rockies (rainfall overestimation) for PERSIANN-CDR and GPCP (Figs. 11g,i). While PERSIANN-CDR and GPCP exhibit significant differences in terms of biases between the eastern and the western United States, CMORPH presents more pronounced differences in the latitudinal direction. At higher latitudes, CMORPH presents a significant rainfall underestimation (Fig. 11h). Over the Rockies, most locations present moderate underestimation while others display rainfall overestimation when compared to USCRN. For negative temperatures, the rainfall underestimation is obvious for CMORPH, while PERSIANN-CDR and GPCP display moderate bias ratios with a mixture of overestimation and underestimation. While it is possible that the length of the reporting periods between stations of the USCRN (12 years) and the 4CCRN (5 years) has an influence on results due to precipitation annual variability, we have not found those differences between both datasets (USCRN only and USCRN with 4CCRN) to be significantly noticeable. This is valid for the bias ratios as well as for the other quantities discussed next [correlation coefficient, variability ratio, Kling–Gupta efficiency (KGE)]. The obvious benefit in adding the 4CCRN to the two other networks (USCRN, ALCRN) despite the different reporting periods, is that it provides complementary information at higher resolution over the complex topography of the Four Corners states and the Rockies.
The correlation coefficient is presented in Fig. 12. The geographical patterns obtained for correlation are similar to those found for bias (Fig. 11). Higher correlations are found over the southern and eastern United States for the three SPPs on an annual basis (Figs. 12a–c) and in summer (Figs. 12d–f). The lowest correlations are found over the Rockies and the western United States. As was the case with volume bias, CMORPH displays the highest correlation coefficients CONUS-wide including over the Rockies. PERSIANN-CDR and GPCP present comparable correlation coefficients that are noticeably lower than for CMORPH. In winter (Figs. 12g,i) and for negative average daily temperatures (T < 0°C: Figs. 12j,l), PERSIANN-CDR and GPCP present comparable correlations. Furthermore, for most of the locations, the correlations are comparable to the values obtained on an annual basis and in summer. CMORPH displays higher correlation coefficients than PERSIANN-CDR and GPCP along an arc going from the northeastern to the northwestern United States, and encompassing the Atlantic coast, Florida, the Gulf of Mexico, the Southwest, and the Pacific Coast. In winter, CMORPH displays lower correlations over the Rockies, the inland domain of the Northwest, and the northern section of the Midwest. For negative average daily temperatures (T < 0°C), the values of the correlation coefficients are very low (<0.2).
As in Fig. 11, but the results are for the correlation coefficient.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
c. KGE performance
Figure 13 displays the KGE for the same conditions as presented for the bias (Fig. 11) and the correlation coefficient (Fig. 12). PERSIANN-CDR and GPCP present comparable KGE values on an annual and seasonal basis (JJA, DJF). The highest KGE values are obtained along the Atlantic seaboard, over the southern United States, and along the Pacific coast. The lowest KGE values are found over the Rockies and at latitudes above 36°N. On an annual basis and in summer, CMORPH presents significantly higher KGE values as compared to PERSIANN-CDR and GPCP with the same spatial distribution for the highest KGE. In winter, CMORPH presents higher KGE values than PERSIANN-CDR and GPCP along the Eastern Seaboard from Virginia (i.e., approximately below 36°N) down to Florida, over the Southeast, along the Gulf of Mexico, over the southwestern United States, and along the Pacific Coast. Conversely, CMORPH displays the lowest KGE scores over the Rockies, the western United States, the Midwest, the Great Lakes region, and the northeastern United States. This decreased performance coincides with higher-elevation and higher-latitude areas that experience mixed or frozen precipitation including a fair amount of snowfall in winter. Further confirmation is provided considering negative daily temperatures (T < 0°C). Note that the value of the ground temperature does not allow refining the analysis by precipitation type (drizzle, sleet, snow, mixed). In this case, CMORPH displays the lowest KGE values (KGE < 0.1) when compared to PERSIANN-CDR and GPCP. Both PERSIANN-CDR and GPCP display relatively similar KGE values comparable to KGE values observed over the same domains regardless of the season considered.
As in Fig. 11, but the results are for the Kling–Gupta efficiency (KGE).
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
Results for the conditional (i.e., precipitation observed simultaneously by the SPP and at the station) and unconditional (i.e., including zeros and nonzeros for USCRN and SPPs) analysis are presented in Fig. 14 for KGE (Fig. 14a), correlation coefficient (Fig. 14b), bias (Fig. 14c), and variability ratio (Fig. 14d). Regardless of the SPP considered, each metric (KGE, correlation coefficient, bias, variability ratio) presents a wider spread in winter than in summer and on an annual basis. The contrast between the warm season (JJA) and the cold season (DJF) is even more important for CMORPH both for the conditional and the unconditional cases.
(a) Kling–Gupta efficiency (KGE), (b) correlation coefficient, (c) bias, and (d) variability ratio for daily precipitation with respect to USCRN stations for PERSIANN-CDR (P), CMORPH (C), and GPCP (G). Statistics are provided for all the USCRN stations on an annual basis (YEA), for the warm season (JJA), for the cold season (DJF), and for stations with average daily temperature below 0°C (T < 0°C). Only stations that had at least 30 days per year of average daily temperature below 0°C are considered. The boxplots represent the 25th percentile, median, and 75th percentile. The whiskers represent the minimum and maximum values not considered outliers. The circles indicate the average value. The results presented conditionally (C: empty circles) and unconditionally (U: full circles). The colors of the circles are red for PERSIANN-CDR (P), green for CMORPH (C), and blue for GPCP (G). The green circles on the vertical axis indicate the perfect score.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
For ease of comparison, we report in Tables 2 and 3, the average values for each metric (i.e., dots in the figures) obtained for the conditional and unconditional cases on an annual basis, for summer and for winter.
Kling–Gupta efficiency (KGE), correlation coefficient (CORR), bias (BIAS), and variability ratio (VAR) for daily precipitation with respect to USCRN in situ data for the three SPPs. The metrics (KGE, CORR, BIAS, VAR) are computed conditionally (i.e., considering only nonzeros simultaneously at the USCRN station and at the corresponding satellite pixel). The best scores as a function of the season (YEA, JJA, DJF) are indicated in bold.
As in Table 2, but with the metrics (KGE, CORR, BIAS, VAR) computed unconditionally (i.e., over the entire period of record including zeros and nonzeros alike).
For the conditional case, CMORPH presents the best average values for KGE (0.53 and 0.44), a higher average correlation coefficient (0.57 and 0.50), the best average value for the variability ratio (0.97 and 0.89), both on an annual basis and in summer, and the lowest average biases (0.92) for summer (Table 2).
CMORPH performances are notably worse in winter for the average values of KGE (KGE = 0.30) and the variability ratio (VAR = 1.15). The relatively close results obtained for the correlation coefficient (CORR = 0.50) and the bias (BIAS = 1.05) when compared to annual and summertime results do not reflect the wider range of values observed for winter (Fig. 14). PERSIANN-CDR and GPCP present comparable values regardless of the season. PERSIANN-CDR has slightly better overall performance than GPCP with KGE values closer to unity on an annual basis (0.33 versus 0.30), for summertime (0.21 versus 0.19), and for wintertime (0.30 versus 0.23). Similarly, PERSIANN-CDR presents better values for correlation than GPCP. Conversely, GPCP presents better bias than PERSIANN-CDR with annual and seasonal biases closer to unity (from 0.90 in summer to 1.10 in winter) when PERSIANN-CDR displays a tendency for rainfall underestimation (from 0.70 to 0.74). Apart for CMORPH (bias = 1.05) and GPCP (bias = 1.10) in winter, other situations indicate rainfall underestimation by the SPPs (Table 2).
Results for the unconditional analysis are comparable to those of the conditional case (Table 3). A noticeable difference is that PERSIANN-CDR and GPCP exhibit negative (rainfall underestimation) and positive biases (rainfall overestimation) for the conditional and unconditional analysis, respectively. In addition, CMORPH exhibits worse performance in winter with lower correlation (0.47 versus 0.50), worse bias (0.75 versus 1.05; that is, −25% underestimation versus +5% overestimation), and worse variability ratio (1.77 versus 1.15) than previously shown for the conditional case, which results in a negative KGE of −0.19; compare this value against 0.18 for PERSIANN-CDR and GPCP (Table 3). The low POD of CMORPH in winter (Fig. 8) and for subfreezing temperatures (Fig. 10h) explain why the unconditional statistics are much worse than the conditional. The unconditional statistics include the missed detections whereas the conditional statistics do not.
The average bias for PERSIANN-CDR indicates an overestimation between +10% and +24% depending on the season for the unconditional analysis, which is slightly better and contrasts with the underestimation observed for the conditional case (between −26% and −30%). GPCP presents a similar overestimation between +14% and +33% for the unconditional when the conditional case presented a ±10% bias. Overall CMORPH presents the best values for the biases with average values between −8% and +5% (conditional and unconditional) except for the unconditional case in winter with a significant underestimation (−25%).
Those results are consistent with Beck et al. (2019) that found the best KGE scores on an annual basis for CMORPH, PERSIANN-CDR, and GPCP, respectively (see Fig. 2 in Beck et al. 2019). In addition to the fact that our analysis is performed over a limited number of gauges, our slightly worse results (comparison of the median values) are likely due to the fact that Beck et al. (2019) uses Stage IV for validation. Because Stage IV and the SPPs use (at least partially) the same gauges for bias adjustment procedures the evaluation is not completely independent and could explain the higher KGE scores. On a seasonal basis, similar ranges are obtained for the KGE scores (see Fig. 3 in Beck et al. 2019). Here and there, we find better results in summer for CMORPH than for PERSIANN-CDR and GPCP. In winter, PERSIANN-CDR and GPCP display better KGE scores than CMORPH which displays an important variability for KGE including negative values from KGE ranging from −0.6 < KGE < +0.6 (Beck et al. 2019).
The biggest differences are found for negative temperatures (T < 0°C), with CMORPH displaying the worst performances for the unconditional case (i.e., lowest KGE and correlation, rainfall underestimation, higher variability ratio). For the conditional case, CMORPH average statistics, although degraded when compared to annual and summer precipitation, are comparable with the other SPPs (Fig. 14). However, results for CMORPH present a wider spread for the values of KGE, correlation, bias, and variability ratio than for PERSIANN-CDR and GPCP.
Results indicate that PERSIANN-CDR displays better results for correlation and bias for the unconditional case and better results for the variability ratio for the conditional case (i.e., when comparing average values regardless of the season and temperature values). GPCP presents better results for the bias and variability ratio for the conditional case and better results for the correlation for the unconditional case. For CMORPH, the distinction is seasonal with better results on an annual basis and for summer for the unconditional case and better results for winter and negative temperature for the conditional case. Although it might be difficult to link the results of the conditional–unconditional analysis to specific differences among three algorithms, the contrast could be partially due to the bias adjustment techniques. The unconditional analysis represents the evaluation of average daily rainfall (i.e., on all events YgYs + NgYs + YgNs + NgNs). Conversely, the conditional analysis is an evaluation of the SPPs against in situ data when rain is detected simultaneously at the SPP pixel and at the gauge (i.e., only on the hits YoYs). For instance, the monthly bias adjustment procedure does not affect the number of rainy days of the unadjusted version of the SPP—just the rainfall totals. This explains why PERSIANN-CDR and GPCP display similar average daily rainfall despite the fact that the number of rainy days for PERSIANN-CDR is almost twofold the number of rain days of GPCP. Differently than for the two other SPPs, CMORPH uses a daily bias adjustment against in situ CPC gauge analysis (with a 16-day moving window). Because the bias adjustment is performed on a daily scale instead of on average monthly rainfall totals, this could explain in part why the differences between unconditional and conditional cases are of a smaller magnitude when compared to the two other SPPs and why CMORPH displays better results for the conditional case during the cold season and for negative temperatures.
d. Overall results for the daily performance analysis
Figure 15 provides a summary of the evaluation of the three SPPs. Results are also provided for spring (MAM) and fall (SON). Although those transition seasons were not part of the results presented previously, they provide a more complete picture of SPP performance on a seasonal basis. For each metric, namely, hits, accuracy, false alarm ratio, probability of detection, probability of false detection, Kling–Gupta efficiency, correlation coefficient, bias, and variability ratio, the figure indicates the dataset that displays the best agreement with USCRN stations with respect to the season or the atmospheric conditions. The results are reported in a winner-takes-all approach based on the average values and do not account for each SPP degree of performance (i.e., range of variability).
SPP performance summary with respect to USCRN for annual (YEA), spring (MAM), summer (JJA), fall (SON), and winter (DJF). Metrics are hits (YY) accuracy (ACC), false alarm ratio (FAR), probability of detection (POD), probability of false detection (POFD), Kling–Gupta efficiency (KGE), correlation coefficient (CORR), bias (BIAS), and variability ratio (VAR). For the last four metrics, results are indicated conditionally (C) and unconditionally (U). For each situation, we report which SPP—CMORPH (C), PERSIANN-CDR (P), or GPCP (G)—presents the best performance.
Citation: Journal of Hydrometeorology 24, 9; 10.1175/JHM-D-22-0108.1
Overall, CMORPH displays the best objective performances when compared to the two other SPPs on an annual basis and for warm precipitation (spring, summer, fall, T > 0°C) both conditionally and unconditionally. For cold precipitation (winter, T < 0°C), the performances of the three SPPs are split over the ensemble of metrics, with PERSIANN-CDR and GPCP presenting the best performances. CMORPH performances are degraded during winter and even more so when considering negative temperatures (T < 0°C) with better statistics for the conditional case versus the unconditional case.
Apart from the annual basis, PERSIANN-CDR exhibits a better POD than the two other SPPs. Because PERSIANN-CDR displays a higher number of rainy days than the two other SPPs (as shown in Figs. 3 and 4), the probability of detecting rainfall events observed at the ground is higher (i.e., higher POD) and so is the probability of false rainfall detection (i.e., higher POFD) (as shown in Figs. 8–10). Counterintuitively, a better statistic is not necessarily an indicator of a better performance. Similarly, a better POFD (i.e., a lower probability of false detection) for CMORPH in winter and for negative average temperatures could be attributed to a degraded detection of cold season precipitation (as shown in Figs. 8–10).
5. Summary and conclusions
We have compared the performance of three SPPs—PERSIANN-CDR, CMORPH, and GPCP—part of the NOAA/Climate Data Record program. The evaluation was performed over CONUS against USCRN stations that are not used in any bias adjustment procedure of the three SPPs. This analysis extends previous work (Prat et al. 2021) by looking specifically at the ability of each SPP to capture cold season precipitation and to quantify the differences between cold and warm season performances. The main conclusions of the study can be summarized as follows:
-
While the three SPPs display comparable long-term average precipitation, significant differences were found in terms of daily precipitation occurrences and maximum daily rainfall. PERSIANN-CDR displayed a higher number of daily rainfall events while CMORPH displays higher daily maximum rainfall. Maximum rainfall for PERSIANN-CDR and GPCP were about 60%–70% and 40%–60% those of CMORPH, respectively (from the 99th percentile to the daily max).
-
In addition to seasonal rainfall variability, differences for rainfall occurrences were particularly significant for CMORPH at higher latitudes in winter due to a lack of rainfall detection over snow covered surfaces.
-
For daily rainfall, the distributions of concurrent events show that all SPPs tend to underestimate daily rainfall regardless of the season and atmospheric conditions. CMORPH presents the best agreement with USCRN throughout the distribution for concurrent rainfall events in summer, when compared to that of PERSIANN-CDR and GPCP.
-
The three datasets displayed better performance during the warm season when compared to the cold season. For the warm season, CMORPH had a higher fraction of hits (49%) when compared to PERSIANN-CDR and GPCP (41%). The fraction of missed events remained relatively low around 10% for the three SPPs, while the fraction of false alarms was higher (from 53% for PERSIANN-CDR to 43% for CMORPH). For the cold season, contingency statistics worsened for the three SPPs with CMORPH displaying the lowest fraction of hits (26%) when compared to PERSIANN-CDR and GPCP (33%–35%). CMORPH displayed a high fraction of missed events (42%) while PERSIANN-CDR exhibited a high fraction of false alarms (59%). The performance of CMORPH was highly dependent on the season while the performances of PERSIANN-CDR and GPCP were more homogeneous throughout the year.
-
During the warm season, CMORPH displayed the best performance among the three SPPs for the selected metrics including accuracy, FAR, POD, and POFD. CMORPH showed smaller biases and higher correlations when compared to PERSIANN-CDR and GPCP, both conditionally (for concurrent rainfall events observed simultaneously by the satellite and the USCRN stations) and unconditionally (for all records indistinctively).
-
Among the three SPPs, CMORPH presented higher values of KGE that combine correlation, bias, and variability into one objective performance score. This was the case on an annual basis and for the warmer seasons (spring, summer, fall) for which there are only sporadic snow, or frozen and mixed precipitation events for conditional or unconditional cases alike.
-
In winter, CMORPH showed the worst performance when compared to the two other SPPs. The degradation in terms of performance was particularly important toward higher latitudes over areas experiencing recurring snow or frozen and mixed precipitation. The lower values of POD, bias, correlation, and KGE observed for CMORPH illustrated the limitations for PMW sensors to detect precipitation over snow covered surfaces.
-
The performance of CMORPH was worse for days with negative average temperature (T < 0°C) than the two other SPPs. However, the performance metrics improved when computed for events detected simultaneously by the satellite and the USCRN station (i.e., conditional case). Although significantly better than for the unconditional case only, CMORPH performance remained lower than PERSIANN-CDR and GPCP.
Acknowledgments.
This research was supported by NOAA through the Cooperative Institute for Satellite Earth System Studies (CISESS) under Cooperative Agreement NA19NES4320002.
Data availability statement.
The three satellite precipitation datasets used during this study are part of the NOAA/Climate Data Record Program (CDR) and are openly available from the NOAA National Centers for Environmental Information. The PERSIANN Climate Data Record dataset (PERSIANN-CDR) is available at https://doi.org/10.7289/V51V5BWQ. The Climate Prediction Center (CPC) Morphing Technique (MORPH) Climate Data Record (CMORPH) is available at https://doi.org/10.25921/w9va-q159. The Global Precipitation Climatology Project (GPCP) Daily Precipitation Analysis Climate Data Record (CDR) dataset is available at https://doi.org/10.7289/V5RX998Z. The hourly data from the U.S. Climate Reference Network (USCRN) are openly available from NOAA/NCEI at ftp://ftp.ncei.noaa.gov/pub/data/uscrn/products/hourly02.
REFERENCES
Adler, R. F., M. Sapiano, and J.-J. Wang, 2017a: Precipitation – Global Precipitation Climatology Project (GPCP) monthly (01B-34). Climate Algorithm Theoretical Basis Doc. (C-ATBD), revision 2, 33 pp., https://www1.ncdc.noaa.gov/pub/data/sds/cdr/CDRs/Precipitation_GPCP-Monthly/AlgorithmDescription_01B-34.pdf.
Adler, R. F., M. Sapiano, and J.-J. Wang, 2017b: Global Precipitation Climatology Project (GPCP) daily analysis precipitation – GPCP daily CDR. Climate Algorithm Theoretical Basis Doc. (C-ATBD), 21 pp., https://www1.ncdc.noaa.gov/pub/data/sds/cdr/CDRs/Precipitation_GPCP-Daily/AlgorithmDescription_01B-35.pdf.
Adler, R. F., and Coauthors, 2018: The Global Precipitation Climatology Project (GPCP) monthly analysis (new version 2.3) and a review of 2017 global precipitation. Atmosphere, 9, 138, https://doi.org/10.3390/atmos9040138.
AghaKouchak, A., A. Behrangi, S. Sorooshian, K. Hsu, and E. Amitai, 2011: Evaluation of satellite-retrieved extreme precipitation rates across the central United States. J. Geophys. Res., 116, D02115, https://doi.org/10.1029/2010JD014741.
AghaKouchak, A., A. Mehran, H. Norouzi, and A. Behrangi, 2012: Systematic and random error components in satellite precipitation datasets. Geophys. Res. Lett., 39, L09406, https://doi.org/10.1029/2012GL051592.
Ashouri, H., K. L. Hsu, S. Sorooshian, D. K. Braithwaite, K. R. Knapp, L. D. Cecil, B. R. Nelson, and O. P. Prat, 2015: PERSIANN-CDR: Daily precipitation climate data record from multisatellite observations for hydrological and climate studies. Bull. Amer. Meteor. Soc., 96, 69–83, https://doi.org/10.1175/BAMS-D-13-00068.1.
Beck, H. E., and Coauthors, 2017: Global-scale evaluation of 22 precipitation datasets using gauge observations and hydrological modeling. Hydrol. Earth Syst. Sci., 21, 6201–6217, https://doi.org/10.5194/hess-21-6201-2017.
Beck, H. E., and Coauthors, 2019: Daily evaluation of 26 precipitation datasets using Stage-IV gauge-radar data for the CONUS. Hydrol. Earth Syst. Sci., 23, 207–224, https://doi.org/10.5194/hess-23-207-2019.
Diamond, H. J., and Coauthors, 2013: U.S. Climate Reference Network after one decade of operations: Status and assessment. Bull. Amer. Meteor. Soc., 94, 485–498, https://doi.org/10.1175/BAMS-D-12-00170.1.
Ferraro, R., B. R. Nelson, T. Smith, and O. P. Prat, 2018: The AMSU-based Hydrological Bundle Climate Data Record—Description and comparison with other data sets. Remote Sens., 10, 1640, https://doi.org/10.3390/rs10101640.
Gebremichael, M., M. M. Bitew, F. A. Hirpa, and G. N. Tesfay, 2014: Accuracy of satellite rainfall estimates in the Blue Nile Basin: Lowland plain versus highland mountain. Water Resour. Res., 50, 8775–8790, https://doi.org/10.1002/2013WR014500.
Griffith, C. G., W. L. Woodley, P. G. Grube, D. W. Martín, J. Stout, and D. N. Sikdar, 1978: Rain estimates from geosynchronous satellite imagery: Visible and infrared studies. Mon. Wea. Rev., 106, 1153–1171, https://doi.org/10.1175/1520-0493(1978)106<1153:REFGSI>2.0.CO;2.
Gupta, H. V., H. Kling, K. K. Yilmaz, and G. F. Martinez, 2009: Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. J. Hydrol., 377, 80–91, https://doi.org/10.1016/j.jhydrol.2009.08.003.
Hirpa, F. A., M. Gebremichael, and T. Hopson, 2010: Evaluation of high-resolution satellite precipitation products over very complex terrain in Ethiopia. J. Appl. Meteor. Climatol., 49, 1044–1051, https://doi.org/10.1175/2009JAMC2298.1.
Hsu, K., H. Ashouri, D. Braithwaite, and S. Sorooshian, 2014: Precipitation PERSIANN-CDR. Climate Algorithm Theoretical Basis Doc. (C-ATBD), revision 2, 30 pp., https://www1.ncdc.noaa.gov/pub/data/sds/cdr/CDRs/PERSIANN/AlgorithmDescription_01B-16.pdf.
Huffman, G. J., R. F. Adler, M. Morrissey, D. Bolvin, S. Curtis, R. Joyce, B. McGavock, and J. Susskind, 2001: Global precipitation at one-degree daily resolution from multi-satellite observations. J. Hydrometeor., 2, 36–50, https://doi.org/10.1175/1525-7541(2001)002<0036:GPAODD>2.0.CO;2.
Joyce, R., J. Janowiak, P. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487–503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.
Katiraie-Boroujerdy, P.-S., A. A. Asanjan, K.-L. Hsu, and S. Sorooshian, 2017: Intercomparison of PERSIANN-CDR andTRMM-3B42V7 precipitation estimates at monthly and daily time scales. Atmos. Res., 193, 36–49, https://doi.org/10.1016/j.atmosres.2017.04.005.
Kidd, C., and G. Huffman, 2011: Global precipitation measurement. Meteor. Appl., 18, 334–353, https://doi.org/10.1002/met.284.
Kidd, C., V. Levizzani, and S. Laviola, 2010: Quantitative precipitation estimation from Earth observation satellites. Rainfall State of the Science, Geophys. Monogr., Vol. 191, Amer. Geophys. Union, 127–158.
Kling, H., M. Fuchs, and M. Paulin, 2012: Runoff conditions in the upper Danube basin under an ensemble of climate change scenarios. J. Hydrol., 424–425, 264–277, https://doi.org/10.1016/j.jhydrol.2012.01.011.
Knapp, K. R., and Coauthors, 2011: Globally gridded satellite observations for climate studies. Bull. Amer. Meteor. Soc., 92, 893–907, https://doi.org/10.1175/2011BAMS3039.1.
Kucera, P. A., E. E. Ebert, F. J. Turk, V. Levizzani, D. Kirschbaum, F. J. Tapiador, A. Loew, and M. Borsche, 2013: Precipitation from space: Advancing Earth system science. Bull. Amer. Meteor. Soc., 94, 365–375, https://doi.org/10.1175/BAMS-D-11-00171.1.
Leeper, R. D., M. A. Palecki, and E. Davis, 2015: Methods to calculate precipitation from weighing-bucket gauges with redundant depth measurements. J. Atmos. Oceanic Technol., 32, 1179–1190, https://doi.org/10.1175/JTECH-D-14-00185.1.
Massari, C., W. Crow, and L. Brocca, 2017: An assessment of the performance of global rainfall estimates without ground-based observations. Hydrol. Earth Syst. Sci., 21, 4347–4361, https://doi.org/10.5194/hess-21-4347-2017.
Menne, M. J., I. Durre, S. Vose, B. E. Gleason, and T. G. Houston, 2012: An overview of the Global Historical Climatology Network-Daily database. J. Atmos. Oceanic Technol., 29, 897–910, https://doi.org/10.1175/JTECH-D-11-00103.1.
Miao, C., H. Ashouri, K.-L. Hsu, S. Sorooshian, and Q. Duan, 2015: Evaluation of the PERSIANN-CDR daily rainfall estimates in capturing the behavior of extreme precipitation events over China. J. Hydrometeor., 16, 1387–1396, https://doi.org/10.1175/JHM-D-14-0174.1.
Michaelides, S., V. Levizzani, E. Anagnostou, P. Bauer, T. Kasparis, and J. E. Lane, 2009: Precipitation: Measurement, remote sensing, climatology and modeling. Atmos. Res., 94, 512–533, https://doi.org/10.1016/j.atmosres.2009.08.017.
Nelson, B. R., O. P. Prat, and R. D. Leeper, 2021a: Using ancillary information from radar-based observations and rain gauges to identify error and bias. J. Hydrometeor., 22, 1249–1258, https://doi.org/10.1175/JHM-D-20-0193.1.
Nelson, B. R., O. P. Prat, and R. D. Leeper, 2021b: An investigation of NEXRAD-based quantitative precipitation estimates in Alaska. Remote Sens., 13, 3202, https://doi.org/10.3390/rs13163202.
Nguyen, P., M. Ombadi, S. Sorooshian, K.-L. Hsu, A. AghaKouchak, D. Braithwaite, H. Ashouri, and A. R. Thorstensen, 2018: The PERSIANN family of global satellite precipitation data: A review and evaluation of products. Hydrol. Earth Syst. Sci., 22, 5801–5816, https://doi.org/10.5194/hess-22-5801-2018.
Prat, O. P., and B. R. Nelson, 2015: Evaluation of precipitation estimates over CONUS derived from satellite, radar, and rain gauge data sets at daily to annual scales (2002–2012). Hydrol. Earth Syst. Sci., 19, 2037–2056, https://doi.org/10.5194/hess-19-2037-2015.
Prat, O. P., and B. R. Nelson, 2020: Satellite precipitation measurements and extreme rainfall. Satellite Precipitation Measurement, V. Levizzani et al., Eds., Vol. 2, Springer, 761–790, https://doi.org/10.1007/978-3-030-35798-6_16.
Prat, O. P., R. D. Leeper, J. E. Bell, B. R. Nelson, J. Adams, and S. Ansari, 2018: Toward earlier drought detection using remotely sensed precipitation data from the Reference Environmental Data Record (REDR) CMORPH. Geophysical Research Abstracts, Vol. 20, Abstract EGU2018-11468-1, https://meetingorganizer.copernicus.org/EGU2018/EGU2018-11468-1.pdf.
Prat, O. P., A. M. Courtright, R. D. Leeper, B. R. Nelson, R. Bilotta, J. Adams, and S. Ansari, 2020: Operational near-real time drought monitoring using global satellite precipitation estimates. 2020 EGU General Assembly, Online, EGU, EGU2020-14871, https://doi.org/10.5194/egusphere-egu2020-14871.
Prat, O. P., B. R. Nelson, E. Nick, and R. Leeper, 2021: Global evaluation of gridded satellite precipitation products from the NOAA Climate Data Record program. J. Hydrometeor., 22, 2291–2310, https://doi.org/10.1175/JHM-D-20-0246.1.
Sadeghi, L., B. Saghafian, and S. Moazami, 2019: Evaluation of IMERG and MRMS remotely sensed snowfall products. Int. J. Remote Sens., 40, 4175–4192, https://doi.org/10.1080/01431161.2018.1562259.
Scofield, R. A., 1987: The NESDIS operational convective precipitation estimation technique. Mon. Wea. Rev., 115, 1773–1793, https://doi.org/10.1175/1520-0493(1987)115<1773:TNOCPE>2.0.CO;2.
Serrat-Capdevila, A., J. B. Valdes, and E. Z. Stakhiv, 2014: Water management applications for satellite precipitation products: Synthesis and recommendations. J. Amer. Water Resour. Assoc., 50, 509–525, https://doi.org/10.1111/jawr.12140.
Sun, Q., C. Miao, Q. Duan, H. Ashouri, S. Sorooshian, and K.-L. Hsu, 2018: A review of global precipitation datasets: Data sources, estimation, and intercomparisons. Rev. Geophys., 56, 79–107, https://doi.org/10.1002/2017RG000574.
Tan, M. L., and H. Santo, 2018: Comparison of GPM IMERG, TMPA 3B42 and PERSIANN-CDR satellite precipitation products over Malaysia. Atmos. Res., 202, 63–76, https://doi.org/10.1016/j.atmosres.2017.11.006.
Tapiador, F. J., A. Y. Hou, M. de Castro, R. Checa, F. Cuartero, and A. P. Barros, 2011: Precipitation estimates for hydroelectricity. Energy Environ. Sci., 4, 4435–4448, https://doi.org/10.1039/c1ee01745d.
Tapiador, F. J., and Coauthors, 2012: Global precipitation measurement: Methods, datasets and applications. Atmos. Res., 104–105, 70–97, https://doi.org/10.1016/j.atmosres.2011.10.021.
Tian, Y., and C. D. Peters-Lidard, 2010: A global map of uncertainties in satellite-based precipitation measurements. Geophys. Res. Lett., 37, L24407, https://doi.org/10.1029/2010GL046008.
Tian, Y., C. D. Peters-Lidard, J. B. Eylander, R. J. Joyce, and G. J. Huffman, 2009: Component analysis of errors in satellite based precipitation estimates. J. Geophys. Res., 114, D24101, https://doi.org/10.1029/2009JD011949.
Xie, P., J. E. Janowiak, P A. Arkin, R. Adler, A. Gruber, R. Ferraro, G. J. Huffman, and S. Curtis, 2003: GPCP pentad precipitation analyses: An experimental dataset based on gauge observations and satellite estimates. J. Climate, 16, 2197–2214, https://doi.org/10.1175/2769.1.
Xie, P., M. Chen, and W. Shi, 2010: CPC unified gauge-based analysis of global daily precipitation. 24th Conf. on Hydrology, Atlanta, GA, Amer. Meteor. Soc., 2.3A, https://ams.confex.com/ams/90annual/webprogram/Paper163676.html.
Xie, P., R. Joyce, S. Wu, S.-H. Yoo, Y. Yarosh, F. Sun, and R. Lin, 2017: Reprocessed, bias-corrected CMORPH global high-resolution precipitation estimates from 1998. J. Hydrometeor., 18, 1617–1641, https://doi.org/10.1175/JHM-D-16-0168.1.
Xie, P., R. Joyce, and S. Wu, 2018: Bias-corrected CMORPH high-resolution global precipitation estimates. Climate Algorithm Theoretical Basis Doc. (C-ATBD), 35 pp., https://www1.ncdc.noaa.gov/pub/data/sds/cdr/CDRs/Precipitation-CMORPH/AlgorithmDescription_01B-23.pdf.
Yang, W., V. O. John, X. Zhao, H. Lu, and K. R. Knapp, 2016: Satellite climate data records: Development, applications, and societal benefits. Remote Sens., 8, 331, https://doi.org/10.3390/rs8040331.