1. Introduction
The United Nations’ World Water Day and World Meteorological Day events shared the same theme in 2020: climate and water. Precipitation redistributes water around the planet and plays a significant role in the interaction between the hydrosphere, atmosphere, and biosphere (Allen and Ingram 2002). With a changing climate, research on the spatiotemporal distribution of precipitation becomes even more important as changes in precipitation patterns directly affect the access to freshwater resources (Hou et al. 2014; Skofronick-Jackson et al. 2017). Furthermore, high-impact precipitation events such as floods, droughts, and blizzards have serious socioeconomic implications being central to the concept of sustainable development. Therefore, high-quality precipitation observations lay the foundation for research in hydrology, meteorology, and climate with major implications to economy, agriculture, transportation, urban developments, and many other applications.
Standard automated precipitation measurements using the surface gauge network provide a relatively accurate precipitation amount at specific point locations. Some drawbacks of gauge networks that make it difficult to precisely estimate the spatiotemporal distribution of precipitation events are their low density in remote areas, their uneven spatial distribution, and the lack of representativeness of some gauges (Morrissey et al. 1995; Villarini and Krajewski 2008). With the rapid development of satellite remote sensing technology and satellite-borne instruments (Huffman et al. 2001), space-based precipitation observations have become one of the crucial methods of acquiring global precipitation data (Adler et al. 2003; Bellerby et al. 2000). Remote sensing is particularly useful for monitoring oceans and inland remote areas where it is difficult to deploy ground-based sensors (Battaglia et al. 2020).
Space-based measurements have varying accuracy and precision due to atmospheric attenuation, sensor sensitivities, time scales, rainfall rates, and terrain types (Huffman et al. 2012). Therefore, the accurate assessment of satellite products is necessary to help developers enhance retrieval algorithm performance, while also benefiting users by the inclusion of uncertainty quantification.
As part of the Global Precipitation Measurement (GPM), the Integrated Multi-satellitE Retrievals for GPM (IMERG) algorithm provides a high-resolution global scale satellite-based precipitation product. With the large domain under its jurisdiction, it is not surprising that Environment and Climate Change Canada (ECCC) has a high interest in IMERG to augment its national observation network, whose uneven distribution and low density of precipitation gauges in some areas present a challenge for monitoring (Mekis et al. 2018). In the Canadian Great Lakes region, on-water gauge locations are nonexistent, despite the high density of ground stations in the area. Hence, it is necessary to use remote sensing products to fill these observation gaps.
Numerous IMERG studies have been performed around the world. In Asia, Tang et al. (2020) pointed out the good quality of IMERG products by comparing them to nine satellite and reanalysis datasets in China. In Europe, O et al. (2017) used a dense gauge network in Austria to evaluate IMERG emphasizing the significant advantage of the Final Run. In the Middle East, Mohammed et al. (2020) compared IMERG with gauges located in the Kingdom of Saudi Arabia, analyzing the seasonal and topographical influence in an arid region. Based on the comparison between IMERG and the Tropical Rainfall Measuring Mission Multi-Satellite Precipitation Analysis (TMPA) in Africa, Dezfuli et al. (2017) demonstrated that IMERG can relatively better capture the diurnal cycle of precipitation. Navarro et al. (2019) assessed IMERG over Europe and pointed out the bias of the Final Run in various regions. In South America, a gridbox-level assessment by Gadelha et al. (2019) showed the potential of IMERG in ungauged areas over Brazil. In North America, Beck et al. (2019) performed a comparison between IMERG and Stage-IV gauge–radar gridded products in the contiguous United States and showed IMERG to perform well among 26 daily precipitation datasets. Asong et al. (2017) analyzed the capacity of IMERG by comparing gauges indicating the various performance of IMERG for different zones around southern Canada. For numerical weather prediction (NWP) model application, Boluwade et al. (2017) verified the potential of IMERG to enhance the performance of a near-real-time operational gridded precipitation product, the Canadian Precipitation Analysis (CaPA) (Fortin et al. 2015), which is a near-real-time operational multisource fusion gridded precipitation product with a minimum 2.5-km spatial and 6-h time resolution, for one summer season.
Many previous studies focused only on a single variable, precipitation rate, at a single time scale, and for a single IMERG Run using v05 or older of the precipitation product. Some studies offered the IMERG validation around coastal areas (Sui et al. 2020; Pradhan et al. 2022) but few for interior waters, distance from the coastline is a major challenge (Derin et al. 2022). Thus, the differences in performance of multiple variables among IMERG runs, seasons and time scales surrounding large lakes are not fully understood. Meanwhile, Tan et al. (2016) validated the capacity difference among sensors but lacked seasonal comparison. There has been little research on the IMERG Quality Index (QI), and this research gap needs to be filled so IMERG users can use better guidance on the IMERG performance for their application. In addition, how lake effect precipitation—a typical characteristic of the Great Lakes—impacts the performance of IMERG is still unknown.
In the present study, within the Great Lakes region of Canada, 19 ECCC surface precipitation gauge locations—independent from IMERG processing input—were used as Ground Validation (GV) reference. The compared data include hourly GV observations and IMERG precipitation retrievals during the 66-month period between April 2014 and September 2019. This period almost covers all months from GPM launched (February 2014) to this study end (September 2019). Only the first 2 months are not included since we believe data quality of IMERG was not stable then. Multiple continuous and categorical metrics were applied to assess the latest IMERG version v06’s performance in this region. The Early and Final Run were analyzed both for near-real-time and non-real-time applications.
The performance of half-hourly IMERG was evaluated around the Great Lakes region at several submonthly scales. In addition to comparing precipitation amounts at time scales of 1, 3, 6, and 24 h, the performance of underlying passive microwave (PMW) contributions was assessed, the IMERG half-hourly Quality Index field was evaluated, and the impact of lake effects to IMERG was discussed. The paper is structured as follows. Section 2 describes the study area and data sources. Section 3 indicates the data matching criteria, the approach for generating various time scales, the selection of precipitation event thresholds, and the statistical validation methods used in the analysis. Section 4 comprises the validation results, and section 5 follows with discussion. The study is concluded in section 6. A glossary is in appendix B.
2. Study area and data source
a. Study area
The Great Lakes is a chain of deep freshwater lakes in east-central North America comprising Lakes Superior, Michigan, Huron, Erie, and Ontario. It is the largest group of freshwater lakes on Earth, and second largest by total volume, containing 21% of the world’s surface freshwater by volume and 84% of the surface freshwater available in North America (EPA 2021). About 34 million people rely on the Great Lakes for drinking water, jobs, and their way of life (ELPC 2019).
The Great Lakes play an important role in influencing local weather patterns across the region. The Great Lakes impact daily weather by 1) moderating temperatures in all seasons, producing cooler summers and warmer winters; 2) increasing cloud cover and precipitation over and just downwind of the lakes during winter; and 3) decreasing summertime convective clouds and rainfall over the lakes (Scott and Huff 1996; Notaro et al. 2013).
The North American Great Lakes region has a humid continental climate with typical upstream influences from dry, cold Arctic systems from the north; mild Pacific air masses from the west; and warm, wet tropical systems from the south (e.g., the Gulf of Mexico) (Bush and Lemmen 2019; Mailhot et al. 2019). The moderating effect of the lakes on the local climatology is due to the water temperature changes lagging land temperature. The lakes are effectively gaining heat in summer and releasing the heat during cooler months. This heat storage results in cooler springs, warmer falls, delayed frosts, and localized lake effect snow. On average, under a range of emission scenarios, most regional climate model (RCM) studies project a lowering of future lake levels by 0.2 m for the 30-yr time period centered on the 2050s, as compared to the 1971–2000 mean (Derksen et al. 2019). Reduced lake ice cover and enhanced evaporation may lead to increased lake-effect snowfall in the near term, but rising temperatures will cause more winter precipitation to fall as rain as opposed to snow across the region later in the century.
The study domain, defined by the boundary of the Great Lakes basin, encompasses the Canadian side of the hydrological drainage extent for which local waterways feed in the North American Great Lakes (Fig. 1). Water quality and quantity are areas of priority for Canada’s environment, both of which are dependent on precipitation. Validating and evaluating the quality of GPM in the Great Lakes area would enable the use of this unique dataset for science, services, and regulations in Canada, complementing in a cost-effective manner the data provided by the surface monitoring network.
Drainage basin outlined by black lines. Ground validation map of 19 ECCC surface precipitation stations in the Great Lakes basin (hydrological boundary in gray). Symbols denote the weighing gauge type, where triangles are Pluvio and circles are Geonor. Colored black are near-lakeshore locations (within 20 km of a lakeshore) with high probability of lake effect, and white are inland sites (greater than 40 km from a lake).
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
b. Data sources
1) IMERG v06
The GPM Core Observatory (GPM-CO) satellite, launched in February 2014, comprises the GPM Microwave Imager (GMI) and the Dual-Frequency Precipitation Radar (DPR) (Hou et al. 2014). GPM-CO is observing precipitation alongside other passive microwave (PMW) low-Earth-orbit (LEO) satellites as well as IR sensors on geostationary-Earth-orbit (GEO) satellites to form the GPM constellation. The best estimate of precipitation by GPM-CO is derived using the combined radar–radiometer (CORRA) algorithm and serves as a calibration reference within the GPM constellation (Grecu et al. 2016).
The IMERG algorithm combines data sources from the GPM constellation, other passive microwave and infrared (IR) satellites, and ancillary input from forecast models and surface type maps, to yield a global high-resolution (30 min and 0.1°) gridded precipitation estimate. As a key component of the IMERG algorithm, MORPHING (Joyce et al. 2004; Joyce and Xie 2011) fills in gaps in the PMW precipitation field using motion vectors. Table 1 shows the attributes of the following five PMW sensors: Advanced Microwave Scanning Radiometer 2 (AMSR2), Special Sensor Microwave Imager/Sounder (SSMIS), Microwave Humidity Sounder (MHS), GPM Microwave Imager (GMI), and Advanced Technology Microwave Sounder (ATMS).
Attributes of passive microwave sensor available to the IMERG algorithm during 2014–19. The last column (carrying satellite) may differ according to recording era.
Further adjustments are made using monthly ground precipitation gauge analyses from the Deutscher Wetterdienst (DWD) Global Precipitation Climatology Centre (GPCC) (Huffman et al. 2020). The monthly IMERG and monthly GPCC are merged with weights inversely based on their estimated error variance to create a monthly satellite–gauge combination. The scaling factors are calculated based on this monthly satellite–gauge product and the monthly IMERG. Additional details can be found within the Algorithm Theoretical Basis Document (ATBD) for IMERG (Huffman et al. 2019b), from which is distilled Table 2, characterizing the differences of IMERG product runs (Early, Late, and Final). Final Run uses reanalysis data instead of forecast data in various parts of the algorithm and has a calibration window that is centered rather than trailing.
The comparison of IMERG product runs.
Five IMERG half-hourly product fields are used in this study: 1) precipitationCal (PreCal), a calibrated precipitation estimate; 2) precipitationUncal (UnCal), a precipitation estimate before gauge correction; 3) HQprecipitation (HQpre), a merged microwave-only precipitation estimate; 4) HQprecipSource (HQsrc), a sensor index of the “HQpre” field; and 5) PrecipitationQualityIndex (QI), a quality index of the PreCal field.
The latest version of IMERG (v06) was released in 2019. Compared with the previous version (v05), major changes are improvements to the satellite intercalibration and to the Kalman filter process (Tan et al. 2019). In this study, the latest IMERG v06 products including the Early and Final Run half-hourly (HHR) were used during the study period from April 2014 to September 2019, corresponding to 66 months. IMERG coverage from 60°N to 60°S can completely observe the Great Lakes region. All IMERG v06 data (Huffman et al. 2019a) were acquired from NASA GPM data portal at https://gpm.nasa.gov/data/directory.
2) Surface gauge precipitation
The ECCC network used in this study (Table 3 and Fig. 1) includes surface precipitation stations located in the Great Lakes area. These measurements as per the Canadian Manual of Surface Weather Observations Standards (MANOBS; ECCC 2021a) are subject to automated quality control procedures and available from ECCC’s Historical Climate Data archive (ECCC 2021b) at various time scales ranging from hourly to monthly. The originally selected 55 locations were reduced to 27 stations; the excluded 28 stations were part of the GPCC calibration for IMERG and are thus not independent. Furthermore, eight locations with tipping-bucket rain gauge (TBRG) instrumentation were excluded due to possible timing issues (the slowness or partial filling of the bucket can lead to delays in reporting). Only the highest-quality stations with hourly weighing gauges (WG) were selected. The most reliable 19 locations consist of 13 Geonor and 6 Pluvio gauges.
List of 19 independent stations not included in GPCC analyses used to adjust IMERG. Letters “G” and “P” stand for Geonor and Pluvio precipitation weighing gauges, respectively.
The Geonor and Pluvio weighing gauges are one of the most studied gauges at present. The Geonor T-200B (600-mm capacity) uses three vibrating wire transducers to weigh the collection bucket (Mekis et al. 2018). This configuration provides redundancy as well as helps to eliminate errors that can be introduced due to diurnal thermal variations or an unlevel instrument. The similarly shaped Pluvio1 (1000-mm capacity) and Pluvio2 (1500-mm capacity) precipitation gauges also operate by measuring the bucket weight transferred through a load transfer mechanism to a load cell (Mekis et al. 2018; OTT Hydromet 2019). In the operational installation of Pluvio1, an internal processor produces a pulse output for each 0.1-mm increase in precipitation accumulation. The reporting resolution can be up to 0.001-mm precision for the newer Pluvio2 and Geonor WG. However, the minimum measurable amount is different for the Geonor and Pluvio gauges, 0.2 versus 0.1 mm, respectively.
The 19 stations used for ground verification (GV) in this study are the highest-quality locations in the region with reliable metadata information and regular station maintenance. The ECCC quality assessment procedure starts with automated basic real time checks including presence, integrity, range, and time scale consistency checks. In addition to the incoming native flags (suppressed, error, suspect), several flags are added for any suspicious data during quality control (missing, error, doubtful, inconsistency) (Mekis et al. 2018).
Currently, Double Fence Automated Reference (DFAR) and Double Fence Intercomparison Reference (DFIR) remain the most reliable ground solid precipitation “truth” measurements according to WMO Guidelines for the solid precipitation observation (Nitu et al. 2019). However, these reference stations can only be found on research sites rather than for operations and none covered the locations of interest for the study period. Hence the 19 weighing gauges operated by ECCC were used as ground reference to effectively assess IMERG seasonal performance through the study period.
The ECCC quality control (QC) flags cannot identify malfunctioning instrumentation leading to continuous zero observations. Smaller than 0.1-mm hourly Pluvio or Geonor weighing gauges values are also difficult to distinguish from signal noise (Nitu et al. 2019). To avoid incorrect model overestimation when the precipitation detected by IMERG was not captured by station data, a small monthly precipitation diagnosis method was introduced. Only total accumulated precipitation monthly values greater or equal to 1.0 mm are included in this analysis. For extreme outliers, reported values greater than 120 mm h−1 (the Canadian record is 112 mm h−1) were identified and excluded from the analysis.
After the application of all the QC steps on the hourly raw data, 93.12% of the observations by the 19 gauges were available for the study period of 66 months.
3. Methodology
To compare GV and IMERG at the same scale, the spatial and time alignment of satellite and station data requires thorough preprocessing. This study uses various matching and time-scale generation methods for comparison. Each of the three matching methods serves different purposes.
a. GV-IMERG matching
1) Spatial matching
Since spatial data obtained by interpolation will introduce artifacts (Tan et al. 2015), this study performs a grid-to-point comparison by matching each gauge to the closest IMERG grid cell. Due to the overall lower spatial variability (less convection) of convective systems in the Great Lakes area compared to convection in lower latitudes, particularly lake effect systems, the grid to point comparison should not be greatly affected by spatial variability differences (Da Silva et al. 2021). As the Great Lakes region is completely covered by IMERG, there is no invalid and missing data for most IMERG variables except for PMW related (HQpre, HQsrc) variables.
2) Observation time matching
Spurious temporal mismatches caused by failing to account for gauge reporting times may result in erroneous comparison results (Beck et al. 2019). Since ECCC’s archive data were stored in local standard time (LST) while IMERG uses coordinated universal time (UTC), a time shift of +5 h was applied to the 19 stations all located in the province of Ontario, Canada. As a result, the UTC time zone is used in all analyses throughout the paper.
3) Precipitation unit matching
The units of precipitation rate in IMERG are in millimeters per hour, while the GV’s precipitation amounts are reported in millimeters for the accumulated precipitation observation period. All IMERG data were thus converted to accumulated precipitation with units in millimeters for the same observation period.
b. Various timescale generation method
1) Upscaling IMERG and GV for PreCal for QI analysis
Apart from the GV original hourly data, which can be used directly, 3-, 6-, and 24-hourly subdatasets were also generated, with accumulated total precipitation aligned to 0000 UTC. Similarly, IMERG subdatasets for 1, 3, 6, and 24 h were calculated by accumulating the original half-hourly PreCal satellite data.
Aiming to ensure a relatively complete sample granularity, a QC approach called “inter-missing check” was applied to guarantee the sample completeness for each time resolution. In the subdatasets generation process any sample with less than 80% data over the accumulated period (samples with less than 3-, 5-, and 20-hourly observations for the 3-, 6-, and 24-h periods, respectively) were excluded from the analysis. As a result, 0.66%, 1.03%, and 1.86% of the samples from these three time scales were filtered out during this upscaling step.
The half-hourly QI (QI-HHR) has unitless values between 0 and 1. There are two simple methods to calculate QI after QI-HHR is upscaled to a higher time scale (TS): accumulating values or computing the mean. In this study, the latter method was used, and we refer to the computed value as QI-TS. Since QI values were averaged for each time scale, the range of each QI-TS value remains between 0 and 1.
2) Downscaling GV for HQpre, HQsrc analysis
At the time scale of 30 min, although PMW observations are instantaneous, they are geographically discontinuous, as they originate from different satellite platforms. Thus, they cannot simply be merged together.
A consistent HQsrc pattern between Early and Final Run was expected since the original swaths of PMW sensors are the same during the whole study period. However, some differences were found for variable count during July, August, and September of 2019 (the last three months of the study period). To avoid misleading results related to this issue, only the first 63 months period was used for the HQ analysis.
The best way to assess HQpre and HQsrc is by using 30-min or finer GV to match half-hourly IMERG. However, since not all instruments have this time scale, a novel approach called middle wet selection (MWS) method was created in this study to allow the use of hourly GV to assess half-hourly IMERG products.
In the MWS method, GV data are first converted to hourly precipitation rate. Then, every hourly GV value is split to two individual half-hourly bins sharing the same original hourly precipitation rate. Considering the discreteness of precipitation events, it is unclear whether precipitation occurs in the first or second half hour, or even both. The MWS method is introduced to pick the half-hourly bins within a certain wet process. The wet spell length is defined as greater than or equal to 3 h. The first and last hour of the precipitation event are discarded as the exact start and end time during an hour are unknown. Then, the middle part is kept, which will be of relatively high quality assuming a continuous rain or snow event during these hours. Following this procedure, the sample loss ratio is 56.86% in this study.
Note that in this study, the MWS method was only applied for continuous validation metrics, and not applied for categorical validation metrics.
c. Threshold selection function for precipitation events
As part of the assessment, the aim is to verify the performance of the satellite precipitation retrieval product (Sat) when precipitation occurs. Criteria for the selection of the lower threshold to distinguish between a precipitation event and a nonprecipitation event is crucial for this research.
In general, the diagnosis depends on GV precipitation events, but the value also can use Sat as reference since the focus of this study is IMERG. For the half hourly time scale, the threshold was set to T(0.5 h) = 0.2 mm h−1 for both GV and IMERG as the precipitation event identification threshold (T) which is the minimal amount with capacity to detect precipitation before gauge adjustment (Tan et al. 2017).
Precipitation thresholds per time scale to identify wet events.
d. Validation metrics
The eight different metrics used to validate the performance of IMERG are as follows: the relative bias (RB), root-mean-square error (RMSE), and correlation coefficient (CC) as continuous verification (CON) metrics; the probability of detection (POD), false alarm ratio (FAR), frequency bias index (FBI-1), equitable threat score (ETS), and Heidke skill score (HSS) as categorical verification (CAT) metrics. These quantities are defined in appendix A.
In terms of CAT, four different thresholds are used to identify precipitation occurrences matching four time scales. In a 6-h time scale, Fortin et al. (2015) used 0.2, 1, 2, 5, 10, and 25 mm to assess 6-h CaPA, so these six thresholds plus the 0.4 mm (6 h)−1 thresholds were used to further analyze the 6-h IMERG performance in this study.
4. Results
a. Quantitative precipitation estimation analysis
1) Overall precipitation background
Considering this was a precipitation focused study, the months November–March are the months in which significant average monthly values snowfall takes place. So we defined two seasons, the cold season (November–March) had lower precipitation amounts than in the warm season (April–October). The difference is not as significant as what could be expected from the lower moisture in the cold season, the previous detailed studies (Baijnath-Rodino and Duguay 2018; Kunkel et al. 2009) for Great Lakes shows the similar results, so it may be attributable to the availability of moisture transport from open waters in the Great Lakes region. This is also known as the lake effect (American Meteorological Society 2021), in which bodies of water acting as warm temperature reservoirs are overridden by colder air masses, thus promoting conditions for increased number of precipitation events.
Monthly mean precipitation over the study period is shown in Fig. 2 for IMERG Early and Final Runs, GV, and a climate reference derived from the 1981–2010 monthly normals of 238 locations across Ontario (ECCC 2021b). Higher monthly fluctuation is observed for GV during the warm season (April–October), compared to more consistent normals during the cold season (November–March). The Final Run monthly means overestimate mostly at the peaks after June but has a similar pattern to GV, which is not the case at the Early Run. The Early Run is more inconsistent and significantly overestimates precipitation amounts for most months.
The comparison of monthly mean total precipitation amounts for the study period April 2014–September 2019. The black line is the climate reference based on 1981–2010 normals. Other lines are amounts from the ground validation (GV) gauges (19 hourly reporting stations), the IMERG Early and Final Run products (produced half-hourly) respectively.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
2) Continuous verification results
In Fig. 3, two monthly CON metrics, RB and CC, were quantitatively compared for the Early and Final Runs at four time scales (1, 3, 6, 24 h). For clarity, only the result of 1-h time scale for the RB is shown in Fig. 3a. Results for other time scales are nearly identical; perhaps due to the nature of the upscaled reference accumulation which appears in both the numerator and denominator of the RB equation. The Early Run (green line) yields an inconsistent score. In particular, a striking increase from −14% to +52% is observed from December to March. This is likely due to the decreased performance of PMW measurements over snow covered surfaces.
Continuous verification (CON) metrics by month. Relative bias (RB) comparison, only 1-h time scale shown, among (a) Early, Final UnCal, and Final Cal. (b) GPCC correction and backward MORPHING. Vertical dotted lines demarcate five climatic periods for discussion. (c) Correlation coefficient (CC) for the Final Run at four time scales (colored). Curves denote the CC value using the left axis. Bars are the CC differences from the Early to Final Runs using the right axis.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
In comparison to the Early Run’s unsteadiness, the Final Run Cal (Fig. 3a, purple line) has a more consistent overestimation bias for all months. Also, the range of RB is tighter in the cold season (November–March) than in the warm season, 10%–15% versus 0%–25%, respectively. Thus, by merit of its lower and narrower RB range, the Final Run is more reliable. The known improvement of the Final Run over the Early Run is due to two extra processing steps: backward MORPHING (B-MORPHING) and GPCC monthly adjustment (Table 2).
Following IMERG ATBD, we use the Final Run UnCal field to study the effect of B-MORPHING application before any GPCC adjustment. Figure 3b shows the RB as B-MORPHING (orange) and GPCC (blue) are incrementally applied. Again, only the result of 1-h time scale is shown for clarity. The Final UnCal Run is also plotted for further analysis and reference on Fig. 3a (yellow line).
We divide the year into five periods as (I, II, III.a, III.b, IV) by four vertical dotted lines (Fig. 3b) based on climate characteristics of the Great Lakes region. The relevant results are as follows:
- (I) Early warm season (April–July), with the exception of April, Early Run’s overestimation is reduced by both B-MORPHING and GPCC adjustment to different extents.
- (II) Late warm season (August–September), it is surprising that the Early Cal outperformed the Final UnCal and the RB increasing after the application of B-MORPHING while GPCC shows its effectiveness in decreasing the RB during this period. This might be attributed to the difficulty of correctly capturing convective precipitation by MORPHING.
- (III)
Early cold season (October–January), this period is subdivided into III.a and III.b subperiods as described below:
- (III.a) Onset of lake effect snows (Baijnath-Rodino and Duguay 2018), in October and November, B-MORPHING apparently has no effect on reducing RB under these frequent lake effect snow events. Another possibility is that B-MORPHING does reduce the RB in synoptic scale snow but the improvement is offset by lake effect snow.
- (III.b) Lake effects over snowy land surface and partially ice-covered lakes, in December and January, B-MORPHING and GPCC act in opposite directions, B-MORPHING increases contribution to a positive RB bias while GPCC reduces the RB positive bias.
- (IV) Late cold season (February–March), lake effects are greatly reduced as the water surface becomes colder and partially ice covered, resulting in a similar RB between Cal and UnCal of the Final Run. Monthly GPCC has no effect to reduce RB while B-MORPHING results in large performance improvement from the Early Cal to Final UnCal without the impact of lake effect.
In summary, monthly GPCC is more effective in reducing the Early Run RB when there is a high frequency of convective precipitation as well as lake effect events, but shows less value in the late cold season. B-MORPHING is more effective at reducing the Early Run RB during snowy and icy land surface periods. This improvement is more obvious when lake effects are not happening. It enhances the RB for heavy precipitation processes in several months with frequent convective weather, whose rapid movement cannot be accurately predicted by B-MORPHING.
The correlation results are consistently improving for all four time scales. Figure 3c shows the larger CC improvement in the Final Run with increasing time scale from the Early Run until July. However, after July, although the CC increases in the Final Run, it does not have a strong dependency on time scale. In fact, the 24-h increasing CC is less than the 1 h in December. For the Final Run, MAM, JJA, and SON have similar good CC, although there is a slight decrease in CC in July during this period. The CC drops to the lowest value in DJF.
3) Categorical verification results
Categorical verification (CAT) is used to assess the precipitation event detection skills for each of the studied time scales (1, 3, 6, and 24 h) and their associated rate thresholds (see Table 4). Only Final Run results will be shown and discussed in Fig. 4 as the differences to the Early Run are unremarkable.
Categorical (CAT) metrics for the Final Run by month, (a) probability of detection (POD), (b) false alarm ratio (FAR), (c) Heidke skill score (HSS), and (d) frequency bias index (FBI-1), at time scales of 1, 3, 6, and 24 h.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
In terms of POD (Fig. 4a), the trends are similar among all time scales with closer to the perfect one POD with increasing time scale. The approximate POD increase is 0.13, 0.16, and 0.3 from 1–3, 6, and 24 h, respectively. Looking at the line of 24 h, which has the highest POD as representative, DJF has a much worse POD of 0.56 than 0.82 in JJA which highlights the summer–winter discrepancy.
In terms of FAR (Fig. 4b) for the 24-h time scale, April and October have the best (lowest) FAR at 0.22 and 0.21, respectively. JJA and DJF both have the higher FAR’s with maxima ∼0.6 in July and February, respectively. The month-to-month variability in JJA is larger than DJF. In terms of the differences among time scales, the FAR decreases with increasing time scale but the difference is not as marked as POD, especially from 1 to 3 h.
The HSS (Fig. 4c) confirms that DJF is the worst season for detecting precipitation events, which is consistent with its low POD and high FAR.
Finally looking at FBI-1 (Fig. 4d), winter season (DJF) has the lowest POD and the highest FAR leading to a significantly negative FBI-1 or underestimation. It means IMERG is not sensitive enough to detect precipitation events during winter. This is not surprising since IMERG does not use PMW measurements over snowy or icy surfaces but rather relies on IR measurements. By contrast, for the warm season, JJA has a dramatically positive FBI-1. This is due to fewer misses in JJA, whose misses’ number is one-third of cold season under the similar amount of hits. Curves show 24-h time scale performing best, with extremes of 0.25 in July, and −0.2 in December. MAM performance is better than SON since the latter declined rapidly to −0.1 in November due to the onset of lake effect snow. Among four time scale lines, it is clear that with the lowest negative FBI-1 in winter and highest FBI-1 in summer the 1-h time scale is the most biased product.
In conclusion, the weak performance of winter POD, FAR, and HSS results indicate the snow events and ice covered surface significantly affect PMW to detect real precipitation occurrence. Although JJA shows the best HSS but at the cost of too high FAR and FBI-1, while MAM and SON present good FAR and FBI-1.
4) Detailed study of the 6-h results
One purpose of this study was to evaluate the seasonal potential to fuse the Early or Final Run satellite product into CaPA. A preliminary experiment assimilating IMERG v04 was implemented (Boluwade et al. 2017) only of the Final Run and a single summer season; so it is important to assess the performance of the CaPA-IMERG v06 fusion over a longer period. To match CaPA characteristics, we focus on 6-h precipitation accumulation amounts.
ETS results are shown in (Figs. 5a,b) for the Early and Final Run. The best ETS scores for MAM and JJA occur at 1 mm, while for SON and DJF it is at 2 mm for the Final Run. Additionally, JJA has the best ETS scores, followed by MAM and SON. All these three seasons are far ahead of DJF with a difference of 0.13 when the threshold is no more than 5 mm. This difference gradually reduces to 0.02 at 25 mm.
CAT metrics for 6-h accumulated amounts by season, (top) equitable threat score (ETS) and (bottom) FBI-1 results for the (a),(c) Early Run and (b),(d) Final Run.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
With respect to FBI-1 results for both runs (Figs. 5c,d), although JJA has the highest overestimation for precipitation less than 5 mm, it presents overall good skill since FBI-1 is at a good level ideally around zero for the higher-precipitation events at 10 and 25 mm. By contrast, MAM and SON are the best two seasons for precipitation events below 5 mm. These results are consistent with results from the previous section (Fig. 4d). Once again, these results may be a reflection of JJA events being characterized by specific convective events while SON and MAM have a larger share of more widespread synoptic events. Contrarily, the winter DJF is the worst performing season with the lowest negative bias for events < 2 mm and extremely large FBI-1 for heavy precipitation at 10 and 25 mm, especially for 25 mm. The outliers at 25 mm may be due to the small sample size of six GV reports in that range. Also comparing the shoulder seasons, MAM is slightly better than SON as lake effect snow starts to affect the region in November.
The overall comparison (Fig. 5) between Early and Final Runs shows little change under 5 mm of precipitation amount. For greater than 5 mm, there are increased differences, and more notable in warmer seasons (MAM and JJA). This is largely because GPCC monthly data can better identify large outlier events to improve the Final Run results.
Figure 6 shows the decreasing trend of RB with intensity for both runs, stratified by seasons. Generally, IMERG overestimates light precipitation amount events while underestimating larger precipitation amount events. With respect to the Final Run (blue bars), the seasonal turning point from overestimation to underestimation occurs at the lowest binned amount (1–2 mm) in DJF (Fig. 6d), and the highest (5–10 mm) in JJA (Fig. 6b). The MAM and SON shoulder seasons (Figs. 6a,c) have their turning point intermediately at 2–5 mm. The RB of the Early Run is generally larger than the Final Run which is consistent with Fig. 3a as well.
Seasonal relative bias (RB) results of 6-h precipitation intensity.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
Looking closer at DJF (Fig. 6d), it is noted that there is a significant large RB at 81% for heavy intensities (>25 mm). Compared with the large winter FBI-1 result (Fig. 3d), it can be hypothesized that the Early Run incorrectly identified snow surface leads to a great number of false alarms in which the satellite retrieval precipitation amounts are far greater than GV. However, this bias is corrected effectively by the monthly gauge analyses with RB of the Final Run close to 0% in the >25-mm events.
The JJA season has the worst RB score, despite other good CAT scores. This is likely due to IMERG’s relatively low skill in extreme precipitation scenes such as convection.
b. PMW sensors comparison
To better assess individual PMW sensor performance, the hourly GV samples were calculated from half-hourly GV as described in the methodology section 3d. The CAT was implemented to assess the capacity of HQpre, and CON was applied to quantify the events picked by the MWS. Since the two runs’ results are very similar, only the results of the Final Run are presented.
1) Seasonal PMW CAT results
First, for half-hourly precipitation events (no less than 0.2 mm h−1), Fig. 7 shows the distribution of categorical raw metrics (hit, miss, false alarm, and correct negative) for each PMW sensor with their fractional percentage from total cases. In terms of hit rate, AMSR2, MHS, and ATMS are similar ranging from 3.3% to 3.4%, while GMI has the lowest hit rate at 3.0% and the greatest miss rate at 6.3%. By contrast, in terms of false alarm rate, GMI is the best with the lowest FAR at 2.3%, AMSR2, MHS, and ATMS have once again a similar performance with FAR between 2.6% and 2.9%, and SSMIS has the highest false alarm rate at 3.7%. Thus, the sensitivity of GMI seems too conservative.
Proportional counts of PMW sensor categorical results for Final Run and for half-hourly precipitation events (no less than 0.2 mm h−1).
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
Second, Fig. 8 shows the CAT seasonal comparison. Figure 8a indicates that all sensors have the maximum POD in JJA at >0.52. MAM has the second highest POD ∼0.42, then SON at 0.29–0.37. The minimum POD for each sensor occurs in DJF, especially for GMI and AMSR2, whose POD is below 0.2. The HSS seasonal trend (Fig. 8d) is consistent with POD.
Seasonal PMW sensor CAT metrics for the Final Run, (a) POD, (b) FAR, (c) FBI-1, and (d) HSS.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
In addition, compared to the previous PreCal results suggesting the worst FAR occurs in winter, it is a little surprising that the impact of winter season to FAR (Fig. 8b) of HQpre is not as visible as before. All sensors have their highest FAR in JJA instead of DJF. ATMS has the lowest (or best) FAR of all sensors in the winter. AMTS’s good score is likely due to the PMW sensor with the most channels, 24, permitting a broader performance range in snowfall observations.
FBI-1 (Fig. 8c) shows that the positive value only appears in JJA. Meanwhile, the negative value of FBI-1 gradually decreases in the order of MAM, SON, and DJF. This trend also agrees with the previous section conclusion. SSMIS has relatively good performance while GMI was the worst.
Finally, the low HSS score (Fig. 8d) in winter is an indication that the snow surface will lead to an increase of PMW uncertainty. However, it should be noted that the extent of decline is different from sensor to sensor. The impact of winter on the ATMS is less than for the other sensors, with an HSS of 0.299 for the ATMS. By contrast, AMSR2 and GMI are more winter influenced, especially for GMI. It shows the best HSS in both JJA and SON while suddenly declining to the second lowest (0.233) in DJF.
2) Overall PMW CON results
Examining the overall PMW CON results (Fig. 9), recall these samples are from precipitation events filtered by MWS.
Overall PMW sensor CON metrics for the Final Run, (a) root-mean-square equal (RMSE) and (b) correlation coefficient (CC). PMW sensor descriptions are in Table 1.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
For the RMSE of individual sensors (Fig. 9a), SSMIS has the lowest error at 2.648 mm h−1, followed by GMI with a little higher value of 2.727 mm h−1. The remaining sensors are all above three with MHS and ATMS in a similar range (3.206 versus 3.335 mm h−1), while AMSR2 has the worst error at 3.966 mm h−1.
In terms of CC (Fig. 9b), GMI is clearly better than the other sensors at 0.48, and ATMS is the second best at 0.433, while the remaining three sensors have a CC of less than 0.4.
Thus, GMI is the best PMW, being ranked second according to RMSE and best according to CC, and this is one of the reasons why GPM-CO satellite can be a primary measurement benchmark for GPM projects.
c. Quality Index analysis
At the half-hourly scale, according to a briefing paper by IMERG developers (Huffman 2019), the Quality Index provided is “some measure of the relative skill that might be expected from the fluctuating mix of different passive microwave- and infrared-based precipitation estimates.” To improve QI, it must be studied closely. What is the change in the reliability of the IMERG products as QI increases? Do metrics move toward their respective ideal values with increasing QI? How much do trends change for different time scales? Findings will help users understand the relationship of the performance of IMERG products with the QI.
1) QI distribution
Figure 10 shows the distribution of QI values for Early and Final Runs, with subplot variation of the QI time scale (QI-TS) from shortest (half-hour) to longest (24 h). The gray shadowed box indicates the breakpoints of 0.4 and 0.6, so the suggested 3-class “stoplight” statements may be used to interpret QI-HHR as per the IMERG developers (Huffman 2019). The classes are (0.0–0.4) = “red,” significant IR contribution with high uncertainty; (0.4–0.6) = “yellow,” the midrange is morphed microwave; (0.6–1.0) = “green,” current half-hour microwave swath data and short morphs.
QI-TS histograms (250 bins of size 0.004) of QI for Early and Final Runs, (a) half-hour (HHR), (b) 1 h, (c) 3 h, (d) 6 h, and (e) 24 h. The gray shadow box (QI-TS between 0.4 and 0.6) demarks boundaries for simplified QI interpretation (see text).
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
For QI for Half-Hourly (QI-HHR) (Fig. 10a), there is a systematic improvement (i.e., higher QI) from Early to Final Run in the interval from 0.2 to 0.65. The lack of B-MORPHING in the Early Run means more IR data with higher uncertainty are used to fill the gaps between swaths leading to a lower QI value. However, for QI-HHR > 0.65 there is no difference between Early and Late Runs is detected. This can be due to the fact that QI-HHR above 0.65 relies on direct observation by conical-scanning and cross-track-scanning radiometers (direct observation swath without morphing).
With increasing time scale (Figs. 10b–e), QI-TS distribution begins to separate into two individual components, with the second component becoming more and more concentrated. It is not surprising since QI-TS is calculated by computing the mean, and according to the Central Limit Theorem, the distribution tends toward the normal distribution even if the original variables are not normally distributed. Compared with the Final Run, the Early Run has the same standard deviation but lower mean.
2) Regression analysis by equal bins
Although the stoplight statements may be good for QI-HHR, it cannot be simply applied for larger time scales. For example, CaPA users will realize there is insufficient data with QI-6h in the last bin of [0.6, 1] when they are using the Early Run. Hence, we defined finer bin widths to get more detailed trends information, and regression analysis methods were introduced to explore the relation between QI-TS and metrics for the Final Run.
For the CON regression analysis, a bin width of 0.05 was chosen to divide QI-TS into 20 equal bins, with each resulting bin containing at less 2% of the data. Larger bins include more values due to more concentrated distribution, higher and narrower waveforms while less or even no result for 0.2–0.4 bins. Figure 11 suggests that the linear regression curve fits well the two CON metrics—RB and CC (Figs. 11a,b). The higher QI-TS has lower bias and higher correlations and obeys a linear relationship. The intercept is higher with larger time scales indicating improved RB (less negative bias) and higher correlations. Note that the only positive RB point appears at QI-24h. The slopes of the regression line, y(x) = a + bx, are stable for the different time scale.
The Final Run linear regression CON results by QI-TS where bin width = 0.05 and available rate ≥ 2%.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
The CAT metrics, however, present a nonlinear relation to QI-TS for all metrics except FAR. To get more valid sampling spots, two looser sample criteria were set for QI-TS: bin width = 0.025 and available ≥0.8%. After this parameter tuning, it is clear that FAR (Fig. 12b) fits the linear relationship very well. However, it is surprising that POD, HSS, and FBI-1 cannot be approximated with a linear curve. For POD (Fig. 12a), the highest skill happens between QI-TS of 0.4 and 0.6 rather than for higher bins around 0.7. FBI-1 (Fig. 12c) shows that the lower QI-TS are overestimated in the middle but underestimated at both sides. HSS (Fig. 12d) shows its best skill appears around 0.6. It is noted that since these three metrics change with different time scale, the regression curve can fit the 1- and 3-h time scales relatively better compared with the 6- and 24-h time scales, whose skill drops nearly at 0.7 of QI-TS, but rapidly recovers with higher QI-TS. Meanwhile, they still present the best performance in the middle level of QI-TS.
The Final Run CAT results by QI-TS, where bin width = 0.025 and available rate ≥ 0.8%.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
In conclusion, RB, CC, and FAR show a linear relationship with QI-TS, with RB and CC increasing with QI-TS while FAR decreasing with QI-TS. However, the higher QI-TS does not always indicate the best skills for HSS, POD, and FBI-1. Instead, the best skill happens at the middle level of QI-TS, between 0.4 and 0.6. It should be noted that the POD, FAR, and FBI-1 metrics will reach a plateau and even slightly downward trend with increasing QI-TS.
These results also provide guidance on whether the original QI-HHR performance can be maintained with larger time scales. The answer varies by metrics. IMERG QI-HHR linear trends can be well maintained to reflect QI-TS trends in all time scales for RB, CC, and FAR. But for POD, FBI-1, and HHS, QI-1h and QI-3h show top quality for the middle bins, while QI-6h and QI-24h almost follow a step function.
d. Spatial characteristics
To gain further insight into the performance of IMERG over land or water, the 19 GV stations were subdivided into two groups: 12 stations were categorized as coastal (near shore) while the remaining 7 stations were located inland (Table 2).
The spatial distributions of 6-h Early run CC for different groups in the Great Lakes are shown in Fig. 13a. The CC for coastal stations is better than for inland stations (lower CC in the lighter grayscale). CC in Fig. 13c also presents that coastal CC is better than inland for both Early and Final Runs at all time scales. Final CC is better than Early and the improvement of Final over early is more significant at longer time scales, because the Early Run is more impacted by lake surface resulting in relatively larger differences between shore and inland areas.
Early Run CC and ETS results by location. (a),(b) Distribution with CC and ETS under 6 h, respectively. (c),(d) Group by inland and coastal (see Table 3) under four time scales.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
For 6-h ETS (Fig. 13b), coastal sites outperform inland (lower ETS in lighter grayscale), although the difference is not as large as the CC differences. Figure 13d indicates coastal ETS is higher than inland and final ETS is better than early at all time scales but there is no discernable time scale dependence.
In terms of the comparison from warm to cold season. ETS generally decreases from the warm (Fig. 14b) to the cold season (Fig. 14c), but the extent of decrease is different. The difference between both groups is larger in the cold season than in the warm season. The likely explanation is because more frequent lake effect events during the cold season lead to more sensitivity of IMERG to diagnose the precipitation occurrence for shore areas than for inland areas.
Seasonal ETS changes between coastal and inland groups: (a) warm season and (b) cold season.
Citation: Journal of Hydrometeorology 24, 6; 10.1175/JHM-D-22-0214.1
5. Discussion
Using gauges as ground-truth allows to assess IMERG spatiotemporal characteristics well in the subcontinental region of the Great Lakes. Our results are comparable to other studies using gridded precipitation products over continental regions, such as the Multi-Radar Multi-Sensor (MRMS) product over Canada by Moazami and Najafi (2021). This study’s Final Run hourly average RB of 11.42% is less than Moazami and Najafi’s RB of 20%, and in terms of CC, both show very similar results at a level of 0.45. Our lower RB is partly attributed to the 19 high quality GV measurements. Strict controls and adjustment for snow undercatch ensure a most accurate precipitation amount in all seasons. These results indicate the importance of high quality gauge data for IMERG ground validation within subcontinental areas.
Overall, the Final Run outperforms the Early Run as expected, but the specific differences of performance depend on the evaluation metric. On one hand, CAT indices show little discrepancy between both runs when IMERG is only applied to detect whether precipitation occurs or not. According to the 6-h detailed analysis, the Early Run capacity is competitive within all thresholds by ETS as well as smaller precipitation thresholds less than 5 mm/6 h by FBI-1 (Figs. 5c,d). However, the Final Run FBI-1 is superior to the Early Run for larger thresholds since the general values of the Early Run’s FBI-1 are considerably overestimated (Fig. 5). On the other hand, the CON analysis yields a closer look, using more exact precipitation amounts, compared to the CAT binary precipitation event detection. Considering the algorithms and data usage difference, there are two stepwise operations from Early to Final Runs. The first, B-MORPHING, is reflected in the Final Uncal results, and the second, adjustment for the GPCC monthly gridded product, results in the Final Cal product. Only nonzero values of precipitation estimates in the Early Run can be GPCC gauge adjusted. Thus, if a particular grid pixel changes from zero to nonzero this is due to B-MORPHING. Hence, the effect of B-MORPHING is minimal with regard to trace precipitation estimates since there is little improvement from the Early to Final Run according to CAT results. However, for heavier precipitation the precipitation features are sufficiently intense to be more accurately captured and interpreted by GPM’s IR instruments leading to more valuable B-MORPHING vector information from PMW sensors. As a result, the extent of differences between the Early and Final Run is not a simple relationship and depends on the meteorological context.
Since the Early Run has the exclusive advantage of minimum latency among the family of IMERG runs, more and more real-time and near-real-time precipitation analysis products, e.g., CaPA, have used or plan to introduce IMERG Early into their products and models. These results demonstrate that there are various areas for IMERG users to improve the Early Run, e.g., by assimilating more reliable gauges as a correction to control the bias of precipitation amount estimation. This can lead to a performance similar to the Final Run, especially for the light precipitation.
The Early and Final Run performances are impacted seasonally. The similar trends of both runs present the best performance in summer, followed by the transitional seasons of MAM and SON. The most challenging season remains winter and is the major issue in the IMERG products, which is consistent with findings of Asong et al. (2017). The poor performance in winter is an indication of the shortcomings in PMW algorithms over snowy or icy surfaces. However, in terms of the contributing parts of the algorithm, 1-h RB results presented for the Final Run (Fig. 3b) indicate that the B-MORPHING algorithm is effective while the GPCC data adjustment is limited during this period. Differences in PMW processing between early and final as also a possible contributing factor along with the fact that B_MORPHNG is used is Final but not Early. For example, it is possible that the standard version of the PMW algorithm used in Final (as opposed to the near-real-time version in Early) has reduced RB. Therefore, IMERG developers can partly solve winter issues via improved B-MORPHING.
It is important to understand PMW seasonal differences separately with regard to HQpre and HQsrc results. Figure 8 implies ATMS is the most well-suited sensor for winter observations, although its performance throughout the year is not outstanding. Increasing the weight of ATMS may improve the overall performance of IMERG. On the other hand, GMI can still be a benchmark for the whole year due to its best performance with CC of 0.371, RMSE of 3.966 mm h−1.
One priority that needs to be considered is the time scale selection for the application of IMERG products in order to achieve acceptable performance while conserving the excellent temporal resolution of the IMERG half-hourly products. Results indicate IMERG’s performance improves with increasing time scales (Figs. 3 and 4) except RB. For RB, the longer accumulation involves more averaging, which “evens out” RB in the half-hour estimates, so availability of PMW data at shorter time scales may also contribute to the results. This is because a longer accumulation time potentially includes additional PMW swaths into the MORPHING calculation. It is worth noting that the POD gain from 1 to 3 h is the largest of all the time scale comparisons. At least one PMW is able to have a direct observation within 3 h, while 1 h may be too short to include in at least a single swath of any PWM. IMERG users should keep this in mind when prioritizing either temporal resolution or accuracy of the estimated precipitation rates.
QI has been added as an important variable of IMERG, but its quantitative applicability needs further clarification. QI temporal upscaling was derived by the parameter QI-TS. Results indicate that there is a linear relationship between QI-TS and RB and CC skill scores at all temporal resolutions (Fig. 11). A QI greater than 0.4 proves useful, otherwise is of limited value because QI below 0.4 is mainly from IR observation with high uncertainty. For the other metrics, FAR shows an inverse linear relationship with QI-TS (Fig. 12). However, the other variables, FBI-1, POD and HSS, display more of a bifurcation where above a QI of 0.4 there is a larger increase in performance, but not much further improvement for higher QI values. The regression linear behaviors between CC, RB, and FAR with QI-TR can help users predict the performance. For example, if an IMERG grid point exhibits a QI-24h value of close to 1, the respective CC may reach 0.8.
IMERG presents spatial differences between coastal and inland. The improved scores of ETS and CC for coastal stations compared to inland stations is an indication of the improvement in PMW measurements with open water as the underlying surface. This effect is visible in all time scales.
Some limitations of this study are the relatively small number of GV gauges, due to the current ECCC network design trading off quantity for high-quality.
6. Conclusions
In this study, multiple variables of the Early and Final Run from the half-hourly IMERG v06 satellite products were evaluated in the Great Lakes region at four submonthly time scales using independent precipitation stations as reference. This assessment provides valuable insights for IMERG precipitation product users and helpful guidance on the development of the satellite precipitation retrieval algorithm. Based on the analysis presented in this study, the following main conclusions can be drawn:
-
The precipitation retrieval performs better when data are accumulated over several hours. The most remarkable gain is for the POD from 1 to 3 h. The Final Run outperforms the Early Run due to the inclusion of the GPCC calibration data and B-MORPHING, especially in the winter. The monthly GPCC can reduce RB for events related to lake effects while B-MORPHING is more effective when there is less influence from lake effects. Threshold selection functions were applied to define a basis of comparison for precipitation event detection under different accumulation times. The two runs show that performance in the summer is the best, followed by spring and fall. Winter exhibits by far the poorest performance.
-
For the 6-h time scales, although summer is the only season with consistent positive FBI-1, the FBI-1 stays under 0.5 for the heavy rain [>10 mm (6 h)−1]. The ETS in the summer period is better than for the spring and autumn. In winter, the most challenging season overall, IMERG severely overestimates heavy precipitation events. Both runs overestimate heavy precipitation amounts while underestimating light precipitation.
-
A MWS method was applied to better match the hourly GV to the half-hourly PMW data. GMI is the most conservative with the lowest overall FAR while SSMIS is assertive leading to high POD and FAR. ATMS is the most consistent over different seasons and is the best sensor to capture snow in winter and spring. AMSR2 and GMI are impacted the most out of all sensors by winter conditions, and GMI has a strong negative bias throughout the year. However, in terms of CON analysis without MWS, GMI shows the best performance with the highest CC of 0.48. The second best CC is reached by RMSE at 2.727 mm h−1 while AMSR2 performs the poorest (CC of 0.371, RMSE of 3.966 mm h−1).
-
Our QI assessment is the first study of the link between QI-TS and IMERG performance. With increasing QI-TS, the metrics of RB, CC, FAR show a linear relationship to QI-TS with higher QI-TS for better performance. However, the better performance of POD, HSS, FBI-1 are not always present in the high bins (>0.6) of QI-TS but tend to appear in the middle range bins (0.4–0.6).
-
Both runs show better performance for the nearshore areas compared to the inland areas. The ETS difference is even larger in the cold season. Most inland sites exhibit a negative bias for FBI-1 while shore sites perform better with the FBI-1 around the ideal value of 0 at time scales of 3, 6, and 24 h.
The Great Lakes area, with its combination of large water bodies and land surfaces and a variety of both warm and cold season meteorological conditions is an ideal testbed for IMERG algorithm development. Future study considerations include adding marine buoy observations to compare in-water, shore, and in-land locations. In addition, an assessment of a new version of IMERG in the Great Lakes Basin with a more comparable and high-quality dataset, such as the ground-radar gridded QPE product is favorable. The mixed precipitation is a challenge for the future.
Acknowledgments.
BZ would like to thank Chris Doyle for contacting Environment and Climate Change Canada (ECCC), Stella Melo and Daniel Michelson for the support during BZ’s working as a Visiting Scientist in ECCC. BZ would like to extend special thanks to Paul Joe for sharing work and life experiences during a weekly morning walk. The work was supported with funding from the China Meteorological Administration, the China Scholarship Council (201905330005), the Service Center for Experts and Scholars of Hebei Province (201756), the S&T Program of Hebei (21567624H), and ECCC.
Data availability statement.
The Integrated Multi-satellitE Retrievals for GPM (IMERG) Early and Final Run data used in this study are available under https://dx.doi.org/10.5067/GPM/IMERG/3B-HH-E/06 and https://dx.doi.org/10.5067/GPM/IMERG/3B-HH/06, respectively. These measurements of Environment and Climate Change Canada (ECCC) Climate Network stations are available from ECCC’s National Climate Data and Information Archive Historical Climate Data portal at various temporal scales ranging from hourly to annual (https://climate.weather.gc.ca/, ECCC 2021c).
APPENDIX A
Definition of Metrics
Continuous verification (CON) statistics can measure the accuracy of a continuous variable such as precipitation amount between estimated and observed data. Relative bias (RB) is used to evaluate the average bias of Sat, normalized by the scale of GV (Wilks 2011). When above 0 it means Sat overestimates GV while less 0 means underestimation. There are also numerous studies suggesting the importance of using correlation coefficient (CC) and root-mean-square error (RMSE) (O et al. 2017; Asong et al. 2017; Wang et al. 2017). The former metric indicates the extent of correlation between Sat and GV while the latter is used to assess the overall performance (Table A1).
Continuous metrics for assessing Sat. The overbarred Sat and GV represent the Sat mean and GV mean, respectively.
Categorical metrics (CAT) based on dichotomous variables are usually applied to evaluate the performance of weather forecast, but they can also be applied to Sat verification (Jolliffe and Stephenson 2012). A certain precipitation threshold (T) is selected first, then the number of occurrences of the Sat and GV precipitation over or under this threshold is counted for the four different possibilities (Table A2).
The categorical event for assessing Sat. HT = hits, MS = misses, FA = false alarms, CN = correct negatives.
Five CAT metrics are employed in this study, probability of detection (POD), and false alarm ratio (FAR) are sensitive for hits and misses, respectively. Equitable threat score (ETS) and Heidke skill score (HSS) are commonly used as an overall skill measure of numerical weather prediction (NWP) and Sat (Rossa et al. 2008; Ebert et al. 2007) and its “equitability” allows scores to be compared more fairly across different regions. ETS has the perfect value of 1 if there are no misses or false alarms; A positive value of ETS indicates is skillful while a negative value shows a product that has no more skill than climatology (Fortin et al. 2018). A frequency bias index (FBI) of 1 typically indicates no bias. However, in this study, we use FBI-1 to modify the perfect value from 1 to 0 in order to meet the condition that zero represents unbiased (Lespinas et al. 2015). Thus, a positive value of FBI-1 indicates a positive bias, and a negative value indicates a negative bias (Table A3).
APPENDIX B
REFERENCES
Adler, R. F., and Coauthors, 2003: The version-2 Global Precipitation Climatology Project (GPCP) monthly precipitation analysis (1979–present). J. Hydrometeor., 4, 1147–1167, https://doi.org/10.1175/1525-7541(2003)004<1147:TVGPCP>2.0.CO;2.
Allen, M., and W. Ingram, 2002: Constraints on future changes in climate and the hydrologic cycle. Nature, 419, 224–232, https://doi.org/10.1038/nature01092.
Asong, Z. E., S. Razavi, H. S. Wheater, and J. S. Wong, 2017: Evaluation of Integrated Multisatellite Retrievals for GPM (IMERG) over southern Canada against ground precipitation observations: A preliminary assessment. J. Hydrometeor., 18, 1033–1050, https://doi.org/10.1175/JHM-D-16-0187.1.
American Meteorological Society, 2021: Lake effect. Glossary of Meteorology, https://glossary.ametsoc.org/wiki/Lake_effect.
Baijnath-Rodino, J. A., and C. R. Duguay, 2018: Historical spatiotemporal trends in snowfall extremes over the Canadian domain of the Great Lakes Basin. Adv. Meteor., 2018, 5404123, https://doi.org/10.1155/2018/5404123.
Battaglia, A., and Coauthors, 2020: Spaceborne cloud and precipitation radars: Status, challenges, and ways forward. Rev. Geophys., 58, e2019RG000686, https://doi.org/10.1029/2019RG000686.
Beck, H. E., and Coauthors, 2019: Daily evaluation of 26 precipitation datasets using Stage-IV gauge-radar data for the CONUS. Hydrol. Earth Syst. Sci., 23, 207–224, https://doi.org/10.5194/hess-23-207-2019.
Bellerby, T., M. Todd, D. Kniveton, and C. Kidd, 2000: Rainfall estimation from a combination of TRMM precipitation radar and GOES multispectral satellite imagery through the use of an artificial neural network. J. Appl. Meteor. Climatol., 39, 2115–2128, https://doi.org/10.1175/1520-0450(2001)040<2115:REFACO>2.0.CO;2.
Boluwade, A., T. Stadnyk, V. Fortin, and G. Roy, 2017: Assimilation of precipitation estimates from the integrated multisatellite retrievals for GPM (IMERG, early run) in the Canadian Precipitation Analysis (CaPA). J. Hydrol. Reg. Stud., 14, 10–22, https://doi.org/10.1016/j.ejrh.2017.10.005.
Bush, E., and D. S. Lemmen, Eds., 2019: Canada’s Changing Climate Report. Government of Canada, 444 pp., https://changingclimate.ca/CCCR2019.
Da Silva, N. A., B. G. M. Webber, A. J. Matthews, M. M. Feist, T. H. M. Stein, C. E. Holloway, and M. F. A. B. Abdullah, 2021: Validation of GPM IMERG extreme precipitation in the Maritime Continent by station and radar data. Earth Space Sci., 8, e2021EA001738, https://doi.org/10.1029/2021EA001738.
Derin, Y., P. Kirstetter, N. Brauer, J. J. Gourley, and J. Wang, 2022: Evaluation of IMERG satellite precipitation over the land–coast–ocean continuum. Part II: Quantification. J. Hydrometeor., 23, 1297–1314, https://doi.org/10.1175/JHM-D-21-0234.1.
Derksen, C., D. Burgess, C. Duguay, S. Howell, L. Mudryk, S. Smith, C. Thackeray, and M. Kirchmeier-Young, 2019: Changes in snow, ice, and permafrost across Canada. Canada’s Changing Climate Report, E. Bush and D. S. Lemmen, Eds., Government of Canada, 194–259, https://doi.org/10.4095/308279.
Dezfuli, A. K., C. M. Ichoku, G. J. Huffman, K. I. Mohr, J. S. Selker, N. van de Giesen, R. Hochreutener, and F. O. Annor, 2017: Validation of IMERG precipitation in Africa. J. Hydrometeor., 18, 2817–2825, https://doi.org/10.1175/JHM-D-17-0139.1.
Ebert, E. E., J. E. Janowiak, and C. Kidd, 2007: Comparison of near-real-time precipitation estimates from satellite observations and numerical models. Bull. Amer. Meteor. Soc., 88, 47–64, https://doi.org/10.1175/BAMS-88-1-47.
ECCC, 2021a: MANOBS manual of surface weather observation standards. 8th ed., amendment 1, accessed 5 October 2021, https://publications.gc.ca/site/eng/9.904265/publication.html.
ECCC, 2021b: Canadian climate normal: 1981–2010 climate normals and averages. Accessed 5 October 2021, https://climate.weather.gc.ca/climate_normals/index_e.html.
ECCC, 2021c: Historical climate data. Accessed 5 October 2021, https://climate.weather.gc.ca/index_e.html.
ELPC, 2019: An assessment of the impacts of climate change on the Great Lakes. Accessed 5 October 2021, https://elpc.org/resources/the-impacts-of-climate-change-on-the-great-lakes.
EPA, 2021: Facts and figures about the Great Lakes. Accessed 5 October 2021, https://www.epa.gov/greatlakes/facts-and-figures-about-great-lakes.
Fortin, V., G. Roy, N. Donaldson, and A. Mahidjiba, 2015: Assimilation of radar quantitative precipitation estimations in the Canadian Precipitation Analysis (CaPA). J. Hydrol., 531, 296–307, https://doi.org/10.1016/j.jhydrol.2015.08.003.
Fortin, V., G. Roy, T. Stadnyk, K. Koenig, N. Gasset, and A. Mahidjiba, 2018: Ten years of science based on the Canadian precipitation analysis: A CaPA system overview and literature review. Atmos.–Ocean, 56, 178–196, https://doi.org/10.1080/07055900.2018.1474728
Gadelha, A. N., and Coauthors, 2019: Grid box-level evaluation of IMERG over Brazil at various space and time scales. Atmos. Res., 218, 231–244, https://doi.org/10.1016/j.atmosres.2018.12.001.
Grecu, M., W. S. Olson, S. Joseph Munchak, S. Ringerud, L. Liao, Z. Haddad, B. L. Kelley, and S. F. McLaughlin, 2016: The GPM combined algorithm. J. Atmos. Oceanic Technol., 33, 2225–2245, https://doi.org/10.1175/JTECH-D-16-0019.1.
Hou, A. Y., and Coauthors, 2014: The Global Precipitation Measurement mission. Bull. Amer. Meteor. Soc., 95, 701–722, https://doi.org/10.1175/BAMS-D-13-00164.1.
Huffman, G. J., 2019: IMERG V06 Quality Index. Accessed 9 November 2019, https://arthurhou.pps.eosdis.nasa.gov/Documents/IMERGV06_QI.pdf.
Huffman, G. J., R. F. Adler, M. M. Morrissey, D. T. Bolvin, S. Curtis, R. Joyce, B. McGavock, and J. Susskind, 2001: Global precipitation at one-degree daily resolution from multisatellite observations. J. Hydrometeor., 2, 36–50, https://doi.org/10.1175/1525-7541(2001)002<0036:GPAODD>2.0.CO;2.
Huffman, G. J., and Coauthors, 2012: Developing the integrated multi-satellite retrievals for GPM (IMERG). Geophysical Research Abstracts, Vol. 14, Abstract 6921, https://meetingorganizer.copernicus.org/EGU2012/EGU2012-6921.pdf.
Huffman, G. J., Stocker E. F., Bolvin D. T., Nelkin E. J., Tan J., 2019a: GPM IMERG final precipitation L3 half hourly 0.1 degree × 0.1 degree V06. Goddard Earth Sciences Data and Information Services Center (GES DISC), accessed 9 January 2020, https://doi.org/10.5067/GPM/IMERG/3B-HH/06.
Huffman, G. J., and Coauthors, 2019b: Algorithm Theoretical Basis Document (ATBD) version 06/NASA Global Precipitation Measurement (GPM) Integrated Multi-satellitE Retrievals for GPM (IMERG). National Aeronautics and Space Administration, accessed 13 March 2019, https://arthurhou.pps.eosdis.nasa.gov/Documents/IMERG_ATBD_V06.pdf.
Huffman, G. J., and Coauthors, 2020: Integrated Multi-satellite Retrievals for the Global Precipitation Measurement (GPM) Mission (IMERG). Satellite Precipitation Measurement, V. Levizzani et al., Eds., Advances in Global Change Research, Vol. 67, Springer, 343–353, https://doi.org/10.1007/978-3-030-24568-9_19.
Jolliffe, I. T., and D. B. Stephenson, Eds., 2012: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. John Wiley & Sons, 296 pp.
Joyce, R. J., and P. Xie, 2011: Kalman filter-based CMORPH. J. Hydrometeor., 12, 1547–1563, https://doi.org/10.1175/JHM-D-11-022.1.
Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487–503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.
Kunkel, K. E., L. Ensor, M. Palecki, D. Easterling, D. Robinson, K. G. Hubbard, and K. Redmond, 2009: A new look at lake-effect snowfall trends in the Laurentian Great Lakes using a temporally homogeneous data set. J. Great Lakes Res., 35, 23–29, https://doi.org/10.1016/j.jglr.2008.11.003.
Lespinas, F., V. Fortin, G. Roy, P. Rasmussen, and T. Stadnyk, 2015: Performance evaluation of the Canadian Precipitation Analysis (CaPA). J. Hydrometeor., 16, 2045–2064, https://doi.org/10.1175/JHM-D-14-0191.1.
Mailhot, E., B. Music, D. F. Nadeau, A. Frigon, and R. Turcotte, 2019: Assessment of the Laurentian Great Lakes’ hydrological conditions in a changing climate. Climatic Change, 157, 243–259, https://doi.org/10.1007/s10584-019-02530-6.
Mekis, E., N. Donaldson, J. Reid, A. Zucconi, J. Hoover, Q. Li, R. Nitu, and S. Melo, 2018: An overview of surface-based precipitation observations at environment and climate change Canada. Atmos.–Ocean, 56, 71–95, https://doi.org/10.1080/07055900.2018.1433627.
Moazami, S., and M. R. Najafi, 2021: A comprehensive evaluation of GPM-IMERG V06 and MRMS with hourly ground-based precipitation observations across Canada. J. Hydrol., 594, 125929, https://doi.org/10.1016/j.jhydrol.2020.125929.
Mohammed, S. A., M. A. Hamouda, M. T. Mahmoud, and M. M. Mohamed, 2020: Performance of GPM-IMERG precipitation products under diverse topographical features and multiple-intensity rainfall in an arid region. Hydrol. Earth Syst. Sci. Discuss., https://doi.org/10.5194/hess-2019-547.
Morrissey, M. L., J. A. Maliekal, J. S. Greene, and J. Wang, 1995: The uncertainty of simple spatial averages using rain gauge networks. Water Resour. Res., 31, 2011–2017, https://doi.org/10.1029/95WR01232.
Navarro, A., E. García-Ortega, A. Merino, J. L. Sánchez, C. Kummerow, and F. J. Tapiador, 2019: Assessment of IMERG precipitation estimates over Europe. Remote Sens., 11, 2470, https://doi.org/10.3390/rs11212470.
Nitu, R., and Coauthors, 2019: WMO Solid Precipitation Intercomparison Experiment (SPICE) (2012–2015). WMO IOM Rep. 131, 1445 pp., https://library.wmo.int/index.php?lvl=notice_display&id=20742.
Notaro, M., A. Zarrin, S. Vavrus, and V. Bennington, 2013: Simulation of heavy lake-effect snowstorms across the Great Lakes basin by RegCM4: Synoptic climatology and variability. Mon. Wea. Rev., 141, 1990–2014, https://doi.org/10.1175/MWR-D-11-00369.1.
O. S., U. Foelsche, G. Kirchengast, J. Fuchsberger, J. Tan, and W. A. Petersen, 2017: Evaluation of GPM IMERG early, late, and final rainfall estimates using WegenerNet gauge data in southeastern Austria. Hydrol. Earth Syst. Sci., 21, 6559–6572, https://doi.org/10.5194/hess-21-6559-2017.
OTT Hydromet, 2019: Operating instructions precipitation gauge OTT Pluvio2 L. 60 pp., https://www.ott.com/download/operating-instructions-precipitation-gauge-ott-pluvio2-l-1.
Pradhan, R. K., and Coauthors, 2022: Review of GPM IMERG performance: A global perspective. Remote Sens. Environ., 268, 112754, https://doi.org/10.1016/j.rse.2021.112754.
Rossa, A., P. Nurmi, and E. Ebert, 2008: Overview of methods for the verification of quantitative precipitation forecasts. Precipitation: Advances in Measurement, Estimation and Prediction, Springer, 419–452, https://doi.org/10.1007/978-3-540-77655-0_16.
Scott, R. W., and F. A. Huff, 1996: Impacts of the Great Lakes on regional climate conditions. J. Great Lakes Res., 22, 845–863, https://doi.org/10.1016/S0380-1330(96)71006-7.
Skofronick-Jackson, G., and Coauthors, 2017: The Global Precipitation Measurement (GPM) mission for science and society. Bull. Amer. Meteor. Soc., 98, 1679–1695, https://doi.org/10.1175/BAMS-D-15-00306.1.
Sui, X., Z. Li, Z. Ma, J. Xu, S. Zhu, and H. Liu, 2020: Ground validation and error sources identification for GPM IMERG product over the southeast coastal regions of China. Remote Sens., 12, 4154, https://doi.org/10.3390/rs12244154.
Tan, J., W. A. Petersen, and A. Tokay, 2016: A novel approach to identify sources of errors in IMERG for GPM ground validation. J. Hydrometeor., 17, 2477–2491, https://doi.org/10.1175/JHM-D-16-0079.1.
Tan, J., W. A. Petersen, P. E. Kirstetter, and Y. Tian, 2017: Performance of IMERG as a function of spatiotemporal scale. J. Hydrometeor., 18, 307–319, https://doi.org/10.1175/JHM-D-16-0174.1.
Tan, J., G. J. Huffman, D. T. Bolvin, and E. J. Nelkin, 2019: IMERG V06: Changes to the morphing algorithm. J. Atmos. Oceanic Technol., 36, 2471–2482, https://doi.org/10.1175/JTECH-D-19-0114.1.
Tan, M. L., A. L. Ibrahim, Z. Duan, A. P. Cracknell, and V. Chaplot, 2015: Evaluation of six high-resolution satellite and ground-based precipitation products over Malaysia. Remote Sens., 7, 1504–1528, https://doi.org/10.3390/rs70201504.
Tang, G., M. P. Clark, S. M. Papalexiou, Z. Ma, and Y. Hong, 2020: Have satellite precipitation products improved over last two decades? A comprehensive comparison of GPM IMERG with nine satellite and reanalysis datasets. Remote Sens. Environ., 240, 111697, https://doi.org/10.1016/j.rse.2020.111697.
Villarini, G., and W. F. Krajewski, 2008: Empirically-based modeling of spatial sampling uncertainties associated with rainfall measurements by rain gauges. Adv. Water Resour., 31, 1015–1023, https://doi.org/10.1016/j.advwatres.2008.04.007.
Wang, Z., R. Zhong, C. Lai, and J. Chen, 2017: Evaluation of the GPM IMERG satellite-based precipitation products and the hydrological utility. Atmos. Res., 196, 151–163, https://doi.org/10.1016/j.atmosres.2017.06.020.
Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. International Geophysics Series, Vol. 100, Academic Press, 676 pp.
