IMERG V06: Changes to the Morphing Algorithm
1. Introduction
The Global Precipitation Measurement (GPM) mission is a joint satellite mission led by NASA and JAXA with contributions from U.S. and international partners “to unify and advance precipitation measurements from space for scientific research and societal applications” (Hou et al. 2014). GPM mission efforts include instrument calibration, algorithm development, data production, ground validation, and science and societal applications. Central to these efforts is the GPM Core Observatory, a satellite launched in February 2014 with advanced spaceborne active and passive sensors for accurate retrievals of rainfall and snowfall. Already, GPM products are used in a diverse range of applications, including aiding hurricane predictions, advancing numerical weather and climate models, studying the monsoons, assisting with water resource management, modeling extreme events such as floods and landslides, contributing to global public health through disease tracking, and even determining insurance baselines and payouts (Kirschbaum et al. 2017; Skofronick-Jackson et al. 2017, 2018).
Vital to many of these applications is the Integrated Multi-Satellite Retrievals for GPM (IMERG), a gridded GPM precipitation product developed by the U.S. Science Team that combines observations from multiple spaceborne sensors to provide the best precipitation estimates at 0.1° every half hour globally (Huffman et al. 2019a; Precipitation Processing System 2019a). The essence of IMERG is to use the sensors on board the GPM Core Observatory as the reference standard to calibrate the partner satellite sensors in the GPM constellation, with a single retrieval framework to estimate the observed precipitation, and then merge them into a grid with advanced interpolation techniques to provide global coverage. Scientific research benefits from a record of sufficient length, while societal applications require near-real-time availability. As such, IMERG spans a record of about 20 years starting with the launch of the Tropical Rainfall Measuring Mission (TRMM) satellite, and has multiple “runs” with different latencies to cater to various applications. Evaluations over different parts of the world find that IMERG is generally outperforms its predecessor, the TRMM Multi-Satellite Precipitation Analysis, with improvements to its representation of intense precipitation and variability (e.g., Guo et al. 2016; Tang et al. 2016; Tan et al. 2017; Dezfuli et al. 2017; He et al. 2017; Kim et al. 2017; Manz et al. 2017; Satgé et al. 2017; Tan and Duan 2017; Wang et al. 2017; Prakash et al. 2018; Gebregiorgis et al. 2018; Jiang et al. 2018; Palomino-Ángel et al. 2019).
IMERG V06 introduces several major changes. Among these are improvements to parent GPM products, modifications to the satellite intercalibrations, the inclusion of additional sensors (particularly in the TRMM era), and refinements to the Kalman filter process (Huffman et al. 2019a,b). This paper focuses on changes to the morphing component in the IMERG algorithm. In short, IMERG V06 derives motion vectors from total precipitable water vapor (or total column water vapor) in numerical models to propagate the precipitation pixels, addressing limitations in the previous infrared (IR)-based approach, as well as extending the scheme to the poles. The following sections will document the development of this new morphing scheme, starting with an overview of the entire IMERG algorithm and of the morphing algorithm in V05 (section 2). This establishes the foundation on which we introduce the changes in V06 and provides our motivations for them (section 3), together with evaluations of the associated changes (section 4). We will also describe some alternatives that were not chosen (section 5) and briefly examine the different IMERG runs (section 6). While this change to the motion vector source is generally beneficial, we will identify several issues that need to be analyzed further and provide the outlook for the morphing scheme in IMERG V07 (section 7).
2. Algorithm
a. An overview of IMERG
IMERG is the gridded precipitation product from the GPM mission that unifies observations from a network of partner satellites in the GPM constellation (Huffman et al. 2019a). In IMERG V06, precipitation estimates are provided at 0.1° grids every half hour globally (within 60°N–60°S in previous versions). IMERG has three runs—Early, Late, and Final—to accommodate different user requirements for latency and accuracy. The Early run, available at a 4-h latency, is suitable for real-time applications such as in the prediction of flash floods. The Late run, with a 12-h latency, can be used for purposes such as water resource management. The Final run is at a 3.5-month latency and is intended for research applications.
IMERG uses precipitation estimates derived from passive microwave (PMW) sensors. The PMW precipitation estimates are derived primarily from the Goddard profiling algorithm (GPROF) (Kummerow et al. 2001, 2011, 2015; Precipitation Processing System 2017), a fully parametric retrieval algorithm that estimates the surface precipitation rate from the PMW brightness temperature using a Bayesian approach conditioned upon surface classes, surface temperature, and total precipitable water. This applies to all PMW sensors except Sondeur Atmosphérique du Profil d’Humidité Intertropicale par Radiométrie (SAPHIR), which uses the Precipitation Retrieval and Profiling Scheme (Kidd 2019; Precipitation Processing System 2019b). The PMW estimates are gridded to 0.1° every half hour. If there are more than two estimates in a grid box, only one will be selected according to the following priorities: estimates from conical-scanning sensors are chosen over estimates from cross-track scanning sensors, followed by the estimate with the observation time closest to the center of the half hour. The gridded PMW estimates are then intercalibrated. IMERG uses the Ku-band Combined Radar and Radiometer (CORRA) product (Precipitation Processing System 2018) as the reference standard for calibration, with seasonal adjustments to the Global Precipitation Climatology Project V2.3 product (Adler et al. 2003, 2016) to control for known deficiencies at certain latitudes for land and ocean. Ideally, all sensors would be directly calibrated to CORRA, but coincident sampling is too sparse for partner sensors due to orbit characteristics and the narrow swath of the Ku-band CORRA estimates. Instead, the intercalibration is performed in two steps. First, mean values of gridded partner sensors are adjusted to the GPM Microwave Imager (GMI) and the TRMM Microwave Imager (TMI) for the GPM and TRMM eras, respectively, using static seasonal calibrations based on one year of coincident estimates. Second, GMI–TMI is calibrated to CORRA using dynamic 45-day calibrations updated every 5 days, and this calibration is also applied to the GMI–TMI-calibrated partner sensors.
Quasi-Lagrangian interpolation is applied to the gridded PMW estimates to fill in gaps in the field using motion vectors computed from ancillary data. This process is called morphing and was first introduced as a central component of the CPC morphing technique (CMORPH) (Joyce et al. 2004; Climate Prediction Center 2011). Changes to this morphing component in IMERG are the focus of this study and will be elaborated in sections 2b and 3. The morphed precipitation is further supplemented, via a Kalman filter approach following Joyce and Xie (2011), with microwave-calibrated precipitation estimates calculated from IR brightness temperatures using the Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks—Cloud Cluster System (PERSIANN-CCS) algorithm (Hong et al. 2004; Nguyen et al. 2018). Traditionally, PMW retrievals suffer from inaccuracies over frozen surfaces, so IMERG masks out PMW and morphed estimates over snow- and ice-covered surfaces as indicated in the NOAA AutoSnow product (Romanov et al. 2000). Within 60°N–60°S, this results in the use of IR-based estimates. At high latitudes, this results in missing values. The output comprises the “best” satellite estimates from IMERG, which in the Final run is then calibrated with gauge analyses from the Global Precipitation Climatology Centre Full and Monitoring products (Schneider et al. 2014, 2015) following the approach employed by Huffman et al. (2007) for the TRMM Multi-Satellite Precipitation Analysis. For the Early and Late runs, due to the need for timely data delivery, a more limited selection of data is employed [see Huffman et al. (2019a) for details].
b. The morphing component
A key component of the IMERG algorithm is the morphing technique. First developed at the NOAA Climate Prediction Center as the central part of the CMORPH product (Joyce et al. 2004) and later improved with a Kalman filter following the Global Satellite Mapping of Precipitation (GSMaP; Ushio et al. 2009) to include IR precipitation (Joyce and Xie 2011), the morphing algorithm is integrated in the IMERG algorithm (Huffman et al. 2019a). The rapid adoption of this concept in leading algorithms of global precipitation is a testament to its ability to produce high temporal resolution in a gridded dataset from the limited sampling of PMW sensors.
The morphing algorithm as implemented in CMORPH and IMERG can be broken down into three subcomponents. First, motion vectors representing precipitation motions are derived. Second, these motion vectors are used to propagate precipitation over areas where there are no instantaneous PMW observations. Third, the propagated precipitation is integrated with IR precipitation through a Kalman filter. This study focuses on the improvements to the first two subcomponents.
To derive the motion vectors, two successive fields of an atmospheric variable with global coverage are compared. In CMORPH and IMERG prior to V06, the variable used is the half-hourly global geosynchronous IR brightness temperature from the CPC 4-km merged IR dataset (Janowiak et al. 2017). To compute the motion vector at a grid box, the IR pixels below 260 K in a template of 5° × 5° are compared to the IR pixels at the next half hour with different spatial offsets to the template. The correlation between the two fields of IR pixels is computed for each spatial offset, and the offset with the highest correlation forms the motion vector for that grid box. Motion vectors derived from this approach reflect the large-scale motions of cloud tops and is relatively independent of the actual values of the brightness temperatures. While motion vectors can be derived at every precipitation grid box (0.1° for IMERG), they are instead computed at every 2.5° because this procedure is computationally intensive. Any missing vectors, due to either data dropouts or an absence of cold clouds within the template, are filled in using temporal and spatial interpolation.
Given motion vectors, PMW precipitation pixels can be propagated both forward and backward in time. For the IR-based motion vectors, comparisons with ground-based precipitation radars in the United States indicate that the motion vectors overestimate the speeds of precipitation systems in both the longitudinal and latitudinal directions (Joyce et al. 2004). Therefore, climatological scaling factors are applied to the motion vectors prior to the propagation of precipitation. The precipitation pixel at 0.1° is propagated using the closest vector at the 2.5° vector grid. If multiple pixels are propagated to a single grid box, they are averaged. If a grid box is empty after the propagation, it is filled using spatial interpolation from neighboring grid boxes.
3. Changes to the morphing system
In CMORPH and IMERG up through V05, the motion vectors are computed from 4-km geosynchronous IR brightness temperatures; hence, the motion vectors reflect cloud-top motions. However, there are two main limitations in using geosynchronous IR to compute motion vectors. The first limitation is that cloud-top motions may not match precipitation motions, primarily due to wind shear and the potential difference in the evolution of precipitation and cloud systems. Indeed, this is the reason for the climatological scaling using U.S. ground-based radars (section 2b), though it is unclear how applicable these scaling factors are to other regions in the world. The second limitation is that the CPC IR dataset is limited to 60°N–60°S. Poleward of 60°N/S, the viewing angle is judged to be too oblique to produce information that is reliably useful. This limitation prevents a direct extension of morphing to high latitudes (defined here as poleward of 60°N/S).
Due to these drawbacks of geosynchronous IR data, IMERG V06 was modified to use numerical model variables to derive motion vectors. For the Final run, the Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2), product is used (Gelaro et al. 2017); for the Early and Late runs, the Goddard Earth Observing System Forward Processing (GEOS FP) forecast product (Lucchesi 2017) is used due to latency requirements. Both products are produced by the Global Modeling and Assimilation Office at NASA Goddard Space Flight Center using the GEOS model, which helps ensure consistency in the vectors between all runs.
MERRA-2 is based on version 5.12.4 of the GEOS model. MERRA-2 variables are available on a 0.5° latitude × 0.625° longitude grid with a temporal resolution of 1 h for most 2D fields. The MERRA-2 record begins in 1980, covering the entire TRMM and GPM eras. Its latency of about 1–2 months is compatible with the processing timeline of the Final run. GEOS FP is produced using the latest version of the GEOS model. GEOS FP variables are available on a 0.25° latitude × 0.3125° longitude grid with a temporal resolution of 1 h for most 2D fields. Being a forecast product, GEOS FP is run four times a day, each with a different forecast length: the 0000 UTC forecast is run for 240 h, the 0600 UTC forecast for 30 h, the 1200 UTC forecast for 120 h, and the 1800 UTC forecast for 30 h. Vectors are always computed using variables from two consecutive hours of the latest available forecast, but the extended forecast lengths mean that vectors can still be produced should there be any interruption to forecast data availability, albeit with slightly lower reliability (see section 6). The latency of GEOS FP is currently about 7 h, although hardware and software improvements may reduce this latency in the future.
In this study, we will evaluate the morphing scheme using vectors computed from six model-based variables: total (surface) precipitation from atmospheric model physics (PRECTOT), total precipitable water vapor (TQV; also known as total column water vapor), total precipitable liquid water (TQL), total precipitable ice water (TQI), specific humidity at 500 hPa (Q500), and specific humidity at 850 hPa (Q850). TQV, TQL, and TQI are the vertically integrated water content in the three respective phases. Due to the lognormal distribution of precipitation, we first transformed PRECTOT by log(P + 1), where P is the precipitation rate (mm h−1), to reduce the otherwise disproportionate weights given to high values in the correlation computation. Furthermore, to avoid anomalous vectors due to small-scale noise in the variables, minimum thresholds are applied to the model variables for the correlation computation. Based on preliminary studies, the thresholds adopted were: 0.03 mm h−1 for PRECTOT, 0.002 kg m−2 for TQL, and 0 for all others. To ensure sufficient sample sizes in the correlation computation, the coarse model data are linearly interpolated to the 0.1° precipitation grid. A vector will be computed only if there are at least 1000 grid boxes (out of a maximum of 2500 grid boxes in the 5° × 5° template) that are above the threshold for computing the correlation; otherwise, it is assigned as missing. (In the previous IR-based morphing scheme, a vector was computed so long as there was at least one grid box above the threshold.) Of the six variables, only TQV and Q500 consistently provide vectors at all locations; Q850 is globally complete except in areas of high surface altitude. Vector fields with missing values are filled in using the same temporal and spatial interpolation schemes developed for the IR-based vectors. Here, we only show results for MERRA-2 variables, since results for GEOS FP are similar. A brief discussion on the similarity between MERRA-2 and GEOS FP vectors is given in section 6.
The vector computation approach for the model variables is similar to that of IR: comparing large-scale templates of about 5° on a 2.5° grid. Different template sizes and grid spacing have been explored, but differences in performance were not sufficiently compelling to warrant a change for V06. However, as we will be extending the vector computation from 60°N/S to the poles, the distortion of the longitudinal size of the template at high latitudes is an issue. At high latitudes, the area of a 5° × 5° template is much smaller than a 5° × 5° template at the equator, leading to inconsistency in the scale the vectors represent. To mitigate this distortion, the template size is given a latitudinal size of 5° and a longitudinal size of 5°/cos(latitude). Furthermore, at the poles, the template “wraps” around to the other longitudinal side of the hemisphere. Note that these fixes are driven by the fact that IMERG’s native grid is a cylindrical equidistant (or equal angle) grid. Future versions of IMERG aim to migrate to an alternative grid in data processing while still providing the 0.1° cylindrical equidistant grid for end users.
One important criterion for the choice of vector source is its temporal resolution. Our experience suggests that a higher temporal resolution is more important than higher spatial resolution in computing the vectors, despite the absence of finescale spatial features in the model fields like those in the 4-km IR. The importance of temporal resolution limits the range of model products that are viable for our purpose. Other than MERRA-2, we have also explored using ERA5—which also has hourly resolution—from the European Centre for Medium-Range Weather Forecasts, but the vectors produced are highly similar to MERRA-2, even though ERA5 has an independent computational approach. This demonstrates that the vector computation method—which is based on the large-scale motion of the variable and not its exact values—is not sensitive to the precise dataset used.
The motion vectors computed at 2.5° every hour are interpolated to 0.1° every half hour for propagating the precipitation pixels. Temporally, this interpolation is necessary to produce half-hourly precipitation fields. Spatially, this interpolation enables smoother motions (Fig. 1). The climatological scaling factors derived for the IR-based vectors are no longer necessary, as the motion vectors better reflect precipitation motions. As with the vector computation, pixels propagated beyond 90° latitude will be wrapped around to the other longitudinal hemisphere. In practice, masking out PMW estimates due to frozen surfaces, which occurs after the Kalman filter, renders this issue moot at both poles for V06.
A snapshot demonstrating the spatial interpolation of vectors from the 2.5° grid to the 0.1° grid box, showing the computed vectors (magenta arrows), every fourth interpolated vector (gray arrows), and the propagated precipitation field in the background. Arrow lengths are relative and do not indicate the physical distance propagated.
Citation: Journal of Atmospheric and Oceanic Technology 36, 12; 10.1175/JTECH-D-19-0114.1
4. Evaluation of the new morphing scheme
a. Evaluation approach
One way to evaluate the morphing scheme is to propagate gridded PMW observations to the next half hour and compare it with available PMW observations at that half hour. This is possible because some sun-synchronous PMW orbits are separated by about 30 min—such that a grid box may have PMW overpasses in two successive half hours—as well as the non-sun-synchronous orbits of GMI and TMI. This allows us to contrast the performance of the new scheme using model data and spatially interpolated vectors against the old scheme, and also between different model variables, at all locations of the globe. However, due to intersensor differences and evolution of the precipitation systems, the propagated precipitation will never be identical to the instantaneous PMW observation. In other words, even if the propagation is perfect, this evaluation will not give perfect scores. At the same time, biases in the retrieval process may inflate the absolute values of the scores. Therefore, to provide a meaningful evaluation, we choose two benchmarks. The first benchmark is the previous morphing scheme as implemented in V05: the IR-based vectors with ground radar scaling. The second benchmark is no motion (i.e., a vector field with zeros everywhere). This evaluation approach will be used in sections 4b and 4c; in section 4d we will complement this with ground validation.
With the propagated precipitation as the “estimate” and the instantaneous PMW precipitation as the “reference,” we compute three metrics that characterize different aspects of performance. The Heidke skill score (HSS) is a generalized skill score evaluating the ability of the estimate to detect precipitation at a threshold—chosen here to be 0.2 mm h−1 following Tan et al. (2018)—compared to random chance. A negative HSS indicates a performance poorer than random chance, a positive HSS indicates a performance better than random chance, and a value of 1 indicates perfect performance. For reference-estimate pairs that are both at least 0.2 mm h−1 (i.e., hits), we further compute the Pearson correlation coefficient (or simply correlation) and the normalized root-mean-square error (NRMSE). The correlation assesses the random error in the hits, quantifying the degree of scatter in the relationship between the reference and the estimate. A value of 1 indicates perfect performance. NRMSE expresses the difference in values between the estimate and the reference, normalized against the average reference value; it contains information on both the systematic and random error. A value of 0 indicates perfect performance. See the appendix for the mathematical definitions of these three metrics. Tests using alternative thresholds of 0.1 and 0.5 mm h−1 did not have an effect on the conclusions (not shown). The evaluation presented here uses data for the year 2018. Due to the large sample sizes (~109), any numerical differences in the rounded values are statistically significant with a negligible p value.
b. Global average
Figure 2 shows the HSS, correlation, and NRMSE of all propagated precipitation within 60°N–60°S for the year 2018. (High-latitude estimates from the model-based morphing scheme are ignored for comparability to the IR-based morphing scheme). The most salient result is that the new scheme using MERRA-2 outperforms both the IR-based propagation as implemented in IMERG V05 and no motion (None) in all three metrics most of the time. Not surprisingly, the IR-based propagation is also better than None. Between the six MERRA-2 variables, all except TQI and Q850 have comparable performance, with slight variations across the three metrics. Q850 has marginally lower performance in correlation and NRMSE, while TQI has the lowest performance of all MERRA-2 variables and is comparable to IR. This relative performance persists when evaluating estimates that were propagated for two and three half hours (Fig. S1 in the online supplemental material) as well as when the evaluation is broken down into different seasons in the Northern Hemisphere (Fig. S2).
(top) HSS, (middle) correlation, and (bottom) NRMSE of the precipitation fields propagated for one half hour using vectors derived from different variables, validated against the instantaneous PMW precipitation field of the second half hour, over the 60°N–60°S band for the year 2018. The variables are IR brightness temperature, total (surface) precipitation from atmospheric model physics (PRECTOT), total precipitable water vapor (TQV), total precipitable liquid water (TQL), total precipitable ice water (TQI), specific humidity at 500 hPa (Q500), and at 850 hPa (Q850). “None” refers to a vector field of all zeros, i.e., no motion.
Citation: Journal of Atmospheric and Oceanic Technology 36, 12; 10.1175/JTECH-D-19-0114.1
Figure 2 also demonstrates an important facet of this evaluation approach. Visually, the use of motion vectors computed from the variables to propagate precipitation fields leads to a clear improvement in the animations of precipitation fields (Figs. S3–S10). Nevertheless, the differences in absolute scores between the vectors and “None” are relatively small. This small numerical difference is likely a consequence of the intersensor differences, which results in a large contribution from observational noise. The actual difference in morphing performance will only emerge from the noise by averaging over a sufficiently large sample of reference-estimate pairs. Indeed, the relative performance between different vectors is consistent across different seasonal subsamples (Fig. S2), suggesting that the results in Fig. 2 are robust. Therefore, even though the differences in the metrics are numerically small, they represent differing morphing capabilities.
Figure 3 shows the evaluation separated by ocean and land, respectively, within 60°N–60°S. The relative performances between the various vectors are broadly consistent over land and over ocean: MERRA-2 variables are better than IR, which is in turn better than None. However, some vectors perform better over one surface type than the other. For example, over ocean, Q850 is comparable to other MERRA-2 variables, but over land, it is less accurate. Similarly, over ocean, TQV, TQL, and PRECTOT have similar scores that are among the best, but over land, TQV falls behind. Comparing the absolute scores between ocean and land, HSS and correlations are better over ocean but NRMSEs are better over land. Likely explanations of this ocean–land contrast are (i) the greater variability in land surface emissivity leading to greater differences between retrievals from different sensors, (ii) the faster evolution of precipitation systems over land, and (iii) the higher precipitation rates over ocean than over land (Adler et al. 2017), which generally increase the absolute values of NRMSE.
As in Fig. 2, but separately for (left) ocean and (right) land.
Citation: Journal of Atmospheric and Oceanic Technology 36, 12; 10.1175/JTECH-D-19-0114.1
c. Zonal average
In section 4b, we evaluated the performance of the propagation without consideration of location (except by surface type). However, given that precipitation motions vary at different latitudes, examining the performance of the morphing scheme as a function of latitude may yield some useful insights.
Figure 4 shows the HSS, correlation, and NRMSE by each 2.5° latitude band, while Fig. 5 shows the anomaly about the precipitation field with no motion (None) at each latitude band to enhance the relative differences. Looking at all precipitation fields, there is decreased performance in the low latitudes, indicating higher variability between the sensors. Comparing between various propagated precipitation fields, the impact of morphing is clearest in the midlatitudes, where all variables (including IR) show a marked improvement over the precipitation field without propagation. This is due to the predominance of extratropical cyclones and fronts at these latitudes, which leads to fast and persistent motions in precipitation. In the tropics, morphing confers a smaller advantage due to the generally lower propagation speeds, such that a lack of morphing is not a severe impediment to predicting precipitation positions. At high latitudes, the advantages of morphing diminish, too, though the sharp decline in absolute scores—caused by the strong intersensor differences over frozen surfaces—limits any useful evaluation at these latitudes (Figs. S3–S10).
As in Fig. 2, but for each 2.5° latitude band.
Citation: Journal of Atmospheric and Oceanic Technology 36, 12; 10.1175/JTECH-D-19-0114.1
As in Fig. 4, but expressed as the anomaly against “None.”
Citation: Journal of Atmospheric and Oceanic Technology 36, 12; 10.1175/JTECH-D-19-0114.1
As with the globally averaged evaluation, the MERRA-2-based vectors generally provide improved performance compared to the V05 IR-based vectors and no motion (Figs. 4 and 5). None of the MERRA-2 variables consistently outperforms the others at all latitudes, though their relative performances broadly resemble the globally averaged metrics (Fig. 2). For example, TQL is among the best performing variable at most latitudes, but in the Southern Hemisphere low latitudes and parts of the Southern Ocean it is falls behind TQV. On the other hand, the performance of TQV is at the top at some latitude bands, but it suffers a drop in performance in the Northern Hemisphere low latitudes.
A surprising result in Figs. 4 and 5 is the poorer performance of IR compared to None in the tropics, consistently in HSS and NRMSE, and occasionally in correlation. That is, the IR-based morphing as implemented in IMERG V05 may actually worsen the performance of the tropical precipitation estimates. Similarly, TQI may lead to a performance less accurate than None. One potential explanation is the possible discrepancy in the motions between the cloud tops that the IR observes and the precipitation systems beneath, and likewise between the cloud ice particles represented by TQI aloft and the underlying precipitation systems. Another possible explanation lies in the more transient and smaller-scale nature of tropical convective clouds, which can impede the effective computation of motion vectors. Coupled with the generally slower motions in the tropics and thus a smaller advantage of morphing over no motion, typically small errors in motion are proportionately more important in the low-motion tropics.
d. Ground validation over CONUS
The evaluation up to this point relies solely on satellite data, with the propagated precipitation as the “estimate” and the instantaneous PMW observations as the “reference.” Alternatively, an approach complementary to this satellite-only evaluation is to use ground observations as the reference. While ground validation comes at the expense of a more limited coverage—in this case, over the conterminous United States (CONUS)—it does not suffer from the limitations of intersensor differences and lack of system evolution that is inherent in the previous evaluation (section 4a).
The ground reference used here is a Multi-Radar Multi-Sensor (MRMS) system product processed in support of GPM ground validation (Kirstetter et al. 2012, 2014, 2015). This product aggregates the MRMS precipitation rates to produce half-hourly accumulated precipitation rates over CONUS (20°–55°N, 130°–60°W) with a high spatial resolution of 0.01°, which we average to 0.1°. Each grid box is also associated with a radar quality index that reflects sampling and estimation uncertainty, which allows us to select only the most reliable grid boxes. Furthermore, MRMS is adjusted with the Hydrometeorological Automated Data System and regional rain gauge networks, which also allows us to exclude grid boxes with gauge correction factors outside the range of 0.5–2—indications of potential discrepancies in the surface data.
Figure 6 shows the HSS, correlations, and NRMSE of the comparisons over CONUS for estimates propagated for one half hour using the various motion vectors. The salient differences compared to Fig. 2 are (i) a better correlation by the IR-based vectors in V05, which can be attributed to the scaling factors derived from the selfsame ground radars (though with a different algorithm), (ii) the strikingly poorer performance by TQI, and (iii) the unexpectedly high HSS by None, for which we do not yet have a satisfactory explanation. On the other hand, TQV, TQL, and Q850 clearly outperform all others (except against IR in correlation).
(top) HSS, (middle) correlation, and (bottom) NMRSE of the precipitation fields propagated for one half hour using vectors derived from different variables, validated against MRMS over CONUS for the year 2018.
Citation: Journal of Atmospheric and Oceanic Technology 36, 12; 10.1175/JTECH-D-19-0114.1
5. Using large-scale winds to propagate precipitation
In the implementation of the new morphing scheme, one obvious candidate for the motion vectors is the large-scale horizontal winds from the models. To test this possibility, we converted MERRA-2 winds at 850 and 500 hPa (variables U850, V850, U500, V500) into vectors directly, instead of deriving them from the spatial offset of the template as described in section 3. The vectors are then ingested into the forward-propagation code.
It is clear from animations of the propagated precipitation that using winds as motion vectors is deficient (Figs. S11 and S12). The precipitation propagated by the winds at either level has an incorrect speed and/or direction, leading to jumps in positions when PMW overpasses occur. Over the South China Sea, the incorrect directions of propagation even led to the distortion of an organized precipitation cell.
The problem with using horizontal winds as vectors is well known from synoptic experience, namely that the “steering level” of the precipitation varies depending on the system. Precipitation originating from warm clouds are likely steered by low-level winds, while precipitation emerging from deep convection may be influenced by winds aloft. In fact, for winds to serve as vectors, they should be vertically integrated with weights based on the amount of precipitation at each level, accounting for fall speeds and evaporation. Not only does this introduce a complexity beyond what can reasonably be implemented or even approximated, the motion of this vertically integrated quantity should be well captured by TQL or TQV.
6. GEOS-FP and MERRA-2
While the changes herein affect all three IMERG runs, we thus far only showed the evaluation results for motion vectors derived from MERRA-2, which is used only in the Final run. We expect these results to be comparable to motion vectors derived from GEOS FP that is used in the Early and Late runs, given the structural similarity between the two products. Indeed, a test comparison of the motion vectors derived from GEOS FP TQV to those derived from MERRA-2 TQV showed high similarity between the two sets of vectors, with correlations between the vector fields consistently above 0.9 (Fig. 7). This result is based on using all four GEOS FP runs of the day over 10 days beginning 1 August 2017 (taking into account the anticipated 7-h latency in data availability). If we were to use a single 0000 UTC GEOS FP forecast that goes out to 240 h, the drop in performance associated with a deteriorating forecast is gradual, with correlations dropping to just below 0.7 at 240 h. This suggests that, should there be an interruption to GEOS FP data availability that forces the IMERG Early and Late runs to use a forecast to an extended period, they can still run for up to 10 days with a tolerable degradation in performance.
Correlation of the vector field at each half hour between MERRA-2 TQV and GEOS FP TQV, using the presumed most recent GEOS FP runs over 10 days (blue) and using a single run of GEOS FP out to 10 days (orange), both starting from 1 Aug 2017, over the 60°N–60°S band.
Citation: Journal of Atmospheric and Oceanic Technology 36, 12; 10.1175/JTECH-D-19-0114.1
7. Concluding remarks
This paper described the changes to the morphing component of the IMERG algorithm in V06. One of the biggest changes is the switch from geosynchronous IR to numerical model data from MERRA-2 and GEOS FP for the derivation of motion vectors used to propagate the PMW precipitation observations. This change is motivated by discrepancies between cloud-top motion and precipitation motion, and by the need to extend IMERG coverage to the poles. In addition, there are some minor changes in the implementation of morphing, including (i) the removal of scaling constants derived by comparing IR-based motion vectors with ground-based radars over CONUS; (ii) the introduction of spatial interpolation of the vectors from 2.5° to the precipitation grid for smoother motions; and (iii) the implementation of a correction to deal with gridbox distortions in longitude at high latitudes associated with the cylindrical equidistant grid.
Our evaluation—comparing the precipitation propagated for one half-hour time step against the instantaneous PMW observation at the second half hour—indicates that the new scheme is an improvement globally over the IR-based morphing scheme implemented in V05. Among the MERRA-2 variables tested, vectors derived from TQV, TQL, PRECTOT, Q500, and Q850 generally have better performance, with slight variations over land and over ocean. Zonally, these variables are comparable to each other, with no one variable consistently outperforming the others. These results are also confirmed by ground validation using radar- and gauge-based estimates over CONUS, with TQV, TQL, and Q850 gaining a distinct edge. For IMERG V06, we chose TQV as the variable for deriving motion vectors because of its competitive performance and global completeness, though this decision may be revisited in future versions as more evaluation results arrive.
One concern regarding this change in motion vector source to TQV is the possibly negative effect of orography. Being a vertically integrated quantity, TQV is reduced at higher elevations due to a shorter vertical column. How this affects the accuracy of the motion vectors is unclear, considering the indirect nature of the motion vector derivation and complex interactions between precipitation motion and orography. In fact, it is also unclear how the IR-based motion vectors are affected by orography. It is worth pointing out that IMERG, unlike GSMaP (Yamamoto and Shige 2015; Yamamoto et al. 2017), does not yet have a scheme to account for orographic enhancement of precipitation, making it harder to disentangle the possible orographic effect on the motion vectors while at the same time limiting the effectiveness of our evaluation approach (section 4a). We encourage users evaluating IMERG V06 to consider assessing the impact of orography on propagated precipitation, keeping in mind that its effect can be distant due to the large template size.
A second issue relating to the extension of IMERG to the poles is the use of a cylindrical equidistant grid, as mentioned in section 3. What was previously an acceptable distortion of the grid boxes in the longitude direction at the 60°N/S limits is now quite severe close to the poles. While corrections have been implemented in V06, their ad hoc nature may introduce artifacts into the propagated estimates at high latitudes. We are actively considering migrating to a more equitable grid for V07. This change will occur inside the processing system for IMERG; the public products will still provide interpolated precipitation estimates on a 0.1° cylindrical equidistant grids. We anticipate that a switch to a grid that accounts for longitude distortion will provide improved accuracy (especially at higher latitudes) and computational speed.
The original morphing component in IMERG was adapted from CMORPH, but developments on both implementations have since diverged. A reprocessed version of CMORPH beginning 1998 has been released (Xie et al. 2017), and the next-generation CMORPH is in active development. Unlike IMERG V06, the new CMORPH will adopt a different approach. It will use IR-based precipitation (instead of IR brightness temperature) from geosynchronous satellites to derive the motion vectors, supplemented by model precipitation and polar-orbiting IR sensors at higher latitudes.
Regardless of how morphing is implemented, it is fundamentally an interpolation technique, so it can only be as good as the PMW estimates that it interpolates from. Hence, frequent observations by high-quality satellite instruments, accurate retrieval algorithms, and reliable calibration data are primary factors in the ultimate performance of such multisatellite algorithms. Therefore, for IMERG to sustain its performance in the future, it is critical to maintain a constellation of high quality sensors with open sharing of data and continued efforts to refine the retrieval algorithms and calibrations.
Acknowledgments
We are grateful to personnel at Precipitation Processing Systems, especially Patty McCaughey and Erich Stocker, for their invaluable assistance both in reprocessing IMERG V06 and in providing the necessary intermediate data for the analysis in this study. We also acknowledge Robert Joyce and Pingping Xie at NOAA CPC for their contributions to the IMERG algorithm. We thank the three anonymous reviewers and the Editor Dr. Luca Baldini for their feedback to the manuscript. All authors are supported by the NASA Precipitation Measurement Missions funding (NNH18ZDA001N-PMMST).
APPENDIX
Definition of Metrics
The propagated precipitation is evaluated against the reference using three metrics: Heidke skill score (HSS), Pearson’s correlation coefficient (or simply correlation), and normalized root-mean-square error (NRMSE). HSS quantifies the precipitation detection while correlation and NRMSE quantifies the accuracy in precipitation intensity.
Correlation is computed using the standard definition of the Pearson correlation coefficient.
REFERENCES
Adler, R. F., and Coauthors, 2003: The version-2 Global Precipitation Climatology Project (GPCP) monthly precipitation analysis (1979–present). J. Hydrometeor., 4, 1147–1167, https://doi.org/10.1175/1525-7541(2003)004<1147:TVGPCP>2.0.CO;2.
Adler, R. F., and Coauthors, 2016: Global Precipitation Climatology Project (GPCP) climate data record (CDR), version 2.3 (monthly). NCEI, accessed 27 June 2019, https://doi.org/10.7289/v56971m6.
Adler, R. F., G. Gu, M. Sapiano, J.-J. Wang, and G. J. Huffman, 2017: Global precipitation: Means, variations and trends during the satellite era (1979–2014). Surv. Geophys., 38, 679–699, https://doi.org/10.1007/s10712-017-9416-4.
Climate Prediction Center, 2011: NOAA CPC morphing technique (CMORPH) global precipitation analyses. NCAR Computational and Information Systems Laboratory, accessed 27 June 2019, https://doi.org/10.5065/d6cz356w.
Dezfuli, A. K., C. M. Ichoku, G. J. Huffman, K. I. Mohr, J. S. Selker, N. van de Giesen, R. Hochreutener, and F. O. Annor, 2017: Validation of IMERG precipitation in Africa. J. Hydrometeor., 18, 2817–2825, https://doi.org/10.1175/JHM-D-17-0139.1.
Gebregiorgis, A. S., P.-E. Kirstetter, Y. E. Hong, J. J. Gourley, G. J. Huffman, W. A. Petersen, X. Xue, and M. R. Schwaller, 2018: To what extent is the day 1 GPM IMERG satellite precipitation estimate improved as compared to TRMM TMPA-RT?: Day-1 IMERG improves upon TMPA-RT. J. Geophys. Res. Atmos., 123, 1694–1707, https://doi.org/10.1002/2017JD027606.
Gelaro, R., and Coauthors, 2017: The Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2). J. Climate, 30, 5419–5454, https://doi.org/10.1175/JCLI-D-16-0758.1.
Guo, H., S. Chen, A. Bao, A. Behrangi, Y. Hong, F. Ndayisaba, J. Hu, and P. M. Stepanian, 2016: Early assessment of Integrated Multi-Satellite Retrievals for Global Precipitation Measurement over China. Atmos. Res., 176–177, 121–133, https://doi.org/10.1016/j.atmosres.2016.02.020.
He, Z., L. Yang, F. Tian, G. Ni, A. Hou, and H. Lu, 2017: Intercomparisons of rainfall estimates from TRMM and GPM multisatellite products over the upper Mekong River basin. J. Hydrometeor., 18, 413–430, https://doi.org/10.1175/JHM-D-16-0198.1.
Hong, Y., K.-L. Hsu, S. Sorooshian, and X. Gao, 2004: Precipitation estimation from remotely sensed imagery using an artificial neural network cloud classification system. J. Appl. Meteor., 43, 1834–1853, https://doi.org/10.1175/JAM2173.1.
Hou, A. Y., and Coauthors, 2014: The Global Precipitation Measurement Mission. Bull. Amer. Meteor. Soc., 95, 701–722, https://doi.org/10.1175/BAMS-D-13-00164.1.
Huffman, G. J., and Coauthors, 2007: The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J. Hydrometeor., 8, 38–55, https://doi.org/10.1175/JHM560.1.
Huffman, G. J., and Coauthors, 2019a: NASA Global Precipitation Measurement (GPM) Integrated Multi-Satellite Retrievals for GPM (IMERG). NASA Algorithm Theoretical Basis Doc., version 06, 38 pp., https://pmm.nasa.gov/sites/default/files/document_files/IMERG_ATBD_V06.pdf.
Huffman, G. J., D. T. Bolvin, E. J. Nelkin, and J. Tan, 2019b: Integrated Multi-Satellite Retrievals for GPM (IMERG) technical documentation. NASA Tech. Doc., 77 pp., https://pmm.nasa.gov/sites/default/files/document_files/IMERG_doc_190909.pdf.
Janowiak, J., R. Joyce, and P. Xie, 2017: NCEP/CPC L3 half hourly 4km global (60S–60N) merged IR, V1. NCEP Climate Prediction Center, accessed 8 July 2019, https://doi.org/10.5067/p4hzb9n27eku.
Jiang, S., L. Ren, C.-Y. Xu, B. Yong, F. Yuan, Y. Liu, X. Yang, and X. Zeng, 2018: Statistical and hydrological evaluation of the latest Integrated Multi-Satellite Retrievals for GPM (IMERG) over a midlatitude humid basin in South China. Atmos. Res., 214, 418–429, https://doi.org/10.1016/j.atmosres.2018.08.021.
Joyce, R. J., and P. Xie, 2011: Kalman filter–based CMORPH. J. Hydrometeor., 12, 1547–1563, https://doi.org/10.1175/JHM-D-11-022.1.
Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487–503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.
Kidd, C., 2019: NASA Global Precipitation Measurement (GPM) Precipitation Retrieval and Profiling Scheme (PRPS). NASA Algorithm Theoretical Basis Doc., version 01-02, 17 pp., https://pmm.nasa.gov/data-access/downloads/gpm.
Kim, K., J. Park, J. Baik, and M. Choi, 2017: Evaluation of topographical and seasonal feature using GPM IMERG and TRMM 3B42 over Far-East Asia. Atmos. Res., 187, 95–105, https://doi.org/10.1016/j.atmosres.2016.12.007.
Kirschbaum, D. B., and Coauthors, 2017: NASA’s remotely sensed precipitation: A reservoir for applications users. Bull. Amer. Meteor. Soc., 98, 1169–1184, https://doi.org/10.1175/BAMS-D-15-00296.1.
Kirstetter, P.-E., and Coauthors, 2012: Toward a framework for systematic error modeling of spaceborne precipitation radar with NOAA/NSSL ground radar–based national mosaic QPE. J. Hydrometeor., 13, 1285–1300, https://doi.org/10.1175/JHM-D-11-0139.1.
Kirstetter, P.-E., Y. Hong, J. J. Gourley, Q. Cao, M. Schwaller, and W. Petersen, 2014: Research framework to bridge from the Global Precipitation Measurement Mission core satellite to the constellation sensors using ground-radar-based national mosaic QPE. Remote Sensing of the Terrestrial Water Cycle, Geophys. Monogr., Vol. 206, Amer. Geophys. Union, 61–79.
Kirstetter, P.-E., J. J. Gourley, Y. Hong, J. Zhang, S. Moazamigoodarzi, C. Langston, and A. Arthur, 2015: Probabilistic precipitation rate estimates with ground-based radar networks. Water Resour. Res., 51, 1422–1442, https://doi.org/10.1002/2014WR015672.
Kummerow, C., and Coauthors, 2001: The evolution of the Goddard profiling algorithm (GPROF) for rainfall estimation from passive microwave sensors. J. Appl. Meteor., 40, 1801–1820, https://doi.org/10.1175/1520-0450(2001)040<1801:TEOTGP>2.0.CO;2.
Kummerow, C., S. Ringerud, J. Crook, D. Randel, and W. Berg, 2011: An observationally generated a priori database for microwave rainfall retrievals. J. Atmos. Oceanic Technol., 28, 113–130, https://doi.org/10.1175/2010JTECHA1468.1.
Kummerow, C., D. L. Randel, M. Kulie, N.-Y. Wang, R. Ferraro, S. Joseph Munchak, and V. Petkovic, 2015: The evolution of the Goddard Profiling algorithm to a fully parametric scheme. J. Atmos. Oceanic Technol., 32, 2265–2280, https://doi.org/10.1175/JTECH-D-15-0039.1.
Lucchesi, R., 2017: File specification for GEOS-5 FP. NASA GMAO Rep., 61 pp., http://gmao.gsfc.nasa.gov/pubs/office_notes.
Manz, B., S. Páez-Bimos, N. Horna, W. Buytaert, B. Ochoa-Tocachi, W. Lavado-Casimiro, and B. Willems, 2017: Comparative ground validation of IMERG and TMPA at variable spatiotemporal scales in the tropical Andes. J. Hydrometeor., 18, 2469–2489, https://doi.org/10.1175/JHM-D-16-0277.1.
Nguyen, P., M. Ombadi, S. Sorooshian, K. Hsu, A. AghaKouchak, D. Braithwaite, H. Ashouri, and A. R. Thorstensen, 2018: The PERSIANN family of global satellite precipitation data: A review and evaluation of products. Hydrol. Earth Syst. Sci., 22, 5801–5816, https://doi.org/10.5194/hess-22-5801-2018.
Palomino-Ángel, S., J. A. Anaya-Acevedo, and B. A. Botero, 2019: Evaluation of 3B42V7 and IMERG daily-precipitation products for a very high-precipitation region in northwestern South America. Atmos. Res., 217, 37–48, https://doi.org/10.1016/j.atmosres.2018.10.012.
Prakash, S., A. K. Mitra, A. AghaKouchak, Z. Liu, H. Norouzi, and D. S. Pai, 2018: A preliminary assessment of GPM-based multi-satellite precipitation estimates over a monsoon dominated region. J. Hydrol., 556, 865–876, https://doi.org/10.1016/j.jhydrol.2016.01.029.
Precipitation Processing System, 2017: GPM GMI (GPROF) radiometer precipitation profiling L2A 1.5 hours 13 km V05. NASA GES DISC, accessed 27 June 2019, https://doi.org/10.5067/gpm/gmi/gpm/gprof/2a/05.
Precipitation Processing System, 2018: GPM DPR and GMI combined precipitation L2B 1.5 hours 5 km V06. NASA GES DISC, accessed 27 June 2019, https://doi.org/10.5067/gpm/dprgmi/cmb/2b/06.
Precipitation Processing System, 2019a: GPM IMERG final precipitation L3 half hourly 0.1 degree × 0.1 degree V06. NASA GES DISC, accessed 27 June 2019, https://doi.org/10.5067/gpm/imerg/3b-hh/06.
Precipitation Processing System, 2019b: GPM SAPHIR on MT1 (PRPS) radiometer precipitation profiling L2 1.5 hours 10 km V06. NASA GES DISC, accessed 27 June 2019, https://doi.org/10.5067/gpm/saphir/mt1/prps/2a/06.
Romanov, P., G. Gutman, and I. Csiszar, 2000: Automated monitoring of snow cover over North America with multispectral satellite data. J. Appl. Meteor., 39, 1866–1880, https://doi.org/10.1175/1520-0450(2000)039<1866:AMOSCO>2.0.CO;2.
Satgé, F., A. Xavier, R. P. Zolá, Y. Hussain, F. Timouk, J. Garnier, and M.-P. Bonnet, 2017: Comparative assessments of the latest GPM mission’s spatially enhanced satellite rainfall products over the main Bolivian watersheds. Remote Sens., 9, 369, https://doi.org/10.3390/rs9040369.
Schneider, U., A. Becker, P. Finger, A. Meyer-Christoffer, M. Ziese, and B. Rudolf, 2014: GPCC’s new land surface precipitation climatology based on quality-controlled in situ data and its role in quantifying the global water cycle. Theor. Appl. Climatol., 115, 15–40, https://doi.org/10.1007/s00704-013-0860-x.
Schneider, U., A. Becker, P. Finger, A. Meyer-Christoffer, B. Rudolf, and M. Ziese, 2015: GPCC full data reanalysis version 7.0 at 1.0°: Monthly land-surface precipitation from rain-gauges built on GTS-based and historic data: Gridded monthly totals. DWD, accessed 27 June 2019, https://doi.org/10.5676/dwd_gpcc/fd_m_v7_100.
Skofronick-Jackson, G., and Coauthors, 2017: The Global Precipitation Measurement (GPM) mission for science and society. Bull. Amer. Meteor. Soc., 98, 1679–1695, https://doi.org/10.1175/BAMS-D-15-00306.1.
Skofronick-Jackson, G., D. Kirschbaum, W. Petersen, G. Huffman, C. Kidd, E. Stocker, and R. Kakar, 2018: The Global Precipitation Measurement (GPM) mission’s scientific achievements and societal contributions: Reviewing four years of advanced rain and snow observations. Quart. J. Roy. Meteor. Soc., 144, 27–48, https://doi.org/10.1002/qj.3313.
Tan, J., W. A. Petersen, P.-E. Kirstetter, and Y. Tian, 2017: Performance of IMERG as a function of spatiotemporal scale. J. Hydrometeor., 18, 307–319, https://doi.org/10.1175/JHM-D-16-0174.1.
Tan, J., W. A. Petersen, G. Kirchengast, D. C. Goodrich, and D. B. Wolff, 2018: Evaluation of Global Precipitation Measurement rainfall estimates against three dense gauge networks. J. Hydrometeor., 19, 517–532, https://doi.org/10.1175/JHM-D-17-0174.1.
Tan, M., and Z. Duan, 2017: Assessment of GPM and TRMM precipitation products over Singapore. Remote Sens., 9, 720, https://doi.org/10.3390/rs9070720.
Tang, G., Z. Zeng, D. Long, X. Guo, B. Yong, W. Zhang, and Y. Hong, 2016: Statistical and hydrological comparisons between TRMM and GPM level-3 products over a midlatitude basin: Is day-1 IMERG a good successor for TMPA 3B42V7? J. Hydrometeor., 17, 121–137, https://doi.org/10.1175/JHM-D-15-0059.1.
Ushio, T., and Coauthors, 2009: A Kalman filter approach to the Global Satellite Mapping of Precipitation (GSMaP) from combined passive microwave and infrared radiometric data. J. Meteor. Soc. Japan, 87A, 137–151, https://doi.org/10.2151/jmsj.87A.137.
Wang, Z., R. Zhong, C. Lai, and J. Chen, 2017: Evaluation of the GPM IMERG satellite-based precipitation products and the hydrological utility. Atmos. Res., 196, 151–163, https://doi.org/10.1016/j.atmosres.2017.06.020.
Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Elsevier, 676 pp.
Xie, P., R. Joyce, S. Wu, S.-H. Yoo, Y. Yarosh, F. Sun, and R. Lin, 2017: Reprocessed, bias-corrected CMORPH global high-resolution precipitation estimates from 1998. J. Hydrometeor., 18, 1617–1641, https://doi.org/10.1175/JHM-D-16-0168.1.
Yamamoto, M. K., and S. Shige, 2015: Implementation of an orographic/nonorographic rainfall classification scheme in the GSMaP algorithm for microwave radiometers. Atmos. Res., 163, 36–47, https://doi.org/10.1016/j.atmosres.2014.07.024.
Yamamoto, M. K., S. Shige, C.-K. Yu, and L.-W. Cheng, 2017: Further improvement of the heavy orographic rainfall retrievals in the GSMaP algorithm for microwave radiometers. J. Appl. Meteor. Climatol., 56, 2607–2619, https://doi.org/10.1175/JAMC-D-16-0332.1.
