a. Objective function and state variables

The GPM combined algorithm is based on a physical (forward) model that simulates satellite radar and radiometer observations as a function of relevant geophysical variables (Grecu et al. 2004). Because the values of these geophysical variables cannot be uniquely determined from satellite observations alone, a probabilistic formulation based on Bayes’s theorem (Marzano et al. 1999) is used to derive the most likely values consistent not only with the satellite observations but also with the climatology or a priori estimates of the geophysical variables.

Mathematically, this is equivalent to the minimization of the following objective function:

where is the set of geophysical variables to be estimated, is the physically based forward model, are the satellite radar and radiometer observations, is the covariance matrix of forward model errors and noise in the observations, is the a priori mean of , and is its covariance matrix. The simulated observation vector includes the Ka-band (35.5 GHz) radar reflectivity, the GMI brightness temperatures, and the path-integrated attenuation at Ku band and the differential Ku–Ka-band path-integrated attenuation. The Ku-band (13.6 GHz) reflectivity is not included in , as the Ku-band radar observations are used along with a Hitschfeld–Bordan radar profiling procedure (Grecu et al. 2011) to reduce the dimension of .

Nominally, the vector incorporates all parameters needed to describe the precipitation particle size distribution (PSD) in each range bin of the radar-observed vertical profile. The PSD is needed to describe the single-scattering properties of the atmosphere in the range bin, and these scattering properties are used to simulate the attenuated Ka-band reflectivities, as well as the upwelling brightness temperatures, associated with the profile. To describe the PSD, the normalized gamma distribution (Testud et al. 2001) is used to relate the number density of precipitation particles to their size, that is,

Here, is the intercept of the normalized distribution (equivalent to in an exponential distribution), is the mass-weighted mean diameter, μ is the distribution shape factor, D is the liquid water equivalent diameter of the particle, and is the spectral number density of particles with diameter D.

Although each range gate is described by three parameters, to reduce the dimensionality of the problem, the shape factor μ is prescribed, leaving and as free parameters. In addition, as pointed out by Grecu et al. (2004), it is advantageous to use the Ku-band radar observations to analytically diagnose the profile of , given a priori assumptions regarding , rather than including them in of the objective function [Eq. (1)]. The diagnosis of the profile given is achieved using the generalized Hitschfeld–Bordan solution described in Grecu et al. (2011). It is therefore necessary to include only in the vector of parameters , and the Ku-band observations need not be included in of Eq. (1), since they are already consistent with the assumed and derived profile. This strategy was also successfully employed by Grecu et al. (2004), Hogan (2007), and Grecu et al. (2011) to reduce the dimensionality of the optimization problem associated with Eq. (1).

To capture the vertical variability of , a strategy similar to that of Grecu et al. (2011) is used. Specifically, is defined at nine different vertical locations within the radar profile and interpolated to intermediate range gates. The nine locations are defined based on the five nodes introduced by Iguchi et al. (2000) to describe typical storm structures in spaceborne radar observations. The five nodes defined by Iguchi et al. (2000) are as follows: (i) A is the highest-altitude bin with detectable echo at the Ku band, (ii) B is the top of the mixed-phase layer, (iii) C is the peak of the bright band for stratiform precipitation and the bin corresponding to estimated freezing height for convective precipitation, (iv) D is the bottom of the mixed-phase layer, and (v) E is the lowest bin above surface clutter. In addition to the five storm structure nodes, three nodes are introduced between nodes A and B, and one node is introduced between nodes D and E to describe the vertical variability of . Although nine values are included in to describe the vertical variability of , these values are not independent, as correlations among the derived values are imposed by the estimation framework. More details on the imposed variations of are provided in section 2b.

Given a profile of Ku-band radar observations, a precipitation profile () is derived as a function of the nine profile values included in and a prescribed μ profile. The μ profile is not optimized (and therefore not included in ), and the strategy used to prescribe it is discussed in section 2c. Two species of hydrometeors are derived from the , μ, and profiles as a function of the storm’s five-node structure. Specifically, above node B, the PSD parameters , μ, and are associated with snow, while below node D the PSD parameters are associated with rain. In the mixed phase—that is, between nodes B and D—the PSD parameters are associated with a snow–rain mixture. In stratiform precipitation, the mixed-phase relationships between PSD parameters and radar–radiometer electromagnetic scattering properties are derived using the melting layer of Liao and Meneghini (2005). In convective precipitation, the scattering lookup tables are a linear combination of the rain and graupel lookup tables (Grecu et al. 2011).

Ideally, all the variables that have an impact on the satellite observations, less the variable that is derived directly from the DPR Ku-band observations, should be included in the state variable . However, variables that have been deemed difficult to accurately estimate using the optimal estimation framework in Eq. (1) are specified based on a priori assumptions and ancillary data, and they are not included in . A description of the additional state variables and related assumptions follows.

The -dependent precipitation profiles derived from the Ku-band radar observations are used to simulate profiles of single-scattering properties, including the bulk extinction coefficient, the single-scattering albedo, the asymmetry parameter, and the backscattering coefficient. These are combined with the extinction properties of cloud and atmospheric gases to compute the bulk single-scattering properties of each range bin in the profile. The bulk single-scattering properties, along with the temperature profiles and the emissivities of the earth’s surface, are required to simulate the remaining satellite radar and radiometer observations that are included in of Eq. (1).

The vertical distributions of water vapor and cloud water in each radar profile are described using low-order representations based on an empirical orthogonal function (EOF) decomposition. The EOFs are derived from Weather Research and Forecasting (WRF) Model (Michalakes et al. 2001) simulations representing diverse meteorological situations (midlatitude cyclones, tropical convection, etc.). Cloud ice is currently not represented in the combined algorithm, and therefore it is neither included in nor prescribed otherwise. Surface emissivity is modeled as a function of surface wind and temperature over oceans (Meissner and Wentz 2012; Munchak et al. 2016) and is prescribed based on the Tool to Estimate Land Surface Emissivities at Microwave Frequencies (TELSEM) database (Aires et al. 2011) over land. The sea surface wind is therefore included in . Given the Ku-band (13.6 GHz) radar observations and , the other GPM observations—that is, the DPR Ka-band (35.5 GHz) radar observations and GMI observations—are simulated, and the agreement between the simulations and the observations is evaluated as . As already mentioned, the derivation of optimal estimates requires the minimization of function J in Eq. (1). This is achieved through a statistical optimization procedure, which is described next.

b. Ensemble filter retrieval

Irrespective of the optimization approach, the minimization of function J requires multiple evaluations. When a gradient-based approach is used, a special technique to evaluate the gradient of J (called adjoint modeling) is necessary to make the solution computationally feasible. However, adjoint modeling can limit the development of new and more advanced physical forward models in the combined algorithm, and so an alternative strategy is employed here. Specifically, instead of sequentially evaluating J and updating , an ensemble of initial potential solutions is generated, and the associated simulated observations are calculated. The initial solutions are derived from the Ku-band reflectivity data only (as described in the previous subsection), given different a priori assumptions concerning the profiles and environmental parameters. The first- and second-order derivatives of J are evaluated statistically from the ensembles, and a second-order update of the initial guess solution is derived. Moreover, because the initial ensemble of state variables is generated in a manner consistent with their a priori distributions, the term does not have to be explicitly evaluated. Instead, an estimate of is generated from a multivariate normal distribution with covariance and updated using

where is the posteriori ensemble mean of (after the satellite observations have been processed), is the a priori ensemble mean of , is the ensemble mean of , and is the sum of forward model and observation uncertainties. Equation (4) is formally identical to the state variable update formula in an ensemble Kalman filter (EnkF); see Evensen (2006). Note that even though the combined algorithm’s framework is technically not an ensemble Kalman filter because it does not involve dynamic models and sequential assimilation of observations, it is, in essence, an ensemble filter because it updates the a priori distribution of state variables using actual observations. A convenient way to estimate the uncertainties in the state variables derived by the application of Eq. (4) is to update not only the ensemble mean but also the state variables for every member of the ensemble. The standard deviation of the ensemble of state variables is an estimate of their uncertainties. This follows because the covariance of the updated state variables is an estimate of the posteriori uncertainties in . Although it has been proven that ”assimilating” the same observations multiple times while multiplying their uncertainty by the number of times the observations are “assimilated” is effective in dealing with nonlinearities in the models (Emerick and Reynolds 2012), in the current version of the algorithm, Eq. (4) is applied only once. This is because the algorithm is computationally intensive and the multiple assimilations are impractical in the current implementation. Nevertheless, multiple “assimilations” may be adaptively applied and will be implemented in future versions of the algorithm.

A particular challenge in the development of optimal estimation frameworks is the quantification of a priori distributions of state variables, that is, variables and in Eq. (1). Cloud-resolving model (e.g., WRF) simulations may be used to derive estimates of and , but evaluations of early versions of the algorithm revealed that the distributions of near the earth’s surface derived from WRF simulations were systematically different from observed distributions. On the other hand, observations alone are not sufficient to derive the vertical distributions of , as these observations are primarily limited to the liquid phase near the ground. Therefore, in the present formulation, a statistical (autoregressive) model is used to describe the vertical variability of . The coefficients of the autoregressive model were chosen to induce a correlation distance of 6 km, which is consistent with correlation distances derived from polarimetric radar retrievals (see, e.g., Bringi et al. 2015). The profiles are generated on a grid coarser than the DPR grid and then interpolated to the DPR footprint locations. The purpose of the spatial interpolation of profiles is to create correlated values of across the precipitation field in any given ensemble member, since such correlations are observed in field studies (e.g., Tapiador et al. 2010; Tokay and Bashor 2010; Bringi et al. 2015). The means of the “a priori” are those of the TRMM 2A25 algorithm (Viltard et al. 2000).

As previously described, the water vapor and cloud water EOFs are derived from four WRF simulations of precipitation systems associated with different large-scale environments. These include a tropical storm that occurred in the observing period and region of the Tropical Composition, Cloud, and Climate Coupling (TC4) Experiment (Toon et al. 2010), two warm season (weak and strong baroclinic) precipitation systems over southeastern Texas (Hopper and Schumacher 2012), and a North Atlantic oceanic extratropical cyclone (Posselt et al. 2008). While the set of four storms used in the EOF analysis is not comprehensive, extending the breadth of the set of simulations is not a priority in the current algorithm development, as other modeling activities have been determined to have a significantly larger impact on precipitation retrievals. Nevertheless, the lack of water vapor and cloud water profiles representative of high-latitude winter storms may have a significant impact on the high-latitude retrievals, and the extension of model simulation set is a priority for the next version of the algorithm. The water vapor and cloud water EOFs calculated from the four simulations are scaled by randomly generated principal components to derive water vapor and cloud water vapor profiles for all DPR profiles characterized by nonzero precipitation. The randomly generated principal components are included in and, consequently, optimally estimated by the algorithm.

Also, as previously mentioned, the microwave surface emissivity is modeled as a function of surface wind over oceans and prescribed based on the TELSEM database over land. The surface wind is included in . Given the variable and the DPR Ku-band observations, simulations of the DPR Ka-band radar observations and the GMI radiometer observations can be readily derived as a function . These components of the forward model are described next.

c. Forward model

As discussed in section 2a, the microwave scattering properties needed to simulate the DPR Ka-band radar and GMI radiometer observations are derived from precipitation profiles consistent with the DPR Ku-band radar observations, for a given , using the generalized Hitschfeld–Bordan profiling methodology of Grecu et al. (2011). For the sake of brevity, details of the profiling methodology are not repeated here.

As in Grecu et al. (2011), scattering lookup tables at all frequencies of interest (i.e., Ku band, Ka band, and all of the GMI frequencies) are generated using the melting layer model of Liao and Meneghini (2005). The densities of snow particles (assumed to be spherical) are 0.1 g cm⁻³ in stratiform precipitation and 0.4 g cm⁻³ in convective precipitation, and the PSD shape factor μ = 2. The use of scattering tables as a function of the storm’s vertical structure (Iguchi et al. 2000) is explained in Grecu et al. (2011). The snow densities and the shape factor are prescribed and not retrieved (i.e., they are not included in ), as preliminary tests suggested more robust overall algorithm performance when these parameters are prescribed rather than retrieved. Moreover, the use of fixed snow density mitigates errors in the quantification of electromagnetic scattering properties using the fluffy sphere approximation (Kuo et al. 2016), which is utilized here. It should be mentioned that ice particles with densities significantly larger than 0.4 g cm⁻³ (e.g., hail) may occur in convective cores. In such cases, the combined algorithm is likely to overestimate the ice-phase precipitation water content. The use of hail detection procedures—for example, Waldvogel et al. (1979) and Leppert and Cecil (2015)—and the inclusion of hail as a distinct hydrometeor class are expected to improve the accuracy of convective precipitation retrieval but are not yet implemented in the current version of the algorithm.

The model used to simulate the DPR Ka-band observations from the lookup table–derived Ka-band true reflectivity and extinction profiles is more complex than that used by Grecu et al. (2011). The reason for this greater complexity is that multiple scattering—that is, the scattering of radar pulse energy several times off atmospheric targets before returning to the receiver—can significantly enhance Ka-band reflectivity observations. The multiple-scattering effect was anticipated by Battaglia et al. (2006) and has since been illustrated by Battaglia et al. (2015), and the reader is referred to these papers for detailed descriptions. To appropriately account for this effect, the computationally fast 1D multiple-scattering model developed by Hogan and Battaglia (2008) is used to simulate the observed Ka-band reflectivity profiles from the Ka-band “true” reflectivity, extinction coefficient, single-scattering albedo, and asymmetry parameter profiles estimated from the Ku-band observations. The Ka-band extinction profiles include extinction due to water vapor and cloud water derived from the EOF-based water vapor and cloud water profiles. The extinction due to water vapor is determined using the model of Rosenkranz (1998).

The extinction coefficient, single-scattering albedo, and asymmetry parameter profiles at the GMI frequencies are also derived from the Ku-band observations, and these parameters are employed to simulate the DPR-resolution brightness temperatures that are required to update the initial ensemble of Ku band–derived profiles using the GMI observations. The radiative transfer model used in the brightness temperature simulations is based on Eddington’s second approximation; see Kummerow (1993). Although the slant-path application of the Eddington model has been proven effective in mitigating 3D effects of microwave radiative transfer in precipitating clouds (Olson et al. 2001; Munchak and Kummerow 2011), a plane-parallel implementation of the Eddington model is used here, because it allows for an almost ideal parallel processing speedup of computations; see Hennessy and Patterson (2011). Specifically, given the large number of intensive radiative calculations carried out to derive optimal combined retrievals, parallel processing is a necessity. The combined algorithm’s implementation is based on the Open Multi-Processing (OpenMP) standard (Dagum and Menon 1998), which allows for the efficient exploitation of the coarse-grained parallelism associated with the ensemble filter approach. A slant-path radiative model, while still amenable to parallelization, would require more complex programming, significantly more memory, and greater communication and synchronization overhead. Moreover, as shown by Bauer et al. (1998), 3D effects can also be addressed through better matching between simulations and observations based on the projection of the spectral center of gravity (Bauer et al. 1998) of observed brightness temperatures rather than their footprint location.

The uncertainties in the radiative transfer forward model are set using the methodology of Grecu and Anagnostou (2002). Specifically, the differences between a 3D Monte Carlo radiative transfer model and the plane-parallel radiative transfer model used in the combined algorithm are quantified for a large number of precipitation profiles, and the corresponding elements in are evaluated. Although these uncertainties do not account for uncertainties in the scattering lookup tables, it has been found that their magnitude (about 7 K at 37 and 89 GHz) is large enough to prevent the derivation of unrealistic estimates due to uncertainties and biases in the scattering lookup tables.

The application of Eq. (4) requires that the simulated and observed brightness temperatures be at the same resolution. Rather than convolving the DPR-resolution brightness temperatures to GMI resolution, the combined algorithm uses a statistical procedure to deconvolve the GMI observations to DPR resolution. The statistical procedure and its benefits are explained in subsection 2e.

d. Nonuniform beamfilling parameterization

The application of the day 1 (at launch) version of the combined algorithm to GPM data revealed significant differences between estimates using both Ku- and Ka-band observations and retrievals that did not use the Ka-band radar observations. These differences mainly occurred in retrievals of convective precipitation over land and were associated with large (more than 50%) reductions of estimated surface precipitation when the Ka-band observations were introduced into the filter. An analysis of simulated (derived from Ku-band radar using the generalized Hitschfeld–Bordan method) and observed Ka-band radar reflectivity factors used in the ensemble filter update [Eq. (4)] indicated that the simulated Ka-band reflectivities were significantly and systematically lower than the actual observations in convective rain. At the same time, the differences between near-surface simulated and observed Ka-band reflectivities were found to be strongly and negatively correlated with the simulated path-integrated attenuation (PIA) at both DPR channel frequencies. Consequently, adjustments to improve the consistency between simulated and observed Ka-band observations were invariably associated with significant reductions in the PIAs and, consequently, significant reductions in the estimated precipitation rate.

However, the retrieved profiles appeared to be unrealistic in several respects [e.g., typically showing unnatural precipitation rate reductions toward the surface and estimated PIAs larger than those derived from the surface return technique (SRT; Meneghini et al. 2000)], which raised some doubt that the radar forward model and parameterizations used in the algorithm were appropriately accounting for all physical processes occurring in nature. The inclusion of the multiple-scattering model of Hogan and Battaglia (2008) in the combined algorithm significantly improved the consistency between simulated and observed Ka-band radar observations in convective precipitation over land, but significant discrepancies (although smaller than those in the day 1 estimates) were still noted between single- and dual-frequency-based estimates. Further analysis indicated that these discrepancies were caused by systematic differences between the a priori–simulated PIAs and those estimated from the SRT; differences that could not be addressed through the inclusion of the multiple-scattering model.

Previous studies—for example, Iguchi et al. (2009)—demonstrated that precipitation variability within the footprint of a spaceborne radar operating at attenuating frequencies can cause significant differences between the effective integrated attenuation, defined as the ratio of the true reflectivity to the measured reflectivity, and the average integrated attenuation, defined as the fraction of power that is essentially lost due to attenuation. This and other nonlinear effects caused by variability within the radar’s footprint are usually termed nonuniform beamfilling (NUBF) effects. NUBF effects are difficult to mitigate in operational applications because the precipitation variability within a given radar footprint cannot be reliably described using a small number of variables that can be estimated from radar-only observations (Takahashi et al. 2006). However, a methodology to account for NUBF in retrievals based on dual-frequency radar observations is necessary to avoid large errors in estimated and precipitation rates.

Therefore, a simple NUBF parameterization, based on a downscaling procedure, has been implemented in the combined algorithm. Specifically, given a profile of Ku-band radar reflectivity observations, , a set of profiles is statistically generated such that , where the angle brackets represent the horizontal averaging operator. This is achieved by generating a set of lognormally distributed random variables and using them to derive through a simple scaling of the measured Ku-band reflectivity profile, that is, . The standard deviation of is prescribed as a function of surface type (i.e., 0.75 for nonocean surfaces and 0.25 for oceans). These values have been determined through a trial and error process, and their impact on precipitation estimates has been systematically studied. The analysis of airborne radar data suggests that f is a function of height, but further investigation is required to formulate a more realistic, yet robust, downscaling procedure. Therefore, the vertical variability of f is not modeled in the current version of the algorithm. The generalized Hitschfeld–Bordan procedure is applied to each of the downscaled observation profiles, and the derived profiles of reflectivities at Ka-band and bulk scattering properties at the GMI frequencies are averaged back to the DPR resolution. For computational efficiency, the Ka-band multiple-scattering reflectivity calculations and the radiative transfer simulations are performed at DPR resolution.

e. Statistical deconvolution of GMI observations

The signal received by the GMI radiometer can be expressed, mathematically, as a weighted average of radiometric signal originating in the GMI footprint. In discrete form, this can be written as

where is the GMI observed brightness temperature at a location indexed by i; is the radiance emerging from an elementary volume, indexed by j, bounded by a small area within the radiometer’s footprint; and is a weighting factor related to the antenna gain pattern. The tilde symbol in the equation above (and anywhere in the paper henceforth) is used to indicate brightness temperatures at GMI resolution, while brightness temperatures without a tilde sign are defined at DPR resolution. The inversion operation—that is, deriving the radiance emerging from subvolumes inside a GMI observing volume from GMI observations—is more difficult and problematic in general, but, in the context of GPM precipitation combined retrievals, there are several advantages to estimating the DPR-resolution radiances from GMI-resolution radiances. Specifically, the application of Eq. (4) requires the evaluation and inversion of a large matrix, that is, . This matrix is sparse, but it is not blockwise invertible (because radiances are characterized by overlapping observing volumes), which makes Eq. (4) irreducible to a set of independent and simpler equations that can be applied sequentially. Although iterative techniques that allow for the evaluation of without the need to explicitly invert exist, the sequential update of state variables and covariance localization (Anderson (2012)) are preferable because they effectively mitigate sampling errors. That is to say, the covariance matrices derived from the ensemble information are noisy (due to the relatively small size of the ensemble), which may induce erroneous updates in the state variables (Anderson 2012). In covariance localization, elements of the covariance matrix that are expected to be small are explicitly set to 0, even though, due to the small ensemble size, their statistical estimates may be different from 0. Although technically not covariance localization, a statistical deconvolution technique that allows for the derivation of DPR-resolution (nonoverlapping) radiometer pseudo-observations is similarly effective in mitigating the impact of the small size of the ensemble by limiting the radius of influence of radiometer observations to the DPR’s footprint size. In addition, the statistical deconvolution followed by sequential filter updates provides a simpler computational framework. The potentially larger uncertainties in the observations (due to noise amplification during the deconvolution step) are not likely to have a significant impact on the retrievals because the information from various channels is processed simultaneously, and the correlation between observations is effective in filtering out noise.

To derive the DPR-resolution brightness temperatures from observed GMI-resolution brightness temperatures, we use a linear formulation of the type

subject to the constraint . The coefficients are determined from a large number of DPR- and GMI-resolution brightness temperatures, and , derived through simulation, and the regularized minimization of the sum of squared residuals, that is,

where γ is a regularization factor introduced to minimize the amplification of noise during the deconvolution process. The regularization factor γ is derived through cross validation. Specifically, several days’ worth of GPM DPR observations are used to derive precipitation retrievals and to simulate DPR-resolution brightness temperatures at the GMI frequencies. The DPR-resolution brightness temperatures are then convolved using Eq. (5) to derive GMI-like brightness temperatures. The synthetic dataset of GMI-resolution and associated DPR-resolution brightness temperatures is randomly divided into two equal-sized sets and the first set (the training dataset) is used to derive coefficients through the minimization of the sum of squared residuals in Eq. (7). The regularization factor γ is chosen to minimize the sum of squared residuals, that is, , in the second dataset (the validation dataset).

In addition to the optimal estimation of and γ, to prevent noise and uncertainty amplification, the deconvolution procedure makes use of DPR information to bound the deconvolved brightness temperatures within physical limits. Specifically, an ensemble of DPR-resolution brightness temperatures at GMI frequencies, , is derived from the DPR Ku-band radar estimates. The DPR-resolution brightness temperatures are convolved to the GMI resolution, and the differences between the DPR-based brightness temperatures and actual observations are evaluated as

where are the actual GMI observations and are the GMI-resolution brightness temperatures derived from convolved ensemble-average DPR-resolution brightness temperatures, . These differences are deconvolved to DPR resolution using Eq. (6), that is,

and the are used to derive the most likely brightness temperatures at DPR resolution, that is,

where e is the deconvolved (i.e., the difference between observed and simulated brightness temperatures at GMI resolution). The deconvolved brightness temperatures are further constrained by the range of possible values determined by the Ku-band ensemble simulations, that is, , where is the set (ensemble) of brightness temperatures simulated from the Ku-band radar retrievals. Although formally similar to resolution enhancement techniques used in previous studies (e.g., Bauer and Bennartz 1998), the deconvolution technique used in the combined algorithm has the unique aspect that it limits the final value to a range derived through physical simulations.

f. Retrieval workflow

The entire retrieval process is illustrated in Fig. 1. The algorithm starts by reading the GMI observations and the ancillary data (lookup scattering tables, emissivity EOFs, etc.). The radar data are processed (and consequently read) in batches of 300 scans. This segmentation of the data is required because available computer memory limits the number of Ku-band radar profile estimates and the associated Ka-band and GMI simulations that can be reliably processed at once. The radar data include the Ku- and Ka-band observed reflectivities, the precipitation type determined from Ku-band observations (Awaka et al. 2007), and the five-node storm structure (Iguchi et al. 2000). The environmental data [derived by interpolating the model-based global analysis (GANAL) data provided by the Japan Meteorological Agency to the DPR scan-ray grid] is also read in batches of 300 scans. For each batch, an ensemble of state variable vectors, defined in section 2a, is generated for each radar footprint, and an ensemble of Ku-band radar–only profile estimates is derived as a function of at each footprint location. Simulations of Ka-band radar reflectivity profiles, path-integrated attenuation at both Ku and Ka bands, and brightness temperatures at the radiometer channel frequencies and polarizations are derived from the Ku-band profile estimates. The ensemble DPR-brightness temperature averages are calculated and used to determine the differences between GMI brightness temperature observations and DPR-based brightness temperature simulations [i.e., Eq. (8) in section 2e]. These differences are deconvolved to DPR resolution and used to derive the most likely brightness temperatures at DPR resolution [i.e., Eq. (10)]. The DPR-resolution brightness temperatures, along with the observed Ka-band radar reflectivity observations and SRT estimates of PIA at both frequencies, are incorporated into the ensemble filter to update the state variables using Eq. (4). At the same time, the precipitation profiles and simulated radar and radiometer observations are updated. For diagnostic purposes, the convolution procedure is applied again to the updated DPR-resolution brightness temperatures to simulate the brightness temperatures at GMI resolution. The main variables in the output file include the precipitation rate and precipitation water content at 250-m vertical resolution, the five nodes defining the storm structure, the fraction of liquid precipitation at ranges corresponding to the mixed phase, the fraction of liquid surface precipitation, the nine values, etc. A detailed description of variables saved in the output file, their meaning, and format may be found in the NASA Precipitation Processing System (PPS) GPM product file specification available online (at https://storm.pps.eosdis.nasa.gov/storm/Product.jsp).

• Download Figure
• Download figure as PowerPoint slide

Combined algorithm workflow.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

The filter update and brightness temperature convolution steps are applied independently twice, resulting in two sets of combined algorithm precipitation estimates. The first set of estimates is based on data within the overlap of the Ku-band radar and GMI radiometer swaths— that is, the Ku-band SRT PIA estimates (Meneghini et al. 2015) and deconvolved GMI brightness temperatures are used—while Ka-band radar observations are not used. These estimates are referred to as the normal scan swath (NS) estimates. The second sets of estimates is based on data within the overlap of the Ku radar, Ka radar, and GMI radiometer swaths; that is, the Ka-band reflectivity observations, the differential SRT PIA estimates (Meneghini et al. 2015), the Ku-band SRT PIA estimates (Meneghini et al. 2015), and the deconvolved GMI brightness temperatures are used. This results in a Ku+Ka+GMI precipitation product,referred to as the matched scan swath (MS) product, that only draws upon input over the narrower Ka (or MS) swath. Differential SRT PIA estimates are used in favor of Ka SRT PIA estimates because the difference between Ka and Ku surface reflection in precipitation-free regions provides a more stable reference for deriving the differential PIA.

a. Example of estimates over land

As suggested by the discussion in sections 2c and 2d, the derivation of physically consistent combined precipitation estimates proves to be more challenging over land than over ocean surfaces due to multiple-scattering and nonuniform beamfilling effects. In addition to these effects, land surfaces are characterized by high emissivities, which render the low-frequency GMI observations as less reliable indicators of attenuation in the DPR Ku-band radar observations. From this perspective, it is important to closely inspect the estimates derived from the GPM observations of convective storms over land, as such inspection can provide valuable insights regarding the combined algorithm’s strengths and limitations.

Shown in Fig. 2 is the combined algorithm’s estimated surface rain distribution derived from GPM (orbit 2901) observations of a storm near the Kansas–Missouri border. As seen in this figure, embedded within the storm there are several intense convective cells, characterized by precipitation rates above 90 mm h⁻¹. A cross-track section through the northwestern region of the storm, shown in Fig. 3, reveals large attenuation in the observed Ku-band reflectivities. Specifically, the reflectivity profiles at rays 38–41 in the scan shown in Fig. 3 are characterized by extremely large attenuation, while those at rays 22–24 exhibit moderate (yet significant) attenuation. It is apparent from the middle panel of Fig. 3 that the combined algorithm is effective in correcting for attenuation in the observed Ku-band reflectivity observations, and the corresponding estimated precipitation rates (shown in the bottom panel of the figure) are realistic. The Ka-band (35.5 GHz) radar reflectivity observations associated with the Ku-band observations in Fig. 3 are shown in Fig. 4. As apparent in the figure, the attenuation is more severe at the Ka band than at the Ku band. Nevertheless, the Ku+Ka+GMI precipitation estimates are in fairly good agreement with the Ku+GMI precipitation estimates shown in Fig. 3, although systematically lower than the Ku+GMI estimates in the highly convective regions of the scan. The systematic differences between the Ku+GMI and the Ku+Ka+GMI surface precipitation estimates (also apparent in Fig. 2) are suggestive of suboptimal a priori parameterization and/or NUBF-related deficiencies in the forward models and are discussed in the subsequent paragraphs.

• Download Figure
• Download figure as PowerPoint slide

(top) Example of NS-estimated surface precipitation over Kansas and Missouri for GPM orbit 2901 on 2 Sep 2014. The continuous (discontinuous) blue lines show the edges of the NS (MS) swath, while the red line indicates the scan corresponding to the cross section in Fig. 3. (bottom) As in (top), but for the MS-estimated surface precipitation.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

• Download Figure
• Download figure as PowerPoint slide

Observed DPR Ku-band reflectivities, attenuation-corrected Ku-band reflectivites, and estimated precipitation rates for scan 5007 of GPM orbit 2901.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

• Download Figure
• Download figure as PowerPoint slide

As in Fig. 3, but for the Ka-band radar and the MS precipitation estimates.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

In Fig. 5, both the combined algorithm’s estimated Ku-band PIA and the SRT estimates of PIA (Meneghini et al. 2015) are shown. The estimates are shown in (relative) scan-ray rather than latitude–longitude coordinates because this representation provides a more accurate illustration of the similarities and differences of the two types of estimates. Only the SRT estimates flagged as reliable (Meneghini et al. 2015) are shown in the figure. Although a relatively large number of profiles are characterized by unreliable SRT PIA estimates, there is good agreement between the remaining (reliable) SRT PIA estimates and the PIA estimates derived from the combined algorithm. It should be mentioned that although the fraction of unreliable single-frequency SRT PIA estimates is significantly larger over land than over oceans, SRT PIA estimates provide valuable information in the derivation of combined precipitation estimates over land. This is most evident when DPR Ku-band reflectivity observations are affected by large attenuation. In such cases, the ensemble of a priori Ku-band radar–only precipitation estimates exhibits a very broad distribution, and the SRT PIA estimates are crucial for narrowing the initial distribution (reducing uncertainties) and mitigating the potentially large impact of attenuation on retrievals.

• Download Figure
• Download figure as PowerPoint slide

Combined-algorithm-estimated and SRT PIA. The vertical axis indices are scans relative to scan 4887 of orbit 2901. The 49 rays of the NS swath correspond to a distance of about 250 km, while the along-track scans are approximately 4.5 km apart.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

The retrieved (computed from the estimated precipitation and attendant geophysical variables) and observed 89-GHz vertical polarization brightness temperatures associated with the storm depicted in Fig. 2 are shown in Fig. 6. It is evident from the figure that there is good agreement between the retrieved and observed brightness temperatures at this frequency and polarization. However, slight misalignments between the scattering signatures in the retrieved and observed brightness temperatures are also apparent in the figure. This is a consequence of the forward radiative transfer model not accounting for slant-path propagation of radiation through the three-dimensional precipitation field. Although the observation matching strategy of Bauer et al. (1998) is not fully implemented in the current version of the algorithm, investigations carried out so far suggest that the use of the plane-parallel radiative transfer calculations has no negative impact on precipitation estimates. The retrieved 37- and 89-GHz horizontal polarization brightness temperatures exhibit similar agreement with observations.

• Download Figure
• Download figure as PowerPoint slide

As in Fig. 5, but for the 89-GHz vertical polarization brightness temperatures.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

In addition, comparisons of NS and MS mode rain estimates indicate good agreement between the two types of estimates, with a correlation coefficient of 0.95 in stratiform rain and 0.92 in convective rain. Also, the average NS estimates are within a few percent of MS estimates for stratiform rain. The systematic differences between NS and MS estimates are larger for convective rain and strongly depend on the NUBF parameterization. As previously mentioned, multiple scattering and NUBF may significantly impact the accuracy of simulated Ka-band reflectivities and surface returns and, if these effects are not properly described in the algorithm’s forward model, largely erroneous drop size distribution (DSD) adjustments may result. To better understand the impact NUBF may have on MS retrievals, the combined algorithm is run for various prescribed values of the parameter describing the Ku-band reflectivity variability within a DPR-observing volume [i.e., the standard deviation of in section 2d] and retrievals are performed for two scenarios. In the first scenario, the differential SRT PIA estimates (the PIA at Ka band minus the PIA at Ku band) are not used in the retrievals, while in the second scenario, the differential SRT PIA estimates are used. The results are summarized in Fig. 7. Specifically, the average NS- and MS-estimated convective precipitation rates are shown as functions of the prescribed NUBF parameter in the left-hand-side panel of the figure, while the average differential PIA is shown in the right-hand-side panel.

• Download Figure
• Download figure as PowerPoint slide

Sensitivity to NUBF parameterization. The combined algorithm is applied to the portion of orbit 2901 shown in Fig. 1 for different values of the NUBF parameter f with and without incorporating the differential SRT PIA estimates into the objective function J. (left) The NS and MS (with and without differential SRT estimates) domain-average convective precipitation as a function of f. (right) The MS (with and without differential SRT PIA estimates) domain-average convective differential PIA.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

Figure 7 provides good insight into the impact that NUBF assumptions have on combined retrievals. Specifically, as the NUBF factor is increased, the convective NS rain estimates increase as a result of larger specific attenuation being associated with the Ku-band radar observations. The MS estimates that do not incorporate differential SRT PIA information are similar to the NS retrievals although somewhat smaller. This is an indication that Ka-band reflectivity observations do not induce large DSD adjustments, which is a consequence of the fact that the apparent Ka-band signal below the freezing level in convective profiles is largely the result of multiple scattering. On the other hand, the MS estimates that incorporate differential SRT PIA information are significantly different from the NS estimates for low values of the NUBF factor. It may be inferred that even though the Ka-band reflectivity observations computed from the a priori DSD (PSD above the freezing level) are consistent with the actual observations, the computed differential surface PIAs (i.e., the ratios of computed surface returns to clear-sky surface returns) are systematically and significantly larger than actual differential SRT PIA estimates, which results in significant DSD adjustments by the algorithm. As the NUBF parameter is increased, the computed differential surface PIA decreases, even though the actual average attenuation increases. The reason for this is that the structures of the downscaled (subfootprint scale) reflectivity profiles in the NUBF parameterization are vertically coherent, and so increasing the horizontal variability of reflectivity within the footprint, combined with the nonlinear relationship between attenuation and reflectivity, actually leads to overall reduced differential PIA. This effect is explained in detail by Tanelli et al. (2012) and illustrated using high-resolution airborne dual-frequency data. While the NUBF model incorporated into the combined algorithm needs additional evaluation, calibration, and validation, it is more likely that convection over land is characterized by significant field-of-view (FOV) variability, and that the nominal DSD assumptions used in NS retrievals are highly biased. Consequently, the NUBF parameter—that is, the standard deviation of —is set to 0.75 over land, which makes the NS estimates reasonably consistent with both the MS estimates and the independent ground radar rain estimates. Over oceans, the FOV variability is expected to be significantly smaller, and the NUBF parameter is set to 0.25, leading to better and NS and MS agreement over those surfaces.

b. Comparison of observed and computed brightness temperatures

As previously mentioned, a major motivation for the implementation and application of the GPM combined algorithm is the development of a priori databases of candidate solution profiles for the constellation radiometer precipitation estimation algorithms (Hou et al. 2008). Since the GPM combined algorithm provides estimates of all geophysical variables required to compute passive microwave brightness temperatures for any radiometer in the GPM constellation, the use of the combined algorithm precipitation estimates in the radiometer databases will ultimately lead to the development of consistent radiometer-derived precipitation estimates across all constellation members (Hou et al. 2008). The consistency among precipitation estimates from different sensors depends upon, among other factors, the accuracy of the forward radiative transfer calculations, and from this perspective the agreement between GMI observed and simulated brightness temperatures is an important indicator of how well the combined algorithm meets its design requirements.

The overocean joint distributions of GMI retrieved and observed brightness temperatures for September 2014 are shown in Fig. 8. Clear-sky brightness temperatures (exhibiting no precipitation within the FOVs) are not included in the joint distributions. As seen in the figure, there is good agreement between computed and observed brightness temperatures, with the vast majority of joint variables close to the 1:1 line for the first nine channels. The largest and most systematic differences occur at high frequencies (166 GHz and higher). Alternative estimates and brightness temperature calculations using scattering tables representing higher ice particle densities (i.e., 0.2 g cm⁻³ for snow and 0.6 g cm⁻³ for graupel) do not result in better agreement, and the computed brightness temperatures at high frequencies are still systematically warmer than the observed brightness temperatures. This is consistent with the results of Olson et al. (2016), who found that adjusting the snow density in combined retrievals is ineffective in improving the consistency between computed and observed brightness temperatures at 166 GHz when simplistic scattering tables based on ice–air spherical mixtures are used. More accurate scattering tables based on nonspherical ice particles models and discrete dipole approximation (DDA) single-scattering parameter calculations (Kuo et al. 2016) are expected to significantly improve the accuracy of ice-phase precipitation estimates and the consistency between computed and observed high-frequency brightness temperatures (Olson et al. 2016). However, such tables cannot be used in the current version of the combined algorithm, as they are not complete yet; that is, additional DDA calculations are needed for large snow particles and higher frequencies (e.g., 183.3 GHz) to warrant robust estimates. In addition, high-frequency brightness temperatures are sensitive to ice-phase particles at the top of the cloud (Hong et al. 2005) that may not be detected by the DPR. A statistical model is being developed to compensate for the DPR’s limited sensitivity, but it is not included in the current version of the algorithm. Moreover, significant polarization (10–20 K) is observed in brightness temperature observations at 166 GHz in stratiform precipitation; this cannot be handled by a spherical particle model, but it could be reasonably represented through optimization of the extinction coefficient for both polarizations with the DDA calculations.

• Download Figure
• Download figure as PowerPoint slide

Overocean joint distribution of GMI computed and observed brightness temperatures for September 2014. All brightness temperatures are incorporated into the retrievals, but the agreement between computed and observed brightness temperatures is significantly better for frequencies below 166 GHz, as the current scattering lookup table is less deficient at these frequencies.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

The brightness temperatures plotted in Fig. 8 are computed during the retrieval process and saved in the combined algorithm output file. Although these brightness temperatures can be directly used to create a priori precipitation–radiance databases for radiometers with the GMI’s frequency, view angle, and resolution characteristics, the ability to calculate accurate brightness temperatures appropriate for any constellation microwave radiometer using the information in the combined algorithm output file is crucial for achieving the objective of deriving physically consistent estimation algorithms for the GPM radiometer constellation members. To test this ability, a procedure similar to that of Kummerow et al. (2011) is used to recompute GMI brightness temperatures using the information produced by the combined algorithm. Specifically, hydrometeor profiles from the combined algorithm, in combination with associated atmospheric and emissivity information for nonraining areas, are used to compute brightness temperatures at frequencies/polarizations and resolutions of the GMI channels using the slant-path Eddington radiative transfer model (Olson et al. 2001; Munchak and Kummerow 2011). Resulting brightness temperatures are compared to GMI observations. The agreement is similar to that shown Fig. 8, but plots are not reproduced here. In addition, similar to the approach of Kummerow et al. (2011) and consistent with the construction of the radiometer databases, small adjustments are made to the frozen hydrometeor profiles to improve the agreement between observed and computed 89-GHz brightness temperatures. Specifically, graupel is added to convective profiles, and monodisperse cloud ice particles are added to pixels where the simulated high-frequency brightness temperatures are higher than observations. Moreover, emissivity retrievals are performed over land in GMI fields of view that are rain free.

Final computed brightness temperatures are in good agreement with GMI observations, with correlations above 0.9 in all channels with the exception of the highest frequencies, where the correlations are between 0.8 and 0.9. Several relevant statistical measures are provided in Tables 1 and 2. As seen in these tables, the agreement between computed and observed brightness temperatures is better than that reported in Kummerow et al. (2011). Moreover, the database construction process is significantly simpler, as no adjustment of estimated rain profiles is necessary. Shown in Fig. 9 are the observed and simulated-minus-observed average 37-GHz horizontal brightness temperatures for September 2014. Consistent with the results in Tables 1 and 2, the agreement between computed and observed temperatures is good. Nevertheless, regional biases are apparent, with warm biases in the subtropical ocean regions and cold biases outside the tropics in the Southern Hemisphere.

Table 1.

Statistical measures quantifying the agreement between computed and observed brightness temperatures over oceans. Measures include correlation coefficient, bias, and root-mean-square error, and are derived using data from 1 Sep 2014 to 30 Nov 2014. H denotes horizontal; V denotes vertical.

Table 2.

As in Table 1, but over land.

• Download Figure
• Download figure as PowerPoint slide

(top) Observed average 37-GHz horizontal brightness temperatures and (bottom) simulated-minus-observed average 37-GHz horizontal brightness temperatures for September 2014.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

The overland joint distributions of GMI computed and observed brightness temperatures for September 2014 are shown in Fig. 10. Similar to the distributions shown in Fig. 8, clear-sky brightness temperatures are not included. The agreement between computations and observations is generally inferior to that over ocean for low frequencies (below 36.6 GHz), but, as shown in Table 2, the agreement can be improved through more complex overland emissivity estimates and the recomputation of low-frequency brightness temperatures. An overland emissivity estimation module is under development and will be included in future versions of the combined algorithm.

• Download Figure
• Download figure as PowerPoint slide

Overland joint distribution of GMI calculated and observed brightness temperatures.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

c. Zonal-mean rainfall accumulations

Monthly zonal-mean combined algorithm rainfall estimates for the period September 2014–August 2015 are shown in Fig. 11. Both the NS and MS combined algorithm estimates are included in the figure. The MS estimates are potentially more accurate than the NS estimates, as Ka-band DPR observations are used in their derivation, which narrows the range of possible estimates that can be derived from coincident DPR and GMI observations. However, as emphasized earlier, multiple scattering and nonuniform beamfilling can have a significant impact on how Ka-band DPR observations are interpreted, and if these effects are not properly parameterized, highly erroneous precipitation estimates (especially over land, where both multiple-scattering and NUBF effects are more severe than over oceans) can result. It is therefore important to compare the combined NS estimates to the combined MS estimates. As shown in Fig. 11, the NS and MS zonal-mean rainfall estimates are in good agreement with the largest differences within the tropics. It should be noted that when plotted for land and ocean surface types separately, the NS and MS estimates are even more distinct, with the MS estimates systematically lower (by about 10%–20%) than NS estimates over land regions characterized by significant convective precipitation and higher (by about 5%–10%) than NS estimates over mid–high-latitude oceans. The combined precipitation rate estimates are also fairly similar to the Global Precipitation Climatology Project (GPCP) estimates (Huffman et al. 2009). Larger differences between the combined and GPCP estimates are noticeable at high latitudes and could be a consequence of the Ku-band DPR’s 12-dBZ detectability threshold (Kulie and Bennartz 2009). These differences will be investigated in detail, and a radiometer-only retrieval module will be incorporated into the combined algorithm to accommodate high-latitude precipitation seemingly undetected by the DPR but still producing a signal capable of being discerned from the background by GMI.

• Download Figure
• Download figure as PowerPoint slide

Zonal-mean monthly rainfall for September 2014–August 2015. Both the MS and NS estimates are shown. The official GPCP estimates are also shown in the figure.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

d. Comparison to MRMS rain estimates

Combined algorithm estimates of surface precipitation rates are evaluated using the Multi-Radar Multi-Sensor [MRMS; see Zhang et al. (2016)] precipitation rate product. The MRMS product is a quality-controlled, rain gauge–calibrated radar product based upon a merger of National Weather Service dual-polarization S-band (WSR-88D) radar network observations over the contiguous United States and Environment Canada’s single-polarization C-band radar network observations. The MRMS surface precipitation rate product is gridded at 0.01° resolution every 2 min, and local hourly rain gauge data are used to bias correct the product. Also, a rain quality index (RQI) provided with the product is used in the present study to filter regions of radar beam blockage or regions where the radar observations are affected by low melting layers.

The MRMS product is subsequently resampled as in Kirstetter et al. (2012) to match the DPR observations. At each DPR footprint location, the MRMS precipitation rates observed closest in time to the DPR observation are convolved by a Gaussian that approximates the gain function of the DPR. Then, to help suppress random errors due to the space–time collocation of the satellite and ground-based data, the footprint-matched combined algorithm and MRMS precipitation rates are both binned at half-degree resolution and averaged. The half-degree matched data are collected from September 2014 through February 2015, and the resulting combined algorithm NS and MS mode precipitation rates are plotted against the corresponding MRMS precipitation rates in Fig. 12.

• Download Figure
• Download figure as PowerPoint slide

(top) Combined Ku+GMI surface precipitation estimates against MRMS surface precipitation estimates. The combined and the MRMS estimates are instantaneous at half-degree spatial resolutions. Six months’ worth of data—i.e., from September 2014 through February 2015—are used in the evaluation. (bottom) As in (top), but for Ku+Ka+GMI estimates.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1

From Fig. 12, it is evident that the correlations of the NS and MS mode rain estimates with the MRMS precipitation rates are reasonable, and there is a modest increase in correlation (0.80–0.84) of estimates with the addition of the Ka data. This increase in correlation is mainly attributed to a decrease of the more intense, high-biased precipitation rates over land areas. Combined algorithm precipitation rate estimates are mostly within 50% of MRMS precipitation rates at 1 mm h⁻¹, with some increase in relative error as rain intensities increase.

Further insight into the differences between the NS and MS precipitation estimates and the agreement between the combined estimates and the MRMS precipitation estimates may be derived from the analysis of the joint distribution of NS and MS surface precipitations estimates shown in Fig. 13. The joint distribution reveals good agreement between NS and MS estimates for surface precipitation rates below 10 mm h⁻¹, with systematic differences for surface precipitation rates larger than 10 mm h⁻¹. These systematic differences are a consequence of the fact that, in convective precipitation over land, the differential PIAs predicted from NS precipitation estimates are systematically higher than the SRT differential PIA estimates, resulting in systematic downward adjustments of initial NS estimates. The NUBF parameterization described in section 2d and investigated in section 3a improves the consistency between the NS and MS estimates, but it does not fully explain the joint distribution of single- and dual-frequency SRT PIA estimates. As the precipitation increases (i.e., surface rates larger than 50 mm h⁻¹), the dual-frequency SRT PIA estimates become unreliable and are not used in the MS estimates, which explains the good agreement between the NS and MS estimates for surface precipitation rates larger than 50 mm h⁻¹. Figures 12 and 13 suggest that MS estimates are more accurate than NS estimates, but the overestimation in the NS estimates may be a consequence of deficiencies in the NUBF parameterization, which was tuned based on MS estimates.

• Download Figure
• Download figure as PowerPoint slide

Joint distribution of combined Ku+Ka+GMI and Ku+GMI surface precipitation. Data from the retrieved precipitation profiles used in the MRMS evaluation shown in Fig. 12 are used in the derivation of the joint distribution.

Citation: Journal of Atmospheric and Oceanic Technology 33, 10; 10.1175/JTECH-D-16-0019.1