A Worldwide Data Science Investigation of Rainfall

Massimiliano Ignaccolo; Carlo De Michele

doi:10.1175/JHM-D-21-0211.1

Jump to Content

Journal of Hydrometeorology

Abstract
1. Introduction
2. The data scientist parameterization of the raindrop size distribution
3. Globalness and localness of the raindrop size distribution
4. Conclusions
Acknowledgments.
Data availability statement.
REFERENCES

ProCite

RefWorks

Reference Manager

BibTeX

Zotero

EndNote

View raw image

Fig. 1.

Sites’ locations: orange circles. Given the map resolution, many locations cannot be fully resolved: we use a group label and shadow the sites’ locations with blue rectangles. For each label in the figure, we report 1) the number of distinct sites associated with it, 2) the number of datasets derived from the sites, and 3) the datasets number in our database.

View raw image

Fig. 2.

(top) RMI results, (middle) CC results, and (bottom) PCA results for all 166 datasets investigated. Results are grouped in four different columns according to dataset instrumentation.

View raw image

Fig. 3.

Cumulative distribution function (CDF) of the generalized relative dispersion for (top left) σ(μ, γ), (top right) κ(μ, γ), and (bottom left) η(μ, γ). Colors indicate the average ball radius of the points (μ, γ) with a given relative error. (bottom right) The number of datasets having a given couple (μ, γ) as part of their LAF fit support.

View raw image

Fig. 4.

The CDF of relative dispersion given all possible couples of datasets: (top left) ↔ σ(μ, γ) curve, (middle left) ↔ κ(μ, γ) curve, (bottom left) ↔ η(μ, γ) curve. The dataset averaged invariant curve: (top right) ↔ σ(μ, γ), (middle right) ↔ κ(μ, γ), (bottom right) ↔ η(μ, γ).

	All Time	Past Year	Past 30 Days
Abstract Views	441	0	0
Full Text Views	831	564	245
PDF Downloads	415	170	8

The Impacts of Interannual Climate Variability on the Declining Trend in Terrestrial Water Storage over the Tigris–Euphrates River Basin

Authors:

Li-Ling Chang

and

Guo-Yue Niu

Evaluation of Snowfall Retrieval Performance of GPM Constellation Radiometers Relative to Spaceborne Radars

Authors:

Yalei You

George Huffman

Veljko Petkovic

Lisa Milani

John X. Yang

Ardeshir Ebtehaj

Sajad Vahedizade

, and

Guojun Gu

A 440-Year Reconstruction of Heavy Precipitation in California from Blue Oak Tree Rings

Authors:

Ian M. Howard

David W. Stahle

Michael D. Dettinger

Cody Poulsen

F. Martin Ralph

Max C. A. Torbenson

, and

Alexander Gershunov

Quantifying the Role of Internal Climate Variability and Its Translation from Climate Variables to Hydropower Production at Basin Scale in India

Authors:

Divya Upadhyay

Sudhanshu Dixit

, and

Udit Bhatia

Extreme Convective Rainfall and Flooding from Winter Season Extratropical Cyclones in the Mid-Atlantic Region of the United States

Authors:

Yibing Su

James A. Smith

, and

Gabriele Villarini

Editorial Type:: Article

Article Type:: Research Article

A Worldwide Data Science Investigation of Rainfall

Massimiliano Ignaccolo

Massimiliano Ignaccolo ^aSAS Institute, Cary, North Carolina

Search for other papers by Massimiliano Ignaccolo in
Current site
Google Scholar
PubMed

and

Carlo De Michele

Carlo De Michele ^bPolitecnico di Milano, Milano, Italy

Search for other papers by Carlo De Michele in
Current site
Google Scholar
PubMed

Online Publication:: 21 Jul 2022

Print Publication:: 01 Jul 2022

DOI:: https://doi.org/10.1175/JHM-D-21-0211.1

Page(s):: 1193–1203

Received:: 15 Aug 2021

Accepted:: 15 Apr 2022

Published Online:: 21 Jul 2022

Displayed acceptance dates for articles published prior to 2023 are approximate to within a week. If needed, exact acceptance dates can be obtained by emailing amsjol@ametsoc.org.

Download PDF

Full access

Abstract

We perform a worldwide analysis of the raindrop size distribution using 166 disdrometer datasets from 76 distinct sites for a total of 1 527 963 one-minute drop counts, 428 410 two-minute drop counts, and a total of 988 922 720 drops. Following data science tenets, we adopt a functional-agnostic description of the raindrop size distribution. In this way, we uncover the presence of an invariant structure of statistical relationship among the distribution parameters, not depending on location, synoptic origin, or type of disdrometer. The features of this structure are 1) count–shape independence (there is no dependence between the drop count N and the shape of raindrop spectra), 2) mean–skewness prominence (the variability of the shape of raindrop spectra can be fully captured by its mean μ and skewness γ), and 3) mean–skewness invariant parameterization [we derive empirical invariant functional forms expressing all other shape describing parameters in terms of the free parameters (μ, γ)]. The presented analysis reveals the global and local properties of the raindrop size distribution offering a coherent and universally applicable methodology to describe the raindrop size distribution.

Corresponding author: Massimiliano Ignaccolo, massimiliano.ignaccolo@sas.com

Abstract

Corresponding author: Massimiliano Ignaccolo, massimiliano.ignaccolo@sas.com

Keywords: Rainfall; Hydrometeorology; Numerical analysis/modeling; Statistical techniques; Atmosphere

1. Introduction

The statistical description of the raindrop size distribution is of fundamental importance in many sciences and applications, including retrieval of rainfall rates from radar echoes, radio-link propagation, and applications in numerical weather modeling. In literature (Jameson and Kostinski 2001b; Villermaux and Bossa 2009), the term “raindrop size distribution” indicates the concentration per unit volume and diameter

N (D)

(mm⁻¹ m⁻³) as sampled in a short interval of time. The function

N (D)

obeys the following normalization condition:

\int N (D) d D = N_{V}

, where N_V is the number concentration (viz., the number of drops per cubic meter). The mathematical description of the drop size distribution is

N (D) = N_{0} f^{*} (D) = N_{V} f (D) = \frac{N}{A_{m} T υ (D)} p (D) .

(1)

The first equality is, for historical reasons (Marshall and Palmer 1948), the ansatz commonly adopted in literature: N₀ is the intercept parameter [although $N_{0} = N (0) \Leftrightarrow f^{*} (0) = 1$ ], while $f^{*} (D)$ , $\int f^{*} (D) d D \neq 1$ , describes the diameter dependency. The second equality describes an ansatz where the normalization condition of $N (D)$ is made explicit so that the function f(D) is a probability distribution function (pdf): $\int f (D) d D = 1$ . The final equality explains how the drop size distribution can be obtained from ground disdrometer measurements. In the above equation, Np(D)dD is the number of drops with diameter D ∈ [D, D + dD] passing with speed υ(D), in meters per second, through the catchment area A_m, in meters squared, of the instrument in the sampling time interval T, in seconds: p(D) is the pdf of drop diameter as $\int p (D) d D = 1$ , and thus N is the total drop count as reported from the instrument. Finally, Eq. (1) indicates that a statistical description of $N (D)$ can be made through one of these couples: $[N_{0}, f^{*} (D)]$ , [N_V, f(D)], [N, p(D)].

The findings reported in the seminal works of Marshall and Palmer (1948) and Ulbrich (1983) have made, for the last 40 years, the gamma distribution the de facto functional form adopted to describe the variability of $N (D)$ . While recent contributions (Ignaccolo and De Michele 2014; Adirosi et al. 2014) have shown the gamma distribution to be a nonstatistically meaningful fit to $N (D)$ , works investigating alternative functional forms are rare: lognormal (Feingold and Levin 1986; Rosenfeld and Ulbrich 2003; Owolawi 2011), Weibull (Jiang et al. 1997; Sekine et al. 2007; Ishii et al. 2011), Johnson system bounded (SB) (Cugerone and De Michele 2015; D’Adderio et al. 2016; Cugerone and De Michele 2017), and generalized gamma (Lee et al. 2004; Thurai and Bringi 2018). The parameters of the gamma distribution are derived from observations, almost exclusively, by use of the method of moments (MoM) (Tokay and Short 1996a). Table 1 reports the list of acronyms while Table 2 lists the symbols used in the text.

Table 1

List of acronyms used in the text. Acronyms in curly brackets are used only in the supplemental material.

Table 2

List of symbols used in the text. Symbols in curly brackets are used only in the supplemental material. When applicable the dimensions are indicated in square brackets: [.] = dimensionless quantity, [—] = dimension not fixed a priori.

Investigations on the invariant properties of the rainfall phenomenon (e.g., identification of the properties of a particular synoptic regime) proceeds along the lines of 1) comparing the relationship among intercept, scale, and shape parameters of the gamma distributions and 2) studying relationship among couple of bulk variables (hence independently on the particular functional form of the drop size distribution) (Liao et al. 2020). More recently, studies (Dolan et al. 2018; Ignaccolo and De Michele 2020) have been made investigating the correlation/relationship among bulk variables by analyzing datasets from many (≃10) different sites in order to derive general properties.

Here, the keywords dynamical system, big data, and functional agnostic define our approach in investigating the raindrop size distribution with the goals of 1) uncovering, if any, its invariant features and 2) using these features as a foundation for an operative description of the phenomenon. We use “dynamical system” in the sense that rainfall is the manifestation of a physical process whose rules governing its dynamics are the same everywhere (the laws of physics are the same everywhere). We are limited in the investigation of the system to the observation of the system output: raindrops [in this sense, the drop size distribution $N (D)$ is what physicists call an observable]. Among the statistical properties of the observable, there must be imprintings reflecting the rules of the dynamical system: e.g., the results shown in Fig. 3 of Liao et al. (2020) and Fig. 2 of Ignaccolo and De Michele (2020) are what we refer to as an imprinting or invariant property of the phenomenon. We use “big data” because the search for imprintings requires the exploration of a large amount of data from different locations on Earth’s surface: Fig. 1 shows how the 166 datasets considered in this analysis are divided among 76 distinct sites. We use “functional agnostic” because we do not use any particular functional form for the drop size distribution $N (D)$ to avoid spurious results depending on the particular form and/or methodology used to derive the parameters of this given functional form (Smith and Kliche 2005): we let the data speak.

Fig. 1.

Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0211.1

2. The data scientist parameterization of the raindrop size distribution

Hereby, we present the results obtained adopting the flux representation [N, p(D)] (the adoption of the cloud [N_V, f(D)] representation leads to similar results, see online supplemental material).

For a 1-/2-min disdrometer record, the number of drops N [the concentration N_V via Eq. (1)] is not a free parameter but a fixed quantity: whatever the instrument reported. Thus, a functional-agnostic description of the drop size distribution for disdrometer records is a functional-agnostic description of the pdf p(D) [f(D) if we adopt the cloud representation]. The knowledge of the moments of a pdf is equivalent to the knowledge of the pdf (Reed and Simon 1975) itself: a functional-agnostic description of p(D) uses variables which are either the moments themselves or a function of them. The choice of such variables should be informed by the known properties of p(D): 1) approximate exponential decay for large diameters (Marshall and Palmer 1948; Ulbrich 1983); 2) variable location of the mode [although p(D) can be occasional slightly bimodal (Ekerete et al. 2015) or multimodal (Sauvageot and Koffi 2000)]; and 3) presence of a left tail with respect to the mode: small value of p(D) for drop diameters smaller than the mode [although in the case of orographic precipitation the left tail might be greatly reduced or even missing (Martner et al. 2008; Ignaccolo and De Michele 2012)]. Given the first property, one should expect that increasingly higher moments (e.g., $≳ 3$ ) to not add new information on p(D) as they are increasingly dominated by larger and larger values of D, and therefore probe only the right tail of the distribution. Conversely, given properties 2 and 3, one should use variables related to smaller moments of p(D) (e.g., ≤3) as these are the moments mostly affected by variability in location of the mode and spreading of the distribution. Overall, a proper set of chosen variables should have a mixture of smaller and higher moments.

The common practice in data science to describe an unknown pdf is to use its mean μ and standard deviation σ together with the third and fourth standardized statistical moments: namely, skewness γ and kurtosis κ. For brevity, we will refer to all these parameters as statistical moments. Equation (1) of the supplemental material shows how the statistical moments are connected to the moments of a distribution, and that μ, σ, γ, κ are a mixture of smaller and higher moments. In this manuscript, in addition to the first four statistical moments, we will consider a fifth one, η: namely, the fifth standardized central moment. The rationale to include η is that of checking that higher-order moments of p(D) do not carry any additional information, as expected given the exponential right tail of p(D). Overall, our statistical description of the drop size distribution consists in casting the couple [N, p(D)] into the 6-tuple [N, μ, σ, γ, κ, η]. We refer to this methodology as to the data scientist parameterization.

The data scientist parameterization of the raindrop size distribution contains the mixture of small and large moments necessary to adequately describe p(D), given the known property. We caution the reader to not interpret the above statement as a dismissal of other possible parameterizations. We are simply stating that whatever might be the variability of the raindrop size distribution, we are confident that adopting the data scientist parameterization we are not missing any relevant information, which is a prerequisite for any possibly useful parameterization.

3. Globalness and localness of the raindrop size distribution

Using the data scientist parameterization, we investigate (for each dataset) the statistical dependence among the 6-tuple of parameters [N, μ, σ, γ, κ, η]. Our goal is to seek if a more parsimonious representation can be found so that few (e.g., 2 or 3) independent parameters can be used instead of 6 without loss of information: the values of the dependent parameters can be derived from the independent ones within a reasonable margin of error.

The most common metric used to assess statistical dependence is the cross correlation (CC). However, the CC is not robust against outliers, and possible nonlinear relation among parameters. To overcome the limitations of the CC, we additionally use (see supplemental material for details) 1) rescaled mutual information (RMI) and 2) principal component analysis (PCA). The mutual information is a measure (robust against outliers and nonlinear relationships) of the entropy of the joint probability distribution of two random variables and therefore is a direct measure of how well the value of a variable is known given the other. The PCA of a dataset composed of p random variables can be thought of as fitting a p-dimensional ellipsoid to the data, where the magnitude of each axis (principal component) of the ellipsoid represents the variance of the data along the axis. PCA is a dimensionality reduction technique: if k < p principal components account for 90% (this is the typical threshold used in the literature) or more of the total variance then k is approximately the dimensionality of the data (the number of independent variables).

We calculate the RMI and CC values for each possible couple of the 6-tuple of parameters [N, μ, σ, γ, κ, η], and for each dataset. The results are reported in the tessellated top and middle panels of Fig. 2: each row of the panels refers to a different couple of parameters, and each column of the panels to a different dataset (datasets are grouped by disdrometer type). For the PCA, we calculate, for each dataset, the percentage of the total variance (%VE) explained by the first j (j ≤ number of parameters) principal components. As different values of %VE are obtained from different datasets, we report our results by plotting the observed probability density function pdf(%VE) in the bottom panels of Fig. 2 (again results are grouped by disdrometer type). From the joint RMI, CC, and PCA results, we draw the following conclusions.

Fig. 2.

(top) RMI results, (middle) CC results, and (bottom) PCA results for all 166 datasets investigated. Results are grouped in four different columns according to dataset instrumentation.

Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0211.1

a. Count–shape independence

Applying PCA to the 5-tuple [μ, σ, γ, κ, η], we see that roughly the first two principal components explain $⪆ 85 % –95 %$ of the variance (orange curves in bottom row panels of Fig. 2). If we apply PCA to the whole 6-tuple [N, μ, σ, γ, κ, η], a similar threshold in percentage of variance explained (see dark blue dashed curves in bottom panels of Fig. 2) is reached with three components, while the adoption of just the first two components explains ≃70%–80% of the variance observed (light blue dashed curves in bottom panels of Fig. 2). The PCA results together with the RMI and CC results relative to the coupling between N and any of the statistical moments of the pdf p(D) (top panels, RMI, and middle panels, CC, of Fig. 2) indicate that N and p(D) are independent from each other, supporting the conjectures in Jameson and Kostinski (2001a, 2002). Some moderate values ([0.3, 0.5]) are observed, especially for Parsivel datasets, for the CC of N and any of the statistical moments [μ, σ, γ, κ, η]. These values reflect a weak tendency to a linear relationship coupled with large variability or the presence of few outliers: the value of N does not fix the values of any of [μ, σ, γ, κ, η] or vice versa (see Figs. 14–16 and Table 15 in the supplemental material).

b. Mean–skewness prominence

Focusing our attention on the pdf p(D), we notice how the RMI analysis (see top panels of Fig. 2) indicates that 1) the mean μ and the skewness γ are fairly independent from each other, 2) the standard deviation σ is strongly dependent on μ, and 3) the fourth and fifth statistical moments (κ and η) are strongly dependent on γ. The CC mirrors the RMI results with correlation values larger than 0.8 for the couples (μ, σ), (γ, κ), (κ, η). Moreover, as stated in the previous subsection, the PCA analysis indicates that the first two components capture a considerable portion of the total variance of the 5-tuple [μ, σ, γ, κ, η]. Given the RMI and CC results, one could choose one parameter from the couple (μ, σ) together with one parameter from the triplet (γ, κ, η) as a couple of independent parameters to describe the variability of the pdf p(D). We choose the couple (μ, γ) as they are the statistical moments with smaller order, and thus the one whose estimate is less affected by statistical fluctuations. Some moderate values [(0.3, 0.5)] are observed, especially for Parsivel datasets, for the CC between μ and γ. The average value of γ increases as μ increases, however the fluctuations around the mean value are large. So the value of μ does not fix the value of μ and vice versa (see Figs. 12 and 13 in the supplemental material).

The fact that mean and skewness are sufficient to describe the variability of p(D) indicates the existence of functional forms σ(μ, γ), κ(μ, γ), and η(μ, γ) relating the parameters σ, κ, η to the couple (μ, γ). In practice, we consider a neighborhood of the couple (μ, γ) to derive empirical fits to these functional forms by means of the local adaptive fit procedure (LAF); see supplemental material for a detailed description.

As the name suggests, the fitting procedure is local, as we cover the μ–γ plane in a dense grid of points (over 15 000), and it is adaptive, as for each point in the grid, a ball, whose radius depends on the local density of points, is used to perform the estimate. Therefore, each point in the grid has a fit accuracy. The accuracy is defined as the ratio between the semi-interquartile range and the median (the estimated fit) of the values of (σ, κ, η) inside the ball: this notion is a common one adopted in data science to generalize the concept of relative dispersion, the ratio between standard deviation and mean, when the errors might not be normally distributed. Thus, we measure the overall accuracy of the fit by calculating the cumulative distribution function of the relative dispersion v, CDF(v). Heuristically, we expect the relative dispersion to depend not only on the natural variability, but also on the radius of the ball used to perform the fit: large balls → large relative dispersion as the hypothesis of “constancy” of (σ, κ, η) inside the ball might not be valid.

c. In situ: Statistical moments functional dependence on (μ, γ)

The CDFs of the coefficient of relative dispersion v for the parameters σ, κ, η are plotted in Fig. 3 (top panels and bottom-left panel): one curve per dataset. We see how 1) the fit of κ(μ, γ) is the most accurate [ $ν < 0.2 \Rightarrow CDF (ν) > 0.8$ ], followed by σ(μ, γ) [ $ν < 0.2 \Rightarrow CDF (ν) \in (0.6,0.9)$ ] and η(μ, γ) [ $ν < 0.2 \Rightarrow CDF (ν) \in (0.4,0.8)$ ], and 2) the larger the value of ν, the larger is the average radius of the ball used in the estimation (strong effect for σ and κ, moderate effect for η) as shown by the dot colors. In reporting the results obtained for η, we excluded values of η ∈ [−0.2, 0.2] because relative dispersion around null values can be arbitrarily large (v → ∞). We refer the reader to Fig. 8 of the supplemental material for a comparison of the CDF(v) curves with/without the values η ∈ [−0.2, 0.2].

Fig. 3.

Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0211.1

Finally, the LAF fit produces an estimate of (σ, κ, η) for a particular value of (μ, γ) only if the local density of points is large enough (see supplemental material). Thus, for each dataset S, there is a support Ω_S, namely, the set of points (μ, γ) for which an estimate is considered to be statistically meaningful. The bottom-right panel of Fig. 3 shows for each point in the μ–γ plane the number of datasets having that point as part of the support of the LAF fit.

d. Ex situ: Invariance of the statistical moments functional dependence on (μ, γ)

To evaluate the datasets’ similarity of the empirical curves σ(μ, γ), κ(μ, γ), and η(μ, γ), we use different types of relative dispersion ξ between two curves: L2RD, wL2RD, MRD, and wMRD. The L2RD relative dispersion is the ratio between the L² norms of the semidifference between the curves, and the average of the two curves. The MRD relative dispersion is the median value [among all possible couples (μ, γ)] of the ratios between the absolute semidifference between the curves and the average of the two curves. The prefix “w” indicates weighted quantities, the weight being the inverse of the radii ρ(μ, γ) used for the fitting procedure. We refer the reader to the supplemental material for a detailed description of these quantities. The left panels of Fig. 4 report for each row (σ ↔ top, κ ↔ middle, η ↔ bottom) the CDFs of the four relative dispersion (distance) coefficients ξ considered (these curves are obtained calculating the relative dispersion for each possible couple of datasets: 166 × 165/2 = 13 695). The kurtosis κ is the parameter with less dispersion (CDF ≃1 when ξ = 0.1 for all types of dispersion coefficients), while the dispersions of the parameters σ and η are quite similar but larger than that of κ (CDF ≃ 1 when relative dispersion ξ = 0.2 for all types of dispersion coefficients). The right panels of Fig. 4 report the functional forms σ(μ, γ), κ(μ, γ), and η(μ, γ) obtained averaging the functional forms of each single dataset [averages are calculated only for the couples σ(μ, γ) for which values from 20 different datasets are available; see bottom-right panel of Fig. 3]. The functional forms shown in the right panels of Fig. 4 can be thought as approximately invariant functional relations for the standard deviation, kurtosis, and fifth central moment of p(D) as a function of its mean and skewness.

Fig. 4.

Citation: Journal of Hydrometeorology 23, 7; 10.1175/JHM-D-21-0211.1

e. Synoptic considerations

Do the curves σ(μ, γ), κ(μ, γ), and η(μ, γ) depend on synoptic origin? We have knowledge of synoptic origin for a very small part of our data (16 701 out of ∼2 million records), so we cannot provide a definitive answer to this question. However, based on the results depicted in Figs. 3 and 4, we estimate that an eventual synoptic dependence to be weak. The rationale is as follows. Figures 3 and 4 are indicative of the error associated with considering the couple (μ, γ) as the sole responsible of whole variability of p(D) not depending on synoptic origin. This error is due partly to the fact the couple (μ, γ) does not fully represent 100% of the variability of p(D) (natural variability error), and partly to the inevitable errors associated with estimating parameters of a distribution from a finite sample (statistical inference errors). Therefore, if we consider the inhomogeneity of synoptic origin is a third source of error, we can (as a rule of thumb) project its intensity to be ∼1/3 of the total error observed in Figs. 3 and 4, and thus a “small” error. To put it differently, the curves σ(μ, γ), κ(μ, γ), and η(μ, γ) are fairly independent on the location on Earth’s surface.

The synoptic origin comes into play in determining the portion of the volume Nμγ available to a rain event (see Fig. 11 of supplemental material). For example, quite large values of N (>1 500 min⁻¹), large values of μ (>1.5 mm), and moderate skewness (γ ∈ [0, 0, 6]) are quite typical of strong convective precipitation (Ignaccolo and De Michele 2010), and a rarely achieved by stratiform precipitation. Similarly quite large values of N (>1 500 min⁻¹), very small value of μ (<0.5 mm) and large value of skewness (γ > 1.5) are quite possible during orographic precipitation (Martner et al. 2008; Ignaccolo and De Michele 2014), rarely occurring during stratiform, and quite impossible during convective precipitation (e.g., Fig. 11 in supplemental material).

f. Deriving invariant empirical formulas for rain bulk variables

The triplet of parameters (N, μ, γ) virtually exhausts the variability of the raindrop size distribution at any site not depending on difference of climate or synoptic origin. Moreover, the moments of the raindrop size distribution can be obtained as a function of the statistical moments (see supplemental material). Thus, the data scientist representation opens the possibility to write for any bulk rainfall variable X, and any site/dataset S,

X = a_{X} N F_{X, S} (μ, γ)

with a_X being a constant, and the shape factor

F_{X, S}

being fairly independent on the site (

F_{X, S} \to F_{X}

). For example, considering the flux pdf p(D), Eq. (1) to obtain the third moment M₃, and the empirical relation for σ depicted in Fig. 4, we can write the rainfall intensity R as

R = \frac{6 π 10^{- 4}}{A_{m} T} N M_{3} = \frac{6 π 10^{- 4}}{A_{m} T} N {μ^{3} + 3 μ {[σ (μ, γ)]}^{2} + {[σ (μ, γ)]}^{3} γ} = a_{R} N F_{R} (μ, γ) .

(2)

Other bulk variables of interest, such as W (liquid water content) or Z (reflectivity), do not have a simple analytical dependence on the statistical moments of p(D) [because they involve speed weighted moments, see Eq. (1)]. In these cases, one can use the LAF algorithm to derive empirical fit to the shape factors $F_{Z} (μ, γ)$ , and $F_{W} (μ, γ)$ .

g. Data scientist parameterization and method of moments

Some considerations can be drawn on the MoM, as currently employed in the literature, based on the findings of this work which reinforce the criticism already expressed in Smith and Kliche (2005). The MoM consists in calculating the empirical values of three moments $(M_{i}, M_{j}, M_{l})$ of $N (D)$ and inverting the moment equations of the gamma distribution to obtain the intercept, scale, and shape parameters. Note that from Eq. (1), $M_{i} = N_{V} M_{i}$ , where $M_{i}$ is the ith moment of f(D). Most common choices are $(M_{2}, M_{3}, M_{4})$ (Smith 2003), $(M_{3}, M_{4}, M_{6})$ (Ulbrich 1983; Tokay and Short 1996b), and $(M_{2}, M_{4}, M_{6})$ (Ulbrich and Atlas 1998).

Smith and Kliche (2005) caution against using the MoM because of the bias affecting the estimation of the parameters and the fact that the relationship observed between the estimated parameters might be just artifacts of the method. In lieu of the results presented in this manuscript, we can understand the relationships between the parameters (derived from the MoM) as the results of the “percolation” of the invariant statistical relationship structure among the moments of f(D) (see Fig. 3 of the supplemental material) through the distorting prism of the functional form of choice (the gamma distribution in this case), and the particular order of moments (i, j, l) selected. The data scientist parameterization properly uses the invariant statistical dependence structure to derive a set of independent parameters and invariant relationships connecting moments to distribution parameters. The MoM takes this same structure and generates a botched pastiche.

One more issue affects the MoM: it ignores that for disdrometric data the drop count N, and thus the concentration N_V, are not free parameters but are uniquely determined by the instrument measurement. In other words, for disdrometer drop counts $M_{0}$ is fixed. Thus, the zeroth moment should be one of the three moments used, unless one is willing to ignore the empirical evidence reported by the disdrometer that N drops passed through its catchment area, e.g., see Fig. 1 in Ignaccolo and De Michele (2014).

Overall, given all these issues, we think that the MoM may actually limit the progress in understanding and describing the rainfall phenomenon. If one really wants to fit a gamma distribution to disdrometric data, a likelihood method would be a better choice (Smith and Kliche 2005). Having done that, one has also to check if the fit is statistically meaningful: the evidence (Ignaccolo and De Michele 2014; Adirosi et al. 2014) is against it.

h. Imprintings

An indirect confirmation (in addition to evidence reported in this manuscript) of the site independence of the shape factor

F_{X}

might come from results already present in the literature (Ignaccolo and De Michele 2020; Liao et al. 2020). In Liao et al. (2020), the authors find that

R / N_{W} \propto D_{m}^{4.706}

, where R is the rainfall rate, N_w is the generalized intercept parameter (Testud et al. 2001) (N_w is proportional to the ratio between the liquid water content and

D_{m}^{4}

), and D_m is the mean volume diameter. In Ignaccolo and De Michele (2020), the authors show the empirical validity of the relation z = Ar² at all 12 sites investigated, where z is the reflectivity per unit drop, r is the rainfall rate per unit drop, and A is a constant value not depending on site. We can recast both findings in term of relationships between shape factors: in the case of Liao et al. (2020), see Eq. (3), while for Ignaccolo and De Michele (2020), see Eq. (4):

(\begin{matrix} R \propto N F_{R, S} (μ, γ); D_{m} = \frac{F_{M_{4}, S} (μ, γ)}{F_{M_{3}, S} (μ, γ)}; N_{w} \propto N \frac{{[F_{M_{3}, S} (μ, γ)]}^{5}}{{[F_{M_{4}, S} (μ, γ)]}^{4}} \\ \frac{R}{N_{w}} \propto \frac{F_{R, S} (μ, γ)}{F_{M_{3}, S} (μ, γ)} \times {[\frac{F_{M_{4}, S} (μ, γ)}{F_{M_{3}, S} (μ, γ)}]}^{4} ≃ D_{m}^{4.706} \Leftrightarrow \\ \Leftrightarrow \frac{F_{R, S} (μ, γ)}{F_{M_{3}, S} (μ, γ)} \times {[\frac{F_{M_{3}, S} (μ, γ)}{F_{M_{4}, S} (μ, γ)}]}^{0.706} ≃ constant at all sites \end{matrix},

(3)

(\begin{matrix} z = Z / N_{V} = F_{Z, S} (μ, γ) / F_{N_{V}, S} (μ, γ) \\ r^{2} = {[R / N]}^{2} = {[a_{R}]}^{2} \times {[F_{R, S (μ, γ)}]}^{2} \\ \log z = - 2 \log a_{R} + \log {\frac{F_{Z, S} (μ, γ)}{F_{N_{V}, S} (μ, γ) \times {[F_{R, S} (μ, γ)]}^{2}}} + 2 \log r ≃ - 2 \log a_{R} + A + 2 \log r \Leftrightarrow \\ \Leftrightarrow \log {\frac{F_{Z, S} (μ, γ)}{F_{N_{V}, S} (μ, γ) \times {[F_{R, S} (μ, γ)]}^{2}}} ≃ A at all sites \end{matrix} .

(4)

Note that

[(R / N_{w}), D_{m}]

in Liao et al. (2020) and (z, r) in Ignaccolo and De Michele (2020) depend only on p(D) and not on N. Thus, the invariant properties (imprintings) reported in Liao et al. (2020) and Ignaccolo and De Michele (2020) are due to the invariant dependence structure among the moments of p(D). When we write

[(R / N_{w}), D_{m}, z, r]

in terms of the shape functions

F_{X, S} (μ, γ)

, we see that the existence of imprintings requires the constancy of some ratios between different shape functions not depending on site (dataset). Heuristically, the constancy would be quite improbable to hold if the shape factors were site dependent.

4. Conclusions

The analysis presented in this manuscript is unprecedented because of the amount of data analyzed (76 different sites, 166 distinct datasets, 4 types of instruments), and the introduction of a novel parameterization of raindrop size distributions, the data scientist parameterization. This parameterization properly captures the distinction between global and local properties of the raindrop size distribution, making it possible to provide an answer to the following question: What do a rain shower in Milan and one in New York have (or not have) in common? Having an answer to this question is of paramount importance for modeling a highly variable phenomenon like rain.

The data scientist parameterization does not suffer the limitations of the existing parameterization (gamma distribution and method of moments) as 1) it properly uses the correlation structure of the moments of p(D) to discover mean and skewness as the independent parameters regulating the shape of p(D); 2) it does not model p(D) with a functional form, the gamma distribution, which often does not pass goodness of fit tests; and 3) it correctly makes use of empirical evidence, since for disdrometer data the drop count N and (hence) the number concentration N_V are not free numbers to be determined by the fitting procedure like in the case of the method of moments.

Finally, these results do not only hold for the flux representation [N, p(D)] of the raindrop size distribution, but also for the cloud representation [N_V, f(D)] (see supplemental material).

Acknowledgments.

Our data are organized in a database and referred to by their number entry: dataset number (DSN). We refer the reader to the supplemental material for a detailed description of each dataset. We wish to sincerely thank colleagues and institutions who have generously provided disdrometer data: 1) Dr. C. R. Williams, the National Oceanic and Atmospheric Administration (NOAA) and Physical Sciences Laboratory (PSL) for the public availability of the datasets recorded at Alta (DSN*19), Bodega Bay (DSN 20), Erie (DSN 61), Cazadero (DSN 32 to DSN 45), Colfax (DSN 21 to DSN 31), Darwin (DSN 162), Davis (DSN 46), Estacion Obispo (DSN 59 and 60), Fort Ross (DSN 47), Grass Valley (DSN 48), Point Sur (DSN 56 to DSN 58), Rocky Mtn. Arsenal (DSN 63), Spanaway (DSN 12), Westport (DSN 13 to DSN 18); 2) Dr. Ali Tokay and National Aeronautics and Space Administration (NASA) for the public availability of the following datasets: Wallops Precipitation Research Facility (DSN 100 to DSN 120), IFloodS Alabama disdrometer data (DSN 87 and 88), Crystal Face Florida disdrometer data (DSN 165), Kwajalein Disdrometer Data (DSN 1 to DSN 11), LPVex Disdrometer Data (DSN 158 to DSN 160), and as part of the Global Precipitation Measurement (GPM) mission Ground Validation campaigns: HYMEX (DSN 149 to DSN 157), IPHEX (DSN 83 to DSN 86, and DSN 89 to DSN 99), MC3E (DSN 64 to DSN 82); 3) the National Institute of Information and Communications Technology, Japan for the Kashima dataset (DSN 164); 4) the Institute of Observational Research for Global Change together with the Japan Agency for Marine-Earth Science and Technology for the Bukit Koto Tabang dataset (DSN 161); 5) Dr. Dan Brawn for the Hassel dataset (DSN 163); 6) Dr. Martin Hagen of the Institut für Physik der Atmosphäre Deutsches Zentrum für Luft- und Raumfahrt, Wessling, Germany for the Macunaga dataset (DSN 166); 7) Dr. Charles Wrench and the British Atmospheric Data Centre (BADC) for Chilbolton (DSN 121 to DSN 137) and Sparsholt (DSN 138 to DSN 148) data archives. Calculation of the Mutual Information was done adopting the Local Non-uniformity Correction (Kraskov et al. 2004) via the python package NPEET_LNC (https://github.com/BiuBiuBiLL/NPEET_LNC). The Local Adaptive Fit algorithm uses the Python package GriSPy (https://grispy.readthedocs.io/en/latest/) to find nearest neighbors (Chalela et al. 2021). We sincerely thank Prof. Remko Uijlenhoet and two anonymous referees for their comments, and suggestions that helped to improve the manuscript’s presentation.

Data availability statement.

The data are not of the authors’ property, but may be provided by the authors upon request. The data are freely available from the institutions reported in the acknowledgments. Reproducibility: in order to make our analysis easily reproducible, we publicly share all the code used (together with a step by step walkthrough) via GitHub at https://github.com/mignaccolo/datascience_dsd.

REFERENCES

Adirosi, E., E. Gorgucci, L. Baldini, and A. Tokay, 2014: Evaluation of gamma raindrop size distribution assumption through comparison of rain rates of measured and radar-equivalent gamma DSD. J. Appl. Meteor. Climatol., 53, 1618–1635, https://doi.org/10.1175/JAMC-D-13-0150.1.
- Crossref
- Search Google Scholar
- Export Citation
Chalela, M., E. Sillero, L. Pereyra, M. A. Garcia, J. B. Cabral, M. Lares, and M. Merchán, 2021: GriSPy: APython package for fixed-radius nearest neighbors search. Astron. Comput., 34, 100443, https://doi.org/10.1016/j.ascom.2020.100443.
- Crossref
- Search Google Scholar
- Export Citation
Cugerone, K., and C. De Michele, 2015: Johnson SB as general functional form for raindrop size distribution. Water Resour. Res., 51, 6276–6289, https://doi.org/10.1002/2014WR016484.
- Crossref
- Search Google Scholar
- Export Citation
Cugerone, K., and C. De Michele, 2017: Investigating raindrop size distributions in the (L-)skewness–(L-)kurtosis plane. Quart. J. Roy. Meteor. Soc., 143, 1303–1312, https://doi.org/10.1002/qj.3005.
- Crossref
- Search Google Scholar
- Export Citation
D’Adderio, L. P., K. Cugerone, F. Porcù, C. De Michele, and A. Tokay, 2016: Capabilities of the Johnson SB distribution in estimating rain variables. Adv. Water Resour., 97, 241–250, https://doi.org/10.1016/j.advwatres.2016.09.017.
- Crossref
- Search Google Scholar
- Export Citation
Dolan, B., B. Fuchs, S. A. Rutledge, E. A. Barnes, and E. J. Thompson, 2018: Primary modes of global drop size distributions. J. Atmos. Sci., 75, 1453–1476, https://doi.org/10.1175/JAS-D-17-0242.1.
- Crossref
- Search Google Scholar
- Export Citation
Ekerete, K.-M. E., F. H. Hunt, J. L. Jeffery, and I. E. Otung, 2015: Modeling rainfall drop size distribution in southern England using a Gaussian mixture model. Radio Sci., 50, 876–885, https://doi.org/10.1002/2015RS005674.
- Crossref
- Search Google Scholar
- Export Citation
Feingold, G., and Z. Levin, 1986: The lognormal fit to raindrop spectra from frontal convective clouds in Israel. J. Climate Appl. Meteor., 25, 1346–1363, https://doi.org/10.1175/1520-0450(1986)025<1346:TLFTRS>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Ignaccolo, M., and C. De Michele, 2010: Statistical collapse of stratiform and convective drop diameter distributions at the ground. Geophys. Res. Lett., 37, L24402, https://doi.org/10.1029/2010GL045454.
- Crossref
- Export Citation
Ignaccolo, M., and C. De Michele, 2012: Skewness as measure of the invariance of instantaneous renormalized drop diameter distributions – Part 2: Orographic precipitation. Hydrol. Earth Syst. Sci., 16, 329–343, https://doi.org/10.5194/hess-16-329-2012.
- Crossref
- Search Google Scholar
- Export Citation
Ignaccolo, M., and C. De Michele, 2014: Phase space parameterization of rain: The inadequacy of gamma distribution. J. Appl. Meteor. Climatol., 53, 548–562, https://doi.org/10.1175/JAMC-D-13-050.1.
- Crossref
- Search Google Scholar
- Export Citation
Ignaccolo, M., and C. De Michele, 2020: One, no one, and one hundred thousand: The paradigm of the Z–R relationship. J. Hydrometeor., 21, 1161–1169, https://doi.org/10.1175/JHM-D-19-0177.1.
- Crossref
- Search Google Scholar
- Export Citation
Ishii, S., S. Sayama, and T. Kamei, 2011: Measurement of rain attenuation in terahertz wave range. Wireless Eng. Technol., 2, 119–124, https://doi.org/10.4236/wet.2011.23017.
- Search Google Scholar
- Export Citation
Jameson, A. R., and A. B. Kostinski, 2001a: Reconsideration of the physical and empirical origins of Z–R relations in radar meteorology. Quart. J. Roy. Meteor. Soc., 127, 517–538, https://doi.org/10.1256/smsqj.57213.
- Crossref
- Search Google Scholar
- Export Citation
Jameson, A. R., and A. B. Kostinski, 2001b: What is a raindrop size distribution? Bull. Amer. Meteor. Soc., 82, 1169–1177, https://doi.org/10.1175/1520-0477(2001)082<1169:WIARSD>2.3.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Jameson, A. R., and A. B. Kostinski, 2002: Spurious power-law relations among rainfall and radar parameters. Quart. J. Roy. Meteor. Soc., 128, 2045–2058, https://doi.org/10.1256/003590002320603520.
- Crossref
- Search Google Scholar
- Export Citation
Jiang, H., M. Sano, and M. Sekine, 1997: Weibull raindrop-size distribution and its application to rain attenuation. IEEE Proc., 144, 197–200, https://doi.org/10.1049/ip-map:19971193.
- Crossref
- Export Citation
Kraskov, A., H. Stögbauer, and P. Grassberger, 2004: Estimating mutual information. Phys. Rev. E, 69, 066138, https://doi.org/10.1103/PhysRevE.69.066138.
- Crossref
- Export Citation
Lee, G. W., I. Zawadzki, W. Szyrmer, and D. Sempere-Torres, 2004: A general approach to double-moment normalization of drop size distributions. J. Appl. Meteor., 43, 264–281, https://doi.org/10.1175/1520-0450(2004)043<0264:AGATDN>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Liao, L., R. Meneghini, T. Iguchi, and A. Tokay, 2020: Characteristics of DSD bulk parameters: Implication for radar rain retrieval. Atmosphere, 11, 670, https://doi.org/10.3390/atmos11060670.
- Search Google Scholar
- Export Citation
Marshall, J., and W. Palmer, 1948: The distribution of raindrops with size. J. Meteor., 5, 165–166, https://doi.org/10.1175/1520-0469(1948)005<0165:TDORWS>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Martner, B. E., S. E. Yuter, A. B. White, S. Y. Matrosov, D. E. Kingsmill, and F. M. Ralph, 2008: Raindrop size distributions and rain characteristics in California coastal rainfall for periods with and without a radar bright band. J. Hydrometeor., 9, 408–425, https://doi.org/10.1175/2007JHM924.1.
- Crossref
- Search Google Scholar
- Export Citation
Owolawi, P., 2011: Raindrop size distribution model for the prediction of rain attenuation in Durban. PIERS Proc., 42, 1068–1075, https://doi.org/10.2528/PIERM15031706.
- Crossref
- Export Citation
Reed, M., and B. Simon, 1975: Fourier Analysis, Self-Adjointness. Academic Press, 384 pp.
- Crossref
- Export Citation
Rosenfeld, D., and C. Ulbrich, 2003: Cloud microphysical properties, processes, and rainfall estimation opportunities. Radar and Atmospheric Science: A Collection of Essays in Honor of David Atlas, Meteor. Monogr., No. 52, Amer. Meteor. Soc., 237–258, https://doi.org/10.1175/0065-9401(2003)030<0237:CMPPAR>2.0.CO;2.
- Crossref
- Export Citation
Sauvageot, H., and M. Koffi, 2000: Multimodal raindrop size distributions. J. Atmos. Sci., 57, 2480–2492, https://doi.org/10.1175/1520-0469(2000)057<2480:MRSD>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Sekine, M., S. Ishii, S. I. Hwang, and S. Sayama, 2007: Weibull raindrop-size distribution and its application to rain attenuation from 30 GHz to 1000 GHz. Int. J. Infrared Millimeter Waves, 28, 383–392, https://doi.org/10.1007/s10762-007-9221-0.
- Crossref
- Search Google Scholar
- Export Citation
Smith, P. L., 2003: Raindrop size distributions: Exponential or gamma—Does the difference matter? J. Appl. Meteor., 42, 1031–1034, https://doi.org/10.1175/1520-0450(2003)042<1031:RSDEOG>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Smith, P. L., and D. V. Kliche, 2005: The bias in moment estimators for parameters of drop size distribution functions: Sampling from exponential distributions. J. Appl. Meteor., 44, 1195–1205, https://doi.org/10.1175/JAM2258.1.
- Crossref
- Search Google Scholar
- Export Citation
Testud, J., S. Oury, R. A. Black, P. Amayenc, and X. Dou, 2001: The concept of ‘normalized’ distribution to describe raindrop spectra: A tool for cloud physics and cloud remote sensing. J. Appl. Meteor., 40, 1118–1140, https://doi.org/10.1175/1520-0450(2001)040<1118:TCONDT>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Thurai, M., and V. N. Bringi, 2018: Application of the generalized gamma model to represent the full rain drop size distribution spectra. J. Appl. Meteor. Climatol., 57, 1197–1210, https://doi.org/10.1175/JAMC-D-17-0235.1.
- Crossref
- Search Google Scholar
- Export Citation
Tokay, A., and D. A. Short, 1996a: Evidence from tropical raindrop spectra of the origin of rain from stratiform versus convective clouds. J. Appl. Meteor., 35, 355–371, https://doi.org/10.1175/1520-0450(1996)035<0355:EFTRSO>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Tokay, A., and D. A. Short, 1996b: Evidence from tropical raindrop spectra of the origin of rain from stratiform versus convective clouds. J. Appl. Meteor., 35, 355–371, https://doi.org/10.1175/1520-0450(1996)035<0355:EFTRSO>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Ulbrich, C. W., 1983: Natural variations in the analytical form of the raindrop size distribution. J. Climate Appl. Meteor., 22, 1764–1775, https://doi.org/10.1175/1520-0450(1983)022<1764:NVITAF>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Ulbrich, C. W., and D. Atlas, 1998: Rainfall microphysics and radar properties: Analysis methods for drop size spectra. J. Appl. Meteor., 37, 912–923, https://doi.org/10.1175/1520-0450(1998)037<0912:RMARPA>2.0.CO;2.
- Crossref
- Search Google Scholar
- Export Citation
Villermaux, E., and B. Bossa, 2009: Single-drop fragmentation determines size distribution of raindrops. Nat. Phys., 5, 697–702, https://doi.org/10.1038/nphys1340.
- Crossref
- Search Google Scholar
- Export Citation

Supplementary Materials

Share Link

Copy this link, or click below to email it to a friend

Email this content

or copy the link directly:

https://journals.ametsoc.org/view/journals/hydr/23/7/JHM-D-21-0211.1.xml

Link copied successfully

Journal of Hydrometeorology

Abstract
1. Introduction
2. The data scientist parameterization of the raindrop size distribution
3. Globalness and localness of the raindrop size distribution
4. Conclusions
Acknowledgments.
Data availability statement.
REFERENCES

ProCite

RefWorks

Reference Manager

BibTeX

Zotero

EndNote

View raw image

Fig. 1.

View raw image

Fig. 2.

(top) RMI results, (middle) CC results, and (bottom) PCA results for all 166 datasets investigated. Results are grouped in four different columns according to dataset instrumentation.