1. Introduction

In this study, we bring to the attention of the hydrometeorological community a little known fact that estimation of the correlation coefficient from a bivariate sample of skewed data results in biased estimates of correlation coefficient. Our focus is on rainfall data, in particular at smaller temporal and spatial scales on which both rainfall rate and rainfall accumulation exhibit considerable skewness. The statisticians have known about this bias problem for years, but numerous works on rainfall analysis neglected the finding. This neglect prompted us to report on it. The issue is important because the bias may be as high as 0.1 or more, painting an overoptimistic picture of model and algorithm performances.

Correlation coefficient is a measure of linear dependency between a pair of random variables. Correlation often is used as a measure of agreement between rainfall estimates based on different sensors; radar–rain gauge comparison studies are a prime example. Correlation of rainfall at two or more gauging stations is used in data quality-control applications, rainfall prediction, and climate characterizations. Correlation has been widely used to characterize the complex spatial structure of rainfall patterns (e.g., Huff and Shipp 1969; Huff 1970; Sharon 1972; Zawadzki 1973). In addition, spatial correlation is a significant component of hydrological design, in particular, design of rainfall networks (e.g., Bras and Rodriguez-Iturbe 1993). Evaluation of the errors of area-averaged rainfall accumulations is largely determined by information provided by the spatial correlation function in the rain fields (Morrissey et al. 1995; Bras and Rodriguez-Iturbe 1993).

For the problem discussed herein, the variables are concurrent rainfall quantities at two locations. In the past, the sample Pearson product–moment correlation coefficient r has been used commonly to estimate the population coefficient ρ [see Eqs. (1) and (2) in section 3]. Many researchers argued that estimation of the correlation coefficient suffers from several shortcomings. For example, in his evaluation of climate model performance, Willmott (1984) illustrated that r is insensitive to additive and proportional differences that may exist between the two variables, which causes a problem if r is used as a performance measure. Like many other statistical measures, r is much more influenced by outliers than by observations near the mean (Moore 1991). Legates and Davis (1997) and Legates and McCabe (1999) discussed the effect of extreme values on the evaluation of goodness-of-fit measures of hydrologic and hydroclimatic models. They illustrated that the existence of extreme events will lead to an inflated estimate of r that may obscure the true relationship over most of the range of observations. Based on an idealized analysis of radar and rain gauge comparisons, Kessler and Neas (1994) showed that the magnitude of the correlation coefficient tends to increase with the range over which the variables are measured.

Researchers also noted other problems with estimating correlation. Cressie (1993) argued that, as compared with other spatial association measures such as the variogram, the correlation function is not preferred given that bias and trend contamination errors heavily affect its estimates. A similar limitation of the use of the correlation coefficient arises when temporal records of observations are used to estimate the spatial correlation structure. Gunst (1995) illustrated that, with meteorological data, correlations calculated from time series overestimate spatial correlations when temporal trends and autocorrelations exist in the observations.

Statistical literature provides evidence that the source of the problems is the underlying distribution of the variables under consideration. In the case of bivariate normal distribution, the distribution of r is known well (e.g., chapter 32 of Johnson et al. 1995; chapter 6 of Stuart and Ord 1987) and can be used to form confidence intervals or to perform significance tests. However, in the case of nonnormal bivariate populations, little is known about the distribution of r. Kowalski (1972) illustrated that the distribution of r may be very sensitive to nonnormality in the data and suggested that normal correlation analysis should be limited to situations in which the variables are nearly normal. Hutchinson (1997) suggested that, for very skewed distributions, the Pearson sample correlation coefficient is a poor estimator of the population value.

Although several studies have investigated the robustness of the correlation coefficient estimation in case of nonnormal data, statistical literature does not provide a practical methodology to determine the distribution of the sample correlation. Contradicting results are often reported about the robustness of the Pearson sample correlation estimator; see Johnson et al. (1995, chapter 32) for an extensive review. Tedious numerical analyses such as cross-cumulant and asymptotic expansions are required (Nakagawa and Niki 1992; Lai et al. 1999) to address the issue of uncertainties in the correlation estimation.

In the hydrological literature, few studies addressed the correlation estimation problems. Stedinger (1981) presented an extensive investigation of alternative estimators of the correlation coefficients of streamflows. He used a simulation-based analysis to show that, in the case of skewed data such as streamflows, better estimates are possible if logarithmically transformed variables are used to estimate the true correlations. Such arguments indicate the need for a similar practice in the case of rainfall data, which are characterized by even higher skewness levels. Our review of the rainfall literature indicated that, despite the extensive use of correlation-based analyses, it is unclear to what extent researchers are aware of these limitations. Correlation estimates for rainfall patterns observed via a variety of sensors are often reported without discussing relevant issues such as the distribution of the analyzed data, sample size effects, and uncertainty bounds of the computed coefficients (e.g., Moszkowicz 2000; May and Julien 1998). However, in recent studies, Young et al. (2000) and Krajewski et al. (2000) discussed the significant scatter in the computed coefficients and the inability of the normal theory (Stuart and Ord 1987) to explain them; these authors indicated the need to analyze and to explain the sources of such scatter.

In the current study, we investigate and illustrate the relevance of such problems to the correlation estimation in rainfall data. We follow the approach presented by Stedinger (1981) and examine it using rainfall data, which, in contrast to streamflows, has a mixed-type distribution because of the intermittent behavior of the precipitation process. We have organized this paper as follows. We begin with a brief description of the experimental rainfall datasets used throughout the study to illustrate our points. We then briefly discuss the simulation-based analysis with which we investigated the robustness of the correlation coefficient. After that, we examine the performance of the transformation-based estimator and apply it to the experimental data. We close with discussion and conclusions.

2. Data description

To illustrate our investigation, we use rainfall data from a high-density network (cluster) of rain gauges. The cluster was deployed near Melbourne, Florida, as a part of the Texas and Florida Underflights field campaign (TEFLUN-B), in August and September of 1998, in support of the ground validation of the Tropical Rainfall Measuring Mission (TRMM; Simpson et al. 1996). The network consisted of 14 8-in. tipping-bucket (TB) gauges. The separation distances between gauges ranged from a few hundred meters up to about 8 km, covering spatial scales typical for resolution of radar-rainfall products used for hydrologic applications. Figure 1 shows a layout of this experimental network. Within this network, another cluster of rain gauges, operated by The University of Iowa, was also deployed in which three dual-gauge platforms (Krajewski et al. 1998) were set with gauge-separation distances of 1, 10, and 100 m. All the gauges in the cluster have high sampling resolution in time (5 or 10 s), and a bucket size that corresponds to 0.25 mm. Herein we focus on the data collected during the 2-month period from August through September of 1998. Despite this relatively short time, the total amount of accumulated rainfall was very high, ranging from 350 to 450 mm, depending on the rain gauge. Using a linear interpolation scheme (Habib et al. 2001), we converted the raw rain gauge data into estimates of 1-min rain rates. These fine-resolution estimates can be used to construct rain-rate averages at any desired time interval for further analysis. For discussion of accuracy of TB-based point rainfall estimates, please see Humphrey et al. (1997) and Habib et al. (2001).

To the extent possible, we performed careful quality control of the gauge records to detect instances of mechanical or electronic failures of the kind that often affect the TB-type gauges. This check was facilitated by the closely located gauges. Thus, we believe that the behavior of the analyses reported in subsequent sections of this paper is not contaminated by the occurrence of outliers in the dataset.

3. Empirical properties of correlation coefficient in rainfall

Let (X, Y) denote a pair of rainfall processes observed at two gauge locations. The population correlation coefficient can be defined as

View Expanded

The following Pearson product–moment sample coefficient r, derived from N observations of the pair (X, Y), is usually used as an estimate of ρ(X, Y):

View Expanded

where the overbars indicate average values over the sample of size N. In this section, we elaborate on some of the problems associated with the estimation of correlation from rainfall data. Effect of sample size was addressed by Berndtsson (1987). Using daily rainfall data, he showed that scatter in correlation variation with distance increases significantly when the sample size is reduced. We performed similar investigations using the TEFLUN-B gauges and computed the correlation coefficients for the entire sample and for smaller-size subsamples at a timescale of 15 min. We observed that when the sample size is reduced, the scatter becomes more pronounced and the estimated values become more biased upward. Application of the commonly used “Fisher z transformation” (Fisher 1921; Stuart and Ord 1987) does not explain the observed scatter. This fact indicates that the correlation estimation in rainfall is highly sensitive to the sample size and cannot be treated with the normal-based traditional statistical methods. To investigate this issue empirically, extremely long records of homogeneous rainfall observations are needed. The question, “How long do the datasets need to be to obtain stable statistics?,” cannot be answered with the usual setups of rainfall-measuring networks. Thus, we will investigate this issue using simulation.

4. Monte Carlo simulations

To investigate the performance of the sample correlation coefficient with respect to the underlying distribution of the rainfall data, we performed a simulation-based analysis. Rain rate has a mixed-type distribution, because there is a positive probability of no rain caused by the intermittent behavior of the rainfall process. One accordingly, has to decide on the continuous-distribution component of rain rate. Kedem et al. (1990) presented a brief review of several models that have been suggested in the literature and indicated that distributions such as gamma, hyperbolic, and lognormal are all possible candidates. Houze and Cheng (1977), Kedem and Chiu (1987), Shimizu (1993), and Ciach and Krajewski (1999) all adopted the lognormal distribution. In addition, researchers often use logarithmic transformation of rainfall data (e.g., McDonald 1960; Kitchen and Blackall 1992; Smith and Krajewski 1991; Anagnostou et al. 1999). Considering that there is no general agreement within the research community on a specific form of the distribution (Kedem et al. 1990), we chose to use the lognormal model as the underlying distribution of rain rate in the simulation analysis. In the application described in next sections, we will evaluate the agreement of actual rainfall data with the lognormal distribution.

a. Bivariate intermittent lognormal distribution

A realistic distribution that takes into account the intermittent nature of rainfall was developed by Shimizu (1993) as a part of a two-point rainfall model and serves as a useful tool for our simulations. The distribution is explained in detail in Shimizu's paper, and we only present a brief description here. The distribution allows representation of situations with zero rainfall at either of the two observing stations. Four cases have to be considered: zero rain at both stations (0, 0), zero at one station and rain at the other (x*, 0) and (0, y*), and rain at both stations (x, y). For a sample size of N, the four cases will have sample sizes of n₀, n₁, n₂, and n₃, where n₀ + n₁ + n₂ + n₃ = N. The distribution accordingly can be described fully by the following parameters: δ₀, δ₁, δ₂, δ₃, μ^*₁, σ^*₁, μ^*₂, σ^*₂, μ₁, μ₂, σ₁, σ₂, and ρ_N. Parameters δ_i (i = 0, 1, 2, 3) are the probabilities of rainfall occurrence for each case. Parameters μ^*_i and σ^*_i (i = 1, 2) are conditional means and standard deviations for cases n₁ and n₂. Parameters μ_i and σ_i (i = 1, 2), and ρ_N are, respectively, conditional means, standard deviations, and correlation coefficient for case n₃. We remind the reader that this set of parameters corresponds to normal distributions of the logarithmic transformation of the variables X and Y. Note also that the set of standard deviations σ_i and σ^*_i of the normal variables determines the skewness of the distribution; that is, higher standard deviations result in a more skewed lognormal distribution. For specific values of these parameters, simulation of such a distribution is possible through assigning the desired probabilities δ_i (i = 0, 1, 2, 3), sampling from a normal distribution for both of the cases (x*, 0) and (0, y*) and sampling from a bivariate normal distribution for the case (x, y). Appropriate exponential transformations then can be applied to get the desired intermittent bivariate lognormal sample.

Following the method of Shimizu (1993), population means and variances of the lognormal variables are expressed as follows:

View Expanded

After substituting these expressions into Eq. (1), an expression of the population correlation coefficient for the mixed-lognormal variables can be obtained.

5. Application to TEFLUN-B dataset

In this section, we illustrate the application of the correlation estimation procedure using the rainfall dataset that was collected during August–September of 1998 as part of the TEFLUN-B field experiment (see section 2). We used a timescale of 15 min, which is of interest in radar-rainfall studies. Also, the shorter the timescale is, the more skewed the rainfall data are; thus it is easier to illustrate the problem at hand. First, we test the assumption of lognormality for this dataset. Testing for lognormality is equivalent to testing for normality after performing log-transformation of the rainfall values. We did this for each gauge separately and then for the bivariate sample of every pair of gauges. We applied two statistical tests to the transformed data: the Shapiro–Wilk W test to univariate samples and the Mardia's multivariate skewness and kurtosis-based test to the bivariate sample. Details of these tests are available in most statistics books, and here we discuss only the results. Using the data collected by the 14-gauge TEFLUN-B cluster, we performed the tests for each pair of rain gauges with the subsets that correspond to cases (x^*, 0), (0, y*), and (x, y). Both sets of tests accepted the lognormality hypothesis for almost all the cases. For illustration, we show in Fig. 5 an example of the lognormal distribution fitted to the time series of one of the gauges. Despite slight deviations at both tails, the plot indicates a fair agreement between the experimental data and the assumed distribution within 95% confidence bounds.

After the tests, we proceeded with the correlation estimation based on the assumption of mixed bivariate lognormal distribution. Following the procedure described in the previous section, we estimated the parameters δ₀, δ₁, δ₂, δ₃, μ^*₁, μ^*₂, μ₁, μ₂, σ^*₁, σ^*₂, σ₁, σ₂, and ρ_N for each pair of gauges. Then, for each set of these estimates, we generated 1000 samples of the same size from the mixed-lognormal distribution corresponding to the same parameter values. We then used each of these samples to estimate the correlation coefficient using the transformation-based estimator described in the previous section. Standard errors for each set of 1000 estimated correlation coefficients were computed to obtain the uncertainty bounds of the estimates. In Fig. 6a, averages of the 1000 estimated correlation coefficients are plotted together with error bars of 2 standard errors. These error bars are comparable to the standard errors in Fig. 4, except for the cases in which the data sample was atypically small because of the gaps in the observations. For comparison, we also plotted the correlation values obtained directly from the standard Pearson product–moment estimator Eq. (2) as shown in Fig. 6b. In addition, we fitted a functional approximation [Eq. (3)] to the estimated correlation points using the Levenberg–Marquardt algorithm (Press et al. 1988). The algorithm minimizes the sum of squares of differences between data points and the desired model but also takes into account the data uncertainty information which is available only for the transformation-based estimates. As expected, transformation-based correlation estimates are systematically lower than the straightforward Pearson product–moment values. The bias of the latter is on the order of 0.1 over most of the distance range in Fig. 6. We also examined the fitting quality using the sum of squares of differences between the analytical function and the corresponding estimated coefficients. This examination revealed that the scatter in the case of the transformation-based estimator is relatively less than that of the Pearson's estimates. It is obvious now that the estimated uncertainty bounds can explain the amount of scatter associated with the correlation estimates. These effects are significant and, without the bias correction, any further correlation-based analysis, such as rain gauge representativeness errors and the area-point differences, are systematically underestimated.

To complete our illustration, we comment on the possible effect of autocorrelation in the used samples. The mixed bivariate model we used in the simulations ignored the serial correlation that may exist in the data. To address this issue, we computed temporal autocorrelations for different time lags, and we noticed low levels of autocorrelation. The autocorrelation value at one time step of 15-min lag is about 0.5. It becomes negligible (∼0.15) for 30-min lag. To address the effect of the serial correlation, we performed a simple experiment in which subsamples were constructed by skipping every other pair of observation in the original time series of the gauges. By doing this, we remove any significant levels of autocorrelation in the samples. We recomputed the correlation coefficients using the transformation-based estimator applied to the constructed subsamples and compared them with the full sample results. The results indicated that the recomputed coefficients show comparable levels to the previously obtained values using the full sample. This result indicates that serial correlation has negligible effect on the reported estimates. We refer the reader to further analysis in Stedinger (1981) on how to deal with this issue in data that have nonnegligible autocorrelation levels.

6. Summary and discussion

In this study, we examined the problems of estimation of correlation coefficients in rainfall data. Researchers need to pay more attention to estimation problems such as effects of sample size, departures from normal distribution, and sensitivity to extreme rain rates. The large deviation of rainfall data from the normal distribution invalidates the usually made assumption of normal distribution of sample correlation. We performed simulation analyses based on bivariate mixed-lognormal distributions to examine the distribution of the sample correlation in conditions closer to reality. The results indicate that the sample correlation coefficient suffers from large bias and estimation variance, which increase with the increase of the skewness of the distribution. We also found that the bias and variance maintain high values, even with large sample sizes not typically available in most archives. We examined the performance of an alternative transformation-based estimator suggested earlier by Stedinger (1981) for streamflows. This estimator provided unbiased and more stable estimates of the correlation coefficient. We illustrated an application of this alternative estimation method to 2-month observations from a high-density rain gauge network deployed in Florida. In the future, it should be worthwhile to examine estimation procedures applied to longer periods of rainfall observation not currently available. Other factors, such as different rainfall regimes and seasonal effects, need to be addressed so that these effects of nonstationarity can be distinguished from the sampling- and distribution-related variability.

There are several other relevant issues with regard to the alternative estimator. First, we consider the procedure we proposed merely as an example of approaches that are needed for improved estimation of correlation over the traditional Pearson's formula. We do not advocate the above procedure as a full remedy of the problem. A major assumption—supported to some degree by the literature and statistical testing—was the underlying bivariate model for two-station rainfall observations. However, other bivariate distributions may offer better fit and/or result in better estimation properties of the sought correlation. For example, the described two-parameter lognormal transformation estimator can be easily formulated to suit a three-parameter case (Stedinger 1981). Another candidate bivariate model (Moran 1969) may be based on the gamma distribution that is known to provide good fit to rainfall observations. Furthermore, testing the validity of the assumed distribution is always required before applying the proposed estimator to a specific rainfall dataset. The robustness of the proposed estimator with respect to the model choice is another important issue.

For some applications it may be appropriate to use nonparametric correlation methods, such as Kendall's τ and Spearman's rank correlations (e.g., Conover 1971; Kendall 1970; Gibbons and Chakraborti 1992). Similar to other nonparametric statistics, rank correlation is free from distributional assumptions and is less subject to some of the limitations we discussed earlier. However, we did not consider this class of estimators given that our focus is on the population correlation coefficient as defined in Eq. (1).