• Adderley, E. E., 1961: Non-parametric methods of analysis applied to large-scale cloud-seeding experiments. J. Meteor.,18, 692–694.

  • ——, and S. Twomey, 1958: An experiment on artificial stimulation of rainfall in the Snowy Mountains of Australia. Tellus,10, 275–280.

  • Changnon, S. A., F. A. Huff, and C. F. Hsu, 1979: On the need to evaluate operational weather modification projects. Bull. Amer. Meteor. Soc.,60, 770–774.

  • Davis, A. W., 1979: On certain ratio statistics in weather modification experiments. Technometrics,21, 283–289.

  • Dennis, A. S., J. R. Miller, D. E. Cain, and R. L. Schwaller, 1975: Evaluation by Monte Carlo tests of effects of cloud seeding on growing season rainfall in North Dakota. J. Appl. Meteor.,14, 959–969.

  • Flueck, J. A., 1976: Evaluation of operational weather modification projects. J. Wea. Modif.,8, 42–56.

  • ——, and B. S. Holland, 1976: Ratio estimators and some inherent problems in their utilization. J. Appl. Meteor.,15, 535–543.

  • Gabriel, K. R., 1988: More about ratio statistics, especially for weather experiments. Tech. Rep. 88/07, University of Rochester, Rochester, NY, 31 pp. [Available from Department of Biostatistics, University of Rochester, Rochester, NY 14642.].

  • ——, 1991: The use of ratio statistics in rain experiments, with special reference to Puglia and Sardinia. TECNAGRO Report, Corso V. Emanuele 173, TECNAGRO, Rome, Italy, 42 pp.

  • ——, and P. Feder, 1969: On the distribution of statistics suitable for evaluating rainfall stimulation experiments. Technometrics,11, 149–160.

  • ——, and D. Rosenfeld, 1990: The second Israeli rainfall stimulation experiment: Analysis of precipitation on both targets. J. Appl. Meteor.,29, 1055–1067.

  • Gagin, A., and J. Neumann, 1981: The second Israeli randomized cloud seeding experiment. J. Appl. Meteor.,20, 1301–1311.

  • Kapoor, R. K., K. Krishna, U. S. De, K. G. S. Nair, I. C. Talwar, S. K. Sharma, and B. V. R. Murthy, 1973: Results of operational cloud seeding experiment over Rihand catchment in northeast India. Indian J. Meteor. Geophys.,25, 379–384.

  • List, R., K. R. Gabriel, B. A. Silverman, Z. Levin, and T. Karakostas, 1999: The rain enhancement experiment in Puglia, Italy: Statistical evaluation. J. Appl. Meteor.,38, 281–289.

  • Mielke, P. W., and J. A. Flueck, 1976: Distributions of ratios for some selected bivariate probability distributions. Proc. Social Statistics Section, Washington, DC, American Statistical Association, 608–618.

  • ——, and J. G. Medina, 1983: A new covariate ratio procedure for estimating treatment differences with application to Climax I and II experiments. J. Climate Appl. Meteor.,22, 1290–1295.

  • ——, J. S. Williams, and S. C. Wu, 1977: Covariance analysis technique based on bivariate log-normal distribution with weather modification applications. J. Appl. Meteor.,16, 183–187.

  • Miller, A. J., D. E. Shaw, and L. C. Veitch, 1979: Analyzing the results of a cloud-seeding experiment in Tasmania. Comm. Stat. Theory Meth.,A8, 1017–1047.

  • Moran, P. A. P., 1959a: The methodology of rainmaking experiments. Rev. Int. Stat. Inst.,38, 105–119.

  • ——, 1959b: The power of a cross-over test for the artificial stimulation of rain. Aust. J. Stat.,1, 47–52.

  • Neumann, J., and E. Shimbursky, 1972: On the distribution of a ratio of interest in single-area cloud-seeding experiments. J. Appl. Meteor.,11, 370–375.

  • Nirel, R., 1994: Bootstrap confidence intervals for the estimation of seeding effect in an operational period. Water-Related Issues, V. Barnett and K. F. Turkman, Eds., Vol. 2, Statistics for the Environment, Wiley, 109–123.

  • Petrondas, D. A., 1983: Two topics in permutation (re-randomization) inference: Double ratio statistics and multiple comparisons. Ph. D. thesis, University of Rochester, 365 pp. [Available from University of Rochester, Rochester, NY 14627.].

  • Siliceo, P., and P. A. Mosino, 1963: Twelve years of cloud seeding in the Necaxa watershed, Mexico. J. Appl. Meteor.,2, 311–323.

  • Smith, E. J., L. G. Veitch, D. E. Shaw, and A. J. Miller, 1979: A cloud-seeding experiment in Tasmania 1964–1970. J. Appl. Meteor.,18, 804–815.

  • Wurtele, Z. S., 1971: Analysis of the Israeli cloud-seeding experiment by means of concomitant variables. J. Appl. Meteor.,10, 1185–1192.

  • View in gallery

    A schema of correlations between targets and controls τ, between targets and their controls κ, and between targets and other controls γ.

  • View in gallery

    Variances of different ratio statistics, general and schematic.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 137 137 3
PDF Downloads 24 24 1

Ratio Statistics for Randomized Experiments in Precipitation Stimulation

View More View Less
  • 1 Department of Statistics, University of Rochester, Rochester, New York
© Get Permissions
Full access

Abstract

A variety of ratio statistics has been used in the design and evaluation of weather modification experiments and their significance has usually been estimated by rerandomization. These ratios, and especially their logarithms, are asymptotically normal with null expectations and variances that can be readily calculated. This paper reviews and generalizes several useful ratio statistics and derives their variances. The variances presented here should make it easier for users of these ratios statistics in large experiments, 100 or more units, to assess significance without going through a large number of rerandomizations. It also shows how these formulas can be used to evaluate power and the required sample sizes. Some illustrations from Israel and from Puglia, Italy, are given.

Corresponding author address: Dr. K. Ruben Gabriel, Department of Statistics, University of Rochester, Rochester, NY 14627.

krg1@db1.cc.rochester.edu

Abstract

A variety of ratio statistics has been used in the design and evaluation of weather modification experiments and their significance has usually been estimated by rerandomization. These ratios, and especially their logarithms, are asymptotically normal with null expectations and variances that can be readily calculated. This paper reviews and generalizes several useful ratio statistics and derives their variances. The variances presented here should make it easier for users of these ratios statistics in large experiments, 100 or more units, to assess significance without going through a large number of rerandomizations. It also shows how these formulas can be used to evaluate power and the required sample sizes. Some illustrations from Israel and from Puglia, Italy, are given.

Corresponding author address: Dr. K. Ruben Gabriel, Department of Statistics, University of Rochester, Rochester, NY 14627.

krg1@db1.cc.rochester.edu

Introduction

This paper is concerned with the randomization distributions of ratio statistics and with the means and standard errors of the asymptotic distributions of these ratios and their logarithms. These distributions are important for the correct application and interpretation of this type of statistics, which have found wide use in the analysis and design of randomized weather modification experiments (Adderley and Twomey 1958; Siliceo and Mosino 1963; Kapoor et al. 1973; Dennis et al. 1975; Smith et al. 1979; Changnon 1979; Miller et al. 1979; Gagin and Neumann 1981; Gabriel and Rosenfeld 1990; List et al. 1999). Nowadays it is not difficult to generate a very large number of rerandomizations of an experiment and obtain a reliable estimate of its P value without recourse to asymptotic formulas. That approach, however, becomes exorbitant and yields little understanding of the design of an experiment and of alternative methods of analysis. These purposes are better served by application of asymptotic variance formulas to hypothetical models of experiments and will be illustrated in section 5.

Ratio statistics relate totals, or means, of precipitation at different times and in different areas—under cloud seeding or in the absence of seeding, on a seeding target or a control area, or on two alternate targets. The totals cumulate all seeded or unseeded experimental units at all stations of a particular area. Since daily, or even weekly, precipitations tend to have highly skewed distributions, this emphasizes the occasions of extreme precipitation at the expense of the many small or moderate precipitations. Ratio methods may therefore be suspected of being nonrobust and prone to the effect of outliers. Replacing them by statistics that censor outliers, or substitute ranks for amounts of precipitation, would increase robustness but do so at the risk of ignoring the large effects of seeding. Not enough is known about cloud seeding to guarantee that rejecting extreme precipitation events might not result in ignoring its largest effects. Current models of the effect of seeding on precipitation are quite speculative, and so it makes sense to proceed conservatively by analyzing the totals of precipitation, especially as that is what weather modifiers usually aim to affect.

This paper does not argue that ratio statistics are best but presents tools for making correct inferences about them, given that they have been much used and are likely to continue being used.

Two experimental designs are discussed: one has a single target, which is randomly seeded or not seeded;the other uses two targets, which are seeded alternately according to random allocation. Either of these designs may be analyzed with or without covariates. In practice, covariates usually consist of precipitation on nearby control areas that are always unseeded. The targets and controls are assumed to be located so that seeding will affect only the target to which seeding is assigned; in practice, there often are serious concerns about contamination (see, e.g., Wurtele 1971), but these are not addressed here. The units of experimentation are assumed to be well defined at the time of seeding assignment and thus usually consist of fixed time intervals, referred to here as “days.” More meteorologically meaningful units, such as individual clouds, would surely enhance the sensitivity of experiments, but defining them (especially the times of their beginning and end) in real time at the moment of seeding assignment, and/or of finding alternate targets and controls, has proved to be quite elusive. Finally, the length of an experiment, as denoted by the number n of days, is assumed to be fixed at the outset of experimentation.

A variety of papers have been concerned with randomization distributions of ratio statistics (Adderley 1961; Moran 1959a,b; Gabriel and Feder 1969; Neumann and Shimbursky 1972; Davis 1979; Petrondas 1983; Gabriel 1988, 1991; Nirel 1994). Other publications have discussed ratio statistics under parametric assumptions (Flueck 1976; Flueck and Holland 1976; Mielke and Flueck 1976; Mielke et al. 1977) and other alternative techniques to avoid normality assumptions (Mielke and Medina 1983).

Definitions and notation

Consider an experiment carried out on days i = 1, . . . , n according to random allocation variables θ1, . . . , θn, such that, on day i, if θi = 1, then one target Y, say, is seeded, whereas if θi = 0, then that target is not seeded. In addition, in an alternating targets experiment there is another target X, say, which is seeded if θi = 0 and not seeded when θi = 1. The allocation variables θ1, . . . , θn are independent, and each has probability ½ for each of θi = 1 and θi = 0.

Precipitations on day i are written yi and xi for targets Y and X, respectively. Data may also be available on covariates, with zi,j denoting the ith day’s observation of covariate Zj, j = 1, . . . , k, and a constant is introduced by setting i,0 ≡ 1 for dummy covariate Z0.

In such experiments, the only part that is considered random is the collection θi, . . . , θn of allocation variables, whereas the statistical analysis is conditioned on the observations yi and xi of precipitation and zi,j of covariates, which are taken as given. This approach is particularly useful for testing a hypothesis that observations are independent of allocation, that is, that seeding has no effect on precipitation.

The means of the precipitation and covariates, computed for all days, irrespective of seeding, are
i1520-0450-38-3-290-eq1
the variances are
i1520-0450-38-3-290-eq2
and the covariances are
i1520-0450-38-3-290-eq3
Precipitation and covariates can be expressed relative to their means during the experiment, and in this paper variance formulas are developed in terms of relative variation. The daily relative values are defined as
i1520-0450-38-3-290-eq4
The means of these relative values are all 1; their variances are the relative variances
i1520-0450-38-3-290-eq5
and their covariances are the relative covariances
i1520-0450-38-3-290-eq6
Note that Y, X, and j are coefficients of variation. Note also that since Z0 has been defined as a dummy covariate, it follows that s20, sY0, sX0, and sj0, as well as 20, Y0, X0, and j0, are all zero.

Ratio statistics are calculated from partial sums of precipitation and covariate(s), one sum for days seeded on Y, another for days not seeded on Y. These sums are obtained by multiplying the summands by θi or 1 − θi, respectively, as shown in Table 1.

Single target experiments

The design

An experiment may have a single target Y, which is seeded or not seeded according to random allocations θ1, . . . , θn, which are independent with Pr(θi = 1) = Pr(θi = 0) = ½, i = 1, . . . , n. The response is target precipitation yi on day i. Covariate observations zi,j, j = 1, . . . , k may also be available; they usually consist of precipitation on upwind control areas.

The single ratio and its approximation

The evaluation of a single target experiment without covariates is naturally carried out by comparing target precipitation means during seeding and without seeding. Thus, an estimate of the proportional effect of seeding is the single ratio (SR)
i1520-0450-38-3-290-e1
Since this ratio is the same for relative values as for actual values, the two are used interchangeably.

Statistical inferences on seeding effect can be based on the SRY statistic by relating it to its distribution over all possible random seeding allocations, conditional on the y’s. This randomization distribution derives from the randomness of the θ’s.

The asymptotic distribution of SRY can be obtained from the statistic’s logarithm
i1520-0450-38-3-290-e2
by Taylor approximation (see the appendix). The distributions of lnSRY and SRY are then found to be asymptotically normal with expectations (E)
YY
and variances
i1520-0450-38-3-290-e3b
as follows from (A4) in the appendix (see also Neumann and Shimbursky 1972). The asymptotic convergence would be faster for lnSRY than for SRY.

For experiments with large n, one may use these approximate moments for testing significance and setting confidence bounds on the proportional effect of seeding. For intermediate-sized experiments one should preferably use lnSRY for this purpose rather than SRY itself. Petrondas (1981) carried out a detailed study of such ratio statistics, noting that “for large datasets . . . Taylor approximation of the variance is adequate. For experiments with fewer than one hundred data points . . . normal approximation is adequate for the central part of the permutation distribution of the logarithm” (of the ratio). He further concluded that “for larger experiments” (over 100 data points) “the normal distribution is adequate” (for the entire permutation distribution).

The double ratio and its approximation

When observations zi,j on covariates are also available, they may be used to improve the precision of estimating the target’s precipitation by calculating the double ratio (DR)
i1520-0450-38-3-290-e4
for a given set of coefficients a0, a1, . . . , ak. This compares the SR on Y with a given weighted combination of similar ratios
i1520-0450-38-3-290-e5
for the covariates. The DR was originally used for a single covariate Z1 consisting of precipitation on a control area associated with the target (Adderley and Twomey 1958). For that case, a1 = 1 and a0 = a2 = · · · = ak = 0), so one may write it DR(0,1).
Conditional on y’s and z’s, the randomization distribution of lnDR(a), as well as DR(a), can then be shown (see the appendix, section b) to be asymptotically normal with expectations
(a)(a)
and variances
i1520-0450-38-3-290-e6b
as follows from (A8a), (A8b) in the appendix. In particular, for the original double ratio DR(0,1), this becomes
i1520-0450-38-3-290-e7
If the coefficients a0, a1, . . . , ak are chosen to minimize the sum of squared residuals
i1520-0450-38-3-290-e8
they become the intercept α0 and coefficients α1, . . . , αk of the multiple regression of the differences i − 1 onto the covariates i,j − 1, j = 1, . . . , k, all relative to their means. In particular, α0 = 0 since both the i − 1 and the i,j − 1, j = 1, . . . , k, sum to zero. The resulting statistic is the regression ratio (RR):
i1520-0450-38-3-290-e9
which seems to have been used first by Smith et al. (1979). The same argument as above (see the appendix, section c) shows the randomization distributions of lnRR and RR, conditional on given y’s and z’s, to be asymptotically normal with expectations
and variances
i1520-0450-38-3-290-e10b
as follows from (A10) in the appendix, where R2Y:1,...,k is the multiple correlation of the differences i − 1 onto the covariates i,j − 1, j = 1, . . . , k.

Note that the fit, and the correlation, of the above regression is the same as that of the regression of the precipitation yi onto the covariates zi,j themselves (not relative). Hence, one could use the latter multiple regression instead of the former to obtain the same result.

Alternating targets experiments

The design

An experiment may have targets Y and X, which are seeded on alternate days where the alternation is according to random allocation θ1, . . . , θn, which are independent with Pr(θi = 1) = Pr(θi = 0) = ½, i = 1, . . . , n. The responses are precipitation yi and xi on alternate targets Y and X on day i. Covariate observations zi,j, j = 1, . . . , k may also be available. The covariates usually are precipitations on several control areas, some upwind of target Y, others upwind of target X.

The root double ratio and its approximation

The evaluation of an alternating targets experiment without covariates is naturally carried out by a comparison of the precipitation means on each target with and without seeding. Thus, estimates of the proportional effect of seeding could use the single ratio SRY and/or the reciprocal of the analogous single ratio
i1520-0450-38-3-290-e11
based on target X precipitation. Note that a positive effect of seeding would increase the reciprocal of SRX, not SRX itself. This is because in the alternating targets experiment X is seeded when θi = 0 and is not seeded when θi = 1. A summary statistic for evaluating the overall effect of seeding on the two targets therefore has to average SRY and 1/SRX. This is done by the root double ratio (RDR; Moran 1959a,b):
i1520-0450-38-3-290-e12
which is the geometric mean of SRY and 1/SRX.

Comparing this with the single area experiment’s DR(0,1), one notes it differs by introducing the square root. This is justified because DR(0,1) compares SRY to the SR1 for control Z1, which could not be affected by seeding, whereas RDR compares SRY to SRX for the alternate target X, which could be reciprocally affected by seeding; the square root then averages over the effects on the two targets.

Statistical inferences on average seeding effect can be based on the RDR statistic by relating it to its distribution over all possible random seeding allocations, conditional on the given x’s and y’s. Asymptotically, lnRDR and RDR are, conditional on the x’s and y’s, asymptotically normal with
and
i1520-0450-38-3-290-e13b
as follows from (A14a), (A14b) in the appendix (Gabriel and Feder 1969; Petrondas 1981).

The root quadruple ratio and its approximation

When observations zi,j on concomitant variables Zj, j = 1, . . . , k are available, one may use them to adjust the comparison of the two targets’ precipitation by calculating the root quadruple ratio (RQR):
i1520-0450-38-3-290-e14
for any given set of coefficients b0, b1, . . . , bk. This compares the RDR, which is the square root of SRY divided by SRX, with the square root given weighted combinations of single ratios SRZj, j = 0, 1, . . . , k. For suitable covariates that may improve the precision of the estimate of the proportional seeding effect. The RQR was originally used for a pair of covariates Z1, Z2, which represented precipitation on control areas upwind of targets Y and X, respectively. For that case, b1 = 1, b2 = −1 and b0 = b3 = · · · = bk = 0 and the statistic becomes RQR(0,1,−1) (Gabriel and Rosenfeld 1990).
Again, conditional on given y’s, x’s, and z’s, the randomization distribution of lnRQR(b), as well as of RQR(b), is shown (see the appendix, section e) to be asymptotically normal with expectations
(b)(b)
and variances
i1520-0450-38-3-290-e15b
as follows from (A18a), (A18b) in the appendix. For the original use of this statistic, this becomes
i1520-0450-38-3-290-e16

The root regression ratio and its approximation

If the coefficients b0, b1, . . . , bk of the RQR are chosen to minimize the sum of squared residuals Σ [ii − Σjbj(i,j − 1)]2, they become the intercept β0 = 0 and coefficients β1, . . . , βk of the multiple regression of the differences ii onto the concomitants i,j, j = 1, . . . , k, all relative to their means. The resulting statistic is the root regression ratio (RRR)
i1520-0450-38-3-290-e17
where the exponents β1, . . . , βk are the intercept and coefficients of the regression of the differences ii onto the concomitants i,j, j = 1, . . . , k, all relative to their means.
The computation of the fractional powers can be effected by calculating this statistic through logarithms as
i1520-0450-38-3-290-e18
which yields
i1520-0450-38-3-290-e19
Once more, the randomization distributions of lnRRR and RRR are asymptotically normal with expectations
and variances
i1520-0450-38-3-290-e20b
as follows from (A19a), (A19b) of the appendix, where R2YX:1,...,k is the multiple correlation of the differences (ii) with the covariates i,j − 1, j = 1, . . . , k.

Note that the fit and the correlation of the above regression is the same as that of the regression of the difference of the targets’ relative precipitation ii onto the covariates zi,j themselves (not relative). Hence, one could use the latter multiple regression instead of the former to obtain the same result.

A schematic comparison of designs and ratio statistics

The schema

The formulas developed in the preceding sections allow calculation of the variances of different ratios when the relative variances and covariances of all variables are known. Given, however, that certain common patterns have been observed in many precipitation experiments, we propose a schema of variances and correlations that roughly represent observed patterns. We then derive the ratios’ variances from this schema and make some fairly general comparisons of various ratio statistics in terms of their variances and thus in terms of the length of experimentation they require.

The schema, Fig. 1, concerns one or two targets, Y or Y and X, with corresponding control(s) Z1, or Z1 and Z2, respectively. (Illustrations follow in Tables 2 and 3 below.) It thus includes both one- and two-target designs. It assumes the relative variances of all variables are the same, that is, 2Y = 2X = 21 = 22 (=σ̃2, say). It also assumes the correlation pattern to be the same for both targets and their respective controls. In particular, the two targets are assumed to have the same correlations with their respective controls, rY1 = rX2 (=κ, say). Also, the correlation of each target with the other’s control is assumed to be the same for both targets, rY2 = rX1 (=γ, say). And finally, the correlation between the targets is assumed to be the same as that between the controls, rYX = r12 (=τ, say). These are not unreasonable approximations to observed patterns, and they lead to an κτγ ≥ 0 order since controls are chosen to be as highly correlated with their targets as possible, more so than alternating targets are to each other. The cross correlation of each target to the other’s control is obviously least.

Some examples

Examples of alternating targets experiments with controls are Israel II (Gagin and Neumann 1981; Gabriel and Rosenfeld 1990) and Puglia (List et al. 1999) with the correlations and coefficients of variation as shown in Table 2.

For the Israeli data, the coefficient of variation is remarkably constant at around 1.25. The target–control correlation average is 0.753, whereas the between-target and between-control correlation average is 0.738, which is a bit lower, as postulated under the schema. The lowest are the target–other control correlations, which average 0.665, again as expected. The two correlations of each kind do not closely correspond to the schema, that is, 0.866 is not close to 0.641, 0.668 to 0.808, and 0.745 to 0.584, so the approximation to the Israeli situation is rough.

For the Puglia data, Table 3, the coefficient of variation is fairly steady around 1.60. The target–control correlation average is 0.832, the between-target and between-control correlation average is 0.508, which is a bit lower than postulated under the schema. Least is the target–other control correlation with an average of 0.498, again as expected. Again, the two correlations of each kind are not very close together, so the schema is likely to be a rough approximation of the Puglia situation.

Relative variability, that is, the coefficient of variation, is somewhat higher in Puglia than in Israel; however, once adjusted for the control covariate, it does not vary widely, which suggests that important features of experiments in these two areas may not be that different.

Some comparisons and practical conclusions

The variances of the various ratios for single and alternating targets experiments are shown in Fig. 2, both as derived generally in sections 3 and 4, and as calculated under the simplified schema.

In a single target experiment the double ratio DR(0,1) is seen to be more precise, that is, to have smaller variance than the single ratio SRY, provided the target–control correlation κ exceeds 0.5. The table also shows RR to be the more precise than either, so it should always be used, unless no suitable control is available and only SRY is available.

In an alternating targets experiment the root quadruple ratio RQR(0,1,−1) is similarly seen to be more precise than the root double ratio RDR, provided RYX:1,2 = (κγ)/(1 − τ) exceeds 0.5. The RRR is, however, found to be the most precise and should always be used if controls are available; in the absence of controls, only the RDR is available.

Comparing an alternating targets experiment with a single target experiment, both without controls, the former’s RDR will have a smaller variance, (1/n)σ̃22(1 − τ), than the latter’s SRY, which has variance (4/n)σ̃2. Indeed, if the correlation between the targets is positive, surely a reasonable assumption, the variance of the former will be less than one-half that of the latter, that is, (1 − τ)/2 of the latter, so it will need less than half as many observations for the same precision. The advantage of the alternating targets design evidently gets more pronounced as the correlation τ between the targets increases.

When the two targets each have a control, the advantage of the alternating targets experiment depends on the ratio
i1520-0450-38-3-290-eq7
and is even greater than in the absence of controls. It increases with the multiple correlation R2YX:1,2 = ((κγ)/(1 − τ))2 of Y–X onto Z1 and Z2, which is particularly large if the correlation κ of each target with its control is much larger than its correlation γ with the other control.

Additional issues

Power and length of experimentation

The above expressions for the expectations and variances of ratio statistics have been computed under the null hypothesis that seeding has no effect and are conditional on the first two moments of target precipitation and the covariates. They can be used to assess the powers of planned experiments and the length of experimentation needed for a given power.

For any ratio statistic R, if seeding were to increase the precipitation every day by a proportion δ, this would add ln(1 + δ) to lnR. The expectation of lnR would increase by ln(1 + δ), whereas the variance would remain the same. Standard formulas for power can therefore be applied provided one has estimates of the relative variation of precipitation and the covariates.

For any ratio statistic R, the variance can be written Var[lnR] ≈ (1/n)ψ2R, for a suitable ψR, see Fig. 2. From standard statistical methodology, the power of an experiment at significance level α which lasts n days then is
i1520-0450-38-3-290-e21
where Φ{ } is the integral and zα is the αth quantile of the standard normal distribution. Conversely, a power of 1 − β at level α is attained by an experiment whose number of days is
i1520-0450-38-3-290-e22
In particular, for a one target experiment using the single ratio, ψSRY = 42Y according to Fig. 2, the power of an experiment of n days is
i1520-0450-38-3-290-e23
and a power of 1 − β at level α is attained by an experiment whose number of days is
i1520-0450-38-3-290-e24
In Puglia, for example, an experiment on the single target of Bari, for which Y = 1.447, 90% power (β = 0.10, so zβ = 1.28), to discover a 15% (δ = 0.15) increase of precipitation at a significance level of 5% (α = 0.05, so zα = 1.645) would have required some
i1520-0450-38-3-290-eq8
days of experimentation. [This differs slightly from an earlier calculation by Gabriel (1991), who used the approximate distribution of SRY rather than of lnSRY that is used here. It is not completely clear whether the latter is preferable for calculating power.]
Similarly, for an alternating targets experiment using RRR, ψRRR = (2Y − 2YX + 2X)(1 − R2YX:1,...,k), and the power of an experiment of n days’ length is
i1520-0450-38-3-290-e25
and a power of 1 − β at level α is attained by an experiment whose number of days is
i1520-0450-38-3-290-e26
Again, in Puglia, an alternating Bari–Canosa experiment with a control covariate for each target for which 2Y − 2rYXYX + 2X = (1.447)2 − 2(0.6073)(1.447)(1.631) + (1.631)2 = 1.8874, and R2YX:1,2 = 0.4880, 90% power, to discover a 15% increase of precipitation at a significance level of 5% would have required some
i1520-0450-38-3-290-eq9
days of experimentation. [This differs slightly from an earlier calculation (Gabriel 1991) using the approximate distribution of RRR as well as more detailed covariates.]

Comparisons of different experimental designs

In choosing a particular experimental design, one may be guided by the sample sizes required for each of them to attain a certain power. To illustrate, consider a single target Y experiment that is to be evaluated by DR(0,1) relative to a control area Z1 = X. Compare this with an alternating Y and X target experiment to be evaluated by RDR. From Fig. 2,
i1520-0450-38-3-290-e27
and
i1520-0450-38-3-290-e28
Since Z1 = X, and thus κ = τ, these expressions differ only by a factor of 4 and the single target experiment would need four times as many days as the alternating targets experiment to achieve the same power.

Actual experiments commonly have controls that are more highly correlated with a target than with an alternate target, that is, κ > τ, so the ratio of Var[lnRDR] to Var[lnDR(0,1)] is larger than 0.25. For the Bari target in the Puglia experiment, τ = 0.61 and κ = 0.77 and so the variance of the alternating targets experiment’s RDR would have been 0.42 of the variance of a single target experiment’s DR(0,1).

Some general comparisons

Under the simplified schema of section 5, the number of days required to attain a power of 1 − β at level α would be
i1520-0450-38-3-290-e29
for a one target experiment using SRY and
i1520-0450-38-3-290-e30
using RR. For an alternating targets experiment the corresponding required sample sizes would be
i1520-0450-38-3-290-e31
for the RDR and
i1520-0450-38-3-290-e32
for RRR.

Taking rough average values from the Israeli II and Puglia experiments, we would put κ = 0.8, τ = 0.65, and γ = 0.6, and coefficients of variation σ̃ = 1.25 for Israel and σ̃ = 1.45 for Puglia, noting also that their average numbers of experimental days per year were 65 and 75, respectively. The resulting approximate required sample sizes then are presented in Table 4.

The data from both experimental regions show that single area experiments on days are impractical; certainly so if there is no control and in practice also in the presence of a control, since it is not realistic to design experiments of 15–20 years’ duration. The data from both experiments also show that alternating targets experiments require some 5–6 years for reasonable power if a 15% seeding effect is tested for and that the use of control covariates does not significantly reduce the time requirement. In the author’s experience, those seem to be the order of magnitude for experiments of this type in general.

Single area experiments could therefore be viable only with some drastic change from the designs studied above. Thus, better covariates might be helpful if they can be found, as would better focused experimental units that reduced variability, and, of course, seeding techniques that justified testing for bigger effects than the ones hoped for from AgI seeding. Without any of these, alternate targeting is the only game in town.

Acknowledgments

Dr. B. A. Silverman’s generous advice and critical encouragement of the formulation of this paper are greatly appreciated.

REFERENCES

  • Adderley, E. E., 1961: Non-parametric methods of analysis applied to large-scale cloud-seeding experiments. J. Meteor.,18, 692–694.

  • ——, and S. Twomey, 1958: An experiment on artificial stimulation of rainfall in the Snowy Mountains of Australia. Tellus,10, 275–280.

  • Changnon, S. A., F. A. Huff, and C. F. Hsu, 1979: On the need to evaluate operational weather modification projects. Bull. Amer. Meteor. Soc.,60, 770–774.

  • Davis, A. W., 1979: On certain ratio statistics in weather modification experiments. Technometrics,21, 283–289.

  • Dennis, A. S., J. R. Miller, D. E. Cain, and R. L. Schwaller, 1975: Evaluation by Monte Carlo tests of effects of cloud seeding on growing season rainfall in North Dakota. J. Appl. Meteor.,14, 959–969.

  • Flueck, J. A., 1976: Evaluation of operational weather modification projects. J. Wea. Modif.,8, 42–56.

  • ——, and B. S. Holland, 1976: Ratio estimators and some inherent problems in their utilization. J. Appl. Meteor.,15, 535–543.

  • Gabriel, K. R., 1988: More about ratio statistics, especially for weather experiments. Tech. Rep. 88/07, University of Rochester, Rochester, NY, 31 pp. [Available from Department of Biostatistics, University of Rochester, Rochester, NY 14642.].

  • ——, 1991: The use of ratio statistics in rain experiments, with special reference to Puglia and Sardinia. TECNAGRO Report, Corso V. Emanuele 173, TECNAGRO, Rome, Italy, 42 pp.

  • ——, and P. Feder, 1969: On the distribution of statistics suitable for evaluating rainfall stimulation experiments. Technometrics,11, 149–160.

  • ——, and D. Rosenfeld, 1990: The second Israeli rainfall stimulation experiment: Analysis of precipitation on both targets. J. Appl. Meteor.,29, 1055–1067.

  • Gagin, A., and J. Neumann, 1981: The second Israeli randomized cloud seeding experiment. J. Appl. Meteor.,20, 1301–1311.

  • Kapoor, R. K., K. Krishna, U. S. De, K. G. S. Nair, I. C. Talwar, S. K. Sharma, and B. V. R. Murthy, 1973: Results of operational cloud seeding experiment over Rihand catchment in northeast India. Indian J. Meteor. Geophys.,25, 379–384.

  • List, R., K. R. Gabriel, B. A. Silverman, Z. Levin, and T. Karakostas, 1999: The rain enhancement experiment in Puglia, Italy: Statistical evaluation. J. Appl. Meteor.,38, 281–289.

  • Mielke, P. W., and J. A. Flueck, 1976: Distributions of ratios for some selected bivariate probability distributions. Proc. Social Statistics Section, Washington, DC, American Statistical Association, 608–618.

  • ——, and J. G. Medina, 1983: A new covariate ratio procedure for estimating treatment differences with application to Climax I and II experiments. J. Climate Appl. Meteor.,22, 1290–1295.

  • ——, J. S. Williams, and S. C. Wu, 1977: Covariance analysis technique based on bivariate log-normal distribution with weather modification applications. J. Appl. Meteor.,16, 183–187.

  • Miller, A. J., D. E. Shaw, and L. C. Veitch, 1979: Analyzing the results of a cloud-seeding experiment in Tasmania. Comm. Stat. Theory Meth.,A8, 1017–1047.

  • Moran, P. A. P., 1959a: The methodology of rainmaking experiments. Rev. Int. Stat. Inst.,38, 105–119.

  • ——, 1959b: The power of a cross-over test for the artificial stimulation of rain. Aust. J. Stat.,1, 47–52.

  • Neumann, J., and E. Shimbursky, 1972: On the distribution of a ratio of interest in single-area cloud-seeding experiments. J. Appl. Meteor.,11, 370–375.

  • Nirel, R., 1994: Bootstrap confidence intervals for the estimation of seeding effect in an operational period. Water-Related Issues, V. Barnett and K. F. Turkman, Eds., Vol. 2, Statistics for the Environment, Wiley, 109–123.

  • Petrondas, D. A., 1983: Two topics in permutation (re-randomization) inference: Double ratio statistics and multiple comparisons. Ph. D. thesis, University of Rochester, 365 pp. [Available from University of Rochester, Rochester, NY 14627.].

  • Siliceo, P., and P. A. Mosino, 1963: Twelve years of cloud seeding in the Necaxa watershed, Mexico. J. Appl. Meteor.,2, 311–323.

  • Smith, E. J., L. G. Veitch, D. E. Shaw, and A. J. Miller, 1979: A cloud-seeding experiment in Tasmania 1964–1970. J. Appl. Meteor.,18, 804–815.

  • Wurtele, Z. S., 1971: Analysis of the Israeli cloud-seeding experiment by means of concomitant variables. J. Appl. Meteor.,10, 1185–1192.

APPENDIX

Asymptotic Randomization Distributions of Ratio Statistics

Single ratio SRY

The statistic’s logarithm is
i1520-0450-38-3-290-ea1
Taylor approximations are applied by expanding the first two terms about Σ yiθi = Σ yi/2 and the last two about Σ θi = n/2. The resulting first-order approximation is
i1520-0450-38-3-290-ea2
After some reduction, this becomes
i1520-0450-38-3-290-ea3
As the θ’s are independent, each with expectation ½ and variance ¼, it follows from the central limit theorem (see Gabriel and Feder 1969) that, conditional on the y’s, the distribution of lnSRY is asymptotically normal. The same is true of the distribution of SRY since SRY ≈ 1 + lnSRY when SRY is near 1, but its convergence is slower. The expectations are readily worked out to be
EYEY
and the variances
i1520-0450-38-3-290-ea4b

Double ratio DR(a)

This statistic’s logarithm is
i1520-0450-38-3-290-ea5
It can be approximated analogously to the argument for (A3) by
i1520-0450-38-3-290-ea6
which becomes
i1520-0450-38-3-290-ea7
As above, it follows that, conditional on the ys and zs, the randomization distributions of lnDR(a) and DR(a) are asymptotically normal with expectations
(a)(a)
and variances
i1520-0450-38-3-290-ea8b

Regression ratio RR

For this statistic, the coefficients a0, a1, . . . , ak of the double ratio DR(a) are chosen to minimize the sum of squared residuals Σ (i − 1 − Σjaj(i, j − 1))2 so that
i1520-0450-38-3-290-ea9
Applying multiple regression theory reduces this to
i1520-0450-38-3-290-ea10
where R2Y:1,...,k is the multiple correlation of the i − 1’s on the i,j − 1’s.

Root double ratio RDR

The logarithm of this statistic is
i1520-0450-38-3-290-ea11
and can be approximated by expansions of the four logarithms in this expression about arguments in which each θ is replaced by its expectation ½. Thus, the first two terms are expanded about Σ yiθi = Σ yi/2 and the last two are expanded about Σ xiθi = Σ xi/2. The resulting first-order Taylor approximation is
i1520-0450-38-3-290-ea12
After some reduction, this becomes
i1520-0450-38-3-290-ea13
Again, since the θ’s are independent, each with expectation ½ and variance ¼, it follows that, conditional on the x’s and y’s, lnRDR and RDR are asymptotically normal with
and
i1520-0450-38-3-290-ea14b

Root quadruple ratio RQR(b)

The logarithm of this statistic is
i1520-0450-38-3-290-ea15
which, by an argument as for (A12) above, is linearly approximated by
i1520-0450-38-3-290-ea16
that is, by
i1520-0450-38-3-290-ea17
Again, conditional on the y’s, x’s, and z’s, the randomization distributions of lnRQR(b), and RQR(b), are asymptotically normal with expectations
(b)(b)
and variances
i1520-0450-38-3-290-ea18b

Root regression ratio RRR

This statistic is obtained from the root quadruple ratio RQR(b) by choosing the coefficients b0, b1, . . . , bk so as to minimize the sum of squared residuals Σ [ii − Σjbj(i,j − 1)]2. This sum then becomes Σ (ii)2(1 − R2YX:1,2,...,k) and so, conditional on the y’s, x’s, and z’s, the randomization distributions of lnRRR and RRR are asymptotically normal with expectations
and variances
i1520-0450-38-3-290-ea19b

Fig. 1.
Fig. 1.

A schema of correlations between targets and controls τ, between targets and their controls κ, and between targets and other controls γ.

Citation: Journal of Applied Meteorology 38, 3; 10.1175/1520-0450(1999)038<0290:RSFREI>2.0.CO;2

Fig. 2.
Fig. 2.

Variances of different ratio statistics, general and schematic.

Citation: Journal of Applied Meteorology 38, 3; 10.1175/1520-0450(1999)038<0290:RSFREI>2.0.CO;2

Table 1.

Totals and means for single target and alternating targets experiments. Note that Σ without an index will be considered to have index i running from 1 to n.

Table 1.
Table 2.

Coefficients of variation and correlations of targets and controls for the Israel II experiment. (The corresponding parameters from the model of Fig. 1 are indicated as approximations.)

Table 2.
Table 3.

Coefficients of variation and correlations of targets and controls for the Puglia experiment. (The corresponding parameters from the model of Fig. 1 are indicated as approximations.)

Table 3.
Table 4.

Number of days (years) needed for different designs and statistics to yield 90% power of detecting a 15% increase with 5% significance; approximations under schematic assumptions for the Israel II and Puglia experiments.

Table 4.
Save