## Introduction

Hail suppression is a weather modification technology that seemingly generates more sustained interest, on a worldwide basis, than any other. Cloud seeding for hail suppression has been carried out in many parts of the world using a variety of techniques. Dennis (1980) discusses hypotheses of how seeding could reduce damaging hailfall. Some randomized experiments based on such hypotheses have yielded significant evidence of seeding effects (e.g., Miller et al. 1975; Rudolph et al. 1994), while others have not (e.g., Crow et al. 1979; Federer et al. 1986). In the face of these conflicting experimental results, operational hail-suppression seeding programs continue, and there are indications that at least some of them produce reductions in hail damage (e.g., Dessens 1986; Markó et al. 1990; Simeonov 1992; Mesinger and Mesinger 1992).

Studies of the climatology of hail damage to crops (Changnon 1977, 1984) show that North Dakota experiences the highest insurance dollar loss of any state in the United States, while southwestern North Dakota has the highest ratio of damage claims paid to insured crop liability (Miller and Fuhs 1987). Operational cloud seeding has been going on in western North Dakota since the 1950s, with regular hail-suppression operations taking place in some areas since 1961 (Rose and Jameson 1986). The North Dakota program claims to be the longest continuing program in the world employing seeding from aircraft. Since 1976 the operations have been organized as the North Dakota Cloud Modification Project (NDCMP) and supervised by the North Dakota Atmospheric Resource Board (formerly the North Dakota Weather Modification Board), an agency of the state of North Dakota. This continuing support is based on a perception that the seeding has been effective in reducing hail damage to crops. It seems reasonable to examine the available data for any indications that may support or contradict this perception.

Rose and Jameson (1986) and Miller and Fuhs (1987) conducted preliminary analyses of crop hail insurance data from western North Dakota and neighboring regions. They found some indications of reduced hail damage in the seeded areas. The purpose of this paper is to present the results of a further exploratory analysis of the same kind of data using more powerful statistical methods. In this analysis we follow the approach advocated in Mielke et al. (1982), which requires historical and treatment period data for the target area along with a control area. Thus, it combines some of the features of both historical and target–control comparisons.

## The North Dakota Cloud Modification Project

The state-managed NDCMP was established and field operations initiated in 1976. The basic treatment strategy has been essentially the same over the succeeding years, although there have been some variations in seeding materials and delivery technology. The general approach is for aircraft to deliver glaciogenic seeding agents to the summertime convective clouds, mostly at cloud base, using guidance from ground-based radar. From 1976 to 1988, the target area included six continuously participating counties, comprising 26 278 km^{2} in western North Dakota (Fig. 1).

The operational procedures followed in the NDCMP are spelled out in a project operations manual (North Dakota Weather Modification Board 1980). Cloud seeding in the NDCMP has a dual purpose; in addition to suppressing hail, a rainfall-enhancement effect is desired. The aircraft conduct seeding in two modes: within updrafts at cloud base and by direct injection during penetration at or near cloud top. The former mode is most often used, as only one out of every four project aircraft is equipped for repeated cloud penetrations and possesses the quality of performance required for cloud-top work.

The seeding agents are primarily glaciogenic, as observations of northern Great Plains cumuliform clouds indicate that most precipitation results from processes involving the formation of ice. In cloud-base seeding, silver iodide (AgI) complexes are produced either by the combustion of acetone-based solutions in wingtip generators or by the burning of flares in racks mounted on the trailing edges of the wings. The ice nuclei produced are intended to be ingested by the target clouds, transported upward to the regions containing supercooled liquid water, and mixed through a significant portion of the cloud volume. Timely treatment ensures that the seeding agent can disperse through much of the supercooled portion of the cloud by the time the cloud top grows through the −10°C level. Ice nucleation beginning at about this time would be some 5°–10°C warmer than most natural nucleation. Given typical cloud-growth rates, the “head start” in precipitation development in the seeded clouds would be on the order of 3–5 min.

In direct-injection seeding, either dry-ice pellets (frozen CO_{2}) or ejectable silver iodate flares are used. Again, clouds growing through the −10°C level are targeted. The seeding agents are placed directly into the supercooled clouds where nucleation is desired; the updrafts in these cases are relied upon only to provide a continuing source of condensate. This delivery technique requires less anticipation on the part of those directing and conducting the seeding, and has the advantage of placing the agent directly into supercooled cloud regions.

In both cases the intent is to glaciate portions of the cloud, initiating ice development minutes earlier than it would naturally have occurred. For smaller or more isolated convective towers, this glaciogenic seeding can accelerate hydrometeor growth sufficiently to allow the clouds to produce precipitation during their short lifetimes. In larger clouds and convective complexes, accelerated hydrometeor development can result in increased total rain volume and also reduce hail production by altering the hailstone-growth environment.

The conceptual model guiding the seeding operations, developed from investigations of hailstorms in surrounding regions including those of Dennis et al. (1970), Knight and Squires (1982a,b), Krauss and Marwitz (1984), Humphries et al. (1987), and Cheng and Rogers (1988), is as follows.

*Natural clouds:*

The main updraft of each mature thunderstorm cell is responsible for a large fraction of the cell’s total mass flux and supports the larger hydrometeors (hailstones) as they grow. Primary hailstorm updrafts are often tilted and frequently possess speeds in excess of 20 m s

^{−1}, which are essential for the production of large hail.Vigorous convective clouds, especially those flanking the mature thunderstorm cells, develop significant quantities of supercooled liquid water. Cloud droplet spectra are usually continental in character, and precipitation development through ice-phase precipitation processes are usually dominant within these growing clouds. Ice development most often occurs after cloud tops attain temperatures of −15° to −20°C.

Because of the initially slow development of natural ice-phase precipitation in these clouds, few hydrometeors are sufficiently large to precipitate at the time that the growing cloud begins to evolve into or merge with a mature cell. Thus, the bases of such transitioning clouds remain rain-free. Any potential hail embryos remain aloft within the maturing cell at this time.

Graupel generated within these clouds then become hail embryos that are resident along the periphery of the updraft as it matures. Rapidly evolving hydrometeors begin to precipitate. Hailstones become too large to melt completely during fallout, transit the warm subcloud air, and reach the ground.

*Cloud treatment:*Treatment of supercooled clouds with ice-forming nuclei intended to activate as the clouds grow through the −10°C level will initiate significant ice development minutes before it would otherwise occur.

Significant glaciation occurs in the treated cells. Much of the supercooled cloud liquid water is converted to ice through either riming or evaporation-deposition. Supercooled liquid water is depleted more rapidly in treated clouds than in untreated clouds. Precipitation mass within the clouds is enhanced.

Updrafts in treated cells remain essentially unchanged in magnitude.

Precipitation from the treated cells begins. A developing precipitation shaft exists where there otherwise would have been a rain-free cloud base (early rainout and trajectory lowering). Most precipitating ice particles are small and melt before reaching the surface.

As a treated cell reaches maturity or is absorbed by a mature cell, less supercooled cloud liquid water remains

^{1}because seeding accelerates the within-cell glaciation. Large populations of precipitating ice-phase hydrometeors continue to deplete the remaining supercooled liquid water (beneficial competition).Many multicell thunderstorms feed on moist boundary layer air, usually drawn in from the southern or eastern quadrants. The precipitation shaft that develops beneath the previously rain-free cloud base (the early rainout) may interfere with such inflow, reducing the “fuel supply” to the maturing cells (fuel starvation).

These northern Great Plains clouds are characterized by continental cloud droplet spectra and are slow to produce ice naturally, yet usually require an active ice phase precipitation process before precipitation is produced. Treatment with ice-forming nuclei ultimately diminishes total supercooled liquid water in the mature updraft. The mature updrafts are not seeded directly because ice particles originating in strong updrafts of mature cells, perhaps as a direct result of artificial nucleation, generally do not grow sufficiently during their brief residence within the mature updraft to develop appreciable terminal velocities or to diminish the supercooled water significantly. Such particles reside within the mature updraft for only a few minutes, remain small, and ultimately exit the updraft as they are transported into the upper reaches of the storm anvil. Thus, seeding the primary updraft of a mature storm likely has little effect on the storm and will not effectively diminish hail development.

The mature updraft may be weakened by mass loading and possibly by fuel starvation. The environment is less favorable for the growth of hail, and less damaging hail results. The rain shaft of the storm is broadened by early rainout. Measurable precipitation falls in some areas that otherwise would have remained rain-free. Other areas that would have received locally intense rain and hail receive less intense rain and significantly less hail damage.

The NDCMP operators recognize that important parts of this conceptual model are not well documented. An intensive, ongoing research program to improve understanding of the basic cloud processes involved is being conducted in parallel with the NDCMP operations (e.g., Smith et al. 1992; Boe et al. 1992).

## Crop hail insurance data

Crop hail insurance data are available for western North Dakota and the adjacent regions from 1924 onward (CHIAA 1978). These data indicate the yearly insured liability and the associated loss ratios (the ratio of damage claims paid, in dollars, to insured liabilities) on a township by township or county by county basis. The use of such data for evaluating seeding effects has limitations, as discussed by Changnon (1969, 1985). Among them are the facts that only part of the crops in any given area are insured and the insured portion varies with time; crop sensitivity to hail damage varies over the season; and farming techniques, cropping patterns, crop yields, and crop values also vary with time. Perhaps most importantly, the insurance forms often include a “deductible” clause stipulating that no payment will be made for losses smaller than a specified fraction (say, 10%) of the insured crop value. Consequently, the loss payments cannot be equated directly to hail damage. However, the insurance data also have important advantages: they cover much larger areas than would be practical with any known hail-measurement instruments, they cover a long historical period, and they are based on a relevant economic measure of the losses due to hail. We base this exploratory analysis on these data because of these advantages, while at the same time recognizing their limitations.

Using the loss ratios, in the form of annual values based on areal totals of liabilities and calculated loss payments, helps to mitigate some of the limitations of the hail-insurance data. Fig. 1 shows a county map of the region of interest in western North Dakota and eastern Montana. Seeding has been conducted from time to time in many of the counties of western North Dakota, but the six counties shown crosshatched were regularly seeded using essentially the same techniques during the NDCMP period of interest here (1976–88). The southwestern counties (Bowman, Hettinger, and Slope) comprise NDCMP District I, while the northern counties (McKenzie, Mountrail, and Ward) comprise District II. These six counties constitute the target area for these exploratory analyses, while the 12 easternmost counties of Montana (shaded in Fig. 1) provide an upwind control area. The control area is larger than the target area, but the insured liabilities for the two areas are similar over the years (Fig. 2). The dollar liabilities, however, vary by a factor of about 10^{3} over the period of record.

A change in the general quality of the crop hail insurance data beginning in the late 1940s has sometimes been suggested. However, Neill (1981) made use of data from 1924 onward and reported no such discontinuity. Data from the control area employed here were tested (using tests similar to those discussed below), and significant differences related to a division around that time were not found. Consequently, these exploratory analyses make use of the whole historical record.

## Historical analyses

Appendix A contains a table of the annual values of the loss ratios for the control area in Montana and the NDCMP target area. This section presents separate historical analyses for the respective areas, while section 5 discusses the target–control comparisons.

### Target area

Figure 3 shows the historical record of the annual loss ratios for the six-county NDCMP target area. The values range from a low of 0.02% in 1939 to a high of 19.75% in 1963, with a median for the 64-yr period 1924–88 of 4.83% (1934 is omitted here because the liability was extremely small, more than a factor 20 lower than that for any other year). There is some visual indication of a downward trend after extensive hail-suppression seeding began in the 1960s. Indeed, Rose and Jameson (1986) found indications of reduced hail damage in District I over this period. However, we concentrate here on the last 13 years, when the NDCMP was in operation over the full six-county target area. Eleven of the 13 loss ratios for those years are below the overall median value, and another is only slightly higher.

To explore whether the hail-loss experience over this 13-yr period differed significantly from that for the earlier period 1924–75, a permutation analysis was run using the multiresponse permutation procedures (MRPPs) in a univariate mode (Mielke et al. 1981a,b; Mielke 1985, 1991). The MRPPs have previously been used to evaluate randomized cloud seeding experiments (Mielke et al. 1982, 1984), and a description of MRPPs as used here appears in appendix B. The analysis proceeds by selecting the loss ratios for 13 years (without replacement) at random from the data population of 64 years. The values for the remaining 51 years form the comparison set. Then a measure of the separation between the two groups (the values for those 13 years and for the remaining years) in relation to the scatter within each group is calculated. The process is repeated for all distinct possible selections of the 13 years from the 64 years. The test statistics are then ranked and compared to the corresponding test statistic for the actual division into NDCMP and remaining years to determine a *P* value (*P*).

The MRPP two-sided *P* value of finding loss ratios as small as, or smaller than, those observed during the NDCMP years in a random sample of 13 from the population of 64 values is *P* = 0.006. Hence, it is unlikely that the 13 NDCMP values are just a random sample from the population. This *P* value cannot be interpreted in quite the same way as one from a randomized experiment because the actual NDCMP years were not chosen at random (Gabriel and Petrondas 1983). They were, however, chosen a priori; a *P* value this small is therefore an indication of a reduction in hail loss experienced in the target area during the NDCMP years.

Whether the difference was due to the NDCMP seeding cannot be determined from the target area data alone. A climatological shift toward lower hail losses might have occurred during the NDCMP operational period. Changnon’s (1984) hail climatology study gives little indication of such a shift, but a more specific examination of the possibility can be made using the control area data.

### Control area

Figure 4 shows the historical record of loss ratios for the 12-county control area in eastern Montana. The values range from a low of 0.25% in 1988 to a high of 19.22% in 1937. The median for the 65-yr period (1934 is included here) is 6.10%. There is no visual indication of a historical trend; losses recorded during the period 1947–60 were consistently low, but 5 of those 14 values were still above the median. During the NDCMP years, 6 of the 13 values were below the median.

The same univariate MRPP test was applied to the control area historical data using 13-yr random samples from the population. The MRPP two-sided *P* value of obtaining loss ratios as extreme as, or more extreme than, those found during the 13 NDCMP years is *P* = 0.62. In other words, the values for the 13-yr NDCMP operational period cannot be distinguished from those for a random sample from the population. This suggests that there was no general climatological shift in hail damage occurrences associated with the NDCMP operational period. Such a conclusion strengthens the indication that the lower target area losses experienced during the NDCMP years may have resulted from the seeding operations.

## Target–control comparisons

The annual ratio of target to control loss ratio (hence a kind of double ratio) ranges from 0.005 to 8.04, with a median for the 64 years of record of 0.89. The 13 NDCMP years include 2 of the 3 highest of these values, but 10 of the 13 values fall below the overall median (a result with an exact *P* value of 0.046 if the ratios were random events). The box-and-whisker plot in Fig. 5, comparing the target–control ratios for the 13 NDCMP years with those for the 51 historical years, gives a distinct visual impression of a generally lower loss experienced in the target area during the NDCMP years.

The principal statistical target–control comparison is based on MRPP tests of residual deviations (differences between predicted and observed values) about regression lines on a scatterplot of target versus control loss ratios. The basic regression analysis employed uses the least absolute deviation (LAD) regression line (Barrodale and Roberts 1973, 1974) constrained to pass through the origin. This regression facilitates establishing a relationship between target and control area loss ratios and provides a means for obtaining an estimate of the magnitude of any temporal difference that may be indicated as significant. The LAD regression has the advantage of not giving undue weight to individual points that may be outliers. Similar MRPP analyses were also completed for residuals about the unconstrained LAD line and the least squares regression line for comparison.

The MRPPs compare residuals about the composite regression line for two groups of points, corresponding to the respective sets of years selected at random as in the historical analyses discussed in section 4. If a low *P* value results for the specific NDCMP years, it is likely that the NDCMP period residual values are not a random selection from the composite group. In that case, separate regression lines can be determined for each of the two time periods, and the difference can be regarded as an indication of a seeding effect. Other factors, such as varying agricultural practices, may also contribute to the difference, and their contributions would be combined with the seeding effect; the methodology does not permit separation of the individual contributions. However, factors other than cloud seeding that would be likely to have effects differing between the control and target areas and also between the two time periods have not been identified.

*r*= 0.34), but this relationship provides a rough prediction of the expected target area loss ratio from the control area value.

Nine of the 13 points for the NDCMP operational years lie below the LAD regression line. The (signed) residual deviation from this line was calculated for every point; the residuals ranged from +10.47% to −9.42%, with a median value of 0.63%. Then an MRPP test similar to that used for the historical record (section 4) was carried out on these residuals. The MRPP two-sided *P* value is *P* = 0.025 and indicates that the residuals for the NDCMP years were significantly more negative than would be expected in a random sample from the population. In other words, the target area loss ratios during those years were significantly lower than would be predicted from the composite LAD regression line.^{2}

*P*value justifies computation of separate LAD regression lines for the 13 NDCMP years and the 51 remaining years. The results, also indicated in Fig. 6, are

Appendix C outlines a method, based on the same LAD regression approach, for deriving a maximum-likelihood estimate of the difference between the target area loss ratios for the historical and NDCMP years. The scale factor *G* obtained with that method can be viewed as an alternative to the previous estimate using the ratio of regression slopes. For the present data the maximum-likelihood estimate *G*_{M} is 0.547, which would again correspond to a 45% reduction in the hail loss ratios during the NDCMP years. The associated lower and upper 90% confidence interval limits are *G*_{L} = 0.383 (62% reduction) and *G*_{U} = 0.937 (6% reduction), respectively. Thus the entire 90% confidence range of values falls on the side of reduced target area loss ratios during the NDCMP years.

*x*and

*y*denote the respective control and target area values of the annual loss ratios, one is the double-ratio statistic (DR) given by

*x̄*

_{t}and

*ȳ*

_{t}denote the sample control and target means for the 13 NDCMP treated years, and

*x̄*

_{h}and

*ȳ*

_{h}denote the corresponding means for the 51 historical years. The other is the least squares slope ratio statistic (SR) given by

*β̂*

_{t}

*β̂*

_{h}

*y*

*βx*

*e*

*e*denotes the residual error. If (

*x*

_{1},

*y*

_{1}), . . . , (

*x*

_{n},

*y*

_{n}) is a sample of

*n*control and target values, then the least squares estimator of

*β*is given by

^{6}configurations was obtained from the total

^{6}sample values of DR and SR were both ordered from smallest to largest. Then the 90% acceptance region boundary values correspond to the 5% and 95% sample quantile values of the ordered 10

^{6}DR or SR values. The 90% acceptance region boundary values for DR are 0.616 and 1.612, while those for SR are 0.582 and 1.723. For both DR and SR the observed value falls outside the 90% acceptance region. This lends further support to the inference that the difference in loss ratios in the target area during the NDCMP years was not due to chance.

These estimates of the NDCMP seeding effect on hail insurance losses may even be conservative. Reports of cloud seeding in North Dakota indicate that portions of the target area received treatment via ground generators as early as 1952. Sporadic seeding activities continued through the 1950s, seeding by aircraft was initiated in 1961, and the level of activity increased until all of the target area was included by 1970. However, the seeding was usually limited due to randomized project design, restriction to daylight delivery, or other constraints. The historical period used in the above analysis includes all of the 1952–75 seeding activity. If this seeding was effective, then hail losses in the target area would also have been reduced during that part of the historical period. However, the MRPP two-sided *P* values do not indicate a significant difference between loss ratios for the two periods 1924–55 (1934 not included) and 1956–75 for either the target (*P* = 0.39) or control (*P* = 0.94) area. Hence no statistical basis was found for subdividing the historical period used in the foregoing analysis. Nevertheless, any reduction in target area hail losses achieved during the 1952–75 period (e.g., as indicated in Rose and Jameson 1986) would dilute the apparent effects of the NDCMP seeding in this analysis.

## Conclusions

This exploratory analysis suggests that the crop hail insurance loss ratios in the NDCMP target area averaged 45% lower than would be expected on the basis of prior experience during the 1976–88 operational period. This estimate of the reduction in hail losses is similar to that reported by Dennis et al. (1981) for a similar operational seeding program in South Dakota, as well as by Dessens (1986) for an operational program in France and by Markó et al. (1990) for operational projects in eastern Europe. The 90% confidence interval lies entirely on the side of reduced losses in the areas seeded by the NDCMP. The control area historical analysis in section 4b indicates that climatological variations were not a major factor in this difference. It therefore seems plausible to infer that the reduction was due to the NDCMP seeding operations. Of course, the possibility of a “second-order” climatological shift, in which the relationship between hail-loss experience in the control and target areas changed just around 1976, cannot be excluded by this (or any) analysis based on insurance data alone.

After 1988 the target area included in the NDCMP changed. One county (Hettinger) voted to drop out of the program from 1989, and financial exigencies restricting the state support of the program caused the other two counties of District I (Bowman and Slope) to suspend operations for the summer of 1990. The latter two counties returned to the program from 1991 onward. These variations in the target area would disrupt the continuity of the analysis presented here, so no years after 1988 have been included. With a few more years of experience, a comparative analysis of data for the current five-county target area and for the 1976–88 six-county target will provide an interesting perspective.

This exploratory analysis should be substantiated by more extensive analysis over a longer operational period. The deductible component in the insurance loss payments means that the loss ratios used here cannot be related directly to hail damage; it may be possible to adjust the insurance data to compensate for the deductible amounts, and this possibility should be pursued. A randomized experiment designed to guard against all possible types of climatological variations would be needed to provide confirmatory evidence of the indicated seeding effects. A more detailed physical explanation of the means by which the seeding reduces hail losses is also needed.

## Acknowledgments

The initial analysis was carried out as an activity of the Consortium for Atmospheric Resources Development, with funding from the North Dakota Weather Modification Board under Contract WMB-CARD-86-1. We thank R. Lynn Rose for providing the opportunity to conduct this investigation. Further work was funded by the National Oceanic and Atmospheric Administration (NOAA) Federal–State Cooperative Program Agreements NA27RA0178-01 and NA47RA0184 between NOAA and the North Dakota Atmospheric Resource Board, State Water Commission through NDARB Contracts ARB-IAS-93-1, ARB-IAS-94-1, and ARB-IAS-95-1. The views expressed herein are those of the authors and do not necessarily reflect the views of NOAA or any of its subagencies.

## REFERENCES

Barrodale, I., and F. D. K. Roberts, 1973: An improved algorithm for discrete L1 linear approximation.

*SIAM J. Numer. Anal.,***10,**839–848.——, and ——, 1974: Solution of an overdetermined system of equations in the L1 norm.

*Commun. Assoc. Comp. Mach.,***17,**319–320.Boe, B. A., and Coauthors, 1992: The North Dakota Thunderstorm Project—A cooperative study of High Plains thunderstorms.

*Bull. Amer. Meteor. Soc.,***73,**145–160.Changnon, S. A., Jr., 1969: Hail measurement techniques for evaluating suppression projects.

*J. Appl. Meteor.,***8,**596–603.——, 1977: The climatology of hail in North America.

*Hail: A Review of Hail Science and Hail Suppression, Meteor. Monogr.,*No. 38, Amer. Meteor. Soc., 107–128.——, 1984: Temporal and spatial variations in hail in the upper Great Plains and Midwest.

*J. Climate Appl. Meteor.,***23,**1531–1541.——, 1985: Use of crop-hail data in hail suppression evaluation.

*Proc. Fourth WMO Scientific Conf. on Weather Modification, Vol. II,*Geneva, Switzerland, WMO, 563–567.Cheng, L., and D. C. Rogers, 1988: Hailfalls and hailstorm feeder cells—An Alberta case study.

*J. Atmos. Sci.,***45,**3533–3545.CHIAA Staff, 1978: Crop-Hail Insurance Statistics. 50 pp and appendices. [Available from Crop-Hail Insurance Actuarial Association, Room 700, 209 W. Jackson Blvd., Chicago, IL 60606.].

Crow, E. L., A. B. Long, J. E. Dye, A. J. Heymsfield, and P. W. Mielke Jr., 1979: Results of a randomized hail suppression experiment in northeast Colorado. Part II: Surface data base and primary statistical analysis.

*J. Appl. Meteor.,***18,**1538–1558.Dennis, A. S., 1980:

*Weather Modification by Cloud Seeding.*Vol. 24, Academic Press, 267 pp.——, C. A. Schock, and A. Koscielski, 1970: Characteristics of hailstorms of western South Dakota.

*J. Appl. Meteor.,***9,**127–135.——, B. G. Brown, and J. R. Miller Jr., 1981: Variations in rainfall and insured crop-hail losses associated with operational cloud seeding in South Dakota.

*J. Wea. Mod.,***13,**150–160.Dessens, J., 1986: Hail in southwestern France. II: Results of a 30-year hail prevention project with silver iodide seeding from the ground.

*J. Climate Appl. Meteor.,***25,**48–58.Federer, B., and Coauthors, 1986: Main results of Grossversuch IV.

*J. Climate Appl. Meteor.,***25,**917–957.Gabriel, K. R., and W. J. Hall, 1983: Rerandomization inference on regression and shifteffects: Computationally feasible methods.

*J. Amer. Stat. Assoc.,***78,**827–836.——, and C.-F. Hsu, 1983: Evaluation of the power of rerandomization tests, with application to weather modification experiments.

*J. Amer. Stat. Assoc.,***78,**766–775.——, and D. Petrondas, 1983: On using historical comparisons in evaluating cloud seeding operations.

*J. Climate Appl. Meteor.,***22,**626–631.Humphries, R. G., M. English, and J. Renick, 1987: Weather modification in Alberta.

*J. Wea. Mod.,***19,**13–24.Knight, C. A., and P. Squires, Eds., 1982a:

*Hailstorms of the Central High Plains.*Vol. I,*The National Hail Research Experiment,*Colorado Associated University Press, 282 pp.——, and ——, Eds., 1982b:

*Hailstorms of the Central High Plains.*Vol. II,*Case Studies of the National Hail Research Experiment,*Colorado Associated University Press, 245 pp.Krauss, T. W., and J. D. Marwitz, 1984: Precipitation processes within an Alberta supercell hailstorm.

*J. Atmos. Sci.,***41,**1025–1034.Markó, T., F. Söver, and P. Simeonov, 1990: On the damage reduction in Bulgarian and Hungarian hail suppression projects.

*J. Wea. Mod.,***22,**82–89.Mesinger, F., and N. Mesinger, 1992: Has hail suppression in eastern Yugoslavia led to a reduction in the frequency of hail?

*J. Appl. Meteor.,***31,**104–111.Mielke, P. W., Jr., 1985: Geometric concerns pertaining to applications of statistical tests in the atmospheric sciences.

*J. Atmos. Sci.,***42,**1209–1212.——, 1991: The application of multivariate permutation methods based on distance functions in the earth sciences.

*Earth Sci. Rev.,***31,**55–71.——, K. J. Berry, and G. W. Brier, 1981a: Application of multi-response permutation procedures for examining seasonal changes in monthly mean sea-level pressure patterns.

*Mon. Wea. Rev.,***109,**120–126.——, ——, P. J. Brockwell, and J. S. Williams, 1981b: A class of nonparametric tests based on multi-response permutation procedures.

*Biometrika,***68,**720–724.——, and ——, and J. G. Medina, 1982: Climax I and II: Distortion resistant residual analyses.

*J. Appl. Meteor.,***21,**788–792.——, ——, A. S. Dennis, P. L. Smith, J. R. Miller Jr., and B. A. Silverman, 1984: HIPLEX-1: Statistical evaluation.

*J. Climate Appl. Meteor.,***23,**513–522.Miller, J. R., Jr., and M. J. Fuhs, 1987: Results of hail suppression efforts in North Dakota as shown by crop hail insurance data.

*J. Wea. Mod.,***19,**45–49.——, E. I. Boyd, R. A. Schleusener, and A. S. Dennis, 1975: Hail suppression data from western North Dakota, 1969–1972.

*J. Appl. Meteor.,***14,**755–762.Neill, J. C., 1981: An approach to crop-hail insurance rate revision.

*J. Appl. Meteor.,***20,**1391–1399.North Dakota Weather Modification Board, 1980: The North Dakota Cloud Modification Project—An Operations Manual. 269 pp. [Available from North Dakota Weather Modification Board, Bismarck, ND58505.].

Rose, R. L., and T. C. Jameson, 1986: Evaluation studies of long-term hail damage reduction programs in North Dakota.

*J. Wea. Mod.,***18,**17–20.Rudolph, R. C., C. M. Sackiw, and G. T. Riley, 1994: Statistical evaluation of the 1984–88 seeding experiment in northern Greece.

*J. Wea. Mod.,***26,**53–60.Simeonov, P., 1992: Comparative study of hail suppression efficiency in Bulgaria and in France.

*Atmos. Res.,***28,**227–235.Smith, P. L., H. D. Orville, B. A. Boe, and J. L. Stith, 1992: A status report on weather modification research in the Dakotas.

*Atmos. Res.,***28,**271–298.

## APPENDIX A

### Crop Hail Insurance Loss Ratios for Montana Control Area and NDCMP Target Area, 1924–88

## APPENDIX B

### Description of Multiresponse Permutation Procedures

**Ω**

*ω*

_{1}

*ω*

_{N}

*N*objects (e.g., years), let

*Z*

_{i}denote a response measurement (e.g., a loss ratio) for object

*ω*

_{i}(

*i*= 1, . . . ,

*N*), and let

*S*

_{1}and

*S*

_{2}represent an exhaustive partitioning of the

*N*objects in

**Ω**into two disjoint groups (e.g., where

*S*

_{1}is the historical period and

*S*

_{2}is the NDCMP period). Also let

_{i,j}

*Z*

_{i}

*Z*

_{j}

*ω*

_{i}and

*ω*

_{j}. The choice of Euclidean distance for Δ

_{i,j}provides a meaningful interpretation of the physical comparisons in question (Mielke et al. 1982). The MRPP test statistic is given by

*δ*

*C*

_{1}

*ξ*

_{1}

*C*

_{2}

*ξ*

_{2}

*C*

_{1},

*C*

_{2}> 0,

*C*

_{1}+

*C*

_{2}= 1,

*S*

_{k}(

*k*= 1, 2);

*n*

_{k}is the number of objects in group

*S*

_{k};

*N*=

^{2}

_{k=1}

*n*

_{k}; Σ

_{i<j}is the sum over all

*i*and

*j*such that 1 ≤

*i*<

*j*≤

*N*; and

*I*

_{k}(

*ω*

_{i}) is 1 if

*ω*

_{i}belongs to

*S*

_{k}and 0 otherwise for

*i*= 1, . . . ,

*N.*While the choice of

*C*

_{k}in this paper is

*n*

_{k}/

*N,*other possibilities have been considered (Mielke 1991).

*H*

_{o}) of MRPP is that equal probabilities are assigned to each of the

*N*objects to the two groups. In the present application of MRPP small values of

*δ*imply a concentration of response measurements within the two groups. The exact

*P*value of MRPP is given by

*δ*

_{o}denotes the observed value of

*δ*. If

*M*is large (e.g., greater than 10

^{6}), then the exact

*P*value can be approximated by a Pearson type III distribution (a standardized gamma distribution), which depends on the exact mean, variance, and skewness of

*δ*under

*H*

_{o}(Mielke 1991).

## APPENDIX C

### Permutation-Based Maximum-Likelihood and Confidence Interval Estimators for Differences between Historical and Treated Period Regression Lines

*y*

_{i}

*β*

*x*

_{i}

*e*

_{i}

*n*

_{h}historical cases (

*i*= 1, . . . ,

*n*

_{h}) and the

*n*

_{t}treated period cases (

*i*=

*n*

_{h}+ 1, . . . ,

*n*

_{h}+

*n*

_{t}). Here

*y*

_{i},

*x*

_{i}, and

*e*

_{i}, respectively, denote the dependent (response or, in this instance, target area), independent (predictor or control area), and deviation (residual error) values of the

*i*th case (year), and

*β*is an unknown scale parameter. The least sum of absolute deviations algorithm (Barrodale and Roberts 1974) minimizes the sum given by

*β*(

*β̃*

*ẽ*

_{i}

*y*

_{i}

*β̃*

*x*

_{i}

*i*

*n*

_{h}

*n*

_{t}

*H*

_{o}) states that the LAD deviations for the

*n*

_{h}historical and

*n*

_{t}treated period cases are from a common population. Multiresponse permutation procedures are used to compare the historical and treated period LAD deviations under

*H*

_{o}. The MRPP statistic used here is given by

*δ*under

*H*

_{o}, the permutation distribution of

*δ*is approximated by a Pearson type III distribution to obtain

*P*values (Mielke 1991).

*y*

^{*}

_{i}

*Gy*

_{i}

*i*= 1, . . . ,

*n*

_{h}, where

*G*is a scale factor that can be chosen to make the median of the historical deviation values essentially indistinguishable from the median of the treated period deviation values. Under

*H*

_{o},

*G*= 1. The maximum-likelihood estimator of

*G*(

*G*

_{M}) is that value of

*G*that yields the largest

*P*value (

*P**) under the MRPP analysis; since MRPP detects both scale and location alternatives,

*P** may be less than 1. Consequently, the lower and upper (1 −

*α*) confidence interval limits of

*G*(

*G*

_{L}and

*G*

_{U}) both yield the

*P*value given by

*α*

*P** since MRPP is presently a two-sided test. While

*G*= 1 yields the

*P*value of MRPP for the observed data, a simple search algorithm yields the maximum likelihood and lower and upper (1 −

*α*) confidence limit estimators

*G*

_{M},

*G*

_{L}, and

*G*

_{U}, respectively.

Insured liability (dollars, log scale) for the control and target areas for the period 1924–88.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Insured liability (dollars, log scale) for the control and target areas for the period 1924–88.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Insured liability (dollars, log scale) for the control and target areas for the period 1924–88.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Annual loss ratios for the target area from 1924 to 1988. Open symbols indicate the NDCMP period. (Note that 1934 was not included because the liability was extremely small.)

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Annual loss ratios for the target area from 1924 to 1988. Open symbols indicate the NDCMP period. (Note that 1934 was not included because the liability was extremely small.)

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Annual loss ratios for the target area from 1924 to 1988. Open symbols indicate the NDCMP period. (Note that 1934 was not included because the liability was extremely small.)

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Annual loss ratios for the control area from 1924 to 1988. Open symbols indicate the NDCMP period.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Annual loss ratios for the control area from 1924 to 1988. Open symbols indicate the NDCMP period.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Annual loss ratios for the control area from 1924 to 1988. Open symbols indicate the NDCMP period.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Box-and-whisker plot comparing annual ratios (log scale) of target to control loss ratios for historical period (left) and NDCMP years (right).

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Box-and-whisker plot comparing annual ratios (log scale) of target to control loss ratios for historical period (left) and NDCMP years (right).

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Box-and-whisker plot comparing annual ratios (log scale) of target to control loss ratios for historical period (left) and NDCMP years (right).

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Scatterplot comparing annual loss ratios (LR) in target and control areas. NDCMP years are represented by unfilled symbols, with constrained LAD regression line dotted. Remaining years are represented by filled symbols, with constrained LAD regression line dashed. Solid line indicates composite regression line.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Scatterplot comparing annual loss ratios (LR) in target and control areas. NDCMP years are represented by unfilled symbols, with constrained LAD regression line dotted. Remaining years are represented by filled symbols, with constrained LAD regression line dashed. Solid line indicates composite regression line.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2

Scatterplot comparing annual loss ratios (LR) in target and control areas. NDCMP years are represented by unfilled symbols, with constrained LAD regression line dotted. Remaining years are represented by filled symbols, with constrained LAD regression line dashed. Solid line indicates composite regression line.

Citation: Journal of Applied Meteorology 36, 5; 10.1175/1520-0450(1997)036<0463:AEAOCH>2.0.CO;2