1. Introduction

The return period of a weather event of a specific large magnitude is of fundamental interest in applied meteorology and climatology. All evaluations of the risks of extreme weather events, such as high winds and heavy rain, require methods to statistically estimate their return periods from the measured data. Such methods are widely used in building codes and regulations concerning the design of structures and community planning, as examples. Furthermore, it is crucial for the safety and economically optimized engineering of future communities to be able to estimate the changes in the frequency of various natural hazards with climatic change, and analyzing trends in the weather extremes (Zhang et al. 2004). For that purpose, corresponding statistical analysis needs to be made to the data simulated by climate models (Meehl et al. 2000; Zhang et al. 2004; Kharin and Zwiers 2005).

The return period R (in years) of an event is related to the probability P of not exceeding this event in one year by

View Expanded

A standard method to estimate R from measured data is the following. One first ranks the data, typically annual extremes or values over a threshold, in increasing order of magnitude from the smallest m = 1 to the largest m = N and associates a cumulative probability P to each of the mth smallest values. Second, one fits a line to the ranked values by some fitting procedure. Third, one interpolates or extrapolates from the graph so that the return period of the extreme value of interest is estimated.

Basically, this extreme value analysis method, introduced by Hazen (1914), can be applied directly by using arithmetic paper (see also Castillo 1988, 129–131). However, interpolation and extrapolation can be made more easily when the points fall on a straight line, which is rarely the case in an order-ranked plot of a physical variable on arithmetic paper. Therefore, almost invariably, the analysis is made by modifying the scale of the probability P, and sometimes also that of the random variable x, in such a way that the plot against x of the anticipated cumulative distribution function P = F(x) of the variable appears as a straight line. Typically, the Gumbel probability paper (Gumbel 1958) is used because in many cases the distribution of the extremes, each selected from r events, asymptotically approaches the Gumbel distribution when r goes to infinity. In modern analysis, graphs based on the Pareto distribution and the generalized extreme value distribution are also used (e.g., Pickands 1975; Brabson and Palutikof 2000). The transformed variable η that replaces P on such plots is called the reduced variate. Figure 1 shows an illustrative example of the extreme value analysis.

In this paper, an important problem of the extreme value analysis—how to assess the correct cumulative probabilities to the ranked values—is solved. It is first shown that there exists a unique plotting formula when P, as such, is being plotted to estimate return periods. Furthermore, it is pointed out that the so-called modified Gumbel method, in which the plotting is made through an initial transformation to a reduced variate (e.g., Kimball 1960; Cunnane 1978; Harris 1996), produces a probability parameter that cannot be used to estimate the return periods.

2. The history of plotting positions

Over the last 90 years, a number of plotting formulas and related computational methods for the extreme value analysis have been proposed and supported by empirical justification. A summary of the most commonly used plotting formulas is shown in Table 1. Reviews on this subject are available in Cunnane (1978), Castillo (1988), Folland and Anderson (2002), and Jordaan (2005).

The extensive and controversial discussions on the subject of plotting formulas are not repeated here, but it is noted that many of them have lacked theoretical basis and that, consequently, a rather fatalistic attitude toward selecting a proper formula has been common historically. For example, Langbein (1960) considered the selection “like taking a stand on a political question” and Benson (1962) wrote that the selection “cannot be made by comparing the principles on which each is based.” The same uncertainty is reflected in the more recent literature. Jordaan (2005), as an example, writes on the plotting positions that “there appear to be almost as many opinions as there are statisticians.”

On the other hand, order ranking and the plotting positions have been under rigorous mathematical analysis (e.g., Gumbel 1958; Castillo 1988), so that the theoretical foundations are well known in principle. It is perhaps the long and controversial history of the plotting formulas and the many different types of probability papers that have hampered the transformation of the mathematical theory into a correct and generally accepted practice. The latest reflection of this rather confusing situation is the paper by Folland and Anderson (2002) in which a mathematical analysis originating from Gumbel (1958) is presented, but the “proof” of the resulting plotting formula, the so-called Jenkinson's method (Beard 1943),

View Expanded

is based on “the idea that a natural estimate for the plotting position is the median of its probability density distribution.” No justification is given by Folland and Anderson (2002) for this idea. They recommend Eq. (2) and deem the so-called Weibull formula (Weibull 1939)

View Expanded

as being “not recommended” because “it gives estimates of return period that are smaller than the other methods.” The statement by Folland and Anderson (2002) is striking in that the Weibull formula in Eq. (3) is generally used and may be considered as an essential part of the standard Gumbel extreme value method (e.g., Gumbel 1958; Cook 1982, 1985; Cook et al. 2003). Obviously, more convincing arguments to support a plotting formula and, indeed, a unique solution of the problem are in demand.

3. The correct probability positions for estimating return periods

Consider a variable x that has a probability density function f (x) and cumulative distribution function F(x). Then, a new variable F(x_m) related to x by order ranking from the smallest (m = 1) to the largest (m = N) value will have the probability density f_m[F(x_m)] given by

View Expanded

where F(x_m) is the cumulative distribution function of the order-ranked values [0 ≤ F(x_m) ≤ 1]. Thus, f_m[F(x_m)] is the probability of observing the mth-order statistic_, and F(x_m) is the probability that x takes a value less than or equal to a value x_m associated with m. Equation (4) can be derived by several approaches (Gumbel 1958; Castillo 1988; Harris 1996; Folland and Anderson 2002). The notations used here are those used by Folland and Anderson (2002), except that the order-ranked variable is denoted by F(x_m) instead of y_m.

As shown by Gumbel (1958), the expected value E of the variable F(x_m), which has the probability density given by Eq. (4), is equal to

View Expanded

The expected value E[F(x_m)] is the mean of F(x_m). Thus, the plotting position for a rank m, as given by the Weibull formula Eq. (3), is the mean value of the parent probability distribution F(x_m) associated with the rank m. It is the mean in the following sense. When there are L independent sets of N samples taken from the same parent distribution, then there will be L individual mth-ranked values of x_m. Equation (5) derived above shows that the mean value of F(x_m) taken over the ensemble of L values converges, for a large L, to the value of m/(N + 1).

For the reason above, the expected value E[F(x_m)] given by Eq. (5), that is, the mean, has been widely considered as the unbiased estimate for the plotting position (e.g., Cunnane 1978; Harris 1996). Folland and Anderson (2002), however, suggested that the median of F(x_m) should be used instead. A mathematical proof is given in the following for the mean of F(x_m) as the correct estimate for the plotting position in the extreme value analysis of return periods.

Let us define P_e = 1 − P as the probability of exceedance of the mth smallest observation in the past N trials. Then, following Castillo (1988, 13–14), the probability w of observing r exceedances in the future n trials is given by

View Expanded

The mean of the binomial variable in Eq. (6) is nP_e. Therefore, the mean number of exceedances r is

View Expanded

Taking into account the total probability rule, and that the mean of the mth-order statistics F(x_m) is given by Eq. (5), the mean number of exceedances in Eq. (7) becomes

View Expanded

Let us now return to the exact definition of the return period R. Let A be an event and T be the random time between consecutive occurrences of A events. Then, the mean value of the random variable T is called the return period R of the event A. It follows from this definition that the mean number of events A in an observation period that is equal to T is 1. An event A is here defined so that a random observation, say an annual extreme value, exceeds a value of x. Then, for the mean number of exceedances r,

View Expanded

Combining Eqs. (8) and (9) gives

View Expanded

In terms of the probability of exceedance P_e = 1 − P, the return period in Eq. (1) becomes R = 1/P_e, and one gets from Eq. (10)

View Expanded

The notations of Folland and Anderson (2002), where the ranking is done is ascending order and the plotting position is defined as the probability of nonexceedance P, have been followed above. Solving P = 1 − P_e from Eq. (11) yields

View Expanded

which is Eq. (3). Equation (12) also results if, instead, the order ranking is done in descending order and the probability of exceedance is considered, the combination of which is a common practice in many applications. It is noteworthy that the result derived above is independent of the underlying distribution f (x).

In summary, it was shown above that when estimating the return period R, the correct plotting position is obtained by the mean of F(x_m), that is, by using Eq. (3). Hence, in the analysis of the return period the other suggested plotting formulas, such as Eq. (2), are incorrect.

4. Plotting positions involving a reduced variate

Most of the plotting formulas suggested historically are, however, not intended to be used for plotting the cumulative probability or the related return period R on arithmetic paper. Instead, they are used when plotting on paper where the probability scale is transformed in order to obtain a linear fit that is convenient to extrapolate. For example, on a Gumbel plot (Fig. 1), the probability scale is transformed into the reduced variate η = −ln(−lnP) = −ln[−ln(1 − 1/R)].

In the classical Gumbel analysis, Eq. (3) is used so that the probability P is approximated by its mean. A nonlinear transformation is then applied to that mean. Kimball (1960), Gringorten (1963), Cunnane (1978), and Harris (1996) have argued that when the plotting involves a reduced variate, a more correct procedure would be to apply the transformation first and then plot the mean value E(η_m) of the reduced variate η_m defined that way. This results in plotting formula of the type

View Expanded

where η⁻¹ is the inverse function of the transformation that gives η. The plotting positions based on Eq. (13), in contrast to those based on Eq. (12), depend on the transformation and, hence, on the postulated parent probability distribution function f (x). The various distribution-tailored plotting formulas and methods presented in the literature reflect this situation, that is, it is believed that the plotting positions for estimating the return periods depend on the underlying distribution when a reduced variate is involved in the analysis.

It was shown above in section 3 that Eq. (12) associates the cumulative probability P to the mth rank in N samples. This fundamental relationship can be expressed in terms of the return period R as Eq. (10). Suppose that we have N years of observation of, say, annual extremes, then in the analysis of these N maxima Eq. (10) provides a unique relationship g. Let us denote this relationship by g, so that

View Expanded

As shown in section 3 by deriving Eq. (10), the relationship g is independent of the underlying distribution. Thus, Eq. (14) underlines that there exists a fundamental connection between the rank of an observation and the estimate of its return period. This relationship is presented in quantitative form in Eq. (10). To estimate the return period, we may plot the N maxima on arithmetic paper using R as the ordinate by applying R = g(m) and try fitting some curve to the points thus plotted. As discussed above, the alternative and more commonly used method is to transform the scale on the ordinate axis so that the points plotted would better fall on a straight line.

Clearly, the fundamental distribution free relationship g that associates the return period R with a rank m cannot be affected by the fitting method. In other words, the plotting positions given by Eq. (10) must not be manipulated based on an arbitrary choice of the scale on the ordinate axis of the graph that is devised to merely alleviate the analysis of the data. Hence, in estimating R, any deviation from the use of the Eq. (10) by applying a plotting formula other than Eq. (12), based on some presumed statistical model, is misuse of the data. The fitting procedure may reflect the scale used, but the probability positions of the data must be the same regardless of the method of fitting. In other words, the plotting formula P = m/(N + 1) is valid regardless of the transformation made.

It is, therefore, concluded that the approach leading to the distribution-specific plotting formulas through Eq. (13) is both unnecessary and incorrect when analyzing return periods. Because this concept has been persistent in the literature for many decades, it is of interest to discuss in detail the origins and nature of the errors involved.

First, confusion has been caused by the temptation to obtain a good linear fit for easy extrapolation on probability paper. This has misled many researchers to manipulate the plotting positions to that end. The comments by Blom (1958 that “a condition to be satisfied by any plotting formula is that the points must lie on the average on a line which deviates only little from a straight line,” and by Castillo (1988) that “the plotting position formulas can affect the linear trend of the cumulative probability distribution so that a careful selection must be made,” illustrate this confusion. Manipulation of the plotting positions in order to obtain a linear fit can be identified as a failure to properly separate the two different procedures required in the data analysis; one must first determine the probability positions, which are independent of the distribution, and only then make transformations hoping to obtain linearity in relation to some model distribution and a good fit to the plotted data. In other words, one should not fit the observations to a model, but fit a model to the observations.

Second, the argument given to justify Eq. (13)—“It is not the probability ordinate that is plotted but the reduced variate” (Harris 1996)—is misleading. This is so because transforming to the reduced variate is merely a method to manipulate the probability scale of the graph, so that a parameter η that appears linear on the ordinate is obtained. In the classical Gumbel analysis it is the probability P that is being plotted, but now on another scale. The transformation then associates E[F(x_m)] to m. Kimball (1960), Gringorten (1963), Cunnane (1978), and Harris (1996, 1999, 2000), on the other hand, plot the reduced variate η by making the transformation before plotting, that is, by associating E(η_m) to m. However, it was shown in section 3 that the foundation for the use of the mean E[F(x_m)] is merely that the return period R is defined as the mean time period T between events that exceed F(x_m). There exists no justification for the use of the mean of the cumulative probability function of the mth-ranked value as the plotting position when the variable is something else than F(x_m). To be useful at all in estimating R, that parameter must be a result of an operator that retains the fundamental relationship in Eq. (14). Hence, it must be such that its application to the distribution of η_m rescales to the mean of F(x_m), that is, to P. This redirects the plotting to the use of P and Eq. (3) in the first place.

Third, in the approach of plotting the reduced variate the transformation is from F(x_m) to E(η_m). This is different from that of the classical Gumbel analysis, in which the transformation is from E[F(x_m)] to η_m because the result of taking a mean and making a nonlinear transformation depends on the order in which these operations are applied. Consequently, the linearity, shown by Gumbel (1958) to exist as a result of plotting E[F(x_m)] by Eq. (3), is lost when E(η_m) is being plotted. This linearity can be returned if one knows the underlining distribution, as shown empirically by Cunnane (1978) and theoretically by Harris (1996). However, this can only be done by manipulating the plotting positions, that is, by violating Eq. (12). Such manipulated plotting positions no more correspond to the probability P that is required to estimate the return period. The error thus made can be described in mathematical terms as follows. By an axiom of probability calculus, a sample probability P is additive. Consequently, its nonlinear transformation η is nonadditive. The best estimate of a sample parameter is its mean value only if that parameter is additive. Hence, keeping in mind that P is being estimated, the transformation must be made in such a way that the mean is taken over P, not over η, that is, the transformation to a reduced variate must not be made before taking the mean.

In summary, in order to use Eq. (1) in the case of order-ranked data, the cumulative probability P in it must be defined as the mean of F(x_m) in an infinite ensemble of ranked observations, each including N samples. The variable η⁻¹[E(η_m)] in Eq. (13), which is a retransformation of the mean of the nonlinearly transformed F(x_m) values, does not meet this definition. Thus, estimation of return periods based on order-ranked data is not possible by interpreting the reduced variate as being transformed before plotting. Consequently, the concept of distribution-specific plotting formulas in analyzing return periods should be abandoned. This causes no problems to the analysis, however, because the Weibull plotting formula P = m/(N + 1) is to be used regardless of the underlining distribution.

5. Discussion and conclusions

It was shown above in section 3 that the Weibull plotting formula P = m/(N + 1) directly follows from the definition of the return period R. Thus, proof was given for Eq. (3) as the correct plotting formula when the return periods are being analyzed by the extreme value method. The proof is valid for any underlying continuous distribution f (x).

It was further pointed out in section 4 that, because P = m/(N + 1) associates the mth-ranked value of x with the cumulative probability and the related return period R in a fundamental way, this relationship holds regardless of the transformations made in the extreme value analysis. Consequently, the various other methods for determining the plotting positions, suggested during the last 90 years, such as the formulas by Blom, Jenkinson, and Gringorten, the computational methods by Yu and Huang (2001), as well as the modified Gumbel method, are incorrect when applied to estimating return periods.

As can be seen in Fig. 1 and in Folland and Anderson (2002), the errors resulting from the use of such incorrect methods are very large. The error resulting from the use of Hazen's formula [used, for example, by Castillo (1988)] can be approximated by Fig. 1. For the wind speed of 35 m s⁻¹ Hazen's formula predicts R of approximately 200 yr instead of the 90 yr predicted by the correct plotting formula, that is, Eq. (3). Jenkinson's formula, supported by Folland and Anderson (2002), predicts R of 35 m s⁻¹ to be about 130 yr in the case of Fig. 1.

An implication of the overall error that results at high extremes is obtained by simply considering the return period R that is predicted through Eq. (1) by the different plotting formulas for the largest extreme in the sample, that is, when m = N. In Table 1 such a comparison is shown for a sample of 21 annual maxima [this period is chosen because for that the numerical result by Harris (1996) is available]. Table 1 shows that an error of more than 70% in the return period of the largest observed extreme is obtained by using both the Gringorten formula, which has been used in the analysis, particularly when utilizing the generalized Pareto distribution (Hosking et al. 1985; Hosking and Wallis 1995; Linacre 1992; Brabson and Palutikof 2000) and the modified Gumbel analysis method (Harris 1996, 1999, 2000).

From the point of view of estimating the risks of extreme weather phenomena in the present and future climates these errors are very serious because overestimating the return period equals underestimating the risk. Because the present estimates of many important weather-related risks are partly based on the conventional methods that have been shown here to be invalid, comprehensive reanalysis of them is suggested. This may make it worthwhile to reevaluate the related building codes and regulations.