Given the reality of anthropogenic global warming, it is tempting to seek an anthropogenic component in any recent change in the statistics of extreme weather. This paper cautions that such efforts may, however, lead to wrong conclusions if the distinctively skewed and heavy-tailed aspects of the probability distributions of daily weather anomalies are ignored or misrepresented. Departures of several standard deviations from the mean, although rare, are far more common in such a distinctively non-Gaussian world than they are in a Gaussian world. This further complicates the problem of detecting changes in tail probabilities from historical records of limited length and accuracy.
A possible solution is to exploit the fact that the salient non-Gaussian features of the observed distributions are captured by so-called stochastically generated skewed (SGS) distributions that include Gaussian distributions as special cases. SGS distributions are associated with damped linear Markov processes perturbed by asymmetric stochastic noise and as such represent the simplest physically based prototypes of the observed distributions. The tails of SGS distributions can also be directly linked to generalized extreme value (GEV) and generalized Pareto (GP) distributions. The Markov process model can be used to provide rigorous confidence intervals and to investigate temporal persistence statistics. The procedure is illustrated for assessing changes in the observed distributions of daily wintertime indices of large-scale atmospheric variability in the North Atlantic and North Pacific sectors over the period 1872–2011. No significant changes in these indices are found from the first to the second half of the period.
There is great interest in how global warming might be affecting the statistics of extreme weather (Meehl et al. 2000; IPCC 2014). Indeed the occurrence of any extreme event not previously observed “within living memory” or “since records began” (in both cases, about 100 years) immediately becomes a candidate for attribution to global warming. Both observational and modeling approaches are taken to the problem. A combined approach is often deemed necessary, given the limitations imposed on observational approaches by the shortness and inaccuracies of historical records, and on modeling approaches by known and unknown model errors.
It is not hard to see how detecting changes in extreme value probabilities using limited records can be problematic. Consider an event with an N-yr return period, whose probability of occurring in any single year is 1/N if the annual occurrences are independent, and of not occurring is 1 − 1/N. The probability of at least one occurrence in M years is then 1 − (1 − 1/N)M, and of exactly one occurrence is [M/(N − 1)] (1 − 1/N)M. These simple formulas highlight the difficulty in pinning down the probability (or equivalently the return period N) of rare events from short records. For example, a single occurrence of an event in an M ~100-yr record does not imply that N ~100 yr. Actually N could be as long as 2000 years, or as short as 22 years, because (using the binomial distribution) there is a 5% chance for a 2000-yr event as well as a 22-yr event to occur just once in 100 years. Similarly, the complete absence of an event in a 100-yr record does not imply that N > 100 yr, as is often assumed. It could be as short as 44 years, because there is a 10% chance for a 44-yr event to occur not even once in 100 years. Thus, although it might be tempting to conclude from the absence of an event in a 100-yr interval and its single occurrence in the subsequent 100-yr interval that its return period N has decreased from “infinity” to 100 years and the event has become “more likely,” the large uncertainties in N would render such a conclusion suspect. Indeed there is an appreciable probability [M/(N − 1)] (1 − 1/N)2M for such a sequence of events to occur with no change in N at all, which is greater than 10% for 80 < N < 800 yr. Similar statements can be made regarding events that occur a few times in one century, and a few more (or less) times in the next.
Given the rarity of extreme events, it is not surprising that detecting significant changes in their probabilities through direct counts or raw (or cosmetically smoothed) histograms is prone to sampling errors. There are three ways around this difficulty. Approach 1 is to fit a probability distribution, usually of a universal form based on generalized extreme value (GEV) theory, to the sampled extreme values. Although more robust than using raw histograms, this approach is also prone to sampling errors, and the links between the parameters of the GEV distribution and underlying physical processes are often unclear. Approach 2 is to fit a probability distribution to all and not just the extreme values, whose more physically transparent parameters related to the mean and variance can also be more robustly estimated from limited records, and then to deduce changes in tail probabilities from the changes in those parameters. This approach is appealing from both sampling and physical perspectives, but can be compromised by using an inappropriate probability distribution, such as a Gaussian in cases where the distribution is clearly not a Gaussian. Approach 3 is to use a numerical climate model to estimate the tail probabilities from raw histograms derived from large ensembles of model integrations. This approach can be made as statistically reliable as desired depending upon computing resources, but may be compromised by known and unknown model errors. All three approaches, individually or in combination, are currently being followed to detect changes in extreme weather and climate risks and to attribute them to natural and/or anthropogenic influences.
Our first goal here is to highlight the need for such detection and attribution efforts to account for the distinctively skewed and heavy-tailed aspects of the probability distributions of daily weather anomalies. As discussed below, departures of several standard deviations from the mean, although rare, are far more common in such a distinctively non-Gaussian world than they are in a Gaussian world. In such a world a mean climate shift is also generally associated with changes in the width and shape of the probability distribution. Consequently, even the sign of the changes in tail probabilities cannot be inferred unambiguously from the mean shift alone. These realities further complicate the problems of detecting significant changes in tail probabilities from historical records of limited length and accuracy. In effect they make it harder to reject a null hypothesis of no change in extreme risks. They also raise the bar for the detection and attribution of changes in extreme risks using climate models, by requiring the models to accurately represent the salient non-Gaussian aspects of the observed distributions.
Our second goal is to suggest a simple way to account for the observed non-Gaussianity by using, as a null hypothesis, a general class of non-Gaussian distributions called stochastically generated skewed (SGS) distributions introduced to the meteorological literature by Sardeshmukh and Sura (2009, hereafter SS09). Sardeshmukh and Penland (2015) further discuss the broad physical foundations of such distributions in the climate system. Here we provide extensive justification for the particular relevance of SGS distributions in extreme value analysis. In addition to capturing the distinctive non-Gaussianity of the observed distributions, SGS distributions have two other notable properties: 1) they include Gaussian distributions as special cases, and 2) they remain SGS distributions following a mean climate shift. We describe how to estimate the parameters of an SGS distribution using the first four statistical moments of the data (mean, variance, skewness, and kurtosis) and, importantly, how to generate a statistically equivalent Markov process with the same parameters, whose physical interpretation is more transparent. The Markov model can be used to test for statistical significance through Monte Carlo simulations, and also to investigate changes in the temporal persistence statistics of extreme anomalies. Our approach may be viewed as an attempt to improve the full distribution–based approach 2 to the analysis of extreme anomalies. It also has implications for the extreme value distribution–based approach 1, and the climate model distribution–based approach 3, as will become clear below.
Figure 1 illustrates how non-Gaussianity can strongly impact tail probabilities, and how ignoring it can compromise the detection, quantification, and attribution of their changes even when they are associated with only a mean shift. The figure compares the probabilities of exceeding 2 and 4 standard deviations (sigma) before and after a positive mean “climate shift” of 1 sigma, for Gaussian and SGS distributions with the same mean and sigma, and a skewness of 1 and excess kurtosis of 5 for the SGS distribution. Before the shift, the probabilities of exceeding 2 sigma are 2.3% and 3.4% for the Gaussian and SGS distributions, respectively. After the shift, they increase to 15.8% and 13.3%, by different factors of 7 and 4. For exceedances of 4 sigma, the changes are more dramatic: from 0.003% (0.34%) before to 0.13% (1.0%) after the shift, and by very different factors (43 and 3 for Gaussian and SGS distributions, respectively). The detection and attribution of such changes would be strongly affected by what one assumes to be the true distribution. If the distribution is actually SGS but is assumed to be Gaussian (i.e., if the non-Gaussianity is ignored), then the occurrence of a 4-sigma event would be interpreted as highly significant (since its preshift risk of 0.003% is very small) and highly consistent with the 43 times higher risk of such events after the mean shift. One would thus, with high confidence, both reject a null hypothesis of no change in the risk of such events and accept the alternative hypothesis of their much higher risk in the shifted climate. If, on the other hand, the distribution is correctly recognized to be SGS, then the occurrence of the 4-sigma event would be interpreted as relatively less significant and less clearly attributable to the climate shift, given its already substantial risk of 0.34% before the shift (which is indeed higher than the 0.13% risk in the Gaussian case even after the shift) and its relatively small threefold increase of risk after the shift. Note that no such disparity of interpretation would occur if the distribution is actually Gaussian but is assumed to be SGS, since a Gaussian is a special case of an SGS distribution with zero skewness and excess kurtosis, as already mentioned.
Granting that there may be substantial disparities in the magnitude of the estimated changes in extreme risks, one might still argue that their sign would be correctly deduced in all cases in the above example from the sign of the mean shift, using the popular rule of thumb that a positive mean shift increases the risk of positive extremes and decreases the risk of negative extremes. This rule, unfortunately, is generally valid only if the mean shift is not accompanied by a change in the width or shape of the distribution. It is easily shown for Gaussian distributions, for example, that if a mean shift is accompanied by any decrease of standard deviation, then the risks of sufficiently large positive and negative extremes both decrease, regardless of the sign and magnitude of the mean shift. More generally, it can be shown (see appendix A) that for shape-preserving changes of any probability density function (PDF) p(x) with a location parameter μ0 (usually but not always identified with the mean of x) and a scale parameter σ0 (usually but not always identified with the standard deviation of x), the sign of the change in probability of exceeding high (x > XT) and low (x < 2μ0 − XT) thresholds is given by the sign of an extremal probability shift index (PSI), defined as
where μ and σ are the location and scale parameters after the change, and the + (−) sign refers to the change in probability of high (low) extreme values. It is obvious from this equation that if σ ≠ σ0, the second term on the right-hand side dominates for sufficiently large XT and yields the same sign for ψ(+) and ψ(−). This is essentially a restatement and extension of Katz and Brown’s (1992) result that “variability is more important than averages” in influencing changes in tail probabilities; that is, the second term can be more important than the first. Our extension is to combine these terms into a single predictive measure ψ in (1) for the sign of the change in tail probabilities. Applying it to the Gaussian case in Fig. 1, for example, shows that the 43-fold increased risk of 4-sigma events associated with a mean shift of 1 sigma could be completely nullified by a 25% reduction of sigma. We stress that ψ is applicable to a wider class of distributions than Gaussian distributions. We also note that the signs of changes in the tail probabilities of x are identical to those of any other variable y that is a monotonically increasing function of x. So even if the changes of p(y) are themselves not shape-preserving, the sign of changes in the tails of p(y) may be inferred through an appropriate transformation of variable and using (1) for the transformed variable, making ψ applicable to an even wider class of distributions. If y is precipitation, for instance, then given the monotonic relationship between precipitation and midtropospheric vertical velocity, one could use ψ to deduce changes in the sign of extreme precipitation probabilities associated with changes in vertical velocity, if the vertical velocity has a Gaussian or some other shape-preserving distribution.
Despite this wider applicability of ψ, one can imagine situations in which knowing even the mean and standard deviation would be insufficient to deduce the sign (let alone the magnitude) of changes in tail probabilities, if the distribution is not reducible to a shape-preserving form through a transformation of variable. Situations involving changes of strongly skewed and kurtotic SGS distributions belong to this category. In such situations the full distribution–based approach 2 and the climate model distribution–based approach 3 to determining tail probabilities are further compromised by ignoring or misrepresenting the SGS character of the observed distributions, eroding their advantages over the extreme value distribution–based approach 1.
The paper is organized as follows. Section 2 highlights the distinctive non-Gaussianity of the observed distributions of daily large-scale weather anomalies. Section 3 shows how this distinctive non-Gaussianity is captured by SGS distributions, and describes procedures for fitting SGS distributions to data and generating a statistically equivalent Markov process with the same parameters. Section 4 discusses the strong links between the distributions of midtropospheric vertical velocity (which are well approximated as SGS distributions) and precipitation anomalies. Section 5 shows how some striking observed asymmetries in the temporal persistence of positive and negative circulation anomalies are consistent with the SGS nature of the anomaly distributions. Section 6 shows how fitting SGS distributions to all available values in limited climate records yields extreme value distributions of block maxima with smaller sampling uncertainties than GEV distributions fitted to only the block maxima. This section also shows how the tails of SGS distributions can be directly linked to the corresponding asymptotic GEV distributions of block maxima and generalized Pareto (GP) distributions of threshold exceedances based on extreme value theory. Section 7 describes how a mean climate shift alters both the width and shape of an SGS distribution, and illustrates their combined impact on tail probabilities to contrast with the impacts of the mean shift alone shown in Fig. 1. Finally, section 8 presents an illustrative application of our SGS distribution–based approach to assess changes in the observed distributions of daily indices of large-scale wintertime atmospheric circulation variability over the North Atlantic and North Pacific Oceans, the North Atlantic Oscillation index and the North Pacific index, over the period 1872–2011. Section 9 follows with a discussion and concluding remarks.
2. The distinctive non-Gaussianity of observed daily weather anomalies
Figure 2 shows hemispheric maps of the skewness S and excess kurtosis K of the wintertime daily anomalies (departures from long-term averages) of upper tropospheric relative vorticity, lower tropospheric temperature, and midtropospheric vertical velocity, defined for an anomalous quantity x as
where , is the variance, and angle brackets refer to long-term averages. The maps are based on the Twentieth Century Reanalysis dataset (20CRv2; Compo et al. 2011) for the 1872 to 2011 period. Similar maps derived using other, shorter, datasets have been presented for these and other atmospheric variables by, for example, SS09 and Perron and Sura (2013). Note that S and K are zero for Gaussian distributions. The maps in Fig. 2 show interesting spatial structure near the midlatitude jets and over the polar region, which will be investigated elsewhere. Our focus here is on the fact that the distributions are not only non-Gaussian, but non-Gaussian in a distinctive way. In particular, for all three variables, regions of large positive or negative S generally coincide with regions of large positive K. The left panels in Fig. 3 depict this relationship more clearly, through joint histograms of the S and K values from Fig. 2. An approximately parabolic K–S relationship, K > (3/2)S2, is evident. This relationship for daily atmospheric variability was first presented and investigated by SS09. Although not our focus here, a similar relationship for daily sea surface temperature (SST) variability was also noted and investigated by Sura and Sardeshmukh (2008).
The right panels of Fig. 3 highlight another distinctive feature of the observed non-Gaussianity. They show PDFs of the transformed standardized variables y = (x/σ)S/|S| from Fig. 2, estimated as histograms at each grid point and averaged over the hemisphere. The local factors S/|S| (equal to +1 or −1 at each grid point) ensure that p(y) is positively skewed at each grid point regardless of whether p(x) is positively or negatively skewed. The values of p(y > 0) and p(y < 0) are shown as blue and red curves, respectively, with the p(y < 0) curve flipped over to the positive side for easier comparison with the p(y > 0) curve. The two curves would coincide if the PDFs were symmetric. Note that the asymmetry is substantial even for small magnitudes of y (i.e., the PDFs do not have a Gaussian core) and changes sign at a “crossover” point between |y| = 1.41 and 1.73, beyond which the tail of p(y > 0) becomes heavier than that of p(y < 0). It is interesting that this heaviness is already evident at moderate anomaly magnitudes of 2 sigma that are often used as thresholds for defining extreme anomalies and assessing their statistical significance. Such heavy tails clearly need to be accounted for in extreme value analysis.
3. SGS distributions and corresponding Markov processes
The distinctive K–S relationship, asymmetry, and heavy tails of the observed distributions highlighted in Fig. 3 are well captured by SGS distributions. An SGS distribution has a PDF of the form
with parameters E, g, and b, and the normalization constant ,
which ensures that p(x) integrates to unity. Here ν = 1/E2, q = 2gν/b, and Γ(z) is the standard gamma function of complex argument z. The PDF has a unique maximum at xmax = −Eg/(1 + E2).
The moments of the distribution are
where the K–S inequality K > (3/2)S2 in (5b) results from the fact that the first term within square brackets on the right-hand side is greater than 1 and the second term is greater than 0 (SS09). The left panels of Fig. 3 are generally consistent with this inequality.
It is clear from (5a) that both E and g must be nonzero for S to be nonzero. Note that E is nondimensional, and g and b have the same dimension as x (and σ). Also, without loss of generality, E and b are positive, whereas g can be positive or negative. Thus the sign of g determines the sign of S.
As already mentioned, SGS distributions include Gaussian distributions as special cases. Although not obvious from (3), these correspond to cases of E = 0. Then both S and K are zero from (5), as for a Gaussian distribution. The convergence of the entire PDF to a Gaussian PDF as E → 0 becomes more obvious upon expressing the PDF of the standardized anomalies using (3) as
where and as E → 0. Note that from (4). For small E, the first term on the right-hand side of (6a) dominates and the PDF approximates a standard Gaussian. The second term, whose magnitude is proportional to S, represents a leading-order departure from Gaussianity, and provides a simple explanation for the distinctive asymmetry crossover of the observed distributions in Fig. 3. It shows that for a positively skewed distribution, is larger (smaller) than for larger (smaller) than 3, and vice versa for a negatively skewed distribution. The predicted asymmetry crossover at is close to the observed values in Fig. 3. This first-order approximation for small E is valid for . Including the second-order terms in (6) gives an even closer match of the PDF and the predicted crossover at with respect to the observations.
SS09 discussed other distinctive non-Gaussian features of SGS distributions, such as a cubic relationship between the fifth moment and S, and tails that decay approximately as a power of x. This is easy to see from (3). For , p(x) decays approximately as
where is a constant, and the + (−) sign applies to positive (negative) tails. SS09 provided examples of such approximate power-law tails in the observed PDFs of upper-tropospheric vorticity and midtropospheric geopotential height anomalies. Note that the inference of such tails from (3) does not require even the variance to exist. Penland and Sardeshmukh (2012) indeed used this fact to suggest an alternative interpretation of power-law distributions encountered in many natural contexts as possibly arising from SGS processes, instead of Lévy processes as is often assumed.
Both g and E are important in determining the skewness and heavy tails of SGS distributions. This is illustrated in Fig. 4, which contrasts the tails of two SGS distributions with the same mean, variance, and E, but one (the same as in Fig. 1) with g ≠ 0 and the other with g = 0. The positive tail is much heavier, and the negative tail much lighter, in the g ≠ 0 case. Note, however, that even the negative tail is much heavier than that of a Gaussian distribution with the same mean and variance, also shown in the figure, beyond about 4σ. Interestingly, a pure power-law decay of the PDF does not obtain until well beyond 10σ, which makes it difficult to establish from even very long time series, as indicated by the gray curves obtained from a long integration of a Markov process (described below) with the same PDF.
There are several ways to estimate the parameters of an SGS distribution from a sample series of x. A simple “method of moments” is to specify the sample σ2, S, and K in (4b) and (5) and solve for E, g, and b as
A necessary and sufficient condition for both E2 and b2 to be positive is that , where . If E2 estimated from (8a) falls below this lower bound, the value of K specified in (8a) is increased until it exceeds the bound, and steps (8b) and (8c) are repeated.
One can fit an SGS distribution to any sample data series using this simple robust method. It may be a poor fit if the sample distribution has more than one peak, or if the sample E2 estimated from (8a) strongly violates the constraints on E2 for an SGS process. The degree to which the fitted SGS distribution differs significantly from the sample distribution may be assessed through Monte Carlo integrations of the SGS process model described in (10) below. It is possible that other methods of SGS parameter estimation and their significance, such as maximum likelihood estimation (MLE) and Markov chain Monte Carlo (MCMC) methods might be superior to our method, especially given the explicit forms of (3) and (10). Although we have not considered it here, this issue is certainly worthy of further exploration.
The constraint on E2, if exists up to some n ≥ 4, implies stricter bounds on S and K than is obvious from (5). Specifically,
The fact that these stricter bounds are also generally satisfied in the left panels of Fig. 3 for n = 5 provides additional evidence for the relevance of SGS distributions, and for the existence of the fifth moments of those variables. The upper bounds for S are generally satisfied even for n = 6, but those for K are not, especially for vertical velocity at many gridpoints. This suggests that the sixth moment of vertical velocity generally does not exist, and that its PDF has relatively heavy tails.
The physical basis for SGS distributions, as discussed by SS09, is their association with a stochastically perturbed damped linear Markov process x(t) whose change dx in a small time interval dt is given by
where η1 and η2 are temporally uncorrelated random Gaussian variables with zero mean and unit variance, and λ > 0 is a damping time constant. [Note that (10) is identical to Eq. (5) of SS09 with a different notation, with E, g, and b in that equation scaled by here, and A in that equation equal to −[1 + (1/2)E2]λ here]. The stationary PDF of this process is identical to p(x) in (3), with identical parameters E, g, and b. Its lag-autocorrelation function is ρ(τ) = . Note that λ does not affect the stationary probability density p(x), since it can be used to standardize time in (10) as t′ = λt. It does, however, crucially affect the temporal evolution and hence the persistence statistics of x. We specify its value as λ = 1/τc, where τc is the temporal correlation scale of x estimated using ρ(τ).
The terms including in (10) represent Stratonovich stochastic forcing: these consist of bη1 and gη2 as “additive” and Exη2 as “multiplicative” noise forcing terms. The combined (Ex + g)η2 term was called correlated additive and multiplicative noise (CAM noise) by SS09. The skewness of p(x) arises from the asymmetry of this term with respect to the sign of x. It immediately makes clear why E and g must both be nonzero in (5a) for S to be nonzero.
This statistically equivalent Markov process model (10) is useful for quantifying the sampling uncertainties of SGS distributions, with a proper accounting of the “serial dependence” (i.e., the temporal correlation) of x. Long integrations of (10) also provide information on the persistence statistics of extreme values of x that is difficult to obtain in any other way.
Figure 5 illustrates the dramatically different behavior of extremes in the Gaussian and SGS (S = 1, K = 5) cases of Fig. 4 from a time series perspective. It compares the daily time series of x(t) in the two cases in long integrations of the Markov model (10) using a Heun method. The length of the integrations is 108 correlation time units (ctu) of standardized time t′ = λt = t/τc, with a specified τc of 1 ctu and time step of 1/24 ctu. For concreteness, if τc is 1 day, one may think of these as 108-day integrations, with the first 105 days displayed in Fig. 5 as 1000-winter series of 100-day winters. Consistent with the heavier tails in Fig. 4, excursions beyond ±3 standard deviations are more frequent in the SGS case. Note that the two series have the same mean and standard deviation, with little sampling variability in the 1000-winter series of their values estimated from each winter. The figure also shows the 100-decade series of the largest and fifth-largest daily values in each 10-winter decade. These are dramatically different in the SGS and Gaussian cases, with excursions beyond +4 standard deviations far more common in the SGS case. Note also the substantial long-term trends in the series of decadal maxima in the SGS case, lasting a century or even longer, that are associated purely with sampling fluctuations. Consistent with the discussion of Fig. 1, such long-term trends of “record-breaking” values in an SGS world would be mistakenly attributed to “climate change” by adopting an inappropriate null hypothesis that they are drawn from a Gaussian distribution.
4. The link between vertical velocity and precipitation distributions
The fact that the observed distributions of midtropospheric vertical velocity are approximately SGS distributions has important implications for the distributions of precipitation and precipitation extremes. In the simplest approximation, the daily precipitation rate R(t) at a location may be related to the standardized daily vertical velocity w(t) = W(t)/σw at that location as
where β is a proportionality constant that depends on the mean local humidity and thermal structure and H(w) is the Heaviside function that is 0 for negative w and 1 for positive w. Note that here is the full (not anomalous) vertical velocity W(t) standardized by its standard deviation σw. One may think of positive w as the standardized precipitation rate . This simple model allows one to relate precipitation probabilities to w probabilities. In essence, it states that the PDF of precipitation is the (appropriately normalized) PDF of positive w, and the probability of no precipitation is the probability of negative w.
Figure 6 illustrates how the probabilities of w in scenarios of = −1, 0, and +1 are impacted by whether the PDF of w′ is Gaussian or SGS (with S = 1, K = 5 as in Fig. 1). The impact on the probability of negative w (and therefore also on that of positive w) is relatively modest. However, the impact on the probability of extreme positive w (defined as w > 3) is dramatic, with the SGS/Gaussian probability ratio increasing from 1.5 in the mean ascent scenario to 115 in the mean descent scenario.
The correspondence of the PDFs of daily precipitation and positive w through (11) also has implications for the PDFs of seasonal-mean precipitation. Focusing only on the PDFs of positive w in Fig. 6, it is clear that they are positively skewed in both the SGS and Gaussian cases, with the skewness relatively larger in the SGS case, increasing from the mean ascent to the mean descent scenarios. One would expect this skewness to be reduced for seasonal averages in accord with the central limit theorem, but to remain strongly positive in regions of strong mean descent. This is indeed observationally evident in regions of mean descent in Fig. 7. The right panels of Fig. 7 demonstrate the robustness of this relationship in five different observational and reanalysis datasets. One would thus expect the SGS versus Gaussian character of the daily w PDFs to have a strong impact on the extremes of even seasonal-mean precipitation in arid regions of mean descent.
5. Implications for the persistence of extreme anomalies
An important consideration in extreme value analysis is not only the magnitude but also the duration of extreme anomaly events. There is substantial observational evidence for the asymmetry of both with respect to the sign of circulation anomalies (e.g., Dole and Gordon 1983; Masato et al. 2009; Woollings et al. 2010). Figure 8a shows an early example from Dole and Gordon (1983) of the asymmetric persistence of positive versus negative 500-hPa geopotential height anomalies in the North Pacific sector. It shows the counts PM(T) of T-day persistent anomaly events of amplitude M, defined as events in which the anomaly extends beyond and then persists beyond a threshold M for at least T days. Results are shown for M = ±0, ±0.5, and ±1 standard deviations. Large positive anomalies evidently last longer than large negative anomalies, and also longer than large anomalies of a Gaussian red noise process with the same variance and lag correlation. Such asymmetries are usually attributed to a greater role of nonlinearity in the dynamics of large-amplitude anomalies. Note, however, that the figure shows an asymmetry in the persistence of all positive and negative anomalies, not just large-amplitude anomalies, and the positive anomalies overall are actually less persistent than negative anomalies. There is also a striking exponential decay of PM(T) with T in all cases. No simple explanations of such features currently exist.
The general character of such persistence probability curves is remarkably well captured, without invoking nonlinearity, by the CAM noise–driven linear Markov model (10) associated with SGS distributions. Figure 8b shows the PM(T) of x(t) from the 108-day model integrations presented earlier for the SGS (S = 1, K = 5) and Gaussian cases, in an identical format to that of Fig. 8a. Note again that time here is actually measured in correlation time units t/τc, and is referred to as “days” using τc = 1 day purely for illustrative purposes. Results are shown for thresholds of M = ±0, ±1, and ±2 standard deviations, with 95% confidence intervals estimated from the sampling uncertainty among 1000 100-winter blocks (“centuries”) of 100-day “winters.” The similarities with Fig. 8a are striking.
The exponentially decaying curves for both the SGS and Gaussian cases in Fig. 8b are associated with the same expected exponential decay from an initial condition x(t). What distinguishes them are the unconditional probabilities of x(t), as well as the conditional probabilities (not just expected values) of x(t + τ) given x(t). Both of these are altered in the SGS case by the asymmetric CAM noise forcing (Ex + g)η2 in (10). Given the heavier positive tails in this positively skewed SGS case, it is then not surprising that large positive anomalies last longer than large negative anomalies. It is also not surprising that, overall, positive anomalies do not last as long as negative anomalies, because the unconditional probability of positive x [i.e., p(x) in (3) integrated over all positive x] is smaller than that of negative x (see Fig. 6b). Note that the signs of all such asymmetries are reversed for negatively skewed SGS distributions.
The ability of the linear Markov process (10) to represent such temporal persistence statistics—correctly discriminating among anomaly persistence beyond positive and negative, as well as large and small, amplitude thresholds—makes it useful for investigating the statistical significance of a wide variety of changes in weather statistics associated with climate change. It also enables any changes that are established as significant in a variable to be attributed, as a first step, to changes in the deterministic dynamics (encapsulated in λ = 1/τc) versus the stochastic forcing (E, g, and b) of the variable.
6. Extreme value analysis using SGS versus GEV or generalized Pareto distributions
In this section we show that fitting SGS distributions to all available values in limited records yields extreme value distributions of block maxima with smaller sampling uncertainties than generalized extreme value distributions fitted to only the block maxima. We also show how the tails of SGS distributions are linked to the corresponding GEV distributions of block maxima and generalized Pareto (GP) distributions of exceedances over specified thresholds.
The widespread use of GEV distributions in extreme value analysis is motivated by their quasi-universal applicability. If F(x) is the cumulative probability distribution function (CDF) of x, and is the maximum value of x in a sample of N independent values (a “block”) of x, then the CDF of is , which for large N approximates a GEV distribution
where , , and are shape, scale, and location parameters. It is remarkable that F(x) need not be known explicitly to estimate the distribution of in this limit. The question is how large does N need to be for the limit to apply, and given that each block yields only one sample value of , how accurately can the distribution be estimated using the relatively small number J of such blocks in available climate records.
Figure 9 illustrates the large sampling uncertainties of GEV fits when only 25 blocks (“winters”) of data are used. The thick black curves show the “true” PDFs of 100-day block maxima of the Markov process (10) for the Gaussian and SGS cases discussed earlier, determined using 106 such blocks in 108-day simulations. The shaded bands indicate 95% sampling error bars on the PDFs if they are estimated using just 25 blocks. Not surprisingly, the errors are large if only the raw histograms of the 25-block maxima are used. Fitting GEV distributions to the 25-block maxima reduces the errors (although not substantially in the Gaussian parent case), but using F(x) fitted to all 2500 daily values of x reduces them even more. [Note that to avoid ambiguity in specifying the precise number N of independent x values in blocks of length Nd = 100 days, the PDFs in the latter case were not estimated directly as , but from 2000 additional 106-day Markov model runs with SGS parameters estimated from 2000 randomly selected 25-block segments of the full 108-day runs, and processed similarly as in estimating the “true” PDFs.]
The results in Fig. 9 are consistent with the intuition that if the parent distribution F(x) is known, the PDF of can be more accurately estimated using F(x) fitted to all J × Nd values of x in the record than by fitting a GEV distribution to just the J block maxima , in essence because the former approach uses more information. The GEV approach entails a trade-off in splitting the J × Nd record into J blocks of length Nd. Ideally, both J and Nd should be large, but in practice a compromise is necessary. A smaller J increases sampling error, whereas a smaller Nd makes the GEV approximation itself less valid, especially if the parent distribution is Gaussian. As shown in appendix B (Fig. B1), the GEV fit to the “true” PDF of Nd-day block maxima from the 108-day Markov model runs described earlier is much less accurate in the Gaussian parent case than in the SGS parent case for Nd less than 200 days.
The extreme value distribution associated with an SGS distribution approaches a GEV form for large enough block sizes N that in each block occurs in the heavy tail of p(x) in (7). It is then easy to derive an expression for and the corresponding GEV distribution (see appendix C), whose parameters are related to those of the parent SGS distribution as
Note that the shape parameter ξ does not depend upon N. The increase of and with N is associated with the fact that a larger is more likely to occur in a longer block.
Insofar as SGS distributions are relevant, the correspondence between GEV and SGS distribution parameters made explicit by (13) enables the GEV parameters to be interpreted physically through the Markov model (10), and provides insight into their variation with block size N. The role of E, the amplitude of the multiplicative noise in (10), is key, since E completely determines ξ and the dependence of and on N. Most importantly, the upper bound 2/(n − 1) of E2, if exists, implies an upper bound 1/n of ξ through (13a). The fact that ξ in most climatic contexts is estimated to be less than 1/4 (e.g., Kharin and Zwiers 2005) is thus consistent with both the existence of the fourth moment (kurtosis) and the relevance of SGS anomaly dynamics. The key role of E also explains the slow convergence of the extreme value distribution to the GEV form for a Gaussian parent associated with E → 0, since for small E the block size N needs to be relatively large to ensure that occurs in the heavy tail of p(x) in (7). (The results in Fig. B1 are consistent with this.)
Extreme value analysis based on generalized Pareto (GP) distributions circumvents some of the sampling issues of GEV distributions, while still retaining some of their quasi-universal applicability. The idea is to shift the focus from the probabilities of just block maxima to the probabilities of exceedances over specified thresholds XT. According to theory (e.g., Pickands 1975; Smith 1989; Davison and Smith 1990), for “high enough” thresholds the conditional PDF of x, given that x exceeds XT, approaches the GP form
where ξ, , and XT are shape, scale, and location parameters. This quasi-universal form of tail PDFs is posited on a self-similarity requirement of “threshold stability,” which in essence is a requirement that “the tail of a tail should look like the tail.” Note that unlike the GEV distribution, which is a model of the sampling distribution of a function of tail values (their maxima in large blocks), the GP distribution is a model of the distribution of the tail values themselves. In general, the tails need not be of GP form, of course. Even in contexts in which they are of GP form, specifying the threshold XT beyond which (14) is valid can be an important concern, requiring additional assumptions about the distribution of nonextreme values that allow for the possibility that a single distribution may not fit all values (e.g., Tancredi et al. 2006).
Fortunately, it turns out that the tails of SGS distributions are precisely of the GP form. Specifically, for high enough XT that p(x) for x > XT can be approximated by (7), the conditional PDF of x given that x exceeds XT is identical to (14) (see appendix C), with a location parameter XT and shape and scale parameters
The shape parameter ξ is identical to that of the GEV distribution in (13a), but the scale and location parameters, which now depend on XT instead of block size, are different.
As in the case of the GEV distribution, the correspondence between the GP and SGS distribution parameters made explicit by (15) enables the GP parameters to be interpreted physically through the Markov model (10). The role of E is again key, since E completely determines ξ. In the limit of E → 0 associated with a Gaussian parent, both ξ → 0 and → 0, underscoring that fact that the tail of a pure Gaussian (E = 0) is not of the form (7) even for XT → ∞.
This analysis highlights the contrast between the extreme value distribution–based approach 1 and the full distribution–based approach 2 to estimating tail probabilities discussed in section 1. The GEV and GP approaches are clearly valuable when the parent distribution F(x) is unknown, but are unnecessary when F(x) is known, especially if F(x) is associated with a process model such as (10) that can be used to quantify sampling uncertainties and provide physical insight. The issue, of course, is to what extent F(x) is known. We have argued for the relevance of SGS distributions as the simplest physically based prototypes for non-Gaussian distributions that include Gaussian distributions as special cases. Even if F(x) is not strictly SGS but is only approximately so, our analysis suggests a way to improve the estimation of GEV and GP parameters by constraining, for example, the shape parameter ξ to be consistent with (13a), with E estimated from (8a) using the full record and not just extreme values. This utility of the SGS approach would clearly be compromised if F(x) deviates significantly and substantially from an SGS distribution, for instance if p(x) has more than one maximum. Note however that even in such instances the GP distribution may not necessarily provide a more accurate model of tail probabilities.
7. Form invariance with respect to climate changes
The SGS distribution is an anomaly distribution associated with a one-dimensional linear Markov model (10). This model is a highly idealized model of the dynamics of individual system components of the multicomponent nonlinear climate system. A rationale for all the multicomponent linear and nonlinear interactions to be approximately represented by parameters E, g, b, and λ in (10) was given in SS09, and the validity of (10) as a process model for the stationary statistics and some temporal persistence statistics has been provided here. No additional assumptions need be made to retain (10) as an equally valid model of anomaly statistics in an altered climate. To that extent, the anomaly distribution of a variable in the new climate should remain an SGS distribution, with possibly different parameters. This rationale is the same as that for expecting anomaly distributions to remain Gaussian in an altered climate if there is a physical basis for them to be Gaussian in the current climate.
For small climate shifts, the form invariance of SGS distributions can be rigorously derived by interpreting the mean shift as the mean anomalous response to a mean anomalous external forcing fext in (10). Incorporating this constant forcing as in (10) modifies it to
If we now write , with , the equation for the anomalies x′ around the new mean becomes
where . This anomaly equation in the new climate is identical to (10) in the original climate, with the same E and b, but with g replaced by . The anomaly distribution in the new climate is then also an SGS distribution, with a PDF (3) in which g is replaced by .
The change in g implies changes in the variance, skewness, and kurtosis of the SGS distribution through (4b) and (5), and hence changes in the width and shape of the distribution. It also implies changes in the types of persistence statistics shown in Fig. 8b, even without a change in λ.
Figure 10 shows how accounting for the changes in σ2, S, and K modifies the tail probabilities of the shifted SGS and Gaussian distributions in Fig. 1. (For a cleaner comparison we specify the same change of σ2 in the Gaussian case.) The probabilities of exceeding 2 sigma, which were previously stated as increasing to 15.8% (13.3%) in the Gaussian (SGS) cases due to the mean shift alone, are modified to 21.2% (16.7%) after also accounting for the changes in width and shape, and the probabilities of exceeding 4 sigma are modified even more strongly, from 0.13% (1.0%) to 0.92% (2.65%). Correctly accounting for the changes in width and shape in addition to the mean shift is clearly important in quantifying changes in tail probabilities.
We stress that for even small climate shifts, the change in the SGS distribution is not shape preserving. However, the change in the associated GEV distribution is shape preserving, because the constancy of E implies the constancy of the shape parameter ξ in (13a), as has indeed been found (e.g., Kharin and Zwiers 2005), and sometimes even assumed (as in Zwiers et al. 2011). This allows the sign of the change in the GEV tail (i.e., in the “extreme extremes”) to be deduced from the sign of the extremal probability shift index (PSI) in (1), by specifying the changes in the estimated location and scale parameters.
For larger climate shifts, it is possible for not just g but the other parameters in (10) to change as well. We have not yet found a simple way to characterize the combined effect of the changes in E, g, and b on the tail probabilities, both of extremes and extreme extremes. It appears that in this more general scenario there would be no simpler alternative to deducing the changes in tail probabilities from SGS distributions fitted to data before and after the mean shift.
8. An application to assessment of twentieth-century circulation changes
Finally, we present an illustrative application of our SGS distribution–based approach to assess the changes in two sea level pressure (SLP) based indices of large-scale wintertime atmospheric circulation variability in the Northern Hemisphere, the North Atlantic Oscillation (NAO) index and the North Pacific (NP) index, over the period 1872–2011. To this end we use the 56-member ensemble 20CRv2 dataset (Compo et al. 2011), in which the time series of each ensemble member represents an equally likely estimate of global atmospheric evolution over the 140-yr period within observational error bounds.
To provide context, Fig. 11 shows the ensemble-averaged 7-yr running mean anomalies of the NAO and NP indices. Corresponding results obtained using the NCEP–NCAR reanalysis and ERA-40 datasets for shorter periods are also shown for comparison. There are large multidecadal variations in both indices, including an apparent upward trend in the NAO index and a downward trend in the NP index during much of the second half the twentieth century. However, the trends over the full 140-yr period are much weaker or even nonexistent. This has important implications for understanding the atmospheric circulation response to global warming, and casts doubt on inferences about this response drawn in studies that focus only on the second half, or other subsets, of the full record.
Despite the absence of a long-term mean trend in these indices, one might still think it possible for the probabilities of their extreme values to have changed in association with possible changes in the width and shape of their PDFs. We consider such a possibility, even though the argument given in the previous section precludes it for small (in these cases, nearly zero) mean shifts. Since our interest is in daily weather extremes and the large-scale circulation contexts in which they occur, we focus on changes in the PDFs of daily NAO and NP indices. These have autocorrelation time scales τc of approximately 5.2 and 7 days, respectively.
For each one of the 56 members of the 20CRv2 ensemble, we estimated the mean, variance, skewness, and kurtosis of the daily index anomalies separately from the first and second halves (1872–1941 and 1942–2011) of the 1872–2011 record, and used (8) to fit SGS distributions in each half. As Fig. 12 shows, the ensemble spread of the 56 fitted PDFs, associated with observational uncertainty, is much smaller than that of the raw histograms. The ensemble-mean fitted PDFs do show some changes from the first half to the second half of the record. This still leaves open the question, however, as to how statistically significant those changes are given that we used only 70 years of data to estimate the distributions in each half. We addressed this question through 106 winter simulations of the Markov process model (10), with parameters specified from the ensemble-mean fitted SGS distributions for the second half. We identified the sampling uncertainty of the fitted observational SGS distributions with the sampling variability of SGS distributions fitted to 70-yr segments of the Markov model simulations. This sampling uncertainty is indicated as the gray shaded bands in Fig. 12.
Figure 12 suggests that there was not only no change in the mean, but also no statistically significant change in the entire PDFs of the NAO and NP indices from the first to the second half of the period 1872–2011. This is consistent with the theoretical expectation from the previous section that if there is no mean shift, there should be no change in the entire PDF if the PDF is an SGS distribution.
9. Discussion and concluding remarks
The difficulties of detecting significant long-term changes in the atmospheric circulation using historical records of limited length and accuracy are well appreciated in the climate research community. They provide a major impetus for ongoing efforts to improve atmospheric reanalyses and extend them as far back in time as feasible. Indeed the very different trends in Fig. 11 over the last 70 years versus the last 140 years in the first such long-term 20CRv2 reanalysis underscore the danger of drawing conclusions about long-term changes from relatively short records. Given the strong link between daily weather statistics (“storm tracks”) and the ambient flow (Whitaker and Sardeshmukh 1998; Compo and Sardeshmukh 2004), there are evident pitfalls in establishing significant long-term changes in weather statistics as well. Changes in extreme weather statistics are even harder to pin down using short records. Since large-scale weather events typically last a day or longer, and there are only about 9000 days of data for each 3-month season in a 100-yr record, a discussion of changes in extreme weather events that occur only a few times in a century is compromised by large uncertainties in their small probabilities on the order of 0.01%. Discussions of such small (and sometimes even smaller) probabilities indeed force one to confront the Bayesian versus frequentist meanings of probability itself based on “degree of belief” versus counts of outcomes. The issue, at its core, is the meaningfulness of assigning quantitative probabilities to events that have never occurred or have occurred only a few times in a finite record. A claim that the probability of an extreme weather event has “doubled” from 0.01% to 0.02% is of a very different character in this regard, given the likelihood of unrealized outcomes in a 100-yr record, than a claim that it has doubled from, say, 10% to 20% or even from 1% to 2%.
The basic note of caution sounded in this paper regarding quantifiable tail probabilities is that they are strongly affected by the non-Gaussianity of the observed distributions, and that their changes cannot generally be deduced from the mean shifts alone. The skewed and heavy tailed character of the distributions warrants accurate representation not only of the mean changes but also of changes of variance, skewness, and kurtosis for deducing the changes in tail probabilities. This has large implications for the detection, attribution, and projection of changes in extreme weather statistics using limited and inaccurate observations and imperfect climate models.
From a physical perspective, understanding the probabilities of extreme “unusual” values of an atmospheric variable requires an understanding of how such values are generated, whether through amplified mechanisms for generating the “usual” values or through concatenation of “a series of unfortunate events.” In either case the probabilities of the unusual and usual values are physically related. The tails of the probability distribution are thus related to the properties of the entire distribution, including its mean, width (variance), and shape (skewness and kurtosis). To get tail probabilities right, one has to get all of these properties right.
We have presented evidence that the distinctively skewed and heavy tailed character of the observed distributions of daily weather anomalies is basically due to their evolution being perturbed by stochastic noise whose amplitude increases linearly with anomaly amplitude, but asymmetrically for positive and negative anomalies. This statistically accurate description is associated with a precise model of the anomaly probability distribution as a stochastically generated skewed (SGS) distribution generated by correlated additive and multiplicative (CAM) noise forcing. The relevance of SGS distributions is not inconsistent with the notion that additional feedbacks play an important role in the evolution and persistence of some extreme anomaly events. Rather, it suggests that such additional feedbacks are not systematic, and can effectively be treated as a CAM noise forcing in modeling anomaly statistics.
We have described how to fit SGS distributions to data using the first four moments (mean, variance, skewness, and kurtosis), and how to generate a statistically equivalent CAM noise–driven Markov process with the same parameters, which is useful for estimating sampling uncertainties and persistence statistics. A virtue of this approach is that it makes use of all values in a time series to estimate tail probabilities, whose sampling uncertainties are therefore smaller than those associated with GEV and GP approaches that use just the extreme values. We stress that our approach is applicable not only to SGS variables but also to other variables that are monotonically related to SGS variables, such as precipitation via its strong link to midtropospheric vertical velocity. Overall, this approach provides a sharper tool for investigating the statistical significance of observed changes in extremes over the twentieth century and of projected changes over the twenty-first century.
Current statements concerning changing extreme weather risks associated with global warming mainly involve quantities related to surface temperature (IPCC 2014) and essentially amount to simple shifts of existing PDFs. Such statements downplay the fact that there is more to regional climate change than surface warming, and that assessing the changing risks of extreme storminess, droughts, floods, and heat waves requires accurate model representations of multidecadal and longer-term changes in the large-scale modes of natural atmospheric circulation variability and the complex nonlinear climate–weather interactions associated with them. The detection of changes in such modes from the limited observational record is much less clear cut than for surface temperature. Figure 12 demonstrated this for indices of large-scale modes of wintertime atmospheric circulation variability in the North Atlantic sector (the NAO index) and the North Pacific sector (the NP index). We found no significant changes either in the mean or in the entire PDFs of these indices over the last 140 years. This is of course not to imply that there have been no changes in other modes of circulation variability in other parts of the globe, just that any such changes need to be carefully evaluated on a case-by-case basis after due consideration of the non-Gaussian aspects of the probability distributions.
To detect, attribute, and make credible projections of changing extreme weather risks in a changing climate using climate models, the models need to represent not only the shifting means but also changes in the width and shape of the distributions of daily weather anomalies. Model misrepresentations of these changes can cast doubt on such efforts. Indeed current models have difficulty in representing even the mean changes, such as their inability to adequately capture the post-1998 “hiatus” in global warming (Fyfe et al. 2013). The mean errors are relatively large on regional scales, in substantial part because of model difficulties in capturing the pattern of tropical ocean temperature changes and their global impacts (Shin and Sardeshmukh 2011) that are associated with misrepresentations of tropical feedback processes (Shin et al. 2010). To estimate changes in tail probabilities using such imperfect models, it is important to account for biases in the probability distributions. However, it is clearly inappropriate to do so through a simple a posteriori “bias correction,” given the links between the mean, variance, and shapes of the predicted probability distributions. There is limited understanding of such links at present, mainly due to the limited understanding of climate–weather interactions on regional scales.
Given the importance of non-Gaussianity highlighted in this paper, it would be interesting to use our SGS–Markov process approach to analyze the tail probabilities of other weather-related variables such as near-surface temperature and precipitation in observational datasets and climate model simulations and projections. A multivariate Markov process model, of which (10) is a univariate approximation as discussed by SS09, could also be used as a more accurate “probability model” for this purpose. These are topics of current research.
We thank C. Smith for generating Fig. 7 and C. McColl for generating Fig. 11. We also thank the reviewers for comments and suggestions that improved the clarity of the paper. This work was supported by NOAA’s Climate Program Office (CPO) and by DOE’s Office for Science (BER). The Twentieth Century Reanalysis Project used resources of the National Energy Research Scientific Computing Center managed by Lawrence Berkeley National Laboratory, and of the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory, which are supported by DOE’s Office of Science under Contracts DE-AC02-05CH11231 and DE-AC05-00OR22725, respectively.
An Extremal Probability Shift Index for Changes in Extremes
Consider a probability density function p(x) with a location parameter μ and scale parameter σ such that the PDF of the standardized variable z = (x − μ)/σ is independent of μ and σ, but may or may not depend upon additional shape parameters. Next consider shape-preserving changes of p(x) associated with changes in the location and scale parameters from μ0 to μ and σ0 to σ. Let ΔP+ and ΔP− be the change in the probability of high (x ≥ XT > μ0) and low (x ≤ 2μ0 − XT) extremes of x. Then for high extremes (x ≥ XT)
where zT0 = (XT − μ0)/σ0 and zT = (XT − μ)/σ. The second equality is satisfied for some by the mean value theorem. Note that . Equation (A1) may be rewritten as
The sign of ΔP+ is thus identical to the sign of .
For low extremes x ≤ 2μ0 − XT, a similar derivation gives
Convergence of GEV Fits to Extreme Value Distributions
Figure B1 shows the goodness of the GEV fit to the “true” extreme value PDFs of Nd-day block maxima in 108-day simulations of the Gaussian and SGS (S = 1, K = 5) Markov processes with a 1-day correlation time scale discussed in the text. The results are shown as cyan and black curves, respectively, for the Gaussian and SGS cases. The goodness of fit is measured in terms of the normalized Kolmogorov–Smirnov distance Dks between the fitted and true distributions, where the normalization is the square root of the number of sample maximum values for each block size. The calculations were performed for specified block sizes increasing from 10 to 90 days in steps of 10 days, from 100 to 900 days in steps of 100 days, etc. Note that the time unit on the abscissa is actually the correlation time scale τc of the Markov process, which is specified as 1 day in the simulations purely to provide meteorological context.
According to extreme value theory, the GEV fit should become more accurate for larger block sizes, and Fig. B1 clearly bears this out. However, the rate of convergence is much slower in the case of the Gaussian parent than the SGS parent, and the GEV fit is much less accurate in the Gaussian parent case for Nd less than 200 “days” (i.e., correlation time units).
Generalized Extreme Value and Generalized Pareto Distributions Associated with Parent SGS Distributions
To see how the extreme value distribution associated with an SGS distribution is approximately of the GEV form for block sizes N large enough that the maximum in each block occurs in the tail of p(x) in (7), consider the CDF F(x) of the SGS distribution in this tail:
where . Note that the second term in (C1) is much smaller than unity. This gives, using the approximation for large N,
where , , and . If the maximum value of x in a sample of N independent values of x (i.e., in a block of size N) occurs in this tail, the CDF of is equal to in (C2):
which is a GEV distribution with location, scale, and shape parameters , , and , as noted in (13).
We next show that the conditional exceedance probability density of an SGS variable x beyond a specified threshold XT, given that x exceeds XT, is precisely of the generalized Pareto (GP) form in (14) if XT lies in the tail given by (7). Let y = x − XT ≥ 0, whose PDF py(y) = p[x(y)] = p(y + XT). We may then write
where ξ = E2/(2 + E2) and . Note that we seek an expression for , which is proportional to p(x ≥ XT) but which, unlike p(x ≥ XT), integrates to unity from x = XT to ∞. Ensuring this normalization gives
Publisher’s Note: This article was revised on 11 December 2015 to correct the online version of Figure 2, which was mistakenly replaced when originally published.