1. Background and motivation
The dynamics of low-frequency variability have been, and still are, the focus of many research studies. This continuous interest is driven by the need to understand the dynamics of tropospheric planetary waves and other synoptic patterns such as blocking for predictability. The issue of climate change has also brought the question related to the existence of preferred flow regimes up front in science media (Palmer 1999; Corti et al. 1999; Hsu and Zwiers 2001; Christiansen 2003). Origins and mechanisms of large-scale and low-frequency variability have also been discussed extensively in the literature. Linear Rossby waves forced from the Tropics have been proposed to explain midlatitude large-scale systems (Hoskins and Karoly 1981; Sardeshmukh and Hoskins 1985; Hoskins and Ambrizzi 1993). A significant body of researchers also maintain that these systems are essentially steady wave patterns that can be resonantly excited and stabilized by nonlinear midlatitude processes such as adiabatic heating and eddy vorticity fluxes (Mitchell and Derome 1983; Marshall and So 1990), or resulting from the instability of the climatological winter flows (Simmons et al. 1983). The nonlinear paradigm was also proposed, which led to the concept of multiple equilibria first mentioned by Rossby (1940) and taken steps further by Charney and Devore (1979), Charney et al. (1981), and others.
Various authors have reported, using empirical studies, robust and statistically significant bimodality in the wave amplitude index (Hansen and Sutera 1986, 1995; Christiansen 2005, 2007; Ruti et al. 2006.) The issue of nonlinearity from observations remains, however, a challenging task given the complexity of the system and the relatively small sample size of reanalyses (see, e.g., Branstator and Berner 2005; Wallace et al. 1991; Nitsche et al. 1994). Recently Sura et al. (2005) presented an alternative theory, namely, that low frequency variability is basically linear (Newman et al. 1997, 2003) and that nonlinearity is primarily rapidly decorrelating and can be approximated by a multiplicative noise. Linear dynamics of planetary waves yield, conceptually, systems whose probability distribution function (pdf) is multivariate Gaussian or multinormal if the forcing is Gaussian (Leith 1971; Delsole and Farrell 1995; Penland 1989; Penland and Sardeshmukh 1995; Toth 1991) but also can result in non-Gaussianity if the external forcing is non-Gaussian. A multivariate non-Gaussian structure of the pdf can reflect nonlinearity (Hannachi et al. 2003; Christiansen 2005, 2007; Berner 2005) but can also provide evidence of state-dependent or multiplicative noise, as shown by Sura et al. (2005).
Tropical forcing is also important in the study of low-frequency variability. An example is the effect of El Niño–Southern Oscillation (ENSO) on the extratropics (Lau and Nath 1996; Horel and Wallace 1981; see also the review of Trenberth et al. 1998). Persistent tropical forcing can cause persistence of extratropical large-scale weather systems, such as the case of ENSO forcing of both phases of the Pacific–North American (PNA) pattern (Livezey et al. 1997; Hoerling et al. 1997, 2001; Hannachi 2001). These extratropical weather patterns, however, are internal to the atmosphere and the (external) tropical forcing can contribute to their excitation and persistence, and can also slightly modify their phase. Therefore nonlinearity and/or nonnormality of tropical forcing could also provide another alternative mechanism. Other alternative views also exist. For example, possible nonlinearity may exist but may result primarily in a roughly piecewise linear response or in a saturation at the extremes.
Investigations using atmospheric models do seem to show some nonlinearity in the space of large-scale flow. Nonlinearity can produce skewness and nondiffusive behavior (Crommelin 2004) and can also yield multiple equilibria as in the theoretical studies using low-order models (Charney and Devore 1979) and intermediate complexity models (Egger 1981; Reinhold and Pierrehumbert 1982; Legras and Ghil 1985; Crommelin et al. 2004; Yang et al. 1997). Similar behavior is also observed using general circulation model (GCM) simulations (Haines and Hannachi 1995; Hannachi 1997a, b; Selten and Branstator 2004; Branstator and Berner 2005) where quasi-stationary (QS) states are identified in the subspace of the large-scale flow. But, it is fair to say that the task is a challenging one; see, for example, the discussion in Lorenz (2006). The difficulty in separating contributions from the linear and the nonlinear dynamics is mainly due to (i) the myriad of spatiotemporal scales of motions interacting with the large-scale flow, contributing therefore to embedding the large-scale signals in the dynamical noise, and (ii) the sampling problem resulting from the relative shortage of observed data. The question of sampling, for instance, becomes apparent when dealing with monthly data from reanalyses or from a short-period GCM simulation (Hsu and Zwiers 2001; Stephenson et al. 2004). It is only by using a very long (low resolution) GCM integration that Branstator and Berner (2005) have shown that large-scale planetary wave dynamics invoke nonlinearity revealed by the state dependence of the mean local tendencies of these waves. For small to moderate sample sizes, such as the case of reanalyses, one can only look at the pattern dependence of planetary wave dynamics.
Now, a main feature of atmospheric dynamics is that preferred low-frequency circulation structures exist in the atmosphere irrespective of whether nonlinearity is or is not significant (Simmons et al. 1983; Newman et al. 2003). For example, Newman et al. (2003) obtain, using a linear model, preferred large-scale flow patterns that look like observed ones. Persistent low-frequency patterns, which differ significantly from climatological circulation and which remain stationary for a week or so have also been objectively identified by various authors (e.g., Dole and Gordon 1983) over the North Pacific and the North Atlantic. The dynamical paradigms of, for example, Legras and Ghil (1985) and Mukougawa (1988) provide possible mechanisms of persistent patterns and their sporadic recurrence.
Various methods have been proposed in the literature to identify large-scale preferred structures from reanalyses and GCMs (see, e.g., Kimoto and Ghil 1993; Marshall and Molteni 1993; Hsu and Zwiers 2001; Hannachi and O’Neill 2001, hereafter HO01; Stephenson et al. 2004 and references therein). Some of these methods have been reviewed by Christiansen (2007). In most of these investigations the search for preferred structures was not addressed systematically nor satisfactorily because of the difficulty in choosing the correct number of these structures (see, e.g., HO01 and Christiansen 2007 for more discussion). In this manuscript, the observed skewness in planetary wave dynamics (Sura et al. 2005) is interpreted in terms of multivariate Gaussian mixture, which is used as a clustering algorithm as in HO01. In Smyth et al. (1999), a cross-validation procedure was used to find the number of components in the mixture. This procedure, however, has a systematic bias, which depends on the sample size (HO01; Christiansen 2007). HO01 used a cross-validation combined with a requirement based on statistical independence to find the number of regimes. The issue of statistical independence is a difficult one since there is no systematic way to identify the independent sample and its size. This difficulty is overcome here by systematically finding the number of regimes in the mixture model using arguments from order statistics. The mixture model and the methodology are described in section 2. In section 3 the method is applied to few simple stochastic dynamical systems and data from a GCM. The application to the National Centers for Environmental Prediction–National Center for Atmospheric Research (NCEP–NCAR) reanalyses is presented in section 4. A summary and conclusions are presented in the last section.
2. Methodology
a. The Gaussian mixture model
It has been shown (e.g., Christiansen 2003) that winter NH stratospheric planetary waves display a bimodal behavior representing the strong/weak polar vortex or cold/warm stratosphere. By looking at the tropospheric wave amplitude index few authors have reported significant bimodality of daily winter NCEP 500-hPa heights (Hansen and Sutera 1995; Christiansen 2005, 2007). Ruti et al. (2006) have reported a characterization of planetary atmospheric waves in terms of the subtropical jet. They show, using NCEP–NCAR reanalyses and the 40-yr European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis (ERA-40) that for intermediate jet strengths a bimodal behavior of an indicator of planetary waves is obtained. The extratropical troposphere as a whole, however, is characterized by highly chaotic dynamics and low signal-to-noise ratio. The background (or dynamical) noise combined with ergodicity (Branstator and Berner 2005) therefore prevents multimodality from emerging, even if the dynamics reveal intermittent behavior between low-frequency flow regimes resulting in a skewness of the planetary wave pdf. For example, Sura et al. (2005) show that large-scale extratropical flow could well be entirely linear but with multiplicative noise resulting from the rapidly decorrelating nonlinearity. The observed skewness of the pdf of planetary wave dynamics (Sura et al. 2005) is interpreted here in terms of a mixture model to diagnose preferred large-scale structures. But it should be noted that this alternative interpretation does not necessarily rule out any current theories.
b. Model identification
Given a set of n data points x1, . . . xn, the model parameters in (1), for a given c, are estimated by maximizing the log-likelihood L = Σnt=1Ln( f (xt)) with respect to the [c(d + 1)(d + 2) − 2]/2 unknown parameters μk, and Σk, k = 1, . . . , c, and αk, k = 1, . . . , c − 1. These parameters are obtained iteratively using the EM algorithm. In HO01, the number c of multivariate Gaussian components is obtained using a cross-validation procedure (Smyth et al. 1999) based on an “independent sample.” The choice of an independent sample, however, is not unique and is controversial because of the difficulty in defining an independent sample and its size.1 In addition, the method suffers a systematic bias as the sample size increases (see, e.g., HO01 or Christiansen 2007 for details). The method proposed here finds the number of components for any given significance level and overcomes the previous drawbacks.
The method is based on finding uncertainties on the estimated parameters. One way to compute these uncertainties is to use information from second derivatives (Hessian) of the likelihood. This, however, defeats the very objective of the EM algorithm, which was introduced, indeed, to avoid direct maximization of the likelihood. Another major problem in finding uncertainties in this model is related to the difficulty in mapping the mixture components of one solution to the mixture components of another solution (of the same data). The simple solution presented here avoids this problem and looks for confidence intervals of the mixing proportions only. This is because the weights α1, . . . , αc play a central role in the model since they represent explicitly prior probabilities of belonging to the associated Gaussian. If, for example, in a mixture of c Gaussians one proportion is not significant (at a given significance level), then the model can only support at most c − 1 Gaussian components (at that significance level).
The method developed here is based on arguments from order statistics, as explained below. For a given number c of Gaussian components the EM algorithm provides, given the initial conditions, the model parameters. For the initial conditions the mixing proportions are randomly selected positive numbers satisfying (2), the centers of the components are randomly chosen from the state space, and the covariance matrices are identical and chosen to be diagonal. To quantify uncertainties on the mixing proportions one hundred such solutions are generated using different initial conditions and the mixing proportions sorted in increasing order. Suppose that the true model has (unknown) proportions2 that are sorted as α1 ≤ α2 . . . ≤ αc. Suppose now that β1, . . . , βc is a realization of these c proportions obtained using the EM algorithm with a mixture of c components. Note that at this stage the remaining parameters are not used. Next, these proportions are sorted to yield β1:c ≤ β2:c . . . ≤ βc:c; that is, β1:c, β2:c . . . , and βc:c are respectively the smallest, the next smallest . . . , and the maximum of β1, β2 . . . , βc. The principle of order statistics implies that the kth smallest number βk:c is precisely a realization of αk, k = 1, . . . , c.3 For details on order statistics see, for example, Wilcox (1997) and also Hannachi (2006) for further references and for an application to climate. Hence, from a generated ensemble of solutions, the mean mk and standard deviation σk of the estimator of αk, for each integer k (1 ≤ k ≤ c), can be obtained. The interval [mk − σkΦ−1(1 − α/2), mk + σkΦ−1(1 − α/2)] provides approximately the 100(1 − α)% confidence interval of the mean of the estimator of αk, where Φ−1(· · ·) is the inverse cumulative distribution function of the normal distribution. This is a rough approximation to the confidence intervals and an improvement to this approximation, which could take into account the positivity of the proportions (2), is left for future research. To find the right number of components we then start from c = 2 and successively increase c by one until the added component becomes insignificant. The method is very successful and provides systematically the correct number of Gaussian components. The method is applied next to few stochastic dynamical systems, and later to NCEP–NCAR reanalyses.
3. Application to few low-order systems and data from a GCM
Two simple low-order systems are considered here, a three-well potential system and the classical Lorenz (1963) model. These models have been used in HO01 and are used here also with the new methodology because they provide a convenient test bed but do not necessarily reflect the dynamics of climate.
a. Three-well potential system
When the number of the multivariate Gaussian components in the mixture is less than or equal to three, all components are found to be highly significant (not shown). However, when a four-component model is fitted, the fourth is found to be nonsignificant. The distribution of the mixing proportions for c = 4 is shown in Fig. 1a, and the corresponding 99% confidence intervals for the means are shown in Fig. 1b. The case of a three-multivariate Gaussian mixture is shown in Fig. 1c. Similar results to those shown in Figs. 1b,c are obtained at a lesser confidence level (e.g., 90% level) and at a higher confidence level (e.g., 99.9%). This is clear evidence that the data contain three components representing the three potential wells. Note from Fig. 1 that, when the number of components increases, the confidence intervals of the mixing proportions can change slightly. However, the strategy adopted here is that we start always with two components, then add an extra component only if the last regime (or component) is significant at a given level, and so on. In this way we avoid the ambiguity that can occur when regime K, in K components, is not significant, but it is for K + 1 components. Figure 1d shows a random subsample of size 103 from the potential system data along with the centers of the bivariate Gaussian components (bold thick dots) and their covariances (continuous circles). The squares show the theoretical centers, that is, the bottom of the wells. Figure 1d shows that the theoretical centers have been well identified. Note that the solution shown in Fig. 1d is chosen among the 100 generated solutions by selecting the one with maximum likelihood. The dashed lines in Fig. 1d show contour plots of the obtained mixture pdf where multimodality is clear in this simple system. The robustness of the method has also been tested by successively splitting the data into smaller trunks of sizes 2500, 1250, 625, 312, 156, and the method applied to these trunks. In all of them, the correct number of components was recovered at the three levels 10%, 5%, and 1%.
The power of the methodology has been addressed by analyzing the rate of type-2 errors, that is, false negatives/misses, which correspond to when the null hypothesis is incorrectly not rejected when, in fact, it is false. The probability p2 of making a type-2 error is a function of the significance level and the sample size. The power of the test is 1 − p2 and provides a measure of how well the methodology avoids false negatives. To compute the power we proceed by progressively decreasing a0 [see Eq. (4)], starting from 0.9, hence reducing the depth of one of the wells (the one centered at a and a
b. The Lorenz system
The approach has been tested also with a simple bivariate first-order autoregressive (AR1) model. Two AR1 time series, x1 and x2, with respective lag-1 autocorrelations 0.6 and 0.65 have been generated. The red-noise time series have been transformed to yield cross-correlation using y1 = x1 + 2x2 and y2 = −x1 + 2.5x2. Various sample sizes have been used, and the results are reported in Table 2 showing the obtained number of components for various confidence intervals.
c. Application to data from a GCM
Haines and Hannachi (1995) found that, when the number d of EOFs used in (8) increases, the minima get deeper. Similar behavior is also observed here using the sampled streamfunction trajectory, that is, PCs, and the mixture model. As the state space dimension increases, the uncertainties on the mixing proportions of the two states get smaller, but the third component remains insignificant (not shown). Figure 4 shows the solutions obtained from the two-component mixture model using three EOFs showing ±PNA (Figs. 4c,d) similar to the dynamical QS states (Figs. 3c,d). The QS state 1 (Fig. 3c) is similar to regime 1 (Fig. 4c). The spatial structure of the QS state 2 (Fig. 3d) is, however, a little different from the experimental regime 2 (Fig. 4d). The distance between QS 1 and regime 1 is around 0.1, and that between QS 2 and regime 2 is around 0.3. One candidate responsible for this difference is the effect of high frequency eddies (Haines and Hannachi 1995). In fact, when these latter are taken into account in Eq. (7), the experimental regimes and the QS states become closer (Haines and Hannachi 1995). The other and most important reason is the nature of these QS states. It is shown in Hannachi (1997a), through a stability analysis using both the reduced EOF space and normal modes, that +PNA is more unstable than −PNA, rather like the situation for the Lorenz system and the three-well system, respectively. Note also, as pointed out in the end of section 2a, that the regime centers are colinear when there are two components.
4. Application to NCEP–NCAR geopotential heights
a. Winter 500-hPa height variability
The data used here consist of the NCEP–NCAR geopotential height field at the 500-hPa pressure level (Z500). The data have a horizontal resolution of 2.5° × 2.5° and span the period 1 January 1949–31 December 2003. The seasonal cycle is computed using the smoothed daily geopotential heights using a 9-day moving average filter. At each grid point, height anomalies are then computed as the departure of unfiltered daily values from the seasonal cycle. Winter daily anomalies, defined by the months December–January–February (starting 1 January 1949), north of 20°N, are then obtained and are used next for the analysis of planetary wave behavior. The obtained winter gridded data are then weighted by the square root of the cosine of the corresponding latitude and EOFs and PCs obtained. We emphasize here that the NH circulation data used are not low-pass filtered and may include synoptic scale. This has the potential to be compared to most previous studies where low-pass filtered data are used.
Figure 5 shows the spectrum of the covariance matrix, where only the first 40 eigenvalues, given in percentage of explained variances of the corresponding EOFs, are shown. The vertical bars show approximately 95% confidence limits, given by the rule of thumb of North et al. (1982). The two leading eigenvalues are degenerate. Figure 6 shows the leading two EOFs of the geopotential heights, and they look similar to those obtained, for example, by Crommelin (2004). EOF1 (Fig. 6a) has a wave pattern over the North Pacific, the North Atlantic sector, eastern North America, and western Europe. The strongest center of action of EOF1 is sitting over southern Greenland. The pattern bears some similarities with the PNA pattern (Wallace and Gutzler 1981) except that the center of action over North America is stretched toward southern Greenland. The EOF2 pattern (Fig. 6b) has zonal and meridional structures with its strongest center of action sitting over the Aleutian Islands. The pattern has two positive centers sitting over the Aleutian Islands and southern Greenland, and two opposite centers over North America and northeastern Atlantic Ocean/western Europe. These two leading EOFs are mostly concentrated over the Western Hemisphere.
These two EOFs explain altogether about 18% of the total winter daily 500-hPa geopotential height variance. The next five EOFs are degenerate as are the following four EOFs (Fig. 7). Most of these EOFs have zonal structure. The strongest centers of action of these patterns are located over the North Pacific (EOF3, EOF8), western North Atlantic (EOF9, EOF10, and EOF11), eastern North Atlantic (EOF5, EOF6), eastern North Pacific (EOF7, EOF8), and Scandinavia (EOF5).
b. Planetary wave dynamics and mixture analysis
Figure 8 shows a scatterplot matrix of the daily 500-hPa height PCs within various two-dimensional state spaces using EOFs 1 to 5. Histograms of the individual PCs are also shown. This is a simple and informative way to explore all two-dimensional spaces within this five-dimensional state space. Some PCs show departure from normality (e.g., skewness in PC1 and PC2). There is also an indication of departure from binormality of some of the scatterplots, for example, the PC1/PC2 scatterplot. The skewness in, for example, the leading PCs is consistent with Sura et al. (2005). Here an alternative interpretation is presented. Planetary-scale flows are analyzed using the mixture model applied to various subspaces spanned by the large-scale 500-hPa height EOFs, and preferred structures are interpreted in terms of the mixture components.
A clustering analysis in probability space (Diggle 1983; Martinez and Martinez 2002; Stephenson et al. 2004) has been conducted and applied to the leading two PCs, as in Stephenson et al. (2004). The PC data have been transformed using the marginal cumulative pdfs of the PCs. The procedure then looks for clusters in the probability space based on the (interpoint) distance between pairs of points. The mean number of points that are within a given distance of a target point is then computed. The ratio of this number to that obtained from a uniform (no clustering) distribution yields a clustering index L. Figure 9 shows an example of a scatterplot of PC1/PC2 in the probability space for the first half of the record (Fig. 9a) and the second half (Fig. 9c) and the associated clustering indexes L (Figs. 9b,d). The dashed curves represent, respectively, the upper and lower envelopes of the clustering indexes obtained using 25 samples simulated from a homogeneous Poisson random process known as complete spatial randomness (CSR; Martinez and Martinez 2002). There is clear indication of significant clustering (i.e., departure from CSR), particularly in the range [0.025, 0.3] of interpoint distances (within the probability space). The clustering is particularly marked for the first half of the record (Fig. 9b) and less so for the second half (Fig. 9d). The test has also been applied to subsamples obtained by considering points that are a number of days apart. Clustering is found to decrease as the sample size decreases but was still observed for data that are 4–5 days apart for the whole record, particularly for the first half of the record but less so for the second half. For example, the slight clustering observed in the second half of the record (Fig. 9d) can still be seen when the data are 2, but not more, days apart.
The mixture model is now used as a way to quantify the departure of the PC1/PC2 scatterplot from binormality, and also as a clustering algorithm within a probabilistic and inferential framework [see, e.g., Fraley and Raftery 1998, 2003; Everitt et al. 2001; Martinez and Martinez 2002 for further details and references]. The PCs are scaled to have unit variance prior to applying the mixture model. The mixing proportions for a three-Gaussian mixture model are shown in Fig. 10a, and the associated 99% confidence limits on the mixing proportion means in Fig. 10b. The mixing proportions for the two-Gaussian mixture model (not shown) produce two significant components at the 99% confidence level. Figure 10b shows clear evidence for the presence of two components. Robustness tests against changes in sample size have also been carried out as for the three-well potential system. The data have been resampled by scrambling years, that is, 90-day blocks and also 45-day blocks. The approach is then applied to the first trunks of respective sizes 2475, 1237, 618, and 308, and they all produce two regimes at 10%, 5%, and 1% significance levels except for the last trunk (of size 308) where the two components are only significant at the 10% level. So at least 4 years of daily data are required to reproduce the results. Other two-dimensional scatterplots have also been investigated for nonbinormality, and the results show some nonuniformity. This nonuniformity can also be obtained by visual inspection of Fig. 8. At 99% confidence limits only (PC1, PC2) and (PC2, PC3) scatterplots each fit a two-bivariate Gaussian mixture model. In the (PC2, PC3) state space the two-component mixture model is significant also at the 99.9% confidence level. It can be seen in fact, from Fig. 8, that the highest density of the scatter is shifted toward the lower right corner in the (PC1, PC2) plane and toward the upper left corner in the (PC2, PC3) plane, that is, evidence for skewness or departure from bivariate normality in agreement with Sura et al. 2005). In all other scatterplots shown in Fig. 8 the mixture model supports only one single two-dimensional Gaussian component model. This also can be noted from these scatterplots where the highest density (dark) region is located in the middle of the scatter. The scatterplot of the (PC1, PC3) from the GCM of the previous section (not shown), where non-Gaussianity is manifest, is similar to the (PC1, PC2) scatterplot of the reanalyses (Fig. 8), showing similar skewness in their respective pdfs. Branstator and Berner (2005) also reported that the linear/nonlinear signature of their two-dimensional phase space tendencies from a long run of a GCM varies a great deal from plane to plane. They found that the projected dynamics in some planes is exclusively nonlinear, while in others it is nearly linear.
The centers of the mixture model in the (PC1, PC2) plane (Table 3) are shown in Fig. 10c along with a random subsample of size 103 from the data, the covariance matrix of each component (continuous line), and contours of the obtained mixture model pdf (dashed lines). The associated preferred patterns are discussed later. Analyses performed on five consecutive 11-yr periods also indicate significant low frequency variability between these periods, as is observed in Christiansen (2005).
Extension of the mixture model to higher state space dimensions is straightforward conceptually. Technically, however, two impediments prevent this extension. The empty space phenomenon in high dimensional spaces requires a power increase of sample size with dimension. For instance, with the available sample size of daily geopotential heights, one can only use a maximum of three or four state space dimensions. The other difficulty is related to the degeneracy of the spectrum. The winter 500-hPa height spectrum (Fig. 5) shows, for example, that EOFs 3 to 7 (Fig. 7) are degenerate and, hence, if one decides to use three-dimensional state space, there is no reason why one should use the leading three EOFs. One could, for example, equally choose EOFs 1, 2, and 6. In the literature, either or both of these difficulties are not taken care of in general (e.g., Kimoto and Ghil 1993; Crommelin 2004).
The data anomalies are next projected onto the obtained rotated EOFs, that is, REOF3 and REOF4, to yield, respectively, RPC3 and RPC4 time series. The mixture model has been extended to three- and four-dimensional state spaces spanned, respectively, by (EOF1, EOF2, REOF3) and (EOF1, EOF2, REOF3, REOF4). In both cases a strong support for a two-component mixture model is obtained. For example, Fig. 12 shows the 99% confidence limits of the mixing proportions for a two- (Fig. 12a) and a three-component (Fig. 12b) model using the winter daily 500-hPa height trajectory within the newly formed four-dimensional state space; see also Table 3. Figure 13 shows the anomaly patterns, that is, flow regimes, associated with the centers of a two-component mixture when PC1 and PC2 are used. These nearly opposite patterns are aligned mainly along PC2 (Fig. 10c) in the two-dimensional case. The solutions are quite robust to changes in the basis, and in three or four dimensions the patterns still look like those of Fig. 13 when either PC1, PC2, PC3, and PC4 or PC1, PC2, RPC3, and RPC4 are used.
The solutions have a strong projection onto ±PNA patterns on the Pacific side and a slightly weaker projection onto the negative/positive North Atlantic Oscillation (NAO) patterns on the Atlantic side, respectively. The first solution (Fig. 13a) projects particularly well onto the so-called cold-ocean/warm-land (COWL) pattern (Wallace et al. 1996; Corti et al. 1999) but with a slightly weaker contribution over the North Atlantic compared to the North Pacific. The COWL pattern describes much of the recent climate change of NH surface air temperature (Houghton et al. 1996; Wallace et al. 1996).
c. Climate change and regime shift
Christiansen (2003) analyzed stratospheric winter-averaged geopotential heights and found a regime shift in the last half of the 1970s from the weak vortex regime to the more zonal strong vortex regime resulting in a change of frequency of occupation of the regimes. We have attempted here to see whether one can observe such a shift in the high frequency part of planetary wave variability. The newly formed four winter PCs, that is, PC1, PC2, RPC3, and RPC4, have been split into two subsamples corresponding respectively to two periods: January 1949–February 1978 and December 1978–December 2003. Both subsets have been submitted, separately, to the same analysis of the mixture model as in the previous section.
For both of the periods highly significant support for a two-multivariate Gaussian mixture is obtained with slightly stronger support for the first period than for the second period, in agreement with the results of the clustering test discussed in section 4b. For example, using the leading two PCs, the preferred structures for the first period are significant at the 0.1% level, whereas they are significant at only the 1% level for the second period. The mixing proportions and the centers of the corresponding mixtures are shown in Table 3. Figure 14 shows the scatterplot, the covariances of each component, and the mixture pdf for the two periods: January 1949–February 1978 (Fig. 14a) and December 1978–December 2003 (Fig. 14b). The filled small circles refer to the regime centers for the first period, whereas the squares refer to the last period. The centers for the whole period, that is, January 1949–December 2003, are indicated by small circles (see also Fig. 10c). The arrows shown in Fig. 14 indicate the shift from the regime centers of the first period to those of the second period.
Figure 14 shows a clear shift in preferred structures between the two periods. The axis of the regime centers is rotated by about 45° clockwise between the two periods. This has resulted in the axis of the regime centers corresponding to the whole period being nearly aligned with the EOF2 direction (Fig. 14; see also Corti et al. 1999). The observed skewness in PC2 (Stephenson et al. 2004) could be a result of this regime shift. Figure 15 shows the regime centers using the first three PCs, that is, (PC1, PC2, RPC3) for the first period (Figs. 15a,b) and the second period (Figs. 15c,d). The Aleutian low center in Fig. 15c is deeper than its counterpart in Fig. 15a, and similarly for the positive centers over North America. Both centers have shifted southwestward. For the second regime, the high center over southern Greenland disappeared during the second period, accompanied by a weakening of the low over the North Atlantic and a strengthening of the high over the southeastern United States.
To investigate the significance of the observed regime shift a Monte Carlo test is conducted to check whether the difference between the pre- and post-1978 regimes could be simply an artifact of sampling. One hundred random subsamples, each with approximately half the size of the entire record, are drawn from the winter 1949–2003 height PCs. To take seriality, and also possible autocorrelation of seasonal forcing into account, each subsample is composed of randomly drawn 90-day blocks. Solution from a two-component mixture model is then obtained for each subsample. The pre- and post-1978 regime centers are then compared to the distribution obtained from the clusters of the Monte Carlo centers. It is found that the regimes from the two periods, particularly those of the second period, are well outside the bulk of the Monte Carlo clusters (not shown). The centers of the first period are on the 85% confidence ellipse of the clusters. For the second period, however, the first regime, associated with a deepening of the Aleutian low (Fig. 15c), is nearly on the 95% confidence ellipse and the second regime, associated with a deepening of the Icelandic low (Fig. 15d), is nearly on the 99% confidence ellipse. So the regime shift is highly significant and points mainly to changes occurring in the second half of the record. To test whether there is a change in the shape of the pdf, both subsamples have been rescaled to have zero mean and unit variance and changes in the pdf shape along various directions, in the (PC1, PC2) plane, are analyzed using a two-sample Kolmogorov–Smirnov test. A significant change in the shape is found along all directions, with EOF1 angles between 138° and 161° at the 5% level and between 141° and 154° (around the second diagonal in, e.g., Fig. 14a) at the 1% level. For example, along the direction 148°, the sample is significantly skewed in the first period (Fig. 16a) and with significant kurtosis in the second period (Fig. 16b). Although the changes in regimes are significant, the spatial correlations between the associated regimes are very high. For example, the correlations between the patterns of Figs. 15a,b and those of Figs. 15c,d are larger than 0.8. The same high correlations are also observed for different state space dimensions. There is also a change in the frequency of regime occupation with an increase (decrease) for the first (Fig. 15c) (second) (Fig. 15d) regime. Note that changes in regime occupation frequency are directly related to changes in the mixing proportions (Table 3). The regimes have therefore undergone some change in frequency of occurrence (Palmer 1999; Corti et al. 1999), also in amplitude but not much in regime phase.
Figure 17 shows the difference between the associated regimes corresponding to the first and second periods. The patterns of change (Fig. 17) reflect the effect of forcing involved in the observed regime shift. The first pattern (Fig. 17a) projects onto +PNA and shows a deepening in the Aleutian low system by about 40 m. The second pattern (Fig. 17b), with a low over southern Greenland and a high stretching from the eastern U.S. coast through to Europe, projects onto the positive phase of the NAO. It is known that the NAO has gone through a positive phase since around the mid-1970s, characterized by a deepening of the Icelandic low and an increase of the subtropical North Atlantic high (see, e.g., Hurrell et al. 2003). This trend toward the positive phase is characterized by a strengthening of westerly winds bringing stormy and rainy weather over Europe. Changes in external forcing can also have an influence on regime shift. On the Pacific side, for example, an increase in ENSO activity has also been observed. El Niño events observed in the last quarter century, for example, the events of 1982/83 or 2001, are undoubtedly stronger than all previously observed events since records began. Since El Niño forces the positive phase of the PNA, the increasing warm ENSO events in the second period have contributed therefore to the observed increase in the PNA (Fig. 17a) (see, e.g., Hoerling et al. 2001; Livezey et al. 1997; Hannachi 2001).
Another possibility of external influence could be the observed changes in SST that occurred in the mid-1970s, when a shift is identified in fall and winter 1976–77. This change was caused by a marked cooling of SSTs in the central North Pacific and a warming of SSTs along the west coast of North America (Venrick et al. 1987; Ebbesmeyer et al. 1990; Trenberth and Hurrell 1994; Miller et al. 1994). This SST shift along with some instigation by forcing from the tropical Pacific could have been partially involved in the basin-scale deepening of the wintertime Aleutian low system (Graham 1994). The deepening of the Aleutian low, by about 4 hPa as a result of tropical SST forcing of the 1976/77 climate transition over the North Pacific, has been simulated recently by Deser and Phillips (2006) using a combined observational and modeling approach.
5. Summary and conclusions
Preferred flow structures in the high-frequency and large-scale flows are investigated using winter Northern Hemispheric 500-hPa NCEP–NCAR heights for the period January 1949–December 2003. The approach followed here is based on the mixture model (Haines and Hannachi 1995; Smyth et al. 1999; Hannachi and O’Neill 2001) whereby the probability distribution function of the atmospheric state is expressed as a mixture of a finite number of multivariate Gaussian distributions. The centers of the individual multivariate Gaussians are interpreted as the regime centers. The components of the mixture are obtained using the efficient expectation–maximization algorithm. Because all of the information regarding the significant components is contained in the mixing proportions, the number of components is obtained using order statistics applied to the mixing proportions of the mixture model. A large number of solutions of the mixture model is obtained, then the mixing proportions sorted and confidence intervals obtained. This method is efficient in finding the number of components and avoids all known problems encountered, for example in cross-validation such as the issues of sample size and independence (Hannachi and O’Neill 2001; Christiansen 2005).
The method has been successfully applied to a simple three-well potential system, the perturbed Lorenz system, and to the output of a GCM. Next, daily winter (December–February) 500-hPa geopotential heights from NCEP–NCAR (January 1949–December 2003) are used and EOFs/PCs computed. A clustering algorithm is applied to the leading two PCs. Significant clustering is found in the data. Clustering is found to decrease when seriality is taken into account, but the data remain clustered, particularly for the first half of the record and less so for the second half. The mixture model is then used to identify the clusters. The model has been applied to the leading two, three, and four PCs where the third and fourth PCs are obtained using rotation of EOFs 3 to 7 and EOFs 8 to 11, respectively. This rotation procedure has been chosen because of the degeneracy of the eigenvalue spectra. The method shows a strong support for a mixture of two multivariate Gaussians. The associated preferred regimes have structures over the North Pacific and the North Atlantic sectors. The preferred large-scale structures show a +PNA/−NAO pattern and its opposite. This behavior is not observed in all EOF subspaces, and there are subspaces in which the distribution is multinormal.
To investigate possible changes in preferred large-scale structures the PCs have been divided into two subperiods: pre- and post-1978. For both periods two highly significant regimes are again observed with a slight support for the first period. In addition, a significant regime shift between the two periods is obtained along with a significant change in the shape of the distribution. For the regime shift, the main changes are in amplitude, but not much in the structure (phase) of the regimes. The pattern of regime difference between the associated regimes of the two periods are the +PNA, associated with a deepening of the Aleutian low by about 40 m, and the positive phase of the NAO, associated with a deepening of the Icelandic low by about the same amount. The patterns associated with the regime shift are consistent with 1) the positive phase shift of the NAO, and the increase in positive ENSO events during the last quarter of the century, and 2) the SST shift observed in the North Pacific around the mid-1970s.
Acknowledgments
This work was funded by the NCAS Centre for Global Atmospheric Modelling (CGAM), the University of Reading. Some of this work was completed while the author was visiting the Department of Hydraulic and Environmental Modelling at the University of Tunis (ENIT). The author would like to thank T. Palmer, J. Berner, and C. Ferro for useful discussions, and three anonymous reviewers for their thorough and constructive comments that helped improve the manuscript.
REFERENCES
Anderson, B. D. O., and J. B. Moore, 1979: Optimal Filtering. Prentice Hall, 357 pp.
Berner, J., 2005: Linking nonlinearity and non-Gaussianity of planetary wave behavior by the Fokker–Planck equation. J. Atmos. Sci., 62 , 2098–2117.
Branstator, G., and J. Berner, 2005: Linear and nonlinear signatures in the planetary wave dynamics of an AGCM: Phase space tendencies. J. Atmos. Sci., 62 , 1792–1811.
Charney, J. G., and J. G. Devore, 1979: Multiple flow equilibria in the atmosphere and blocking. J. Atmos. Sci., 36 , 1205–1216.
Charney, J. G., J. Shukla, and K. C. Mo, 1981: Comparison of a barotropic blocking theory with observation. J. Atmos. Sci., 38 , 762–779.
Christiansen, B., 2003: Evidence for nonlinear climate change: Two stratospheric regimes and a regime shift. J. Climate, 16 , 3681–3690.
Christiansen, B., 2005: Bimodality of the planetary-scale atmospheric wave amplitude index. J. Atmos. Sci., 62 , 2528–2541.
Christiansen, B., 2007: Atmospheric circulation regimes: Can cluster analysis provide the number? J. Climate, 20 , 2229–2250.
Corti, S., F. Molteni, and T. N. Palmer, 1999: Signature of recent climate change in frequencies of natural atmospheric circulation regimes. Nature, 398 , 799–802.
Crommelin, D. T., 2004: Observed nondiffusive dynamics in large-scale atmospheric flow. J. Atmos. Sci., 61 , 2384–2396.
Crommelin, D. T., J. D. Opsteegh, and F. Verhulst, 2004: A mechanism for atmospheric regime behavior. J. Atmos. Sci., 61 , 1406–1419.
Delsole, T., and B. F. Farrell, 1995: A stochastically excited linear system as a model for quasigeostrophic turbulence: Analytic results for one- and two-layer fluids. J. Atmos. Sci., 52 , 2531–2547.
Deser, C., and A. S. Phillips, 2006: Simulation of the 1976/77 climate transition over the North Pacific: Sensitivity to tropical forcing. J. Climate, 19 , 6170–6180.
Diggle, P. J., 1983: Statistical Analysis of Spatial Point Patterns. Academic Press, 148 pp.
Dole, R. M., and N. D. Gordon, 1983: Persistent anomalies of the extratropical Northern Hemisphere wintertime circulation: Geographical distribution and regional persistence characteristics. Mon. Wea. Rev., 111 , 1567–1586.
Ebbesmeyer, C. C., D. R. Cayan, D. R. McLain, F. H. Nichols, D. H. Peterson, and K. T. Redmond, 1990: Step in the Pacific climate: Forty environmental changes between 1968–1975 and 1977–1984. Proceedings of the Seventh Annual Pacific Climate (PACLIM) Workshop, J. L. Betancourt and V. L. Tharp, Eds., Interagency Ecological Studies Program for the Sacramento-San Joaquin Estuary, 115–126.
Egger, J., 1981: Stochastically driven large-scale circulations with multiple equilibria. J. Atmos. Sci., 38 , 2606–2618.
Everitt, B., and D. J. Hand, 1981: Finite Mixture Distributions. Chapman and Hall, 143 pp.
Everitt, B., S. Landau, and M. Leese, 2001: Cluster Analysis. 4th ed. Oxford University Press, 237 pp.
Fraley, C., and A. E. Raftery, 1998: How many clusters? Which clustering method? Answers via model-based cluster analysis. Comput. J., 41 , 528–588.
Fraley, C., and A. E. Raftery, 2003: Enhanced software for model-based clustering, discriminant analysis, and density estimation: MCLUST. J. Classification, 20 , 263–286.
Graham, N. E., 1994: Decadal-scale climate variability in the tropical and North Pacific during the 1970s and 1980s: Observations and model results. Climate Dyn., 10 , 135–162.
Haines, K., and A. Hannachi, 1995: Weather regimes in the Pacific from a GCM. J. Atmos. Sci., 52 , 2444–2462.
Hannachi, A., 1997a: Weather regimes in the Pacific from a GCM. Part II: Dynamics and stability. J. Atmos. Sci., 54 , 1334–1348.
Hannachi, A., 1997b: Low-frequency variability in a GCM: Three-dimensional flow regimes and their dynamics. J. Climate, 10 , 1357–1379.
Hannachi, A., 2001: Toward a nonlinear identification of the atmospheric response to ENSO. J. Climate, 14 , 2138–2149.
Hannachi, A., 2006: Quantifying changes and their uncertainties in probability distribution of climate variables using robust statistics. Climate Dyn., 27 , 301–317.
Hannachi, A., and B. Legras, 1995: Simulated annealing and weather regimes classification. Tellus, 47A , 955–973.
Hannachi, A., and A. O’Neill, 2001: Atmospheric multiple equilibria and non-Gaussian behaviour in model simulations. Quart. J. Roy. Meteor. Soc., 127 , 939–958.
Hannachi, A., D. B. Stephenson, and K. R. Sperber, 2003: Probability-based methods for quantifying nonlinearity in ENSO. Climate Dyn., 20 , 241–256.
Hannachi, A., I. T. Jolliffe, D. B. Stephenson, and N. Trendafilov, 2006: In search of simple structures in climate: Simplifying EOFS. Int. J. Climatol., 26 , 7–28.
Hansen, A. R., and A. Sutera, 1986: On the probability density distribution of planetary-scale atmospheric wave amplitude. J. Atmos. Sci., 43 , 3250–3265.
Hansen, A. R., and A. Sutera, 1995: The probability density distribution of the planetary-scale atmospheric wave amplitude revisited. J. Atmos. Sci., 52 , 2463–2472.
Hasselmann, K., 1999: Climate change: Linear and nonlinear signature. Nature, 398 , 755–756.
Hoerling, M. P., A. Kumar, and M. Zhong, 1997: El Niño, La Niña, and the nonlinearity of their teleconnections. J. Climate, 10 , 1769–1786.
Hoerling, M. P., A. Kumar, and T. Xu, 2001: Robustness of the nonlinear climate response to ENSO’s extreme phases. J. Climate, 14 , 1277–1293.
Horel, J. D., and J. M. Wallace, 1981: Planetary-scale atmospheric phenomena associated with the Southern Oscillation. Mon. Wea. Rev., 109 , 813–829.
Hoskins, B. J., and D. J. Karoly, 1981: The steady linear response of a spherical atmosphere to thermal and orographic forcing. J. Atmos. Sci., 38 , 1179–1196.
Hoskins, B. J., and T. Ambrizzi, 1993: Rossby wave propagation on a realistic longitudinally varying flow. J. Atmos. Sci., 50 , 1661–1671.
Houghton, J. T., L. G. Meira, Filho, B. A. Callandar, N. Harris, A. Kattenberg, and K. Maskell, 1996: Climate Change 1995: The Science of Climate Change. Cambridge University Press, 572 pp.
Hsu, C. J., and F. Zwiers, 2001: Climate change in recurrent regimes and modes of Northern Hemisphere atmospheric variability. J. Geophys. Res., 106 , 20145–20160.
Hurrell, J. W., Y. Kushnir, G. Ottersen, and M. Visbeck, 2003: An overview of the North Atlantic Oscillation. The North Atlantic Oscillation: Climate Significance and Environmental Impact, Geophys. Monogr., Vol. 134, Amer. Geophys. Union, 1–35.
Jolliffe, I. T., 2002: Principal Component Analysis. 2d ed. Springer, 486 pp.
Kaiser, H. F., 1958: The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23 , 187–200.
Kimoto, M., and M. Ghil, 1993: Multiple flow regimes in the Northern Hemisphere winter. Part I: Methodology and hemispheric regimes. J. Atmos. Sci., 50 , 2625–2643.
Lau, N-C., and M. J. Nath, 1996: The role of the “atmospheric bridge” in linking tropical Pacific ENSO events to extratropical SST anomalies. J. Climate, 9 , 2036–2057.
Legras, B., and M. Ghil, 1985: Persistent anomalies, blocking and variations in atmospheric predictability. J. Atmos. Sci., 42 , 433–471.
Leith, C. E., 1971: Atmospheric predictability and two-dimensional turbulence. J. Atmos. Sci., 28 , 145–161.
Livezey, R. E., M. Masutani, A. Leetmaa, H. Rui, M. Ji, and A. Kumar, 1997: Teleconnective response of the Pacific–North American region atmosphere to large central equatorial Pacific SST anomalies. J. Climate, 10 , 1787–1820.
Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20 , 130–141.
Lorenz, E. N., 2006: Regimes in simple systems. J. Atmos. Sci., 63 , 2056–2073.
Marshall, J., and D. K. W. So, 1990: Thermal equilibrium of planetary waves. J. Atmos. Sci., 47 , 963–978.
Marshall, J., and F. Molteni, 1993: Toward a dynamical understanding of planetary-scale flow regimes. J. Atmos. Sci., 50 , 1792–1818.
Martinez, W. L., and A. R. Martinez, 2002: Computational Statistics Handbook with MATLAB. Chapman and Hall, 591 pp.
McLachlan, G. J., and K. E. Basford, 1988: Mixture Models: Inference and Applications to Clustering. Marcel Dekker, 253 pp.
Miller, A. J., D. R. Cayan, T. P. Barnett, N. E. Graham, and J. M. Oberhuber, 1994: The 1976–77 climate shift of the Pacific Ocean. Oceanography, 7 , 21–26.
Mitchell, H. L., and J. Derome, 1983: Blocking-like solutions of the potential vorticity equation: Their stability at equilibrium and growth at resonance. J. Atmos. Sci., 40 , 2522–2536.
Mukougawa, H., 1988: A dynamical model of “quasi-stationary” states in large-scale atmospheric motions. J. Atmos. Sci., 45 , 2868–2888.
Newman, M., P. D. Sardeshmukh, and C. Penland, 1997: Stochastic forcing of the wintertime extratropical flow. J. Atmos. Sci., 54 , 435–455.
Newman, M., P. D. Sardeshmukh, C. R. Winkler, and J. S. Whitaker, 2003: A study of subseasonal predictability. Mon. Wea. Rev., 131 , 1715–1732.
Nitsche, G., J. M. Wallace, and C. Kooperberg, 1994: Is there evidence of multiple equilibria in planetary wave amplitude statistics? J. Atmos. Sci., 51 , 314–322.
North, G. R., T. L. Bell, R. F. Cahalan, and F. J. Moeng, 1982: Sampling errors in the estimation of empirical orthogonal functions. Mon. Wea. Rev., 110 , 699–706.
Palmer, T. N., 1993: Extended-range atmospheric prediction and the Lorenz model. Bull. Amer. Meteor. Soc., 74 , 49–65.
Palmer, T. N., 1999: A nonlinear dynamical perspective on climate prediction. J. Climate, 12 , 575–591.
Penland, C., 1989: Random forcing and forecasting using principal oscillation pattern analysis. Mon. Wea. Rev., 117 , 2165–2185.
Penland, C., and P. Sardeshmukh, 1995: The optimal growth of tropical sea surface temperature anomalies. J. Climate, 8 , 1999–2024.
Reinhold, B. B., and R. T. Pierrehumbert, 1982: Dynamics of weather regimes: Quasi-stationary waves and blocking. Mon. Wea. Rev., 110 , 1105–1145.
Richman, M. B., 1986: Rotation of principal components. J. Climatol., 6 , 293–335.
Rossby, C-G., 1940: Planetary flow patterns in the atmosphere. Quart. J. Roy. Meteor. Soc., 66 , 68–87.
Ruti, P. M., V. Lucarini, A. D. Aquila, S. Calmanti, and A. Speranza, 2006: Does the subtropical jet catalyse the mid-latitude atmospheric regimes? Geophys. Res. Lett., 33 .L06814, doi:10.1029/2005GL024620.
Sardeshmukh, P. D., and B. J. Hoskins, 1985: Vorticity balances in the tropics during the 1982–83 El Niño-Southern Oscillation event. Quart. J. Roy. Meteor. Soc., 111 , 261–278.
Selten, F. M., and G. Branstator, 2004: Preferred regime transition routes and evidence for an unstable periodic orbit in a baroclinic model. J. Atmos. Sci., 61 , 2267–2282.
Simmons, A. J., J. M. Wallace, and G. W. Branstator, 1983: Barotropic wave propagation and instability, and atmospheric teleconnection patterns. J. Atmos. Sci., 40 , 1363–1392.
Smyth, P., K. Ide, and M. Ghil, 1999: Multiple regimes in Northern Hemisphere height fields via mixture model clustering. J. Atmos. Sci., 56 , 3704–3723.
Stephenson, D. B., A. Hannachi, and A. O’Neill, 2004: On the existence of multiple climate regimes. Quart. J. Roy. Meteor. Soc., 130 , 583–605.
Sura, P., M. Newman, C. Penland, and P. Sardeshmukh, 2005: Multiplicative noise and non-Gaussianity: A paradigm for atmospheric regimes? J. Atmos. Sci., 62 , 1391–1409.
Titterington, D. M., A. F. M. Smith, and U. E. Makov, 1985: Statistical Analysis of Finite Mixture Distributions. John Wiley, 243 pp.
Toth, Z., 1991: Circulation patterns in phase space: A multinormal distribution? Mon. Wea. Rev., 119 , 1501–1511.
Trenberth, K. E., and W. Hurrell, 1994: Decadal atmosphere-ocean variations in the Pacific. Climate Dyn., 9 , 303–319.
Trenberth, K. E., G. W. Branstator, D. Karoly, A. Kumar, N-C. Lau, and C. Ropelewski, 1998: Progress during TOGA in understanding and modeling global teleconnections associated with tropical sea surface temperatures. J. Geophys. Res., 103 , 14291–14324.
Venrick, E., J. McGowan, D. Cayan, and T. Hayward, 1987: Climate and Chlorophyll a: Long-term trends in the central North Pacific Ocean. Science, 238 , 70–72.
Wallace, J. M., and D. S. Gutzler, 1981: Teleconnections in the geopotential height field during the Northern Hemisphere winter. Mon. Wea. Rev., 109 , 784–812.
Wallace, J. M., X. Cheng, and D. Sun, 1991: Does low-frequency atmospheric variability exhibit regime-like behavior? Tellus, 43A–B , 16–26.
Wallace, J. M., Y. Zhang, and L. Bajuk, 1996: Interpretation of interdecadal trends in Northern Hemisphere surface air temperature. J. Climate, 9 , 249–259.
Wilcox, R. R., 1997: Introduction to Robust Estimation and Hypothesis Testing. Academic Press, 296 pp.
Yang, S., B. Reinhold, and E. Källén, 1997: Multiple weather regimes and baroclinically forced spherical resonance. J. Atmos. Sci., 54 , 1397–1409.
Distribution of the mixture proportions α1, . . . , α4, for a mixture of four bivariate Gaussians (a) applied to the three-well potential system, the 99% confidence limits of the means of the mixture proportions for (b) a four- and (c) three- component mixture models respectively, and (d) a random subsample from the data of size 103 along with the centers (thick dots) and the covariances of the individual bivariate Gaussians of the three-component mixture model. The theoretical foci of the system are shown by small squares in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Distribution of the mixture proportions α1, . . . , α4, for a mixture of four bivariate Gaussians (a) applied to the three-well potential system, the 99% confidence limits of the means of the mixture proportions for (b) a four- and (c) three- component mixture models respectively, and (d) a random subsample from the data of size 103 along with the centers (thick dots) and the covariances of the individual bivariate Gaussians of the three-component mixture model. The theoretical foci of the system are shown by small squares in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Distribution of the mixture proportions α1, . . . , α4, for a mixture of four bivariate Gaussians (a) applied to the three-well potential system, the 99% confidence limits of the means of the mixture proportions for (b) a four- and (c) three- component mixture models respectively, and (d) a random subsample from the data of size 103 along with the centers (thick dots) and the covariances of the individual bivariate Gaussians of the three-component mixture model. The theoretical foci of the system are shown by small squares in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions corresponding to (a) a two- and (b) three- bivariate normal mixture models applied to the Lorenz system and (c) as in Fig. 1d but for the Lorenz system for a two-bivariate Gaussian mixture model.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions corresponding to (a) a two- and (b) three- bivariate normal mixture models applied to the Lorenz system and (c) as in Fig. 1d but for the Lorenz system for a two-bivariate Gaussian mixture model.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions corresponding to (a) a two- and (b) three- bivariate normal mixture models applied to the Lorenz system and (c) as in Fig. 1d but for the Lorenz system for a two-bivariate Gaussian mixture model.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Logarithm of the flow tendency I, given by Eq. (6), of the streamfunction using (a) two EOFs and (b) three EOFs, and (c), (d) the dynamical quasi-stationary states minimizing the flow tendency using three EOFs and identified as ±PNA; units of I are 3.8 × 10−21 m6 s−4. Note that (b) is a cross-section through the origin and the two minima, and the contour interval is chosen so that both minima appear simultaneously. Contour interval is 2.37 × 106 in (c) and 3.4 × 106 m2 s−1 in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Logarithm of the flow tendency I, given by Eq. (6), of the streamfunction using (a) two EOFs and (b) three EOFs, and (c), (d) the dynamical quasi-stationary states minimizing the flow tendency using three EOFs and identified as ±PNA; units of I are 3.8 × 10−21 m6 s−4. Note that (b) is a cross-section through the origin and the two minima, and the contour interval is chosen so that both minima appear simultaneously. Contour interval is 2.37 × 106 in (c) and 3.4 × 106 m2 s−1 in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Logarithm of the flow tendency I, given by Eq. (6), of the streamfunction using (a) two EOFs and (b) three EOFs, and (c), (d) the dynamical quasi-stationary states minimizing the flow tendency using three EOFs and identified as ±PNA; units of I are 3.8 × 10−21 m6 s−4. Note that (b) is a cross-section through the origin and the two minima, and the contour interval is chosen so that both minima appear simultaneously. Contour interval is 2.37 × 106 in (c) and 3.4 × 106 m2 s−1 in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions (a) of a mixture of two-bivariate Gaussian mixture model using the leading two PCs and (b) three-multivariate Gaussian mixture model using the leading three PCs, and (c), (d) the corresponding flow regimes obtained using a two-component mixture model with the leading three PCs of the UGAMP model showing the ±PNA regime centers. Contour interval is 7.86 × 105 in (c) and 1.38 × 105 m2 s−1 in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions (a) of a mixture of two-bivariate Gaussian mixture model using the leading two PCs and (b) three-multivariate Gaussian mixture model using the leading three PCs, and (c), (d) the corresponding flow regimes obtained using a two-component mixture model with the leading three PCs of the UGAMP model showing the ±PNA regime centers. Contour interval is 7.86 × 105 in (c) and 1.38 × 105 m2 s−1 in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions (a) of a mixture of two-bivariate Gaussian mixture model using the leading two PCs and (b) three-multivariate Gaussian mixture model using the leading three PCs, and (c), (d) the corresponding flow regimes obtained using a two-component mixture model with the leading three PCs of the UGAMP model showing the ±PNA regime centers. Contour interval is 7.86 × 105 in (c) and 1.38 × 105 m2 s−1 in (d).
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Spectrum of the covariance matrix obtained using the daily winter (DJF) NCEP–NCAR 500-hPa heights for the period January 1949–December 2003. Vertical bars show approximate 95% confidence limits, and only the leading 40 eigenvalues are shown.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Spectrum of the covariance matrix obtained using the daily winter (DJF) NCEP–NCAR 500-hPa heights for the period January 1949–December 2003. Vertical bars show approximate 95% confidence limits, and only the leading 40 eigenvalues are shown.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Spectrum of the covariance matrix obtained using the daily winter (DJF) NCEP–NCAR 500-hPa heights for the period January 1949–December 2003. Vertical bars show approximate 95% confidence limits, and only the leading 40 eigenvalues are shown.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The leading two EOFs of the winter daily 500-hPa heights.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The leading two EOFs of the winter daily 500-hPa heights.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The leading two EOFs of the winter daily 500-hPa heights.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
(a) EOF3 through to (i) EOF11 of the daily winter 500-hPa heights.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
(a) EOF3 through to (i) EOF11 of the daily winter 500-hPa heights.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
(a) EOF3 through to (i) EOF11 of the daily winter 500-hPa heights.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot matrix of the daily winter 500-hPa height PCs 1 to 5. The diagonal shows the histograms of the individual PCs 1 to 5.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot matrix of the daily winter 500-hPa height PCs 1 to 5. The diagonal shows the histograms of the individual PCs 1 to 5.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot matrix of the daily winter 500-hPa height PCs 1 to 5. The diagonal shows the histograms of the individual PCs 1 to 5.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot of PC1/PC2 data within the probability space for (a) the first half of the record and (c) the second half and (b), (d) the associated clustering indexes L. The dashed curves represent respectively the upper and lower envelopes of 25 clustering index curves obtained using simulations from a homogeneous Poisson random process.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot of PC1/PC2 data within the probability space for (a) the first half of the record and (c) the second half and (b), (d) the associated clustering indexes L. The dashed curves represent respectively the upper and lower envelopes of 25 clustering index curves obtained using simulations from a homogeneous Poisson random process.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot of PC1/PC2 data within the probability space for (a) the first half of the record and (c) the second half and (b), (d) the associated clustering indexes L. The dashed curves represent respectively the upper and lower envelopes of 25 clustering index curves obtained using simulations from a homogeneous Poisson random process.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Distributions (a) of the mixture proportions for a three-component bivariate Gaussian mixture model, (b) the 99% confidence limits on the means of the previous three-component mixture proportions, and (c) the centers along with the covariances of the bivariate normals of a two-component mixture model and a random subsample of the data of size 103 using the leading two 500-hPa height PCs. The mixture pdf is also shown in (c) by the dashed line.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Distributions (a) of the mixture proportions for a three-component bivariate Gaussian mixture model, (b) the 99% confidence limits on the means of the previous three-component mixture proportions, and (c) the centers along with the covariances of the bivariate normals of a two-component mixture model and a random subsample of the data of size 103 using the leading two 500-hPa height PCs. The mixture pdf is also shown in (c) by the dashed line.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Distributions (a) of the mixture proportions for a three-component bivariate Gaussian mixture model, (b) the 99% confidence limits on the means of the previous three-component mixture proportions, and (c) the centers along with the covariances of the bivariate normals of a two-component mixture model and a random subsample of the data of size 103 using the leading two 500-hPa height PCs. The mixture pdf is also shown in (c) by the dashed line.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
VARIMAX (a) REOF3 obtained by rotating the five EOFs, EOF3 to EOF7, and (b) REOF4 obtained by rotating the next four EOFs, EOF8 to EOF11.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
VARIMAX (a) REOF3 obtained by rotating the five EOFs, EOF3 to EOF7, and (b) REOF4 obtained by rotating the next four EOFs, EOF8 to EOF11.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
VARIMAX (a) REOF3 obtained by rotating the five EOFs, EOF3 to EOF7, and (b) REOF4 obtained by rotating the next four EOFs, EOF8 to EOF11.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions for the (a) two- and (b) three-component mixture model using the newly formed four PCs: PC1, PC2, RPC3, and RPC4.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions for the (a) two- and (b) three-component mixture model using the newly formed four PCs: PC1, PC2, RPC3, and RPC4.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
The 99% confidence limits on the means of the mixture proportions for the (a) two- and (b) three-component mixture model using the newly formed four PCs: PC1, PC2, RPC3, and RPC4.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Patterns of the regime centers obtained from a two-component mixture model using the leading two height PCs. Contour interval is 10 m.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Patterns of the regime centers obtained from a two-component mixture model using the leading two height PCs. Contour interval is 10 m.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Patterns of the regime centers obtained from a two-component mixture model using the leading two height PCs. Contour interval is 10 m.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot, covariances, and centers of the bivariate Gaussians along with the mixture pdf of a two-component mixture model using the leading two PCs restricted to the periods (a) January 1949–February 1978 and (b) December 1978–December 2003. The centers for the first period are shown by small filled circles, and those for the second period by small squares. The centers corresponding to the entire dataset (see also Fig. 10c) are shown by small empty circles. The arrows indicate the regime shift.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot, covariances, and centers of the bivariate Gaussians along with the mixture pdf of a two-component mixture model using the leading two PCs restricted to the periods (a) January 1949–February 1978 and (b) December 1978–December 2003. The centers for the first period are shown by small filled circles, and those for the second period by small squares. The centers corresponding to the entire dataset (see also Fig. 10c) are shown by small empty circles. The arrows indicate the regime shift.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Scatterplot, covariances, and centers of the bivariate Gaussians along with the mixture pdf of a two-component mixture model using the leading two PCs restricted to the periods (a) January 1949–February 1978 and (b) December 1978–December 2003. The centers for the first period are shown by small filled circles, and those for the second period by small squares. The centers corresponding to the entire dataset (see also Fig. 10c) are shown by small empty circles. The arrows indicate the regime shift.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Regime centers obtained from a two-component mixture model using PC1, PC2, and RPC3, restricted to (a), (b) the first period and to (c), (d) the second period. Note that the PC splitting is simply applied to the entire record of the PCs. Contour interval 10 m.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Regime centers obtained from a two-component mixture model using PC1, PC2, and RPC3, restricted to (a), (b) the first period and to (c), (d) the second period. Note that the PC splitting is simply applied to the entire record of the PCs. Contour interval 10 m.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Regime centers obtained from a two-component mixture model using PC1, PC2, and RPC3, restricted to (a), (b) the first period and to (c), (d) the second period. Note that the PC splitting is simply applied to the entire record of the PCs. Contour interval 10 m.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Histograms of PC1/PC2 along a direction making 148° with PC1 axis for (a) the first (scaled) subsample and (b) the second (scaled) subsample. The shapes of the distributions are significantly different at less than 1% level. The standard normal density function is also shown.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Histograms of PC1/PC2 along a direction making 148° with PC1 axis for (a) the first (scaled) subsample and (b) the second (scaled) subsample. The shapes of the distributions are significantly different at less than 1% level. The standard normal density function is also shown.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Histograms of PC1/PC2 along a direction making 148° with PC1 axis for (a) the first (scaled) subsample and (b) the second (scaled) subsample. The shapes of the distributions are significantly different at less than 1% level. The standard normal density function is also shown.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Difference between the corresponding regime centers of Fig. 15, showing (a) the difference between Figs. 15c and 15a and (b) the difference between Figs. 15d and 15b.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Difference between the corresponding regime centers of Fig. 15, showing (a) the difference between Figs. 15c and 15a and (b) the difference between Figs. 15d and 15b.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Difference between the corresponding regime centers of Fig. 15, showing (a) the difference between Figs. 15c and 15a and (b) the difference between Figs. 15d and 15b.
Citation: Journal of the Atmospheric Sciences 64, 10; 10.1175/JAS4045.1
Power of the methodology applied to the three-well potential system as a function of a0 and the significance level.
Obtained number of components of the mixture model using a two-dimensional AR(1) model for various sample sizes and various confidence intervals.
Mixing proportions (%) and centers of a mixture model (1) with two multivariate Gaussian components using the first two, three, and four (new) PCs, for three different periods: January 1949–December 2003; January 1949–February 1978, and December 1978–December 2003.
The very existence of an independent sample is questionable. The usual way of defining independence based on decorrelation is not conceptually valid since independence and uncorrelation are two different concepts. Independence is stronger and is based on factorization of the joint pdf.
In a Bayesian framework these unknown parameters are random variables.
As an illustration, suppose that c = 2 and we generate only two solutions. The first one provides, say, β1 = 0.3 and β2 = 0.7, then β1:2 = 0.3 ≤ β2:2 = 0.7. The second solution provides, say, β′1 = 0.85 and β′2 = 0.15, then β′1:2 = 0.15 ≤ β′2:2 = 0.85. Then {0.3, 0.15} is a sample of α1, and {0.7, 0.85} is a sample of α2.
In fact, bimodality of the Lorenz (1963) system shows up after a smoothing has been applied to reduce the effect of transitions near the unstable fixed point (the origin); see Palmer (1993).
Note, however, that these patterns (or their time series) are not orthogonal to (or uncorrelated with) the residual; there is no clear partition between this four-dimensional space and its complement.