a. Derivation of the general form

If no more specific prediction for x_ν is available than the climatological mean, then the uncertainty in the system’s state is the climatological or prior uncertainty associated with the climatological probability density that characterizes the state vector X_ν. The effect of the predictor is to provide predictive information on the system’s state, thus reducing the state’s prior uncertainty to the posterior uncertainty that remains after a specific prediction has become available. A state is not predictable when its posterior uncertainty is as large as its prior uncertainty—that is, when the prediction does not contain predictive information in excess of the climatological mean—and its predictability increases with increasing predictive information.

Rendering this intuitive notion of predictability quantitative requires a precise definition of the degree of uncertainty associated with the probability density p_X(x) of a random variable X. Such a definition, which is at the heart of information theory (Brillouin 1956; Shannon and Weaver 1949), was introduced by Shannon (1948), who showed that the entropy

View Expanded

is a natural measure of the uncertainty associated with a random variable X. (The quantity S_X is sometimes called the information of the random variable X, meaning that on average the additional information S_X is needed to specify completely a realization of X.) Shannon derived the entropy functional from a set of heuristic requirements that any measure of uncertainty should fulfill and showed that the entropy is, up to the constant factor k, the unique measure fulfilling these requirements. The value of the constant k determines the units in which the entropy is measured. For thermodynamic systems, k is Boltzmann’s constant. For discrete random variables, the integration in (1) must be replaced by a sum, and k = 1/log2 is chosen so that the entropy S_X becomes the expected number of binary digits, or bits, needed to specify a particular realization of X. We set k = 1/m, where m is the dimension of the state space, so that S_X becomes the mean entropy per state vector component. Defining the entropy relative to the state space dimension makes it possible to compare the entropies of random vectors of different dimensions.

The PP can be interpreted geometrically. If, as is common practice, the entropy S_X is evaluated with k = 1 in definition (1), then the exponential exp S_X is the state space volume enclosing “reasonably probable” or “typical” realizations of the random vector X (Shannon and Weaver 1949, p. 59; Papoulis 1991). We evaluate the entropy S_X with k = 1/m and call the exponential expS_X the typical range. The typical range is the mth root of the volume of typical realizations and measures the mean size of the set of values typically taken by a random vector component. Thus, the term exp(−R_ν) = exp S_Eν/exp S_Xν in the predictive power (3) is a ratio of typical ranges: it is the “typical range of a state vector component given a prediction” over the “typical range of a climatological state vector component.” That is to say, the term exp(−R_ν) is the fraction of the climatological typical range that lies within a prediction’s “range of uncertainty.” The complement 1 − exp(−R_ν), the PP, is the typical range fraction that the predictor eliminates from the climatological typical range. Therefore, the PP indicates the efficacy of the predictor in narrowing the typical range of a state vector component.

Besides exhibiting proper limiting behavior and having an intuitive interpretation, any adequate predictability index should also be independent of the basis in which state vectors are represented. If, for example, the state is a compound of several geophysical fields, its predictability index should not depend on the units in which these fields are measured. Changing the dimensional scaling of some components of a state vector amounts to transforming state space vectors x to the rescaled vectors x′ by multiplication with a diagonal matrix. Such a transformation should leave a predictability measure unchanged. More generally, we require the predictability measure to be invariant under linear coordinate transformations that transform state space vectors x to x′ = Ux with arbitrary nonsingular matrices U. To check if the PP is invariant under such transformations, note that the probability density functions p_X in the original coordinates and p_X′ in the transformed coordinates are related by p_X(x) dx = p_X′(x′) dx′, from which it follows that p_X(x) = |detU| p_X′(x′). Using these relations, one finds that the entropy (1) of the transformed variable X′ compared to that of the original variable X is changed only by the additive constant klog|detU|, which involves the determinant detU of the transformation matrix U (Shannon and Weaver 1949, p. 59). In the predictive information—the difference (2) between the prior and the posterior entropies—the constant terms klog|detU| cancel. Thus, the PP is indeed invariant under arbitrary linear transformations of state space coordinates.

The PP hence has desirable properties and is defined under general circumstances; neither assumptions about the modeling framework nor assumptions about the dimension or distributions of the relevant random variables were needed for the derivation. For univariate and possibly for low-dimensional state vectors, the entropy can be estimated using standard procedures, which involve estimation of the probability density (see, e.g., Silverman 1986; Scott 1992) and of the entropy as a functional thereof (see, e.g., Prakasa Rao 1983; Dmitriev and Tarasenko 1973; Ahmad and Lin 1976; Joe 1989; Hall and Morton 1993). Thus, it may be possible to obtain a predictability measure for, say, local precipitation, a field for which neither the climatological distribution nor the prediction error distribution is Gaussian and for which a predictability index based on rms errors may be inappropriate.

Whereas the PP in its most general form is applicable to low-dimensional predictions, for high-dimensional states estimation of the entropy from (1) may not be feasible when the available dataset is small. Our emphasis, however, is on intraseasonal to interannual climate predictability, as opposed to the predictability of shorter-term weather processes. That the former kind of variability follows an approximately Gaussian distribution (see, e.g., Toth 1991) considerably simplifies the discussion.

b. Simplifications for Gaussian random variables

For an m-dimensional Gaussian random vector X, the probability density takes the form

View Expanded

where 〈X〉 is the mean of X, the superscript (·)^T indicates the transpose of (·), and Σ⁻¹ is the inverse of the nonsingular covariance matrix Σ. The entropy integral (1) of the Gaussian density p_X is readily carried out and yields the entropy

View Expanded

as a function of the covariance matrix determinant. Denoting the covariance matrix of the state, the climatological covariance matrix, by

Σ_νX_ν

and the covariance matrix of the prediction error by

C_νE_ν

one finds from (2) the predictive information

View Expanded

Using the product theorem for determinants and substituting k = 1/m leads to the PP

α_νΓ_ν^1/(2m)(5)

where

Γ_νC_νΣ⁻¹_ν(6)

is called the predictive information matrix. The predictive information matrix is well-defined, provided that the climatological covariance matrix Σ_ν is positive definite so that its inverse exists and is likewise symmetric and positive definite. Positive definiteness of the climatological covariance matrix is assumed in the following theoretical developments. The complications arising in practice from singular covariance matrix estimates will be dealt with in section 4c.

The interpretation of the PP as a ratio of typical ranges can now be made more concrete. From the entropy (4) of an m-dimensional Gaussian random vector X with covariance matrix Σ follows, again taking k = 1/m, the typical range,

View Expanded

For a univariate random variable with m = 1, the covariance matrix Σ is a scalar variance, and the square root of this variance is the standard deviation σ. Therefore, the typical range of a univariate Gaussian random variable, exp S_X = 2πeσ ≈ 4.13σ, is proportional to the standard deviation. For an m-dimensional Gaussian random vector X, the ellipsoid E_p(X) that is centered on the mean of X and encloses some fraction 0 < p < 1 of the cumulative probability distribution has a volume proportional to (detΣ)^1/2 (Anderson 1984, 263). Since the volume of an ellipsoid is proportional to the product of the lengths of its semiaxes, the factor (detΣ)^1/(2m) in the typical range (7) is proportional to the geometric mean of the semiaxis lengths of the ellipsoid E_p(X). Hence, the term (detΓ_ν)^1/(2m) = (detC_ν)^1/(2m)(detΣ_ν)^−1/(2m) in the PP is the ratio of the geometric mean of the semiaxis lengths of the prediction error ellipsoid E_p(E_ν) over the geometric mean of the semiaxis lengths of the climatological ellipsoid E_p(X_ν). This interpretation of the PP as a ratio of geometric means of semiaxis lengths specializes the above general interpretation of the PP to Gaussian random variables.

For univariate state vectors, the covariance matrices in (6) are scalar variances; the predictive information matrix is the ratio of these variances; and the square root of these variances, the standard deviations, are the rms errors. Therefore, the predictive power (5) reduces to one minus the ratio of the rms error of a prediction over the rms error of the climatological mean prediction. Similar predictability measures have been employed by several authors, for example, Hayashi (1986), Murphy (1988), and Stern and Miyakoda (1995). Thus, the PP can be understood as a generalization of the univariate error-ratio predictability measures to multivariate states with arbitrary probability distributions.

When the distribution of states is multivariate Gaussian, one might think that arguments based on a comparison of prediction errors also lead to an adequate predictability measure. The mean-squared prediction error corresponds to the sum of the diagonal elements, or the trace trC_ν, of the prediction error covariance matrix C_ν. Analogously, the trace trΣ_ν of the climatological covariance matrix Σ_ν gives the mean-squared error of the climatological mean prediction. Taking one minus the ratio of the rms errors as a predictability index, one obtains

View Expanded

Traces, however, are only invariant under orthogonal transformations, a subclass of the general linear transformations considered above. A scaling transformation, for example, generally changes the predictability index (8). The expression (5) for the PP, on the other hand, involves a ratio of determinants that remains invariant under arbitrary linear coordinate transformations, including scaling transformations. The invariance under linear coordinate transformations is a principal advantage of information theory arguments over those based on considerations of prediction errors.

a. State space decomposition

The state vector X_ν consists of m components X¹_ν, . . . , X^m_ν, which are univariate random variables. If the state vector is a grid representation of a geophysical field, for example, the component X^k_ν is the random variable associated with the geophysical field at grid point k. These univariate random variables are generally correlated and are not ordered by PP. From the m components X^k_ν, we want to construct m linear combinations Y^k_ν = (u^k_ν)^TX_ν such that the first component Y¹_ν has the largest PP attainable by any linear combination of state vector components, and subsequent components Y²_ν, . . . , Y^m_ν are mutually uncorrelated and ordered according to decreasing PP.

In order to find the linear combination of state vector components that has the largest PP, we must determine the predictive power α^k_ν of an arbitrary linear combination Y^k_ν = (u^k_ν)^TX_ν and then maximize this PP with respect to the weight vector u^k_ν. The predictive information matrix of the univariate component Y^k_ν is a ratio of scalar variances. These scalar variances are the diagonal elements of the covariance matrices

Σ^′_νU^T_νΣ_νU_νC^′_νU^T_νC_νU_ν(10)

of the transformed state vector Y_ν = U^T_νX_ν and the transformed prediction error U^T_νE_ν. The predictive information matrix (6) of the kth component thus reduces to the ratio of the kth diagonal elements,

View Expanded

The scalar γ^k_ν is called the Rayleigh quotient of the weight vector u^k_ν (see, e.g., Golub and van Loan 1993, chapter 8). Substituting the Rayleigh quotient for the predictive information matrix in (5) gives the predictive power of the kth component

α^k_νγ^k_ν^1/2(12)

Maximizing the predictive power α^k_ν is thus equivalent to minimizing the Rayleigh quotient γ^k_ν.

We will now argue that an analysis of the remaining eigenvectors of the predictive information matrix leads to a decomposition of the state space into uncorrelated subspaces that are ordered according to decreasing PP. In making this point, we need some properties of the eigendecomposition of the predictive information matrix.

As detailed in the appendix, the eigenvector matrices U_ν and V_ν can be obtained from a sequence of real transformations and are thus real themselves. This means that, despite the fact that the predictive information matrix is not necessarily symmetric, its eigenvalues and eigenvectors are real. Moreover, as the predictive information matrix is a product of positive semidefinite matrices, the eigenvalues γ^k_ν are greater than or equal to zero. Since no predictor of a linear combination of state vector components should have a prediction error variance that exceeds that of the climatological mean prediction, the eigenvalues γ^k_ν should also be less than or equal to one; thus, 0 ⩽ γ^k_ν ⩽ 1.

Expressing the predictive information matrix in the predictable pattern basis makes manifest properties of both the PP and the predictable component analysis. Since the determinant of a matrix is the product of its eigenvalues, the determinant of the predictive information matrix can be written as det Γ_ν = Π^m_k=1γ^k_ν, whence we infer, from (5), the overall PP

View Expanded

Written in this form, it is evident that the overall PP is unity if one or more eigenvalues of the predictive information matrix vanish, that is, if the prediction error variance vanishes for at least one state vector component. At the other extreme, the PP is zero if all eigenvalues of the predictive information matrix are unity. The PP is nonzero if at least one eigenvalue of the predictive information matrix is smaller than unity, that is, if there is at least one predictable component with nonzero PP. Conversely, if the overall PP is nonzero, the predictable component analysis discriminates the state vector components with large, nonzero PP from those with small, possibly vanishing PP.

More generally, if the overall PP is nonzero, the predictable component analysis discriminates a more predictable “signal” from an uncorrelated background of less predictable “noise.” The overall PP in the subspace spanned by the first r ⩽ m predictable patterns is 1 − (Π^r_k=1γ^k_ν)^1/(2r), which is greater than or equal to the overall PP in any subspace of dimension r′ > r. This dimension dependence of the PP particularly implies that the PP α¹_ν = 1 − (γ¹_ν)^1/2 in the subspace of the first predictable pattern is always greater than or equal to the overall PP in any other subspace, regardless of its dimension. We also conclude that the first r < m predictable patterns span the r-dimensional state space portion with the largest PP, the signal, which is uncorrelated with the (m − r)-dimensional complement, the noise.

b. Relation to principal component analysis

The transformation Y_ν = U^T_ν X_ν simultaneously diagonalizes the climatological covariance matrix Σ_ν, the prediction error covariance matrix C_ν, and the predictive information matrix Γ_ν. That is to say, when the states and the prediction error are expressed relative to the predictable pattern basis, their components at any fixed instant ν are uncorrelated; nevertheless, predictable components at different instants ν may be correlated. If we again think of the state vector as a representation of a geophysical field on a spatial grid, the predictable component analysis yields components that are uncorrelated spatially but that may be correlated temporally.

Principal component analysis and predictable component analysis pursue different goals and optimize different criteria (cf. Fukunaga 1990, chapter 10.1). Expanding state vectors in terms of EOFs and truncating the expansion at some r < m gives the r-dimensional subspace that is uncorrelated with the neglected (m − r)-dimensional subspace and has minimum rms truncation error (e.g., Jolliffe 1986, chapter 3.2). The principal component analysis thus yields an optimal representation of states in a reduced basis. As the rms truncation error is invariant solely under orthogonal transformations but is not invariant under, for example, scaling transformations, the EOFs are invariant under orthogonal transformations only; a dimensional rescaling of variables generally changes the outcome of the principal component analysis.

By way of contrast, expanding state vectors in terms of predictable patterns and truncating at some r < m gives the r-dimensional subspace that is uncorrelated with the neglected (m − r)-dimensional subspace and has maximum PP. The predictable component analysis thus yields an optimal discrimination between more predictable components and less predictable components. As the predictive power is invariant under arbitrary linear coordinate transformations, so the predictable component analysis is invariant under arbitrary linear transformations of state vectors; in particular, the predictable component analysis does not depend on the dimensional scaling of variables.

c. Rank-deficient covariance matrices

The expressions for the PP of Gaussian predictions and the predictable component analysis were derived under the assumption that the climatological covariance matrix Σ_ν be nonsingular. Yet when the climatological covariance matrix is estimated from data, restrictions in sample size may lead to a sample covariance matrix that is singular. For a sample of size N, the sample covariance matrix is singular if N − 1, the number of degrees of freedom in the covariance matrix estimate, is smaller than the state space dimension m. The sample covariance matrix has at most rank N − 1 or m, whichever is smaller. In typical studies of climatic predictability, the number N of independent data points is much smaller than the dimension m of the full state space of, say, a general circulation model; hence, sample covariance matrices usually do not have full rank.

The correspondence between predictable component analysis and the principal component analysis of the prediction error for whitened state vectors suggests a heuristic for dealing with rank deficiency of sample covariance matrices. Instead of applying the whitening transformation to the full m-dimensional state vectors, one retains and whitens only those principal components of the climatological covariance matrix that correspond to eigenvalues significantly different from zero. The predictable component analysis is then computed in this truncated state space.

Complications similar to those with the climatological covariance matrix Σ_ν may arise with the prediction error covariance matrix C_ν. If the number of degrees of freedom n available for the estimation of C_ν is smaller than the state space dimension m, the estimated prediction error covariance matrix is singular. A singular error covariance matrix leads to vanishing eigenvalues of the predictive information matrix Γ_ν. Vanishing eigenvalues of the predictive information matrix correspond to states that have zero prediction error variance for at least one state vector component, but if n < m, at least m − n of the vanishing eigenvalues may be spurious: they correspond to state space directions in which the prediction error variance is zero because of sparse sampling but could become nonzero if the sample were larger. As above, a way to circumvent these difficulties is to perform a principal component analysis of the climatological covariance matrix, retaining at most n components for further analysis.

If the state vectors consist of variables with different dimensions, the principal component analysis depends on the dimensional scaling of the variables. For state vectors that are, for example, compounds of different geophysical fields, it is therefore advisable to compute the principal components of each field separately and assemble the state vectors for the predictable component analysis from selected principal components of each field. The principal components should be selected in such a way that the resulting state space dimension is small enough to ensure adequate sampling and nonsingular covariance matrix estimates. Section 7a contains an example that illustrates how principal components may be selected for a predictable component analysis.

Estimating predictable components from sparse data is an ill-posed problem, a problem in which the number of parameters to be estimated exceeds the sample size. Methods for solving ill-posed problems are known as regularization techniques (see, e.g., Tikhonov and Arsenin 1977; Engl et al. 1996; Hansen 1997; Neumaier 1998). We refer to the above approach as regularization by truncated principal component analysis. The computational algorithm in the appendix shows that regularization by truncated principal component analysis amounts to replacing the ill-defined inverse of the estimated climatological covariance matrix by a Moore–Penrose pseudoinverse (see, e.g., Golub and van Loan 1993, chapter 5). Since the principal component analysis and the pseudoinverse can be computed via a singular value decomposition of a data matrix (see, e.g., Jolliffe 1986, chapter 3.5), regularization by truncated principal component analysis is equivalent to regularization by truncated singular value decomposition, a method extensively discussed in the regularization literature (e.g., in Hansen 1997, chapter 3). More sophisticated regularization techniques (e.g., McLachlan 1992, chapter 5;Friedman 1989; Cheng et al. 1992; Krzanowski et al. 1995) may yield better estimates of the predictive information matrix; however, these techniques are less transparent than regularization by truncated principal component analysis.

d. Related work in the statistics and climatic predictability literature

Predictable component analysis is a variant of a method known in multivariate statistics as discriminant analysis. [For introductory surveys, see Ripley (1996, chapter 3) and Fukunaga (1990, chapter 10).] Discriminant analysis seeks those linear combinations of state variables that optimize a criterion called the discriminant function. Discriminant functions are usually ratios of determinants or of traces of covariance matrices and thus resemble the PP.

In discriminant analysis, one considers only the weight vectors u^k_ν and the associated components Y^k_ν, which are commonly referred to as canonical variates. Our additional interest in the predictable patterns v^k_ν led us to the above generalizations of standard results from discriminant analysis. Instead of focusing solely on the left eigenvectors u^k_ν of the predictive information matrix (or on the right eigenvectors of its transpose), we have considered both the right and the left eigenvectors as well as their interdependence. In this respect, the above derivations extend those in the literature on discriminant analysis.

In climate research, a number of authors have used some of the above methods, particularly in the detection of climate change [e.g., Bell (1982, 1986); Hasselmann (1993); see Hegerl and North (1997) for a review]. Hasselmann (1993), for example, takes a climate change signal v¹_ν as given and determines the linear combination Y¹_ν = (u¹_ν)^TX_ν of climatic variables that best discriminates between the climate change signal and a background noise of natural variability. He obtains from the signal v¹_ν the optimal fingerprint u¹_ν via the relation u¹_ν = Σ⁻¹_νv¹_ν, which is a special case of the relation (15) between predictable patterns v^k_ν and weight vectors u^k_ν.

Another example of a method that is used in climate research and resembles discriminant analysis is the state space decomposition discussed by Thacker (1996). Thacker’s state space decomposition formally parallels the predictable component analysis but derives from a different motivation, namely, seeking dominant modes of variability in datasets in which the data are affected by uncertainties.

The predictable component analysis unifies these approaches. Grounding the analysis in the literature on multivariate statistics should make a host of further methods accessible to climate research.

a. Predictability studies of the first kind

Studies of the first kind address the evolution of uncertainties in the initial condition for a prediction. In studies using ensemble integrations of a numerical model, M initial model states x¹₀, . . . , x^M₀ are chosen such as to sample a probability distribution that represents uncertainties in the initial condition. Each initial state xⁱ₀ is then integrated forward in time, evolving into the state xⁱ_ν at instant ν. Just as the initial states x¹₀, . . . , x^M₀ form a sample of a distribution that represents uncertainties in the initial condition, the states x¹_ν, . . . , x^M_ν form a sample of a distribution that represents uncertainties in the prediction for lead time ν.

The predictive information matrix is the product of the prediction error covariance matrix and the inverse of the climatological covariance matrix, and these covariance matrices are estimated as sample covariance matrices from the ensemble of model integrations. Since the climatological statistics are often almost stationary over typical forecast lead times, the climatological covariance matrix Σ = Σ_ν is usually assumed to be independent of the lead time ν. The climatological covariance matrix depends only, for example, on the month or the season for which a forecast is made. If, in addition to the ensemble integration, a longer control integration of the model is available, the climatological covariance matrix can be estimated from this control integration as the sample covariance matrix

View Expanded

The sample mean

View Expanded

is an estimate of the climatological mean, and the index ν runs over those N instants of the control integration that have the same climatological statistics as the instant for which the forecast is made. The sample covariance matrix Σ̂ is an estimate of the unknown climatological covariance matrix Σ.

The mean of the M-member ensemble

View Expanded

is a prediction of the model state x_ν at lead time ν that evolved from some initial state x₀ drawn from the distribution representing initial uncertainties. The ensemble mean prediction is unbiased because the residuals

eⁱ_νxⁱ_νx̂_ν

which form a sample of the prediction error distribution, have zero mean. The sample covariance matrix of the residuals

View Expanded

is an estimate of the prediction error covariance matrix.

The predictive information matrix is estimated from the sample covariance matrices as Γ̂_ν = Ĉ_νΣ̂⁻¹, and the estimate Γ̂_ν is substituted for the actual predictive information matrix Γ̂_ν in all of the above analyses. Thus, one can estimate predictive information matrices for a sequence of forecast lead times ν and obtain the overall PP at each ν from (5). Examining the PP as a function of lead time ν will reveal typical timescales over which the predictability varies. As illustrated in section 7b, one can test by Monte Carlo simulation at which lead times ν, if at any, the PP estimate is significantly greater than zero. At those lead times ν at which the PP is significantly greater than zero, there exist predictable state vector components, and these can be identified by a predictable component analysis. The sequence of predictable patterns with a PP significantly greater than zero will disclose the system’s predictable features as functions of forecast lead time ν. The first predictable pattern is that pattern whose component is predictable with the smallest rms error relative to the rms error of the climatological mean prediction.

Since the estimate Ĉ_ν of the prediction error covariance matrix and the estimate Σ̂ of the climatological covariance matrix are computed from different datasets, finite sample effects may cause their difference Σ̂ − Ĉ_ν not to be positive semidefinite; that is, for some components, the prediction error variance may exceed the climatological variance. If the difference Σ̂ − Ĉ_ν is not positive semidefinite, the estimate of the predictive information matrix Γ̂_ν has eigenvalues γ̂^k_ν that are greater than one, and such eigenvalues may lead to negative PPs. Negative PPs can be avoided by setting all estimated eigenvalues γ̂^k_ν > 1 to γ̂^k_ν = 1. The predictable patterns and weight vectors that belong to eigenvalues greater than one are not reliably estimated; however, since they correspond to state space portions with small PP, they are of little interest.

b. Predictability studies of the second kind

Studies of the second kind address the predictability of the response of a system to changes in boundary conditions. Internal variability of the system renders a multitude of states consistent with a particular boundary condition, but the distributions of possible state realizations may differ from one boundary condition to another. Predictability of the second kind rests on the separability of the distributions of possible realizations: the more separable the distributions are according to different boundary conditions and the more the distributions are localized in state space, the more a prediction, based on knowledge of a particular boundary condition, reduces the uncertainty of which state may be realized.

In ensemble studies, each member i = 1, . . . , M of the ensemble is a model state that is consistent with a given boundary condition. The scatter of the M ensemble members around their mean reflects the internal variability. The climatic variability, reflected by the scatter of states around the climatological mean, is composed of the internal variability plus the variability of states induced by variability in the boundary conditions. In ensemble studies, variability in the boundary conditions is accounted for by determining the model’s response to J different boundary conditions j = 1, . . . , J, which are chosen so as to sample the climatological distribution of boundary conditions. Thus, the simulated data consist of model states x^ij_ν, where the indices i and j label the ensemble member and the boundary condition, respectively, and ν designates the time for which predictability characteristics are being examined. For example, in a study that aims to assess the predictability of the response of the atmosphere to changes in SST, ν may label the season and j a particular configuration of SST drawn from the climatological distribution of SST in season ν. To perform such a study in practice, time-varying SST observations of various years may be prescribed as a boundary condition in a GCM. For each season ν, the SST configurations in the years j = 1, . . . , J form a sample of the climatological distribution of SST. Each of the model states x^ij_ν is one possible atmospheric state consistent with the SST configuration j in season ν.

The analysis of such ensemble integrations uses techniques from the multivariate analysis of variance (MANOVA). [See, e.g., Johnson and Wichern (1982, chapter 6) for an introduction to MANOVA; among others, Harzallah and Sadourny (1995), Stern and Miyakoda (1995), and Zwiers (1996) have used univariate analysis of variance techniques in predictability studies of the second kind.] MANOVA tests whether J groups of multivariate random variables are separable. Similarly, predictability studies of the second kind are concerned with the separability of state distributions according to J different conditions on the system’s boundary.

The climatological covariance matrix is estimated as the sample covariance matrix

View Expanded

which measures, at time ν, the scatter of the N = JM sample vectors x^ij_ν around the sample mean

View Expanded

The sample mean is an estimate of the climatological mean.

Given a boundary condition j at time ν, the ensemble mean

View Expanded

provides a prediction of the model state. As above, this prediction is unbiased because the residuals

e^ij_νx^ij_νx̂^j_ν

which form a sample of the prediction error distribution, have zero mean. The sample covariance matrix of the residuals

View Expanded

is an estimate of the prediction error covariance matrix. From the estimate Ĉ^j_ν of the prediction error covariance matrix and the estimate Σ̂_ν of the climatological covariance matrix, one could compute the predictive information matrix and hence the PP and the predictable component analysis for each individual boundary condition j at time ν. However, attention is seldom focused on predictability characteristics associated with individual boundary conditions but is often focused on predictability characteristics averaged over all boundary conditions that typically occur at time ν. For example, atmospheric predictability characteristics associated with a particular SST configuration are often of less interest than average atmospheric predictability characteristics associated with SST configurations that typically occur in season ν. For this reason, the estimated covariance matrices of the prediction error are often combined to an average covariance matrix

View Expanded

where N − J = J(M − 1) is the number of degrees of freedom in the averaged estimate. Taking the average covariance matrix Ĉ_ν in place of the individual covariance matrices Ĉ^j_ν has the advantage of increasing the number of degrees of freedom in the estimate of the prediction error covariance matrix from M − 1 to J(M − 1). Averaging thus regularizes the estimate of the prediction error covariance matrix (Friedman 1989).

The predictive information matrix is estimated from the sample covariance matrices as Γ̂_ν = Ĉ_ν Σ̂⁻¹_ν. With the estimates Γ̂_ν of predictive information matrices, the overall PP at a sequence of ν can be obtained from (5). If the index ν labels seasons, for example, examining the PP as a function of ν will reveal how the system’s average predictability varies seasonally. By Monte Carlo simulation or with Wilks’ lambda statistic (see, e.g., Anderson 1984; chapter 8.4), it can be tested at which times ν, if at any, the estimated PP is significantly greater than zero. At times ν when the PP is significantly greater than zero, the predictable component analysis will yield the predictable patterns. The first predictable pattern is that pattern whose component varies most strongly, relative to its climatological variability, from one boundary condition to another.

In studies of the second kind, the climatological covariance matrix and the covariance matrix of the prediction error are estimated from a single dataset. Therefore, one would expect that the predictive information matrix can be estimated consistently in that all estimated eigenvalues γ̂^k_ν lie between zero and one. When the estimated covariance matrices Σ̂_ν and Ĉ_ν are computed from (24) and (27), respectively, it can be verified that the eigenvalues γ̂^k_ν of the predictive information matrix estimate Γ̂_ν are greater than zero and are bounded above by (N − 1)/(N − J). In the limit of large sample sizes N, the upper bound approaches unity from above, but for finite N, the eigenvalues γ̂^k_ν are not guaranteed to be less than or equal to unity. For the sake of consistent estimation, one may use biased covariance matrix estimates in which both the factor 1/(N − 1) in the climatological sample covariance matrix (24) and the factor 1/(N − J) in the sample covariance matrix (27) of the prediction error are replaced by 1/N. These replacements ensure that the predictive information matrix estimate Γ̂_ν has eigenvalues between zero and one so that the PP always lies between zero and one as well. However, the resulting PP estimate is biased toward larger values.

a. Model identification

The model identification process comprises three phases (Tiao and Box 1981): (i) selecting the model order p; (ii) estimating the coefficient matrices A₁, . . . , A_p, the intercept vector z, and the noise covariance matrix S; and (iii) diagnostic checking of the fitted model’s adequacy to represent the given time series.

3) Checking model adequacy

After one has obtained the AR model that fits the given data best, it is necessary to check whether the model is adequate to represent the data. Adequacy of a fitted model is necessary for analyses of its predictibility characteristics or of its dynamical structure to be meaningful. A variety of tests of model adequacy are described, for example, in Lütkepohl (1993, chapter 4), Brockwell and Davis (1991, chapter 9.4), and Tiao and Box (1981).

As one approach to testing model adequacy, one can test whether the fitted model and the data are consistent with the assumptions intrinsic to AR models. A principal assumption in AR models is that the noise vectors ϵ_ν be uncorrelated. Uncorrelatedness of the noise vectors is, for example, invoked in the derivation of LS estimates and will be implicit in the discussion of predictions with AR models in section 6b. To test if the fitted model and the data are consistent with this assumption, the uncorrelatedness of the residuals

View Expanded

can be tested. The superscript (;af·) refers, as above, to estimated quantities, here to the LS estimates of the AR model parameters. Uncorrelatedness of the residuals can be tested by examining their autocorrelation function (Brockwell and Davis 1991, chapter 9.4) or by performing statistical tests such as the multivariate portmanteau test of Li and McLeod (1981).

b. Predictions with an estimated model

After an AR(p) model has been identified that is adequate to represent a given time series of state vectors, future state vectors can be predicted with the estimated model. Suppose that p initial states x₀, x₋₁, . . . , x_1−p are given independently of the sample from which the AR model was estimated and that the state x_ν, ν steps ahead of x₀, is to be predicted. The ν-step prediction

View Expanded

with x̂_j = x_j for j ⩽ 0, predicts the state x_ν optimally in that it is the linear prediction with minimum rms prediction error (Lütkepohl 1993, chapters 2.2 and 3.5).

We take into account two contributions to the error in predictions with estimated AR models. The first contribution to the prediction error arises because AR models are stochastic models whose predictions are always subject to uncertainty, even when the model parameters are known. The second contribution to the prediction error arises because the AR parameters are estimated, as opposed to being known, and are thus afflicted with sampling error. The uncertainty in the estimated parameters adds to the uncertainty in the predictions. A third contribution to the prediction error arises from uncertainty about the correct model order and uncertainty about the adequacy of an AR model to represent the given data. This third contribution, which results from uncertainty about the model structure, will be ignored in what follows. Draper (1995) discusses how the uncertainty about a model structure affects predictions.

The predictive information matrix of an AR model is estimated from the prediction error covariance matrix Ĉ_ν and the climatological covariance matrix Σ̂ as Γ̂_ν = Ĉ_νΣ̂⁻¹. The PP and the predictable component analysis can then be computed, just as in ensemble studies of the first kind, for a sequence of forecast lead times ν.⁵

As mentioned above, in ensemble studies of the first kind, finite sample effects may cause the estimated PPs to be negative. The same remark applies to AR models because the referenced estimators of the sampling error covariance matrices are asymptotic estimators that become exact only in the limit of infinite sample sizes. In finite samples, the difference Σ̂ − Ĉ_ν between the estimated climatological covariance matrix and the estimated prediction error covariance matrix is not necessarily positive semidefinite. Negative PPs can again be avoided by setting to unity all those eigenvalues of the estimated predictive information matrix that are greater than unity.

a. Principal component analysis of the dynamic topography

The EOFs and the variances of the principal components would traditionally be estimated as the eigenvectors and eigenvalues of the sample covariance matrix of the control integration. The eigenvalues of the sample covariance matrix are, however, biased estimates of the variances (Lawley 1956). The bias of the eigenvalues is positive for the leading principal components and negative for the trailing principal components. The positive bias of the eigenvalues of the leading principal components, the components retained in a truncated principal component analysis, implies a positive bias of the regularized estimate of the climatological covariance matrix. This selection bias would lead to a PP estimate that is biased toward larger values.

We avoid the selection bias of the PP estimate by partitioning the control integration into two parts with N = 100 randomly drawn years each and by selecting principal components from one part of the dataset and estimating the climatological covariance matrix from the other (cf. Miller 1984). We denote the sample covariance matrices of the two parts of the control integration by Σ̂⁽¹⁾ and Σ̂⁽²⁾ and refer to the eigenvectors ŵ^k (k = 1, . . . , m) of the matrix Σ̂⁽¹⁾ as EOFs and to the projections (ŵ^k)^Tx_ν of state vectors x_ν onto the EOFs as principal components. If the r EOFs retained in a truncated principal component analysis form the columns of the matrix Ŵ ∈ ℜ^m×r, then the generally nondiagonal estimate Ŵ^TΣ̂⁽²⁾Ŵ of the covariance matrix of the retained principal components is not affected by selection bias.

Selecting a truncation of the principal component analysis means finding a trade-off between reducing the sampling variability of the estimated covariance matrices by including fewer EOFs and reducing the truncation error by including more EOFs. The fewer EOFs retained, the smaller the sampling error in the estimated covariance matrices. The eigenvalue associated with an EOF gives the mean-squared truncation error that omission of this EOF would entail.

Figure 1 shows the spectrum of the eigenvalues associated with the first 20 EOFs of the dynamic topography. The eigenvalues are normalized such as to indicate the percentage of total sample variation accounted for by each of the EOFs. Figure 1 suggests a truncation after the fourth EOF, at the eigenvalue spectrum’s point of greatest curvature. However, the third and fourth EOFs represent centennial trends in the model’s deeper ocean (Griffies and Bryan 1997b). Since we are interested in multidecadal oscillations and not in long-term model trends, we focus on the first two EOFs.

Figure 2 shows the first and second EOF patterns. The EOF patterns, normalized to norm unity, are in this plot multiplied by the standard deviation of their principal components, so the amplitude of the patterns indicates the rms variability of the dynamic topography. EOF1 represents variations in the strength of the North Atlantic Current’s northeastward drift. The dynamic topography variations have maximum amplitude in the model’s western boundary region. EOF2 is more elongated meridionally than EOF1, and it extends farther northward into the sinking region of the model’s subpolar North Atlantic. EOF2 represents gyre-shaped variations in the North Atlantic circulation with the strongest current variations directed northeastward and located in the central portion of the basin. The principal components associated with these patterns exhibit irregular oscillations with a dominant period of 40–45 yr. Since the two principal component time series are roughly in quadrature, the EOFs can be viewed as different phases of an oscillatory mode of dynamic topography variability.

Together, the first two EOFs account for 39% of the total sample variation in North Atlantic dynamic topography. In the central and western North Atlantic, where these EOFs have the largest amplitude, the correlation coefficients between the principal component time series and the local variations in dynamic topography exceed 0.8 (Griffies and Bryan 1997b). Hence, predictability of the two EOFs would imply predictability of a large portion of local variability in these dynamically active regions.

b. Predictability estimates from the ensemble integration

To obtain regularized estimates of the predictive information matrices, we project the data from the ensemble integration onto the r = 2 EOFs retained in the truncated principal component analysis. For each lead time ν = 1, . . . , 30 yr, we estimate a prediction error covariance matrix Ĉ_ν ∈ ℜ^r×r from the sample covariance matrix of the residuals (23). The full sample covariance matrix Σ̂⁽²⁾ ∈ ℜ^m×m of N = 100 yr of the control integration is used in the estimate Σ̂ = Ŵ^TΣ̂⁽²⁾Ŵ ∈ ℜ^r×r of the climatological covariance matrix in the truncated EOF basis. With the Cholesky factorization technique in the appendix, PPs and the predictable component analysis are computed from the covariance matrices Ĉ_ν and Σ̂.

This approach to estimating the predictive information matrices avoids selection bias. However, the estimation of the climatological covariance matrix draws upon only 100 yr out of the 200 yr of available data. But since for the estimation of the prediction error covariance matrices Ĉ_ν only 11 degrees of freedom are available, against 99 degrees of freedom for the estimation of the climatological covariance matrix Σ̂, the sampling variability of the predictive information matrices Γ̂_ν = Ĉ_νΣ̂⁻¹ is dominated by the sampling variability of the prediction error covariance matrices Ĉ_ν. Therefore, ignoring one-half of the control integration in the estimation of the climatological covariance matrix has little effect on the accuracy of the predictive information matrix estimates.

Figure 3a shows the overall PP of the first two EOFs as a function of forecast lead time ν. At each ν we estimate, by Monte Carlo simulation of 1000 samples, a 95% confidence interval for the PP estimate. Each of the 1000 samples consists of M = 12 random vectors drawn from a Gaussian distribution with a covariance matrix equal to the sample covariance matrix Ĉ_ν of the residuals and N = 100 random vectors drawn from a Gaussian distribution with a covariance matrix equal to the estimated covariance matrix Σ̂ = Ŵ^TΣ̂⁽²⁾Ŵ of the principal components. A PP is computed from each sample. Adding the difference between the 97.5th percentile and the mean of the simulated PPs to the overall PP gives the upper bound of the estimated 95% confidence interval, and subtracting the difference between the mean of the simulated PPs and the 2.5th percentile gives the lower bound. Because the thus estimated 95% confidence interval is centered on the estimated overall PP, it does not account for a bias of the PP estimate.

The difference between the mean of the Monte Carlo simulated PPs and the PP estimate from the GCM ensemble is a measure of the bias of the PP estimate. The mean of the Monte Carlo simulated PPs is always greater than the PP estimate, indicating a bias of the PP estimate toward larger values, but the average PP difference of 0.03 is negligible compared with the sampling error in the PP estimates.

The bias of the PP estimate is small enough that the PP can be considered significantly greater than zero when the 95% confidence interval does not include zero. To justify this heuristic for establishing significance of the PP estimate, we test, also by Monte Carlo simulation, the null hypothesis that the residuals and the state vectors of the control integration are drawn from distributions with equal covariance matrices. If the null hypothesis is true that the climatological covariance matrix Σ and the prediction error covariance matrix C_ν are equal, then there are no predictable components in the state space of the first two EOFs. The null hypothesis is rejected at the 5% significance level for PPs greater than 0.28, the bound marked by the dash-dotted line in Fig. 3a. The lead times at which the 95% confidence interval for the PP estimate does not include zero approximately coincide with the lead times at which the estimated PP is greater than the 0.28 significance bound. The overall PP decays rapidly over the first 10 yr of the forecasting lead time, remains marginally significant up to about year 17, and becomes insignificant beyond year 17.

When the overall PP is significantly greater than zero, there exist predictable linear combinations of state variables, and the predictable component analysis identifies the most predictable of those. Figure 3b shows the predictive power α̂¹_ν of the first predictable pattern as well as the individual PPs of the two EOFs. The first predictable pattern is the linear combination of EOF1 and EOF2 with the largest PP. As discussed in section 4a, the PP of the first predictable pattern is always greater than both the overall PP of the two EOFs combined and the individual PPs of EOF1 and EOF2.

The confidence interval for the estimated PP of the first predictable pattern is obtained from the same Monte Carlo simulation as the confidence interval for the estimated overall PP. Because of the selection bias introduced by selecting the most predictable linear combination of state variables, exclusion of zero from the confidence interval is not sufficient for a PP estimate α̂¹_ν to be considered significantly greater than zero. The fact that, beyond year 17, the 95% confidence interval for the estimated PP of the first predictable pattern does not include zero cannot be taken as evidence of a PP significantly greater than zero.

Questions as to which combination of state variables contributes most to the predictability of state realizations are only meaningful when the overall PP is greater than zero. However, in a statistical test of whether the overall PP is consistent with zero, it is possible that the null hypothesis of zero overall PP is accepted because of a lack of power of the test, not because it is in fact true. With subset selection techniques that exclude from the state space components of small PP (cf. McLachlan 1992, chapter 12), it might then be possible to identify a lower-dimensional subspace in which the test has greater power and yields a significant overall PP. For an example of a similar phenomenon in a different multivariate test, see Johnson and Wichern (1982, chapter 5B). In our two-dimensional example, however, Fig. 3b shows that, at most lead times, both EOFs contribute to the overall PP, so analyzing them jointly seems appropriate. From the above Monte Carlo simulations we therefore conclude that the overall PP is insignificant beyond year 17, and we only consider the first predictable patterns and their PPs up to this lead time.

Figure 3b suggests that during most of the first 13 yr, EOF1 has a greater PP than EOF2; conversely, between years 14 and 17, EOF2 has a greater PP than EOF1. The succession of first predictable patterns v̂¹_ν at lead times ν = 1, 7, 13, and 17 yr, displayed in the left column of Fig. 4, also reflects the relative magnitudes of the individual PPs of the two EOFs. EOF1 dominates the first predictable pattern at lead times 1 and 7 yr. At year 13, EOF2 starts to contribute significantly to the first predictable pattern. At year 17, EOF2 dominates the first predictable pattern.

Both the normalization and the sign of the predictable patterns are matters of convention. According to the normalization convention (16), the predictable patterns are orthogonal with respect to the inverse climatological covariance matrix Σ̂⁻¹. Therefore, amplitudes of the patterns in Fig. 4 indicate the rms variability of the dynamic topography. The sign of the first predictable pattern v̂¹_ν at lead time ν is chosen so as to minimize the squared Mahalanobis distance (v̂¹_ν − v̂¹_ν−1)^TΣ̂⁻¹(v̂¹_ν − v̂¹_ν−1) to the pattern v̂¹_v−1 at lead time ν − 1. The first predictable pattern v̂¹₀ at ν = 0 is taken to be the normalized model state at the initialization of the ensemble integration. This sign convention ensures that the first predictable patterns evolve smoothly with forecast lead time.

In the initial conditions for the ensemble integration, the Atlantic overturning circulation is anomalously weak. Figure 4 shows that what is most predictable 1 and 7 yr in advance is the state vector component associated with a weak-drift anomaly in the North Atlantic Current. Most predictable 13 yr in advance is the state vector component associated with a spreading of the weak-drift anomaly into subpolar regions and with the beginning formation of a gyre-shaped current anomaly in the central North Atlantic. Most predictable 17 yr in advance is the state vector component associated with a decrease in amplitude of the weak-drift anomaly in the North Atlantic Current and with an increase in amplitude and a northward spreading of the gyre-shaped anomaly in the central North Atlantic. The first predictable patterns represent those features of the dynamic topography whose components are predictable with the smallest rms error relative to the rms error of the climatological mean prediction.

The examination of the PP, of the significance of the PP, and of the first predictable patterns confirms and complements the analyses of Griffies and Bryan (1997b). The results presented above were found to be robust: they do not depend on the particular random partition of the control integration chosen to estimate the EOFs and the climatological covariance matrix.

c. Empirical predictability estimates from an AR model

Predictability characteristics can also be derived empirically from AR models fitted to the same set of GCM data projected onto two EOFs. The AR model identification is performed with the software package ARfit (Schneider and Neumaier 1997).

Among AR(p) models of order p = 0, . . . , 6, the order selection criteria SBC and MSC indicate that an AR(1) model best fits the first 100 yr of the GCM control integration. To check whether the fitted AR(1) model is adequate to represent the GCM data, we test the residuals (29) of the fitted model for uncorrelatedness. For N realizations of a white noise process, approximately 95% of the sample autocorrelations are expected to lie within the bounds ±1.96(N)^−1/2 (e.g., Brockwell and Davis 1991, chapter 7.2). For the 99 bivariate residuals of the fitted AR(1) model, all but 2 of the 40 sample autocorrelations between lags 1 and 20 lie within the bounds ±1.96(99)^−1/2. Additionally, the modified portmanteau statistic of Li and McLeod (1981) does not reject, at the 5% significance level, the null hypothesis that the residuals are uncorrelated. Therefore, within the class of AR models, an AR(1) model fits the first 100 yr of the control integration best; the residuals of the fitted model provide no grounds for rejecting the hypothesis that the model is consistent with the GCM data.

The AR model identification procedure was repeated with segments of the GCM data of various lengths. AR(1) models were consistently found to be the best fitting, and diagnostic tests of the fitted models provided no grounds for rejecting the hypothesis of model adequacy. But the fact that an AR(1) model seems to represent adequately the particular set of oscillatory principal component time series considered here is not to be taken as a justification for the indiscriminate use of first-order models. The adequacy of a fitted model must be assessed, and, when linear stochastic models are at all adequate, then higher-order models will often be more appropriate than first-order models. [For an example of how models of inappropriately low order can produce misleading results, see Tiao and Box (1981).] As discussed in section 6, the PP concept and the predictable component analysis are applicable to AR models of any order.

A χ²-test of the skewness and kurtosis of the residuals (29) of the fitted AR(1) model does not reject, at the 5% significance level, the hypothesis that the residuals are a realization of a Gaussian process. The estimators of the prediction error covariance matrices in section 6b are valid for Gaussian processes and can thus be expected to yield reliable estimates of the PP of the AR model (cf. Lütkepohl 1993, chapters 3.5 and 4.5).

Figure 5 shows the PP of the fitted AR(1) model as a function of forecast lead time. Since the prediction error variance of an AR model is a monotonically increasing function of lead time (Lütkepohl 1993, chapter 2.2), the PP decreases monotonically. The overall PP of the AR(1) model fitted to the first 100 yr of the GCM control integration reaches zero at a lead time of about 20 yr. Beyond year 20, the error variances of a model prediction are estimated to be as large as the error variances of the climatological mean prediction.

Included in Fig. 5 is the overall PP of an AR model fitted to only 30 yr of GCM data. Sampling errors contribute to the uncertainty in predictions, and since the sampling errors in models decrease with increasing length of the time series from which the model parameters are estimated, the PP of the model fitted to 30 yr of data is smaller than the PP of the model fitted to 100 yr of data. The PP of the model fitted to 30 yr of data already vanishes at a lead time of 11 yr. However, if sampling errors in the estimated parameters are not taken into account and the PP is estimated only from the generic prediction error matrices Ĉ^mod_ν and Σ̂^mod, then the PPs of the two AR models are almost indistinguishable and nonzero up to year 40. Therefore, the difference between the two PP curves in Fig. 5 can be attributed entirely to larger sampling errors in the parameters of the model fitted to the shorter time series. Neglecting sampling errors leads to a gross overestimation of the predictive capabilities of fitted AR models.

The PP of the first predictable pattern of the AR(1) model fitted to 100 yr of GCM data is shown in Fig. 5, and the right column of Fig. 4 displays the first predictable pattern itself. The sign and normalization conventions of section 7b are applied. The qualitative features of the first predictable patterns of the AR model are the same as those of the first predictable patterns inferred from the ensemble integration. Since the predictive information matrix of an AR model does not depend on the initial condition for a prediction, the significance of the agreement between the first predictable patterns inferred from the ensemble integration and from the AR model goes beyond indicating that the AR model fits the GCM data well. At lead time 1 year, the first predictable pattern of the AR model is dominated by the anomaly, represented by EOF1, in the strength of the North Atlantic Current’s northeastward drift. This pattern suggests that predictions of the AR model are particularly reliable when initialized during either an extremely strong or an extremely weak phase of the oscillation in the strength of this drift. The GCM ensemble was initialized during an extremely weak phase of the North Atlantic Current, which explains the agreement between the predictable patterns.

Thus, the estimation of predictability characteristics from the ensemble integration and from an AR model fitted to a small fraction of the GCM data leads to similar results. For the GCM and the AR model, the lead-time scales over which the overall PP is distinguishable from zero coincide, and the same features of the dynamic topography field are associated with large PP. Such an agreement of results is possible if, as in our example, the process in question can be modeled as a linear superposition of stochastically forced damped-oscillatory and relaxatory modes, modes that an AR model is able to represent.