1. Introduction

This paper proposes a new measure of the overall predictability of a system, independent of lead time. Such a measure allows one system to be characterized as “more predictable” than another. One motivation for this new measure is that traditional measures of the overall predictability of a system often have no obvious generalization to multivariate systems. For instance, the limit of predictability, as defined by Lorenz (1969), is the time beyond which the mean square error exceeds a predefined threshold. However, mean square error depends on the coordinate system used to represent the state and is problematic if different variables with different units and natural variances are mixed. Alternatively, an integral time scale, defined as the integral of some moment of the autocorrelation function with respect to lag, is used often in turbulence studies. However, the autocorrelation function is even more restrictive than mean square error in that it is defined only for a single time series. The predictability time scale of a process is sometimes suggested to be related to the peak of its power spectrum. As an example, ENSO has a spectral peak around 4 yr (Julian and Chervin 1978), which under this assumption implies predictability for about 4 yr. However, real-time forecasts of ENSO demonstrate little skill beyond 1 yr, suggesting an inconsistency. Moreover, even if a reasonable relation between predictability and power spectra could be ascertained for the univariate case, the generalization to multivariate systems would not necessarily be straightforward.

Our proposal for overcoming these limitations has two key elements. The first is adoption of a measure of predictability that is applicable to multivariate systems and invariant to linear transformation of the variables—measures satisfying these properties have been proposed by Leung and North (1990), Schneider and Griffies (1999), and Kleeman (2002) (see DelSole and Tippett 2007 for a review)—and the second is the integration of this measure of predictability with respect to all lead times. The resulting measure has several attractive properties. First, the measure is consistent, in the sense that applying it to the same system represented using a different basis set gives the same result. Second, if the predictability decay follows the same form in different systems, then systems that are predictable at longer lead times will have a larger measure. We restrict our attention to predictability averaged over all initial conditions and call the resulting integral the average predictability time (APT). The APT may not be interpretable as a time scale if the decay of predictability is pathological, but nonetheless the integral still gives useful information about overall predictability.

When only predictability averaged over initial conditions is considered, two natural measures satisfying the above properties are mutual information and the Mahalanobis metric. Unfortunately, mutual information depends on the complete forecast and climatological distributions and is therefore difficult to estimate from finite time series. Even in the case of normally distributed variables, mutual information is difficult to integrate with respect to lead time. In contrast, the Mahalanobis metric depends only on second-order moments and is easier to integrate. Furthermore, it turns out that choosing the Mahalanobis metric as a basis for APT has several other attractive properties. Specifically, the resulting APT can be expressed in terms of power spectra, thus clarifying the connection between predictability and power spectra. This point is demonstrated in the present paper. In addition, the resulting APT can be decomposed into independent components that optimize it, analogous to the way that principal component analysis decomposes variance. This decomposition clarifies the usefulness of APT even if the system is characterized by a wide range of time scales—the decomposition separates different components according to their APT, allowing the full spectrum of APTs to be diagnosed. This decomposition will be derived and illustrated with meteorological data in Delsole and Tippett (2009, hereafter Part II).

The present paper may be summarized briefly as follows: In section 2, we review the measure on which the APT will be based, namely the Mahalanobis metric. We then show in section 3 that the resulting APT gives sensible results for one-dimensional stochastic models; that is, the APT is inversely proportional to the damping rate and comparable to traditional integral time scales in appropriate cases. On the other hand, we show that the APT gives clearly superior results compared to a widely used integral time scale. Next we discuss the relation between predictability and power spectra. Specifically, we show that the APT equals the integral of the squared modulus of the power spectrum and explain how APT depends on the shape of the power spectrum. In section 6 we evaluate the APT for autonomous, linear stochastic models excited by Gaussian white noise. In addition, we show how the multivariate expressions parallel the univariate results if the variables are transformed appropriately. In section 7, we show that the APT for a set of independent, uncoupled stochastic models equals the average APT of the individual systems. In section 8 we derive the relation between APT and multivariate power spectra and show that the minimum APT occurs when the system is a set of independent, uncoupled white noise processes, consistent with intuition. Bounds on the APT of linear stochastic models are derived in section 9 and a surprising interpretation of APT is discussed in section 10. The bounds are illustrated in section 11 with a simple stochastic model. We conclude with a summary and discussion of results.

2. Definition of average predictability time

Consider a system whose state vector is the K-dimensional vector x. Let the forecast distribution at a fixed lead time τ have covariance matrix Σ_τ. This covariance matrix quantifies the uncertainty in the forecast as a function of lead time. If the system is stationary and the forecast is independent of the initial condition at asymptotically long lead times, then the climatological distribution can be identified with the forecast distribution in the limit of large lead time (DelSole and Tippett 2007), namely Σ_∞.

As discussed in the introduction, the first key to defining APT is to choose a measure of predictability that is invariant to nonsingular linear transformation. Although several such measures exist, one turns out to have certain advantages compared to the others. Specifically, we adopt a measure of predictability called the Mahalanobis signal (DelSole and Tippett 2007), defined as

View Expanded

where “tr” denotes matrix trace. The Mahalanobis signal can be interpreted as 1 minus the mean square error, but with the error measured in a space in which the climatological covariance matrix is the identity matrix [DelSole and Tippett (2007) and Schneider and Griffies (1999) effectively show this too]. The Mahalanobis signal equals 1 for a perfect forecast and vanishes when Σ_τ = Σ_∞; that is, it vanishes for a forecast that is no better than a randomly drawn state of the system. The Mahalanobis signal is appropriate only if forecast uncertainty is well characterized by second-moment statistics. As we shall see later, the factor 1/K is used to ensure that a collection of independent, identical systems has the same APT as any subsystem. Schneider and Griffies (1999) note that this factor also allows the predictability of random processes with different dimensions to be compared. Given the above metric, we define APT by

View Expanded

The factor of 2 is introduced to ensure that APT corresponds to an e-folding time if the autocorrelation is exponential (see section 3). For discrete time, we define APT as

View Expanded

where the summation starts at time step τ = 1.

3. One-dimensional case

To illustrate the reasonableness of the above definitions, consider the one-dimensional stochastic model

View Expanded

where a is a negative number and w is a Gaussian white noise process with zero mean and time-lagged covariance given by

View Expanded

where the angle brackets denote an ensemble average and δ denotes the Dirac delta function. The solution to (4) at time τ for a particular realization of the forcing can be solved by elementary methods as

View Expanded

Taking the mean of an ensemble of solutions with mean initial condition mean μ₀ gives

View Expanded

It follows that the mean of the ensemble decays exponentially with a decay rate determined by a. The total variance of the ensemble is

View Expanded

where the initial ensemble and stochastic forcing are assumed to be independent. For simplicity, we assume the initial variance vanishes; that is, σ₀² = 0. In this case, the only source of uncertainty is the stochastic forcing, and the variance of the forecast ensemble given by (8) is

View Expanded

where we have used σ_∞² = −σ_w²/(2a) from (8). If the initial condition is not perfect (i.e., σ₀ ≠ 0), then the predictability is reduced relative to the perfect initial condition case.

The predictability of the system is computed by substituting (9) into the predictability measure (1), which gives

View Expanded

This result shows that the predictability decays monotonically with lead time τ. The APT is found by integrating this expression with respect to lead time and multiplying by 2. The result is

View Expanded

This result shows that APT is inversely related to the decay rate, consistent with physical intuition—systems with strong damping have less “memory” and thus less predictability.

4. Connection between APT and autocorrelation

Intuitively, processes with large autocorrelations are more predictable than processes with small autocorrelations. To formalize this intuition, the relation between forecast uncertainty and time-lagged covariance needs to be defined. For linear prediction models of stationary processes, this relation can be computed readily. Consider the linear prediction model

View Expanded

Standard regression yields the parameters that minimize the mean square prediction error as

View Expanded

where the time-lagged covariance c_τ is defined as

View Expanded

and μ is the stationary mean. The forecast error variance of this model is thus

View Expanded

For stationary processes, the autocorrelation is ρ_τ = c_τ/c₀ and c₀ = σ_∞², in which case the autocorrelation can be written in terms of the forecast covariance as

View Expanded

Substituting this expression into the expressions for APT gives

View Expanded

We see that the APT of a one-dimensional regression model is twice the integral of the square of the autocorrelation function. This function was proposed by DelSole (2001) as a measure of the time scale of a process; it emerges here as a measure of overall predictability. As a check, note that the time-lagged covariance for the stochastic model (4) is

View Expanded

which implies ρ_τ = exp(aτ). Substituting this into (17) gives S = −1/a, consistent with (10).

Having related the APT of a linear regression model to the autocorrelation function of the random process, we may now choose any permissible autocorrelation function for demonstration purposes. Accordingly, consider the damped oscillation function

View Expanded

which is a permissible autocorrelation function for a stationary process (Gelb 1974, p. 82). The APT of a regression model for this process is

View Expanded

The limit ω₀ → 0 recovers the one-dimensional case S = −1/a. However, the limit ω₀ → ∞ gives S = −1/2a, or half the APT of the case with no oscillations. It is instructive to compare the APT with the familiar integral time scale

View Expanded

For the damped oscillation (19), this integral time scale is

View Expanded

If ω₀ = 0, then the integral time scale T₁ coincides with the APT given in (11). This shows that the APT is consistent with the integral time scale T₁ for random processes with nonoscillating autocorrelation functions. However, if ω₀ ≠ 0, then the integral time scale is less than the APT, and in fact T₁ = 0 in the limit ω₀ → ∞ while S = −1/2a. Thus, the APT and integral time scale T₁ have dramatically different dependence on the oscillation frequency. To illustrate this fact, consider the autocorrelations for select values of a shown on the right-hand side of Fig. 1. As ω₀ increases, the frequency of oscillation increases, but the bounding envelope remains the same. In the case a = −1 and ω₀ = 4, the integral time scale T₁ evaluated from (22) is 1/17, but strong correlations persist well beyond this value. It is evident, then, that the integral time scale defined in (21) is not appropriate for oscillatory correlation functions because the oscillations lead to cancellations in the integral. In contrast, the APT gives a more appropriate estimate of the time scale of a process.

5. Connection between APT and power spectrum

We now derive the relation between APT and power spectra. Certain basic points about the relation between predictability and the power spectrum can be inferred by considering two extreme cases. First, white noise can be considered to be the least predictable process because its value at one time is completely independent of its value at any other time. The power spectrum of white noise is constant, or “flat.” Conversely, consider a perfect sine wave. A sine wave is perfectly predictable because the value at a finite number of times can be used to specify the value at all other times. The power spectrum of a sine wave is a delta function. These considerations suggest that strongly peaked power spectra correspond to highly predictable time series, whereas flat or near-flat power spectra correspond to weakly predictable time series.

The above arguments suggest that predictability is related to the degree of flatness or “peakiness” of power spectra, but they are not precise enough to suggest a quantitative relation. We show in this section what properties of power spectra influence predictability as measured by APT. As is well known, the power spectrum p_ω of a stationary process equals the Fourier transform of the time-lagged covariance c_τ; specifically,

View Expanded

Substituting the latter relation into the APT (17) gives

View Expanded

where we have used the relation

View Expanded

We call p_w/σ_∞² the whitened power spectrum. By definition, the whitened power spectrum has a unit integral. The above result shows that the APT for a linear regression model is proportional to the integral of the squared modulus of the power spectrum. To see how this relation relates to the shape of the power spectrum, consider the identity

View Expanded

where overbars denote the mean value over any chosen set of frequencies. The first term on the right-hand side measures the deviations about the mean value. The closer the spectrum is to its mean value (on average), the smaller the first term. A spectrum can be “flattened,” and hence rendered less predictable, by setting the spectrum equal to its mean value over any set of bands; that is, by setting p = p. This flattening process preserves the mean spectrum over the band and therefore preserves the total integral over all frequencies; it also produces positive power spectra because the mean spectrum can never be negative. These considerations suggest that the integral of the squared whitened spectrum is a measure of the degree to which the spectrum differs from a constant—that is, APT is a measure of the peakiness of a power spectrum. The closer the spectrum is to its mean value, in the sense that the first term in (26) is small, and the “flatter” it is, the more closely it approximates white noise and the less predictable the corresponding random process. Conversely, the more the spectrum deviates from the mean, in an average square sense, the stronger the peakiness and the more predictable the corresponding process.

It is perhaps worth mentioning that the above definition of peakiness is not the only one possible. For instance, the Burg entropy is defined as the integral of the log of the power spectrum, and it is known (Priestley 1981, p. 604) that its minimum value, for all power spectra with the same total power, is achieved when the spectrum is a constant (i.e., flat). We propose (24) as a measure of flatness because it has certain practical and theoretical advantages compared to other definitions; that is, it can be estimated from only second-order moments and can be decomposed into components ordered by their APT (the latter point will demonstrated in Part II).

As an illustration of the above results, consider again the autocorrelation (19) for a damped oscillation. The corresponding power spectrum is simply the Fourier transform of (19), which is

View Expanded

(we have used the fact that for any real stationary process c_−τ = c_τ). The power spectrum and autocorrelation function for various choices of a and ω₀ are illustrated in Fig. 1. As expected, the autocorrelation function generally increases as a approaches zero (for fixed τ) and the peak of the power spectrum becomes sharper (i.e., less flat) as a approaches zero. Results for different values of ω₀ are shown in the right panel, but inferences of APT given the power spectrum are not as straightforward (e.g., they require visually integrating the square of the power).

These considerations clarify the error in relating predictability to the location of spectral peaks. Returning to the ENSO example mentioned in the introduction, we note that the Niño-3 index is tolerably fit by the autocorrelation function (19) with a⁻¹ = 16 months and ω₀⁻¹ = 48/(2π) months, as shown in Fig. 2 (We do not recommend this fitting procedure in general; it is used here primarily for illustration purposes.). These parameter values give S = 9.5 months, which differs considerably from the period of the spectral peak, which is 2π/ω₀ = 48 months. In the limit of large ω₀, there is only a modest effect on S, S → −1/(2a) = 8. Thus, the location of the peak is almost irrelevant to the predictability time scale. The reason is that predictability depends on the peakiness of the power spectrum, which is more strongly controlled by the damping time −1/a, rather than on the location of the peak itself (ω₀).

6. Multivariate linear stochastic models

In this section we derive the APT for linear stochastic models of the form

View Expanded

where 𝗔 is a K × K matrix, called the dynamical operator, and w is a Gaussian white noise process with zero mean and covariance matrix 𝗤. The dynamical operator is assumed to be independent of time and stable—that is, it possesses K distinct eigenvalues with negative real parts. Let λ_k(𝗔) denote the kth eigenvalue of 𝗔. Also, let the eigenvector decomposition of 𝗔 be

View Expanded

where the columns of 𝗭 are the eigenvectors of 𝗔, and Λ is a diagonal matrix whose diagonal elements are the eigenvalues of 𝗔. As in the one-dimensional case, assume that the initial ensemble has zero variance. Tippett and Chang (2003) show that the ensemble of solutions to the stochastic model (28) has covariance matrix

View Expanded

where superscript H denotes the conjugate transpose, −H denotes the inverse of the conjugate transpose,

View Expanded

and 𝗖 ∘ 𝗗 denotes the Hadamard product of 𝗖 and 𝗗; that is, (𝗖 ∘ 𝗗)_ij = 𝗖_ij𝗗_ij). [This result also is derived in DelSole and Tippett (2007); the stationary form of this solution can be found in Horn and Johnson (1999, p. 301)]. The solution (30) is the multivariate generalization of (8). A further standard result is that the forecast covariance matrix (30) also satisfies

View Expanded

which is the multivariate generalization of (9).

Analogous to the one-dimensional case, the climatological covariance matrix is identified with the asymptotic forecast covariance matrix Σ_∞. Substituting (32) into the predictability measure (1) gives

View Expanded

The APT is obtained by integrating this expression over all lead times τ. Adopting the convention that repeated indices are summed, we have

View Expanded

Substituting the expression in (30) for Σ_∞ into (34) gives

View Expanded

The last expression relates APT directly to known parameters of the stochastic model.

An alternative, and particularly revealing, expression for predictability can be obtained by introducing the matrix square root Σ_∞^1/2 of the climatological covariance, which satisfies

View Expanded

where H/2 denotes the Hermitian of the square root matrix. A square root matrix always exists for positive definite Σ_∞, although it is unique only up to a unitary matrix multiplied on the right-hand side. Substituting (36) into the predictability measure (33) and invoking standard properties of the trace, determinant, and exponential operators yields

View Expanded

View Expanded

Expression (37) is the multivariate generalizations of (11). Here, is called the whitened dynamical operator and is the dynamical operator governing the transformed variable

View Expanded

Transformation (39), called the whitening transformation, appears frequently in information theory, predictability theory, and pattern recognition theory (Fukunaga 1990; Schneider and Griffies 1999; Majda et al. 2002; Tippett and Chang 2003). See DelSole and Tippett (2007, 2008) for a review of this transformation and its importance to predictability theory.

7. Normal case

We now consider linear stochastic models of the form (28) with diagonal dynamical operator 𝗔 having diagonal elements λ_k(𝗔) and diagonal noise covariance matrix 𝗤 having diagonal elements Q_kk. This case corresponds to a set of uncoupled, damped oscillators that are excited independently. In this case, the eigenvector matrix 𝗭 is the identity matrix, and the forecast covariance matrix computed from (30) is diagonal, with the kth diagonal element being

View Expanded

We assume that the noise variances Q_kk are strictly positive so that all the eigenmodes are excited and that the climatological is invertible. Substituting this expression into the predictability measure (1) gives

View Expanded

Integrating this expression over all lags gives

View Expanded

which is the mean damping time of the eigenmodes. Comparing this expression with its one-dimensional counterpart (11) shows that the APT of a set of independently excited, uncoupled oscillators equals the average APT of the individual oscillators.

Because APT is invariant to linear transformations, the above results hold for a wider class of models than merely those with a diagonal dynamical operator and noise covariance matrix. In fact, if the dynamical operator has the eigenvector decomposition (29), then the above result holds if the noise covariance matrix is of the form

View Expanded

where 𝗗 is a diagonal matrix with real positive entries. Thus, a more general result is that if there exists a basis set in which both the dynamical operator and noise covariance matrix are diagonal, then the APT of the system is given by (42). Intuitively, if both the dynamical operator and noise covariance matrix can be rendered diagonal by a suitable linear coordinate transformation, then the dynamical system is fundamentally equivalent to a set of independently excited, uncoupled oscillators. It is interesting to note that this condition is precisely the condition for the stochastic model to satisfy the principle of detailed balance (Weiss 2003).

The condition (43), where 𝗭 is the eigenvector matrix of 𝗔, can be expressed in a more revealing way. Substituting (43) into the forecast covariance matrix (30) gives

View Expanded

Let us now compute the whitened dynamical operator defined in (38). Because the square root matrix is unique up to a unitary matrix, the most general square root matrix is

View Expanded

where 𝗨 is an arbitrary unitary matrix (this square root is well defined because 𝗗 ∘ 𝗘_τ is a real, positive definite, diagonal matrix). Substituting this expression into the definition of the whitened dynamical operator (38) gives

View Expanded

where we have used 𝗭⁻¹𝗔 𝗭 = Λ and the fact that diagonal matrices commute. This result shows that if the noise covariance matrix 𝗤 is of the form (43), then is normal. We conclude then that if the whitened dynamical operator is normal, then the system is fundamentally a set of independent, uncoupled oscillators and the APT is given by (42), which depends only on the real part of the eigenvalues. It is worth noting that although the question of whether 𝗔 is normal depends on the coordinate system, the normality of is invariant under nonsingular linear coordinate transformations.

8. Relation between the power spectrum and predictability

We now derive the relation between APT and power spectrum for multivariate systems, thereby generalizing the results of section 5. We assume the time series is stationary and band limited, in which case the power spectra are limited to finite frequencies and only discrete time steps are needed. According to the Wiener–Khinchin theorem, the time-lagged covariance matrix

View Expanded

is related to the power spectrum matrix 𝗣_ω as

View Expanded

The time-lagged covariance matrix 𝗖_τ can be related to the forecast covariance matrix Σ_τ only after a forecast model is invoked explicitly. Here we consider linear forecast models of the form

View Expanded

where x̂ is a prediction of x and 𝗟_τ is a linear operator. Determination of the linear operator that minimizes the sum of squared forecast errors is a standard regression problem with solution

View Expanded

The forecast error covariance matrix for this model is therefore

View Expanded

This expression is the multivariate generalization of (15). Furthermore, this expression is equivalent to (32) with the identification 𝗟_τ = exp(𝗔τ), reflecting the close relation between linear stochastic models and linear regression models.

Having computed the forecast covariance matrix (51) for the linear prediction model, we can now substitute it into the Mahalanobis signal (1), which gives

View Expanded

A subtlety is that the linear prediction model (50) is valid only for positive time lags τ. However, for any stationary process, 𝗖_−τ = 𝗖_τ^H. Substituting this identity into the above equation implies S_−τ = S_τ; that is, our expression for S_τ is an even function of lag. Therefore, we may sum this expression over positive and negative lags. Substituting the Wiener–Khinchin relation for the power spectrum, while noting that S₀ = 1, gives

View Expanded

View Expanded

View Expanded

Using the fact that

View Expanded

gives

View Expanded

Factoring the zero-lag covariance matrix as in (36) and invoking standard properties of the trace operator yields

View Expanded

where _τ and _ω are the whitened time lag covariance and power spectrum matrices, respectively, defined as

View Expanded

The relation between APT and power spectrum given in (58) is the multivariate generalization of (24) (but restricted to band-limited spectra).

The restriction to band-limited power spectra allows us to place rigorous bounds on the APT. Specifically, we show in the appendix that

View Expanded

with equality if and only if the power spectra is

View Expanded

The later condition implies that minimum predictability occurs when the power spectrum of all variables can be transformed into a set of uncorrelated, white noise processes, as intuition would predict. We suggest that (58) is a kind of multivariate generalization of spectral peakiness, as defined in section 5, although we emphasize that it is not the only reasonable measure of peakiness. The intuitive result follows that the absolute minimum APT occurs when all processes are independent white noise (i.e., flat). In the next section, we consider the minimum APT subject to the constraint that the dynamical operator is known.

9. Bounds on APT

In this section we derive bounds on the APT of linear stochastic models (28). Bounds on predictability at fixed lead time have been derived in Tippett and Chang (2003) and Weiss (2003) and reviewed in DelSole and Tippett (2007). Tippett and Chang (2003) show that a lower bound for the Mahalanobis signal is given by

View Expanded

Because the lower bound holds at each lead time individually, a lower bound for APT can be found by integrating both sides over all lead times and multiplying by two, which yields

View Expanded

Note that the right-hand side is identical to (42). It follows that the minimum APT occurs when there exists a basis set in which the dynamical operator 𝗔 and noise covariance matrix 𝗤 are diagonal, which in turn implies that the system is fundamentally a set of uncoupled, damped oscillators that are excited independently. This condition for minimum APT is precisely the condition for minimum predictability at fixed lag, as well as the condition for detailed balance (Tippett and Chang 2003; Weiss 2003). These correspondences are not surprising because predictability is minimized at each individual lead time and hence minimized for the integral over all lead times.

An upper bound on APT can be derived from a majorization property due to Cohen (1988). Specifically, for matrix exponentials, Cohen (1988) proves that

View Expanded

where n = 1, 2, …, K. As is well known, any increasing, convex function preserves majorization (Horn and Johnson 1999, p. 173). Because exp(2x) is an increasing, convex function, it follows that an upper bound on Mahalanobis signal is

View Expanded

where 𝗔_s = ( + ^H)/2 is the symmetric part of . These inequalities become equalities if and only if is normal, in which case these upper bounds equal the lower bound (62). Because this upper bound holds at each lead time individually, the upper bound for APT can be found by integrating both sides over all lead times and multiplying by two, which yields

View Expanded

We see that the upper bound (66) and lower bound (63) are proportional to a sum of inverse eigenvalues, but the respective eigenvalues are based on different operators. The operator 𝗔_s appearing in the upper bound (66) arises in a variety of contexts related to the instantaneous rate of change of variables. For instance, the eigenvectors of 𝗔_s are called instantaneous optimals and define an orthogonal set of initial states that optimize the instantaneous rate of change of x̃^Hx̃ (see DelSole 2004 for review). In the present situation, however, the initial condition plays no role in the predictability because all unpredictability arises from stochastic forcing (i.e., the initial condition is assumed to be known perfectly). The relevance of the operator in the present situation can be seen as follows: It is straightforward to show from (30) that dΣ̃_τ/dτ = at τ = 0. Furthermore, the Lyapunov equation 𝗔Σ_∞ + Σ_∞𝗔^H + 𝗤 = 0 implies = −( + ^H), where we have used the fact that Σ̃_∞ = 𝗜, by definition. Combining these results implies that

View Expanded

Thus, 2𝗔_s is the rate of change of the whitened forecast covariance matrix at τ = 0. Interestingly, we would obtain the same rate of change if the original dynamical operator had been 𝗔_s. It follows that the upper bound (66) can be derived by replacing the original dynamical operator by the normal dynamical operator 𝗔_s, which gives precisely the same rate of predictability loss at the initial time as the original system, then integrating the predictability of the system over all times.

The upper bound (66) can be written explicitly in terms of model parameters using the fact that

View Expanded

where we have used (30) and the fact that λ_k(𝗔𝗕) = λ_k(𝗕𝗔) for any two matrices 𝗔 and 𝗕. Using this identity, the upper bound (66) becomes

View Expanded

If the noise covariance matrix is of the form (43), then the above upper bound becomes identical to the lower bound (62). It is also apparent that if 𝗤 is singular, then the upper bound given above is not defined, consistent with the fact that the original upper bound (66) diverges to infinity in this case. Thus, this upper bound is useful only when 𝗤 is full rank. On the other hand, this upper bound requires knowing the same number of parameters as the full solution, whereas the other bounds require fewer parameters. Nevertheless, the above upper bound is conceptually useful because it relates the APT of linear stochastic models to the predictability at short times.

An alternative upper bound on predictability can be derived from the conjecture of Tippett and Chang (2003) that maximum predictability occurs when the noise covariance matrix is rank 1 and excites all eigenmodes. As Tippett and Chang (2003) show, this conjecture gives the upper bound

View Expanded

Remarkably, the response to rank-1 forcing is independent of detailed structure of the forcing. This independence can be explained as follows: A rank-1 noise covariance matrix implies that each eigenmode is excited in sync with other eigenmodes. The response thus depends only on the amplitude with which each mode is excited, but any set of noise amplitudes can be transformed into any other set by multiplying each eigenmode equation by an appropriate factor, which corresponds to a linear transformation that does not affect predictability. Standard calculus gives

View Expanded

where the definition (31) has been used. Therefore, integrating (70) and multiplying by 2/K gives the conjectural upper bound

View Expanded

An advantage of this conjectured upper bound is that it is an explicit solution, thus ensuring that it is actually achieved by a specific stochastic model. In contrast, the upper bound (66) is achieved only when the whitened dynamical operator is normal, but then the upper bound equals the lower bound, which is not an interesting limit. Another distinction is that the upper bound (72) depends only on the eigenvalues of the dynamical operator, in contrast to the upper bound (66). Finally, it should be recognized that this upper bound is conjectural. These bounds will be illustrated with a simple example in section 11.

10. APT as a solution to an alternative stochastic model

A useful concept in the application of information theory predictability measures to normally distributed variables is that techniques for analysis of predictability are often equivalent to techniques for analysis of variance to applied whitened variables (Schneider and Griffies 1999; DelSole and Tippett 2007). Here we show how APT fits in this framework. Generalized stability analysis relates the variance of linear dynamics forced by homogeneous isotropic stochastic forcing to the stability properties of the linear dynamics (Farrell and Ioannou 1996; Tippett and Marchesin 1999). Here we show that the variance of whitened linear dynamics excited by homogeneous isotropic stochastic forcing is related to the predictability of the system as measured by APT.

The asymptotic solution to the stochastic model (28) can be written in the alternative form (Lancaster and Tismensetsky 1985)

View Expanded

This expression can be transformed into the form of (37) by making the substitutions 𝗔 → and 𝗤 → 𝗜 in (73), then taking the trace and multiplying by 2/K. This equivalence implies that the APT can be interpreted as the “total variance” of a stochastic model with dynamical operator and noise covariance matrix 𝗜. This result bears repeating: APT can be expressed as

View Expanded

where Σ_y is the stationary covariance matrix of y obtained from the stochastic model

View Expanded

where v is a Gaussian white noise process with zero mean and covariance matrix 𝗜. The fact that APT equals the total variance of a stochastic model implies that the intuition developed about total variance produced by stochastic models can be applied directly to APT. As an example, a system with a nonnormal dynamical operator generates more variance than a system with a normal dynamical operator with the same eigenvalues (Ioannou 1995). Applying this theorem to the stochastic model (75) immediately implies that nonnormality of enhances APT, consistent with the result derived in section 9. Another consequence of the above equivalence is that results from dynamical stability theory can be used to understand APT. For instance, it is known that the covariance matrix Σ_y derived from the stochastic model (75) satisfies the Lyapunov equation

View Expanded

Furthermore, there exists a wide variety of bounds on the solutions to Lyapunov equations (Kwon et al. 1996). For instance, bound (106) from Kwon et al. (1996) implies the upper bound (66).

11. Example

We illustrate some properties of APT with a simple stochastic model; a more detailed example based on meteorological data will be given in Part II. Consider a two-dimensional linear stochastic model of the form (28) with noise covariance matrix 𝗤 = 𝗜 and dynamical operator

View Expanded

where c is a parameter measuring the coupling between the two components. Because 𝗔 is upper triangular, its eigenvalues are simply the diagonal elements −1/3 and −1. The negative eigenvalues imply that the dynamical operator is stable and that the stochastic model gives statistically stationary solutions. The predictability of the system is minimized and achieves the lower bound (63) when there is no coupling between the two components of the system (i.e., when c = 0). The time-dependent predictability for the case c = 1 and c = 4 are shown in the top panels of Fig. 3. Also shown are the upper and lower bounds. As expected, as the coupling parameter increases, so too does the predictability, at all lead times. For c = 1 the APT is S = 2.54; for c = 4 the APT is S = 3.35. For comparison, the conjectured upper bound (72) is 3.5, whereas the lower bound is 2. In general, predictability is an increasing function of coupling parameter c. The two upper bounds provide tight constraints in different extremes—the bound based on instantaneous optimals performs well at small lead times, whereas the conjectured bound performs well at long lead times. The lower panels show the average predictability time as a function of coupling parameter c. The upper bound based on instantaneous optimals greatly overestimates the APT at large values of the coupling parameter. A good upper bound is the minimum of the two upper bounds.

12. Summary and discussion

This paper introduced average predictability time (APT) for characterizing the overall predictability of a system. The APT is defined as the integral of the Mahalanobis signal with respect to lead time. As such, APT measures an inherent property of a system that is independent of the basis set used to represent the system. The appropriateness of APT was illustrated with a one-dimensional stochastic model in which the APT depends inversely on the damping rate, consistent with the intuition that systems with stronger damping have less memory and hence are less predictable. Furthermore, the APT is comparable to integral time scales in appropriate cases. However, if the autocorrelation is a damped oscillation, then the integral time scale defined in (21) becomes arbitrarily small for large frequencies, which is unrealistic, whereas APT is always within 50% of the e-folding time. Thus, the APT provides a more suitable measure of time scale than the integral time scale (21). For nonlinear or non-Gaussian processes, the Mahalanobis signal may decay very differently from that of linear stochastic systems (e.g., it may exhibit long tails). Nevertheless, just as the correlation coefficient has meaning even if the variables are not linearly related, the APT still has meaning even if it cannot be interpreted directly as a time scale.

The APT also clarifies the connection between predictability and power spectra. Specifically, if a process is stationary, then the APT of a linear regression forecast is proportional to the integral of the square of the normalized power spectrum. The appropriateness of this relation can be seen from the fact that the latter quantity can be decreased simply by replacing the power in any spectral band by its mean value within the band; that is, the quantity decreases as the spectrum becomes flatter. If the time series is band-limited, then the minimum value occurs when the system comprises a set of independent and identically distributed white noise processes, which is obviously the least predictable system. In essence, predictability is related to the width of spectral peaks, with strong narrow peaks associated with high predictability and nearly flat spectra associated with low predictability. As extreme examples, white noise has a constant power spectrum and is minimally predictable, whereas a sine wave has a delta-function power spectrum and is perfectly predictable. Expressing the APT in terms of the power spectra rigorously quantifies this intuitive relation.

Closed form expressions for the APT of linear, multivariate stochastic models were derived. If the dynamical operator and noise covariance matrix are both diagonal, then—remarkably—the APT depends only on the damping rates and equals the average APT of the individual eigenmodes. Because the APT is invariant to linear transformation, this result is true for any system for which there exists a basis set in which both the dynamical operator and noise covariance matrix are diagonal. We further show that this particular system minimizes the APT compared with all linear stochastic models with the same dynamical operator. Thus, the least predictable system can be transformed into a set of uncoupled, independent stochastic models. It follows that systems that are irreducibly coupled have more predictability than those that are fundamentally uncoupled. Simply put, coupling enhances predictability. APT rigorously justifies this intuitive notion.

As reviewed in DelSole and Tippett (2007), applying the whitening transformation to a system allows familiar concepts from analysis of variance and generalized stability analysis to be applied directly to predictability analysis. We show that this equivalence carries over to APT because, surprisingly, APT itself can be interpreted as the total variance of an alternative stochastic model. The alternative stochastic model is driven by homogeneous white noise and has a dynamical operator equal to the whitened dynamical operator of the original stochastic model. This connection allows one to anticipate that the lower bound for APT occurs when the whitened dynamical operator is normal, which in turn is equivalent to the condition that there exists a basis set in which both the dynamical operator and noise covariance matrix are diagonal—consistent with the lower bound of APT discussed above.

The remarkable equivalence noted above is worth further reflection. Loosely speaking, one way to understand the stability of a dynamical system is to examine its response to white noise forcing. The more variance produced, the less stable the system. This approach is the basis of generalized stability theory, which in turn utilizes concepts from dynamical stability theory. In defining APT, we proposed something apparently different: we proposed that the overall predictability of a system can be quantified by the integral of predictability with respect to lead time, reasoning that more predictable systems have larger integrals. Surprisingly, the integral of predictability is itself the total variance of the system consisting of the whitened dynamical operator forced by white noise. APT measures the stability of the whitened dynamics. Thus, the two methods for understanding predictability turn out to be fundamentally equivalent. An important benefit in this equivalence, however, is that the whitened dynamical operator is unique up to a variance preserving unitary transformation. Thus, the framework proposed in this paper imposes a particular stochastic model for studying predictability, which is essential for applying generalized stability theory. In contrast, the latter theory does not impose the norm or coordinate system; rather, it provides a framework for understanding variance after the norm and coordinate system have been chosen.

An upper bound on the APT of linear stochastic models also was derived. The upper bound is proportional to the sum of inverse eigenvalues of the symmetric part of the whitened dynamical operator. The latter operator arises frequently in the investigation of instantaneous growth rates of variance. It is interesting that the upper bound on APT (or the predictability at any lead time) is related to the predictability at very short time scales in linear stochastic models. The upper bounds imply that as the short time predictability decreases, so too does the maximum possible APT. This intuitive notion is used widely in predictability studies as a justification for drawing conclusions about overall predictability from the characteristics of error growth at short times. Two key points about this relation are worth emphasizing. First, although intuitive, we are aware of no rigorous demonstration of a connection between predictability at short times and overall predictability. Thus, our result provides such a demonstration, at least for linear stochastic models. Second, the connection between short time error growth and APT is most direct when the error growth is measured in whitened space. In other words, our result clarifies the existence of a preferred norm for relating predictability at short and long lead times. Attempts to draw conclusions about long-term predictability from short-term error growth in other norms may actually be misleading (see DelSole and Tippett 2008).

A conjecture for an upper bound for APT follows from the conjecture of Tippett and Chang (2003), which states that out of all linear stochastic models with the same dynamical eigenvalues, the model with rank-1 noise covariance matrix (with nonzero diagonal elements) has maximum predictability. This conjecture can be motivated by the fact that if minimum predictability occurs when all eigenmodes are forced independently, then perhaps maximum predictability occurs when all eigenmodes are forced in perfect correlation. The resulting upper bound depends only on the eigenmode damping rates; in particular, the upper bound is independent of the detailed structure of the forcing, provided all modes are excited.

It should be recognized that APT can be applied to more general forecast models than linear stochastic models. For instance, APT can be evaluated for regression models with physically different variables for predictors and predictands. Interestingly, if only a single variable is predicted, but more than one predictor is used, then the APT relation (17) still is applicable, but the parameter ρ_τ denotes the multiple correlation. This result is discussed further in Part II of this paper.

We note that other measures of predictability can be used to define APT, and these alternative measures may be attractive in some cases. Mutual information in particular is appropriate for non-Gaussian or nonlinear systems. Furthermore, mutual information is integrable for linear stochastic systems, even though it is unbounded as lead time approaches zero for perfect initial conditions. For linear stochastic models with Gaussian white noise, the APT derived from mutual information turns out to be proportional to the APT (42) for normal whitened dynamical operators. However, the relation between this alternative form of APT and power spectra is obscure, and the decomposition of this form of APT is not straightforward.

In climate systems, the APT of different components can vary greatly owing to the widely different time scales associated with land, ice, and ocean processes. Thus, characterizing the predictability of the full climate system by a single number might seem to be a gross simplification. However, the APT can be decomposed into uncorrelated components that can be ordered by their fractional contribution to APT, such that the first component maximizes APT, the second maximizes APT subject to being uncorrelated with the first, and so on. This decomposition therefore allows the full spectrum of predictability times to be diagnosed. This decomposition can be used to study predictability on different time scales without time filtering, provided the predictabilities on different time scales are characterized by different spatial structures. This decomposition and its practical implementation are discussed in Part II.