a. Model distance functional

Let x₀, … , x_T ∈ Ψ ⊂ Rⁿ be the observed n-dimensional time series with T + 1 snapshots in time interval [0, T]. In the following we will assume that in this time interval the considered time series x_t is approximated by a time-discrete output of the certain direct mathematical model

View Expanded

where F(·) is the model operator, τ is the model time step, mτ is the memory depth (m = 1 for Markov models), and

View Expanded

is a (time-dependent) set of the model parameters (including, if necessary in some model contexts, some initial and/or boundary values), where d is the dimension of a model parameter space. Let

View Expanded

be a functional (called the model distance functional) describing the distance between some given x_t at time t and the output of the model (1) calculated for a fixed set of parameters θ(t). In this case, for a given observation series x₀, … , x_T and some fixed functional form g(·), the inverse problem (or the parameter identification problem) can be approached via the solution of the following variational problem:

View Expanded

subjected to the constraints given in (2). Problem (4) is clearly ill posed if no special assumptions about the temporal dependence of the unknown parameters θ(t) can be made. In the following we will assume that for any t ∈ [0, T] model distance, functional (3) can be represented as a convex linear combination of K ≥ 1 stationary model distance functionals—that is, model functionals dependent on some constant (time-independent) model parameters θ_i ∈ Ω, i = 1, … , K:

View Expanded

with some time-dependent model affiliations γ_i(t) fulfilling the convexity condition

View Expanded

View Expanded

In another words, we assume here that at any time t the global time-dependent (or nonstationary) model distance functional (3) can be approximated by one of K local time-independent (or stationary) model distance functionals chosen according to some time-dependent probabilities (or model affiliations) Γ(t) = [γ₁(t), … , γ_K(t)]. This idea for the inverse numerical problems is widely used in the context of data clustering (Höppner et al. 1999); in the presented general form it was introduced in Horenko (2010a) and stems from the classical spline interpolation approach for direct numerical problems (see, e.g., Deuflhard 2004). Inserting the ansatz (5) into (4) results in the minimization of the average clustering functional

View Expanded

subject to (2), (6), and (7) with Θ = (θ₁, … , θ_K).

To comprehend the above concepts it is instructive to consider a case where the direct model (1) has a following simple form:

View Expanded

where ϵ_t is some independent identically distributed (i.i.d.) stochastic variable with zero expectation and θ(t): [0, T] → Rⁿ is a time-dependent parameter describing the evolution of the expectation value of the process x_t. The model distance functional (3) in such a case has the form

View Expanded

and the corresponding average clustering functional can be numerically minimized applying the standard K-means clustering algorithm (Bezdek 1981; Höppner et al. 1999). This means that with the help of the ansatz (5), the solution of the nonstationary inverse problem for dynamical system (9) can be approached via the iterative clustering algorithm based on the minimization of the average clustering functional (8).

In the following it will be explained in detail how these concepts can be interpreted and applied in context of more general nonstationary multivariate models with external forcing (VARX models).

b. Nonstationary VARX models and VARX model distance functional

The stationary VARX model is a widely used dynamical multivariate tool to investigate the time series subject to external forcing (Brockwell and Davis 2002; Tsay 2005). If, in addition to the time series x₀, … , x_T ∈ Rⁿ (describing the internal degrees of freedom of the considered dynamical system), the time series u₀, … , u_T ∈ R^l of the external influences (or forcing) is available, the nonstationary nonlinear VARX model has the following form:

View Expanded

where ϕ₁(x_t−τ, … , x_t−mτ): R^nm → R^d is some (in general nonlinear) function connecting the previous observations x_t−τ, … , x_t−mτ; ϕ₂(u) = {ϕ₂¹[u(t)], … , ϕ₂^k[u(t)]} : R^l → R^k is some fixed (nonlinear) function of the external factors; ϵ_t : [0, T] → R^h(h ≪ n) is a Gaussian process with zero expectation; ; μ(t) : [0, T] → Rⁿ; 𝗔(t) : [0, T] → R^n×d; 𝗕(t) : [0, T] → R^n×k; and 𝗖(t) : [0, T] → R^n×h.

The most simple and straightforward form of the VARX model used in the literature is the linear autoregressive factor model with ϕ₁(x_t−τ, … , x_t−mτ) = [x_t−τ, … , x_t−mτ] and a VARX model equation (Brockwell and Davis 2002):

View Expanded

We will further assume that the noise matrix 𝗖(t) (describing the coupling between the h-dimensional Gaussian noise process to the analyzed time series x_t) for any t ∈ [0, T] can be represented as

View Expanded

where 𝗣(t) : [0, T] → R^n×h is an orthogonal matrix function and Λ(t) : [0, T] → R^h×h is a diagonal matrix function with nonnegative diagonal elements. Defining θ(t) = [μ(t), 𝗔(t), 𝗕(t), 𝗖(t)] and under the assumption (13), the VARX model distance functional (3) of dynamical system (11) can be written as

View Expanded

where the 𝗣(t)-weighted norm ‖·‖_𝗣(t) = [·𝗣(t), 𝗣^†(t)·]₂ is used and † denotes the matrix transposition. The main advantage of the above definition of the least squares residual norm (14)—as compared to the standard Gaussian norm based on the 𝗖(t)-weighted norm ‖·‖_𝗖(t) = [·, 𝗖(t)·]₂—is that it preserves the norm of the original residuals of the least-squared problem in the essential noise dimension. If the aforementioned assumptions are fulfilled, then it is easy to demonstrate that the time series of model distances g[x_t, θ(t)], t = 0, … , T is a χ² process. In the application examples below it will be demonstrated how this property of the process can be used to estimate the confidence intervals of the model parameters. In the context of stationary VARX models (e.g., in the case of the time-independent parameter matrices 𝗔, 𝗕, and 𝗖) this property can be deployed to a posteriori check the model assumption (11) and to demonstrate the asymptotical normality of the resulting parameter estimates (i.e., to demonstrate that for T → ∞ the parameter estimates are distributed according to the multivariate Gaussian; Reinsel 1993). Applying the ansatz (5), the nonstationary parameter identification problem can be approached via the solution of the minimization problem (8) [with θ_i = (μⁱ, 𝗔ⁱ, 𝗕ⁱ, 𝗖ⁱ)]. This can be done by means of the iterative clustering algorithm (just as described for the K-means clustering; Höppner et al. 1999; Horenko 2010b).

c. Incorporation of additional information

Direct numerical treatment of the problem (8) is hampered by the following issues: (i) the problem is ill posed since the number of unknowns can be higher then the number of known parameters and (ii) because of the nonlinearity of g, the problem is in general nonconvex and the numerical solution gained with some sort of local minimization algorithm depends on the initial parameter values (Deuflhard 2004). Perhaps it is even more important that the solution Γ of the above constrained minimization task might be an irregular function: to see this, let us assume that we already know the minimizing values Θ_* for Θ. Then, the minimizer Γ_* for the affiliation vector Γ has the following form:

View Expanded

Thus, the datum x_t has perfect affiliation with state i if the model distance functional for x_t is minimal in state i. That is, if the process exhibits strong variability, then the affiliations are rather nonsmooth functions. Whenever the affiliation functions just take values 0 or 1, we will call them deterministic, in the sense that then for every datum in the time series it is certain to which cluster it belongs.

As demonstrated in the literature (Horenko 2009, 2010b), one of the possibilities for approaching the two aforementioned problems simultaneously is first to incorporate some additional information about the regularity of the observed process [e.g., in the form of smoothness assumptions in the space of time-continuous functions Γ(·)] and then to apply a finite Galerkin discretization (e.g., a FEM discretization) of this infinite-dimensional Hilbert space. In context of Tykhonov-based FEM-clustering methods, this was done assuming the weak differentiability of the time-continuous functions γ_i; that is,

View Expanded

where W_1,2(0, T) is the Sobolev space of weakly differentiable functions [e.g., functions with ₂(0, T) integrable first derivatives].

As demonstrated in Horenko (2010a), one possibility to incorporate this a priori information from (16) into the optimization is to modify the functional (8) and to write it in the Tykhonov-regularized form

View Expanded

However, introduction of the ϵ²-dependent penalty term in the formulation of the Tykhonov-regularized problem (17) changes the functional form of the original clustering problem (8) and biases the position of the solution of the respective minimization problem with growing ϵ² (e.g., the solution of the regularized problem may have a significant deviation from the global minimum of the original problem). As was demonstrated in Horenko (2009, 2010b), increasing the value of ϵ² leads to the “smoothing out” of the sharp transitions between the cluster states.¹ Moreover, the formulation (16) of the persistency condition in W_1,2 sense relies on the differentiability and continuity of the underlying cluster affiliation functions γ_i(t). This cannot in general be assumed to be granted in the cases where the transitions between the cluster states are sharp [e.g., when the function γ_i(t) has jumps and is discontinuous].

In the following, an alternative way to incorporate the persistency assumption into the original clustering problem (8) (avoiding the two abovementioned problems) will be presented, based on the formulation of the persistency condition in a functional space allowing for a discontinuity of its elements [functions with bounded variation, BV(0, T)].

d. Persistence in the BV(0, T) sense: The constrained BV-clustering method and FEM discretization

Instead of limiting ∂_tγ_i in the ² sense [which relies on the differentiability and continuity assumption for the cluster affiliation function γ_i(t)], we will consider the time-discrete functions γ_i(t) defined only at the time instances where the observations x_t are available. We will formulate the persistency condition in the time-discrete BV(0, T) sense

View Expanded

where the persistency parameter C defines the maximal number of transition between the cluster state i and all other states in time interval (0, T). Note that since at least in the time-continuous case W_1,2(0, T) ⊂ BV(0, T) (Moreau 1988), this kind of persistency condition will also allow us to preserve the “smooth” W_1,2 transitions between the cluster states in the continuous limit by an appropriate choice of the persistency parameter C.

Let 𝗗 be the discrete difference operator (the right-hand derivatives) with regard to t:

View Expanded

For a given Θ = (θ₁, … , θ_K), let

View Expanded

Then problem (8) transforms to

View Expanded

where † denotes the transposition operation. The above problem (21) is subject to the constraints

View Expanded

View Expanded

View Expanded

The direct numerical optimization of the problem (21)–(24) is hampered by the fact that the persistency constraint (22) makes the overall problem nondifferentiable. In the following, an adequate transformation of the above problem to the differentiable formulation will be introduced. This will allow us to use the standard numerical optimization methods in the context of (21)–(24). We define υ_i := Dγ_i^† and split this into nonpositive and nonnegative parts (following the theorem about the unique representation of the BV functions; cf. Moreau 1988):

View Expanded

Moreover, let 𝗗⁻¹ be the discrete integration operator; for example,

View Expanded

for some variable (defining the initial value for the cluster affiliation i at time 0). Now we can express γ_i^† as

View Expanded

Defining

View Expanded

we can express the original nondifferentiable optimization problem [(21)–(24)] as the following differentiable optimization problem (for more details, see the appendix):

View Expanded

in the vector space of the higher dimensionality [since by construction the dimension of variable x̃_i defined in (28) is almost twice as high as the dimension of the original variable γ_i]. The solution of the above minimization problem can be approached via the subspace iteration procedure (e.g., via the solution of the restrained optimization problems in parameter subspaces x̃ and θ subsequently). Completely analogously to the Tykhonov-regularized FEM-clustering case (Horenko 2009, 2010b), it can be demonstrated that this subspace iteration procedure converges toward the (local) minimum of the problem (29) if some appropriate assumptions (convexity and differentiability) of the model distance functional (3) are fulfilled.

However, since the dimensionality of the variable x̃ is growing as K(2T + 1) with the length of the analyzed time series, the numerical solution of the restrained problem (29) for a fixed value of Θ can become increasingly expensive for long time series. In the following the finite element method will be deployed to reduce the dimensionality of the above problem.

e. FEM discretization

Let {0 = τ₀, τ₁, τ₂, … , τ_N−1, τ_N, τ_N+1 = T} be a finite subset of the interval [0, T] with uniform time steps δ_t. We can define a set of N ≪ (T + 1) time-discrete functions with bounded variation {f₁(t), f₂(t), … , f_N(t)} defined at T + 1 time points of the observation series x, where each function f_i(t) shall take positive values at the observation time instances of x in the time interval (τ_i−1, τ_i+1) and be zero at the time instances outside this interval.² Now we represent the υ’s from (25) by these functions; thus,

View Expanded

where χ_N⁺ and χ_N⁻ are discrete discretization errors. As δ_t goes to 1, the discretization errors become zero. This develops into a reduced discrete representation of the discrete problem (29) by using

View Expanded

where 𝗪_N ∈ R^(T+1)×N is a FEM-basis matrix and ṽ_i⁻ is defined analogously. Then the previously defined x̃_i can be approximated as

View Expanded

Defining

View Expanded

results in the FEM-discretized problem in the BV sense:

View Expanded

Note that x has a dimensionality of K(2N + 1), which is much less then the dimension K(2T + 1) of the original variable x̃ if N ≪ T. Analogously to the Tykhonov-regularized FEM-clustering problem described in Horenko (2009), adaptive FEM techniques can be deployed to find the optimal set of time intervals δ_t for a given total discretization error δ̃.³

For a fixed Θ the above minimization problem is a linear minimization with regard to x with linear equality and inequality constraints. This problem can be solved by means of some standard numerical methods of linear programming like simplex or interior point methods. For the fixed parameter x, the above problem is an unconstrained minimization problem that can be solved analytically if the model distance functional g is convex with regard to θ and the problem ∂g(x_t, θ)/θ = 0 has a unique analytical solution with regard to θ (as will be demonstrated in the following, this is the case for the VARX models). The iterations can be repeated until the change of the functional value does not exceed some predefined threshold for the change of the functional value. The algorithm is:

• choose an arbitrary x⁰

• set s = 1

• while not converged repeat

  • step 1: solve (34) with respect to (w.r.t.) Θ for a fixed x^s−1 analytically and identify Θ^s

  • step 2: solve (34) w.r.t. x for a fixed Θ^s numerically (via linear programming) and identify x^s

  • set s = s + 1

Compared to the Tykhonov-regularized FEM-clustering problem as described in Horenko (2009, 2010b), the above problem [(34)] has two major numerical advantages. First, while for the Tykhonov-regularized FEM-clustering problem the persistence could be influenced only indirectly through the choice of the regularization parameter ϵ², in the context of (34) it is directly controllable via the persistency threshold C. Second, a sparse linear programming problem is solved in each iteration of the above algorithm (compared to the more numerically expensive quadratic programming in the case of the Tykhonov-regularized FEM-clustering problem).

In the next section it will be demonstrated how the above numerical procedure can be formulated for the parameter identification of nonstationary VARX models (11).

a. Derivation of the estimator formulas

In every iteration s of the subspace iteration algorithm described above, the unconstrained minimization problem

View Expanded

is solved for a fixed value of x̃^(s−1) (step 2 of the above algorithm). If the dynamics of the analyzed time series is assumed to be governed by the VARX process (11) with memory m, the model distance functional in the clustering problem formulation will take the form (14). Let 𝗔𝗕ⁱ = (μⁱ, 𝗔ⁱ, 𝗕ⁱ) ∈ R^n×(1+d+k) [where d is the dimensionality of the output of function ϕ₁(·)] and Γ^s−1(t) be the cluster affiliations reconstructed from the current values x̃^s−1 via the subsequent application of Eqs. (30) and (32). After inserting Γ^s−1(t) into the function g_i(θ_i), i = 1, … , K and taking the derivative of the functional L₀ (34) w.r.t. the new variables 𝗔𝗕ⁱ, we get the optimal estimators for 𝗔𝗕_sⁱ in the iteration s as the solution of the following system of linear equations:⁴

View Expanded

where ϕ₁^t = ϕ₁(x_t−τ, … , x_t−mτ) and ϕ₂^t = ϕ₂[u(t)]. If the matrix (𝗫*_s𝗫_s) is invertible, then the solution of the above system exists, is unique, and can be expressed as

View Expanded

where * denotes the matrix conjugate (Golub and Loan 1989). The optimal estimates of the noise parameters 𝗖ⁱ are straightforwardly calculated from the covariance matrices of the model residuals in the cluster state i:

View Expanded

Equations (37) and (38) give explicit estimator expressions that are used in step 2 of the subspace iteration algorithm.

As can be seen from the above estimator formulas, from the view point of the inverse numerical problem there is no difference between linear (12) and nonlinear (11) factor models: both result in the solution of a linear least squares problem (36). This is explained by the fact that in both cases the right-hand sides of the models are linear functions of model parameters. However, as will be demonstrated in the following, linear autoregressive models can in some situations provide more insight, allowing us to use the available tools of linear data analysis, and can be successfully applied to analyze realistic (nonlinear) data.

b. FEM–VARX model order selection for fixedK and C

To select a proper model order m and the optimal functional form ϕ₂(u) = {ϕ₂¹[u(t)], … , ϕ₂^k[u(t)]} for the external factors, standard tools of information theory such as the Akaike information criterion (AIC) or the Bayesian information criterion (BIC) McQuarrie and Tsai (1998) developed for the linear stationary VARX models can be applied a posteriori to the locally stationary VARX models identified via the FEM–VARX procedure. In terms of the information criteria, various models are compared wrt the special functional consistent of the model log likelihood with an added regularization term penalizing the total number of parameters involved in the model (to avoid overfitting). For example, the BIC functional (being in general more robust than AIC; cf. McQuarrie and Tsai 1998) for the cluster state i will have the form

View Expanded

where _i = γ_ig_i^†(θ_i) and N_i is the number of the model parameters in the cluster state i. Given any two estimated cluster models [e.g., models with different memory depth and/or different functions ϕ₂(u)], the model with the lower value of BIC(i) is the one to be preferred. In the following it will be demonstrated how the criterion (39) can be applied in the praxis to make a decision about the optimal functional form of the local VARX models and to test the significance of different external factors u for the dynamical process explaining the analyzed time series data.

c. Choosing the optimal number of local models K

The upper bound for the number of statistically distinguishable cluster states for each value of the persistency threshold C can be algorithmically estimated in the following way: starting with some a priori chosen (big) K, one solves the optimization problem (34) for different fixed value of C and calculates the confidence intervals of the resulting local parameter matrices Θⁱ, i = 1, … , K (this can be done applying standard bootstrap sampling procedures; see Chernik 2007). If two of the estimated parameter sets for two of the identified cluster states have confidence intervals that overlap in all components, this means that respective clusters are statistically indistinguishable and the whole procedure must be repeated for K = K − 1. If at a certain point all of the matrices are statistically distinguishable, the procedure is stopped and K_max(C) = K.

Another possible means of estimating the optimal number of clusters can be used if the identified transition process Γ(t) is shown to be Markovian for given K, C. Markovianity can be verified applying some standard tests (e.g., one can check the generator structure of the hidden process; see Metzner et al. 2007).⁵ In such a case the hidden transition matrix can be calculated and its spectrum can be examined for a presence of the spectral gap (a gap separating the fast and the slow time scales in the Markov dynamics). If the spectral gap is present, then the number of the dominant eigenvalues (i.e., eigenvalues between the spectral gap and 1.0) gives the number of the metastable clusters in the system (Schütte and Huisinga 2003). Positive verification of the Markovianity of the hidden process has an additional advantage: it allows us to construct a reduced dynamical model of the analyzed process and to estimate some dynamical characteristics of the analyzed process (e.g., one can calculate relative statistical weights, mean exit times, and mean first passage times for the identified clusters) (Horenko et al. 2008a).

Verification of the Markov assumption also allows us to construct predictive Markovian models of the persistent dynamical process Γ(t) switching between the cluster states (Horenko et al. 2008a; Horenko 2009). As will be demonstrated for the numerical examples in the following, the respective K × K Markovian transition matrix, together with the parameter estimates (37) and (38) of the locally stationary cluster states, can help to construct the dynamical ensemble prediction models. It is important to mention that without this a posteriori Markov verification, the minimum of the functional (34) cannot be directly used to generate the data-based predictions since the identified cluster affiliation function Γ is just an abstract BV function with some predefined persistency C and has no intrinsic representation in terms of the underlying dynamical process.

d. Selection of the optimal persistency threshold C for a given number of states K

Linearity of the functional (34) wrt x̃ for fixed values of model distance parameters θ_i can help to apply the standard instruments from the theory of ill-posed linear problems, such as the L-curve approach (Calvetti et al. 2000), to identify the optimal value of the persistency threshold C for a fixed number of clusters K. For example, the optimal value of C can be determined as the edge point (or the point of maximal curvature) on a two-dimensional plot. This plot depicts a dependence of the residuum-norm of the solution from C (Horenko 2009). Alternatively, as would be demonstrated in the numerical examples, decreasing C up to some point [meaning the increasing regularity of the optimal solutions of (34) in the BV sense] results in decreasing the respective values L₀ of the clustering functional in the solution point (meaning the increasing quality of the resulting clustering). The point with the minimal value of L₀ in such a case indicates the optimal persistency threshold guaranteeing the best clustering quality.

a. Subgrid-scale modeling for Lorenz ’96 system with forcing

The Lorenz ’96 model of type II (Lorenz 1996; Orrell 2003) is a two-scale simplified ordinary differential equation (ODE) describing advection, damping, and forcing of some (slow) resolved atmospheric variable x̃_i being coupled to some (fast) subscale variables y_i,j:

View Expanded

with all of the model parameters ( = 8, = 4, h = 1, b = c = 10) set to be the same as in the paper of Orrell (2003). In contrast to the paper of Orrell (2003), we choose the external forcing to be an explicitly time-dependent function of the form F(t) = 10 sin(2π/80t), resulting in switching between “deterministic” and “chaotic” regime behavior (Orrell and Smith 2003) on the time scale of the forcing (e.g., producing a system with three different time scales).

We generate one realization of (41) in the time interval [0, T] with an adaptive Runge–Kutta method of the fourth order (MATLAB command ode45), resulting in a 40-dimensional time series with 4000 instances (time step τ = 0.1). We further aim at parameterization of the subgrid-scale influences x_tⁱ = F_t^y(t) in the nonstationary VARX form (11) with external factors defined by the resolved variables {ϕ₂[u(t)] = [x̃₁(t), … , x̃₈(t)]}. Figure 1 demonstrates the application of the FEM–VARX clustering to the time series of x_t and ϕ₂[u(t)] defined in such a way for K = 2 (number of clusters), q = 1 (memory depth), and with N = 200 (number of finite elements) for various numbers of persistency threshold parameter. The minimum is achieved for C = 6, which corresponds to the number of transitions through the bifurcation point of (41). As can be seen from Fig. 2, the FEM–VARX clustering procedure for K = 2 and C = 6 identifies both dynamical regimes of the model (41), whereas the minimization of (34) without the persistency constraint (or, equivalently, C = 2N = 400) results in a much worse clustering result (in terms of the optimal value of L₀) and is due to the trapping in the local minimum of the clustering functional. Inspection of the confidence intervals of the estimated local stationary VARX model parameters in Fig. 3 reveals that whereas the matrices 𝗔 are statistically indistinguishable (e.g., the “own dynamics” of the subgrid-scale process is basically the same in both states), there are statistically significant differences in 𝗕 (in this case it governs the coupling from the resolved degrees of freedom x̃ to subgrid-scale degrees of freedom). Also, the noise intensities 𝗖 are different in both cases (in the identified chaotic regime the noise intensity is significantly higher).

Finally, we verify the Markov assumption for the identified affiliation function Γ(t), generate the ensemble predictions (with 10 000 ensemble members) based on the estimated eight-dimensional FEM–VARX parameterization (based on the first 90% of the data), calculate the expectation values of the relative prediction errors (in 2-norm, based on the last 10% of the data; black solid line in Fig. 4), and compare them with predictions obtained by other data-driven models (see Fig. 4): (i) stationary constant (gray dotted line) and regressive (black dashed–dotted line) subgrid-scale models (Orrell 2003); (ii) stationary linear stochastic model (black dashed line) (Wilks 2005); and (iii) one-dimensional FEM–VARX model, estimated under the assumption that different subgrid-scale processes in different dimensions do not interact (e.g., the matrices 𝗔 are diagonal) (gray solid line). Figure 4 shows that the fully interactive eight-dimensional persistent FEM–VARX model has a much better predictive skill for the considered nonstationary model series.

b. Analysis of ERA-40 geopotential data in Europe (1958–2003)

Using the FEM–VARX method introduced above, we analyze daily mean values of the 500-hPa air temperature field from the ERA-40 reanalysis data (Simmons and Gibson 2000).⁷ We consider a region with the coordinates 32.5°–75.0°N, 27.5°W–47.5°E, which includes Europe and a part of the eastern North Atlantic. The resolution of the data is 5°, which implies a grid with 16 points in the zonal and 9 in the meridional direction. For the analysis we have considered temperature values only for the period 1 December 1958 to 31 July 2003; thus, we end with a 144-dimensional time series of 16 314 days. To remove the seasonal trend we apply a standard procedure, where from each value in the time series we subtract a mean build over all values corresponding to the same day and month (e.g., from the data on 1 January 1959 we subtract the mean value over all days that are the first of January, and so on). The resulting deviations ΔH(t) of the geopotential heights from their seasonal mean values are the subject of the data analysis in the following.

Since the overall dimensionality of 144 of the analyzed data series will induce uncertainties too high for the given time span of 16 314 days, the scope of the data analysis in the following is reduced to the projections on 20 dominant EOFs (describing approximately 99% of the total data variance).

Three different external influence factors u_t are tested on their significance for the change of the EOF projection data x_t in the setting of the VARX models (11): (i) atmospheric CO₂ values u_t¹ from the Mauna Loa observatory (data are available online at http://cdiac.ornl.gov/ftp/trends/co2/maunaloa.co2), (ii) the seasonal trend factor in the form u_t² = sin(2π/365.4t), and (iii) the sunspot cycle data u_t³ (data are available online at http://solarscience.msfc.nasa.gov/SunspotCycle.shtml).

We start the data analysis parameterizing the globally stationary VARX model for 20 dominant EOFs with different sets of external factors described above for various values of the memory parameter q, applying the BIC test to verify the statistical significance of the factors and testing the independent identically distributed (i.i.d.) assumptions for the model residuals with the AR(1) test (Brockwell and Davis 2002) (to check the validity of the underlying estimator assumptions). It is found that the optimal globally stationary VARX model for the analyzed data is the one with q = 2 and no external factors; that is, in the globally stationary framework all of the external factors are found to be statistically nonsignificant.

Application of the nonstationary FEM–VARX framework according to the K selection procedure described above results in determining K = 4 as the maximal number of statistically distinguishable local VARX states (valid for a wide range of parameters C, N, q). For a fixed value of K = 4 we further determine the optimal value of the persistency threshold C (in the same manner demonstrated in Fig. 1), determine the optimal value of memory q = 2 via the BIC test (e.g., the same as for the globally stationary VARX model), and test the i.i.d. assumptions for the model residuals with the AR(1)-test (Brockwell and Davis 2002). Next, we repeat the verification of external factors in the same way as described above for the globally stationary VARX model. As shown in Fig. 5, the BIC test confirms the hypothesis that the optimal FEM–VARX model is the one with the two external factors u_t¹ (atmospheric CO₂ values) and u_t² (seasonal factor), whereas the influence of the sunspot cycle data u_t³ is found to be statistically negligible. Therefore, all of the further tests are conducted only with the factors u_t¹ and u_t².

Inspection of the respective cluster affiliation functions γ_i(t) (see Fig. 6) together with the mean equilibrium positions in the cluster states (40) reveals (see Figs. 7 and 8) that two of the identified cluster states, namely states 3 and 4, are describing the blocking situation in the upper troposphere. Figure 6 shows the comparison of the sum of cluster affiliations γ₃(t) + γ₄(t) for the par of the time series with the scaled zonally averaged Lejenas–Okland blocking index. It indicates the appearance of a blocking anticyclone and the duration of the event. We have a blocking if the geopotential height difference at 500 hP between 40° and 60°N is negative over a region with 20° zonal extent. The exact formula is given in Lupo et al. (1997); for the purpose of representation, we have computed a zonally averaged value of the index, rescaled it, and reversed its sign.

Next we aim at demonstrating how the postprocessing methods described in section 3 can help us gain additional insight into the data to understand the impact of the external factors. Figures 7 and 8 show the EOF back-projection to the original 144-dimensional space of the mean dynamical equilibrium positions in clusters 3 and 4 calculated at different times with the respective values of the factors u_t¹ and u_t² As can be seen from the graphics, the impact of the seasonal factor u_t² results in significant weakening of the blocking states in summer, a finding consistent with the observations reported in the literature. The impact of the increasing CO₂ concentration also results in a weakening of the blocking situation in both states 3 and 4 that is less pronounced but still statistically significant (see the respective confidence intervals in Figs. 7 and 8).

Finally, in the same manner as in the previous nonstationary Lorenz ’96 example, we construct an ensemble prediction model (with 10 000 ensemble members) based on the FEM–VARX clustering results and compare the resulting predictions with the ones obtained by other methods. First, we verify the Markov assumption in the time-homogenous approximation for the identified affiliation function Γ(t), generate the ensemble predictions based on the estimated 20-dimensional FEM–VARX parameterization (based on the first 90% of the data), and calculate the expectation values of the relative prediction errors (in 2-norm, based on the last 10% of the data). As can be seen from Fig. 9, FEM–VARX ensemble predictions produce much better predictions than the constant model (where the prediction is always the expectation value over the whole history) and the “same as today model” (where the prediction for the next day is just the state of the system now). Compared with the globally stationary VARX model (where the ensemble prediction is calculated from 10 000 ensemble members of the global VARX model), FEM–VARX produces only slightly better predictions (approximately 2% better). This observation could be explained by the low overall persistence of the process Γ(t) compared, for example, to the previous Lorenz ’96 example where the hidden process was very persistent. This issue needs a deeper understanding and is a matter of future research.