1. Introduction

Data assimilation (DA) refers to the estimation of oceanic–atmospheric fields by melding sensor data with a model of the dynamics under study. Most DA schemes are rooted in statistical estimation theory: the state of a system is estimated by combining all knowledge of the system, like measurements and theoretical laws or empirical principles, in accord with their respective statistical uncertainty. The present challenge is that the state of the atmosphere and ocean system is complex, evolving on multiple time and space scales (Charnock 1981;Charney and Flierl 1981). Direct observations can be difficult and costly to acquire on a sustained basis, especially in oceanography. The large breadth of scales and variables also leads to costly and challenging numerical simulations. Future advances in coupled ocean–atmosphere estimation will thus require efficient assimilation schemes. In Part I of this two-part paper, the main goal is to develop the basis of a comprehensive, portable, and versatile four-dimensional DA scheme for the estimation and simulation of realistic geophysical fields. The adjective realistic emphasizes that the scheme should capture the time and space scales of the real processes of interest. It implies the use of real ocean data, as well as appropriate theoretical models and numerical resources. The primary focus is on the physics;acoustical, biological, and chemical phenomena will be investigated later. The implementation presented is compatible with the Harvard Ocean Prediction System (HOPS; e.g., Lozano et al. 1996; Robinson 1996b) and the future of this work involves ocean–atmospheric data-driven estimations. In Part II (Lermusiaux 1999a) of this paper, identical twin experiments based on Middle Atlantic Bight shelfbreak front simulations are employed to assess and exemplify the capabilities of the present DA scheme.

A description of the goals and uses of DA, with a review of most methods, is given in Robinson et al. (1998a, and references therein). The issue is that, with most existing approaches, the combination of our practical, accuracy, and realism goals is difficult to satisfy (sections 3, 4). In fact, several directions have been taken so as to determine feasible schemes for realistic studies. Examples of such attempts include simpler physics models to integrate errors (e.g., Dee et al. 1985;Dee 1990; Daley 1992b), variance-only error models (e.g., Daley 1991, 1992b), steady-state error models (e.g., Fukumori et al. 1993), and reduced dimension or coarse-grid Kalman filters (KF; e.g., Fukumori and Malanotte-Rizzoli 1995). Other reductions deal with the explicit computation of non-null elements of linearized transfer matrices (e.g., Parrish and Cohn 1985; Jiang and Ghil 1993), banded approximations (e.g., Parrish and Cohn 1985), extended filters (e.g., Evensen 1993), ensemble and Monte Carlo methods (e.g., Evensen 1994a,b; Miller et al. 1994), and possibly using the optimal and breed perturbations for assimilation (Ehrendorfer and Errico 1995; Toth and Kalnay 1993). It is important to realize that several of these attempts are based on incompatible hypotheses. Briefly, the coarse-grid KFs imply global-scale forecast errors while the variance-only and banded approximations assume local errors. Pure Monte Carlo methods acknowledge the importance of nonlinear terms, extended schemes neglect their effects locally in time, and linearized methods neglect them at all times. The forward integration of error fields using simpler physics models assumes that the dominant predictability error is never correlated to the complex physics. Steady-state error models are somewhat limited to fixed data arrays and statistically steady dynamics. For each attempt, the list of arguments for and against is long. Even if most a priori reduced methods have been successful data interpolators, they have logically led to controversies.

If one accepts that, in general, relatively little is known about dynamical and observational error fields, it is rational to limit the a priori assumptions. For the present comprehensive aims, the conditional mean, a minimum of several cost functionals or estimation criteria, is chosen for the optimal estimate. An approximation to the estimation criterion is obtained using heuristic characteristics of geophysical measurements and models. The resulting suboptimal approach is based on an objective, evolving truncation of the number and dimension of the parameters that characterize the conditional probability or error space. The ideal error (probability) subspace spans and tracks the scales and processes where the dominant errors (low probabilities) occur. The notion of dominant is naturally defined by the error measure used in the chosen optimal criterion: to each estimation criterion corresponds an error subspace definition. For the present minimum error variance approach, the logical definition yields an evolving subspace, of variable size, characterized by error singular vectors and values, or, similarly, the error empirical orthogonal functions (EOFs) and coefficients. Data assimilation via error subspace statistical estimation (ESSE) combines data and dynamics in accord with their respective dominant uncertainties. Once successfully applied, ESSE can rationally validate specific a priori error truncations for future tailored applications. Organizing the error space as a function of relative importance in fact defines a theoretical basis for quantitative intercomparison of today’s numerous reduced dimension methods. A first issue of course is the meaning and validity of the truncation of geophysical error spaces (section 5 and Part II). Another is that it is easy to define the concept of an evolving subspace, but it is harder to determine mathematical systems that describe and track its evolution. Most of the schemes for filtering and smoothing via ESSE (derived next) have variations. Focusing on the dominant errors fosters dynamical model testing and corrections. The error subspace also helps to identify areas and variables for which observations are most needed. ESSE provides a feasible quantitative approach to both dynamical model improvements and adequate sampling. Historically, these have been challenging issues. The accurate specification and tracking of the dominant errors hence appears of paramount importance from a fundamental point of view.

The text is organized as follows. Selected definitions and generalities are stated in section 2. Section 3 deals with the focus and specific objectives of the paper. Section 4 develops the main issues in realistic and comprehensive data assimilation today: the efficient reduction of error models and the powerful use of all information present in the few observations available. Section 5 addresses the meaning of the variations of variability and error subspaces. The correlations in geophysical systems are emphasized and the ESSE criteria introduced. Sections 6 and 7 derive schemes for filtering and smoothing via ESSE, respectively. These filtering and smoothing algorithms with nonlinear systems are succinctly compared with existing “suboptimal” procedures. Section 8 consists of the summary and conclusions. Appendix A describes most of the notation and assumptions. Appendix B addresses important specifics and variations of the ESSE schemes presented.

2. Definitions and generalities

Since the dynamical (1a) and measurement models (1b) are approximations, statistical estimation theory formulates stochastic hypotheses for their respective imperfections or errors. This defines the statistics of the true models (2a)–(2b),

View Expanded

The definitions and hypotheses employed here allow parameter estimations and time-correlated model errors (appendix A). All nonzero mean phenomena are assumed included into (1a)–(1b). Once the truth is stochastically described, the estimation or melding criterion determines the respective influence of the dynamics and observations on the state estimate. A DA system consists of three components: a dynamical model, a measurement model with sensor data, and an estimation criterion. The data assimilation problem is to determine the best possible field estimate that is in accord with the dynamical and measurement models, within their respective uncertainties. By “best” it is meant “in closest statistical accord with the truth.” The notion of close accord is defined mathematically by the estimation criterion. Within the criterion, all constraints are weak a priori, but to ease computations some may be assumed strong, depending on the relative estimated accuracy of each constraint. For instance, the nonlinear dynamical model can be considered either as a strong or weak constraint (e.g., Reid 1968; Sasaki 1970; Daley 1991; Bennett 1992; Wunsch 1996). Finally, it must be remembered that the central notion of an optimal estimate is a function of the melding criterion and associated statistical hypotheses. The ultimate arbiter consists of using the optimal estimate for ocean prediction.

3. Focus and specific objectives

A comprehensive assimilation system is suited to most nonlinear oceanic–atmospheric phenomena and most data types and coverage. Even though weather and ocean modeling differ (dynamics, data and models available, constraints), the approach chosen is general. The main restriction is a focus on the synoptic/mesoscale circulation and processes. The assimilation period is left arbitrary; it could be days to years. It is only assumed that the observations are time–space adequate with regard to the predictability limits of the phenomena of interest. This notion of adequate is subtle since even for known predictability limits and validated models (1a)–(1b), data sufficiency is still a function of the assimilation scheme. The better the scheme, the fewer the necessary data (e.g., Lorenc et al. 1991; Todling and Ghil 1994). The application considered in Part II mainly involves simulated temperature and salinity data for the control of shelfbreak front phenomena.

As far as specific objectives, the scheme should address model nonlinearities, estimate the uncertainty of the forecast with sufficient information on a posteriori errors, be suitable for assimilation in real time as well as for parallel computations, and allow adaptive filtering and parameter estimation. Most of the existing optimal schemes (section 4) cannot yet satisfy our practical objectives. The information, accuracy, and realistic goals may reduce the success of today’s operational methods like the optimal interpolation (OI), commonly used in weather forecasting (Bengtsson et al. 1981; Ghil 1989;Lorenc et al. 1991) and in real-time, at-sea ocean prediction (Robinson et al. 1996a,b). One would like to improve the existing nowcast/forecast capabilities and hopefully increase general understanding; data assimilation should feed back to fundamental science.

4. Constraints and issues

The assimilation problem of section 2 is a nonlinear statistical estimation problem. Existing nonlinear schemes have first been analyzed with regard to our goals. In this text, it is simply our intention to focus on the essentials of nonlinear schemes and issues, and provide references for more complete discussions. Within estimation theory, Bayesian estimation and maximum likelihood estimation are the common approaches (e.g., Jazwinski 1970; Gelb 1974; Lorenc 1986; Boguslavskij 1988). In control theory, the DA problem is seen as a deterministic optimization issue, and for quadratic cost functions it amounts to weighted least squares estimation (Le Dimet and Talagrand 1986; Tarantola 1987;Sundqvist 1993). Statistical assumptions can be implicitly associated with all approaches (Robinson et al. 1998a). The discussion to follow involves Bayesian ideas.

The Bayesian conditional mean is the optimal estimator with respect to several cost functionals for arbitrary statistics. In this study, it is chosen as the optimal estimate. For nonlinear systems, it depends on all moments of the conditional density function, hence, on an infinite number of parameters. Solving for the conditional probability density governed by the Fokker–Planck or Kushner’s equation (Jazwinski 1970) is today an informative guide with simple systems (Miller et al. 1994; Miller et al. 1998). In real ocean–atmosphere nonlinear estimation, the aim is to approximate the quantities of primary interest, the state, and its uncertainty. Taylor expansions and local linear extensions yield the common approximate schemes: the linearized, extended, higher-order and iterated Kalman filters/smoothers (KF/KS) and associated simplifications (e.g., Boguslavskij 1988). They provide an estimate of the uncertainty, but their truncated expansion may diverge (Evensen 1993, 1994a,b) and require frequent reinitialization. Most of the control and weighted least squares methods were derived for linear systems but can be iterated locally to solve nonlinear generalized inverse problems (e.g., Bennett et al. 1996; Bennett et al. 1997). The representer method considers model errors and minimizes the cost function in the data space but can be as costly as the estimation schemes if a posteriori state error covariance estimates are required. Direct global minimizations (e.g., Robinson et al. 1998a) are alternatives but a physically realizable solution is not assured. To limit expensive model integrations, a good first guess is required. In fact, the convergence of the iterated weighted least squares schemes is not yet proven (Evensen and van Leeuwen 1996). Smoothing prior to the minimization appears necessary (section 7).

The main advantages of all the above methods are the update and dynamical forecast of the error covariance. Even with linear models, forecast errors can differ considerably from the ones currently prescribed in OI (Parrish and Cohn 1985; Cohn and Parrish 1991; Miller and Cane 1989; Todling and Ghil 1990; Daley 1992a–c). Nonetheless, this error forecast and update is very expensive. For nonlinear systems, the required dimension is infinite. For discrete linear systems with n degrees of freedom, one needs O(n²) numbers to represent the error covariance, with n of O(10⁵–10⁶) or more. Even if the real-time aim is relaxed, today’s parallel computers cannot manage such sizes. Since most schemes are only optimal for linear models, several sometimes conflicting hypotheses have been made to derive practical/operational reduced schemes (e.g., Parrish and Cohn 1985; Lorenc et al. 1991; Toth and Kalnay 1993; Todling and Cohn 1994), hence leading to controversies.

There are two constraints that a DA system needs to address. First, the dimension of the full error model associated with (2a)–(2b) is too large. Since less is known about errors than about dynamics, a careful reduction is necessary. The a priori hypotheses should be limited. With experience, for specific regions and processes, and particular data properties, some of today’s hypotheses (e.g., Todling and Cohn 1994; appendix B) may be validated for use as a priori information. Even though the data coverage, type, and quality have increased in the past decades, the second constraint is the limited data sets. This concern is of special relevance in oceanography (Robinson et al. 1998a). To optimize the assimilation, it is very important to utilize at once all information contained in the few observations available. In summary, the issues are (a) how to reduce the size of the error model while explaining as accurately as possible the error structure and amplitude (many degrees of freedom, limited resources) and (b) how to optimize the extraction of reliable information from observations limited both in type and coverage.

5. Error subspace statistical estimation

Estimation criteria that address the objectives and questions raised in sections 3–4 are now identified. The approach is dynamic, reflecting basic properties of oceanic–atmospheric systems. Based on essential characteristics of geophysical measurements and models, the first property argues that efficient bases to describe, respectively, the dynamics, the variations of variability, and the errors, exist and are meaningful (section 5a). Section 5a is detailed since confusions between these evolving bases lead to controversies. The second property relates to the correlations between geophysical fields (section 5b). These facts are then used to determine the optimal representation of the error and a criterion for data–dynamics melding, leading to the ESSE concept (section 5c).

c. ESSE

The conditional mean was chosen for optimal estimate. For all statistics, minimizing the expectation of a convex measure of the estimate’s uncertainty leads to that optimum (appendix B, section d). Approximate definitions of uncertainty and measures thereof lead to approximate estimates. Here, both notions are determined using sections 5a and 5b.

The estimate’s uncertainty can be represented by an error covariance, a common convex statistical measure of error fields.⁴ The first conclusion (section 5a) supports its truncation to a most energetic error subspace while the second (section 5b) implies that the melding criterion should be 3D multivariate. A reduced-rank approximation of the error covariance P at time t_k is thus optimal for a given rank p_k if it explains the maximum variance and structure of the multivariate P_k that is possible. For the structured, power-decay property (section 5a) there is a relatively small number p_k for which the optimal reduction explains most of P_k. Denoting for convenience this time-variant p_k by p, the associated reduction is called the principal error covariance P^p_k. The difference between P_k and P^p_k, or complementary error covariance P^c_k,

P^c_kP_kP^p_k(4)

should have a minimal norm. An orthonormal decomposition of P^p_k, E_kΠ_kE^T_k, with variance Π_k ∈ R^p×p and structure E_k ∈ R^n×p, hence satisfies

View Expanded

In the sense of any unitarily invariant norm (e.g., the two-norm ‖P^c_k‖₂ and Frobenius norm ‖P^c_k‖_F), the optimum in (5) is the dominant rank-p singular value decomposition of P_k (Horn and Johnson 1985, 1991). The matrix Π_k is the ordered diagonal of dominant-p singular values and the columns of E_k are the associated singular vectors. Since P_k is positive semidefinite, E_kΠ_kE^T_k is also its rank-p eigendecomposition. The columns of E_k form a basis for the 3D multivariate error subspace; Π_k is the error subspace covariance. At a given time, t_k, and for a given p, these matrices characterize the error subspace (ES). They answer the first issue raised in section 4.

With the ES theoretically defined by (5) at all t_k, only the error measure in this ES still needs to be chosen. Several exist (e.g., Horn and Johnson 1985), but the most logical is the Euclidean one. Combining the criterion (5) with the Euclidean minimum error variance approach leads to the present notion of ESSE. Data and dynamics are melded such that the a posteriori ES variance is minimized. The estimation criteria are

View Expanded

for objective analysis (OA) via ESSE at t_k;

View Expanded

for filtering via ESSE at t_k; and

View Expanded

for smoothing via ESSE at t_k within the data interval [t₀, t_N]; where each of (6a)–(6c) are subject to the dynamical and measurement model constraints (2a)–(2b). To distinguish between a priori and a posteriori quantities, the symbols (−) and (+) are employed (see appendix A for more on notation). Except for the ignored small errors, criteria (6a)–(6c) follow the Bayesian minimum error variance nonlinear estimation. They lead to efficient, 3D multivariate analyses since the meldings occur within the ES. The second question raised in section 4 is answered. The criteria (6a)–(6c) also relate to the efficient concept of “minimax assimilation”: the maximum errors, here in the Euclidean sense, are minimized. The general goal of ESSE is to determine the considered ocean–atmosphere state evolution by minimizing the dominant errors, in accord with the full dynamical and measurement models, and their respective uncertainties. Of course, the criteria (6a)–(6c) are only theoretical. In practice, efficient schemes for finding (6a) and tracking (6b)–(6c) the ES have to be determined (sections 6, 7).

Objective analysis via ESSE (6a) or “fixed-time ESSE” emphasizes the general applicability of the approach. In fact, Anderson (1996) has shown that for the Lorenz model (Lorenz 1963) the projection of classical OA error correlation onto the local attractor sheet is the most effective selection of initial conditions for ensemble forecasts. This conclusion has also been exemplified in primitive-equation (PE) modeling (Lermusiaux 1997). Since the ES is a subset of the local variations of variability, (6a) is the most efficient analysis for given resources and in the Euclidean framework. In Part II of this study, the focus is on filtering via ESSE (6b). The problem statement is given in Table 1.

Section 6 outlines filtering via ESSE schemes using a Monte Carlo approach; dynamical systems for adaptive ESSE are also derived in Lermusiaux (1997). The smoothing via ESSE problem statement is as in Table 1, but with criterion (6c) replacing (6b). For linear models (2a), this corresponds to the generalized inverse problem restricted to the dominant errors. Such smoothing schemes with a posteriori error estimates are obtained in section 7. The five main components of the present ESSE system (initialization, field and ES nonlinear forecasts, minimum variance within the ES, and smoothing) are illustrated in Fig. 1.

Some general properties are now discussed. First, in (6a)–(6c) the data only correct the most erroneous parts of the forecast. The accurate portion of the forecast is corrected by dynamical adjustments and interpolations. Second, the ES in (5) is time dependent. The reduction to the principal error covariance is dynamic. The ES tracks the scales and processes where the dominant errors occur. The time rate of change of the ES is a function of the (i) initial uncertainty conditions; (ii) evolving model errors; and (iii) data type, quality, and coverage;and of the nonlinear, interactive evolution of these three components. All these factors influence the nature of the ES (e.g., multiscale, anisotropic, homogeneous, or not). In fact, the successes of the optimal perturbations (OP) to determine initial conditions for ensemble forecasting (e.g., Mureau et al. 1993; Toth and Kalnay 1993;Molteni and Palmer 1993; Molteni et al. 1996) can be improved by quantitatively taking into account data quality and model errors. The OP spectrum and structures, which only consider the predictability error, should be modified accordingly, especially in oceanography. ESSE provides a theoretical framework to do so. Third, the error and dynamics subspaces in general differ. Fourth, the statistical estimation of the ES (sections 6, 7) yields the notion of error EOFs. In fact, there are several quantitative definitions for the ES, each associated with slightly modified criteria (6a)–(6c): for example, singular or normal error modes; extended, complex, or frequency error EOFs; error POPs and PIPs;and synoptic and wavelet-based ES. If the maximum norm had been chosen in (5), a maximum measure would have been logical in (6a)–(6c). These particular formulations are not discussed further. Priority is given to the central concept, common to all representations. Fifth, the ideal ES dimension p (e.g., appendix B, section b) evolves in time, in accord with the dynamics, model errors, and available data. It is only for statistically stationary data, dynamics, and model errors that p should stay constant. In all schemes of sections 6 and 7, the size of the ES is thus time dependent.

6. Filtering via ESSE schemes

A recursive scheme is now derived for filtering in the ES (5) corresponding to the models (2a)–(2b). One needs to track the ES evolution, which is not trivial. The two-step root of the algorithm consists of a forecast–data melding when data are available (section 6a), followed by the dynamical state and ES forecasts to the next assimilation time (section 6b). It is assumed that an estimate of the conditional mean state and associated ES have been integrated from t₀ to t_k using (1)–(2) and (6b) for [d₀, . . . , d_k−1]. Hence, ψ̂_k(−), E_k(−), and Π_k(−) are available. Specifics and variations of the algorithm are discussed in appendix B.

a. ES melding or ES analysis scheme

As in most existing schemes, the data–forecast melding is chosen linear a priori. This causes a departure from the strict Bayesian approach (e.g., Jazwinski 1970), which would solve (6a)–(6c) without making this simplification. Nonetheless, the melding weights are determined using criterion (6b) in the nonlinearly evolved ES (5). To simplify notations, the index k is omitted. The minimum error covariance melding is first decomposed exactly in terms of error eigenvalues and eigenvectors [section 6a(1)]. Its ES truncation is given in Lermusiaux (1997). In section 6a(2), the sample or empirical ES melding is derived.

1) Eigendecomposition of the minimum error variance linear update

Since linear melding is enforced, determining the optimum of

View Expanded

is equivalent to minimizing the trace of P(+) (e.g., Gelb 1974). Taking the derivative of (7) with respect to ψ^t yields the melded estimate

ψ̂(+) = ψ̂(−) + K[d − Cψ̂(−)](8)

and updated error covariance

PPKCP(9)

where K is the Kalman gain

KPC^TCPC^TR⁻¹(10)

Note that (8) contains the data update of all boundary variables and external forcings since they are assumed to be part of (1a)–(2a). Introducing the eigendecompositions of the a priori and a posteriori error covariances (see appendix A for notation),

View Expanded

with U₋U^T₋ = U^T₋U₋ = U₊U^T₊ = U^T₊U₊ = I, into (8)–(10), the Kalman gain and error covariance update are exactly rewritten, respectively, as

View Expanded

where C̃ ≐ CU₋. The eigendecomposition of the nonnegative definite Λ̃(+) yields

View Expanded

where Λ(+) is diagonal and the columns of H ∈ R^n×n are a set of orthonormal eigenvectors for Λ̃(+). Hence,

P(+) = U₊Λ(+)U^T₊U₋HΛ(+)H^TU^T₋(15)

The a posteriori diagonal matrix of error eigenvalues Λ(+) is given by (14) and the a posteriori error eigenvectors by

U₊U₋H(16)

The a posteriori error covariance derives from (15). For linear melding, the state (8), (12) and error eigenupdates (14), (16) are the minimum error variance estimates (Table 2).

2) Minimum sample ES variance linear update

In this section, a sample ES forecast described by E₋ and Π(−) is assumed available (section 5b). The data-forecast melding within the sample ES is outlined, with updates of the fields and ES covariance. For the details, we refer to (Lermusiaux 1997).

(i) Dynamical state update

The field update can be derived either by truncation of (8) and (12) to the sample ES or by minimization analogous to (7), but within the ES. One gets

View Expanded

where C̃^p ≐ CE₋. For adequate a priori sampling, that is, P^p(−) = E₋Π(−)E^T₋ converges to P(−), (17)–(18) converge to (8) and (12) at the infinite sample limit.

(ii) Sample ES update

The derivation of the ES covariance update requires care since the present ES forecast (section 5b) is obtained from an ensemble forecast. As discussed in (Evensen 1997b; Lermusiaux 1997; Burgers et al. 1998), the original ensemble update algorithm (e.g., Evensen1994a) underestimates a posteriori errors. The two ES algorithms outlined next give a correct error estimate at the infinite ensemble limit. The scheme A directly estimates Π(−) and E₊. The scheme B updates the SVD of the ensemble spread.

Scheme A: Update of the sample ES covariance. An ensemble of q unbiased dynamical states is denoted by ψ̂^j(−), j = 1, . . . , q. The associated a priori and a posteriori error sample matrices, M(−) and M(+) ∈R^n×q, are⁵

View Expanded

These expressions are matrices whose column j consists of the ensemble member j minus the mean estimates ψ̂(−) and ψ̂(+), respectively. The update of the sample error covariance, P^s ≐ MM^T/q, is now obtained. Denoting by d^j a set of q data vectors perturbed with noise of zero mean and covariance R, the updates of the ensemble (e.g., Burgers et al. 1998) and conditional mean estimate are, respectively,

View Expanded

where the gain K^s has to be optimized. Subtracting (20b) from (20a) and using (19a)–(19b),

M(+) = (I − K^sC)M(−) + K^sV.(21)

The columns of V = [v^j] = [d^j − d] ∈ R^m×q are realizations of the random processes v. The update of the sample error covariance derives from (21)

View Expanded

where R^s ≐ (1/q)VV^T = E^q{v^jv^jT} and Ω^s ≐ (1/q)M(−) V^T = E^q{[ψ̂^j(−) − ψ̂(−)]v^jT}. For the gain K^s minimizing the trace of P^s(+), K^s = P^s(−)C^T(CP^s(−)C^T + R^s)⁻¹, one obtains

View Expanded

By hypotheses (appendix A), Ω^s → 0 for q → ∞. Neglecting the associated symmetric sum in (23) yields an estimate of P(+) of standard deviation error decay of O(1/q),

P^s(+) = (I − K^sC)P^s(−).(24)

The sample ES update derives from the dominant rank-p reduction of (19)–(24). It is efficiently estimated based on the SVD of M(−) and unknown M(+), respectively,

View Expanded

where the operator SVD_p( · ) selects the dominant rank-p SVD. After melding, the p dominant left singular vectors, columns of E₊, form the ordered basis for the ES of dimension p ⩽ q. The corresponding singular values yield the diagonals Π_k(−) and Π_k(+),

View Expanded

Performing computations similar to (21)–(24), but starting from (25)–(26), leads to the optimal gain (18) and to the equations for the ES update. Using the orthogonality of singular vectors, these are

View Expanded

The columns of H in (27) are ordered orthonormal eigenvectors of Π̃(+), projection of P^p(+) onto the columns of E₋. The corresponding eigenvalues form the diagonal Π(+) and I^p ∈ R^p×p is the identity matrix. With (18) and (24)–(25), the columns of E₋ and E₊ in (28) span the same space. Table 3 summarizes the sample ES scheme. Algorithmic and computing issues are discussed in appendix B, section e.

In this scheme A, the ensemble update (20a) is not carried out. It was only utilized to derive the ES covariance update. Only one melding (17) is necessary. The number p of significant error EOFs or singular vectors is smaller than the ensemble size q, which reduces computations in Table 3 (appendix B, section c). Of course, for efficient ensemble forecasts, q should not be much larger than p (i.e., at most, an order of magnitude larger).

Scheme B: Update of the SVD of the ensemble spread. A disadvantage of (27)–(28) is that the information contained in the right singular vectors of (25a) is lost. Once E₊, Π(+), and ψ̂(+) are computed, an adequate ensemble still has to be constructed [section 6b(2)]. Hence, the complete rank-p SVD update is now sought. Using (25a)–(25b) to reduce (19a)–(19b), rewritten for clarity in a vector form [with (A)^j denoting the column j of A], one has

View Expanded

Deriving such an update (29b) based on the original ensemble algorithm (e.g., Evensen 1994a), Lermusiaux (1997) showed that a posteriori errors were underestimated; that is, that the terms K^sV in (21) and thus K^sR^sK^sT in (22) were missing in the original ensemble algorithm. The approach of Lermusiaux (1997) is now utilized but based on the modified ensemble update (20a)–(20b). This reduces the analysis of Burgers et al. (1998) to its significant subspace. The direct derivation evaluates the SVD of (21),

E₊Σ(+)V^T₊ = SVD_p[(I − K^sC)M(−) + K^sV]. (30)

To reduce computations, the ensembles in the rhs of (30) can be truncated a priori to their significant rank-p SVDs. These are denoted by (25a) for the ensemble of states and by SVD_p(V) = V^p ∈ R^p×p for the perturbed data. With these rank-p approximations, using (25b) and the optimal gain (18), (21) reduces to, at O(1/p) in standard deviation,

View Expanded

Inserting V₋V^T₋ = I^p at the right of the second term in the rhs of (31) gives, with (18),

View Expanded

In (32), computing the SVD of the term in bracket, already of rank p, leads to

View Expanded

With (17)–(18), (33a)–(33c) update the complete rank-p SVD (29b), as summarized in Table 4. The advantage of (30) or (33a)–(33c) over (27)–(28) is the right singular vector update. The a posteriori states [(29b)] are physically balanced in the sense of the estimation criterion (section 2). If one uses Table 3, techniques to create an ensemble of a posteriori states from (27)–(28) are needed [section 6b(2)]. On the other hand, scheme A is an efficient procedure to compute the a posteriori ES covariance (27)–(28). It is in fact straightforward to show that the a posteriori rank-p sample error covariance obtained from (29b) and (33a)–(33c) is, at O(1/p) in standard deviation, identical to that obtained in Table 3. In practice, the simulation requirements dictate the adequate scheme from Table 3 or 4. In both cases, one expects the size p of the ES to be time variant and algorithms for evolving p are discussed next in section 6b.

b. Dynamical state and error subspace forecast

The quantities ψ̂_k(+), E_k(+), and Π_k(+) obtained in (section 6a) are now known. The goal is to issue their forecast to the next data time t_k+1. For large nonlinear models, we expect that for adequate sampling of the initial error conditions, an ensemble forecast is efficient to estimate the evolution of the state and its uncertainty. Monte Carlo forecasting is, thus, the approach followed. Several alternatives are discussed in appendix B, section c.

1) Dynamical state forecast

The conditional mean of (2a), ψ̂^t (appendix A), evolves according to

dψ̂^tf̂ψ^ttdt.(34)

The nonlinear central forecast to t_k+1, ψ̂^cf_k+1(−), is obtained from

dψ̂ = f(ψ̂, t) dt,ψ̂_k = ψ̂_k(+)(35)

Statistically, it is the first-order estimate of the conditional mean ψ̂^t_k+1 (Jazwinski 1970). It is also the classic deterministic forecast (1a). With the ensemble ES forecast approach [section 6b(2)], each member evolves during Δt_k+1 as in (2a),

dψ̂^j = f(ψ̂^j, t) dt + dw,ψ̂^j_kψ̂^j_k(36a)

The corresponding ensemble mean at t_k+1 estimates ψ̂^t_k+1 with a standard deviation of O(1/q):

ψ̂^em_k+1(−)E^qψ̂^j_k+1(−)(36b)

Other estimates are the forecast of minimum data misfits (section 7c) or the most probable forecast (maximum likelihood). They are further discussed in appendix B, section d. In section 6b(2), the algorithm simply denotes the chosen conditional mean estimate as ψ̂_k+1(−).

2) Error subspace forecast

Using (2)–(3), the error covariance of ψ̂^t, P, evolves during Δt_k+1 according to

View Expanded

Here, the forecast of the principal covariance of (37), P^p_k+1(−), is estimated by an ensemble of stochastic evolutions to t_k+1 of states initially sampling the a posteriori ES structure E_k(+) and amplitude Π_k(+). The three local steps involved are described next.

(i) Create an ensemble whose covariance from ψ̂_k(+) tends to P^p_k(+)

The a posteriori ensemble is defined by

View Expanded

where the coefficients π^j_k(+) ∈ R^p have to be determined. The simplest choice is

π^j_k(+)Π^1/2_k(+)u^j(39a)

The vectors u^j ∈ R^p are q realizations of a random vector u of zero mean and covariance I^p. For q = q_k(+) → ∞ in (38)–(39a), the sample covariance with respect to ψ̂_k(+) tends by construction toward P^p_k(+). Constraints can be added to (39a) so that all realizations ψ̂^j_k(+) are physically acceptable,

View Expanded

One may wonder why (39b) should be used since the states ψ̂^j_k(+) are in accord with the data and dynamics, and their dominant error covariances. One argument is that only dominant covariances are considered in (39a), but not higher moments. The signs of the u^j are free in (39a). Because of the orthogonality condition, some combinations of singular vectors can also lead to unrealistic variability, even if the true error subspace is spanned. Finally, some of the randomly generated u^j (e.g., Gaussian) can have values quite far from their statistical mean and variance. Hence, constraining the combinations (39a) is useful to reject the few states of possibly too unrealistic or unlikely physics. A simple constraint is to force the ψ̂^j_k(+) vectors to have data residuals in strong accord with the measurement model errors. Rejecting members that have residuals of horizontal-averaged variance smaller than a factor of the local observation error variance has been implemented. Such residual constraints are limited to data regions. Other restrictions can be applied globally, for example, a water column in static equilibrium, weak geostrophic balance, or other considerations (e.g., Mureau et al. 1993). A third option for the π^j_k(+) in (38) is to directly use scheme B. From (29b), the combination coefficients are then

π^j_k(+)Σ(+)(V^T₊)^j(39c)

where the rhs is obtained from (32) or (33a), (33c). The three linear combinations (39a)–(39c) have been utilized in Lermusiaux (1997) and in several regions of the world’s oceans (section 8) for weeks of simulations. For the lack of sufficient time–space data coverage, real data cases for which an approach has always been found to be better than the others have not yet been determined. However, using (39b) instead of (39a) is advantageous since it involves little extra cost compared to that of running a very unlikely simulation in (36a).

An important advantage of (38)–(39) is that the size of the a posteriori ensemble q_k(+) is easily made larger than that of the a priori one, q_k(−). Since q_k(+) = q_k+1(−), this is carried out as a function of the convergence of the ES forecast to t_k+1 (appendix B, section b). Note that increasing the size of the ensemble using (38)–(39) does not extend the base spanning the ES at t_k [in our two-step recursive assumption, E_k(+) is assumed adequate]. However, for nonlinear models, each new integration of (36a) during Δt_k+1 increases the size of the ES forecast to t_k+1, the significance of which grows with the duration of integration. The nonlinearities lead to an evolving ES (section 5c). This fact is illustrated in Part II. Stochastic model errors in (36a) also excite growing modes of variability and thus favor the ES evolution. With linear models, the size of the ES is only modified by this stochastic forcing. If model errors are null, to evolve the ES one must then add new columns to E_k.

(ii) Integrate each ensemble member from t_k+1 using the sample path (2a)

Renumbering (36a), an ensemble of stochastic forecasts are evolved during Δt_k+1,

View Expanded

where dw(t) is a vector Brownian motion process, representing a priori model errors. Its covariance over dt is E{dw(t)dw^T(t)} ≐ Q(t)dt ≐ B(t)B^T(t)dt, where B(t) ∈ R^n×r. The ES concept argues that restricting B(t) to a few r ≪ n columns corresponding to the rank-r eigendecomposition of Q(t) is efficient. Several authors have implicitly used this fact (e.g., Dee at al. 1985; Phillips 1986; Cohn and Parrish 1991; Daley 1991; Cohn 1993; Jiang and Ghil 1993). Stochastic modeling of the dominant model errors B(t) is becoming an area of active research. Their specification is discussed in Lermusiaux (1997).

(iii) Compute the forecast of the ES structure and amplitude at t_k+1

Once ψ̂_k+1(−) is chosen [section 6b(1)], the matrix of sample error forecasts, M_k+1(−) = [ψ̂^j_k+1(−) − ψ̂_k+1(−)] ∈ R^n×q, is evaluated. The sample estimates Π_k+1(−) and E_k+1(−) of rank p ⩽ q are then most efficiently obtained from the rank-p SVD of M_k+1(−),

View Expanded

with Π_k+1(−) ≐ (1/q)Σ²_k+1(−). Table 5 summarizes the steps (i–iii). Among the conditional mean estimates (appendix B, section d), only the central and ensemble mean forecasts are stated.

The basis of the present ESSE filtering scheme consists of Tables 3 and 5. A flow schematic of the algorithm is shown in Fig. 2. As in Fig. 1, each operation consists of several subcomputations and options, with corresponding equations (e.g., appendix B). For instance, the ES initialization and the adoptive error subspace learning are challenging (Lermusiaux 1997; Lermusiaux et al. 1998, manuscript submitted to Quart. J. Roy. Meteor. Soc.).

7. Smoothing via ESSE schemes

The improvement of the filtering solution ψ̂_k(+) (section 6) based on future data leads to a smoothing solution ψ̂_k/N. The present smoothing criterion was defined by (6c). The data interval [t₀, t_N] is fixed: fixed-interval smoothing is considered. The complete problem statement is as in Table 1, but with (6b) replaced by (6c). The issues in nonlinear smoothing are that a forward integration of (2a) is rarely invertible and that running nonlinear geophysical models backward in time can be difficult. In fact, most nonlinear smoothers are based on some of sort of localized (and iterated) approximation (Jazwinski 1970). The present approach falls into that category. However, the philosophy somewhat differs from that of some schemes previously utilized in geophysical studies: it is argued that i) for the lack of data, an accurate filtering solution (section 6) is an essential prerequisite to the smoothing; ii) the linearization, if any, should be local enough; and iii) in real-time smoothing, a few iterations of approximate schemes can be very valuable (section 7c). With this in mind, smoothing via statistical approximation is developed in section 7a. Its ESSE version is outlined in section 7b. A discussion, with additional approximate ESSE algorithms, is presented in section 7c. Specifics are provided in appendix B.

a. Smoothing via statistical approximation

The approach consists of nonlinear filtering until t_N (sections 4–6), followed by the update of the conditional mean and error covariance, backward in time, based on future data. This later component is now outlined. It is recursive as was the filtering. The derivation presumes that the smoothing estimates ψ̂_k+1/N and P_k+1/N have been obtained. The unknowns are ψ̂_k/N and P_k/N. A statistical approximation (e.g., Gelb 1974; Austin and Leondes 1981) to the forward integration of (2a) between data times t_k and t_k+1 is first derived, assuming that ψ^t_k+1 is perfectly known. The approximation is written in a backward form. Based on the smoothing conditions at t_k+1, it is used to compute ψ̂_k/N and P_k/N. The resulting smoothing is shown to include a few classic schemes as particular cases. Its truncation to a significant subspace is outlined in section 7b.

The present statistical approximation is chosen to be locally linear. This implies that Δt_k+1 is small enough and that the statistical linearization is made around the data-corrected filtering solution, characterized by the initial conditions ψ̂_k(+) and P_k(+), and forecasts, ψ̂_k+1(−) and P_k+1(−). We seek a linear relation that estimates how to correct ψ̂_k(+) based on its forecast error. Assuming for now that ψ^t_k+1 is known, this relation is of the form

ψ̂_kψ̂_k(+)L_kψ^t_k+1ψ̂_k+1(−)(42)

Within the present minimum error variance approach, the unknown matrix L_k should be such that (42) minimizes the error variance of ψ̂_k, hence,

View Expanded

Subtracting ψ^t_k from both sides of (42) and forming P_k in (43) leads to

View Expanded

where P_k+1(−) is the error covariance forecast carried out from P_k(+) based on (1a)–(2a). Inserting (44) into (43) and taking the derivative with respect to L_k yields

View Expanded

which is a minimum since P_k+1(−) is positive semidefinite. The optimum (45) and relation (42) define the statistical backward linearization sought. It is now employed to compute the smoothing conditions at t_k. The best available unbiased estimate of ψ^t_k+1 is ψ̂_k+1/N, of error covariance P_k+1/N. Using it in (42) gives the smoothing estimate

ψ̂_k/Nψ̂_k(+)L_kψ̂_k+1/Nψ̂_k+1(−)(46)

with ψ̂_N/N = ψ̂_N(+). The error covariance associated with (46) is derived as follows. Subtracting ψ^t_k from (46) and rearranging the terms gives

View Expanded

Multiplying (47) by its transpose and taking expectations leads to

View Expanded

since the cross-terms, E{(ψ̂_k(+) − ψ^t_k)ψ̂^T_k+1(−)} and E{(ψ̂_k/N − ψ^t_k)ψ̂^T_k+1/N}, are null. The first of these is null because it involves unbiased error fields multiplied by the expected state ψ̂^T_k+1(−). To see that the second is also null, one can replace ψ̂_k+1/N using the linear relation (46), use the previous argument for ψ̂^T_k+1(−), and invoke the orthogonality principle (Davis 1977a), which states that the error ψ̂_k/N − ψ^t_k is orthogonal to functions of already used measurements, hence to ψ̂_k/N and ψ̂_k(+). For similar reasons, the relation E{ψ^t_k+1ψ^tT_k+1} = P_k+1 + E{ψ̂_k+1ψ̂_k+1} holds for all estimates ψ̂_k+1. Using it for ψ̂_k+1/N and ψ̂_k+1(−) into (48) yields the error covariance of ψ̂_k/N,

P_k/NP_kL_kP_k+1/NP_k+1L^T_k(49)

with P_N/N = P_N(+). The complete scheme is stated in Table 6. In passing, if ψ̂_k+1/N in (46) had been perfect, (49) would have been equal to (44). One can verify that (44) does not contain the positive-definite term involving P_k+1/N. In simple words, L_kP_k+1/NL^T_k in (49) is the error cost incurred for using (46) instead of (42).

Some of the classic smoothing algorithms are simplifcations of Table 6. The extended Kalman smoother (EKS) is first considered. In the EKS, the nonlinear integrations (34) and (37) are reduced using Taylor series expansions. Successive relinearizations about the last available estimate of the conditional mean are carried out. The resulting state estimate is the central forecast (35). To obtain a covariance estimate, the evolution of the perturbation (ψ^t − ψ̂) is linearized about (35), leading to

View Expanded

with initial conditions [ψ^t_k − ψ̂_k(+)]. Integrating (50) over Δt_k+1 and denoting by Φ(t_k+1, t_k) the corresponding state transition matrix, one has

View Expanded

where w_k+1 accounts for the integrated effects of dw over Δt_k+1 in (50). Inserting this simplification (51) into (45) yields

L_kP_kΦ^Tt_k+1t_kP⁻¹_k+1(52)

which, with (46) and (49), defines the Rauch–Tung–Striebel EKS (e.g., Jazwinski 1970). The linearized KS is also based on (52), but without the relinearizations. If (1a)–(2a) were linear models, Φ in (52) would simply be the transition matrix of (1a). The statistical approximation (45) hence encompasses several truncations of lower order. In (45), all nonlinearities are kept in computing the expectations; derivatives do not need to exist. In fact, truncating the models (1a)–(2a) prior to computing these expectations, as was done in (50)–(52) using Taylor series, has been shown to be less accurate in several filtering cases (e.g., Austin and Leondes 1981; Uchino et al. 1993). In passing, expansions of higher order than (50)–(52) would still replace the global statistical properties of (37) and (45) by local derivatives.

b. ESSE and smoothing via statistical approximation

The smoothing scheme in Table 6 is now reduced to its significant components. The recursive derivation presumes that the smoothing estimates ψ̂_k+1/N, E_k+1/N and Π_k+1/N defining P^p_k+1/N, have been obtained based on (6c). The unknowns are ψ̂_k/N, E_k/N, and Π_k/N. Starting from the nonlinear filtering ESSE scheme run forward until t_N (section 6), a Monte Carlo sample estimate of L_k in (45) is, at O(1/q) in standard deviation,

View Expanded

where M_k(+) and M_k+1(−) are defined as in section 6. The † logically denotes the Moore–Penrose generalized inverse. Note that in (53), q = q_k(+) = q_k+1(−) so that all multiplications are feasible. Truncating the sample matrices in (53) to their dominant SVD,

View Expanded

and using the orthogonality properties of the singular vectors yields the estimate

View Expanded

The smoothing ESSE gain L^p_k is thus defined by SVD_p[M_k(+)]SVD_p[M_k+1(−)]†. The covariance associated with (46) and (55) can be derived similarly to (47)–(49); one obtains

P^p_k/NP^p_kL^p_kP^p_k+1/NP^p_k+1L^pT_k(56)

For Γ_k ≐ Σ_k(+)V^T_k(+)V_k+1(−)Σ⁻¹_k+1(−) and θ_k+1 ≐ E^T_k+1(−)E_k+1/N, (56) becomes

View Expanded

where Π_k ≐ (1/q_k)Σ²_k, ∀k. Hence, the smoothing update of the ES characteristics consist of

View Expanded

The a posteriori covariance can then be obtained from P^p_k/N = E_k/NΠ_k/NE^T_k/N. The sequential scheme is summarized by Table 7. In cases with strong nonlinearities or long intervals without data, Table 7 can be iterated. Once ψ̂¹_0/N is known, a new nonlinear ESSE filtering up to t_N can be carried out, followed by the statistical smoothing up to ψ̂²_0/N and so on. Such iterations are discussed for the EKS in Jazwinski (1970); similar equations apply here.

c. Discussion

An essential property of Tables 6–7 is that the four-dimensional statistics of the dynamical variability, model errors, and data available enter the cost function (6c). Another is that the statistical smoothing is made on the nonlinear filtering ESSE estimate, which is already a state evolution corrected by data. In fact, this contrasts with the ensemble smoother (Evensen 1997a) or iterated representer methods (e.g., Bennett et al. 1996, 1997). Evensen and van Leeuwen (1996) showed that the ensemble Kalman filter was superior to the ensemble representer smoother. An explanation for this unexpected issue is that in the linear representer approach, the first guess is the pure (without data assimilation) deterministic forecast up to t_N. Predictability errors, as well as numerical and analytical model deficiencies can then dominate before reaching t_N. Driving the pure forecast back to the true ocean evolution by quadratic minimization is therefore numerically difficult. This issue could be solved by modifying the original representer equations, ψ̂_k = ψ̂^f_k + R̃_klb_l (Robinson et al. 1998a), into

View Expanded

where R̃_kl(+) ≐ [r¹_kl(+), . . . , r^m_kl(+)] ∈ R^n×m are a posteriori representers and b_l(+) ∈ R^m their coefficients. Equations for R̃_kl(+) and b_l(+) can be derived from the linearization of Tables 6–7. Other advantages of Table 7 include the efficient reduction to the dominant errors;the orthogonality properties of the SVD leading to simplified, efficient computations; and the evaluation of the a posteriori error covariances (58a)–(58b). Table 7 has been successfully utilized for PE process studies in the Levantine Basin, whose first results are partially described in Lermusiaux (1997).

In real-time smoothing with very large multivariate states, approximate approaches can be very useful benchmarks, as the OI scheme continues to be in filtering studies (e.g., Lermusiaux 1999b). For instance, one may impose a priori the form of the smoothing gains or simply best-fit the dynamics to the data. In the latter case, for an ensemble (40) of nonlinear forecasts during Δt_k+1, a first approach is to set ψ̂_k/N to the initial conditions of the forecast that minimizes the forecast–data misfits:

View Expanded

This smoothing is a shooting method toward future data. It is local, ψ̂_k/N = ψ̂_k/k+1, but it can be iterated. Taking the data errors into account, one obtains the cost function

View Expanded

Examples of such smoothing are given in Lermusiaux (1997). A few gradient descents in an adjoint approach (Sundqvist 1993; Rabier et al. 1996) extend (60b) to the whole interval [t₀, t_N], with an additional term for the model parameters (e.g., initial and boundary conditions). Deriving simple nonlinear subspace methods for iterative adjoint strategies is valuable. Miller and Cornuelle (1999) have successfully utilized such a reduced-state inverse method for initialization. Assuming perfect dynamics and a diagonal data error covariance matrix, the authors employed an ensemble of large-scale horizontal structure functions to adjust the initial conditions to the future data.

8. Summary and conclusions

Utilizing basic properties and heuristic characteristics of geophysical measurements and models, a nonlinear assimilation approach has been outlined. The concept of an evolving error subspace (ES), of variable size, that spans and tracks the scales and processes where the dominant errors occur was defined. The word “dominant” is naturally specified by the error measure used in the chosen optimal estimation criterion. Truncating this criterion to its evolving ES defines the error subspace statistical estimation approach (ESSE). The general goal of ESSE is to determine the considered ocean–atmosphere state evolution by minimizing the dominant errors, in accord with the full dynamical and measurement model constraints, and their respective uncertainties. This rational approach, satisfying realistic and practical goals while addressing geophysical issues, leads to efficient estimation schemes. For the minimum error variance criterion, the ES is characterized by time-variant principal error vectors and values. In general, the size and span of the ES evolve in time. The meaning and validity of the dominant truncation of the error space was addressed, with a focus on synoptic/mesoscale to large-scale geophysical applications. The ES was intercompared to variations of variability subspaces and its properties were discussed. In Part II of this study, primitive-equation simulations of Middle Atlantic Bight shelfbreak front evolutions are employed to assess and exemplify the capabilities of an ESSE system. The approach has also been used in real-time forecasting for North Atlantic Treaty Organization (NATO) operations in the Strait of Sicily and in the simulation and study of the spreading of the Levantine intermediate water (Lermusiaux 1997). Other real-time simulations were carried out in the Ionian Sea and Gulf of Cadiz (Robinson et al. 1998b).

In this first part, filtering and smoothing schemes for nonlinear assimilation via ESSE are derived (Figs. 1, 2). The time integration of the ES is based on Monte Carlo ensemble forecasts. The a posteriori members sample the current ES and are forecast using the full nonlinear stochastic model. The melding criterion minimizes variance in the ES and is much less costly than classical analyses involving full covariances. The statistical smoothing keeps all nonlinearities in computing expectations and is carried out from the nonlinear filtering solution, which is already corrected by data. For given computer resources, the representation of errors is energetically optimal. The dynamical forecast is corrected by data where the forecast errors are most energetic. The assimilation is multivariate and three-dimensional in physical space. The model nonlinearities and model errors are accounted for, and their effects on forecast errors explicitly considered. The SVD facilitates the analysis of the evolving error covariance. The scheme is well suited to parallel computers. The formalism, while conceptually simple, can be complex in its details. Determining mathematical systems that describe and track the dominant errors is challenging. Several specific components and variations of the present schemes are provided in appendix B. Dynamical systems for tracking and learning the ES (Brockett 1991) are derived in Lermusiaux (1997).

The concepts introduced by the subspace approach are as useful as the practical benefits. The ESSE formalism defines a theoretical basis for rational intercomparison of other reduced dimension methods. As discussed in the text, these methods have led to numerous controversies and, in fact, several of their respective assumptions were simply shown to be incompatible. By definition, the present approach can rationally validate specific a priori error hypotheses for tailored applications. In accord with the evolution of the deterministic dynamics, model errors, and data available, ESSE may, for instance, lead to different simplified assimilation schemes for weather or ocean forecasting. In dynamical modeling, specific interests lead to verified a priori dynamical assumptions; similarly, in assimilation there are a priori error assumptions to verify. The focus on the dominant errors also fosters the testing and correction of existing dynamical models (e.g., coding mistakes are often associated with a large error growth). It equally implies future observations that span the dominant forecast errors or a search for data optimals (Lermusiaux 1997). ESSE in fact provides a feasible quantitative approach to both dynamical model improvements and adequate field sampling. Finally, aside from the assimilation framework, the present scheme has other applications. Turning off the assimilation in Fig. 2, one can study the impact of the dominant stochastic model errors. This is a new research area and ESSE can validate specific stochastic models for use in simulations. Without model errors and assimilation, the statistical estimation of the variations of variability and stability subspaces is considered. Predictability and stability properties (e.g., Farrell and Ioannou 1996ab) can thus also be decomposed and analyzed. Fixing the estimation time yields an objective analysis scheme. In general, the range of applications includes nonlinear field and error forecasting, data-driven simulations, model improvements, adaptive sampling, and parameter estimation. The accurate tracking and specification of the dominant errors hence appears of paramount importance, even from a fundamental point of view.