a. Observational update in square root or ensemble Kalman filters

In Kalman filters, the standard formula to compute the observational update increment δx of the model state vector is

View Expanded

where δy = y − 𝗛x^f is the innovation vector, representing the difference between the observation vector y (size y) and the model equivalent to the observation in the forecast state vector x^f (size x), and 𝗣^f(x × x) is the forecast error covariance matrix. The computational complexity (leading behavior for large x and y) of this standard formula is

View Expanded

In the computation of C₀, it is assumed that a linear system is solved to compute (𝗛𝗣^f𝗛^T + 𝗥)⁻¹δy, with asymptotic complexity y³/6 (for a symmetric matrix). The second terms in C₀ corresponds to the left multiplication by (𝗛𝗣^f)^T. In addition, the cost of application of the observation operator 𝗛 is assumed negligible throughout this discussion. It is negligible for instance if every observation is related to a small number of state variables. If 𝗛 is more complex, it is straightforward to add the cost of 𝗛 to the computational complexity formulas and transform the conclusions accordingly.

If the forecast error covariance matrix is available in square root form:

View Expanded

as in square root or ensemble Kalman filters, the standard formula (2) can be transformed into

View Expanded

as suggested by Pham et al. (1998), using the Sherman–Morrison–Woodbury formula:

View Expanded

Formula (5) is advantageous with respect to formula (2) only if 𝗥 can be inverted at low cost. For instance, if 𝗥 is diagonal, the asymptotic computational complexity of formula (5) (leading behavior for large x, y and r) is

View Expanded

where r is the number of columns in 𝗦^f (the maximum rank of 𝗣^f). The first term corresponds to the computation of the r × r matrix between brackets; the second term, to the solution of the linear system; and the last term, to the left multiplication by 𝗦^f. The main difference between formula (5) and formula (2) is that the linear system to solve is of size r (complexity r³/6) instead of y (complexity y³/6).

The key advantage of formula (5) with respect to formula (2) is that the computational complexity C₁ is linear in the number of observations y (a property that disappears if a general matrix 𝗥 is inverted). With formula (5), larger observation vectors become numerically tractable. Asymptotically, for large values of x, y, and r, with fixed ratios y/x and r/x (this means in practice that any of these numbers is small with respect to the products of the other two: x ≪ yr, y ≪ xr, r ≪ yx), the gain factor that is obtained by using formula (5) instead of formula (2) simplifies (only the cubic terms remain) to

View Expanded

Even in the full rank problem (r ≥ x), formula (5) is cheaper than formula (2) (asymptotically) as soon as y/r > 2.53. But the benefit of formula (5) becomes really clear in the small rank problems (r ≪ x), that result from the application of reduced rank or ensemble Kalman filters. In these problems, it is often useful to reach very small r/y ratios, for which formula (5) is by far preferable. Nevertheless, the main drawback of using formula (5) is that it leads to assuming a diagonal observation error covariance matrix. It is the purpose of this paper to show how it is possible to introduce parameterizations of the observation error correlations that preserve the numerical efficiency of formula (5).

It is worth noting here that, in realistic applications, the observational update is often performed locally by dividing the full model state in subdomains, and by performing a separate observational update for every subdomain using a subset of the global observation dataset (see, e.g., Anderson 2003; Houtekamer and Mitchell 1998; Ott et al. 2004; Tippett et al. 2003). With local methods, the size of the observation vector can be significantly reduced with respect to a global observational update, thus modifying the computational complexity (3) and (7) of the algorithms and the gain factor (8) that is obtained by using formula (5) instead of formula (2). The same expressions can however still be applied providing that x and y are defined as the size of the local state and observation vectors. In addition, if r is still the number of columns in 𝗦^f, it can usually be set smaller with local methods. The use of low-rank 𝗣^f parameterizations (or small size ensembles) is indeed one important reason for which local methods are required (Houtekamer and Mitchell 1998).

b. Observational update of the error covariance in square root or ensemble Kalman filters

In this section, we examine how the conclusions of the previous section must be modified if the problem requires the observational update of the error covariance. (This is always done if the Kalman filter is not approximated by an optimal interpolation scheme.) In Kalman filters, the standard formula to compute the observational update of the error covariance, corresponding to formula (2) is

View Expanded

where 𝗣^a is the updated (analysis) error covariance matrix. The computational complexity (leading behavior for large x and y) of this standard formula is

View Expanded

The first term corresponds to the symmetric matrix inversion, and the two last terms to matrix multiplications. This complexity includes complexity C₀ if formulas (2) and (9) are applied together because most operations of formula (2) are included in formula (9).

If the forecast error covariance matrix is available in square root form (4), it is shown (Pham et al. 1998) that the updated matrix, corresponding to formula (5) can be obtained in square root form (𝗣^a = 𝗦^a𝗦^aT) using the formula

View Expanded

If 𝗥 is diagonal, the computational complexity (leading behavior for large x, y, and r) of formula (11) is

View Expanded

The first term corresponds to the computation of the r × r matrix between brackets, the second term includes the computation of the inverse matrix and the Cholesky decomposition of the inverse (the cheapest square root), and the last term corresponds to the left multiplication by 𝗦^f. This complexity includes complexity C₁ if formulas (5) and (11) are applied together because most operations of formula (5) are included in formula (11).

Again, the key advantage of formula (11) over formula (9) is that the computational complexity C₁^P is linear in the number of observations y. However, new cubic terms appear so that the (asymptotic) gain factor is not as simple as (8):

View Expanded

However, as soon as x/y remains moderate, the conclusions of section a remain valid: the gain behaves proportionally to (r/y)² for small r/y. And if x/y is large, formula (11) is even more favorable since the gain behaves like (r/y)²y/x for small r/y. It is also worth noting that with formula (11), the additional cost of computing the update of error covariance, with respect to formula (5) is usually moderate, behaving at most (for small r/y) like C₁^P/C₁ ∼ 1+x/y.

In the ensemble Kalman filter (Evensen and van Leeuwen 1996), the update of the error covariance is performed differently: the ensemble forecast (describing the error covariance) is updated by the application of formula (2) to an ensemble of innovation vectors (representing the difference between the observation vector and each member of the ensemble forecast, perturbed by a random vector of covariance 𝗥). However, the complexities C₀ and C₁ are not simply multiplied by the size r of the ensemble because it is cheaper here to explicitly invert the matrix, rather than solving the r linear systems, so that (3) transforms to

View Expanded

The first term corresponds to the inversion of the matrix 𝗛𝗣^f𝗛^T + 𝗥, the second term includes the computation of the matrix 𝗛𝗣^f𝗛^T from the square root representation and the application of the inverse matrix to the ensemble innovations, and the third term corresponds to the left multiplications by (𝗛𝗣^f)^T to obtain the ensemble corrections. On the other hand, (7) transforms to

View Expanded

The first term includes the computation of the r × r matrix between brackets and the application of the inverse matrix to the ensemble innovations, the second term corresponds to the inversion of the matrix between brackets, and the last term to the left multiplication by 𝗦^f to obtain the ensemble corrections. New cubic terms appear in (14) and (15) so that the gain factor is analog to (13):

View Expanded

However, as soon as x/y remains moderate [x/y ≪ (y/r)²], the conclusions of section a remain qualitatively the same: the computational complexity C₁^E is linear in y, and for small r/y, the gain (16) behaves proportionally to (r/y)². [If x/y is large, the leading behavior of C₁^E/C₀^E for small r/y is proportional to r/y instead of (r/y)², because the leading terms in (14) and (15) become the last terms, proportional to x, whose ratio is equal to r/y.]

c. Modal parameterization of the observation error covariance matrix

Any observation error covariance matrix can be written in the form:

View Expanded

where 𝗗 is a positive definite diagonal matrix such that 𝗥 − 𝗗 remains positive definite. There indeed always exists a real square root Θ(y × q) of the positive definite symmetric matrix 𝗗^−1/2𝗥𝗗^−1/2 − 𝗜. Using (6), the inverse of 𝗥 can be written

View Expanded

The application of the transformed algorithm, using formula (5) or (11), with the expression (18) of 𝗥⁻¹ requires the computation of:

View Expanded

and

View Expanded

where 𝗟𝗟^T is the Cholesky decomposition of (𝗜 + Θ^TΘ)⁻¹. The leading behavior of the additional computational complexity comes from the second term of (19):

View Expanded

The first term corresponds to the computation of the matrix (𝗜 + Θ^TΘ), the second term to the multiplication 𝗔 = (𝗛𝗦^f)^T𝗗^−1/2 Θ, the third term includes the inversion of (𝗜 + Θ^TΘ)and the Cholesky decomposition of the inverse, and the two last terms are matrix multiplications (𝗕 = 𝗔𝗟 and 𝗕^T𝗕).

Formula (5) or (11) with parameterization (17) for 𝗥 can only be advantageous with respect to formula (2) or (9) (i.e., C₁ + C₁^R < C₀, C₁^P + C₁^R < C₀^P or C₁^E + C₁^R < C₀^E) if the number of columns q of Θ is small with respect to the number of observations (q ≪ y); that is, if the observation error covariance matrix 𝗥 is the sum of a diagonal matrix and a low rank matrix. [This is why expression (17) is chosen: the diagonal term is necessary to make the matrix regular.] If this can be done, the computational complexity remains linear in the number of observations y and the numerical efficiency of formulas (5) and (11) is preserved.

A further difficulty is that Θ needs to be computed. Obviously, it cannot be computed by decomposition of a full size 𝗥 matrix (followed by rank reduction), because the computational complexity of such operation is again proportional to y³. A possibility (for spatially distributed observations) is to define the correlated part of 𝗥 at the nodes of a regular grid (𝗥^g), compute the square root 𝗥^g = Θ^gΘ^gT (once for all) on that grid (with rank reduction if possible), and interpolate the modes Θ^g at observations locations (for every spatial distributions of the observations) to obtain Θ = 𝗛^gΘ^g. Such approximation is valid if the error modes Θ^g contain only scales that are large against the regular grid resolution; that is, if the 𝗥 matrix can be represented by the superposition of a white noise (the diagonal part 𝗗) and a large-scale red noise (the correlated part 𝗗^1/2ΘΘ^T𝗗^1/2). In such case, two observations that are close together (much closer than the red noise correlation scales) are assumed fully independent with respect to the white noise, and fully dependent with respect to the red noise.

This parameterization is thus particularly efficient if the typical distance between observations is small with respect to the correlation scales, because then the number q of error modes can be made small with respect to the number of observations y (q ≪ y), and the additional cost C₁^R, given by (21), remains tractable (asymptotically for large y): the linear term in y is only increased to y(r ² + q² + rq) instead of yr ². In other situations, this parameterization cannot preserve the efficiency of formulas (5) and (11) and other solutions must be found (see next sections).

An even more efficient parameterization can be built by using directly a reduced rank parameterization for the inverse observation error covariance matrix 𝗥⁻¹ = ΘΘ^T, with square root Θ(y × q), q ≪ y. With respect to parameterization (17), the linear term in y is reduced to y(r ² + rq) instead of y(r ² + q² + rq). Such a simplified parameterization is used in the oceanographic applications described in Brankart et al. (2003) and Testut et al. (2003). However, singular parameterizations of 𝗥⁻¹ are dangerous because they imply that the null space of 𝗥⁻¹ is assumed unobserved (infinite observation error), which may lead to neglecting important observational information. Again, this amounts to building superobservations (presumably the most useful ones) by projecting the original observations on the error modes (the columns of Θ), and dropping everything that is orthogonal to that. In this paper, we prefer to follow our original plan to keep all observations and thus only propose regular parameterizations of 𝗥.

d. Simulating correlations by linear transformation of the observation vector

The observational update given by formula (2) or (5) also minimizes

View Expanded

In this equation [as well as in (25) below], it is the Moore–Penrose pseudoinverse of 𝗣^f that must be used if the matrix is rank-deficient. In this cost function, we can transform the observation vector by a regular (rank equal to y) linear transformation operator 𝗧: δy⁺ = 𝗧δy, 𝗛⁺ = 𝗧𝗛, in such a way that J remains unchanged, providing that the observation error covariance matrix is also transformed according to

View Expanded

It can be easily verified that the same transformation also leaves unchanged formulas (2), (5), (9), and (11). It follows that any observation error covariance matrix 𝗥 can be simulated by a diagonal matrix 𝗥⁺ in a transformed observation space. An immediate solution is to choose 𝗥⁺ as the matrix of eigenvalues of 𝗥 and 𝗧 as the matrix with the corresponding normalized eigenvectors (so that 𝗥 = 𝗧^T𝗥⁺𝗧, with 𝗧 unitary and 𝗥⁺ diagonal). Obviously, this is not the solution that we are looking for, since the computational complexity of the eigenvalue problem is again proportional to y³.

Moreover, for a general linear operator 𝗧, the computational complexity of the application of the operator (e.g., to compute δy⁺ = 𝗧δy) is equal to yy⁺, where y⁺ is the size of the transformed observation vector (y⁺ ≥ y for a regular transformation). Hence, this complexity can only be linear in y if the structure of 𝗧 is simple. It can even become negligible (asymptotically) if every transformed observation (in the vector y⁺) is related to a small number of original observations. (It is the same argument that leads to neglecting the cost of 𝗛, see section 2a.) On the other hand, because the cost of the observational update is linear in y in formulas (5) and (11), we have the freedom to imagine a transformation 𝗧 that increases the number of observations (y⁺ > y), without prohibitive consequence on the numerical cost. Essentially, as soon as 𝗧 is known and is simple enough, the same computational complexity as (7), (12), and (15) applies, with y replaced by y⁺. (Thus the relative cost is multiplied by y⁺/y.) An example of such simple transformation, consisting of adding gradient observations to the original observation vector, is examined in section 3.

In addition, it is interesting to point out that, with uncorrelated observation errors in the transformed observation space, the observational update described by formulas (2) and (9) can be replaced by a repeated application of these formulas, using the observations in y⁺ one by one. The updated x^a, 𝗣^a obtained at each stage of the sequence are used as background state and background error covariance (x^f and 𝗣^f) for the next update. This is the serial processing algorithm that is also often used in ensemble filtering to reduce the numerical cost (at the expense of the assumption that observation errors are independent). By constructing an augmented observation vector with a diagonal error covariance matrix, the transformation method proposed in this paper thus also allows the application of this serial algorithm in presence of observation error correlations. On the other hand, in many applications, there can be several observation datasets with independent errors (e.g., if they originate from different instruments) so that the matrix 𝗥 is block-diagonal. Such observation error covariance matrices can also be easily simulated by this method using separate transformations to the corresponding segments of the observation vector, for instance by augmenting the observation vector with discrete gradients computed inside each observation dataset.

Finally, in order to prepare some of the developments of section 3, it is useful to present how the transformation problem must be reformulated if the system is continuous instead of being discrete. The continuous model state x(ξ′) is assumed observed by a continuous observation y(ξ) through a general linear observation operator H:

View Expanded

where ε(ξ) is the observational noise. Then (22) becomes:

View Expanded

where δw(ξ) is the observation residual:

View Expanded

and R⁽⁻¹⁾(ξ,η) is the inverse observation error covariance:

View Expanded

and P^{f(− 1)}(ξ′,η′) is the inverse (or pseudoinverse) forecast error covariance. If the observation y(ξ) is transformed by a general linear transformation T:

View Expanded

J remains unchanged if

View Expanded

If the observation error is assumed spatially uncorrelated R⁺(ξ⁺, η⁺) = R⁺(ξ⁺)δ(ξ⁺, η⁺), this last formula simplifies to

View Expanded

Given the transformation operator T(ξ⁺, ξ) and the observation error variance in the transformed space R⁺(ξ⁺), the corresponding observation error covariance in the original space R(ξ, η) can be computed using (30) together with (27).

a. One-dimensional problem

If we assume that the observations are distributed spatially along a one-dimensional line, we can think of simulating error correlations along the line by adding gradient observations in the observation vector. Starting with the continuous problem, if y(ξ) is the original observation (where ξ is a curvilinear abscissa along the line), the transformed observation vector is then composed of the original function together with its first derivative:

View Expanded

where T₁(ξ⁺, ξ) = δ(ξ⁺ − ξ) is the identity operator and T₂(ξ⁺, ξ) is the derivative operator. Assuming that R⁺(ξ⁺) is spatially homogeneous:

View Expanded

where σ₀ is the observation error standard deviation and σ₁ is the gradient error standard deviation, (30) can be rewritten as

View Expanded

Using (33), the definition of T₁ and T₂, and taking benefit of the homogeneity of the solution R(ξ, η) = R(ρ) with ρ = ξ − η, (27) transforms to

View Expanded

whose solution is

View Expanded

Adding gradient observations in the observation vector is thus equivalent to assuming that the observation error correlation decreases exponentially with the distance |ρ|, with a decorrelation length ℓ equal to the ratio between observation error and gradient error standard deviations (while the observation error variance is divided by 2).

The discrete problem is similar to the continuous problem, except that no explicit solution can be found analytically. Assume that observations y_i are available along the line, at abscissas ξ_i, i = 1, …, y, and that we add to this observation vector, observations of the discrete gradient (left difference)

View Expanded

The size of the new observation vector is thus y⁺ = 2y − 1, and the transformation is

View Expanded

where 𝗧₁ is the identity matrix, and T_2,ij = (δ_ij − δ_{i − 1,j})/(ξ_i − ξ_{i − 1}). From this, it is easy to compute the observation error covariance matrix 𝗥 on y corresponding to a diagonal observation error covariance matrix 𝗥⁺ on y⁺ using (23). If 𝗥⁺ is homogeneous,

View Expanded

and if the observations are regularly distributed (ξ_i − ξ_i−1 = Δξ ∀i), it follows that

View Expanded

Equation (39) is a consistent discretization of (33) [on a limited domain of size (y − 1)Δξ, with Neuman homogeneous boundary conditions], except for a factor Δξ/ℓ in the discretization of the Dirac function, so that (35) provides the asymptotic solution of (39) (multiplied by ℓ/Δξ) as Δξ → 0 and yΔξ → ∞.

Figure 1 shows the solution of (39) computed numerically (by inversion of a tridiagonal matrix) for σ₀ = 1 and different values of ℓ/Δξ, as compared with the continuous solution (35). The solution is drawn for ℓ = 1 and decreasing Δξ (left panel), showing the convergence toward the exponential decorrelation as Δξ → 0; and for Δξ = 1 and decreasing ℓ, showing how small correlation length scales (smaller than the observation resolution Δξ) are parameterized with this approach.

Anticipating possible mistakes in the applications, it is useful to examine the problems that occur if we replace the original observations by gradient observations (instead of adding gradient observations, as suggested in this paper), and assume a diagonal error covariance matrix in this transformed space. To make the transformation regular, we keep the first observation of y as first element of y⁺: y₁⁺ = y₁ (with error variance σ₀²), and then use the observation differences as next elements: y_i⁺ = y_i − y_i−1, i = 2, …, y (with error variance σ₁²Δξ², assuming a regular distribution). The transformation matrix is thus square (y⁺ = y) and regular, and can be inverted:

View Expanded

(Each original observation y_i is the sum of the first i elements of y⁺, the first observations plus the sum of the i − 1 first differences until y_i is reached.) Since 𝗧 is square and regular, (23) can be inverted explicitly:

View Expanded

so that the elements of 𝗥 are

View Expanded

It follows that the resulting error variance R_i = σ₀² + iσ₁²Δξ² increases linearly with distance with respect to the reference observation. (Of course, the increase can be reduced by placing the reference observation in the middle of the line, or by using the mean of the observations, but the effect remains essentially the same.) Hence, assuming independent errors on the observation differences means that their error variances (σ₁²Δξ²) add up to form the error variances on the original observations, y_i. Even if the errors on the gradient are assumed small, such a transformation is inappropriate because it leads to large errors on the original variable.

b. Two-dimensional problem

In the same way, if we assume that the observations are distributed spatially over an n-dimensional manifold, error correlations along the manifold can be simulated by adding gradient observations in the observation vector. The continuous problem is formally identical with several dimensions, except that ξ is an n-dimensional vector of curvilinear coordinates, and T₂ is the n-dimensional gradient, so that (34) becomes

View Expanded

where Δ is the n-dimensional Laplacian operator and ρ = ||ξ − η|| is the Euclidean distance. Equation (43) stands for the homogeneous and isotropic problem, but it is straightforward to introduce inhomogeneity or anisotropy by a nonlinear change of the ξ coordinates. In two dimensions, the solution of (43) is

View Expanded

where K₀ is the second kind modified Bessel function of order 0. It can indeed be easily verified [using the properties of K₀ in Abramowitz and Stegun (1970)] that (44) is the solution of the homogeneous equation in (43) [i.e., δ(ρ) replaced by 0] everywhere except at the origin, and that the coefficient σ₀²/2π is scaled so that the logarithmic singularity of (44) at ρ = 0 has the right amplitude to be the solution of (43) [viewed as a Green equation; see Morse and Feshbach (1953), chapter 7].

The two-dimensional discrete problem is also similar to the one-dimensional version. We assume that the observations y_ij are available on the two-dimensional surface at the nodes of a grid, with coordinates ξ_ij, η_ij, i = 1, …, y₁, j = 1, …, y₂ (where y₁ and y₂ are the number of rows and columns of the grid), and that we add to the observation vector, observations of the two components of the discrete gradient (left difference):

View Expanded

The size of the new observation vector is thus y⁺ = 3y₁y₂ − (y₁ + y₂), that is, almost a factor 3 with respect to the number of original observations (y = y₁y₂). The transformation is

View Expanded

where 𝗧₁ is the identity matrix and 𝗧_2,1, 𝗧_2,2 are the discrete gradient operators. Note that each line of the 𝗧 operator combines only one or two observations, so that the cost of application of 𝗧 remains always negligible (the computational complexity is equal to 2y). The application of (23) with 𝗥⁺ homogeneous leads to an expression of 𝗥⁻¹ similar to (39). However, the matrix is no longer tridiagonal because each element is combined with its four neighbors in the grid (), which cannot always be consecutive in the y vector. It can be easily seen that this provides a consistent discretization of (43) in two dimensions (except for a factor Δξ²/ℓ² in the discretization of the two-dimensional Dirac function), so that (44) is the asymptotic solution of the discrete problem (with a scale factor ℓ²/Δξ²) as the grid steps tend to zero. Figure 2 presents the same information as Fig. 1 for the two-dimensional problem (with regular and isotropic grid spacings), illustrating the shape of the simulated covariance for σ₀ = 1 and for various values of ℓ/Δξ, and showing the convergence toward the analytical solution (44) as Δξ → 0.

c. Higher order derivatives

The solution of (43), which is valid for homogeneous and isotropic problems, can also be found in the spectral domain; the solution is then the error power spectrum , which is the Fourier transform of the covariance function. In isotropic problems, it only depends on the modulus of the wave vector (κ = ||k||), and the solution is

View Expanded

Equation (47) gives the spectral distribution of the observational error: no error on the small scales (κ ≫ 1/ℓ) and constant spectral distribution for the large scales (κ ≪ 1/ℓ). The corresponding isotropic covariance function R(ρ) given by (35) and (44) (valid for ℓ ≠ 0) can then be found as the inverse Fourier transform (for the one-dimensional function) or the inverse Hankel transform (for the two-dimensional isotropic function) of (47) [see the general formulas (1.0) and (2.0) in Table 1].

More complex observation error power spectra can be simulated by adding p successive derivatives of the observations in the observation vector (with error standard deviation σ_i, i = 1, …, p). Equation (47) then generalizes to

View Expanded

Adding constraints on the successive observations derivatives (e.g., gradient, curvature) is thus equivalent to assuming a specific shape of the observation error power spectrum or of the observation error covariance function. This equivalence is similar in nature to the one-to-one correspondence (Kimeldorf and Wahba 1970; McIntosh 1990; Brankart and Brasseur 1996) between spline analysis (minimizing curvature and gradient) and statistical analysis (with specific shapes of the background error covariance). There, the correspondence is on the background constraint in the cost function (25) (first term) instead of being on the observation constraint (second term of the cost function).

Table 1 provides explicit expressions of the covariance function R(ρ) that can be obtained from (48) for some specific values of the parameters σ_i. However, with expression (48), any shape of the observation error spectrum (provided that it is indefinitely continuously differentiable) can virtually be simulated; even negative σ_i² are possible (for 0 < i < p) providing that . For instance, simulating a Gaussian observation error spectrum (corresponding to a Gaussian covariance function, whatever the number of dimensions) requires an infinite derivative sequence σ_i = ℓⁱ, i = 1, …, ∞ [functions (1.7) and (2.5) in Table 1], but can be approximated by a truncated sequence. Going to a second-order derivative is nevertheless always necessary to simulate a correlation function with zero derivative at ρ = 0 [as functions (1.2)–(1.6) and (2.2)–(2.4) in Table 1].

It is interesting to note that, in one dimension (n = 1) and for one derivative included in the observation vector (p = 1), the error power spectrum given by (47) is characteristic of a random function ε(ξ) that is governed by the differential equation:

View Expanded

where σ₀w(ξ) is a white noise with standard deviation σ₀. Equation (49) is the Langevin equation, which is used in statistical physics to describe the time evolution of particle velocities in the Brownian motion (Reif 1965) or the behavior of random fluctuations in thermodynamical systems (Landau and Lifshitz 1951, chapter 12). The corresponding correlation model given by (35) describes thus also the time correlation of these important physical processes. More generally, any observational noise that is related to a white noise with such a linear differential equation (written here in one dimension):

View Expanded

is characterized by a power spectrum given by (48), with a number of derivatives p equal to the degree of the differential equation in (50). The parameters σ_i of the power spectrum can be easily deduced from the coefficients a_i, by transforming (50) in the spectral domain. For instance, for p = 2, an observational noise governed by

View Expanded

is characterized by a correlation function that can be simulated by adding the first and second derivatives to the observation vector with associated error variances σ₁² = σ₀²ℓ⁻²/(λ² − 2) and σ₂² = σ₀²ℓ⁻⁴ [correlation functions (1.2), (1.3), and (1.4) in Table 1]. Such relationships can help to determine the appropriate parameterization of the error power spectrum as soon as it is possible to find approximate linear differential equations governing the observational noise (e.g., if it is due to unresolved physical processes).

The generality of the method is more directly obvious for discrete problems since any transformation 𝗧 can be obtained by adding finite differences of successive orders. However, increasing the number p of derivatives added to the observation vector also increases the numerical cost, so that the most effective parameterization always results from a compromise between a fine representation of a target observation error spectrum and the numerical efficiency of the observational update. In this respect, two critical elements are always the identification of an accurate prior model for the observation error correlations and the validation of this model using the observed information.

a. Description of the experiment

The North Brazil Current is a surface western boundary current flowing westward along the north Brazilian coast. It is fed from the southeast by the tropical surface current, and brings the water to the northwest into the Caribbean Sea. The current sheds large anticyclonic rings (with diameter of about 200 km), that are also transported toward the Caribbean Sea, covering the 2000 km in about 3 months [see Fratantoni et al. (1995) for more details]. The total transport of the mean current is about 21 Sv (1 Sv ≡ 10⁶ m³ s⁻¹; da Silveira et al. 1994), with typical surface velocities of 1 m s⁻¹ for the main current and for the rings, corresponding to dynamic height differences of about 0.2 m.

A reference simulation of the circulation is computed using a primitive equation model covering the tropical Atlantic between 15°S and 20°N. It is a subregion of the Drakkar global ocean configuration at a 1/4° resolution of the Nucleus for European Modelling of the Ocean model (NEMO; Barnier et al. 2006; Penduff et al. 2007), using boundary conditions extracted from a global simulation. The model atmospheric forcing is computed from European Centre for Medium-Range Weather Forecasts 40-year reanalysis (ERA-40) atmospheric data using bulk aerodynamic formula. A 5-yr reference simulation of the tropical Atlantic model (computed by repeating 5 times the 2002 atmospheric data) is illustrated in Figs. 3 and 4. In this study, we focus on the results obtained in the region of the North Brazil Current (between 4.5° and 12.5°N, and between 6° and 46.5°W), that is shown on the figures. Figure 3 presents two snapshots of the sea surface height, together with its gradient and surface velocity for 2 and 14 December of the first year, showing the rings moving westward, and illustrating the close relation between altimetry and surface velocity. Figure 4 shows the mean circulation (sea surface height, gradient and surface velocity) averaged over the 5 yr of the simulation, together with the corresponding standard deviation. The order of magnitude of the sea surface height variability is similar to the bulk error standard deviation on satellite altimetric measurements, which is presently about 0.04 m. This variability is thus only marginally observed by such satellites, so that a fine tuning of the statistical parameters is particularly needed.

To test the observational update with different kinds of observation error parameterization, we need to define (i) the background (or forecast) state x^f, and (ii) the true state x^t, from which the observations y are sampled, and to which the estimation must be compared. As background state, we use the mean circulation (shown in Fig. 4, top panels); as true state, we use one of the model snapshots (illustrated in Fig. 3). And as observation, we assume that altimetry is observed over the full domain, with a 4-cm error standard deviation, at every node of the model grid. To test the sensitivity of the solution to the kind of observation error, two observation vectors are generated from the true state x^t: a first one by adding uncorrelated observation noise, and a second one by adding a correlated observation noise, with a covariance matrix given by (23), with transformation (46) (for various values of ℓ= σ₀/σ₁). The noise is scaled to have a uniform standard deviation σ = 0.04 m. To randomly draw Gaussian noise vectors with known covariance 𝗥, we use the method described in the appendix of Fukumori (2002). Figure 5 (top panels) shows an example of such noise vectors, generated for three correlation lengths: ℓ = 0, 5, and 15 grid points. (In this section, ℓ = 0 stands for uncorrelated noise.) The figure also shows the corresponding error on the right difference between adjacent grid points, illustrating how the observational error on the discrete gradient decreases with ℓ.

It is interesting to make the link between this simulated observational noise and the characteristics of observation error in real altimetry data. As explained at the beginning of section 2, observation error is always the sum of a measurement error and a representation error. On the one hand, the altimetric measurement is affected by several kinds of error (altimetric measure, orbit error, atmospheric correction error) with a bulk standard deviation of about 3–5 cm, and horizontal correlation patterns that can depend on the satellite orbit and on the state of the atmosphere. On the other hand, altimetric data are actually spatial averages over about 5–10 km along track, which is about a factor of 3 smaller than the resolution of our model. The resulting representation error, which corresponds to this limited range of spatial scales in the continuous ocean system, is thus here likely to remain small with respect to measurement errors. Consequently, the properties of our randomly simulated observational noise (4-cm standard deviation, with various correlation length scales) is chosen quite adequately to be in the range of what can be expected for real altimetric data in this region and for that kind of ocean model.

In addition, in order to increase the robustness of the test, each experiment is repeated by using, as true state, every snapshot of the sequence (one every 6 days for 5 yr), and the results are averaged over that ensemble of experiments. There is thus an ensemble of true states x_i^t, i = 1, …, N (with N = 300) and the corresponding ensemble of observations y_i sampled from them. (Observational errors are drawn independently for every member of the sample.) Hence, as soon as the ensemble of true states can be viewed as representative of all possible states of the system, our indicator gives the average error that is committed using the observation error parameterization that is being tested (starting from the mean as background state).

To parameterize the background (or forecast) error covariance matrix 𝗣_i^f, we use the covariance of all snapshots of the model simulation (sampled every month over the 5 yr of the simulation), except those that are less than 1 month away from the true state (larger than the typical decorrelation time scale), in order to avoid any influence of the true state on the input error covariance matrix (𝗣_i^f, is thus recomputed for every member i = 1, …, N). With an ensemble of about 60 independent realizations (one per month), it is only for correlations >0.26 that the 95% confidence interval for correlations (assuming normal pdfs) does not include zero. Correlations of <0.26 are thus not significant. Thus, if the size of the region is much larger than the spatial decorrelation scale, the integrated influence of distant observations with nonsignificant correlations can be as large as that of close observations with significant correlations. To avoid the spurious effect of these inaccurate long-range correlations (resulting from the use of a small size ensemble), we perform a separate local observational update [as in Brankart et al. (2003) or Testut et al. (2003)] for each water column, with an additional weight on the observations decreasing with the distance r as exp(−r ²/d²), with d = 200 km (the typical distance at which the correlation ceases to be significant). Figure 6 illustrates the resulting local structure of the background covariance that is used to perform the observational update. The figure shows the observational update increment that would result from one single observation (with 0.04-m error standard deviation), located at 9°N, 54°W (in the middle of the area traversed by the mesoscale rings). The long-range (nonsignificant) influence is effectively set to zero, without affecting much the local covariance structure described by the ensemble.

b. Uncorrelated errors

In experiment 1 (see Table 2), we use the observation vector that is perturbed by a white noise (observation errors are thus spatially uncorrelated) and the observation error covariance is parameterized using a diagonal matrix (𝗥 = σ²𝗜). The parameterization is thus fully consistent with the simulated errors. Figure 7 (top panels) shows a map of the ensemble standard deviation of errors (difference with respect to the true state) after the observational update corresponding to experiment 1. It is shown for altimetry (ϵ_ζ), for the altimetric gradient, and for velocity (ϵ_υ). It is computed as (the formula for the gradient is similar to the formula used for velocity)

View Expanded

where ζ_i^t, u_i^t, υ_i^t is the ith true state (corresponding to the ith snapshot of the model sequence), ζ^f,u^f,υ^f is the forecast state (corresponding to the mean of the model sequence), and δζ_i, δu_i, δυ_i is the ith observational update. This result can be directly compared with Fig. 4 (bottom panels), which represents the same quantity before the observational update. Indeed, since the background state is the mean state and since the ensemble of true state is the ensemble of all snapshots of the sequence, the standard deviation of the sequence is equal to the ensemble standard deviation of errors before the observational update; that is, (52) with δζ_i = 0, δu_i = 0, and δυ_i = 0. The comparison shows that the error on altimetry is significantly reduced by the observational update, becoming much smaller than both the background error standard deviation (Fig. 4, bottom panels) and the observational error standard deviation. This is because background errors are correlated over an area of about L × L, with L ∼ 125 km (see Fig. 6), including about L²/Δξ² ∼ 25 observations with uncorrelated errors. The resulting errors are thus about 1/ϵ_ζ² ∼ 1/σ_f² + 25/σ². Observations are dense and very accurate and therefore the background has only little influence and the resulting error is ϵ_ζ ∼ σ/5 = 0.008 m, a rough estimation that is quite consistent with the results is observed in Fig. 7. Filtering off a white noise is easy if the background error correlation scales (about L) are large with respect to typical data spacing (Δξ).

However, the error reduction factor (with respect to background error) is less favorable for the gradient of altimetry. This is because computing altimetric difference Δζ between adjacent model cells amplifies the relative errors. Relative errors on velocities are again slightly worse because the relation between surface velocity and altimetry is not perfectly geostrophic (and thus not perfectly linear).

On the other hand, the observational update of the error covariance is illustrated in Fig. 7 (bottom panels), showing the estimated error standard deviation (the square root of the diagonal of 𝗣^a). This estimation is quite consistent in amplitude and structure with the ensemble standard deviation of the error (measured by difference with respect to the true state), also shown in Fig. 7 (top panels). The good quality of the error estimate (for all variables) is the consequence of the consistent parameterization of the observation error covariance matrix; it also indicates that the background error covariance matrix (𝗣^f) is quite accurately parameterized.

c. Correlated errors, with diagonal 𝗥 parameterization

In experiment 2, we use the observation vector that is perturbed by the correlated noise (with ℓ = 5 grid points), but keep the same diagonal parameterization of the observation error covariance matrix (𝗥 = σ²𝗹). The parameterization is thus inconsistent with the simulated errors, which are assumed uncorrelated, even though they are not. Figure 8 shows the corresponding error maps to be compared with Fig. 7. The comparison shows that the error on altimetry is significantly larger than in experiment 1. This is because the number of independent observations in the L × L area is reduced to about (L/ℓ)² ∼ 1. Here, the background error keeps an influence, and the typical error is about ϵ_ζ ∼ 0.03 m, a rough estimation that is again quite consistent with the results observed in Fig. 8.

However, larger errors on altimetry do not necessarily mean larger errors on the altimetry gradient or larger errors on velocity. In experiment 2, the error increase on the gradient with respect to experiment 1 is actually smaller than the error increase on altimetry. This is because the gradient is better observed through correlated observations than through uncorrelated observations (see Fig. 5). Sticking the solution to correlated observations (even with inappropriate diagonal error parameterization, as in experiment 2) thus partly compensates the easiness of filtering off a white noise from a large-scale (L = 125 km) signal (with optimal parameterization, as in experiment 1). This better observation of the gradient is the reason why keeping all available observations in the observation vector (even if the errors are very correlated) is always a better solution than subsampling the observations.

On the other hand, the observational update of the error covariance is illustrated in Fig. 8 (bottom panels), showing the estimated error standard deviation (the square root of the diagonal of 𝗣^a). This estimation is identical to that of experiment 1 (Fig. 7, bottom panels), since all statistical parameters (𝗣^f,𝗥) are kept identical. This largely underestimates the standard deviation of the true error (measured using the ensemble of differences with respect to the true state), that is shown in the top panels of Fig. 8. The estimation is about a factor of 3 below reality. This situation is the consequence of the inconsistent parameterization of the observation error covariance matrix. The diagonal 𝗥 parameterization lets the scheme believe that the data are more accurate than they are, so that it underestimates the error that is effectively in the system.

d. Correlated errors, with consistent 𝗥 parameterization

In experiment 3, we use the same observation vector as in experiment 2 (perturbed by the correlated noise), but add gradient observations to simulate correlations in the observation error covariance matrix, with the diagonal covariance matrix (38). By choosing σ₀ = 0.275 m and σ₁ = σ₀/ℓ (for ℓ = 5 grid points, see Table 3), this observation error parameterization is thus perfectly consistent with the simulated errors. According to the theory presented in section 3, these observations with correlated errors are thus equivalent to much less accurate observations (σ₀ = 0.275 m), together with accurate observations of the gradient (σ₁ = 0.055 m per grid point). This is consistent with the idea suggested above that increasing ℓ reduces the number of independent observations. Figure 9 shows the corresponding error maps to be compared to Figs. 8 and 7. The comparison shows that the errors on altimetry are still larger than in experiment 1 (Fig. 7), because the quantity of information in the observation vector is still the same as in experiment 2, but the errors are smaller than in experiment 2 (Fig. 8), because the observation error parameterization is now consistent, so that the observational update is closer to optimality. (𝗣^f is still an approximation: it cannot be considered that the background error is drawn randomly from a pdf of covariance 𝗣^f.)

However, the error reduction (with respect to experiment 2) on altimetry gradient and velocity is significantly larger because the better confidence that we must give to the gradient is now explicitly taken into account in the observational update, through the nondiagonal parameterization of the observation error covariance matrix 𝗥. Better, this parameterization is effectively (and equivalently) applied in practice by the addition of gradient observations in the observation vector. (This addition of gradient observations can only bring the estimated gradient closer to the observed gradient, which is more accurate if the observation errors are spatially correlated.) The improvement of the gradient resulting from the parameterization of error correlations (if they exist) is thus clearly demonstrated by this experiment.

On the other hand, the observational update of the error covariance is illustrated in Fig. 9 (bottom panels), showing the estimated error standard deviation (the square root of the diagonal of 𝗣^a). As in experiment 1, it is consistent with the standard deviation of the true error (measured using the ensemble of differences with respect to the true state), which is shown in the top panels of Fig. 9. Again, this is due to the consistent parameterization of the observation error covariance matrix, which has been restored by the addition of gradient observations (with adequate values for σ₀ and σ₁).

e. Sensitivity to the correlation scale

In this last section, we examine how the results presented above depend on the observation error correlation length. For that purpose, the same experiment is repeated for various simulated observation noises, with a correlation length ℓ_o ranging from 0 to 10 grid points. And, for each of these simulated noises, several parameterizations of the observation error covariance matrix are tested, using a correlation length ℓ_p also ranging from 0 to 10 grid points. Figure 10 shows the resulting error standard deviation for sea surface height and velocity (as measured by the ensemble of differences with respect to the true states), averaged over the domain of interest, as a function of ℓ_o and ℓ_p. The figure also shows the ratio between the averaged estimated error and the averaged measured error. It is only if ℓ_o = ℓ_p that the parameterization is consistent with the simulated errors: it is thus along that line that the measured error should be minimum (for a given ℓ_o) and that the ratio between estimated and measured errors should be equal to 1.

The results show that underestimating the observation error correlation length scale (ℓ_p ≪ ℓ_o) leads to a moderate increase of the error on sea surface height, but to a very significant increase of the error on velocity. The estimation of the error standard deviation is also well below reality. This situation indeed corresponds to giving too much importance to the observations and to imposing too weak a constraint on the gradient. On the contrary, overestimating the observation error correlation length scale (ℓ_p ≫ ℓ_o) leads to a moderate increase of the error on velocity but to a significant increase of the error on sea surface height. The estimation of the error standard deviation is also well above reality for sea surface height, whereas no sensitivity can be observed for velocity. A correct tuning of ℓ_p is thus required to accurately estimate both variables, with consistent error estimates.

However it must be noted that in these experiments, the optimal parameterization is not on the line ℓ_o = ℓ_p as it should be, but noticeably below that value, especially for the large values of ℓ_o. The benefit obtained by giving to the observations more credit than they deserve (by using ℓ_p < ℓ_o) can only be explained by inaccuracies in the parameterization of the 𝗣^f matrix, which is here approximated by a limited size ensemble. Overestimating the confidence in the observations is thus somewhat useful here to compensate suboptimalities in the statistical parameterization of the scheme.