1. Introduction

Data assimilation addresses the problem of producing useful analyses and forecasts given imperfect dynamical models and observations. The Kalman filter is the optimal data assimilation method for linear dynamics with additive, state-independent Gaussian model and observation errors (Cohn 1997). An attractive feature of the Kalman filter is its calculation of forecast and analysis error covariances, in addition to the forecasts and analyses themselves. In this way, the Kalman filter produces estimates of forecast and analysis uncertainty, consistent with the dynamics and prescribed model and observation error statistics. However, the error covariance calculation components of the Kalman filter are difficult to implement in realistic systems because of (i) their computational cost, (ii) the nonlinearity of the dynamics, and (iii) poorly characterized error sources.

The ensemble Kalman filter (EnKF), proposed by Evensen (1994), addresses the first two of these problems by using ensemble representations for the forecast and analysis error covariances. Ensemble size limits the number of degrees of freedom used to represent forecast and analysis errors, and Kalman filter error covariance calculations are practical for modest-sized ensembles. The EnKF algorithm begins with an analysis ensemble whose mean is the current state estimate or analysis and whose statistics reflect the analysis error. Applying the full nonlinear dynamics to each analysis ensemble member produces the forecast ensemble; tangent linear and adjoint models of the dynamics are not required. Statistics of the forecast ensemble represent forecast errors; in its simplest form, the EnKF only accounts for forecast error due to uncertain initial conditions, neglecting forecast error due to model deficiencies. The forecast ensemble mean and covariance are then used to assimilate observations and compute a new analysis ensemble with appropriate statistics, and the cycle is repeated. The new analysis ensemble can be formed either stochastically (Houtekamer and Mitchell 1998; Burgers et al. 1998) or deterministically (Bishop et al. 2001; Anderson 2001; Whitaker and Hamill 2002). Deterministic methods were developed to address the adaptive observational network design problem and to avoid sampling issues associated with the use of “perturbed observations” in stochastic analysis ensemble update methods.

The EnKF and other ensemble data assimilation methods belong to the family of square root filters (SRFs), and a purpose of this paper is to demonstrate that deterministic analysis ensemble updates are implementations of Kalman SRFs (Bierman 1977; Maybeck 1982; Heemink et al. 2001). An immediate benefit of this identification is a framework for understanding and comparing deterministic analysis ensemble update schemes (Bishop et al. 2001; Anderson 2001; Whitaker and Hamill 2002). SRFs, like ensemble representations of covariances, are not unique. We begin our discussion in section 2 with a presentation of the Kalman SRF; issues related to implementation of ensemble SRFs are presented in section 3; in section 4 we summarize our results.

2. The Kalman SRF

Kalman SRF algorithms, originally developed for space-navigation systems with limited computational word length, demonstrate superior numerical precision and stability compared to the standard Kalman filter algorithm (Bierman 1977; Maybeck 1982). SRFs by construction avoid loss of positive definiteness of the error covariance matrices. SRFs have been used in earth science data assimilation methods where error covariances are approximated by truncated eigenvector expansions (Verlaan and Heemink 1997).

The usual Kalman filter covariance evolution equations are

View Expanded

where 𝗣^f_k and 𝗣^a_k are, respectively, the n × n forecast and analysis error covariance matrices at time t_k; 𝗠_k is the tangent linear dynamics; 𝗛_k is the p × n observation operator; 𝗥_k is the p × p observation error covariance; 𝗤_k is the n × n model error covariance matrix and 𝗞_k ≡ 𝗣^f_k𝗛^T_k(𝗛_k𝗣^f_k𝗛^T_k + 𝗥_k)⁻¹ is the Kalman gain; n is the dimension of the system state; and p is the number of observations. The error covariance evolution depends on the state estimates and observations through the tangent linear dynamics 𝗠_k. The propagation of analysis errors by the dynamics with model error acting as a forcing is described by Eq. (1). Equation (2) shows how an optimal data assimilation scheme uses observations to produce an analysis whose error covariance is less than that of the forecast.

The forecast and analysis error covariance matrices are symmetric positive-definite matrices and can be represented as 𝗣^f_k = 𝗭^f_k𝗭^fT_k and 𝗣^a_k = 𝗭^a_k𝗭^aT_k, where the matrices 𝗭^f_k and 𝗭^a_k are matrix square roots of 𝗣^f_k and 𝗣^a_k, respectively; other matrix factorizations can be used in filters as well (Bierman 1977; Pham et al. 1998). A covariance matrix and its matrix square root have the same rank or number of nonzero singular values. When a covariance matrix 𝗣 has rank m, there is an n × m matrix square root 𝗭 satisfying 𝗣 = 𝗭𝗭^T; in low-rank covariance representations the rank m is much less than the state-space dimension n. This representation is not unique; 𝗣 can also be represented as 𝗣 = (𝗭𝗨)(𝗭𝗨)^T, where the matrix 𝗨 is any m × m orthogonal transformation 𝗨𝗨^T = 𝗨^T𝗨 = 𝗜. The projection ‖x^T𝗭‖² = x^T𝗣x of an arbitrary n-vector x onto the matrix square root 𝗭 is uniquely determined, as is the subspace spanned by the columns of 𝗭.

Covariance matrix square roots are closely related to ensemble representations. The sample covariance 𝗣^a_k of an m-member analysis ensemble is given by 𝗣^a_k = 𝗦𝗦^T/(m − 1), where the columns of the n × m matrix 𝗦 are mean-zero analysis perturbations about the analysis ensemble mean; the rank of 𝗣^a_k is at most (m − 1). A matrix square root of the analysis error covariance matrix 𝗣^a_k is the matrix of scaled analysis perturbation ensemble members 𝗭^a_k = (m − 1)^−1/2𝗦.

Next, analysis error covariance equation (2) is replaced with an equation for the analysis error covariance square root 𝗭^a_k. This equation determines how to form an analysis ensemble with appropriate statistics. Initial implementations of the EnKF formed the new analysis ensemble by updating each forecast ensemble member using the same analysis equations, equivalent to applying the linear operator (𝗜 − 𝗞_k𝗛_k) to the forecast perturbation ensemble 𝗭^f_k. This procedure gives an analysis ensemble whose error covariance is (𝗜 − 𝗞_k𝗛_k)𝗣^f_k(𝗜 − 𝗞_k𝗛_k)^T and includes analysis error due to forecast error; the Kalman gain 𝗞_k depends on the relative size of forecast and observation error, and the factor (𝗜 − 𝗞_k𝗛_k) shows how much forecast errors are reduced. However, in this procedure the analysis ensemble does not include uncertainty due to observation error and so underestimates analysis error. A stochastic solution to this problem proposed independently by Houtekamer and Mitchell (1998) and Burgers et al. (1998) is to compute analyses using each forecast ensemble member and, instead of using a single realization of the observations, to use an ensemble of simulated observations whose statistics reflect the observation error. This method is equivalent to the analysis perturbation ensemble update

^a_kkk^f_kkk(4)

where 𝗪_k is a p × m matrix whose m columns are identically distributed, mean-zero, Gaussian random vectors of length p with covariance 𝗥_k/m. The perturbed observation analysis equation (4) gives an analysis perturbation ensemble with correct expected statistics:

View Expanded

However, the perturbed observation approach introduces an additional source of sampling error that reduces analysis error covariance accuracy and increases the probability of underestimating analysis error covariance (Whitaker and Hamill 2002). A Monte Carlo method avoiding perturbed observations is described in Pham (2001). The singular evolutive interpolate Kalman (SEIK) filter uses both deterministic factorization and stochastic approaches.

Kalman SRFs provide a deterministic algorithm for transforming the forecast ensemble into an analysis ensemble with consistent statistics. The “Potter method” for the Kalman SRF analysis update (Bierman 1977) is obtained by rewriting (2) as

View Expanded

where we define the m × p matrix 𝗩_k ≡ (𝗛_k𝗭^f_k)^T and the p × p innovation covariance matrix 𝗗_k ≡ 𝗩^T_k𝗩_k + 𝗥_k. Then the analysis perturbation ensemble is calculated from

^a_k^f_kkk(7)

where 𝗫_k𝗫^T_k = (𝗜 − 𝗩_k𝗗⁻¹_k𝗩^T_k) and 𝗨_k is an arbitrary m × m orthogonal matrix. As formulated, the updated ensemble 𝗭^a_k is a linear combination of the columns of 𝗭^f_k and is obtained by inverting the p × p matrix 𝗗_k and computing a matrix square root 𝗫_k of the m × m matrix (𝗜 − 𝗩_k𝗗⁻¹_k𝗩^T_k).

3. Ensemble SRFs

a. Analysis ensemble

In many typical earth science data assimilation applications the state-dimension n and the number of observations p are large, and the method for computing the matrix square root of (𝗜 − 𝗩_k𝗗⁻¹_k𝗩^T_k) and the updated analysis perturbation ensemble 𝗭^a_k must be chosen accordingly. A direct approach is to solve first the linear system 𝗗_k𝗬_k = 𝗛_k𝗭^f_k for the p × m matrix 𝗬_k, that is, to solve

_k^f_k ^T_kkkk^f_k(8)

as is done in the first step of the Physical-space Statistical Analysis System (PSAS) algorithm (Cohn et al. 1998). Then, the m × m matrix 𝗜 − 𝗩_k𝗗⁻¹_k𝗩^T_k = 𝗜 − (𝗛_k𝗭^f_k)^T𝗬_k is formed, and its matrix square root 𝗫_k computed and applied to 𝗭^f_k as in (7). Solution of (8), even when p is large, is practical when the forecast error covariance has a low-rank representation and the inverse of the observation error covariance is available (see the appendix). Iterative methods whose cost is on the order of the cost of applying the innovation covariance matrix are appropriate when the forecast error covariance is represented by a correlation model.

When observation errors are uncorrelated, observations can be assimilated one at a time or serially (Houtekamer and Mitchell 2001; Bishop et al. 2001). For a single observation, p = 1, 𝗩_k is a column vector, and the innovation 𝗗_k is a scalar. In this case, a matrix square root of (𝗜 − 𝗩_k𝗗⁻¹_k𝗩^T_k) can be computed in closed form by taking the ansatz,

⁻¹_kk ^T_kβ_kk ^T_kβ_kk ^T_k^T(9)

and solving for the scalar β_k, which gives β_k = [𝗗_k ± (𝗥_k𝗗_k)^1/2]⁻¹. The analysis ensemble update for p = 1 is

^a_k^f_kβ_kk ^T_k(10)

see Andrews (1968) for a general solution involving matrix square roots of p × p matrices. At observation locations, the analysis error ensemble is related to the forecast error ensemble by 𝗛_k𝗭^a_k = (1 − β_k𝗩^T_k𝗩_k)𝗛_k𝗭^f_k. The scalar factor (1 − β_k𝗩^T_k𝗩_k) has an absolute value less than or equal to one and is positive when the plus sign is chosen in the definition of β_k.

In Whitaker and Hamill (2002) the analysis perturbation ensemble is found from

^a_kK̃_kk^f_k(11)

where the matrix K̃_k is a solution of the nonlinear equation

K̃_kk^f_kK̃_kk^Ta_k(12)

In the case of a single observation, a solution of (12) is

K̃_kkk^1/2−1_kβ_k^f_kk(13)

where the plus sign is chosen in the definition of β_k. The corresponding analysis perturbation ensemble update,

View Expanded

is identical to (10). Observations with correlated errors, for example, radiosonde height observations from the same sounding, can be handled by applying the whitening transformation 𝗥^−1/2_k to the observations to form a new observation set with uncorrelated errors.

Another method of computing the updated analysis ensemble is to use the Sherman–Morrison–Woodbury identity (Golub and Van Loan 1996) to show that

_k ⁻¹_k ^T_k^fT_k ^T_k ⁻¹_kk^f_k⁻¹(15)

The m × m matrix on the right-hand side of (15) is practical to compute when the inverse observation error covariance matrix 𝗥⁻¹_k is available. This approach avoids inverting the p × p matrix 𝗗_k and is used in the ensemble transform Kalman filter (ETKF) where the analysis update is (Bishop et al. 2001)

^a_k^f_kkΓ_k^−1/2(16)

𝗖_kΓ_k𝗖^T_k is the eigenvalue decomposition of 𝗭^fT_k𝗛^T_k𝗥⁻¹_k𝗛_k𝗭^f_k. Note that the matrix 𝗖_k of orthonormal eigenvectors is not uniquely determined.¹ Comparison with (15) shows that 𝗖_k(Γ_k + 𝗜)⁻¹𝗖^T_k is the eigenvalue decomposition of 𝗜 − 𝗩_k𝗗⁻¹_k𝗩^T_k and thus that 𝗖_k(Γ_k + 𝗜)^−1/2 is a square root of (𝗜 − 𝗩_k𝗗⁻¹_k𝗩^T_k).

In the ensemble adjustment Kalman filter (EAKF) the form of the analysis ensemble update is (Anderson 2001)

^a_kk^f_k(17)

the ensemble adjustment matrix 𝗔_k is defined by

_kkkC̃_kΓ̃_k^{−1/2 −1}_k ^T_k(18)

where 𝗣^f_k = 𝗙_k𝗚²_k𝗙^T_k is the eigenvalue decomposition of 𝗣^f_k and the orthogonal matrix C̃_k is chosen so that C̃^T_k𝗚_k𝗙^T_k𝗛^T_k𝗥⁻¹_k𝗛_k𝗙_k𝗚_kC̃_k = Γ̃_k is diagonal.² Choosing the orthogonal matrix C̃_k to be C̃_k = 𝗚⁻¹_k𝗙^T_k𝗭^f_k𝗖_k gives that Γ̃_k = Γ_k and that the ensemble adjustment matrix is

_k^f_kkΓ_k^{−1/2 −1}_k ^T_k(19)

The EAKF analysis update (17) becomes

^a_k^f_kkΓ_k^{−1/2 −1}_k ^T_k^f_k(20)

The EAKF analysis ensemble given by (20) is the same as applying the transformation 𝗚⁻¹_k𝗙^T_k𝗭^f_k to the ETKF analysis ensemble. The matrix 𝗚⁻¹_k𝗙^T_k𝗭^f_k is orthogonal and is, in fact, the matrix of right singular vectors of 𝗭^f_k. Therefore, 𝗖_k(𝗜 + Γ_k)^−1/2𝗚⁻¹_k𝗙^T_k𝗭^f_k is a matrix square root of (𝗜 − 𝗩_k𝗗⁻¹_k𝗩^T_k).

Beginning with the same forecast error covariance, the direct, serial, ETKF, and EAKF methods produce different analysis ensembles that span the same state-space subspace and have the same covariance. Higher-order statistical moments of the different models will be different, a relevant issue for nonlinear dynamics. The computation costs of the direct, ETKF, and EAKF methods are seen in Table 1 to scale comparably (see the appendix for details). There are differences in precise computational cost; for instance, the EAKF contains an additional singular value decomposition (SVD) calculation of the forecast with cost O(m³ + m²). The computational cost of the serial filter is less dependent on the rank of the forecast error covariance and more sensitive to the number of observations. This difference is important when techniques to account for model error and control filter divergence, as described in the next section, result in an effective forecast error covariance dimension m much larger than the dynamical forecast ensemble dimension.

4. Summary and discussion

Ensemble forecast/assimilation methods use low-rank ensemble representations of forecast and analysis error covariance matrices. These ensembles are scaled matrix square roots of the error covariance matrices, and so ensemble data assimilation methods can be viewed as square root filters (SRFs; Bierman 1977). After assimilation of observations, the analysis ensemble can be constructed stochastically or deterministically. Deterministic construction of the analysis ensemble eliminates one source of sampling error and leads to deterministic SRFs being more accurate than stochastic SRFs in some examples (Whitaker and Hamill 2002; Anderson 2001). SRFs are not unique since different ensembles can have the same covariance. This lack of uniqueness is illustrated in three recently proposed ensemble data assimilation methods that use the Kalman SRF method to update the analysis ensemble (Bishop et al. 2001; Anderson 2001; Whitaker and Hamill 2002). Identifying the methods as SRFs allows a clearer discussion and comparison of their different analysis ensemble updates.

Accounting for small ensemble size and model deficiencies remains a significant issue in ensemble data assimilation systems. Schur products can be used to filter ensemble covariances and effectively increase covariance rank (Houtekamer and Mitchell 1998, 2001; Hamill et al. 2001; Whitaker and Hamill 2002). Covariance inflation is one simple way of accounting for model error and stabilizing the filter (Hamill et al. 2001; Anderson 2001; Whitaker and Hamill 2002). Hybrid methods represent forecast error covariances with a combination of ensemble and parameterized correlation models (Hamill and Snyder 2000). Here we have shown deterministic methods of including model error into a square root or ensemble data assimilation system when the model error has large-scale representation and when the model error is represented by a correlation model. However, the primary difficulty remains obtaining estimates of model error.

The nonuniqueness of SRFs has been exploited in estimation theory to design filters with desirable computational and numerical properties. An open question is whether there are ensemble properties that would make a particular SRF implementation better than another, or if the only issue is computational cost. For instance, it may be possible to choose an analysis update scheme that preserves higher-order, non-Gaussian statistics of the forecast ensemble. This question can only be answered by detailed comparisons of different methods in a realistic setting where other details of the assimilation system such as modeling of systematic errors or data quality control may prove to be as important.