• Bishop, C. H., and D. Hodyss, 2007: Flow adaptive moderation of spurious ensemble correlations and its use in ensemble based data assimilation. Quart. J. Roy. Meteor. Soc., 133 , 2029–2044.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., and D. Hodyss, 2009a: Ensemble covariances adaptively localized with ECO-RAP. Part 1: Tests on simple error models. Tellus, 61 , 84–96.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., and D. Hodyss, 2009b: Ensemble covariances adaptively localized with ECO-RAP. Part 2: A strategy for the atmosphere. Tellus, 61 , 97–111.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., 2005: Ensemble-derived stationary and flow-dependent background error covariances: Evaluation in a quasi-operational NWP setting. Quart. J. Roy. Meteor. Soc., 131 , 1013–1043.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010a: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part I: Description and single-observation experiments. Mon. Wea. Rev., 138 , 1550–1566.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010b: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part II: One-month experiments with real observations. Mon. Wea. Rev., 138 , 1567–1586.

    • Search Google Scholar
    • Export Citation
  • Dee, D. P., 1991: Simplification of the Kalman filter for meteorological data assimilation. Quart. J. Roy. Meteor. Soc., 117 , 365–384.

    • Search Google Scholar
    • Export Citation
  • El Akkraoui, A., P. Gauthier, S. Pellerin, and S. Buis, 2008: Intercomparison of the primal and dual formulations of variational data assimilation. Quart. J. Roy. Meteor. Soc., 134 , 1015–1025.

    • Search Google Scholar
    • Export Citation
  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for a spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230 , 112–126.

    • Search Google Scholar
    • Export Citation
  • Kepert, J. D., 2009: Covariance localisation and balance in an Ensemble Kalman Filter. Quart. J. Roy. Meteor. Soc., 135 , 1157–1176.

    • Search Google Scholar
    • Export Citation
  • Liu, C., Q. Xiao, and B. Wang, 2009: An ensemble-based four-dimensional variational data assimilation scheme. Part II: Observing system simulation experiments with Advanced Research WRF (ARW). Mon. Wea. Rev., 137 , 1687–1704.

    • Search Google Scholar
    • Export Citation
  • Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWPβ€”A comparison with 4D-VAR. Quart. J. Roy. Meteor. Soc., 129 , 3183–3203.

    • Search Google Scholar
    • Export Citation
  • Miyoshi, T., and S. Yamane, 2007: Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution. Mon. Wea. Rev., 135 , 3841–3861.

    • Search Google Scholar
    • Export Citation
  • Parrish, D. F., and J. C. Derber, 1992: The National Meteorological Center’s spectral statistical-interpolation analysis system. Mon. Wea. Rev., 120 , 1747–1763.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., E. J. Kostelich, G. Gyarmati, E. Kalnay, B. R. Hunt, E. Ott, E. Satterfield, and J. A. York, 2008: A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A , 113–130.

    • Search Google Scholar
    • Export Citation
  • Wang, X., C. Snyder, and T. M. Hamill, 2007: On the theoretical equivalence of differently proposed ensemble–3DVAR hybrid analysis schemes. Mon. Wea. Rev., 135 , 222–227.

    • Search Google Scholar
    • Export Citation
  • Yang, S. C., E. Kalnay, B. R. Hunt, and N. E. Bowler, 2009: Weight interpolation for efficient data assimilation with the Local Ensemble Transform Kalman Filter. Quart. J. Roy. Meteor. Soc., 135 , 251–262.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 74 70 2
PDF Downloads 52 50 3

Efficient Ensemble Covariance Localization in Variational Data Assimilation

View More View Less
  • 1 Naval Research Laboratory, Monterey, California
  • | 2 Bureau of Meteorology, Melbourne, Australia
  • | 3 Met Office, Exeter, United Kingdom
  • | 4 Meteorological Research Division, Environment Canada, Dorval, Canada
Β© Get Permissions
Full access

Abstract

Previous descriptions of how localized ensemble covariances can be incorporated into variational (VAR) data assimilation (DA) schemes provide few clues as to how this might be done in an efficient way. This article serves to remedy this hiatus in the literature by deriving a computationally efficient algorithm for using nonadaptively localized four-dimensional (4D) or three-dimensional (3D) ensemble covariances in variational DA. The algorithm provides computational advantages whenever (i) the localization function is a separable product of a function of the horizontal coordinate and a function of the vertical coordinate, (ii) and/or the localization length scale is much larger than the model grid spacing, (iii) and/or there are many variable types associated with each grid point, (iv) and/or 4D ensemble covariances are employed.

Corresponding author address: Craig H. Bishop, Naval Research Laboratory, Marine Meteorology Division, 7 Grace Hopper Ave., Stop 2, Building 702, Room 212, Monterey, CA 93943-5502. Email: bishop@nrlmry.navy.mil

This article included in the Intercomparisons of 4D-Variational Assimilation and the Ensemble Kalman Filter special collection.

Abstract

Previous descriptions of how localized ensemble covariances can be incorporated into variational (VAR) data assimilation (DA) schemes provide few clues as to how this might be done in an efficient way. This article serves to remedy this hiatus in the literature by deriving a computationally efficient algorithm for using nonadaptively localized four-dimensional (4D) or three-dimensional (3D) ensemble covariances in variational DA. The algorithm provides computational advantages whenever (i) the localization function is a separable product of a function of the horizontal coordinate and a function of the vertical coordinate, (ii) and/or the localization length scale is much larger than the model grid spacing, (iii) and/or there are many variable types associated with each grid point, (iv) and/or 4D ensemble covariances are employed.

Corresponding author address: Craig H. Bishop, Naval Research Laboratory, Marine Meteorology Division, 7 Grace Hopper Ave., Stop 2, Building 702, Room 212, Monterey, CA 93943-5502. Email: bishop@nrlmry.navy.mil

This article included in the Intercomparisons of 4D-Variational Assimilation and the Ensemble Kalman Filter special collection.

1. Introduction

Work by Lorenc (2003), Buehner (2005), Wang et al. (2007), Buehner et al. (2010a,b), Liu et al. (2009), and others indicate a growing interest in using localized ensemble covariances in three- and four-dimensional variational data assimilation (3D-VAR and 4D-VAR, respectively) schemes. Buehner et al. (2010a,b) found that a 4D-VAR scheme that used localized 3D ensemble covariances for a background error covariance matrix and a tangent linear model (TLM) and adjoint, outperformed versions of operational 4D-VAR and ensemble Kalman filter (EnKF) data assimilation (DA) schemes. They also showed that an ensemble-4D-VAR scheme that only used localized 4D ensemble covariances and did not use a TLM or adjoint, outperformed a version of the operational 4D-VAR scheme both in the tropics and Southern Hemisphere, but not in the northern extratropics. They also mention that, without including the cost of generating the ensemble, ensemble-4D-Var could be performed at a fraction of the cost of a conventional 4D-Var scheme. However, the descriptions of how ensemble covariances can be incorporated into variational schemes given in Lorenc (2003), Buehner (2005), Wang et al. (2007), and Buehner et al. (2010a,b) give scant details or understanding of the factors that influence efficiency. This paper provides understanding vital to the design and implementation of fast ensemble covariance localization algorithms for variational DA. The fast algorithm derived pertains to Buehner et al’s (2010a,b) ensemble-4D-VAR in which the 4D ensemble covariances are nonadaptively localized in space alone, with no localization in time or modulation of covariances between variable types. The fast algorithm draws heavily from techniques that have been developed for nonensemble variational DA.

Variational DA algorithms perform a global minimization of a penalty function containing a background or prior term written in terms of a background error covariance matrix 𝗣0f. This is a nonsparse matrix defining the error covariance between every pair of points and variables in a forecast model, which for modern NWP systems may have a billion degrees of freedom. Not only is it computationally impossible to store and manipulate matrices of this size, but it is also impossible to know them (Dee 1991): there is not enough information in all the forecasts made in the lifetime of an NWP system to accurately estimate even a static covariance. Another problem is that the poor conditioning of the covariance prevents efficient minimization. In 4D-VAR algorithms, these problems are addressed together, by representing 𝗣0f in terms of a square root that is implied by a sequence of simple transformations. These simple transformations give the necessary cost savings and, just as importantly, can impart covariances based on a physical understanding of the dynamics governing forecast error. The transformations define the covariance, rather than being chosen to fit a known or hypothesized covariance. Traditional variational assimilation methods use a parameter transform to balanced and unbalanced variables that are expected to be uncorrelated, together with vertical and horizontal transforms that model the three-dimensional correlation structure in each of the transformed variables (e.g., Parrish and Derber 1992).

In traditional variational methods, the designer of the background error covariance matrix selects transform operators that implicitly define the background error covariance matrix. In contrast, designers of ensemble covariance localization methods such as Bishop and Hodyss (2007, 2009a,b) explicitly design localization functions with which to multiply explicit ensemble covariance functions. Apart from Kepert’s (2009) sophisticated formulation of balance constraints for ensemble Kalman filters, expressions for localized ensemble covariance matrices are, in general, trivial to write down in matrix form. In contrast, the background error covariance matrix implied by the sequence of transformations and balance operators that define the background error covariance matrices of traditional variational methods are nontrivial to write down in matrix form. This note shows how the easily written down matrix vector product associated with a nonadaptively localized ensemble covariance matrix can be reduced to a sequence of cost-efficient transform operators similar to those used in variational DA schemes.

Section 2 shows how a straightforward coding of the matrix-vector multiplications would be extremely expensive for forecast error covariance matrices based on localized ensemble covariance matrices. Section 3 derives a fast algorithm for performing this multiplication. Conclusions follow in section 4.

2. The problem

Consider the localized 4D ensemble covariance matrix given by π—£β€Šf = 𝗣Kf oΜ‡ , where oΜ‡ indicates the element-wise matrix product, and = 1/21/2T is the correlation matrix used to localize the ensemble covariance matrix , where xi is the ith ensemble perturbation (ensemble member minus the ensemble mean) outputted at a series of nq times across the data assimilation time window, and 1/2 is the left square root of .

As shown in the appendix and as discussed in Buehner (2005) and Bishop and Hodyss (2009b), the square root of 𝗣Kf oΜ‡ is given by [diag(x1)1/2, diag(x2)1/2, … , diag(xK)1/2], where diag(xi) is the diagonal matrix whose diagonal is composed of the elements of xi.

Variational algorithms that minimize a measure of the distance of the estimated state from observations and the first guess require the repeated evaluation of terms like
i1520-0493-139-2-573-e1
and
i1520-0493-139-2-573-e2
Here fi has as many elements as there are columns in 1/2 while h has as many elements as the 4D state vector. The meaning of the vectors fi and h and the order in which the operations (1) and (2) are performed depends on whether the primal or dual forms of variational data assimilation are being usedβ€”see El Akkraoui et al. (2008) for details. For the purposes of this paper, fi and h are simply symbols that denote the vector part of the matrix vector products defined by (1) and (2). The symbol pi will be used as shorthand for [diag(xi)1/2]Th.

If the sole component of the forecast error covariance matrix is localized ensemble covariances, the analysis correction xa βˆ’ xf takes the following form: . In other words, the analysis correction is a weighted sum of the ensemble perturbations xi. The vector of weights for the ith ensemble perturbation is given by the vector (1/2fi). The less the values in the vector (1/2fi) vary with space, time, and variable type, the more the global analysis approximates a simple linear combination of global ensemble perturbations. The more the values in the vector (1/2fi) vary with space, time, and variable type, the freer the global analysis is to fit observations. Here, however, we consider the nonadaptive, purely spatial localization used with Buehner et al’s (2010a,b) ensemble-4D-VAR scheme. In this scheme, the values of (1/2fi) depend only on the spatial location of the grid point to which they pertain and are independent of time and variable type. The scale of the correlation functions defining determine the penalty associated with allowing the ensemble weights to vary. Broadscale correlation functions in only permit broadscale fluctuations of the weights while short-scale correlation functions allow shorter-scale fluctuations of the weights. In this way, the localization correlation matrix controls the spatial variation of the ensemble weights defining the analysis correction. In contrast, in the local ensemble transform Kalman filter the variation of ensemble weights is controlled by the assigned sizes of observation volumes and distance-dependent observation error variance inflation (Hunt et al. 2007; Miyoshi and Yamane 2007; Szunyogh et al. 2008; Yang et al. 2009).

We assume that the columns of 1/2 are separable functions; that is, each column is the product of a member of a set of basis functions of the horizontal coordinate and a member of a set of basis functions of the vertical coordinate. Under this assumption, the number of columns nc in 1/2 is given by the product of the number of horizontal basis functions Lh and the number of vertical basis functions M; in other words, nc = MLh. We will also assume that the model has Γ±h horizontal grid points and Γ±z vertical levels. For the sake of simplicity, we shall assume that the number of variable types at each model grid point is equal to the constant nΟ–. The total number of variables is then Γ±T = Γ±hΓ±znqnΟ– (recall that nq is the number of time levels). The cost of computing (nonsparse) matrix-vector products like those in (1) and (2) is order MLhΓ±T operations. Assuming [Γ±h, Γ±z, nΟ–, nq] ∼ O[106, 102, 10, 10] so that Γ±T ∼ O(1010), and assuming that [Lh, M] ∼ O[105, 10] so that MLh ∼ 106, it follows that MLhΓ±T ∼ O(1016) operations would be required for the matrix vector products in (1) and (2). This study shows how this cost can be reduced by many orders of magnitude.

3. Derivation of a rapid algorithm for ensemble covariance localization

A first step to a rapid algorithm is to recognize that parts of it can be performed on a reduced resolution grid. Consider a coarse-resolution grid that only has nh Γ— nz grid points giving a total of just nT = nqnhnznΟ– variables instead of the Γ±h Γ— Γ±z grid points and Γ±T variables of the high-resolution grid. Assume that the coarse-resolution grid has nh = Γ±h/u grid points in the horizontal and nz = Γ±z/s grid points in the vertical, where u and s are scalars.1 Let π—Ÿ be a Γ±T Γ— nT sparse rectangular matrix operator that interpolates coarse-resolution nT vectors to high-resolution Γ±T vectors. When the horizontal and vertical length scales of the functions that describe the columns of 1/2 are broad, 1/2 β‰ˆ π—Ÿπ—–1/2, and (1) and (2) can be approximated by
i1520-0493-139-2-573-e3
and
i1520-0493-139-2-573-e4
Notice that for the sake of conciseness, in (3) and (4) and throughout the rest of the paper, we drop the ensemble index subscript i from f, x, and p (except in section a of the appendix where it is needed again). The cost of mapping from high- to low-resolution grids and vice versa is order Γ±T operations. Without any further simplification, the use of the coarse grid reduces the cost of (1) and (2) to order MLhΓ±T/(su) operations. However, as we will see below, there are even more profound gains in computational cost that can be obtained by judicious use of the separable nature of the functions that define the columns of 𝗖1/2.
When each column of 𝗖1/2 is proportional to a product of vertical and horizontal structure functions, we can use the modulation product with symbol β€œβ€β€”defined in the appendix and in Bishop and Hodyss (2009b)β€”to write
i1520-0493-139-2-573-e5
where each column of the nT Γ— Lh matrix π—ͺh consists of a single horizontal structure (such as a scaled spherical harmonic in a global model) replicated at all vertical levels, all time levels, and all of the nΟ– variable types. Each column of the nT Γ— M matrix π—ͺΟ… consists of a single vertical structure (such as a scaled eigenvector of a vertical correlation matrix) that is the same for all variable types and has no horizontal or temporal variation. In other words, the single vertical structure is replicated in the horizontal and through time for all variables. According to the definition of the modulation product given in the appendix, (5) means that 𝗖1/2 has nc = LhM columns.
To define π—ͺh algebraically, let π—₯h be a nT Γ— nh replication matrix and let π—š be a nh Γ— Lh matrix listing the Lh horizontal basis functions defining a square root of the horizontal correlation matrix associated with 𝗖1/2. We then have
i1520-0493-139-2-573-e6
where the replication matrix π—₯h is defined by the submatrices for all vertical levels m = 1, 2, … , nz, all times q = 1, 2, … , nq, and all variable types r = 1, 2, … , nΟ–, where π—œ is the nh Γ— nh identity matrix. Note that in (6), we have assumed that the nhnznqnΟ– elements of the state vector have been listed such that the first nh elements of the state vector correspond to the horizontal field defining the variation of the first variable at the first time at model level 1. The second nh elements of the state vector correspond to the horizontal field defining the variation of the first variable at model level 2 and so on, until all nhnz variables defining the 3D state of the first variable at the first time level have been defined. The next nhnz variables define the state at the second time level until all nhnznq variables have been listed that define the 4D state of the first variable. The next nhnznq define the 4D state of the second model variable, and so on until all nhnznqnΟ– elements defining the 4D state of all model variables have been listed.
Here π—ͺΟ… is completely analogous to π—ͺh, but its algebraic form changes to accommodate the noncontiguous location of the vertical indices. To define π—ͺΟ… algebraically, let the nz Γ— M matrix 𝗩 = [v1, v2, … , vj, … , vM] be the square root of the vertical correlation matrix associated with 𝗖1/2. The jth column of the nznh Γ— M matrix
i1520-0493-139-2-573-e7
then gives a 3D function with vertical columns equal to vj and no horizontal variation. It then remains to replicate this 3D structure function for all variable types and all output times using the replication matrix π—₯Ο… defined by
i1520-0493-139-2-573-e8
Using (5) in the RHS of (3) gives
i1520-0493-139-2-573-e9
The elements of f are coefficients of the columns of π—ͺΟ… π—ͺh. Hence, [π—ͺΟ… π—ͺh]f is a mapping from coefficient space to physical space. In section b of the appendix, we prove that , where fjT is a Lh vector and fT = [f1T, f2T, … , fjT, … , fMT]. Using this, (6) and (8) in (9) gives
i1520-0493-139-2-573-e10
To rephrase (10) in a form that avoids the explicit use of the replication matrices π—₯Ο… and π—₯h, let denote a vertical column of the vector corresponding to the rth variable type, qth time level, and lth horizontal location. One may then deduce from (10) and (6)–(9) that
i1520-0493-139-2-573-e11
where glT is the lth row of π—š.

The following list of tasks enables x oΜ‡ [π—Ÿ(𝗖1/2f)] to be computed in a computationally efficient way. Each task is associated with an operator or subroutine identified with capitalized script letter.

  1. Use the operator h to compute π—šfj for j = 1, 2, … , M at a cost of order nhLhM operations. Note that the operation π—šfj returns the scalar value glTfj at the lth horizontal grid column for l = 1, 2, … , nh, ready for use in (11).

  2. Use the operator z to compute the right-hand side of (11) for all horizontal grid points. This operation provides the ensemble weight for the ith ensemble perturbation at each grid point and costs order nhnzM operations.

  3. Use the operator tr to replicate the weights at each grid point for all variable types and all output times to obtain the full coarse resolution multivariate weight nT vector (𝗖1/2f) at a cost of order nqnhnΟ‰nz = nT operations.

  4. Use the operator to interpolate the weight vector from the coarse resolution grid to the high-resolution grid (using π—Ÿ) and perform the element-wise product x oΜ‡ [π—Ÿ(𝗖1/2f)] at a cost of order Γ±T operations.

In operator form, the algorithm is x oΜ‡ [π—Ÿ(𝗖1/2f)] ≑ rtzπ—ͺh(f). Using parameter values from section 2, and assuming that the coarse resolution domain has about an order of magnitude fewer horizontal grid points and about the same order of magnitude of vertical grid points gives u ∼ O(10) and s ∼ O(1), its cost is order nhLhM + nhnzM + nT + Γ±T ∼ O(1011 + 108 + 109 + 1010) ∼ O(1011) operations. This is five orders of magnitude fewer operations than the approach described in section 2.

It now remains to describe how similar approaches can be used to speed the adjoint operation defined by (4). Defining y = π—ŸT(x oΜ‡ h) and using (5) in (4) gives
i1520-0493-139-2-573-e12
Section c of the appendix shows that
i1520-0493-139-2-573-e13
Now from the definition of π—ͺh given in (6):
i1520-0493-139-2-573-e14
where ymqr and are each nh vectors describing horizontal fields of the qth model variable type on the mth vertical level at the qth time level. To see the connection between (14) and vertical functions 𝗩, let denote the lth horizontal element of the nh vector . From (8), . Consequently,
i1520-0493-139-2-573-e15
where the nz vector dlT is given by
i1520-0493-139-2-573-e16
Using (16) to define the nz Γ— nh matrix 𝗗 gives
i1520-0493-139-2-573-e17
it follows that
i1520-0493-139-2-573-e18
and using (18) in (14) gives
i1520-0493-139-2-573-e19
We can then use (19) and (14) in (13) to obtain the following:
i1520-0493-139-2-573-e20
The above analysis allows us to see that the operation in (4) may be computed by applying the following list of operators.

  1. Use the operator π—ŸT to compute y = π—ŸT[x oΜ‡ h] at a cost of order Γ±T operations.

  2. Use the operator to compute the arithmetic sum over variables and time [see (16)] at each model grid point defining the 3D grid and hence derive the nz Γ— nh matrix 𝗗 [see (17)] at a cost of order nhnznqnΟ– = nT operations.

  3. Use zT to compute 𝗗Tvj for j = 1, 2, … , M with order nhnzM operations.

  4. Use hT to compute π—šT(𝗗Tvj) for j = 1, … , M and build the vector p in accordance with (20) at a cost of order MLhnh operations.

In operator form, the above algorithm may be summarized as (𝗖1/2)Tπ—ŸT(x oΜ‡ h) ≑ hTzTrtTT(x oΜ‡ h). Unsurprisingly, the cost for these adjoint operations is the same order of magnitude as it was for the forward algorithm at O(1011) operations and is again five orders of magnitude fewer operations than the approach described in section 2.

4. Conclusions

A fast algorithm for including nonadaptively localized ensemble covariances in variational algorithms has been derived and explained. This algorithm requires several orders of magnitude fewer computations than a simple-minded evaluation of the equations that were originally given in Lorenc (2003), Buehner (2005), and Buehner et al. (2010a,b). Elements of the fast algorithm were actually implemented by both Lorenc and Buehner in their work on localized ensemble covariances. However, their papers did not describe or explain the numerous nontrivial aspects of creating a fast algorithm. This paper serves to provide this information.

Some but not all of the techniques introduced here to speed nonadaptive ensemble covariance localization can also be applied to adaptive localization methods such as those discussed in Bishop and Hodyss (2009b). These include (i) defining the adaptive localization correlation matrix on a coarser resolution grid, and (ii) summing over variable type if the same adaptive localization functions are being applied to all variable types.

Acknowledgments

The authors wish to acknowledge the contributions of Ricardo Todling whose comments, as a reviewer, significantly improved the paper. Among other things, Ricardo improved the conciseness of the proof of the square root theorem. The authors also acknowledge the helpful criticism of the Editor, Herschel Mitchell, and the insightful suggestions of an anonymous reviewer. CHB expresses his gratitude to the β€œCentre for Australia Weather and Climate RESEARCHβ€”a collaboration between the Bureau of Meteorology and CSIRO” for their support of a visit to the Centre during which this paper was conceived. CHB and DH gratefully acknowledge financial support from ONR Project Element 0602435N, Project BE-435-003, and ONR Grant N0001407WX30012.

REFERENCES

  • Bishop, C. H., and D. Hodyss, 2007: Flow adaptive moderation of spurious ensemble correlations and its use in ensemble based data assimilation. Quart. J. Roy. Meteor. Soc., 133 , 2029–2044.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., and D. Hodyss, 2009a: Ensemble covariances adaptively localized with ECO-RAP. Part 1: Tests on simple error models. Tellus, 61 , 84–96.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., and D. Hodyss, 2009b: Ensemble covariances adaptively localized with ECO-RAP. Part 2: A strategy for the atmosphere. Tellus, 61 , 97–111.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., 2005: Ensemble-derived stationary and flow-dependent background error covariances: Evaluation in a quasi-operational NWP setting. Quart. J. Roy. Meteor. Soc., 131 , 1013–1043.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010a: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part I: Description and single-observation experiments. Mon. Wea. Rev., 138 , 1550–1566.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010b: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part II: One-month experiments with real observations. Mon. Wea. Rev., 138 , 1567–1586.

    • Search Google Scholar
    • Export Citation
  • Dee, D. P., 1991: Simplification of the Kalman filter for meteorological data assimilation. Quart. J. Roy. Meteor. Soc., 117 , 365–384.

    • Search Google Scholar
    • Export Citation
  • El Akkraoui, A., P. Gauthier, S. Pellerin, and S. Buis, 2008: Intercomparison of the primal and dual formulations of variational data assimilation. Quart. J. Roy. Meteor. Soc., 134 , 1015–1025.

    • Search Google Scholar
    • Export Citation
  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for a spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230 , 112–126.

    • Search Google Scholar
    • Export Citation
  • Kepert, J. D., 2009: Covariance localisation and balance in an Ensemble Kalman Filter. Quart. J. Roy. Meteor. Soc., 135 , 1157–1176.

    • Search Google Scholar
    • Export Citation
  • Liu, C., Q. Xiao, and B. Wang, 2009: An ensemble-based four-dimensional variational data assimilation scheme. Part II: Observing system simulation experiments with Advanced Research WRF (ARW). Mon. Wea. Rev., 137 , 1687–1704.

    • Search Google Scholar
    • Export Citation
  • Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWPβ€”A comparison with 4D-VAR. Quart. J. Roy. Meteor. Soc., 129 , 3183–3203.

    • Search Google Scholar
    • Export Citation
  • Miyoshi, T., and S. Yamane, 2007: Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution. Mon. Wea. Rev., 135 , 3841–3861.

    • Search Google Scholar
    • Export Citation
  • Parrish, D. F., and J. C. Derber, 1992: The National Meteorological Center’s spectral statistical-interpolation analysis system. Mon. Wea. Rev., 120 , 1747–1763.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., E. J. Kostelich, G. Gyarmati, E. Kalnay, B. R. Hunt, E. Ott, E. Satterfield, and J. A. York, 2008: A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A , 113–130.

    • Search Google Scholar
    • Export Citation
  • Wang, X., C. Snyder, and T. M. Hamill, 2007: On the theoretical equivalence of differently proposed ensemble–3DVAR hybrid analysis schemes. Mon. Wea. Rev., 135 , 222–227.

    • Search Google Scholar
    • Export Citation
  • Yang, S. C., E. Kalnay, B. R. Hunt, and N. E. Bowler, 2009: Weight interpolation for efficient data assimilation with the Local Ensemble Transform Kalman Filter. Quart. J. Roy. Meteor. Soc., 135 , 251–262.

    • Search Google Scholar
    • Export Citation

APPENDIX

a. Definitions and Proofs

Definition of the modulation product and square root theorem

It is convenient to define a modulation product u π—ͺ of an n-vector u with a n Γ— L matrix π—ͺ = [w1, w2, … , wL] to be the n Γ— L matrix that lists the modulations (or element-wise products) of each column of π—ͺ by u; in other words:
i1520-0493-139-2-573-ea1
We define the modulation product of the n Γ— M matrix 𝗨 = [u1, u2, … , uM] with an n Γ— L matrix π—ͺ = [w1, w2, … , wL] to be the n Γ— (LM) matrix 𝗨 π—ͺ such that it lists all possible modulations of the columns of π—ͺ by the columns of 𝗨; in other words,
i1520-0493-139-2-573-ea2
It follows that for a general LM vector f,
i1520-0493-139-2-573-ea3
where flm represents elements of the LM vector f partitioned as
i1520-0493-139-2-573-eqa1
Squareroot theorem for elementwise products
If the matrices 𝗨 and 𝗩 are n Γ— M matrices and π—ͺ and 𝗫 are n Γ— L matrices such that
i1520-0493-139-2-573-ea4
then
i1520-0493-139-2-573-ea5
Proof: The proof is by construction. First, note that
i1520-0493-139-2-573-ea6
where um and vm represent the mth columns of 𝗨 and 𝗩, respectively; and wl and xl represent the lth columns of π—ͺ and 𝗫, respectively. Second, note that if the subscripted variables {[𝗨 π—ͺ][𝗩 𝗫]T}ij and uij, Ο…ij, wij and xij denote the elements on the ith row and jth column of [𝗨 π—ͺ][𝗩 𝗫]T, 𝗨, 𝗩, π—ͺ, and 𝗫, respectively, then (A5) implies that
i1520-0493-139-2-573-ea7
and, hence, [𝗨 π—ͺ][𝗩 𝗫]T = 𝗔 oΜ‡ 𝗕 as was required.
The square root theorem for localized ensemble covariances is a corollary of the theorem above in the special case when the Schur product involves symmetric, semipositive definite matrices. To be specific, let 𝗭 = 1/[x1, x2, … , xK] so that
i1520-0493-139-2-573-ea8
then (A5) implies that
i1520-0493-139-2-573-ea9

b. Proof that , where fjT is a Lh vector such that fT = [f1T, f2T, … , fjT, … , fMT] and where π—ͺΟ… and π—ͺh have M and Lh columns, respectively.

Let wkh denote the kth column of π—ͺh and fjk be the kth element of fj then
i1520-0493-139-2-573-ea10
as was required.

c. Proof of (13)

To prove (13) note that each column of π—ͺΟ… π—ͺh can be expressed in the form wjΟ… oΜ‡ wkhβ€”the element-wise product of the jth column of π—ͺΟ… and the kth column of π—ͺh. Hence, each column of π—ͺΟ… π—ͺh is associated with two indices j and k. Hence, the row element of (π—ͺΟ… π—ͺh)Ty corresponding to wjΟ… oΜ‡ wkh is given by
i1520-0493-139-2-573-ea11
Equation (13) follows from (A11). [In (A11) β€œr” does not refer to a variable type as it does in the text; it is just an index.]

1

Recent implementations of efficient localization at Environment Canada and the Met Office did not degrade vertical resolution; however, for the reasons given in section 3, moving to a coarser vertical resolution should provide almost equivalent results at a smaller computational cost.

Save