• Courtier, P., and Coauthors, 1998: The ECMWF implementation of three-dimensional variational assimilation (3D-VAR). I: Formulation. Quart. J. Roy. Meteor. Soc., 124 , 17831807.

    • Search Google Scholar
    • Export Citation
  • Daley, R., 1991: Atmospheric Data Analysis. Cambridge University Press, 472 pp.

  • Fisher, M., 2003: Background error covariance modelling. Proc. ECMWF Seminar on Recent Developments in Data Assimilation for Atmosphere and Ocean, Reading, United Kingdom, ECMWF, 45–63.

    • Search Google Scholar
    • Export Citation
  • Giering, R., and T. Kaminski, 1998: Recipes for adjoint code construction. ACM Trans. Math. Softw., 24 (4) 437474. doi:10.1145/293686.293695.

    • Search Google Scholar
    • Export Citation
  • Kloeden, P. E., and E. Platen, 1992: Numerical Solution of Stochastic Differential Equations. Springer, 636 pp.

  • Mirouze, I., and A. Weaver, 2010: Representation of correlation functions in variational assimilation using an implicit diffusion operator. Quart. J. Roy. Meteor. Soc., in press.

    • Search Google Scholar
    • Export Citation
  • Oksendal, B., 2003: Stochastic Differential Equations: An Introduction with Applications. Springer, 374 pp.

  • Pannekoucke, O., 2009: Heterogeneous correlation modeling based on the wavelet diagonal assumption and on the diffusion operator. Mon. Wea. Rev., 137 , 29953012.

    • Search Google Scholar
    • Export Citation
  • Pannekoucke, O., and S. Massart, 2008: Estimation of the local diffusion tensor and normalization for heterogeneous correlation modelling using a diffusion equation. Quart. J. Roy. Meteor. Soc., 134 , 14251438.

    • Search Google Scholar
    • Export Citation
  • Pannekoucke, O., L. Berre, and G. Desroziers, 2008: Background error correlation length-scale estimates and their sampling statistics. Quart. J. Roy. Meteor. Soc., 134 , 497508.

    • Search Google Scholar
    • Export Citation
  • Purser, R., W-S. Wu, D. Parrish, and N. Roberts, 2003a: Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Rev., 131 , 15241535.

    • Search Google Scholar
    • Export Citation
  • Purser, R., W-S. Wu, D. Parrish, and N. Roberts, 2003b: Numerical aspects of the application of recursive filters to variational statistical analysis. Part II: Spatially inhomogeneous and anisotropic general covariances. Mon. Wea. Rev., 131 , 15361548.

    • Search Google Scholar
    • Export Citation
  • Szymczak, P., and A. J. C. Ladd, 2003: Boundary conditions for stochastic solutions of the convection-diffusion equation. Phys. Rev. E, 68 , 036704. doi:10.1103/PhysRevE.68.036704.

    • Search Google Scholar
    • Export Citation
  • Weaver, A., and P. Courtier, 2001: Correlation modelling on the sphere using a generalized diffusion equation. Quart. J. Roy. Meteor. Soc., 127 , 18151846.

    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    (a) A few correlation functions computed with nearest-neighbor interpolation with a large N, (b) difference between correlation functions obtained with the linear and the nearest-neighbor interpolation, and (c) length scale fields for the two interpolation methods compared with the theoretical length scale. (See text for details).

  • View in gallery
    Fig. 2.

    Relative errors vs the number of particles per grid points for Lh = (a) 250, (b) 500, and (c) 1000 km. See text for details.

  • View in gallery
    Fig. 3.

    Test of the normalization approximated from the local diffusion tensor in the heterogeneous case. See text for details.

  • View in gallery
    Fig. 4.

    Illustration of the correlation functions in (a),(c) the homogeneous Dirichlet case and (b),(d) the homogeneous Neumann case. With (a),(b) an approximated normalization and (c),(d) an exact normalization. See text for details.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 157 123 4
PDF Downloads 41 11 2

Stochastic Integration for the Heterogeneous Correlation Modeling Using a Diffusion Equation

Olivier PannekouckeCNRM/GAME, Météo-France/CNRS, Toulouse, France

Search for other papers by Olivier Pannekoucke in
Current site
Google Scholar
PubMed
Close
and
Laurent VezardCNRM/GAME, Météo-France/CNRS, Toulouse, France

Search for other papers by Laurent Vezard in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

In this note, a stochastic integration scheme is proposed as an alternative to a deterministic integration scheme, usually employed for the diffusion operator in data assimilation. The stochastic integration scheme is no more than a simple interpolation of the initial condition in lieu of the deterministic integration. Furthermore, this also presents a potential in high performance computing. For the classic preconditioned minimizing problem, the stochastic integration is employed to implement the square root of the background error covariance matrix, while its adjoint is obtained from the adjoint code of the square root code. In a first part, the stochastic integration method and its weak convergence are detailed. Then the practical use of this approach in data assimilation is described. It is illustrated in a 1D test bed, where it is shown to run smoothly for background error covariance modeling, with nearest-neighbor interpolations, and O(100) particles.

Corresponding author address: Olivier Pannekoucke, Météo-France/CNRM/GMAP, 42 Av. Coriolis, 31057 Toulouse, CEDEX France. Email: olivier.pannekoucke@meteo.fr

Abstract

In this note, a stochastic integration scheme is proposed as an alternative to a deterministic integration scheme, usually employed for the diffusion operator in data assimilation. The stochastic integration scheme is no more than a simple interpolation of the initial condition in lieu of the deterministic integration. Furthermore, this also presents a potential in high performance computing. For the classic preconditioned minimizing problem, the stochastic integration is employed to implement the square root of the background error covariance matrix, while its adjoint is obtained from the adjoint code of the square root code. In a first part, the stochastic integration method and its weak convergence are detailed. Then the practical use of this approach in data assimilation is described. It is illustrated in a 1D test bed, where it is shown to run smoothly for background error covariance modeling, with nearest-neighbor interpolations, and O(100) particles.

Corresponding author address: Olivier Pannekoucke, Météo-France/CNRM/GMAP, 42 Av. Coriolis, 31057 Toulouse, CEDEX France. Email: olivier.pannekoucke@meteo.fr

1. Introduction

The purpose of data assimilation is to provide the most likely numerical representation xa (the analysis) of the atmosphere, or of the ocean, from known observations yo. Nevertheless, observations are known to be noisy; that is, if xt denotes the true numerical state then yo = 𝗛(xt) + εo, with εo as the observation error and 𝗛 the (linear in what follows) observation operator that maps the model space into the observation space. Moreover, the observational network is heterogeneous in space and in time, with not enough observed area, which leads to a closure problem. A practical solution consists in adding a background xb (generally the most recent forecast) and to compute the analysis state as a correction δxa of the background, so that xa = xb + δxa.

The variational approach for resolving the issue consists in minimizing the cost function:
i1520-0493-138-8-3356-eq1
where is the background error covariance matrix with εb = xbxt, is the observation-error covariance matrix, d = yo − 𝗛(xb) is the innovation vector, and v = 𝗕−1/2δx. The minimum of is reached in va = 𝗕−1/2δxa. The square root matrix 𝗕1/2 is defined so that 𝗕 = 𝗕1/2𝗕T/2. The variational problem is solved by iterations along a minimizing process. Classic minimizing algorithms (e.g., conjugated gradient) require the gradient of the cost function:
i1520-0493-138-8-3356-eq2
where it clearly appears that 𝗕1/2 and its adjoint 𝗕T/2 are needed. In practice, 𝗕 is flow dependent, with heterogeneous correlation functions. The local length scale can stand for the local heterogeneity and can be defined as the scale for which the tangential parabola of the correlation function at the origin is equal to 0.5 (Daley 1991; Pannekoucke et al. 2008). Ensemble methods provide a way to estimate this matrix (Fisher 2003), but the usually small size of the ensemble leads to noisy statistics due to sampling effects. The huge size of 𝗕 implies that it must be modeled.

Different models exist for correlation matrix and among them, the diagonal assumption in spectral space (Courtier et al. 1998), which leads to homogeneous correlation functions. Amid the heterogeneous correlation models, one can cite the diagonal assumption in wavelet space (Fisher 2003), the recursive filter approach (Purser et al. 2003a,b), and the diffusion equation (Weaver and Courtier 2001).

The formulation based on the diffusion equation is now considered. It appears that along the minimization process of the cost function , a time integration of the diffusion equation is required for each computation of 𝗕1/2 (or 𝗕T/2) applied on an input vector. Since one deterministic integration is numerically costly, it would be desirable to find alternative ways. In this note, a stochastic time integration is considered as a possible alternative in lieu of the deterministic integration. The advantage of this approach is to replace a deterministic time integration by simple independent interpolations. This requires the offline computation of interpolation positions. The method is illustrated within a one-dimensional test bed.

The note is organized as follow. Section 2 recalls the correlation model based on the diffusion equation and its link with probabilities. The relationship between stochastic differential equations (SDEs) and elliptic partial differential equations (EPDEs) is described in section 3. This section also gives details about the numerical stochastic resolution of SDE for different boundary conditions. A one-dimensional illustration is then proposed in section 4, with emphasis on the technical implementation. The present work focuses on the horizontal covariance modeling, and details about 3D models can be found in Weaver and Courtier (2001) or in Pannekoucke (2009) for a nonseparable formulation. This is justified by the fact that in 3D models, it is classic to represent the horizontal component separately from the vertical component.

2. Covariance modeling with the diffusion operator

On the plane ℝ2 with coordinates x = (xi)i∈[1,2], the heterogeneous diffusion equation of a field u is given by
i1520-0493-138-8-3356-e1
where ν(x) = [νij(x)] is the local diffusion tensor. In this equation, the Einstein summation convention is used and ∂i denotes the partial derivative along the xi coordinate. The Gaussian function is a particular solution of the homogeneous diffusion equation (i.e., when the diffusion tensors are constant).
In fact, it corresponds to a Green solution (also called, in this situation, the heat kernel):
i1520-0493-138-8-3356-eq3
with Γ = 2νt and | · | denotes the matrix determinant, so that
i1520-0493-138-8-3356-e2
This has led some authors to take advantages of this Gaussian solution to design models of background error correlation functions. Thus, following Weaver and Courtier (2001), the square root of the background error covariance matrix is formulated as a product (for monovariate formulation):
i1520-0493-138-8-3356-e3
where Σ is the diagonal matrix of standard deviations and 𝗟 is the time integration over t = 0 to t = 1 (this is the convention adopted in the present note) of the heterogeneous diffusion Eq. (1), so that Γ = 2ν (Pannekoucke and Massart 2008). Thus, 𝗟1/2 denotes the half time integration. Here 𝗪 and Λ are two normalizations matrix, the former is a metric tensor that depends on the spatial discretization (Weaver and Courtier 2001; Pannekoucke and Massart 2008), and the latter is so that 𝗖1/2 = Λ𝗟1/2 𝗪−1/2 leads to a correlation matrix 𝗖 = 𝗖1/2𝗖T/2.

A crucial aspect of this model is related to the choice of the local diffusion tensor ν(x). A solution to the estimation of these tensors has been proposed by Pannekoucke and Massart (2008), and it relies on the ensemble methods to compute ν(x) from the local length scale (Pannekoucke et al. 2008), under a local homogeneous assumption of the tensor field: the matrix Γ(x) is first estimated from an ensemble of forecasts, then the local diffusion tensor is set as ν(x) = Γ(x)/2. This is the diffusion tensor field used in the heterogeneous diffusion in Eq. (1).

Moreover, an efficient numerical implementation of the diffusion in Eq. (1) is needed to construct 𝗟1/2. The numerical solution can be obtained with a time and space discretization of Eq. (1) (Weaver and Courtier 2001) or also by using a spectral approach instead of a space discretization (Pannekoucke and Massart 2008).

The next section introduces another way to integrate Eq. (1) by mean of stochastic processes. Before describing this tool, it is relevant to observe the particular form of Eq. (2). In a homogeneous case, the Gaussian form of the heat kernel is a probability density function and thus, Eq. (2) can be written as the following expectation:
i1520-0493-138-8-3356-e4
where X′ is a random variable with the density of probability px,t(x′) = Gν,t(x, x′), and where stands for the expectation operator. At this stage, the idea is to approximate the last distribution by an empirical one [i.e., px,t(x′) ≈ 1/Nkδ(x′ − Xk), where Xk is an ensemble of N samples of the random variable X′, and where δ(x) denotes the Dirac distribution]. In that context, the expectation is simply approximated as , where the value u(Xk, 0) can be obtained by interpolations in the discrete case. Each sample is obtained as a path of a stochastic process that remains to be found. The density of the probability is a solution of the Fokker–Planck equation (or Kolmogorov’s forward equation; Oksendal 2003) , simplified as ∂tp = ∂i(νijjp), with the initial density distribution p(x′, t = 0) set to the Dirac distribution δ(x′ − x).

Finally, the benefit is that the time integration is simply reduced to local interpolations of the initial condition at appropriate positions. This property is the key idea that motivates this note.

3. A stochastic integration scheme for EPDEs

The aim of this section is to recall how the solution of 1D EPDEs can be obtained as the expectation under a stochastic process (to be determined). A rigorous, but easily grasped, mathematical description can be found in Oksendal (2003), with an extension to n-dimensional cases. Thus, in a first time, the required background on stochastic calculus is depicted and then illustrated, next section, within a simple one-dimensional test bed.

a. Link between stochastic calculus and EPDEs

An Itô diffusion process is a stochastic process Xt defined by
i1520-0493-138-8-3356-e5
where a(x) and b(x) are two bounded functions. The first integral corresponds to a classic integration while the second is an Itô integral related to a Brownian process Bt (i.e., a stochastic process so that, for a duration τ, Bt+τBt is a Gaussian variable with a mean equal to zero and a variance to τ). The Itô integral is defined as the limit when δt → 0 of the sum ∑ib(Xti)(Bti+1Bti), as in the Riemann integral. In this expansion, the first integral of Eq. (5) is the deterministic part of the process, while the second integral is the Martingale term (Oksendal 2003), so that . Physically, Xt is the coordinate, at time t, of the random movements of a particle (e.g., pollen particles floating in water under the microscope). Because of the randomness, each realization leads to a new trajectory for the particle. The Itô process in Eq. (5) can also be written as the SDE:
i1520-0493-138-8-3356-e6
Later, a process Xt starting from x at t = 0 is denoted by Xtx. The extension of the deterministic chain rule for stochastic processes is a fundamental tool in stochastic calculus. Let f be a function, then the stochastic process associated with the change of variable f (Xt), is given by the Itô formula:
i1520-0493-138-8-3356-e7
where A is the generator of the stochastic process Xt, defined by
i1520-0493-138-8-3356-e8
Note that a stochastic chain rule gives rise to extra terms compared with a deterministic chain rule. At this stage, the first link between the stochastic process and the diffusion equation is sensitive to the occurrence of a second-order derivative in the generator A. The goal is now to write this second-order differential operator within a prognostic relation.
The form of Eq. (7) indicates that Yt is also an Itô diffusion, whose integral form is
i1520-0493-138-8-3356-eq4
Therefore, the expectation of the previous equation is formally written [Dynkin’s formula; see Oksendal (2003)] as
i1520-0493-138-8-3356-eq5
Then, it appears that
i1520-0493-138-8-3356-e9
is differentiable with respect to t and that its derivative is
i1520-0493-138-8-3356-eq6
The last step is to permute the generator A and the expectation. This is done thanks to the Kolmogorov’s backward equation theorem (Oksendal 2003) stating that u is the solution of the EPDE:
i1520-0493-138-8-3356-e10
with the initial condition f (x) = u(x, 0).

Conversely, as previously mentioned, the solution u(x, t) of the second-order partial differential equation in Eq. (10), with the initial condition u(x, 0), can be obtained as the expectation, at time t, of the random variable u(Xtx, 0), where Xt is the stochastic process of generator A. In practice, the solution u(x, t) is reached as follows. From a given generator A, one has to construct an Itô diffusion Xt, deduced from Eqs. (8) and (6), and integrate Eq. (10) as the expectation , where Xtx start from x at time t = 0.

b. Numerical solution and convergence properties

The solution of Eq. (10), is numerically approximated as the empirical mean:
i1520-0493-138-8-3356-e11
where , for k ∈ [1, N], is an ensemble of N particles randomly generated and starting in x. Each path is numerically generated from the discrete version of Eq. (6), for a given time scheme. In what follows, an Euler–Maruyama time scheme (Kloeden and Platen 1992) is used. The integration over the period [0, t] is achieved along the discrete time ti = iδt, with i ∈ [0, n] where δt = t/n (n is the number of time discretization). Thus, the Euler–Maruyama scheme is written as
i1520-0493-138-8-3356-e12
where ζ(ti) is a sample of a random Gaussian variable ζ with zero mean and with a standard deviation of one.
The convergence of Eq. (11) depends on the time step δt and on the numbers of particles N. The solution only requires the expectation of the random variable u(Xtx, 0), thus the accuracy of the numerical solution is considered only through a weak convergence (Kloeden and Platen 1992). This weak convergence, of the approximation t toward Xt, is given by
i1520-0493-138-8-3356-eq7
with α = 1 for the Euler–Maruyama scheme, and K1 a positive constant that depends on t. Furthermore, the weak convergence of the empirical mean to the expectation is in the classic form O(N−1/2) according to the central limit theorem, implying almost surely the inequality:
i1520-0493-138-8-3356-e13
Therefore, with a triangular inequality, the convergence of Eq. (11) almost surely is
i1520-0493-138-8-3356-e14

In practice, the functions u(x, 0), a(x) and b(x) are known by their values on a grid. Out of the grid, an interpolation method is required. In this paper, only two kinds of interpolation are considered: the nearest neighbor and the linear interpolation. This is equivalent to assume the functions piecewise constant in the former interpolation method, and continuously piecewise linear in the latter. Of course, such interpolations introduce discrepancies with the continuous case, and the right-hand-side bound in Eq. (14) must be completed with a third term, which depends on the spatial discretization. In the nearest-neighbor case, this error is in O(δx), while it is in O(δx2) in the linear case. As a consequence, the numerical heat kernel in the homogeneous case is no more Gaussian.

As suggested in the previous section, the solution u(x, t) simply results from an empirical average of local particles: in Eq. (1) the value u(x, t) can be estimated from an ensemble of particles whose paths are limited within the neighborhood around x. The width of this area depends on the local diffusion magnitude. From common results on normal law and considering a 1D homogeneous diffusion of diffusion coefficient ν = L2/2, there is a probability bigger than 99.99% that, at time t = 1, a particle stays inside the segment of radius 3.9L around the starting point. Similar thresholds can be found for n-dimensional cases thanks to the chi-square law (e.g., in 2D, the equivalent threshold is 4.3L). These bounds can serve for high performance computing by distributing the computation of Eq. (11) over the global domain into several independent computations over local areas with an appropriate halo. For this purpose, the width of the halo can be fixed thanks to a length scale L chosen as the largest length scale over the local area (including anisotropic effects). The numerical cost can be reduced by replacing the normal law ζ with a random variable of a same mean and variance, such as the Bernoulli variable of values −1 and 1, with equal probability. The weak convergence is unchanged. Also, samples of the Bernoulli variable (a switch on a uniform law) are cheaper to obtain than normal law [often generated with a Box–Muller algorithm; Kloeden and Platen (1992)].

c. Boundary conditions

The domain considered here is an open bounded set D ⊂ ℝ2 with an external boundary ∂D. Periodic, homogenous Neumann and homogeneous Dirichlet boundary conditions are detailed, since they are used in data assimilation (Weaver and Courtier 2001).

The homogeneous Neumann condition corresponds to the case where the derivative is null either on the boundary (or as part of the boundary). The Neumann conditions are also called “wall conditions.” The last denomination is well illustrated in stochastic calculus: homogeneous Neumann conditions match the case where the particles are reflected on the boundary. The reflection 𝗫′(t + dt) of a particle 𝗫(t + dt) escaping the domain, is computed as 𝗫′(t + dt) − 𝗢 = 𝗦[𝗫(t + dt) − 𝗢], where 𝗢 is the nearest point of 𝗫(t + dt) owning to the boundary, and 𝗦 = 𝗜 − 2nnT is the matrix of the local symmetry with n the local unit vector normal to the boundary at position 𝗢 (Szymczak and Ladd 2003).

The Dirichlet condition is related to cases where, on the boundary, the solution must verify u(x, t) = ΦD(x) for x ∈ ∂D, where ΦD is some arbitrary function. In stochastic calculus, this kind of condition is constructed as follow. If a particle crosses the boundary ∂D at a time τ (stopping time), then it is stopped on ∂D so that for all t > τ, 𝗫t = 𝗫τ. In fact, it can be shown (Oksendal 2003) that the solution of Eq. (1), with the Dirichlet condition u(x, t) = ΦD(x), is given by , with tτ = min(t, τ), and where particles Xτ on the boundary are evaluated as u(𝗫τ, 0) = ΦD(𝗫τ). The homogeneous condition is so that ΦD is a null function.

Thus, after each time step in Eq. (12), a test is required to verify if the particle is still inside the domain or if it has left it. Then, an appropriate reflection or stopping is applied, depending on the boundary condition. Of course, if the domain is periodic, the particle path always stays inside the domain and there no testing is necessary.

4. Applications in data assimilation

The previous formalism is now applied, for data assimilation purposes, in a simple one-dimensional test bed. The aim of this section is to document the use of the stochastic method for the time integration of the correlation model based on the diffusion operator.

The domain is a segment of length 2πa, where a = 6480 km, and it is discretized with 241 points. Three kinds of boundary conditions are considered: periodic, homogeneous Dirichlet, and homogeneous Neumann. Since the domain is similar to the great earth’s circle in the periodic case, then the horizontal coordinate x is considered either in degree or in kilometer. The step size is δx = 166 km. In this simplified framework, the diffusion equation of a field u is
i1520-0493-138-8-3356-e15
where ν(x) is a field of local diffusion coefficients. Under a local homogeneous approximation, the diffusion field is linked to the length scale field L(x) by ν(x) = L2(x)/2 (Pannekoucke and Massart 2008). Two cases are considered: an homogeneous case, where L(x) = Lh and an heterogeneous case, where L(x) = Lh[1 + cos(x/a)/2], with Lh = 250 km (Lh/δx ≈ 1.5). Thereafter, the homogeneous (heterogeneous) covariance matrix modeled with the homogeneous (heterogeneous) diffusion is denoted 𝗕ho(𝗕he). Moreover, the matrices computed by the mean of the stochastic process and using N paths by points are denoted with a subscript index N.

First of all, the stochastic process with the generator of Eq. (15) is determined with respect to section 3. Then, the stochastic time integration is tested under various forms and boundary conditions to construct 𝗕N1/2 and its adjoint 𝗕NT/2. Of course, these matrices are not explicitly stored in computer memory but implemented so that computations are equivalent in their overall effect to simulation of these operators being applied multiplicatively to input vectors.

a. Stochastic process associated with the 1D diffusion

The mathematical tool described in the previous section is applied here to integrate the diffusion in Eq. (15). The current equation is not under the practical form in Eq. (8). This can be done by rewriting Eq. (15) as
i1520-0493-138-8-3356-e16
where the generator Af = ∂xν(x)∂xf + ν(x)∂x2f is clearly visible. Then the unknown functions a and b are easily found by identification with Eq. (8), the result being
i1520-0493-138-8-3356-e17
The acting Itô diffusion is finally written as
i1520-0493-138-8-3356-e18
See the appendix for the n-dimensional case.

b. Implementation of 𝗕1/2 and 𝗕T/2

In practice, 𝗕1/2 is not stored in the computer memory but is evaluated on a vector as 𝗕1/2v. The evaluation by 𝗕1/2 is achieved through the product in Eq. (3). Products matrix/vector, with diagonal matrices 𝗪−1/2, Λ and Σ, are reduced to element-wise products.

In 1D framework, the diagonal values of 𝗪−1 are δx for points inside the domain. For points on the boundary, the diagonal value is δx for periodic conditions and δx/2 for other cases.

Under a local homogeneity assumption, a good approximation for the normalization Λ can be deduced from the local diffusion tensor (Pannekoucke and Massart 2008). In a 2D, the normalization applied at position x is Λ(x)2 = 2π. In a 1D, it is Λ(x)2 = L(x), where L(x) is the local length scale in x. In the following, either the exact or the previous approximated normalization are used.

The principal difficulty rests with the computation of the operator 𝗟1/2. In this paper, the operator is a half-time stochastic integration (from t = 0 to t = ½) of the heterogeneous diffusion equation. It requires the computation at time t = ½ of N particles per grid points, governed by an Itô diffusion. Each path is integrated according to Eq. (12), with appropriate boundary conditions as described in section 3c. These endpoints, at half-time, are computed once and for all, and are stored in computer memories. Then endpoints are used at each computation of 𝗟N1/2v in the empirical mean evaluation in Eq. (11), where the initial condition u(x, 0) is replaced by interpolations of v at the endpoint paths.

The adjoint 𝗕NT/2 can be carried out from the direct code of 𝗕N1/2, by using classic adjoint coding rules (Giering and Kaminski 1998). The endpoints involved in the computation of 𝗕N1/2 are reused in the computation of 𝗕NT/2. In this way, 𝗕NT/2 leads to the exact transpose of the matrix 𝗕N1/2.

Note that the computation of the background-error covariance matrix as a product of 𝗕N = 𝗕N1/2𝗕NT/2 ensures the symmetry and the positiveness of the covariance model.

c. Numerical experiments and results

Several points for the use of the stochastic approach need to be documented: the sensitivity to the interpolation method; the convergence versus the number of particles, the ratio Lh/δx, and the time step; the validity of the approximated normalization; and the use of Dirichlet–Neumann boundary conditions. These are now detailed.

1) Sensitivity with interpolation methods

As explained in section 3b, an interpolation is needed in the numerical resolution of the dynamics in Eq. (12). The discrepancies between the background error covariance matrix resulting from the linear interpolation and the nearest-neighbor interpolation are illustrated in Fig. 1. In these simulation, the diffusion is heterogeneous with periodic boundary conditions and δt = 1/200. The number of particles (N = 105) is large enough to damp the sampling noise. The correlation functions obtained by using the exact normalization are presented in Fig. 1a (only 1 out of 10 is represented). Note that these correlation functions can be considered as analysis increments associated to one observation assimilation experiments. In that case, a single observation is located at the position of the maximum of each correlation function. The correlation function related to the positions 90° and 180° are in bold. It appears that correlation functions are quasi-Gaussian. The difference between correlations obtained with nearest-neighbor interpolation minus those obtained with linear interpolation is represented in Fig. 1b (again only 1 out of 10 is represented). The difference related to the positions 90° and 180° are also in bold. The linear interpolation leads to greater correlation values than the nearest-neighbor interpolation. This is confirmed by the length scale diagnosis, reported in Fig. 1c, where the length scale of 𝗕N in the linear interpolation case (dash–dotted line) is larger than in the nearest-neighbor case (dashed line). Note that both interpolations over estimate the true length scale field (solid line).

Results from this sensitivity test show that the correlation functions obtained with a nearest-neighbor interpolation are similar to those of the functions obtained with the linear interpolation. Moreover, the nearest-neighbor interpolation is less costly than the linear interpolation. Therefore, the nearest-neighbor interpolation appears as a good compromise for data assimilation constraints.

2) Convergence versus the number of particles, the length scale magnitude, and the time step

The sampling noise effect, arising from the number of particles N, is investigated for the homogeneous diffusion with periodic boundary conditions. For such a case, the theoretical correlation function is known and equal to ρ(x) = exp(−x2/2Lh2). The theoretical correlation matrix 𝗕ho is compared with simulated matrices 𝗕N for N in (100, 400, 1600, 6400). The 𝗕N is computed with the nearest-neighbor interpolation and with the integration time step sets to δt = 1/200. Moreover, an exact normalization is employed for Λ. The discrepancy between 𝗕ho and 𝗕N is quantified by the relative error eN1 = 100(‖𝗕N − 𝗕ho‖/‖𝗕ho‖), where ‖·‖ is the Hilbert norm and ‖𝗕‖ = , where Tr denotes the matrix trace. This relative error is represented by a solid line in Fig. 2. It appears that 𝗕N do not converge toward 𝗕ho. This is related to the error due to the interpolations and to the time scheme, leading to a solution 𝗕 distinct from the theoretical one. However, the error decreases when the ratio Lh/δx increases: for N = 6400, the error is close to 4% for Lh = 250 km (see Fig. 2a), while it is less than 2% for Lh = 500 km (Lh/δx = 3) and Lh = 1000 km (Lh/δx = 6) (see Figs. 2b,c). The errors due to the interpolation and to the time discretization (due to δt = 1/200) are damped with the ratio Lh/δx.

The convergence of 𝗕N toward 𝗕 is theoretically in O(N−1/2) since the convergence of 𝗕N1/2 to 𝗕1/2 is in O(N−1/2) [according to Eq.(13)]. Confirmation can be seen in Fig. 2 where the dash–dotted line represents the relative error eN2 = 100(‖𝗕N − 𝗕‖/‖B‖), where 𝗕 is approximated by 𝗕N for a large enough N (here N = 105). The slope of eN2 is asymptotically −½ (represented by the dashed line). But the error increases with Lh/δx for a small ensemble of particles: for N = 100, the error is less than 10% for Lh = 250 km (see Fig. 2a), while it is close to 15% for Lh = 1000 km (see Fig. 2c). Same results are found for heterogeneous case (not shown here).

Some sensitivity experiments (not shown here) on the time step δt = 1/2n (n is the number of discretization of [0, ½], with N = 400, indicate that correlations are overestimated for small n (of the order of 10) but converge rapidly for large n (of the order of 50). Note that in any case, the stochastic integration is stable, even with the extreme case δt = ½, and the resulting correlation functions are quasi-Gaussian.

With previous results, a practical ensemble size of N = 100 (around 10% error) or N = 400 (around 5% error) is accurate enough [that is in O(100)]. But this should be confirmed for real case applications since the error depends on the ratio Lh/δx. Small values for δt = 1/2n increase the cost of the integration of particle paths, but this is not limiting as these trajectories are computed one time, only to retain the endpoints; thus, large n can be used. In most of experiments performed in the note n is set to 100.

3) Discussion on the normalization

As mentioned in section 4b, the normalization Λ can be approximated from the local diffusion tensor. Under this approximation, the local normalization at the point x is Λ(x)2 = L(x). This normalization is tested with a periodic boundary condition, for the heterogeneous diffusion operator, with nearest-neighbor interpolation, N = 400 and δt = 1/200. The resulting variance is reproduced in Fig. 3. The approximated normalization overestimates the exact normalization that would lead to a homogeneous variance field equal to 1 everywhere. Indeed, the mean of the resulting variance is 3% lower than 1. But the standard deviation of the variance is 0.05, which is small enough to propose a global inflation that forces the global variance to be equal to 1 in mean.

The approximated normalization is thus still efficient in this stochastic framework. However, some boundary effects can be encountered and are discussed hereafter.

4) Illustration with nonperiodic boundary conditions

In data assimilation, bounded domains are common (e.g., in local area model or in ocean modeling with coast treatment). Homogeneous Dirichlet and Neumann boundary conditions are classically considered in covariance modeling with the diffusion operator (Weaver and Courtier 2001). They are now tested for a homogeneous diffusion operator for N = 400 and δt = 1/200. The boundary conditions are set as follow: homogeneous Dirichlet at x = 0° and homogeneous Neumann at x = 360°. The correlation functions obtained with the approximated normalization Λ are reported in Figs. 4a,b. As mentioned in section 4c(1), each function can be viewed as an increment analysis obtained from a single observation assimilation. It appears that these functions are not correlation functions since the variance is not close to one. In Fig. 4a, the functions vanish with a decreasing x, this is consistent with the Dirichlet condition that imposes the solution to be null at x = 0°. At the opposite, in Fig. 4b, the functions increase with an increasing x, this is consistent with the Neumann condition that imposes with reflections, an accumulation of particles at x = 180°. These understandable behaviors are not balance by the normalization Λ, sets as a constant Lh in a homogeneous case, and do not lead to correlation functions. These results can be compared when applying the exact normalization (see Figs. 4c,d). The correlations resulting from the Dirichlet condition are similar to those obtained with the Neumann condition, but the major difficulty is to find a good approximation for the normalization (Weaver and Courtier 2001). Recently, Mirouze and Weaver (2010) have improved the approximated normalization for these two boundary conditions (not tested here). As there are few differences between the correlation functions obtained with Dirichlet or with Neumann, the homogeneous Dirichlet condition is to be favored within this stochastic approach. Indeed, the Dirichlet condition is cheaper than the Neumann condition: there is no reflection to implement and the contribution of particles stopped at the boundary is null in Eq. (11).

5. Conclusions

In this note, a stochastic integration of the diffusion equation has been proposed and tested to be used in the model of covariance matrix related on the diffusion operator. The idea is to replace the deterministic integration by local interpolations of the initial condition. This method is naturally adapted for parallel computation since the computation only depends on combination of the initial condition taken on independent areas.

The stochastic integration is achieved thanks to a particle method whose path dynamic is described by a stochastic differential equation deduced from the heterogeneous diffusion equation.

It has been shown, within a one-dimensional test bed, that this approach can be considered in a data assimilation framework for background error covariance modeling. It rests on the expansion of the covariance matrix as the product of a matrix (the square root matrix) and its transposition. The transpose of the square root matrix is achieved through the adjoint code of the square root matrix direct code. It can be inferred from experiments that the nearest-neighbor interpolation (at a low cost compared with other interpolation methods), and a relatively small ensemble of particles O(100), are enough to build quasi-Gaussian correlation functions.

However, there is still some work to achieve for the two-dimensional case where the number of particle might be much larger then the simple one-dimensional case. This study is considered as a preliminary change of perspective in the refinement of the diffusion formulation, and it requires further developments and optimizations to be ready to use.

Acknowledgments

Isabelle Mirouze, Sebastien Massart, and Jean-Antoine Maziejewski are warmly thanks for their careful reading of the manuscript. The authors are grateful to Christophe Baehr for fruitful discussions on stochastic calculus and numerous advice on this note.

REFERENCES

  • Courtier, P., and Coauthors, 1998: The ECMWF implementation of three-dimensional variational assimilation (3D-VAR). I: Formulation. Quart. J. Roy. Meteor. Soc., 124 , 17831807.

    • Search Google Scholar
    • Export Citation
  • Daley, R., 1991: Atmospheric Data Analysis. Cambridge University Press, 472 pp.

  • Fisher, M., 2003: Background error covariance modelling. Proc. ECMWF Seminar on Recent Developments in Data Assimilation for Atmosphere and Ocean, Reading, United Kingdom, ECMWF, 45–63.

    • Search Google Scholar
    • Export Citation
  • Giering, R., and T. Kaminski, 1998: Recipes for adjoint code construction. ACM Trans. Math. Softw., 24 (4) 437474. doi:10.1145/293686.293695.

    • Search Google Scholar
    • Export Citation
  • Kloeden, P. E., and E. Platen, 1992: Numerical Solution of Stochastic Differential Equations. Springer, 636 pp.

  • Mirouze, I., and A. Weaver, 2010: Representation of correlation functions in variational assimilation using an implicit diffusion operator. Quart. J. Roy. Meteor. Soc., in press.

    • Search Google Scholar
    • Export Citation
  • Oksendal, B., 2003: Stochastic Differential Equations: An Introduction with Applications. Springer, 374 pp.

  • Pannekoucke, O., 2009: Heterogeneous correlation modeling based on the wavelet diagonal assumption and on the diffusion operator. Mon. Wea. Rev., 137 , 29953012.

    • Search Google Scholar
    • Export Citation
  • Pannekoucke, O., and S. Massart, 2008: Estimation of the local diffusion tensor and normalization for heterogeneous correlation modelling using a diffusion equation. Quart. J. Roy. Meteor. Soc., 134 , 14251438.

    • Search Google Scholar
    • Export Citation
  • Pannekoucke, O., L. Berre, and G. Desroziers, 2008: Background error correlation length-scale estimates and their sampling statistics. Quart. J. Roy. Meteor. Soc., 134 , 497508.

    • Search Google Scholar
    • Export Citation
  • Purser, R., W-S. Wu, D. Parrish, and N. Roberts, 2003a: Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Rev., 131 , 15241535.

    • Search Google Scholar
    • Export Citation
  • Purser, R., W-S. Wu, D. Parrish, and N. Roberts, 2003b: Numerical aspects of the application of recursive filters to variational statistical analysis. Part II: Spatially inhomogeneous and anisotropic general covariances. Mon. Wea. Rev., 131 , 15361548.

    • Search Google Scholar
    • Export Citation
  • Szymczak, P., and A. J. C. Ladd, 2003: Boundary conditions for stochastic solutions of the convection-diffusion equation. Phys. Rev. E, 68 , 036704. doi:10.1103/PhysRevE.68.036704.

    • Search Google Scholar
    • Export Citation
  • Weaver, A., and P. Courtier, 2001: Correlation modelling on the sphere using a generalized diffusion equation. Quart. J. Roy. Meteor. Soc., 127 , 18151846.

    • Search Google Scholar
    • Export Citation

APPENDIX

Stochastic Process for n-Dimensional Heterogeneous Diffusion

In the general n-dimensional case, with x = (xi)i∈[1,n], Eq. (1) can be written as
i1520-0493-138-8-3356-ea1
The aim is to express the n-dimensional Itô diffusion dXt = (dXt,i)i∈[1,n], where
i1520-0493-138-8-3356-eqa1
with Bt an n-dimensional Brownian process (i.e., a stochastic process so that, for a duration τ, Bt+τBt is a Gaussian vector with mean equal to zero and covariance matrix τ𝗜).
Following section 3 and its n-dimensional extension, the generator is thus . The unknown functions ai and σij are fixed by identification according to the n-dimensional version of Eq. (8), which results in the following:
i1520-0493-138-8-3356-ea2

Fig. 1.
Fig. 1.

(a) A few correlation functions computed with nearest-neighbor interpolation with a large N, (b) difference between correlation functions obtained with the linear and the nearest-neighbor interpolation, and (c) length scale fields for the two interpolation methods compared with the theoretical length scale. (See text for details).

Citation: Monthly Weather Review 138, 8; 10.1175/2010MWR3239.1

Fig. 2.
Fig. 2.

Relative errors vs the number of particles per grid points for Lh = (a) 250, (b) 500, and (c) 1000 km. See text for details.

Citation: Monthly Weather Review 138, 8; 10.1175/2010MWR3239.1

Fig. 3.
Fig. 3.

Test of the normalization approximated from the local diffusion tensor in the heterogeneous case. See text for details.

Citation: Monthly Weather Review 138, 8; 10.1175/2010MWR3239.1

Fig. 4.
Fig. 4.

Illustration of the correlation functions in (a),(c) the homogeneous Dirichlet case and (b),(d) the homogeneous Neumann case. With (a),(b) an approximated normalization and (c),(d) an exact normalization. See text for details.

Citation: Monthly Weather Review 138, 8; 10.1175/2010MWR3239.1

Save