a. Data assimilation and the representation of control space

In this paper, we are not directly interested in the nature of the control fields or in a careful choice of uncorrelated control fields. This choice is assumed to already have been made. Because the physics are modeled through numerics, they have to be discretized and the choice of a resolution should be made. This resolution problem generalizes to the question of the representation of the control fields.

Note that in numerical simulation (no assimilated observations) the issue of choosing a proper representation (e.g., an adaptive grid) is well studied. It allows speeding up the simulations and refines the computations in which the numerical error could be large (see, e.g., Saad 2003). An example in geophysics, and in particular air quality, can be found in Constantinescu et al. (2008).

However, in data assimilation, control variables and observations are equally important components. Because they do not share the same representation space but are nevertheless combined in the data assimilation process, the optimal representation problem is much less simple there.

Note that the control space representation can be different from the forward model (state) space discretization, even if the control space is a subspace of the state space. For instance, their resolutions may differ resulting from use of regular meshes of the same underlying domain.

b. Can the representation be chosen?

A fundamental issue is to decide how control fields should be discretized. In particular, can we choose, in a reasonably nonarbitrary fashion, the resolution of the control space in data assimilation? Hints could come from information theory. Provided adequate tools exist, one would compare the information content of the observations and the capacity of the control space grid to contain this information filtered out by the data assimilation analysis. In addition to the Bayesian inference principle used in data assimilation, a (typically variational) side principle would help construct an optimal representation. Such a cost function could establish how fine the grid resolution should be if it has to accommodate for the set of available data. Such a choice may stem from the balance found between the information content of the observation (but also the background) and the amount of information that the control variables are able to hold.

In this paper, a simple criterion of optimality is chosen based on the trace of the inverse analysis error covariance matrix, with known links to an information content analysis to be discussed and recalled.

c. Regularization, continuum limit, and scaling divergence

In a data assimilation system, what if the control space is discretized in too fine a manner? This question is actually central to inverse problems. Without regularization, the problem is underdetermined and therefore ill-posed. Regularization has been devised to circumvent this difficulty. In the data assimilation literature, regularization is performed through background implementation with a clear physical interpretation. In many cases, common regularization [such as Tikhonov; i.e., a least square constraint between the system state and a first guess (Rodgers 2000)] is sufficient to accommodate a finer resolution of control space (Bocquet 2005a). As the resolution increases, no new information coming from the observation is strengthening the inference or data assimilation analysis. Obviously, the computation load is heavier. However, the solution (the analysis) is only marginally changed.

Yet, it has been shown recently that, in some circumstances (e.g., the identification of an atmospheric pollutant accidental-release source), this regularization may not be enough to support the increase in the resolution (Bocquet 2005a; Furbish et al. 2008). As the resolution increases, the solution ignores the benefit of the observations more and more. With a Tikhonov regularization scheme, it may converge to the first guess. The analysis is mathematically sound, and the degeneracy is understood on physical grounds (Bocquet 2005a). However, the solution is no longer impacted by the observations except in the vicinity of the observation sites. In this context, it is therefore mandatory to make an educated choice on a (if not the) proper scale of the representation.

Building on Bocquet (2005a), general results will be obtained on this issue in section 2.

d. Greenhouse gas fluxes estimation: An optimal representation problem

The driving force of global warming is very likely to be greenhouse gases and their relative balance (Pachauri and Reisinger 2007). The first one, in terms of impact, is carbon dioxide. From climate theory, it is therefore crucial to evaluate precisely the fluxes and exchange of carbon on earth. Therefore, biogeochemists have built up inventories of fluxes that are still uncertain, especially biogenic emissions and uptakes. A complementary approach is to use partial flux data and carbon dioxide concentration measurements and, through inverse modeling and because of prior hypotheses and a global dispersion model, estimate carbon fluxes on the globe. This is the so-called top-down approach. However, until the arrival of carbon satellite data, the pointwise concentration and flux measurements are sparse on the globe, albeit very reliable. Their information content is insufficient to accommodate the global-scale high-resolution map of fluxes that the carbon community is hoping for. So, it was decided very early to split the world into a few areas, leading to clearly identified aggregation errors (Fan et al. 1998; Bousquet et al. 2000). With more data available, the community was then inclined toward high-resolution reconstruction of the fields (Peylin et al. 2005). But then the information content of the observations does not match this too-fine control space. So here, too, the choice of the representation of the control space is thought to be crucial, and quantitative prescription of the right scale would be decisive. The formalism developed in this paper should help in this respect.

e. Objectives and outline

The aim of this paper is to demonstrate that the representation of the control parameter space can be chosen optimally to better match the available sources of information introduced in the data assimilation system.

In section 2, we reinterpret old data assimilation results by studying, in general terms, how the data assimilation analysis, as a measure of the inference quality, evolves through changes in the representation of control space defined on the same domain. This justifies how important an optimal choice of representation can be. In section 3, a simple optimality criterion is chosen to select a representation. A mathematical framework is then developed to make the idea explicit and implementable. In section 4, the formalism is applied in the field of atmospheric dispersion, in which an optimal representation is obtained given a fixed number of grid cells. Using this optimal representation, inverse modeling of a tracer source is then performed and a comparison is made with earlier results on regular grids. A summary and perspectives are given in section 5.

a. Reduction of error and analysis error covariance matrix

In the framework of the best linear unbiased estimator (BLUE) analysis, the posterior error covariance matrix reads as follows:

View Expanded

where 𝗕 is the background covariance matrix (which will be assumed full rank in the following), 𝗥 is the observation covariance matrix, and 𝗛 is, broadly speaking, the observation operator. Here, T denotes the usual vector or matrix transposition. Alternatively, the inverse of the error covariance matrix, or the confidence matrix, is the following:

View Expanded

This matrix is also called the Fisher information matrix (for a general definition in a geophysical context, see Rodgers 2000, p. 32), which is a measure of the information conveyed by the observations (typically a vector of measurements μ ∈ ℝ^d, where d is the number of observations) to the control variables (a vector σ ∈ ℝ^N, where N is the number of variables) through the data assimilation optimality system, when μ is seen as a random vector.

It is assumed that the control space has an underlying spatial and temporal structure, with a discretization parameter r and a spacing that could be spatial, temporal, or both. This structure has a continuum limit (r → 0), which is considered as the “truth” in geophysics. For instance, one may be attempting to retrieve a ground emission field (two spatial dimensions and time; 2D + T). Then 𝗛, as a Jacobian matrix, relates the concentration measurements to the space–time emissions. It is computed because of numerical schemes solving consistently, for some discretization r′(possibly different from r), a set of partial differential equations.

As is well known, the analysis systematically reduces the uncertainty because 𝗣_a ≺ B, where ≺ denotes partial ordering between semidefinite positive matrices. Alternatively, the confidence is systematically increased: 𝗣_a⁻¹ ≻ 𝗕⁻¹. A similar analysis based on the same dataset is contemplated but the resolution, or more generally the representation of the control space, is now changed. When both observations and model are perfect (i.e., 𝗥 = 0), then 𝗣_a = 𝗕 − 𝗕𝗛^T(𝗛𝗕𝗛^T)⁻¹𝗛𝗕. A limiting case is reached when a representation of the control space has d cells, which is the number of observations. As a consequence, 𝗛 may become invertible, at least in an infinite precision numerical context, so that in this specific case 𝗣_a = 0. Still assuming 𝗥 = 0, suppose that the number of variables is greater than the number of observations N > d. This obviously happens when r → 0. Then the uncertainty is not null anymore but its reduction is controlled by , where and 𝗜_N is the identity matrix in ℝ^N. Therefore, the posterior uncertainty is essentially controlled by the ability of the averaging kernel to reproduce the identity. [The averaging kernel is the sensitivity matrix of the analysis state with the true state; see Rodgers (2000) for a clear definition and properties.]

One invariant measure of the total variance of the error is the trace of 𝗣_a, which serves as the optimality criterion for establishing BLUE. The expression 𝗕^−1/2𝗣_a𝗕^−1/2 reads the posterior covariance matrix in the basis where all parameters, viewed as random variables, are independent with variance unity. Therefore, it actually measures the reduction of uncertainty with respect to the uncertainty attached to the independent degrees of freedom of the prior. Then Tr(𝗜_N − 𝗣_a𝗕⁻¹) = d measures the number of degrees of freedom that are elucidated in the analysis. In an errorless context (𝗥 = 0), this equals the number of (independent) observations, as noted by Rodgers (2000). Similarly, Tr(𝗣_a𝗕⁻¹) = N − d rigorously states the obvious: whereas the degrees of freedom are increasing with increasing mesh resolution, the reduction of uncertainty from observation remains the same.

Note that 𝗣_a𝗕⁻¹ also appears in the measure of the average gain of information (average reduction of entropy) in the BLUE analysis (Fisher 2003):

View Expanded

where |𝗠| is the determinant of matrix 𝗠 and 𝗔 = 𝗜_N − 𝗣_a𝗕⁻¹ is the averaging kernel matrix. Here, K_σ is the relative entropy or Kullback–Leibler information measure (Cover and Thomas 1991), using Gaussian hypotheses. The reduction of uncertainty and the reduction of entropy are closely related and coincide in the limit where the reduction of entropy (gain of information) is small with respect to the prior, up to a conventional factor measuring the information unit: . In the same limit, a closely related expansion of the information measure can also be performed:

View Expanded

This expression will be used in section 3 as an optimality criterion.

These elementary data assimilation statements show that increasing the sharpness of the representation (e.g., the resolution by decreasing r), and hence increasing the number of unknown variables N beyond the available observational information (d observations), could be detrimental to the quality of the analysis. The background acts as a mathematical regularization in the data assimilation system when the information content of the d observations cannot account for the N unknown variables. However, the background is usually inducing extra confidence that is often criticized, especially within the atmospheric chemistry applications where emission inventories are very uncertain (Elbern et al. 2007). So, it may often come as a surprise that increasing the resolution in data assimilation decreases the performance of the analysis.

This may be considered less surprising when data assimilation is regarded as an inverse problem. Then, this quest for information (Tarantola and Valette 1982) amounts to matching just enough unknown variables for the available information content. This is obviously a trivial task when inverting an algebraic linear system because there is only one flux of information (the individual equations of the system). In data assimilation, the difficulty comes from quantifying the fluxes of information because the sources are multiple (e.g., background, model, and observations) and of distinct nature.

Note that the background modeling is also dependent on representation (or scale) because variances of control parameters usually vary with the scale. For instance, considering intensive variables, statistically independent subelements have a stronger variance than the variance attached to their aggregated mother element. However, taking into account the proper scaling does not necessarily prevent the degradation of the analysis. If we do not assume that the variables are statistically independent, then a correlated proper background might remove the degrading effect. But this popular trick actually amounts to reducing the effective number of independent degrees of freedom, though in a smoother manner.

b. Continuum limit for the posterior analysis

We have proved in the previous section that increasing the resolution beyond some threshold determined by the information content of the observations is always detrimental to the analysis. Let us now show that it may even lead to a severe failure of the analysis.

As mentioned earlier, 𝗛 could be a Jacobian matrix that directly relates the observations to the control space. Again, let us change the representation of the control space (e.g., the resolution) and consider its consequence on the analysis. In the continuum limit, at least two kinds of behavior can actually occur, which have been described in Bocquet (2005a). The actual regime is decided by the behavior of the innovation statistics matrix 𝗥 + 𝗛𝗕𝗛^T in the continuum limit. A possible divergence was first pointed out by Issartel and Baverel (2003). To stress the significance of a proper representation of control space, let us first exhibit a simplified case relevant to physical applications in which such divergence may occur.

1) Anomalous regime: example of divergence

Essentially following Bocquet (2005a), let us give a simple example of the singular regime. A Fickian diffusion equation is considered. A field of pollutant c obeys the following diffusion equation on the space–time domain Ω:

View Expanded

where σ is the source field of the pollutant and κ is a diagonal diffusivity tensor. The choice of a Fickian operator may look overly simplified, but it does capture qualitative features of atmospheric transport, such as convective diffusion and mixing for a sufficiently long time, that are relevant to data assimilation and inverse modeling applications. To directly relate a pointwise concentration measurement μ_i to the source field, the adjoint equations can be introduced, one for each observation:

View Expanded

where π_i is a retro source, which, in the case of a pointwise and instantaneous concentration measurement at (x_i, t_i), reads π_i(x, t) = δ(x − x_i, t − t_i) so that

View Expanded

where errors ϵ_i can also be accounted for.

In the discrete case, the vector of measurements μ ∈ ℝ^d is related to the source or flux components σ ∈ ℝ^N through the Jacobian matrix 𝗛 ∈ ℝ^d×N, whose lines are approximated by the discretization c*_i of the adjoint solutions c*_i.

Assume that the source or emission field has a prior model formalized by a simple background, a Tikhonov regularization, given by

View Expanded

where 𝗕 = m²𝗜_N and m is a scalar, homogeneous to a root-mean-square error. Then, in this example, the innovation statistics diverge in the continuum limit. In particular, one has 𝗥 + 𝗛𝗕𝗛^T ≃ m²𝗛𝗛^T. Indeed the elements of this Gram matrix are given by scalar products of adjoint solutions. For noncoinciding observations, the scalar product of two elementary diffusion solutions has a proper continuum limit: the two singularities of the integral coincide with the two observations that are integrable because the species concentrations are locally integrable by mass conservation. However, this is not the case for coinciding observations, so that the diagonal of 𝗛𝗛^T diverges. In particular, it was shown (Bocquet 2005a) that in the continuum limit one typical element of the diagonal either reaches a finite limit or behaves like , where Ω_r is the full space and time domain, punctured by a ball with radius r around the related observation site. The problem-dependent exponent α > 0 qualifies the rate of divergence. Note that this divergence can also characterize a wide range of non-Fickian diffusion operators and maybe other types of evolution operators that are relevant to many geophysical situations.

Keeping in mind this possible divergence in the continuum limit, we now look at the consequences for the analysis and the space–time spread of the observational information in quite general terms.

2) Consequences for the analysis

Following the previous example, 𝗛 is assumed to relate the vector of fluxes or source elements σ in ℝ^N to the vector of observations μ in ℝ^d so that μ = 𝗛σ + ϵ, where ϵ is the errors vector. However, this model equation is more general, even though it assumes a linear model. It was shown that the innovation statistics may diverge in some well-founded geophysical applications when r vanishes. Let us analyze the implications on the BLUE analysis.

Here, 𝗛 has a singular value decomposition 𝗛 = 𝗨𝗗𝗩^T, where 𝗨 is an orthogonal matrix of size d × d, 𝗗 is a diagonal d × d matrix made of the singular values λ_i (which implicitly depend on the spacing r), and 𝗩 is a N × d matrix satisfying 𝗩^T𝗩 = 𝗜_d.

In the following, for the sake of simplicity, the background and observation error covariance matrices will be assumed diagonal of the form 𝗕 = m²𝗜_N and 𝗥 = χ𝗜_d. As long as physical correlations are short range, the conclusions of the following developments will remain general. In particular, it is possible to perform similar calculations in the so-called standardized form, in which the singular value decomposition is applied to 𝗥^−1/2𝗛𝗕^1/2 instead of 𝗛 (for details on the standardized form, see Rodgers 2000). Depending on the exact definition of the coarse-graining operation, m may or may not be scale dependent. Also, χ may depend on the scale because of the representativeness error, but this effect is neglected here.

If 𝗩 = (υ₁, … , υ_d), then the averaging kernel 𝗔 = (𝗕⁻¹ + 𝗛^T𝗥⁻¹𝗛)⁻¹𝗛^T𝗥⁻¹𝗛 can be written as follows:

View Expanded

where 𝗔 is a projector on the {υ_i}_{i=1, … ,d} followed by a contraction when errors are taken into account. In the singular regime, the vectors υ_i converge to c*_i/‖c*_i‖, whereas the singular values λ_i diverge. Therefore, in the continuum limit r → 0,

View Expanded

provided m⁻²/χ does not diverge faster than λ_i² (this issue is settled by the exponent α mentioned above and is therefore case dependent). In this limit, the averaging kernel is a mere projector onto the vector space generated by the adjoint solutions. If m⁻²/χ competes with one λ_i (or more) in this limit, then this projection is followed by a contraction.

As for the gain matrix 𝗞 = (𝗕⁻¹ + 𝗛^T𝗥⁻¹𝗛)⁻¹𝗛^T𝗥⁻¹, if 𝗨 = (u₁, … , u_d), then

View Expanded

In the singular regime, the vectors u_i converge to e_i, where (e₁, … , e_d) is the canonical basis of the physical space. Therefore, in the continuum limit,

View Expanded

so that the gain 𝗞 goes to zero in the singular limit case. To estimate the performance of data assimilation within a single event context (only one set of measurements μ), an objective root-mean-square-like score (Bocquet 2005a; Krysta and Bocquet 2007) is given by

View Expanded

where σ_t is the true control state, σ_b is the first guess, and ϵ_t is the true errors set. It can be shown that 0 ≤ ρ ≤ 1 (Bocquet 2005a). The more resolved the control state is, the weaker the image of σ_t − σ_b through the averaging kernel, unless all the mass of the state vector concentrates on the observation sites. In the singular limit case, for a randomly chosen σ_t, one almost surely has the following:

View Expanded

In the control space, the information matrix 𝗕⁻¹ + 𝗛^T𝗥⁻¹𝗛 should also hint at this singular behavior. The question is whether the confidence that is attached to 𝗥⁻¹ is or is not propagating through the action of 𝗛. This is clearly the case in the absence of singularity as follows:

View Expanded

The singular regime leads to a clear divergence in the confidence. As r goes to zero, the adjoint solution is more and more peaked near the observation sites so that the propagation is prominent in an increasingly close neighborhood. The average gain of information in the analysis is supporting this:

View Expanded

Thus, paradoxically, the gain of information diverges in the singular regime. A likely reason is that when the sources are localized in the vicinity of the observation sites, the inference becomes very informative compared to the prior assumption of an omnipotent emission field. The averaging kernel Eq. (10) tells what one can really hope for in this limit: it converges to a projector on the observation site support. This projector has no spread in the space–time domain. The normalized information attached to an observation will not propagate to the distributed control space. It will only resolve the vicinity of the observation sites.

Although it depends on scale, the analysis is not flawed. However, it does depend on the magnification chosen to investigate the control space, which, depending on the analysis purpose, may be inappropriate. If the objective is to tell anything about the whole distributed control space, then this situation is not desirable and the high-resolution limit should be avoided. To a lesser extent, this conclusion also applies to data assimilation systems of the nonsingular regime.

3) BLUE estimators in the two regimes

The generic BLUE results depend on two typical behaviors. First, suppose that the limit of the innovation statistics matrix 𝗛𝗕𝗛^T is well defined. Then both the analysis error covariance matrix and the estimator tend to a proper nontrivial limit. The continuum limit of the analysis covariance matrix is still given by Eq. (1), and the estimator has the following usual form:

View Expanded

Even though there is a finite continuum limit for the data assimilation problem, the analysis performance may degrade when the resolution is decreased, down to a lower nonzero threshold. This has been checked in Bocquet (2005a) and Furbish et al. (2008). In the later paper model, error is also taken into account. On the contrary, when 𝗛𝗕𝗛^T is divergent, the posterior matrix then merely tends to the background covariance; that is, 𝗕^1/2𝗛^T(𝗥 + 𝗛𝗕𝗛^T)⁻¹𝗛𝗕^1/2 tends to a less and less relevant operator in ℝ^N×N, along with the BLUE gain matrix. Thus, almost surely,

View Expanded

where σ_t is a randomly chosen true state of the geophysical system.

a. Simple criterion, simpler problem

We shall first drastically simplify the matter by fixing the number of parameters to be controlled. It is chosen on the order of the number of observations: a few control variables for one observation at most. Matching the resolution of a regular grid on Ω to the fixed number of cells is too rigid an approach, especially for distributed geophysical systems with very inhomogeneous control parameters. If the control space is multidimensional (three-dimensional for this paper application), a few hundred control parameters to match a few hundred observations is likely to be insufficient to describe the multidimensional distributed system through a regular grid.

Hence, the number of control parameters is fixed but not the partition of the control space that defines them. Call ω this representation, which is a partition of the space–time domain Ω. Here, ω is a set of (unnecessarily regular) cells that fully covers the space–time control domain (which is a 2D + T manifold for the typical application here). Besides, a point in the control domain Ω belongs to one and only one cell of the representation.

The criterion that we could choose to select a particular representation would consist in maximizing the total posterior confidence [i.e., the trace of the Fisher information matrix Tr(𝗣_a⁻¹)] at a fixed number of control parameters. To make the criterion dimensionless and coordinate free and following the discussion of section 2, we prefer the closely related J = Tr(𝗕𝗣_a⁻¹ − 𝗜_N). It measures the gain in confidence obtained from observation relatively to the initial confidence in the background. It also reads J = Tr(𝗕𝗛^T𝗥⁻¹𝗛). The physical interpretation is clear; confidence in the measurements 𝗥⁻¹ is propagated by the model 𝗛 and measured in the basis for which the background confidence 𝗕⁻¹ is standardized. It is also connected to the related information gain via Eq. (4).

b. Space–time multiscale grid and Jacobian matrix

To perform multiscale data assimilation, the Jacobian matrix 𝗛 needs to be defined at several scales. The reference scale will be the scale that characterizes the finest available regular grid of size (N_x, N_y, N_t), whose total number of space and time cells is N_fg ≡ N = N_xN_yN_t. In this paper, physics at other scales will be derived from this reference scale by simple arithmetic averaging. However, the formalism developed in this paper could also accommodate physics that is based on more complex coarse-graining approaches with dedicated subgrid parameterizations that depend on the scale.

Recall that the physics of the problem is encoded in μ = 𝗛σ + ϵ, which is written in the reference regular grid. This equation could be written at a different scale or more generally with components of σ that could be defined at different scales. In our context, one can think about coarse-graining components of σ in ℝ^N_fg at the reference scale into a coarser σ_ω in ℝ^N(N ≤ N_fg), whose components are unions of size-varying blocks of components of σ.

We have chosen to precalculate coarse-grained versions of the finest Jacobian obtained from numerical models on the reference regular grid of the control domain Ω (other choices are possible, such as computing them on the fly). As a result, a multiscale Jacobian made of coarse-grained versions of the original Jacobian on the finest regular grid is stored in computer memory. This precomputation saves time in the optimizations. However, the size of this multiscale Jacobian matrix is a requirement to consider in practice, so that an estimation of its maximum occupancy will be given below.

The multiscale extension consists in recursive dyadic coarse grainings of the original finest regular grid. By a basic coarse graining in one direction, two adjacent grid cells are gathered into one and the physical quantities (Jacobian components) defined on them are averaged accordingly. For each direction, one considers a predefined number of scales: n_x, n_y, and n_t for the application ahead. For each vector of scales l = (l_x, l_y, l_t) such that 0 ≤ l_x < n_x, 0 ≤ l_y < n_y, and 0 ≤ l_t < n_t, there is a corresponding coarse-grained version of the finest grid, with N_fg 2^{−lx−ly−lt} coarse cells (also called tiles) and a related coarse-grained Jacobian matrix (we assume N_x, N_y, and N_t are multiples of 2^n_x, 2^n_y, and 2^n_t, respectively). The full multiscale grid on Ω is the union of all these coarser grids. The total number of tiles of the multiscale grid is 8N_fg(1 − 2^−n_x)(1 − 2^−n_y)(1 − 2^−n_t). Obviously, that is also the dimension of the space that the multiscale Jacobian matrix acts on. At worst, the multiscale grid is 8 times larger than the finest regular grid, which could be costly in terms of memory. A graphical representation of the memory occupancy is pictured in Fig. 1a.

c. Adaptive tiling

Optimizations on the dictionary R(Ω) of representations require that this set be handled mathematically and computationally. The following formalism is based on the multiscale grid detailed in the previous section.

In the representations ahead, a distinction is being made between space and time dimensions. Following the dyadic structure of the multiscale grid, the partition of time line is implemented by a binary tree T with n_t levels.

In a first attempt and following a 2D + T example, the partition of the 2D space domain S_xy was built up because of a “quaternary tree” with n_s levels. It means that each 2D space cell could be divided into four subcells, down to a depth of n_s levels (see Knuth 1997; Fig. 1b). A tensorial product of the time and space structures was then peformed. Two natural distinct sets of representations can be obtained: or . As far as computer memory saving is concerned, these sets are rather economical. A (2D + T) emission field would be represented with 8/3N_fg(1 − 4^−n_s)(1 − 2^−n_t) scalars, in both multiscale structures. In the worst case (full downscaling), this involves a factor 8/3 as compared to the finest regular grid representation. However these representations are quite demanding in terms of coding. In particular, parsing efficiently all tree configurations in an optimization process can be very time-consuming.

That is why, in this paper, the set of representations that encompass all space–time “tilings” of a domain, based on the multiscale grid and Jacobian matrix defined earlier, is chosen instead. This set is the tensorial product of dyadic (binary) trees on time direction, O_t, and the spatial directions, O_x and O_y. It is a richer space than the two previous double-tree structures, but it is also much more memory consuming because the total number of cells to be run through is 8N_fg(1 − 2^−n_x)(1 − 2^−n_y)(1 − 2^−n_t). It corresponds to all grid cells, or tiles in this context, of the multiscale grid. At worst, it is 8 times larger than the finest regular grid. Again, this could be avoided if the Jacobian 𝗛 is not stored in memory, or as a compromise a quaternary tree representation is chosen instead.

A schematic of a double-tree partition is displayed in Fig. 1b, and a schematic of a tiling partition (triple tree) is displayed in Fig. 1c. Note that tensorial products of space and time structures are preferred to Cartesian products because the resulting dictionary of representations is much richer. Cartesian products would have implied separation of the space and time degrees of freedom, which is too poor an assumption.

d. Vector space mathematical representation

The dictionary R(Ω) of representations has been described geometrically. We shall now describe it algebraically.

Let us consider the vector space ℝ^N_fg attached to the finest grid of dimension N_fg. To each tile is attached a vector υ_l,k in this space, in which l = (l_x, l_y, l_t) is the set of scales of the tile and k is the index of this tile in the set of all tiles of the same scale l (one coarse version of the original grid). For the finest grid, one has l = (0, 0, 0) and k = 1, … , N_fg. The cells of the finest grid (smallest tiles) are represented by the canonical basis of this vector space. A coarser tile, made of these finest tiles, has vector υ_l,k ∈ ℝ^N_fg, defined as the sum of the vectors of the canonical vectors {e_i,j,h} in ℝ^N_fg with 1 ≤ i ≤ N_x, 1 ≤ j ≤ N_y, and 1 ≤ h ≤ N_t that are in one-to-one correspondence with the finest tiles that make up this tile: , where (i_k, j_k, h_k) are the coordinates on the finest grid of the tile indexed by k in the coarse-grained representation of Ω at level l.

A tiling ω of Ω is a partition, in the strict mathematical sense, of the space and time control space with tiles. For each tiling ω in R(Ω), we define an operator Π_ω : ℝ^N_fg → ℝ^N_fg built with the vectors attached to the tiles as follows:

View Expanded

where n_l is the number of tiles in the set of tiles with scale vector l; l runs on all predefined scales 0 ≤ l_x < n_x, 0 ≤ l_y < n_y, and 0 ≤ l_t < n_t; and are coefficients that define the representation ω. From now on, the superscript ω on will be dropped to simplify the notation.

In a multigrid approach, a location in space and time is represented several times. On the contrary, in a single representation of Ω, such a point is accounted for once, according to the strict definition of a partition. In this paper context, a point is a cell of the finest reference grid. A representation corresponding to a strict partition will be qualified as admissible. To obtain an admissible representation of the control space, one requires that α_l,k be 0 or 1 for all (l, k) and that any point in the domain be represented by one and only one tile:

View Expanded

Furthermore, we impose that the number N of variables (hence tiles) be set as follows:

View Expanded

Obviously, for an admissible representation, N_cg ≤ N ≤ N_fg, where N_cg = N_fg2^{−n_x−n_y−n_t}. On these conditions, it is not difficult to check that Π_ω = Π_ω^T and (Π_ω)² = Π_ω so that Π_ω is an orthogonal projector.

e. Restriction and prolongation

For a given representation ω, σ is coarse-grained into σ_ω = 𝗜_ωσ, where 𝗜_ω : ℝ^N_fg → ℝ^N is a restriction map that defines the coarse-graining operation. Although its precise definition depends on the context, it is always unambiguously defined. The prolongation operator 𝗜*_ω : ℝ^N → ℝ^N_fg relates σ_ω back to σ. Contrary to the restriction operator, several consistent definitions can be used for the prolongation operator. However, it should consistently satisfy 𝗜_ω𝗜*_ω = 𝗜_N (for a discussion of these operators, see Rodgers 2000). In addition, we impose that 𝗜_ω*𝗜_ω = Π_ω. This identity means that the net effect of coarse graining followed by an interpolation back in the finest grid is a local averaging on the area covered by each tile. Note that what follows does not depend on the precise definition of 𝗜*_ω. Then 𝗛 is coarse-grained into 𝗛_ω = 𝗛𝗜*_ω, and the coarse-grained observation equation μ = 𝗛_ωσ_ω + ϵ is made explicit in the finest grid vector space, because of the projection operator Π_ω:

View Expanded

In principle, ϵ should be made dependent on ω through the representativeness and model errors. Besides, whereas 𝗛_ω could be obtained from scale-dependent models at several scales, it is a mere coarse graining of a finescale Jacobian here. These important issues are beyond the scope of this paper, but the formalism developed ahead should accommodate them.

This allows to define a mathematically explicit multiscale criterion that depends on the choice of the representation ω ∈ R(Ω):

View Expanded

The background error covariance matrix 𝗕 has been coarse-grained into 𝗕_ω = 𝗜ω𝗕𝗜_ω^T. In the following, 𝗕 will be again assumed of the form m²𝗜_Nfg so that it commutes with Π_ω. As a consequence, . This formula can be generalized to any positive definite 𝗕, provided that Π_ω is generalized to a projector which is 𝗕⁻¹ orthogonal. This is not reported here because the application of section 4 does not require it.

The objective is to maximize the average reduction of uncertainty in the analysis J_ω on all admissible tilings (i.e., on the α_l,k that satisfy the above conditions).

f. Lagrangian approach

To optimize J_ω on ω, a Lagrangian is written with the above conditions implemented because of Lagrange multipliers. A fixed number of tiles is imposed from a single multiplier ζ. The one point–one tile requirement is imposed because of a vector λ of N_fg multipliers. The Lagrangian reads as follows:

View Expanded

Here, 𝗪 stands for 𝗕𝗛^T𝗥⁻¹𝗛 or any other appropriate criterion. The sum on k = 1, … , n₀ = N_fg runs on all cells of the finest grid. In this sum, α_{l, k̃} is the coefficient attached to the tile at scale l that covers cell k from the finest grid. This tile has rank k̃ among the n_l tiles related to scale l. The Lagrangian can also be written as follows:

View Expanded

Then, the maximum can formally be taken on all representations, admissible or not, with any number of tiles in [N_cg, N_fg], on the set of coefficients α_l,k that have been freed from the constraints through the multipliers. This is made easier (to a limited extent) by the fact that α_l,k can only be 0 or 1. Then, if ,

View Expanded

Because this cost function is dual to L(ω), it needs to be minimized, not maximized. Because it depends on λ and ζ only, its evaluation is not too time consuming. Its complexity scales like an optimization on the finest gridcell variables. However, the cost function is not smooth because it is nonderivable on edges of a polytope. Hence, optimization on the Lagrange parameters cannot make direct use of gradient-based minimization techniques. Besides, it is unlikely that this function is convex; nor is it guaranteed that it has a single minimum. To overcome these potential problems, a regularization of this effective cost function is proposed.

g. Statistical physics interpretation

From now on, is denoted by ϵ_l,k. If 𝗪 = 𝗕𝗣_a⁻¹ − 𝗜_Nfg, then ϵ_l,k is a measure of the extra confidence in the posterior estimate of the scalar defined on tile (l, k) resulting from the observations. Then the criterion can be written as , where Σ_l,k is shorthand for . Building on a statistical mechanics analogy introduced by Jaynes (1957a,b), we interpret the ϵ_l,k as individual energies, one for each tile. The system is thermalized at inverse temperature β, which is the regularization parameter. The average number of cells of the system is set to N. Also, any point must be covered by an average of one, and only one, tile. From statistical mechanics, the partition function of this system is

View Expanded

where λ and ζ are conjugate parameters to the occupation constraint and to the tiles number. Because the number of tiles is fixed and because a point is covered only once, the right statistical potential is the free energy functional that derives from the partition function:

View Expanded

By construction, this free energy function is strictly convex, which guarantees the existence of a single minimum, provided the constraints can be satisfied by at least one configuration ω in R(Ω). In particular, N needs to satisfy N_cg ≤ N ≤ N_fg. This allows for the use of classical optimization tools.

The minimization of the free energy yields λ and . Then, it is possible to compute the average filling factors:

View Expanded

which is essentially the main result of the optimization. The total reduction of uncertainty,

View Expanded

is given by the minimum of the free energy with an average reduction of ϵ_l,kα_l,k per tile.

This approach has been checked to be a consistent regularization of our main optimization problem. In the low temperature limit, β → +∞, the variables can be rescaled by β and the cost function becomes

View Expanded

and the Lagrangian Eq. (26) is recovered, up to β. Hence, this statistical approach is a way to regularize our problem at a small but finite temperature. Any other regularization is possible but this one offers a statistical interpretation and a guarantee of convexity.

A similar Lagrangian to Eq. (26) has been obtained in the case of binary–quaternary trees representations, as well as an analog formalism for this set of representations. This will be reported elsewhere because no application is given for it next.

The optimal structure obtained from this technique is also the least committed representation given that all the constraints are satisfied on average. At finite temperature, the optimal representation is a compromise between the criterion and the relative entropy of the representation as compared to the (nonadmissible) geometry in which all tiles are equiprobable. The proof is not essential here and will be reported elsewhere.

h. Properties

A cost function Eq. (28) has been introduced to qualify representations. Several aspects of its minimization will be discussed now.

2) Links with the concept of resolution

In data assimilation, a concept of resolution quantifies the ability of the observations to locally resolve the control parameter space. It is notably applied in the assimilation of radiances for the reconstruction of profiles in the atmosphere (Purser and Huang 1993; Rodgers 2000). However, its quantification is empirical. As seen in section 2, the trace of the averaging kernel 𝗔 is a measure of the gain of information because of the observations. The diagonal elements of the averaging kernel are a local gain of information within the control space. The inverse of each diagonal element is then an empirical measure of the local resolution, or precision, in control space.

A similar but nonempirical concept can be derived from our formalism. For each point k, 1 ≤ k ≤ N_fg (cell in the finest reference grid), a mean size of the local tile S_k can be obtained, along with a measure of the average resolution _k:

View Expanded

where A is the volume of the tile, which would scale as 2^l_x+l_y+l_t in our dyadic representation in the units of cells of the finest grid. (For the sake of simplicity, the volumes are not converted in distance and time units.) The definition of S_k may be unpractical because the size of the tile grows geometrically in our multiscale coarse-graining picture. Therefore, either ln(S_k) or another average, a “mean logarithmic size,” can be used: s_k = Σ_lα_l,k̃(l_x + l_y + l_t). This definition of the resolution implies that 0 ≤ _k ≤ 1. The best achievable resolution _k = 1 means that the observations allow us to determine the parameter attached to the local cell in the finest reference grid. Similarly, one has 0 ≤ s_k ≤ n_x + n_y + n_t − 3 and A(0,0,0) ≤ S_k ≤ A(n_x − 1, n_y − 1, n_z − 1), with the lower bounds corresponding to the finer scale.

a. Finding an optimal representation

This application is taken from (Bocquet 2005a,b,c, 2007) and is focused on source inversion of the European Tracer Experiment (ETEX). This European Joint Researcher Centre 1994 campaign (Girardi et al. 1998) consisted of a pointwise 12-h-long release into the atmosphere of an inert tracer at continental scale. In the papers mentioned earlier, several three-dimensional inversions of the source were attempted successfully on either the synthetic or the real concentration measurements from the campaign. However, these inversions were based on a regular discretization of the 2D + T source space. These meshes comprised from a few thousand to one million control variables (from 2.25° × 2.25° to 0.5625° × 0.5625°).

The aim is to construct an optimal adaptive representation (a tiling) for inversions based on a much more limited number of cells, using the mathematical formalism developed in section 3.

As proposed in section 3, the gain in confidence J_ω = Tr(𝗕_ω𝗛_ω^T𝗥⁻¹𝗛ω) (or reduction in uncertainty) will be optimized on a dictionary of tilings, with the following multiscale structure: The finest grid has dimensions N_x = 64, N_y = 32, and N_t = 160 for a spacing of Δ_x = Δ_y = 0.5625° and Δ_t = 1 h, whereas n_x = 5 scales are used for O_x, n_y = 5 scales are used for O_y, and n_t = 5 scales are used for O_t. The inverse temperature is set to β = 10⁵.

In the following, a small set of 201 observations is used. It was diagnosed to have a high information content for the inversion (Bocquet 2008). The number of required tiles (402) is twice the number of observations, which is very limited when compared to the number of cells of regular grids.

The resulting mean logarithmic tile size s_k is displayed in Fig. 2. A suboptimal admissible tiling of 403 tiles is displayed in Fig. 3. Its reduction of uncertainty is J_ω = 802.98, whereas the optimal reduction of uncertainty is J = 803.89. For the coarsest grid in this multiscale structure, J_ω = 23.42, whereas for the finest grid, J_ω = 1324.74. The latter is the maximum over all configurations for all N, which is due to the scaling divergence described in section 2.

Note how small the number of tiles N = 402 is compared to N_fg = 327 680. Nonetheless, 60% of the average gain of confidence would potentially be captured using this optimal representation compared to the choice of the finest grid. The average reduction of uncertainty for optimal representations with a tile number in the range [80, 327 680] is plotted on Fig. 4. This graph emphasizes how well performing an adaptive representation of a very limited number of cells can be for the purpose of data assimilation, especially in comparison to regular grid representations.

b. Inverse modeling using the optimal representation

Inverse modeling will now be performed on the basis of the optimal representation (adaptive grid) previously obtained (Fig. 3). We follow the methodology of Bocquet (2007) and Krysta et al. (2008), which was based on a nonoptimal regular grid. The difference is that the inversion is now based on the adaptive optimal grid for N = 402.

A standard Gaussian prior for the source is chosen from the generic form, Eq. (8). However, it is now defined on an adaptive grid . As mentioned earlier, 𝗕_ω = E_νω[σ_ωσ_ω^T] is related to 𝗕 through 𝗕_ω = 𝗜_ω𝗕𝗜_ω^T. We assume that 𝗕 = m²𝗜_Nfg, where m sets the release rate magnitude. Here, m = 0.025M/12 is chosen arbitrarily, where M = 340 kg is the true released mass of tracer in ETEX-I and 12 is the number of time steps of the true release in the finest grid. The errors are also chosen from the Gaussian a priori: , where the error covariance matrix is diagonal and homoscedastic 𝗥 = χ𝗜_d, where χ is the only degree of freedom left to weight the departure from the observations and the departure from the prior.

The cost function, the by-product of the Bayesian inference, is then as follows:

View Expanded

which leads to the usual normal equations and estimators of the BLUE analysis. Here, ω is the conditional representation that has been obtained earlier. The value of χ is selected from an L-curve analysis following Davoine and Bocquet (2007) and Krysta et al. (2008): χ = 0.3 ng m⁻³, which is equal to the value obtained from the regular grid (2.25° × 2.25° × 1h).

The integrated map of the reconstructed source is plotted on Fig. 5, as well as the retrieved profile in the vicinity of the true released site. The total retrieved mass of tracer is 879 kg, as compared to 684 kg obtained on a regular grid (Krysta et al. 2008). It has been diagnosed that 242 kg of tracer have been released in the vicinity of the true release site, as compared to 241 kg in Krysta et al. (2008). Therefore, the inversion result is very similar to the inversion on the regular grid but at a much lower cost. A difference is the excess in the total mass estimate, which can be attributed to observation–unconstrained aggregation error resulting from large tiles over the Atlantic and Ireland. Note that in Krysta et al. (2008), 81 920 and 20 480 control variables are solved for, as compared to 403 tiles here. On the practical side, optimizations are significantly accelerated.

We have also tested this representation within a non-Gaussian prior framework. A Bernoulli prior, which ensures the positivity of the source, has been selected. The two parameters that enter the inversion have been selected using an iterative L-curve approach. As reported in Krysta et al. (2008), on a regular grid, this prior leads to a better estimate of the source than the Gaussian prior. However the profile is less satisfying, with significant fluctuations within the true release period.

The results obtained on the optimal adaptive grid are displayed in Fig. 5. The profile is smoother than with a regular grid. This is because the optimal representation determines the proper balance between the space and time scales. Given N = 402, the optimal time step is about 3–4 h.