a. The IVDL scheme

Note that Eq. (3) presents a mathematically consistent interpretation of the approach in Wang et al. (2008). In their implementation, they use recursive filters instead of the eigen-approach mentioned above and apply the same transform to each variable, i.e., ${\mathsf{U}}_p^{\alpha }={\mathsf{U}}_q^{\alpha }={\mathsf{U}}_r^{\alpha }$. This precludes the possibility of using a different length scale for each variable (although this can be relaxed if required). In the L22 approach, however, a different spatial correlation matrix for each variable is used to achieve variable-dependent localization but forces full multivariate localization. Note that extra memory cost is required to store the eigenvectors and eigenvalues for each variable. In our implementation, this is the case even if, e.g., two variables share the same h^α; the same eigenvectors and eigenvalues are stored twice—once for each variable.

One should also note that with the current approach—using eigenvectors decomposed from correlation matrices—special care must be taken when dealing with periodic domains since the correlation matrix is circulant and may not be positive semidefinite. The implication is that if the localization length scales for p and q are different and any eigenvectors associated with negative eigenvalues are truncated, ${\mathsf{U}}_p^{\alpha }{\mathsf{U}}_q^{\alpha \mathrm{T}}$ and ${\mathsf{U}}_q^{\alpha }{\mathsf{U}}_p^{\alpha \mathrm{T}}$ (the associated off-diagonal blocks) will not strictly be cross-correlation matrices (not shown). The off-diagonal block matrices with this extra symmetry are explored with SVDL in the next section.

Here, we have described IVDL in detail to provide clarity on how one might technically implement it in an NWP system. This approach is outlined in Wang and Wang (2023) and is referred to as basic scale-dependent localization variable-dependent localization Basic-SDLVDL (with scale-dependent aspects in their case), but they implemented and tested a variant minor extension scale-dependent localization variable-dependent localization (MinorE-SDLVDL) instead. Mathematically, Basic-SDLVDL and MinorE-SDLVDL are equivalent (Wang and Wang 2023). Prior to this study, L22 had already discussed the technical implementation of IVDL (although not named IVDL then) and the approach to apply alpha fields to one or all variables, along with the proof of equivalence. Another more computationally efficient approach has since been proposed by Menetrier (https://doi.org/10.5281/zenodo.7547230).

b. The SVDL scheme

We note that while the full localization matrix is always symmetric, the off-diagonal blocks of $\mathsf{L}$ are not themselves symmetric if the localization length scales (and hence transforms) for p and q are different (i.e., ${\mathsf{U}}_p^{\alpha }{\mathsf{U}}_q^{\alpha \mathrm{T}}\ne {\mathsf{U}}_q^{\alpha }{\mathsf{U}}_p^{\alpha \mathrm{T}}$). Buehner and Shlyaeva (2015) also found the asymmetry in their between-scale cross-covariances when applying scale-dependent localization. This is not necessarily a criticism of the existing approaches but is rather a prompt to pose the question if imposing extra symmetry in $\mathsf{U}$^α could improve the performance of a multivariate localization scheme.

To prescribe the off-diagonal correlation matrices (e.g., $\mathsf{L}$_p,q), the average correlation length scale ${\overline{h}}^{\alpha }$ of two associated variables (p and q in this example) is computed, which is then used as the length scale in the Gaspari–Cohn localization function to construct $\mathsf{L}$_p,q. Other approaches may also be considered instead of using ${\overline{h}}^{\alpha }$, e.g., computing localization functions separately and taking their average. As the off-diagonal matrices are constructed like autocorrelation matrices, they are symmetric, unlike in IVDL. Additionally, each off-diagonal matrix pair (e.g., $\mathsf{L}$_p,q and $\mathsf{L}$_q,p) uses exactly the same ${\overline{h}}^{\alpha }$ to construct the localization function, so the full localization matrix is automatically symmetric. Furthermore, to apply selective multivariate localization, one could set selective off-diagonal correlation matrices of $\mathsf{L}$ to $\mathsf{0}$, similar to Eq. (8).

It is also important to note that in SVDL, $\mathsf{L}$ is constructed with block correlation matrices (or with $\mathsf{0}$ in selected off-diagonal blocks) but may not be a correlation matrix as a whole. The implication is that without further adjustment, it is not possible to guarantee positive semidefiniteness. Stanley et al. (2021) proposed how one might prescribe the off-diagonal correlation matrices such that the implied $\mathsf{L}$ is positive semidefinite. SVDL does not use their approach; this is to maintain flexibility to prescribe some of the off-diagonal blocks to $\mathsf{0}$. This also means that an extra step is required to guarantee positive semidefiniteness. Following L22, any negative eigenvalues are truncated and the remaining eigenvalues are rescaled (e.g., by a uniform factor given by the ratio of the original sum of eigenvalues to the sum of eigenvalues after truncation) to restore the original total variance. Figure 1 shows how variable-dependent localization can be explicitly prescribed in SVDL, ensuring that the off-diagonal block matrices are symmetric. After decomposing the original localization matrix and reconstructing, the implied localization matrix is not identical to the original. Even with rescaling, the truncation of negative eigenvalues has the effect of damping, particularly on the cross correlations. The kurtosis of the correlations is also slightly altered. This effect is more severe for periodic domains where circulant matrices may be involved. There may be alternatives to using a uniform factor to rescale, but they are not investigated here.

View Full Size

Full localization matrix [Eq. (9)] with variable-dependent localization using SVDL for a one-dimensional periodic domain of 50 points and three variables (p, q, and r; localization length scale of 5, 10, and 15 points, respectively). (top left) The original matrix is prescribed explicitly, whereas (top right) the implied matrix is reconstructed from eigenvectors after truncating negative eigenvalues and rescaling. Autocorrelations and cross correlations with respect to the midpoint (index 25) of variable p are shown for the (bottom left) original matrix and (bottom right) implied matrix. The black dotted lines in the bottom panels are at value 1, which is the desired value of the peak correlations.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

SVDL allows full control of the localization, including multivariate error cross-covariances, but is computationally expensive. As mentioned, it does enforce symmetry in the off-diagonal correlation matrices unlike previous approaches (IVDL or in Wang and Wang 2023) and allows the specification of null matrices on certain cross-correlation components unlike in Stanley et al. (2021). Nevertheless, it remains to be seen if symmetry is beneficial as there may not be a physical justification (see section 3b for figures illustrating selective multivariate localization with IVDL and the differences between IVDL and SVDL). Due to the expensive—but flexible—formulation of SVDL, one could also easily apply a cross-localization weight factor to diminish the error cross-covariances, similar to that discussed in Stanley et al. (2021), but this is not investigated here.

a. Development of the ABC-DA system

To evaluate variable-dependent and selective multivariate localization for the tropics, we use the ABC model (Petrie et al. 2017), which solves a modified set of the compressible Euler equations. This model uses a vertical slice formulation (a two-dimensional longitude–height plane) and contains only dry dynamics. It is named after its key parameters: the pure gravity wave frequency A, the controller of acoustic wave speed B, and the constant of proportionality between pressure and density perturbations C. Additionally, a Coriolis parameter f can be set based on the desired latitudinal position of the chosen longitude–height plane. This allows for a deep tropical environment to be mimicked by selecting a very small value for f. In this configuration, a value of f = 10⁻⁵ s⁻¹ is used. This corresponds approximately to a value of f at a latitude of 4°N. The other model parameters are also set as A = 0.02 s⁻¹, B = 0.01, and C = 10⁴ m² s⁻². There are five prognostic variables: zonal wind u, meridional wind υ, vertical wind w, scaled density perturbation ${\tilde{\rho }}^{\prime }$ (a pressure-like variable), and buoyancy perturbation b′ (a potential temperature–like variable), which govern the model dynamics. The ABC model is thus sufficiently complex as a multivariate dynamical system, while retaining simplicity to expedite research and development.

The associated data assimilation was introduced in Bannister (2020), supporting incremental 3D-Var and 3D-Var first guess at appropriate time (3D-Var-FGAT). The system is solely based on variational data assimilation. Initial implementation had an arguably crude form of generating an ensemble by considering multiple latitudinal slices from a three-dimensional operational model’s output file (a version of the Unified Model). This was used for calibrating the background error covariance matrix, and no ensemble-based methods (e.g., ensemble Kalman filter or square root filters) were implemented. Further development of the ABC-DA system by L22 introduced hybrid ensemble–variational data assimilation via the alpha control variable transform approach (Lorenc 2003). Concurrently, L22 also introduced other ensemble generation and propagation approaches. The random field perturbations method (Magnusson et al. 2009) was used to cold start an ensemble, and the ensemble bred vectors (EBV) method (Balci et al. 2012) was introduced to propagate the ensemble at each data assimilation cycle—this was computationally cheaper than the traditional ensemble Kalman filter or square root filters and did not suffer from filter collapse (see L22 for details). This parallel-run ensemble was necessary to support hybrid 3D-Var and hybrid 3D-Var-FGAT in the ABC-DA system. From the ensemble forecasts, the error modes ${x^{\prime }}_t^k$ can be computed and used with $\mathsf{U}$^α as in Eq. (1). Using these newly implemented features in the ABC-DA system, L22 showed that hybrid 3D-Var outperformed 3D-Var and pure EnVar methods in the ABC-DA configured for the tropical environment. However, for the purpose of this study, we will focus on the pure EnVar framework.

b. Illustration of IVDL and SVDL

Before exploring the research questions using variable-dependent and selective multivariate localization in assimilation experiments, we illustrate how IVDL and SVDL can control the localization with ABC model variables. First, selective multivariate localization is illustrated using IVDL. For demonstration, the state variables have been grouped into two sets: (i) u and υ; (ii) w, ${\tilde{\rho }}^{\prime }$, and b′. This means that variables in the same set retain their cross correlations (and thus cross-covariances after the Schur product with the ensemble-derived error covariances), but variables in different sets have cross correlations knocked out by localization. Figure 2 shows the implied localization functions with respect to u and ${\tilde{\rho }}^{\prime }$ points. It is clear that only u and υ cross correlations are retained in the first set, and w, ${\tilde{\rho }}^{\prime }$, and b′ cross correlations are retained in the second set. Between variables of different sets, no cross correlations are retained by the design of $\mathsf{U}$^α. Other grouping options have been implemented in the ABC-DA system, which we use to isolate important multivariate error relationships (see section 5a). This will enable us to explore question 2 on which multivariate error relationships are beneficial for EnVar data assimilation in the tropics.

View Full Size

Implied localization functions with respect to a point (yellow cross) using IVDL for cross correlations of all variables with respect to (a) u and (b) ${\tilde{\rho }}^{\prime }$. The state variables have been grouped into two sets: (i) u and υ; (ii) w, ${\tilde{\rho }}^{\prime }$, and b′ to illustrate selective multivariate localization.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

Next, variable-dependent localization is illustrated using both IVDL and SVDL. For demonstration, only the vertical localization length scale is changed between variables. Figure 3 shows the comparison of IVDL- and SVDL-implied localization functions with respect to u and b′ points. There are subtle differences in Figs. 3a and 3b due to the truncation of negative eigenvalues in SVDL. Additionally, note how there are differences in the u–b′ and b′–u cross correlations using IVDL (Figs. 3c,d; left), which is due to the asymmetry in the off-diagonal block matrices. Using SVDL on the other hand (Figs. 3c,d; right), the u–b′ and b′–u cross correlations are identical. As discussed in section 2b, it is not known a priori whether the extra symmetry imposed by SVDL is beneficial to data assimilation, but SVDL is certainly more flexible than IVDL. This will enable us to explore question 3 on whether variable-dependent spatial localization is beneficial for EnVar data assimilation in the tropics.

View Full Size

Implied localization functions with respect to a point (yellow cross) using (left) IVDL and (right) SVDL for autocorrelations of (a) u and (b) b′ and cross correlations of (c) b′ with respect to u and (d) u with respect to b′. The vertical localization length scales are larger in b′ to illustrate variable dependence.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

a. Setup for the ABC-DA system

To evaluate the performance of IVDL and SVDL in data assimilation to learn about tropical multivariate covariances and the best localization settings, we conduct a series of hourly cycling observation system simulation experiments (OSSEs) similar to those in L22. To represent the incompleteness of the observation network in an NWP system, only u, υ, and ${\tilde{\rho }}^{\prime }$ are observed at a set of points in a subdomain (longitudinal distance between 50 and 500 km; height between 9 and 14 km, i.e., the upper portion of vertical slice of 546-km length by 16-km height). The observation operator used is bilinear interpolation. This setup is more akin to an NWP system with only satellite-related point observations (e.g., satellite-derived wind) that are available in the upper troposphere and stratosphere. This setup may also accentuate the impact of selective multivariate localization because unobserved variable fields are updated solely on localized multivariate error cross-covariances. At each cycle, 100 observations of each of the abovementioned variables are assimilated. The observations are sampled from a “truth” run (using a time step of 4 s) with the same values of A, B, and C, but with added Gaussian noise based on the observation error standard deviation. For this setup, the observation error standard deviations are 0.1, 0.1, and 1.5 × 10⁻⁴ m s⁻¹, respectively. All observations are valid at the analysis time of each cycle and all experiments use the same observations.

The random field perturbations method (section 3.1 of L22) is first used to generate a 1000-member ensemble set (i.e., the pairs of states are sampled from a 150-day truth run with hourly outputs; 24 × 150 = 3600 available states). The number of available states has to be sufficiently large to avoid repetition in the pairs of states drawn (with replacement back into pool) by the random field perturbations method. The new 1000-member ensemble is centered around one randomly chosen ensemble analysis from the last cycle of the experiment labeled EBVd in L22 (cold start) and allowed to spin up for 75 cycles (75 h) so that the system would have lost memory of the cold start initialization using random field perturbations. At each cycle, the ensemble perturbations are updated using the EBV method, similar to L22. No inflation is used, but by definition of the EBV method, a scaling based on a fixed factor divided by the max norm of the perturbations from the previous cycle (see Balci et al. 2012 or L22 for details) is used to get the updated perturbations. After spinning up for 75 cycles, the ensemble perturbations from the last spinup cycle are used for the start of the experiments, computed according to the number of ensemble members chosen for a particular experiment (these are subsets of the 1000-member ensemble; see section 4b). For example, for a 1000-member ensemble, 999 perturbations are computed using the member-minus-mean approach; for a 100-member ensemble, the first 100 members are retained and 99 perturbations are computed. Each experiment in section 5 is run for 100 cycles.

b. Guidance on ensemble size for experiments

Figure 4 shows the degree of linear independence for each successive ensemble perturbation for the five prognostic variables at the start of the experiment. It reflects the capability of each new random field perturbation to sufficiently explore the additional direction in the subspace. This is more likely if the system dynamics develop with strong nonlinearities within the first hour (since we are using 1-h forecast ensemble perturbations from a randomly generated analysis ensemble). Based on Fig. 4, and using the 20-member rolling average in red, we note that the degree of independence of each successive ensemble perturbation varies for each variable. We use the ${\widehat{N}}_k=0.3$ level to help decide on whether a sufficient level of linear independence is reached. For w, the degree of independence remains above 0.3 threshold until after the 400th member. On the other hand, for υ, this occurs at about the 40th member. For u and ${\tilde{\rho }}^{\prime }$, the threshold is met after the 100th member, and for b′, the threshold is met at the 120th member. This suggests that nonlinearities largely develop in the w field when computing the “truth” run, which are captured by the random field perturbations method, and/or are evolved within the first hour. The nonlinearities captured by the random field perturbations method in other variables are weaker. The results suggest that if the ensemble size is larger than about 100, most of the ensemble perturbations would virtually be linear combinations of others and thus have limited impacts on EnVar data assimilation [since the analysis increment is a linear combination of perturbations, Eq. (1)]. This method does not, however, give any indication that the sampling error is sufficiently small for localization to be unnecessary. The results do highlight though how each field has its own characteristics based on the system dynamics, so variable-dependent localization in particular should be worth exploring.

View Full Size

Degree of linear independence ${\widehat{N}}_k$ of each successive ensemble perturbation for all five prognostic variables. Perturbations are valid at the start of the experiments. An arbitrary threshold of 0.3 is indicated by a gray dotted line. The 20-member rolling averages are indicated in red.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

We further conducted a sensitivity test to the number of ensemble members using the OSSE framework described above. We tried experiments with 10, 50, 100, 200, and 1000 members. Horizontal and vertical localization are not applied in this sensitivity test to reveal the impacts of sampling noise on the (raw) ensemble-derived error covariances. The runs are evaluated using the RMSE with respect to the truth run. This was the approach taken in Bannister (2020, 2021) and L22 to assess the performance of their experiments. It is also a straightforward metric to measure the deviation of the forecasts from the truth run.

Figure 5 contains the time series of RMSE for each variable over the 100 cycles and a summary of the cycle-averaged RMSE for each variable compared to the free background run (FreeBG, which is the reference forecast without any data assimilation starting from the cold start background state). It shows that in general, the cycle-averaged RMSE decreases as the number of ensemble members increases, particularly for u and ${\tilde{\rho }}^{\prime }$. The error reduction is about 2%–4% depending on the variable when the ensemble size is increased from 10 to 1000. The highest cycle-averaged RMSE is seen in the runs with 10 and 50 members, respectively, as expected due to the lack of localization to address sampling error. Also, the reduction in cycle-averaged RMSE between the runs with 200 and 1000 members is small compared to that between the runs with 10 and 50 members, for almost all variables. For w, the decrease in RMSE with an increasing number of ensemble members is less pronounced beyond 50 members. As w is highly nonlinear, it is unsurprising that the FreeBG deviates from the truth run more rapidly than the other variables and so the error increases over time.

View Full Size

All panels except bottom right show time series of root-mean-square analysis errors for the ensemble sensitivity experiments (10, 50, 100, 200, and 1000 members) and FreeBG. No localization is used in these experiments. The vertical yellow lines are the analysis times. Analysis errors are defined with respect to the “truth” run, computed every 10 min within the respective assimilation windows for experiments and every hour for the FreeBG. The bottom right panel shows the ratio of the cycle-averaged RMSE for each experiment with respect to the FreeBG for the five ABC model variables.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

Also, note the different cycle-averaged RMSE patterns for u and υ. This difference may be due to the two-dimensional nature of the ABC model (no latitude dependence). In this setup, vorticity (∂υ/∂x) is associated with υ and divergence (∂u/∂x) is associated with u. We would expect vorticity and divergence to have different time scales, which are indeed observed in the different cycle-averaged RMSE patterns for u and υ.

Next, we examine the raw ensemble-derived error covariances between u and ${\tilde{\rho }}^{\prime }$ using the ensemble perturbations drawn from the first cycle of the sensitivity test. Figure 6 shows how a selection of covariance structures changes as the number of ensemble members is increased. It shows that the ensemble-derived error covariances become less noisy, particularly in the u–u and ${\tilde{\rho }}^{\prime }\text{–}{\tilde{\rho }}^{\prime }$ autocovariances. Notably, the ensemble size threshold at which the covariance structures start to appear consistent is 100 members. Above the threshold (i.e., 200 and 1000 members), the structures do not differ substantially. We also note that with 1000 members, the u–u autocovariances contain wavelike patterns which result in nonnegligible longer-range spatial covariances. These patterns have previously been seen in Bannister (2020) and L22, albeit for ${\tilde{\rho }}^{\prime }$ autocovariances instead. The wavelike patterns suggest that the main error modes in the ABC-DA system could be strongly influenced by periodic waves resonating in the domain, as also noted by L22 (i.e., they are real features rather than artifacts of sampling error).

View Full Size

Raw ensemble-derived error autocovariances of (left) u, (center) cross-covariances of u with respect to ${\tilde{\rho }}^{\prime }$, and (right) autocovariances of ${\tilde{\rho }}^{\prime }$ as a function of the number of ensemble members N (increasing from top to bottom). Negative values have contours that are dashed, and contour intervals are nonuniform to elucidate any features. The covariances are computed with respect to a point (yellow cross) near the center of the domain. The ensemble perturbations are drawn from the first cycle of the sensitivity test.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

Given the results in Figs. 5 and 6, and weighing the computational costs of running very large (1000-member) ensembles, a suitable ensemble size of 100 is used in further experiments to explore the possibilities of variable-dependent and selective multivariate localization. This number of members is large enough to show coherent structures in the unlocalized covariances, but still shows evidence of sampling errors.

a. Exploring the importance of multivariate error relationships

In this section, we run EnVar data assimilation experiments to test different multivariate localization options. All data assimilation experiments start with the same initial background and ensemble perturbations as the 100-member experiment in section 4b. The observations are also the same. The details of the selective multivariate localization experiment variants are listed here. As a reminder, variables that are in different groups do not retain cross-covariances with variables in other groups. For example, in experiment 1c below, localization is used to ensure that u and υ errors are made to be completely uncorrelated with w, ${\tilde{\rho }}^{\prime }$, and b′ errors.

• 1a) One set: limiting case where all multivariate error cross-covariances are retained, as in traditional EnVar implementations.

• 1b) Five sets: limiting case where no multivariate error cross-covariances are retained (full intervariable localization).

• 1c) Two sets: (i) u and υ and (ii) w, ${\tilde{\rho }}^{\prime }$, and b′.

• 1d) Two sets: (i) υ, w, ${\tilde{\rho }}^{\prime }$, and b′ and (ii) u.

• 1e) Two sets: (i) u, υ, w, and b′ and (ii) ${\tilde{\rho }}^{\prime }$.

• 1f) Two sets: (i) u, w, ${\tilde{\rho }}^{\prime }$, and b′ and (ii) υ.

• 1g) Three sets: (i) u, ${\tilde{\rho }}^{\prime }$, and b′, (ii) υ, and (iii) w.

For this subsection, all variables use the same horizontal localization length scales of 20 km and use the IVDL scheme (see first row of Table 1 and section 5b for justification of choice). No vertical localization is used, following appendix C of L22 which showed that the vertical localization of ${\tilde{\rho }}^{\prime }$ and b′ could result in hydrostatic imbalances. Here, IVDL is first used to enable selective multivariate localization; variable dependence is incorporated in the next subsection.

Table 1.

Horizontal and vertical localization length scales $h_{\text{horiz}}^{\alpha }$ and $h_{\text{vert}}^{\alpha }$ for the experiments to evaluate variable-dependent localization. “NIL” means no localization is used in the relevant direction (horizontal, vertical). IVDL refers to the isolated variable-dependent localization scheme (section 2a) and SVDL refers to the symmetric variable-dependent localization scheme (section 2b).

View Table

To demonstrate the impact of selective multivariate localization on the analysis, the analysis increments for the first cycle of experiments 1a, 1b, 1c, 1d, and 1g are plotted in Fig. 7. These experiments represent the diverse possibilities and the implications of choosing a different number of sets and/or different number of variables in each set (experiments 1e and 1f are not shown as they are similar to 1d; partitioning into two sets: four variables and one variable). A comparison of successive experiment pairs allows for the impact of specific multivariate error relationships to be disentangled. For example, comparing experiments 1a and 1d reveals the impact of isolating u. Note that since EBV is used to propagate the ensemble (without data assimilation), the analysis increments shown are for the control member which assimilates the observations. Here, the impact of u observations on all other variables through the error cross-covariances is substantial; the analysis increments in experiment 1d are less widespread than in 1a, especially over unobserved regions. For the full intervariable localization limiting case (experiment 1b), there are analysis increments for observed variables only, as expected. The ${\tilde{\rho }}^{\prime }$ analysis increment patterns are broadly similar to those from experiments where ${\tilde{\rho }}^{\prime }$ is not influenced by u observations [i.e., experiments 1c, 1d, and 1e (latter not shown)]. Similarly, the υ increment patterns are broadly similar to those from experiments 1f (not shown) and 1g. When υ is isolated in experiment 1g, the impact on the analysis increments of other variables is subtle due to relatively small error cross-covariances associated with υ. Note that the widespread analysis increments in experiment 1a do not yet reveal if the multivariate error relationships associated with u are meaningful or undesirable. Unlike the midlatitudes, where the multivariate error covariances may be explained largely by geostrophic and hydrostatic theories, the essential multivariate error covariances in the tropics are not well known.

View Full Size

Analysis increments from the first cycle of experiments 1a, 1b, 1c, 1d, and 1g (shown from left to right; see row 1 of Table 1), for the five prognostic variables (shown from top to bottom). All experiments start with the same background ensemble, assimilate the same observations (of u, υ, and ${\tilde{\rho }}^{\prime }$, which are equally spaced within the yellow box), and use the same spatial localization. Selective multivariate localization is applied with IVDL; variables are partitioned into sets (see text for description), demarcated by underscores (e.g., $u\_w\upsilon {\tilde{\rho }}^{\prime }{b}^{\prime }$ refers to experiment 1d).

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

To identify important/beneficial multivariate error relationships in the ensemble-derived error covariances, we analyze the performance of each experiment over the course of 100 cycles, benchmarked against two limiting cases: (i) where all multivariate error cross-covariances are retained (experiment 1a) and (ii) where no multivariate error cross-covariances are retained (experiment 1b). Experiments that have smaller cycle-averaged RMSE than benchmark (i) suggest that certain multivariate error cross-covariances are not important and may be introducing more noise into the analysis than signal. Likewise, experiments that perform better than benchmark (ii) suggest that certain multivariate error cross-covariances are important/beneficial for EnVar data assimilation in the tropics. The experiments which isolate individual variables (1d, 1e, and 1f) are plotted in Fig. 8. The remaining experiments (1c and 1g) are plotted in Fig. 9.

View Full Size

As in Fig. 5, but for experiments 1a (EnVar; limiting case), 1b (EnVar-ivl; limiting case), 1d (EnVar-vwrb_u), 1e (EnVar-uvwb_r), and 1f (EnVar-uwrb_v) compared to the FreeBG.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

As in Fig. 5, but for experiments 1a (EnVar; limiting case), 1b (EnVar-ivl; limiting case), 1c (EnVar-uv_wrb), and 1g (EnVar-urb_v_w) compared to the FreeBG.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

The most salient feature from both figures is that the experiments have similar cycle-averaged RMSE to either experiment 1a or 1b (the two limiting cases), for all variables except w. This may be unsurprising based on the comparison of analysis increments in Fig. 7. Experiments 1c, 1d, and 1e have similar values as experiment 1b (except w), while experiments 1f and 1g have similar values as 1a. Note that experiment 1b has smaller cycle-averaged RMSE than 1a for all variables except w. The key points from the seven experiments are as follows.

• Experiments that remove the ${\tilde{\rho }}^{\prime }\text{–}{b}^{\prime }$ error cross-covariances (1b and 1e) lead to a substantially larger RMSE for w.

• Experiments that remove the $u\text{–}{\tilde{\rho }}^{\prime }$ error cross-covariances (1b, 1c, 1d, and 1e) lead to a notably smaller RMSE for two observed variables u and ${\tilde{\rho }}^{\prime }$ and one unobserved variable b′.

The first point relates to hydrostatic imbalances in the analysis. From the prognostic equations for w, there are source and sink terms which relate to b′ and the vertical gradient of ${\tilde{\rho }}^{\prime }$. Any imbalance between the two will result in changes to w as the system evolves. Clearly, this imbalance is undesirable, as seen from the experiments. Lorenc (2003) previously discussed this issue in the context of mass–wind balance. Here, we find that maintaining signals of hydrostatic balance where the ensemble forecasts are hydrostatically balanced—by retaining the ${\tilde{\rho }}^{\prime }\text{–}{b}^{\prime }$ error cross-covariances—is important/beneficial for the tropical ABC-DA system. Vetra-Carvalho et al. (2012) previously showed that for regions (in the United Kingdom) where convection was weak, hydrostatic balance holds very well. However, in regions where moist convection was involved, hydrostatic balance was not preserved. To generalize our results for the tropics, moist processes would need to be considered in the ABC model (Zhu and Bannister 2023), but that version does not yet have data assimilation incorporated. Notwithstanding the limitations, the tropical dry dynamics representing vertical wind (dry convection) and the mass–wind interactions are still relevant to explore within the ABC-DA system.

The second point relates to mass–wind sampling errors in the analysis. As discussed previously, there is no clear mass–wind balance relationship for the tropics and many centers implicitly treat mass and wind variables univariately in their climatological background error covariance matrices. Here, we find that explicitly treating the mass and wind variables univariately in EnVar data assimilation (by knocking out covariances between u and ${\tilde{\rho }}^{\prime }$) is beneficial for the tropical ABC-DA system. From Fig. 6, it appears that even with 100 members, the mass–wind error relationship still contains some sampling noise. Referencing earlier results from Fig. 7, the impact of u observations on other mass variables (${\tilde{\rho }}^{\prime }$ and b′) is likely negative when u-related error cross-covariances are retained. The results suggest that the mass–wind error relationship prescribed directly from a 100-member ensemble is not beneficial for the tropical ABC-DA system. It remains a challenge to find a scale-dependent balance between mass and wind errors for the tropics (e.g., handling the larger scales based on large-scale balances, but handling the smaller scales based on convective-scale balances, if they exist at all). Until then, our results suggest that they should be treated entirely univariately.

We further examined if other multivariate error relationships are important/beneficial in experiments 1f and 1g, but isolating υ and w does not have a substantial impact on the analysis increments nor cycle-averaged RMSE. We further repeated experiments 1a, 1c, 1d, and 1e two more times with different random seeds for the observations (not shown), arriving at the same conclusions. There is limited additional benefit of repeating the other experiments, since they would yield similar results (as seen above) which would lead to the same conclusions. The abovementioned two points therefore suggest that (i) where dry dynamics are concerned, hydrostatic balance is important for EnVar data assimilation in the tropics and (ii) treating mass and wind errors univariately is also beneficial for EnVar data assimilation in the tropics.

b. Exploring the benefits from variable-dependent spatial localization

Next, we explore the benefits of variable-dependent localization for EnVar data assimilation in the tropics by assimilating with different localization length scale values for different model variables. Thus far, all experiments have included only horizontal localization of uniform length scales. For some of the subsequent experiments, we use both horizontal and vertical localization of length scales $h_{\text{horiz}}^{\alpha }$ and $h_{\text{vert}}^{\alpha }$, respectively (see section 2 for localization function details, but applied separately for each spatial dimension). As before, the details of the variable-dependent localization experiment variants are listed here:

• 2a) As in experiment 1a, but applying SVDL and changing horizontal localization for horizontal wind variables only.

• 2b) As in experiment 1a, but applying SVDL and varying horizontal and vertical localization for wind variables.

• 2c) As in experiment 1c, applying IVDL but changing horizontal localization for horizontal wind variables only.

• 2d) As in experiment 1c, applying IVDL but changing horizontal and vertical localization for horizontal wind variables only.

For reference, Table 1 shows a summary of $h_{\text{horiz}}^{\alpha }$ and $h_{\text{vert}}^{\alpha }$ used for the experiments. The default length scales for experiments 1a–1g were determined based on the horizontal distance between adjacent observations (≈23 km), following L22. Experiments 2a and 2c implement variable-dependent horizontal localization only. This is to assess if the flexibility granted by horizontal localization alone is beneficial. Experiments 2b and 2d further implement variable-dependent vertical localization to assess if it is also beneficial. Experiments using SVDL (2a and 2b) retain all multivariate error relationships, so they are compared to experiment 1a as the benchmark. Experiments using IVDL (2c and 2d) require that variables in the same set use the same $h_{\text{horiz}}^{\alpha }$ (see issues with periodic domains highlighted in section 2a), and they are compared to experiment 1c as the benchmark. Experiment 1c is also suitable to be the benchmark since it was the best performing out of the selective multivariate localization experiments.

Figure 10 shows how variable-dependent localization changes the analysis increments for the first cycle for experiments 1a, 2a, 2b, 2c, and 2d. As expected, the u and υ analysis increments in experiments 2a and 2c have broader structures than before since $h_{\text{horiz}}^{\alpha }$ is larger. Similarly, when vertical localization is further introduced in experiments 2b and 2d, the u and υ analysis increments are confined to the observed regions. Note how the w, ${\tilde{\rho }}^{\prime }$, and b′ analysis increments are similar between experiments 2c and 2d despite changes to the vertical localization length scales of the wind variables. This highlights how both variable-dependent and selective multivariate localization can be simultaneously applied to the ensemble-derived error covariances (using IVDL) to constrain the observation impact on specific variables.

View Full Size

As in Fig. 7, but for experiments 1a (NoVarDep), 2a (SVDL-VarDepH), 2b (SVDL-VarDepHV), 2c (IVDL-VarDepH), and 2d (IVDL-VarDepHV).

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

Next, we examine the impacts of variable-dependent localization on the cycle-averaged RMSE. Figure 11 shows that the cycle-averaged RMSE for experiment 2a (when u and υ use different $h_{\text{horiz}}^{\alpha }$ from the rest of the variables) is marginally smaller than 1a for most variables. The oscillations in the u and ${\tilde{\rho }}^{\prime }$ RMSE evolution are also marginally more pronounced, likely related to the gravity waves (and their associated frequencies) within the domain. Similar results are seen comparing experiment 2c with 1c (Fig. 12); using different $h_{\text{horiz}}^{\alpha }$ for both u and υ leads to marginally improved forecasts, despite both experiments omitting error cross-covariances between u and ${\tilde{\rho }}^{\prime }$ (unlike in experiment 2a, which uses SVDL). When variable-dependent vertical localization is further introduced in experiments 2b and 2d, the cycle-averaged RMSEs in general are further reduced, even for b′ which is unobserved. Across all experiments, we find that 2d leads to the smallest cycle-averaged RMSE, about 3%–4% smaller (for u, ${\tilde{\rho }}^{\prime }$, and b′) compared to experiment 1a (the benchmark) which uses the widely adopted traditional localization approach. We have further repeated experiments 1a and 2d two more times with different random seeds for the observations. We have also repeated the experiments using a 50-member ensemble instead of a 100-member ensemble to ensure the conclusions are not heavily dependent on ensemble size. Figure 13 shows that the cycle-averaged RMSE differences of u, ${\tilde{\rho }}^{\prime }$, and b′ within groups of experiment 1a or 2d configurations are much smaller than the differences between runs of experiment 1a and their respective 2d configurations.

View Full Size

As in Fig. 5, but for experiments 1a (EnVar; limiting case), 2a (EnVar-VarDepH), and 2b (EnVar-VarDepHV) compared to the FreeBG.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

As in Fig. 5, but for experiments 1c (EnVar-uv_wrb), 2c (EnVar-VarDepH), and 2d (EnVar-VarDepHV) compared to the FreeBG.

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Comparison of the cycle-averaged RMSE for experiments 1a (crosses) and 2d (pluses) for all five prognostic variables. Each experiment is run three times with different random seeds for the observations (blue, red, and green) and using a 50- or 100-member ensemble (a total of six runs).

Citation: Monthly Weather Review 152, 4; 10.1175/MWR-D-23-0201.1

• Download Figure
• Download figure as PowerPoint slide

We use a paired t test and a Kolmogorov–Smirnov test to quantify the statistical significance. The two tests compare the distributions of the pairs of experiments 1a and 2d using a 100-member ensemble (three samples). The paired t test assumes that the cycle-averaged RMSE follows a normal distribution, while the Kolmogorov–Smirnov test does not. For a small sample size, it is useful to use both tests. Using the paired t test, the p values for u, ${\tilde{\rho }}^{\prime }$, and b′ are much less than 0.01, while the p values for υ and w are 0.301 and 0.0163, respectively. Using the Kolmogorov–Smirnov test, the Kolmogorov–Smirnov statistics for u, ${\tilde{\rho }}^{\prime }$, and b′ are each 1, with a p value of 0.1. For υ and w, the Kolmogorov–Smirnov statistic is each 0.667, with a p value of 0.6. A p value of less than 0.1 indicates that the result is statistically significant at the 90% confidence level. Note that experiments with 50-member ensemble are omitted from the statistical significance tests as they are repeats of the 100-member ensemble experiments (same corresponding observations) and are therefore not independent samples from the 100-member experiments. Overall, the results show that experiment 2d produces a smaller cycle-averaged RMSE than 1a, which is statistically significant at the 90% confidence level (at least) for u, ${\tilde{\rho }}^{\prime }$, and b′ but not statistically significant for υ and w.

a. Summary and key results

In this study, the benefits of retaining or rejecting multivariate error relationships and controlling the localization length scales separately for each variable in ensemble–variational (EnVar) data assimilation in the tropics are explored. This is conducted using a simplified nonhydrostatic model, the ABC model. Two approaches are implemented within the EnVar framework of the ABC model, which we refer to as the isolated variable-dependent localization (IVDL) and symmetric variable-dependent localization (SVDL) schemes. These grant the ability to (i) prescribe different spatial localization length scales for different variables and (ii) control (i.e., knockout by localization) multivariate error cross-covariances. The IVDL determines the multivariate localization in an implicit way, while the SVDL prescribes the multivariate localization in an explicit way.

Using IVDL and SVDL in multicycle observation system simulation experiments (OSSEs) with the ABC-DA system, we explore which multivariate relationships in the ensemble-derived error covariances are important/beneficial and the benefits of variable-dependent spatial localization for EnVar data assimilation in the tropics.

Using up to 1000 ensemble members (generated at each cycle as ensemble bred vectors), we first decide on a suitable ensemble size to test the localization methods (question 1 posed in section 1). We compute the degree of linear independence of each successive ensemble perturbation. This provides an indication of whether the ensemble bred vectors suitably span the subspace. This analysis shows that each variable has different characteristics due to the system dynamics. The fact that the degree of independence is different for each variable suggests that variable-dependent spatial localization is justifiable. We further show that for the ABC-DA system, increasing the number of ensemble members results in a decrease in the cycle-averaged root-mean-square error (RMSE) in the OSSEs. We find that the threshold at which covariance structures start to appear consistent (but still have appreciable sampling error) is about 100 members for the ABC model. This ensemble size is used for most of the remaining experiments.

Further experiments reveal the following:

• Using selective multivariate localization is beneficial, particularly when covariances associated with hydrostatic balance are retained and when the zonal wind errors are decoupled from the mass (scaled density perturbation) errors in the background error covariances (question 2). These lead to reduced cycle-averaged RMSE in corresponding experiments. The results suggest that when tropical dry dynamics is concerned (as with the ABC model), hydrostatic balance can still be important even at convective scales. There is also little basis for retaining the mass–wind error covariances that are presented by the ensemble, which we believe could introduce more sampling noise than useful signal in the error cross-covariances, even with as many as 100 ensemble members.

• Using variable-dependent localization is beneficial for EnVar data assimilation in the tropics, albeit to a smaller extent compared to selective multivariate localization (question 3). For this particular setup, we show that using different horizontal length scales for wind and mass variables (longer length scales for wind) reduces the cycle-averaged RMSE, perhaps because they are more optimal for this system.

• The best-performing experiment uses both variable-dependent localization and selective multivariate localization (retaining covariances associated with hydrostatic balance and omitting covariances associated with mass–wind error relationships). It leads to a 3%–4% smaller cycle-averaged RMSE than the experiment using a traditional EnVar setup, where there is often little control over multivariate localization nor over separate localization length scale for each variable. This is of the same order of magnitude improvement that is typical of upgrades to, for instance, the EnVar system of Environment and Climate Change Canada (e.g., Caron and Buehner 2022).

b. Discussion and future work

In section 2b, we highlighted how SVDL allows for extra symmetry to be enforced in the off-diagonal block matrices. Although there is no mathematical need for such extra symmetry in $\mathsf{L}$ (only that $\mathsf{L}$ as a whole is symmetric), we thought that this could be nonetheless investigated as a plausible option. Comparing experiments 2a with 1a and 2c with 1c shows that variable-dependent spatial localization using SVDL and IVDL, respectively, reduces the RMSE of respective benchmarks by about the same percentages. From other further experiments (not shown), there also does not appear to be additional benefits of having symmetric off-diagonal block matrices using SVDL compared to IVDL. For computational efficiency, it may be more prudent to implement IVDL over SVDL, especially over nonperiodic domains (see section 2a for this caveat). This is because the potential lack of positive semidefiniteness in circulant block matrices only arises in IVDL because of periodic boundary conditions, but they frequently arise in SVDL regardless of boundary conditions.

A more complete study of the optimal horizontal and vertical length scales, and the correlation shapes for variable-dependent localization, is beyond the scope of this paper. In the above experiments, the specific length scale and correlation function choices for wind variables led to improved forecasts, but this could be because the initial choices for experiments 1a and 1c were severely suboptimal. To briefly explore this further, we apply the empirical optimal localization approach of Necker et al. (2023) to find the optimal tapering factors—the factors for populating the optimal localization matrix for a given ensemble size. The optimal tapering factors are computed using the mean of middle five vertical levels, 28–32, and with respect to the middle horizontal grid point as a simple test. The 1000-member background ensemble and 100-member subsamples from the first cycle of the 1000-member experiment (section 4b) are used. The cutoff length of the optimal tapering factor curve with respect to the middle horizontal grid point is about 150 km for u, 80 km for ${\tilde{\rho }}^{\prime }$, and 40 km for b′ (not shown), corresponding to horizontal localization length scales of 75, 40, and 20 km, respectively (half of cutoff length), based on the commonly used Gaspari–Cohn localization function (Necker et al. 2023). The optimal tapering factor curves for υ and w were too noisy, owing to the limited number of vertical levels used in this simple test. This simple test suggests that the initial choices were moderately suboptimal for some variables. Nonetheless, the key point from these experiments is that variable-dependent localization can be beneficial—the impacts are sizeable even in a simplified multivariate system. These results support the findings by Necker et al. (2023), Lei et al. (2015), and Wang and Wang (2023), who found that the localization length scales depend on the variable. Future work may focus on identifying suitable correlation functions and length scales, as above, to implement with variable-dependent localization to achieve further improvements in the forecasts.

There are still many avenues to explore with regard to localization. It has been shown that scale-dependent localization is useful in general (Caron and Buehner 2022; Wang and Wang 2023), although its performance in the tropics is still questionable (Caron and Buehner 2022), and so requires further study. The localization function is not necessarily best specified as a fixed function of distance. Anderson (2012) for instance showed that the localization function is a function of the sample correlation itself in addition to the ensemble size. This issue may be dealt with using adaptive localization schemes such as those proposed by Bishop and Hodyss (2007, 2009). These schemes define the localization function from the ensemble itself, but these schemes are quite expensive to apply.

One limitation of this study is that the ABC model is a dry dynamics model and so does not represent moist processes, even though moist processes are obviously important for convective-scale data assimilation. However, as mentioned in section 5a, the tropical dry dynamics representing vertical wind (dry convection) and the mass–wind interactions are still interesting and relevant to explore as these are captured by the ABC model. Another limitation relates to the lower dimensionality of the ABC model (two-dimensional plane), so divergence and vorticity are only related to either zonal or meridional wind. Despite the limitations, this study could still pave the way for further work in testing IVDL and SVDL in full NWP systems, especially in the tropics, to assess their feasibility and benefits in applying variable-dependent or selective multivariate localization to improve EnVar data assimilation. Should computational costs permit, one might even combine this work with that of Buehner and Shlyaeva (2015) to have scale-dependent, variable-dependent, and selective multivariate localization altogether.