a. Optimal interpolation

OI is written as a linear inverse problem such as

with x^b the background or a priori information, $\mathsf{H}$ the transformation from state x to observations y, ${\mathit{\eta }}^b~\mathcal{N}\left(\mathbf{0},\mathsf{B}\right)$ the background error and $\mathit{ϵ}~\mathcal{N}\left(\mathbf{0},\mathsf{R}\right)$ the observation error. Here, x^b, $\mathsf{B}$, and $\mathsf{R}$ are prescribed by the users. OI is a reanalysis and has a direct Gaussian solution given by $\mathcal{N}\left({\mathbf{x}}^s,{\mathsf{P}}^s\right)$ such that

with $\mathsf{K}$ = $\mathsf{BH}$^T($\mathsf{HBH}$^T + $\mathsf{R}$)⁻¹ the gain controlling the influence of the observations and the background.

The quality of OI results largely depends on the choice of the $\mathsf{B}$ and $\mathsf{R}$ matrices (Tandeo et al. 2018). The matrix $\mathsf{R}$ represents the error covariances in the observational model. It can be measured or estimated offline if we assume that the observation error is stationary, which is the case in this article. However, in realistic applications, $\mathsf{R}$ can be nonstationary and should be estimated online (Minamide and Zhang 2017). The matrix $\mathsf{R}$ is not necessarily diagonal, i.e., the observation errors can be correlated. But, in practice, $\mathsf{R}$ is often assumed diagonal in order to reduce computational costs (Miyoshi et al. 2013). In our experiment, we set

where r is a scalar and $\mathsf{I}$ is the identity matrix.

The choice of $\mathsf{B}$ should be consistent with the choice of x^b. If x^b is chosen to be the climatological mean state field $\overline{\mathbf{x}}$, then it is reasonable to choose $\mathsf{B}$ as the spatial–temporal climatology background covariance matrix. However, saving the complete spatial–temporal climatology covariances is not possible in large dimensional applications because of the prohibitive requirement for storage space. Therefore, a parameterized covariance matrix is often used to substitute the complete climatology covariances (Wu 1995; Gaspari and Cohn 1999). A popular choice of $\mathsf{B}$ has the following form:

with dt = |t₁ − t₂| and where $\mathsf{B}$^spatial(i, j) is the (i, j)th component of a predetermined symmetric positive-definite matrix that represents the spatial climatology distribution of the state variable x, f is a predetermined function that defines the shape of the temporal correlation of each component of x and L_t is a prescribed parameter that defines a uniform decay rate for the temporal correlation. The matrix $\mathsf{B}$^spatial can be a parameterized matrix or the sample covariances computed from a long time series of x. Technically, $\mathsf{B}$ must be a symmetric positive-definite matrix. Hence, the choice of f cannot be arbitrary. When the dimension of x is large, directly inverting the full matrix $\mathsf{HBH}$^T + $\mathsf{R}$ is numerically demanding. In the present study, we implement OI locally in the spatial dimension, as presented in algorithm 1 (Table 1). The choice of $\mathsf{B}$^spatial and f depends on the application problem and will be discussed in each experimental section. Note that OI can also be implemented locally in both spatial and temporal dimensions.

Table 1.

Algorithm 1. Local optimal interpolation.

View Table

b. Analog data assimilation

AnDA is a combination of analog forecasting and data assimilation. For the part of data assimilation, we use the ensemble Kalman smoother (EnKS), which is commonly used in many classic data assimilation problems (see, for instance, Compo et al. 2011). The EnKS requires an ensemble run of N_e simulations starting from different initial states. This ensemble run provides sample covariances for data assimilation at every time step. The EnKS consists of a forward filter and a backward smoother. In the forward process, the forecast of each ensemble member is calculated separately. And each member is updated by ensemble Kalman filter whenever observations are available. In the backward smoother, each member is updated recursively in the backward direction. The EnKS is summarized in algorithm 2 (Table 2). The subscript i refers to the ith member, t the time, and the superscripts p, f, a, and s refer to the forecast without noise, forecast with noise, analysis, and reanalysis, respectively. In the forward Kalman filter, ε_i,t is artificially created and added to y_t to compensate for the loss of variance (Houtekamer and Mitchell 1998). Line 4 of algorithm 2 implements covariance localization which consist in the Schur product $\mathsf{P}\circ {\mathsf{C}}_{\text{loc}}$ with $\mathsf{C}$_loc a prescribed spatial covariance localization matrix (e.g., the Gaussian function or the Gaspari–Cohn matrix; Gaspari and Cohn 1999). In AnDA, the forecasting operator F in line 7 of algorithm 2 is replaced by the analog forecasting.

Table 2.

Algorithm 2. Ensemble Kalman smoother (with covariance localization).

View Table

The major difference between AnDA and the classic data assimilation is that AnDA uses the technique of analog forecasting to predict the state at the next time step, instead of running the numerical model. In many applications, the analog forecast method could be an interesting alternative since it can simulate variable dynamics that are not necessarily represented in a numerical model. For instance, if an underlying variable of the system is not modeled by the numerical model but is present in the analog database, the analog forecast will be able to describe its relationship to other variables and predict its evolution. To ensure the good performances of the analog forecast method and consequently of AnDA, a large historical dataset of state variables is needed: the catalog.

The quality of the analog forecasting procedure highly depends on the quality and the space of the catalog. First, the catalog has to be as rich as possible to cover all the possible situations. Larger catalogs usually lead to better performance of AnDA. Second, the analogs have to live in an informative space. In practice, it can be a subspace to reduce the dimensionality of the problem (e.g., the EOF space used in section 4) or an augmented space when the dimension of the system is too low to distinguish situations that are not real analogs (e.g., the time delayed state space used in section 3). The catalog is then saved in a k-dimensional tree structure so that the relevant analogs at each time step can be accessed efficiently (Bentley 1975). The technique of analog forecasting at each time step can be briefly summarized by the following three steps.

• Step 1: For a given state estimate x_t, search for k analogs ($\mathsf{A}$₁, …, $\mathsf{A}$_k) that are nearest to x_t within the catalog, where k is prechosen. At the same time, we are also given the successors of $\mathsf{A}$_i, denoted by $\mathsf{S}$₁, …, $\mathsf{S}$_k. Here $\mathsf{S}$_i is the physical state at one time step later than $\mathsf{A}$_i.

• Step 2: Build a local model ${\mathcal{M}}_t$ between $\mathsf{A}$₁, …, $\mathsf{A}$_k and $\mathsf{S}$₁, …, $\mathsf{S}$_k, i.e., ${\mathsf{S}}_i={\mathcal{M}}_t\left({\mathsf{A}}_i\right)+{\mathit{\eta }}_{i,t}$, where η_i,t is assumed to be some white and independent identically distributed noise, the distribution of which can be calculated from $\mathsf{A}$_i and $\mathsf{S}$_i.

• Step 3: Apply the local model ${\mathcal{M}}_t$ to x_t: ${\mathbf{x}}_{t+1}\leftarrow {\mathcal{M}}_t\left({\mathbf{x}}_t\right)+{\mathit{\eta }}_t$, where η_t, describing the model error of ${\mathcal{M}}_t$, is drawn randomly and follows the same distribution as η_i,t.

It has been pointed out in Lguensat et al. (2017) that there are various choices of local models in the second step. Lguensat et al. (2019) compared these local models and the numerical results show that the locally linear model outperforms the others. In our applications, the local model ${\mathcal{M}}_t$ is the locally linear model that regresses $\mathsf{S}$_i over the anomalies of analogs ${\mathsf{A}}_i^{\prime }={\mathsf{A}}_i-\overline{\mathsf{A}}$, where $\overline{\mathsf{A}}$ refers to the weighted mean of $\mathsf{A}$_i. Or equivalently, the local model we choose is the linear model that regresses the anomalies of successors ${\mathsf{S}}_i^{\prime }={\mathsf{S}}_i-\overline{\mathsf{S}}$ over ${\mathsf{A}}_i^{\prime }$. In the numerical implementation, this linear regression can be done with respect to the leading components of ${\mathsf{A}}_i^{\prime }$. In the case that x_t represents the full state, this local model can be thought of as an approximation of the tangent linear model restricted on the attractor if the current state estimate x_t lies on the attractor and the distribution of analogs is dense enough. Furthermore, the distribution of the residuals η_i,t is always assumed to be Gaussian in our applications. Hence, ${\mathit{\eta }}_t~\mathcal{N}\left(0,{\mathsf{Q}}_t\right)$, where $\mathsf{Q}$_t is the weighted covariance matrix of the residues ${\mathsf{S}}_i-{\mathcal{M}}_t\left({\mathsf{A}}_i\right)$, mentioned in step 3. The details of analog forecast with locally linear model is described in algorithm 3 (Table 3).

Table 3.

Algorithm 3. Analog forecast.

View Table

c. Conceptual differences

In this subsection we discuss, from a conceptual point of view, the differences between AnDA and OI based on the formulations of these two algorithms. These differences are then assessed in sections 3 and 4 on numerical experiments.

The OI is a purely spatial–temporal interpolation method. The performance of OI completely relies on the choice of the static matrices $\mathsf{B}$ and $\mathsf{R}$. Hence, the interpolation does not account for the dynamics of the underlying state variable. As a consequence, the estimated posterior variance of the OI reanalysis shall only depend on the positions of observations and the physical locations of the state variables. On the other hand, AnDA automatically learns the dynamics from the catalog at every time step. Hence, the posterior variance of AnDA should be flow dependent.

In operational usages of OI, it is usually not realistic to construct the full spatial–temporal climatological covariance. Hence, $\mathsf{B}$ is often assumed to be the tensor product of a spatial covariance matrix and a temporal correlation matrix. The temporal correlation matrix is uniquely determined by a scalar parameter L_t which defines the temporal correlation scale. Numerically, this artificial temporal correlation smooths out the temporal fluctuations of the reanalysis that have periods shorter than L_t. Hence, the OI should not be able to reconstruct the signal for modes of periods less than L_t. In contrast, AnDA does not have this limitation since the state variables are propagated under the dynamics learned from the catalog.

a. Implementation of AnDA

Applying AnDA directly on the first L63 component cannot lead to a good estimation. Indeed, if x_t = a, the intersection of the section x = a and the L63 attractor has two branches, which is the case for a large proportion of possible values of a. Then whether x_t+1 would be greater than or smaller than x_t depends on which branch the full state variable (x_t, y_t, z_t) lies on. Hence, it is roughly equally likely for x_t to increase or decrease in the next time step. Therefore, we would not be able to have an informative prediction of x_t+1 by merely looking at the analogs of x_t. A solution to this problem is to consider the time-delayed states x_t = (x_t, x_t−τ, x_t−2τ)^T for the implementation of AnDA. Experimentally, we find the optimal τ = 11 value. Figure 1 shows the original attractor and the attractor of the time-delayed state variable. By using the time-delayed states as analogs, the details of the implementation of AnDA shall change correspondingly, which is explained in detail in the appendix. We use an ensemble of size N_e = 50. At each time step, we apply analog forecasting separately to each ensemble member with k = 50, which is the parameter mentioned in step 1 of analog forecasting. For the Kalman smoother, we use $\mathsf{R}$ = 2, which is the same as the observation error variance used to create the observations.

The attractors of (left) the original state variable and (right) the time-delayed state variable of L63.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

The attractors of (left) the original state variable and (right) the time-delayed state variable of L63.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

The attractors of (left) the original state variable and (right) the time-delayed state variable of L63.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

b. Implementation of OI

Since we only consider the first component x of the full system, we choose the following prior background covariance:

where B₁₁ is the climatology covariance of x, which can be calculated from a long-time simulation. The parameters of OI, namely, r and L_t as indicated by Eqs. (4) and (7), are tuned to guarantee that OI algorithm produces the minimal root-mean-square error (RMSE): here we set r = 2 and L_t = 0.2.

c. Comparison of mean estimates

Let $\widehat{\mathbf{x}}$ be the reanalysis estimates of AnDA or OI, and x^true be the truth such that ${\mathbf{x}}^{\text{true}}$ and $\widehat{\mathbf{x}}$ exist for t₁, t₂, …, t_T. Suppose that $\widehat{\mathbf{x}}={\left({\widehat{\mathit{x}}}_j\right)}_{1,\dots ,n_x}$ and ${\mathbf{x}}^{\text{true}}={\left(x_j^{\text{true}}\right)}_{1,\dots ,n_x}$ are of dimension n_x (which equals 1 in the present L63 case), the RMSE of $\widehat{\mathbf{x}}$ is then defined as

Although the x_t we use for analog forecast with time-delayed states has three components, we only take the first component x_t to compute the RMSE. The time-delayed estimates (i.e., the second and the third components of the state reanalysis) are not used to evaluate the performance.

The RMSE for AnDA is 0.77, and the minimal (after tuning the parameters) RMSE for OI is 1.177. The top panel of Fig. 2 shows the trajectory of the truth, the observation, and the reanalysis estimates of the L63 first component. The state reanalysis produced by OI apparently has large errors when the state is near the origin. In contrast, the trajectory of AnDA manages to reproduce the L63 dynamics even when the observation errors are large. In this experiment, we do not meet the curse of dimensionality, since we have 10⁴ samples in the catalog while the Hausdorff dimension (Schleicher 2007) of the L63 attractor is around 2.06. Therefore, the dynamics represented by the analog forecast method approximates the true dynamics very well.

(top) The trajectory of the truth, the observations, and the AnDA and OI estimates. The RMSE of AnDA estimates and OI estimates are 0.77 and 1.177, respectively. (middle) Estimated reanalysis standard deviation and absolute error of reanalysis estimate for AnDA. The estimated standard deviation is strongly correlated to the absolute error. (bottom) Estimated reanalysis standard deviation and absolute error of reanalysis estimate for OI. In this example, the estimated standard deviation is periodic since it only depends on the observation frequency and the magnitude of $\mathsf{R}$.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

(top) The trajectory of the truth, the observations, and the AnDA and OI estimates. The RMSE of AnDA estimates and OI estimates are 0.77 and 1.177, respectively. (middle) Estimated reanalysis standard deviation and absolute error of reanalysis estimate for AnDA. The estimated standard deviation is strongly correlated to the absolute error. (bottom) Estimated reanalysis standard deviation and absolute error of reanalysis estimate for OI. In this example, the estimated standard deviation is periodic since it only depends on the observation frequency and the magnitude of $\mathsf{R}$.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

(top) The trajectory of the truth, the observations, and the AnDA and OI estimates. The RMSE of AnDA estimates and OI estimates are 0.77 and 1.177, respectively. (middle) Estimated reanalysis standard deviation and absolute error of reanalysis estimate for AnDA. The estimated standard deviation is strongly correlated to the absolute error. (bottom) Estimated reanalysis standard deviation and absolute error of reanalysis estimate for OI. In this example, the estimated standard deviation is periodic since it only depends on the observation frequency and the magnitude of $\mathsf{R}$.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

d. Comparison of estimated standard deviations

Another interesting way of comparing AnDA and OI is assessing the quality of the estimated standard deviation of the state reanalysis versus the true absolute error. Indeed, the absolute error directly quantifies how far the estimate is from the truth. However, the truth is usually unknown hence the absolute error is often not accessible. When this is the case, estimated standard deviations are often used as a reference to inform on the actual error of the state estimate. Hence, providing an estimated standard deviation that corresponds to the absolute error is a key feature for a reconstruction method. These quantities are defined as follows:

It is not surprising to see that the OI algorithm produces a periodic estimate of standard deviation (Fig. 2, bottom panel). Indeed, the estimated error is only based on the observation sampling. This is a strong limitation of OI. In contrast, the estimated standard deviation of AnDA is much more flow dependent (Fig. 2, middle panel). The absolute error of AnDA increases each time the state variable is close to the bifurcation point or the furthest points of the two wings. At those times, the AnDA estimated standard deviation manages to inform on the error made as the complexity of the L63 dynamics renders the state estimation harder.

a. Targeted region and dataset

In this section we test the OI and AnDA algorithms in an OSSE aiming at interpolating along-track SSH onto gridded SSH maps. We focus here on a 10° × 10° region in the eastern Gulf of Mexico (centering at 85°W, 25°N; see Fig. 3). In terms of grid points, the region of interest is 41 × 41 large, including n_x = 1353 ocean grid points in total (the rest being landmasses) thus giving the dimension of the state variable x.

Snapshots of the “true” SSH in the region of interest on different days of year 2004, featuring the formation and shedding of a “Loop Current eddy.” The SSH here comes from the OCCIPUT ensemble simulation (see text). The two symbols on the maps mark the location of the Loop Current and the Florida coast grid points, respectively, at 25°N, 85°W and at 26.03°N, 82°W.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Snapshots of the “true” SSH in the region of interest on different days of year 2004, featuring the formation and shedding of a “Loop Current eddy.” The SSH here comes from the OCCIPUT ensemble simulation (see text). The two symbols on the maps mark the location of the Loop Current and the Florida coast grid points, respectively, at 25°N, 85°W and at 26.03°N, 82°W.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

Snapshots of the “true” SSH in the region of interest on different days of year 2004, featuring the formation and shedding of a “Loop Current eddy.” The SSH here comes from the OCCIPUT ensemble simulation (see text). The two symbols on the maps mark the location of the Loop Current and the Florida coast grid points, respectively, at 25°N, 85°W and at 26.03°N, 82°W.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

The ocean circulation in this region features the Loop Current (LC), an anticyclonic flowing meander entering the Gulf through the Yucatan Channel (Yucatan Current), and exiting along the southern tip of the Florida Peninsula (Florida Current). The Loop Current is known as an unstable system and episodically sheds large anticyclonic eddy rings of scale 200–400 km with periods ranging from about 100 to 450 days (see Fig. 3). The shedding of these Loop Current eddies is a complicated process as eddies can detach and reattach to the Loop Current, before propagating westward across the Gulf. SSH variability in the region is also related to smaller-size cyclonic eddies (80–120 km) that are observed moving along the outer edge of the LC [Loop Current frontal eddies (LCFEs)], both on subannual and submonthly time scales, and to coastally trapped waves that responds to wind variability, and especially to winter cold surges [see Jouanno et al. (2016) for a review].

We perform the OSSE using daily SSH maps from one of the Ocean Chaos–Impacts, Structures, Predictability (OCCIPUT) ensemble simulations (Penduff et al. 2014; Bessières et al. 2017). This is a regional North Atlantic Ocean–sea ice 50-member ensemble simulation performed at eddy-permitting horizontal resolution (1/4°). After a common 20-yr spinup, the 50 members are restarted from slightly perturbed initial conditions and forced over 20 years (1993–2012) with identical surface forcing. In the following, the SSH of the last year of the first ensemble member is taken as the ground truth. We then use the location of the real along-track AVISO observations available for 2004 (that include 4 satellites: TOPEX/Poseidon, GFO, Jason-1, Envisat), to generate our pseudo-observations by locally and linearly interpolating the truth along the observed tracks. No observation error is artificially added to the simulated observations (i.e., $\mathsf{R}$_true = 0).

The historical catalog from which AnDA learns the forecast model is thus made of the daily maps of SSH from the 19 remaining years of the 49 remaining ensemble members (meaning 19 × 49 × 365 = 339 815 daily SSH maps in total). As an element of comparison, the historical catalog in Lguensat et al. (2019) for a similar problem is 34 years of 3-day data (4017 SSH maps).

b. Implementation of AnDA

First, we reduce the dimension of the state variable. We take the coefficients of the first 100 leading EOFs as the reduced state ${\mathbf{x}}^{\text{red}}\in {\mathbb{ℝ}}^{100}$. In practice, we calculate the spatial climatology covariance ${\mathsf{B}}^{\text{clim}}\in {\mathbb{ℝ}}^{1353\times 1353}$ based on the OCCIPUT simulation:

where ${\mathbf{x}}_{i_N,i_Y}\left(t\right)$ refers to the SSH on the tth day of year i_Y of the i_Nth ensemble member. The EOFs (denoted by e_i) are the eigenvectors of $\mathsf{B}$^clim:

for i = 1, 2, …, 1353. Then for a given state variable $\mathbf{x}\in {\mathbb{ℝ}}^{1353}$, the reduced state is defined by

The first 100 EOFs explains more than 99% of the variance of SSH. This explained variance is stable over the whole time series of 20 years.

AnDA is implemented with respect to x^red. Our catalog consists of the x^red that were calculated using 49 members (member 2 to member 50), from year 1 to year 19. Therefore, the catalog and the truth come from different members and years. By dimension reduction, the corresponding observation operator $\mathsf{H}$^red is different from the original $\mathsf{H}$:

And the corresponding observation error variance ${\mathsf{R}}_{\text{obs}}^{\text{red}}$ is no longer zero since the small components (i.e., ⟨x, e₁₀₁⟩, …, ⟨x, e₁₃₅₃⟩) are missing in the reduced state variable. In this reduced space, covariance localization is implemented as ${\mathcal{T}}^{-1}\left[\mathcal{T}\left(\mathsf{P}\right)\circ {\mathsf{C}}_{\text{loc}}\right]$ in line 4 of algorithm 2 (Table 2), where $\mathcal{T}$ transforms the covariances of x^red to the covariances of the original physical state x.

We choose N_e = 1000 (ensemble size for data assimilation), k = 1000 (the parameter mentioned in algorithm 3; Table 3) and $\mathsf{R}$ = 4 cm². A different choice of analogs was made in Lguensat et al. (2019) where the analogs and successors were chosen to represent only the small-scale modes of the complete simulated SSH. The large-scale modes of SSH were first reconstructed using the OI method. Then the small-scale modes were reconstructed using AnDA. Although this space reduction strategy was shown to be promising, its success requires a catalog of high-resolution SSH data which are not often available.

c. Implementation of OI

We considered the following background covariance matrix:

where i, j = 1, 2, …, dim(x), d_ij refers to the physical distance between x_i and x_j, d_t = |t₁ − t₂|, B_ij = Cov(x_j, x_j) refers to the spatial climatology covariance, and L_t is the scalar parameter defining the temporal scales of the covariance matrices.

The parameters B_ij are directly calculated from the SSH dataset and the parameter L_t is tuned so that the OI algorithm produces the minimal RMSE. Often in real applications, when the true spatial climatology covariances are not accessible, they are also parameterized and the background covariance matrix is approximated by $\mathsf{B}\left[x_i\left(t_1\right),x_j\left(t_2\right)\right]=\sqrt{B_{ii}{B}_{jj}}\hspace{0.17em}\exp \left\{-d_{ij}^2/L_x^2-d_t^2/L_t^2\right\}$, with L_t a scalar parameter defining the spatial scales of the covariance matrices. However, for the sake of a fair comparison between OI and AnDA and since the simulated dataset in our experiments is large enough, we are able to estimate the spatial climatology covariances B_ij and fully compute Eq. (11). This formulation indeed yields the best results in the experiments of the present section (comparison not shown).

Note that, for the OI used in Data Unification and Altimeter Combination System (DUACS), the choice of the parameters L_x and L_t is usually made as a best global trade-off to achieve global resolution of the mesoscale features (e.g., Dibarboure et al. 2011; Pujol et al. 2012), and could, in principle, be better optimized in a specific regional context (hence the tuned OI in this study).

In this study, we also consider an OI optimized with conventional objective analysis (Le Traon et al. 1998) and here named OI_COA. The OI_COA experiment is used as a point of comparison in order to show that (i) an OI is difficult to tune (conventional objective analysis fails to do so) and (ii) an incorrectly tuned OI can lead to significant errors. The covariance function $\mathsf{B}$^COA is chosen to be

where

with α = 3.34. In Le Traon et al. (1998), the parameters are chosen to be L_x = 150 km, L_t = 20 days. We tune R so that the method produces the minimal RMSE based on the given L_x and L_t. A sensitivity test (not shown here) demonstrated that the difference between OI computed from Eq. (11) and OI_COA is mainly due to the difference in the parameter L_t. The correlation functions are also different but do not make a significant difference in our numerical results.

d. RMSE results

The RMSE values for SSH, vorticity and velocity reconstructed with the three methods (AnDA, OI and OI_COA) are summarized in Table 4. Here, the velocity refers to the two-dimensional vector of the geostrophic velocities which is defined as (u, υ) = (−∂ssh/∂x, ∂ssh/∂x) (g/f) and the vorticity is defined as q = (∂υ/∂x − ∂u/∂y) (g/f), where g = 9.81 m s⁻² is the gravity acceleration and f is the Coriolis force. Table 4 shows that AnDA does as good as the best-tuned OI (i.e., tuned and optimized specifically for the region of interest) for these three variables, resulting in very similar RMSE values for the two methods in the full region of interest. In the case of SSH, the RMSE value for AnDA is smaller than the one for OI (1.40 and 1.68 cm, respectively). However, this difference fades off when the RMSE is computed over the central region only, i.e., excluding coastal areas. In the following, we will show that this is due to the fact that AnDA can reconstruct the high-frequency SSH fluctuations of the coastal areas much better than OI. These SSH high-frequency fluctuations are likely related to the coastally trapped waves responding to winter windstorm surges as mentioned in Jouanno et al. (2016).

Table 4.

Summary of the RMSE values obtained with the three methods for year 2004 (6 Jan–31 Dec 2004): AnDA, OI, and OI_COA, for SSH (in cm), geostrophic velocity (in cm s⁻¹), and vorticity [(100 s)⁻¹], computed over the full domain, in the central region only (indicated with an asterisk), i.e., excluding the coastal areas (23.78°–27.13°N, 83.75°–90°W), and in the Florida and Yucatan coastal area.

View Table

It is also clear from Table 4 that OI_COA is systematically less accurate than AnDA and OI in terms of RMSE. The time series of the reconstructed SSH at two example grid points, displayed in Fig. 4, provide an illustration to why this is the case. With parameter L_t set to 20 days for the temporal correlation scale of OI_COA, the reconstructed SSH misses the high-fluctuations of the signal. These high-frequency fluctuations are particularly strong near the Florida coast (bottom panel), while in the Loop Current (top panel), the large amplitude fluctuations appear to be of monthly and subannual time scales as they are associated with the fluctuations of the LC meander and LCE shedding. On the other hand, the tuning of the best-tuned OI with L_t = 6 days results in a better behavior of the reconstructed SSH in the high frequencies. We quantify this further in the following with a dedicated temporal spectral analysis. At this point, we wish to emphasize the fact that AnDA is as accurate as the best-tuned OI, without the need to explicitly tune the parameters L_x and L_t. In AnDA, the information is implicitly provided by the historical catalog. It should be reminded, however, that these results are produced in the context of an OSSE with pseudo-observations derived from the simulated truth, and so the historical catalog from which AnDA learns is fully consistent with those observations. We reserve for future investigations the case where the catalog and the truth come from different sources.

Time series of the reconstructed daily SSH for year 2004 at the two grid points marked on the maps in Fig. 3: (top) in the Loop Current (25°N, 85°W) and (bottom) near the Florida coast (26.03°N, 82°W). The reconstructed SSH is shown for AnDA, OI and OI_COA and compared with the true SSH.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Time series of the reconstructed daily SSH for year 2004 at the two grid points marked on the maps in Fig. 3: (top) in the Loop Current (25°N, 85°W) and (bottom) near the Florida coast (26.03°N, 82°W). The reconstructed SSH is shown for AnDA, OI and OI_COA and compared with the true SSH.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

Time series of the reconstructed daily SSH for year 2004 at the two grid points marked on the maps in Fig. 3: (top) in the Loop Current (25°N, 85°W) and (bottom) near the Florida coast (26.03°N, 82°W). The reconstructed SSH is shown for AnDA, OI and OI_COA and compared with the true SSH.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

An additional sensitivity test has been performed in order to assess the impact of the catalog size on the reconstruction performances. Three other catalog sizes have been implemented: using 1 member (19 × 365 daily SSH maps), 20 members (20 × 19 × 365 daily SSH maps), and 30 members (30 × 19 × 365 daily SSH maps) of the 19-yr OCCIPUT ensemble and compared to the current catalog using 49 members (49 × 19 × 365 daily SSH maps). The resulting SSH reconstruction RMSE are, respectively, 1.82, 1.46, 1.45, and 1.40 cm. As expected, the performance of AnDA are improved by a larger catalog. However, the dependence is not linear and the difference between using 20 members, 30 members, and 49 members is relatively small. In fact, both the 20 member catalog and the 30-member catalog also lead to smaller RMSE than OI.

e. Temporal spectral results

The top panel of Fig. 5 shows the temporal power spectral densities (PSD) averaged over the entire domain for the reconstructed SSH with the three methods and for the true SSH. The PSD of the three reconstructed signals are very close to the PSD of the truth at time scales longer than about 30 days, confirming that all three methods produce equivalent energy reconstructions of the monthly to subannual fluctuations. But at higher frequencies, we find that only the PSD for the AnDA-reconstructed SSH stays close to the truth. A drop-off in the PSD is clearly seen for the OI- and OI_COA-reconstructed SSH at approximately 6 and 20 days, respectively, which is consistent with the values set for L_t in each case.

(a) Temporal power spectral density (PSD) of the reconstructed SSH (AnDA, OI, OI_COA) and true SSH. (b) Temporal signal-to-noise ratio (R) measuring the temporal coherence of each of the reconstructed SSH (AnDA, OI, OI_COA) with the true SSH. Both PSD and R are averaged over the entire domain. Both panels share the same x axis in log scale for temporal frequency [cycles per day (cpd)]. The tick labels on the top axis give the corresponding periods in days.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

(a) Temporal power spectral density (PSD) of the reconstructed SSH (AnDA, OI, OI_COA) and true SSH. (b) Temporal signal-to-noise ratio (R) measuring the temporal coherence of each of the reconstructed SSH (AnDA, OI, OI_COA) with the true SSH. Both PSD and R are averaged over the entire domain. Both panels share the same x axis in log scale for temporal frequency [cycles per day (cpd)]. The tick labels on the top axis give the corresponding periods in days.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

(a) Temporal power spectral density (PSD) of the reconstructed SSH (AnDA, OI, OI_COA) and true SSH. (b) Temporal signal-to-noise ratio (R) measuring the temporal coherence of each of the reconstructed SSH (AnDA, OI, OI_COA) with the true SSH. Both PSD and R are averaged over the entire domain. Both panels share the same x axis in log scale for temporal frequency [cycles per day (cpd)]. The tick labels on the top axis give the corresponding periods in days.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

We also check the noise-to-signal ratio between the reconstructed signals and the truth (Fig. 5, bottom panel). For this purpose, and following Dufau et al. (2016) and Ballarotta et al. (2019), we compute the spectral noise-to-signal ratio as

where PSD_Error is the PSD of the difference between the reconstructed SSH and the truth, and PSD_Truth is the PSD of the true signal. This metric provides a measure of the coherence between the two signals that takes into account differences in both amplitude and phase (Ballarotta et al. 2019). Figure 5 (bottom panel) shows that the spectral noise-to-signal ratio for AnDA remains far above 0.5 down to time scales of ~5 days, which confirms that a good coherence exists between the AnDA-reconstructed and the true SSH. The coherence is not as good for the two reconstructed SSH signals of OI and OI_COA. For the best-tuned OI, R drops below 0.5 at time scales of ~15 days, even if the PSD drops off only around 6 days. In other words, the OI method manages to reconstruct enough energy at high frequencies (although not below 6 days) yet fails to produce a coherent signal at those scales. As for OI_COA, both PSD and R drop off at time scales of about 25 days.

We thus find confirmation in Fig. 5 that AnDA is able to reconstruct an SSH signal with good coherence to the truth at higher frequencies than OI, and so even when AnDA is compared with the OI specifically tuned for the region of interest. This is consistent with what we had already pointed out from the example gridpoint time series in Fig. 4. As already discussed, in the domain we examine here, submonthly fluctuations are the strongest in coastal regions because of the response to wind bursts (Jouanno et al. 2016). Operational systems such as DUACS, based on OI, are not able to capture well those fast fluctuations, but can partially get around this limitation by using additional products such as the AVISO Dynamic Atmosphere Correction (DAC) to propose an a posteriori correction for the missed and aliased part of the signal corresponding to the dynamical high-frequency ocean response to wind and pressure forcing (e.g., Ponte and Ray 2002). Note that this correction, however, is only based on this specific source of high-frequency fluctuations, while our study shows that a method such as AnDA is able to capture high-frequency signals originating from all kind of sources (to the extent that the fluctuations are well represented in the historical catalog). We find indeed that the spectral results shown in Fig. 5 remain robust also when restricting the area of the spectral analysis to the central ocean-only region (not shown here), meaning that AnDA is able to capture high-frequency fluctuations of any kind, and not only the coastally trapped waves.

f. Estimated standard deviation results

Another interesting result of this study is that, consistently with the L63 application in section 3, AnDA produces a more informative estimated standard deviation. Indeed, it has similar spatial patterns as the absolute error (i.e., the difference with the true signal), and does not only depend on the tracks of the observations to interpolate. This is illustrated in Fig. 6 where estimated standard deviation and absolute error are averaged over a 1-month period on two example dates (8 March and 8 September 2004) for AnDA, OI, and OI_COA. For visual purposes, we show the monthly averaged distribution of the absolute error as it presents a clearer flow-dependent feature than the daily distribution.

Monthly averages of (top) estimated standard deviation and (bottom) absolute error centered on (a) 8 Mar and (b) 8 Sep 2004. The $\mathsf{P}$^s produced by OI (top-middle panel for each month) and OI_COA (top-right panel for each month) only depends on the tracks of satellite altimetry and the background covariance $\mathsf{B}$. Therefore, the estimated standard deviation does not seem relevant to approximate the absolute error (bottom-middle and bottom-right panels for each month). On the other hand, the estimated standard deviation produced by AnDA is flow dependent (top-left panel for each month) and closer to the absolute error.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Monthly averages of (top) estimated standard deviation and (bottom) absolute error centered on (a) 8 Mar and (b) 8 Sep 2004. The $\mathsf{P}$^s produced by OI (top-middle panel for each month) and OI_COA (top-right panel for each month) only depends on the tracks of satellite altimetry and the background covariance $\mathsf{B}$. Therefore, the estimated standard deviation does not seem relevant to approximate the absolute error (bottom-middle and bottom-right panels for each month). On the other hand, the estimated standard deviation produced by AnDA is flow dependent (top-left panel for each month) and closer to the absolute error.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

Monthly averages of (top) estimated standard deviation and (bottom) absolute error centered on (a) 8 Mar and (b) 8 Sep 2004. The $\mathsf{P}$^s produced by OI (top-middle panel for each month) and OI_COA (top-right panel for each month) only depends on the tracks of satellite altimetry and the background covariance $\mathsf{B}$. Therefore, the estimated standard deviation does not seem relevant to approximate the absolute error (bottom-middle and bottom-right panels for each month). On the other hand, the estimated standard deviation produced by AnDA is flow dependent (top-left panel for each month) and closer to the absolute error.

Citation: Journal of Atmospheric and Oceanic Technology 37, 9; 10.1175/JTECH-D-20-0001.1

• Download Figure
• Download figure as PowerPoint slide

Figure 6 shows that the absolute error of AnDA on 9 March and 9 September is smaller than that of OI and OI_COA, especially along the Florida coast and in the Loop Current. It means that on that dates, the AnDA-reconstructed SSH is closer to the truth, which is consistent with time series given in Fig. 4. For instance, on 9 September, the absolute errors concentrate near the anticyclonic flowing meander. And it is clear that in this region, the absolute error of AnDA is smaller than that of OI and OI_COA.

Figure 6 also illustrates the fact that the estimated standard deviation for OI depends on the satellite observation sampling (here, along tracks) and on the background error covariance matrix $\mathsf{B}$. This is consistent with the results given in Fig. 2 (bottom panel) in the case of the L63 system. The estimated standard deviation for OI is thus noninformative. In contrast, the estimated standard deviation of AnDA does not only depend on the observation sampling but also on the flow. Therefore, its pattern is more correlated to the absolute error (see top and bottom left panels of each snapshot in Fig. 6).