## 1. Introduction

Rooted in the general theory of sensitivity analysis in data assimilation (Le Dimet et al. 1997), the evaluation of the model forecast sensitivity to observations (FSO) provides an efficient tool for assessing the observing system performance in numerical weather prediction (NWP). The adjoint-based derivation of FSO was put forward by Baker and Daley (2000) and further extended by Langland and Baker (2004) to provide an *all-at-once* assessment of the observation impact on reducing a functional aspect of the model forecast error, including the impact of various observation types categorized by their distribution in the time-space domain. Forecast sensitivity and observation impact (FSOI) methodologies have been formulated and implemented for variational, ensemble, and hybrid data assimilation systems (DAS) and are now incorporated in routine observation monitoring activities at operational NWP centers (Liu and Kalnay 2008; Cardinali 2009; Gelaro et al. 2010; Kalnay et al. 2012; Todling 2013; Ota et al. 2013; Lorenc and Marriott 2014; Buehner et al. 2018; Ishibashi 2018; Kim and Kim 2019; Ruston and Healy 2021). FSOI tools are used along with observing system experiments (OSEs) in synergistic efforts to assess the interaction between various data types in the DAS and their role in improving the model forecasts.

The experience gained from FSOI studies is that in practice, only a small majority of the assimilated observations improve the forecasts (Gelaro et al. 2010; Lorenc and Marriott 2014; Kotsuki et al. 2019). However, caution must be exercised in the design of a data thinning strategy that is effective in reducing the forecast errors. Research to directly embed sensitivity information into a feedback mechanism for improving the observation performance is receiving an increased interest. The FSO has been extended to include error covariance parameters (Daescu 2008) and provide guidance for tuning the observation error covariance representation in the DAS (Daescu and Langland 2013; Lupu et al. 2015; Hotta et al. 2017b; Kim and Kim 2018). A method to flow dependent proactive quality control (PQC) based on FSOI guidance was developed by Hotta et al. (2017a) using the ensemble forecast sensitivity to observations (EFSO) approach of Kalnay et al. (2012) and positive outcomes from PQC through data denial OSEs have been obtained by Chen and Kalnay (2019, 2020).

Our work seeks to improve the observation performance through an improved weighting of the observed-minus-forecast (omf) residuals (innovation vector), without removing observations from the DAS. A new FSO-based optimization methodology is developed to identify and correct deficiencies in the assigned innovation weights through an iterative gradient-based optimization process. By performing parameter updates carried over a time period and effective in reducing a forecast error aspect, the expected outcome is to correct systematic deficiencies in the way information is processed in the DAS and reach a configuration that will sustain improved performance. This methodology is implemented and tested in numerical experiments performed with the Naval Research Laboratory (NRL) Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR; Xu et al. 2005; Rosmond and Xu 2006) and the U.S. Navy’s Global Environmental Model (NAVGEM; Hogan et al. 2014). The article is organized as follows. Section 2 outlines the FSO and FSOI evaluation in a formulation that is relevant to our study. In section 3, key ingredients of a general theoretical formulation are introduced and discussed as follows: specification of the error functional, the parameterization of the analysis equation, the FSO-based identification of a descent direction for gradient-based optimization, and a computationally efficient procedure for step length selection. Section 4 presents the experimental setup, the parameter tuning process, and a detailed analysis of the validation results. Summary and further research directions are in section 5.

## 2. Outline of adjoint-based FSO and FSOI

*δ*

**y**=

**y**−

**h**(

**x**

*) is the innovation vector,*

_{b}**h**evaluated at

**x**

*.*

_{b}**x**

_{0}valid at the analysis time

*t*

_{0}and a forecast lead time

*τ*(e.g.,

*τ*= 24 h), a measure of the forecast error at

*t*=

_{f}*t*

_{0}+

*τ*is typically specified as

*t*

_{0}from

**x**

_{0}and valid at

*t*,

_{f}**x**

*is the verifying analysis at*

^{υ}*t*, and

_{f}**E**is a symmetric positive definite matrix that defines the norm in the state space. A total energy norm is often used in the error measure (3) and observation-space metrics have been also considered (Todling 2013; Cardinali 2018). The gradient of

*e*with respect to the initial condition

**x**

_{0}is

**M**

_{0}denotes the Jacobian matrix (tangent linear model) of the nonlinear model

**x**

_{0}and

**x**

_{0}=

**x**

*and*

_{a}**x**

_{0}=

**x**

*, respectively,*

_{b}The particular case when a verification state is available at the initial time, *t _{f}* =

*t*

_{0}, corresponds to

**M**

_{0}is the identity matrix. In this case,

*e*(

**x**

*) and*

_{a}*e*(

**x**

*) measure, respectively, the error in the analysis state and the error in the background state.*

_{b}### a. FSO evaluation

**x**

*=*

_{a}**x**

*(*

_{a}**y**), the FSO is defined as the gradient of the functional

*e*[

**x**

*(*

_{a}**y**)] with respect to

**y**. From (1) and (7), with chain rule differentiation the FSO is expressed as

**y**−

**h**(

**x**

*) to assess the forecast sensitivity with respect to weight parameters in the observation error covariance representation*

_{a}*i*∈

*I*, and

*e*(

**x**

*) to the observation error covariance weight parameters (FSR) in the reference DAS specification*

_{a}This approach was implemented for tuning the covariance model **R** in a 4D-Var DAS by Lupu et al. (2015) and Kim and Kim (2018), and the ensemble-based formulation to FSR evaluation was developed by Hotta et al. (2017b).

### b. FSOI evaluation

*e*measure,

Data components with *δe* expressed as (13) and are distinguished by their specification and evaluation of the weight vector (14). Analysis of FSOI approximation properties, derivation of higher-order measures, and extensions to fully nonlinear variational schemes are provided in references Gelaro et al. (2007), Trémolet (2008), and Daescu and Todling (2009).

## 3. An FSO-based optimization framework

*linear*innovation-weight parameterization of the analysis state that was first introduced in our work Daescu and Langland (2016). This approach is further developed here and embedded into an iterative process to identify and correct deficiencies in the weights associated to various observation input components based on guidance provided by a short-range forecast error measure. The parametric representation of the analysis is expressed as

*s*,

_{j}*j*= 1:

*m*. The parameter dimension, 1 ≤

*m*≤

*p*, is specified by experiment criteria to partition the data

*m*= 1 when the innovation vector is scaled through a single scalar parameter

*sδ*

**y**and

*m*=

*p*when a scalar parameter is associated to each observation in the DAS. In the latter case, (17) becomes the standard elementwise vector product,

*s*,

_{i}y_{i}*i*= 1:

*p*. It is noticed that for any specification of the weight parameter

**s**, the analysis (16) is obtained by minimizing a quadratic cost functional (single outer loop iteration) for the analysis increment

*δ*

**x**=

**x**−

**x**

*,*

_{b}The reference analysis (1) corresponds to all weights set to one in (16)/(18), **x*** _{a}* =

**x**

*(*

_{a}**1**

*) where*

_{m}**s**, possibly different from the control DAS, we seek a correction

*δ*

**s**such that the updated weights

**s**

^{new}will reduce the error measure defined by (3) and (4),

*improved observation performance*on reducing the forecast errors induced by the parameter correction

*δ*

**s**. When the update (19) is applied at

**s**=

**1**

*, the goal is to improve the analysis forecast produced by the control DAS,*

_{m}**s**≠

**1**

*, the goal is to improve a parameter configuration that has already been updated in a previous analysis, within an iterative parameter tuning process performed over several data assimilation and forecast cycles. In any situation, the corrected analysis*

_{m}**x**

*(*

_{a}**s**

^{new}) is used to produce the background to the next analysis cycle, e.g., for a 6-h assimilation window

**s**

^{new}may be expressed as

*α*> 0. Before addressing (23) and (24), we extend the FSO (9) and FSOI (15) to the weighted DAS representation (16).

### a. FSO and FSOI in the weighted DAS representation

**K**as

**x**

*(*

_{a}**s**). The evaluation of (27) is performed with the same computational tools of the control DAS to evaluate the observation space vector

**s**=

**1**

*. By analogy with (14) and (15), the FSOI calculation associated with the weighted representation (16) is*

_{m}**g**

*replaced by*

_{a}**g**

_{a}_{,}

**. Therefore, as compared with the control DAS, the software development efforts and the added computational cost of implementing and evaluating the weighted formulations (16)/(26), (30), and (31) are negligible.**

_{s}### b. FSI

**x**

*(*

_{a}**s**) to parameters is

*unweighted*innovations. From (30) and (33), we establish the relationship between the FSO and FSI in the weighted analysis (16):

Each gradient component in (33) represents the forecast sensitivity to the corresponding innovation-weight parameter: a positive value *δ***y*** _{j}* is of potential benefit to the forecasts, whereas a negative value ∂

*e*[

**x**

*(*

_{a}**s**)]/∂

*s*< 0 indicates that an increased weight

_{j}*δ*

**y**

*is of potential benefit to the forecasts.*

_{j}The FSO-developed software tools allow the simultaneous evaluation of the sensitivities FSR (11) and FSI (33): the FSR relies on FSO and oma residual **y** − **h**(**x*** _{a}*), whereas the FSI relies on FSO and omf residual

**y**−

**h**(

**x**

*). The correlation between the FSR and the FSI guidance is analyzed in our work Daescu and Langland (2016). A practical advantage of the FSI parameterization is that the analysis (16) is linearly related with the weight parameters. This property facilitates the search for improved parameter estimates through quadratic approximations to the forecast error measure, as explained below.*

_{b}#### 1) The FSI guidance for the analysis increment

*m*= 1 corresponds to a single scalar parameter

*s*and (16) is represented as

**x**

*(*

_{a}*s*),

*s*= 0 and 1 correspond to the background state

The first-order optimality condition is *s* = 1, a negative sensitivity *s* > 1 is a descent direction, i.e., *δ***x*** _{a}* is “too short,” whereas a positive sensitivity

*s*< 1 is a descent direction, i.e.,

*δ*

**x**

*is “too long.” Strategies for estimating the optimal scaling to the analysis increment are presented below. In the following section, we explain that this process is also highly relevant to the problem of finding an appropriate step length*

_{a}*α*along a generic direction

**p**in the parameter update (23).

##### Approximation to the optimal scaling s^{⋆}

*t*

_{0}to

*t*(Gelaro et al. 2007). Under this assumption, an estimate

_{f}*s*

^{⋆}for the analysis increment may be obtained from a quadratic approximation to the nonlinear functional (37),

**x**

*(*

^{f}*s*) with the secant line passing through

*s*= 0 and

*s*= 1,

^{1}

**E**norms as

*δ*

**x**

*in the control DAS expressed in terms of the observation impact*

_{a}*δe*and the forecast increment

For completeness, Table 1 provides the coefficients (40) for each of the quadratics *q _{a}*,

*q*, and

_{b}*q*defined in (41), (42), and (45), respectively, and the estimated scaling factor

_{ab}Quadratic approximations to the parametric forecast error measure and the associated estimates to the optimal scaling of the analysis increment.

#### 2) Parameter updates along a descent direction **p**

**p**= −∇

_{s}*e*[

**x**

*(*

_{a}**s**)] associated with the weight specification

**s**in the analysis (16). Aside from the steepest descent iteration, various strategies may be considered in the practical implementation of the parameter tuning procedure. For example, a block coordinate descent approach (Beck and Tetruashvili 2013; Wright 2015) may be used to update selected parameter components

*s*, e.g., the weights assigned to innovation components of increased sensitivity,

_{j}##### Step length specification

*α*> 0 is essential in determining the performance of the corrected weights (23): a cautiously small step may not produce a substantial forecast improvement, whereas a large step may prove to be detrimental. In general, a correction applied to the weight parameters along a direction

**p**requires the computational overhead of performing one additional analysis. For a trial step length specification

*α*

_{0}> 0, we express the

*trial analysis*as

*optimal step length α*

^{⋆}, suitable for practical implementation. The key to the process is the linear relationship

**x**

*(*

_{a}**s**) in (16) as

*α*along the direction

**p**is expressed using (53) as

**x**

*(*

_{a}**s**) and

**x**

*(*

_{a}**s**+

*α*

_{0}

**p**) in (57) as the background state

**x**

*and the analysis state*

_{b}**x**

*=*

_{a}**x**

*(1) in (35), respectively, and the step ratio*

_{a}*α/α*

_{0}as the scaling factor to the analysis increment. In this context, a quadratic approximation such as (42) or (45) may be used to provide an improved step length estimate. For example, the

*q*approximation (47) to the optimal step

_{ab}*α*

^{⋆}in the direction

**p**is expressed as

**x**

*(*

_{a}**s**) in (16). The updated weights (23) are defined as

*corrected analysis*is evaluated as

It is noticed that the computational overhead required to implement the corrected analysis (60), as compared with the *trial analysis* (52), is roughly given by the model forecast

The computational steps required to perform the correction to the assigned innovation-weight parameters within a data assimilation cycle.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The computational steps required to perform the correction to the assigned innovation-weight parameters within a data assimilation cycle.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The computational steps required to perform the correction to the assigned innovation-weight parameters within a data assimilation cycle.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The theoretical concepts developed in this section provide the main ingredients to the formulation of a gradient-based parameter tuning procedure. We emphasize that the practical implementation must be tailored to the characteristics of the data assimilation and forecast system and the specific goals of the application at hand.

## 4. Experiments with NAVDAS-AR/NAVGEM

The practical applicability of the theoretical framework developed in this work is illustrated in numerical experiments performed with the 4D-Var system NAVDAS-AR and the NAVGEM forecasting model at a resolution of T425/L60 (Hogan et al. 2014). The vertical coordinate is a hybrid sigma-pressure coordinate system with 60 vertical levels (L60) and a model top of 0.04 hPa. The adjoint-DAS tools developed at NRL facilitated all FSI computations and have been extended in our work to incorporate the innovation-weight parameterization. The specific goals of the FSI analysis and parameter tuning procedure are to identify and address systematic deficiencies in the way the information provided by various observing system components is processed in NAVDAS-AR and achieve a weighting configuration capable of sustaining an improved performance over an extended time period, suitable for use in an operational environment. Not all of the theoretical features of the adaptive framework outlined in Fig. 2 have been exercised and in particular, we have not attempted to optimize parameter values at each assimilation cycle.

The experiments are valid for the time period 1 April–31 May 2018 and include all 6-h analyses and the associated 24-h forecasts produced by NAVDAS-AR/NAVGEM. The verification fields have been specified from the THORPEX Interactive Grand Global Ensemble (TIGGE; Bougeault et al. 2010) datasets produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) model, retrieved at 0.5° resolution with 18 vertical pressure levels and interpolated to the NAVGEM Gaussian grid at T119 spatial resolution and 60 vertical levels in sigma-pressure coordinates.

The experimental setup has two distinctive stages: parameter tuning and validation. Tuning of the innovation-weight parameters in NAVDAS-AR is performed for the time period of 1–14 April 2018 and validation of the tuned system is performed for 15 April–31 May 2018.

The parameter tuning stage is directly aimed at improving the DAS performance on reducing the initial-condition errors and implemented with a verification state specified at the initial time *t*_{0}. In this particular context, the FSI represents the sensitivity of the analysis error with respect to the innovation weights. Throughout the tuning stage, a systematic assessment of the weighting procedure has been performed over time intervals of increased length, ranging from 24 to 120 h. In this process, adjustments to the weight parameterization and the assigned parameter values have been implemented, with the FSI (33) reevaluated in the most recent weight configuration. The performance of the weighted NAVDAS-AR/NAVGEM system is assessed during the validation stage in terms of the reduction in the analysis errors *and* the 24-h model forecast errors, as compared with the control DAS. The specification of the verification fields at the initial-time *t*_{0} is a distinctive feature from traditional FSO/FSOI studies that rely on self-analysis fields and relates directly to the way information from observations is processed in the DAS. This experimental setup alleviates ambiguities in the parameter tuning associated with the model forecast errors (biases), forecast lead time, and self-analyses (Gelaro et al. 2007; Privé et al. 2021). The FSOI analysis is performed for both initial condition errors and the 24-h forecast errors to provide further insight on the role of observations at various stages of the assimilation and forecast process. In all computations, the error measure is taken as a dry total energy norm (Rosmond 1997) truncated to NAVGEM vertical levels 7–60 (atmospheric region below ∼0.50 hPa). The interpretation of the numerical results must take into account the limitations associated with the energy norm selection that emphasizes the tropospheric region of the atmosphere (Todling 2013; Diniz and Todling 2020). Notwithstanding these limitations, the energy norm has been the FSOI metric used most commonly in the NWP community, and it is also the metric with which we have most-extensive baseline of results for comparison.

### a. Training stage: FSI guidance and parameter tuning

NAVDAS-AR assimilates observations of atmospheric temperature (*T*), wind longitudinal (*u*) and latitudinal (*υ*) components, wind speed (ws), relative humidity (rh), surface pressure (*p*), radiance, and bending angle. A summary of the observation types and the associated observed variables is given in Table 2. Over the time period of the study, only clear-sky radiances have been assimilated and with the Cross-track Infrared Sounder (CrIS) data at nominal spectral resolution.

Observation types and observed parameters in NAVDAS-AR. The third column provides the default values assigned to the innovation weights in EXP, and the fourth column provides additional parameter corrections performed in the tuning process.

Insight on the potential deficiencies in the weighting of innovations was obtained from a comprehensive analysis of the FSI guidance in the control DAS (CTL) and only a few illustrative examples are discussed below. The average FSI values in CTL valid at 0000 UTC are shown for various observation types in Fig. 3a. Positive FSI values indicate that in general, innovations are overweighted and significantly increased sensitivity values are associated with SATWND data,^{2} as compared with other observation types. The vertical profile of the FSI for SATWND is shown in Fig. 4 and provides a consistent guidance of overweighted innovations throughout the vertical dimension and with larger sensitivities associated to upper-level wind data. Information on the fraction of observations from total SATWND number of observations (nobs) is also included in Fig. 4, e.g., observations assimilated above the 500-hPa level represent in average ∼64% of SATWND observations assimilated at 0000 UTC. In Fig. 3a, radiosondes (RAOBS) display the second largest FSI guidance as an overweighted data type. However, the vertical profile of the FSI analysis (not shown) indicated that relatively large values are associated mainly with the data collected in the upper levels of the atmosphere, particularly RAOBS winds above 100 hPa, where a reduction in the assigned innovation weight parameters was deemed to improve the analysis errors. By contrast, the FSI profile indicated that in general, RAOBS temperature innovations should receive an increased weight in the atmospheric region below 525 hPa. The analysis of radiance data revealed systematic tendencies in the FSI guidance associated with various instruments, closely determined by instrument channels and the sign (positive or negative) of the innovations, an indication of biases in innovations and potential issues in the variational bias correction.

The FSI values for various observation types in the DAS: (a) control and (b) weighted experiment. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The FSI values for various observation types in the DAS: (a) control and (b) weighted experiment. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The FSI values for various observation types in the DAS: (a) control and (b) weighted experiment. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical profile of FSI values for SATWND data. Displayed are average values in CTL and EXP valid at 0000 UTC with observations binned in 50-hPa pressure intervals.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical profile of FSI values for SATWND data. Displayed are average values in CTL and EXP valid at 0000 UTC with observations binned in 50-hPa pressure intervals.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical profile of FSI values for SATWND data. Displayed are average values in CTL and EXP valid at 0000 UTC with observations binned in 50-hPa pressure intervals.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

These examples illustrate the use of the FSI information as a diagnostic tool as well as challenges in the process of identifying a parameterization effective in improving the DAS performance. The practical approach taken was to assign a weight coefficient to each data type and analyze tendencies in the FSI guidance over the training period. Valuable insight on the initial weight specification was obtained from the quadratic approximation (45) and the estimates (47) evaluated in various norm components. This capability is further illustrated and discussed in the validation experiments (see Figs. 6 and 7). A first trial experiment performed with a parameter value of *s* = 0.5 proved that the quadratic (45) provided an accurate a priori representation of the experiment’s outcomes for all components of the energy norm. Results from the FSI analysis in the trial experiment are shown in Fig. 5. Specifically, the scaling factor of *s* = 0.5 applied to the analysis increment is significantly reducing the magnitude of the sensitivities to SATWND (the dominating FSI component) and GPS-RO observations; however, as indicated by the negative FSI values, this DAS configuration underweights the information from most observation types and in particular, RAOBS and MDCRS observations and radiances assimilated from the AMSU-A, ATMS, AIRS, and IASI instruments. These results indicate that a systematic assessment of the FSI guidance is needed to improve the performance of various observation types by tuning a multidimensional weight parameter.

The FSI values for various observation types in the trial experiment with a scaling factor of *s* = 0.5 applied to the analysis increment. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The FSI values for various observation types in the trial experiment with a scaling factor of *s* = 0.5 applied to the analysis increment. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The FSI values for various observation types in the trial experiment with a scaling factor of *s* = 0.5 applied to the analysis increment. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The experience gained from the training stage was that an effective tuning of the weights associated with conventional data, satellite-derived winds, and GPS-RO observations can be achieved using only a few parameters. We adopted a scaled version of the sign gradient descent update (50) based on the FSI guidance collected over several assimilation cycles of the training period and successive parameter corrections have been applied at time intervals distributed as follows: 2, 3, 4, 5, 7, 10, 14 April. An initial step length of *α* = 0.5 was taken for SATWND and GPS-RO observations and a more cautious initial step length of *α* = 0.25 was specified for conventional observations. The default values assigned to the innovation weights for various observation types are provided in Table 2 together with a summary of the additional corrections performed throughout the training stage based on guidance from the recalculated vertical distribution of the FSI. In particular, weight corrections have been applied to SATWND data where the assigned weight values ranged from 0.35 to 0.45 in the region below 200 hPa to 0.5–0.7 in the region above 200 hPa. RAOBS temperature observations received an increased weight of *s* = 1.5 in the region below 525 hPa, whereas all RAOBS observations in the region above 100 hPa received a weight of *s* = 0.5. To facilitate the comparison with CTL, the FSI values reevaluated in the final weighted configuration of the experiment (EXP) are shown for each observation type in Fig. 3b and included in Fig. 4 for the vertical profile of the SATWND data.

Assigning appropriate weight coefficients to radiance data proved to be a difficult task and in particular, the interaction between the flow dependent variational bias correction (VarBC) applied to radiances in NAVDAS-AR and the predefined innovation weighting, determined from a prior time period, requires further investigation. Several configurations have been tested and the final version adopted was a parameterization with instrument channels weighted by the tendencies in the sign associated with omf (positive or negative) and five geographical regions: polar, middle latitudes, and tropics. It is noticed that a weight adjustment *s* = 1 + *δs* to the innovation component *δy* may be interpreted as a correction *δsδy* and by analogy, an FSO-based approach may be used to adjust the weight assigned to the bias estimates produced by VarBC. Since VarBC computes bias corrections in an adaptive fashion, as part of the analysis scheme (Dee 2005; Eyre 2016), further research is needed to investigate the potential use of a flow dependent FSO-based procedure to improve the VarBC applied to radiances in a formulation suitable for operational implementation.

### b. Validation results

The ability of the weighted configuration EXP to improve the assimilation performance is investigated in validation experiments over the time period 15 April–31 May 2018 with data collected at all 6-h analysis cycles. Performance is assessed in terms of the error reduction in analyses (e0), the 6-h background forecasts (e6), and the associated 24-h (e24) and 30-h (e30) forecasts valid at *t*_{0} + 24 h. Results are analyzed in the dry total energy norm and individually for the vorticity, divergence, and temperature components. Bias reduction is investigated for analyses and 24-h forecasts and the FSOI assessment is calculated at *t*_{0} and *t*_{0} + 24 h for various observation types. Time series analysis of errors is used to investigate the ability of the EXP configuration to sustain an improved performance. Comparative results of the background fit to radiosonde observations are provided for CTL and EXP. The validation experiments have been initiated at 0600 UTC 15 April 2018 assimilation cycle when the 6-h forecast produced by the CTL analysis valid at 0000 UTC 15 April 2018 was used to provide the background state to both CTL and EXP.

Diagnosis of the optimal scaling to the analysis increment is an essential stage of the tuning process. Insight on the suboptimal observation performance with the default-weighted innovations (CTL) and the performance gain achieved in EXP through the innovation-weighting procedure may be obtained from the parametric representation to the analysis Eq. (35). A representative example is provided in Fig. 6 where the initial condition errors associated with the CTL and EXP in the first analysis cycle of the validation period, at 0600 UTC 15 April 2018, are shown as a function of the scaling factor applied to the analysis increment based on the quadratic approximation measure (45). The results in Fig. 6a indicate that in a single analysis cycle, a substantial reduction in the total energy error may be achieved by adjusting the *δ***x*** _{a}* scaling factor to a value of

*s*∼ 0.5.

The parametric representation of the initial condition error in CTL and EXP as a function of the scaling factor to the analysis increment. Results valid at 0600 UTC 15 Apr 2018 are displayed for errors measured in (a) the total energy norm and (b) the temperature component of the total energy norm.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The parametric representation of the initial condition error in CTL and EXP as a function of the scaling factor to the analysis increment. Results valid at 0600 UTC 15 Apr 2018 are displayed for errors measured in (a) the total energy norm and (b) the temperature component of the total energy norm.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The parametric representation of the initial condition error in CTL and EXP as a function of the scaling factor to the analysis increment. Results valid at 0600 UTC 15 Apr 2018 are displayed for errors measured in (a) the total energy norm and (b) the temperature component of the total energy norm.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

As outlined in Table 2, EXP assigns a default weight of *s* = 0.45 to the majority of innovations associated with the wind observations and we emphasize that the kinetic energy (divergence and vorticity components) represents a large fraction of the total energy norm (in our experiments ∼69%). The additional gains in the observation performance achieved from the tuning applied in EXP, as compared with the *δ***x*** _{a}* scaling of

*s*= 0.5, are best represented by the error reduction in the temperature component of the total energy norm. To illustrate this aspect, the parametric representation of the initial condition error associated with the temperature component is shown in Fig. 6b.

Estimates (47) to the optimal scaling factor

Estimated optimal scaling to the analysis increment for errors measured in the total energy norm and its vorticity and temperature components. Displayed are 28-cycle simple moving average values calculated over the validation period in CTL (solid line) and in EXP (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Estimated optimal scaling to the analysis increment for errors measured in the total energy norm and its vorticity and temperature components. Displayed are 28-cycle simple moving average values calculated over the validation period in CTL (solid line) and in EXP (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Estimated optimal scaling to the analysis increment for errors measured in the total energy norm and its vorticity and temperature components. Displayed are 28-cycle simple moving average values calculated over the validation period in CTL (solid line) and in EXP (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

*p*may be obtained as

*E*denotes the statistical expectation operator and

**x**

*and*

_{a}**x**

*, respectively. A time series of the values*

_{b}Time series of the estimated value Tr(**HK**)/*p* in CTL and EXP at various 0000 UTC analysis cycles of the validation period. The legend provides the corresponding mean value and standard deviation.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Time series of the estimated value Tr(**HK**)/*p* in CTL and EXP at various 0000 UTC analysis cycles of the validation period. The legend provides the corresponding mean value and standard deviation.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Time series of the estimated value Tr(**HK**)/*p* in CTL and EXP at various 0000 UTC analysis cycles of the validation period. The legend provides the corresponding mean value and standard deviation.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

#### 1) Error reduction in the e norm

Table 3 provides a summary of the average percentage reduction in the errors e0, e6, e24, and e30 produced by the EXP, as compared with CTL based on data collected from all analysis/forecast cycles over the validation period, measured in the total energy norm and the vorticity, divergence, and temperature components. The results indicate a substantially increased performance of the EXP in improving the initial condition errors (e0), ranging from an average reduction of 12.6% in the temperature component to an average reduction of 21.7% in the vorticity component. The EXP also provided noticeable improvements in all components of the e24 and e30 errors and in particular, the average reduction achieved in the total energy norm error was of 3.1% and 1.8%, respectively. A detailed analysis of the error reduction throughout the validation period is provided below.

Average percentage reduction in the state errors produced by EXP as compared with CTL, measured in various norm components.

##### (i) Performance on the analysis and background errors

The time series of the total energy errors e0 and e6 associated with CTL and EXP during the first week (28 analysis cycles) of the validation period are shown in Fig. 9a and the 28-cycle moving average of the errors throughout the validation period is shown in Fig. 9b. Remarkable in Fig. 9a is the fast rate of decay of the errors in EXP at the beginning of the validation period when the experiment corrects the initial state received from CTL, an indication that the benefit from the weighting procedure is compounded over several assimilation cycles. Also noticeable in Fig. 9a is the underperformance of observations in the CTL configuration in reducing the initial condition error at the 0600 and 1800 UTC analysis cycles. The 28-cycle moving average values of the e0 and e6 errors in the vorticity, divergence, and temperature components are shown in Fig. 10. Throughout the validation period, the EXP provided a significant reduction in both e0 and e6 errors. In particular, as shown in Fig. 10, the errors in all components of the 6-h forecasts produced by EXP are significantly lower than the e0 errors in CTL, indicating the potential for a 6-h gain in the forecast lead time. A comparison of the vertical distribution of the globally averaged e0 error components for vorticity, divergence, and temperature in CTL and EXP is shown in Fig. 11 together with the associated average percentage error reduction. The EXP produced a substantial reduction in the analysis errors at most vertical levels and in particular, a consistently large improvement was achieved between NAVGEM vertical levels 30–45 (middle-tropospheric region) where the error reduction in each norm component is in the range of 10%–20% or higher.

Time series of the e0 and e6 errors associated with CTL and EXP, measured in a dry total energy norm (J kg^{−1}). Displayed are (a) errors for the first 28 assimilation cycles and (b) the 28-cycle simple moving average values over the validation time period.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Time series of the e0 and e6 errors associated with CTL and EXP, measured in a dry total energy norm (J kg^{−1}). Displayed are (a) errors for the first 28 assimilation cycles and (b) the 28-cycle simple moving average values over the validation time period.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Time series of the e0 and e6 errors associated with CTL and EXP, measured in a dry total energy norm (J kg^{−1}). Displayed are (a) errors for the first 28 assimilation cycles and (b) the 28-cycle simple moving average values over the validation time period.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Time series of the e0 and e6 errors associated with CTL and EXP for the (a) vorticity, (b) divergence, and (c) temperature components of the total energy norm (J kg^{−1}). Displayed are the 28-cycle simple moving average values over the validation time period.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Time series of the e0 and e6 errors associated with CTL and EXP for the (a) vorticity, (b) divergence, and (c) temperature components of the total energy norm (J kg^{−1}). Displayed are the 28-cycle simple moving average values over the validation time period.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Time series of the e0 and e6 errors associated with CTL and EXP for the (a) vorticity, (b) divergence, and (c) temperature components of the total energy norm (J kg^{−1}). Displayed are the 28-cycle simple moving average values over the validation time period.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical distribution of the globally averaged e0 errors in CTL and EXP for the vorticity, divergence, and temperature components of the total energy norm (J kg^{−1}). The panels on the right display the corresponding average percentage reduction of the errors produced by EXP, as compared with CTL.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical distribution of the globally averaged e0 errors in CTL and EXP for the vorticity, divergence, and temperature components of the total energy norm (J kg^{−1}). The panels on the right display the corresponding average percentage reduction of the errors produced by EXP, as compared with CTL.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical distribution of the globally averaged e0 errors in CTL and EXP for the vorticity, divergence, and temperature components of the total energy norm (J kg^{−1}). The panels on the right display the corresponding average percentage reduction of the errors produced by EXP, as compared with CTL.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

##### (ii) Performance on the model forecast errors

In average, EXP produced improved e24 and e30 errors in all norm components, as compared with CTL, and with increased percentage reductions in the 24-h forecast errors. The 28-cycle moving average values of the errors in analysis forecasts (e24) and background forecasts (e30) produced by CTL and EXP are shown in Fig. 12. A large improvement was achieved in the divergence component of the forecast errors where the average EXP e30 errors are close to the average CTL e24 errors. However, the 6-h gain in the forecast time lead achieved by EXP in the e6 errors has not been materialized in the e30 errors of the vorticity and temperature components. In particular, as noted in Table 3 and shown in Figs. 10 and 12, whereas the initial condition errors in the vorticity component exhibit a large percentage reduction in EXP, consistent throughout the validation period, the associated percentage reduction in the 24- and 30-h forecasts is significantly lower. Possible factors include the accumulated errors in the model forecast and the fast growth of initial-condition errors that is not properly reflected in the e0 and e6 measures. The vertical profile of the globally averaged e24 errors in CTL and EXP is shown in Fig. 13 together with the corresponding average percentage error reduction achieved in EXP. In general, EXP provided error improvements in all norm components and at the majority of model levels. However, by comparison with the vertical distribution of the e0 errors in Fig. 11, some components of the e24 errors are increased in EXP at various model levels as noticed in particular, for the vorticity component near the surface.

As in Fig. 10, for the e24 and e30 errors.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

As in Fig. 10, for the e24 and e30 errors.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

As in Fig. 10, for the e24 and e30 errors.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

As in Fig. 11, for the e24 errors.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

As in Fig. 11, for the e24 errors.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

As in Fig. 11, for the e24 errors.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

#### 2) Bias analysis

Biases in the analyses and forecasts produced by CTL and EXP have been investigated to assess whether the error reduction in EXP produced an increase in bias. The time series of the biases, evaluated against ECMWF verification fields at each vertical level over the global horizontal domain, has been analyzed in CTL and EXP and in general, biases have been reduced in the EXP analyses at the majority of vertical levels. The root-mean-square (rms) values associated with the time series of biases in the analyses and in the 24-h forecasts have been evaluated for the *T*, *u*, *υ* variables and the vertical distribution of the difference rms(bias_EXP) − rms(bias_CTL), is displayed in Fig. 14. Negative values associated with the analyses of *T*, *u*, *υ* variables at the majority of vertical levels show that in general, rms(bias_EXP) values are lower than rms(bias_CTL) values and provide an indication of the ability of the weighting procedure to improve the bias in analyses. For the 24-h forecasts, temperature biases have been in general improved at most vertical levels and only small differences are associated with the biases in the wind components.

Vertical distribution of the difference between the rms values of the time series of biases in EXP and the rms values of the time series of biases in CTL. Displayed are differences in rms values associated to biases in analyses (solid line) and to biases in the 24-h forecasts (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical distribution of the difference between the rms values of the time series of biases in EXP and the rms values of the time series of biases in CTL. Displayed are differences in rms values associated to biases in analyses (solid line) and to biases in the 24-h forecasts (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical distribution of the difference between the rms values of the time series of biases in EXP and the rms values of the time series of biases in CTL. Displayed are differences in rms values associated to biases in analyses (solid line) and to biases in the 24-h forecasts (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

#### 3) Background fit to radiosonde observations

A valuable indication of the improved weighting may be obtained from the statistical analysis of the omf residuals (innovations) in CTL and EXP. In Fig. 15 we provide the vertical profile of the rms values of the omf for radiosonde (RAOBS) temperature and zonal wind observations in CTL and EXP, evaluated at 0000 UTC analyses over the validation period. For temperature observations, the rms values are shown in Fig. 15a and the normalized values rms(EXP)/rms(CTL) are shown in Fig. 15b. For zonal wind observations, the rms values are shown in Fig. 15c and the normalized values rms(EXP)/rms(CTL) are shown in Fig. 15d. The normalized rms values in Fig. 15b indicate that in general, the EXP provided an improved background fit to RAOBS temperature data, with largest relative improvement in the atmospheric region between 500 and 800 hPa. The normalized rms values in Fig. 15d indicate that in general, the EXP also provided an improved background fit to RAOBS data for the zonal wind component, although an increase in the EXP rms values is noticed in the region below 950 hPa. Similar results (not shown) have been obtained for the background fit to RAOBS data for the meridional wind component.

Vertical profile of rms values of the innovations (omf) associated with radiosonde observations in CTL and EXP. Displayed are values valid at 0000 UTC for (a) temperature and (c) zonal wind observations. (b),(d) The corresponding normalized values rms(EXP)/rms(CTL) are shown.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical profile of rms values of the innovations (omf) associated with radiosonde observations in CTL and EXP. Displayed are values valid at 0000 UTC for (a) temperature and (c) zonal wind observations. (b),(d) The corresponding normalized values rms(EXP)/rms(CTL) are shown.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical profile of rms values of the innovations (omf) associated with radiosonde observations in CTL and EXP. Displayed are values valid at 0000 UTC for (a) temperature and (c) zonal wind observations. (b),(d) The corresponding normalized values rms(EXP)/rms(CTL) are shown.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

#### 4) Observation impact assessment

As noticed in Fig. 9, the innovation-weighting procedure applied in EXP has significantly reduced both the initial condition errors and the 6-h forecast errors, as compared with CTL. Larger observation impacts *δe*_{0} = e0–e6 are achieved in EXP with analysis increments *δ***x*** _{a}* of smaller magnitude, as compared with CTL. This aspect is shown in Fig. 16 where the 28-cycle simple moving average values of the magnitude of

*δ*

**x**

*calculated in CTL and EXP are displayed throughout the validation period. Therefore, the analysis and background forecasts are closer in EXP, as compared with CTL. In general, an improved data assimilation and forecast system will be characterized by smaller analysis and forecast errors and reduced observation impacts. As compared with EXP, in the CTL configuration the reduced observation impacts*

_{a}*δe*

_{0}are associated with larger analysis increments and provide further indication of suboptimal observation performance due to overweighted innovations. The interpretation of the results must account for the fact that our experiments rely on ECMWF verification fields that are independent from the analyses and forecasts produced by NAVDAS-AR/NAVGEM.

The magnitude of the analysis increment measured in the total energy norm and its vorticity and temperature components. Displayed are the 28-cycle simple moving average values over the validation time period calculated in CTL (solid line) and in EXP (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The magnitude of the analysis increment measured in the total energy norm and its vorticity and temperature components. Displayed are the 28-cycle simple moving average values over the validation time period calculated in CTL (solid line) and in EXP (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The magnitude of the analysis increment measured in the total energy norm and its vorticity and temperature components. Displayed are the 28-cycle simple moving average values over the validation time period calculated in CTL (solid line) and in EXP (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

FSOI has been performed in CTL and EXP to assess the impact of various observation types in reducing the initial-condition errors (*δe*_{0}) and in reducing the 24-h forecast errors (*δe*_{24}). The FSOI evaluation is implemented in CTL based on (15) and in EXP based on the weighted formulation (31). Comparative results of observation impact on *δe*_{0} in CTL and EXP are provided in Fig. 17. The dominant feature in Fig. 17 is the large detrimental impact on the initial condition errors associated with SATWND data in CTL, which is due to significantly overweighted innovations. By comparison, as a result of the improved specification of the innovation weights, in EXP the SATWND data are the second largest contributor to the error reduction, only surpassed by RAOBS. Also notable in Fig. 17 is the improved use of GPS-RO and LEOGEO observations in EXP, as compared with CTL.

Observation impact (FSOI) on reducing the initial condition errors: (a) CTL and (b) EXP.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Observation impact (FSOI) on reducing the initial condition errors: (a) CTL and (b) EXP.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Observation impact (FSOI) on reducing the initial condition errors: (a) CTL and (b) EXP.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The FSOI assessment for *δe*_{24} in CTL and EXP is provided in Fig. 18. A major distinction between the CTL results for *δe*_{0} and *δe*_{24} is the beneficial impact of SATWND data on the 24-h forecast errors. By comparison with CTL, the FSOI analysis in EXP identifies SATWND as the largest contributor to the reduction of 24-h forecast errors and with a significantly larger impact value as compared with CTL. In general, other observation types exhibit only small variations in their 24-h forecast error impacts between CTL and EXP.

Observation impact (FSOI) on reducing the 24-h forecast errors: (a) CTL and (b) EXP.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Observation impact (FSOI) on reducing the 24-h forecast errors: (a) CTL and (b) EXP.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Observation impact (FSOI) on reducing the 24-h forecast errors: (a) CTL and (b) EXP.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

##### FSOI analysis for SATWND data

The weighting applied to SATWND data in EXP increased the average fraction of observations with a beneficial impact on the error reduction, from 48.9% in CTL to 52.3% in EXP for *δe*_{0} and from 50.1% in CTL to 51% in EXP for *δe*_{24}. A comparison between the vertical profiles of the FSOI analysis for SATWND data in CTL and EXP is provided in Fig. 19. The results obtained in CTL are shown in Figs. 19a and 19c and indicate that, in general, the SATWND data produced an increase in the initial condition errors; however, the upper-level observations contributed to the 24-h forecast error reduction. The EXP results are shown in Figs. 19b and 19d and indicate that the weighting procedure significantly improved the use of SATWND data both for *δe*_{0} and *δe*_{24} errors, at a large majority of the vertical levels.

Vertical profile of SATWND data impact on initial condition errors (a) CTL and (b) EXP and on 24-h forecast errors (c) CTL and (d) EXP. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical profile of SATWND data impact on initial condition errors (a) CTL and (b) EXP and on 24-h forecast errors (c) CTL and (d) EXP. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Vertical profile of SATWND data impact on initial condition errors (a) CTL and (b) EXP and on 24-h forecast errors (c) CTL and (d) EXP. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

In the interpretation of the *δe*_{0} and *δe*_{24} results, it is noticed that the observation impact on forecasts errors is closely determined by the growth of the initial-condition errors, which is not represented in the *δe*_{0} measure, and therefore, observations with a small impact on *δe*_{0} may in fact provide a much larger impact on reducing the forecast errors. Our results corroborate the findings of Privé et al. (2021) where observation impacts evaluated for forecast errors in the range of 6–48 h have shown significant dependencies on the specified forecast lead time, including sign variations. Our results also indicate that attempts to address deficiencies in the DAS based on the FSOI assessment of forecast errors may lead to ambiguous results. A judicious assessment of the observation performance must include complementary diagnostics for quantifying the observation influence on the analysis such as observation-space diagnostics and estimates based on the degrees of freedom for signal (Cardinali et al. 2004; Lupu et al. 2011).

## 5. Summary and further research directions

This work presents a new methodology for embedding FSO information into a feedback mechanism aimed at improving the observation performance in data assimilation. The main ingredients are the innovation-weight parameterization of the analysis equation, the FSO-based evaluation of the gradient (FSI) to identify a descent direction in the parameter space, and a computationally efficient mechanism for step length selection guided by a quadratic approximation to the parametric forecast error measure. Unlike data denial approaches, our methodology retains all observations assimilated in the DAS and improvements are sought through diagnosis and adaptive adjustments to innovation weight parameters. This approach may be implemented and tested in a variety of data assimilation systems where FSO tools have been developed.

The specification of the validation state and the selection of the error metric are key ingredients for the procedure to be effective in improving the DAS performance. Feasible selections of the validation fields include high-resolution reanalyses valid at the initial time and aimed directly at improving the initial-condition to the model forecasts, observation space metrics or, used in a cautious fashion, self-analyses with a time-varying forecast lead time. The total energy norm adopted in this study facilitated the FSI calculations based on the software tools developed at NRL for routine FSO/FSOI monitoring. Covariance metrics provide an improved quantification of the errors and computationally feasible implementations can be obtained from ensemble forecast estimates.

Diagnosis of suboptimal observation performance and identification of high-impact parameters require a systematic analysis of tendencies in the FSI associated with various data types and geographical regions. High impact innovation-weight parameters identified through FSI are then iteratively optimized over a time period spanning various assimilation cycles. The dimension of the parameter vector of weights may be adaptively refined during the training stage; however, it is emphasized that short-term gains that may be achieved from tuning a high-dimensional parameterization do not entail an improved long-term performance. The practical goal is to identify and correct a relatively small number of key parameters that capture systematic deficiencies in the way information is processed in the DAS and have a potentially large impact on improving the observation performance. As shown in our experiments, this process may be initiated through diagnosis and tuning of the scaling applied to the analysis increment and followed by additional refinements in the specified parameterization. The practical ability to implement this methodology for NWP applications has been demonstrated in preliminary numerical experiments with the 4D-Var NAVDAS-AR system and the NAVGEM forecasting model. It is emphasized that the availability of the verification fields at the initial time, here produced by ECMWF, facilitated the parameter tuning procedure implemented in NAVDAS-AR. Further research, testing, and validation experiments are necessary to develop an approach suitable for operational implementation.

Theoretically, the parameterization (16)/(18) may be extended to a nonlinear analysis produced through multiple outer loop iterations and FSI calculations may be achieved based on the FSO evaluation presented by Trémolet (2008). As new software capabilities are developed, further applications are envisaged for adaptive tuning of innovation-weight parameters and in particular, a flow-dependent implementation for improved assimilation of radiance data. The extension of the PQC framework developed by Hotta et al. (2017a) to the FSI-based optimal weight formulation could provide a feasible approach to advance research in this area. Flow-dependent adaptive weighting of innovations based on a short-range forecast error guidance will also be investigated for goal-oriented applications such as targeting an extreme weather event.

This is the vector identity −2(**u**, **v**) = ǁ**v**ǁ^{2} + ǁ**u**ǁ^{2} − ǁ**u** + **v**ǁ^{2} applied to

SATWND incorporates observations from NOAA *GOES-15* and *GOES-16*, EUMETSAT *Meteosat-8*–*Meteosat-11*, JMA *Himawari-8*, and INM INSAT satellites.

## Acknowledgments.

We are grateful to Dr. Ricardo Todling at NASA Global Modeling and Assimilation Office for many discussions and insightful comments throughout the development of this work. We thank two anonymous reviewers for their constructive criticism that helped to improve the presentation. This research is supported by the Office of Naval Research (ONR) through the NRL Base Program PE 62435N. The first author was supported by the NRL BAA 75-18-01 under Award N00173-19-2-C003. This work is based on TIGGE data obtained from the ECMWF Data Server. The THORPEX Interactive Grand Global Ensemble (TIGGE) is an initiative of the World Weather Research Programme (WWRP).

## Data availability statement.

The NAVDAS-AR analyses fields used in this research are archived at the Naval Research Laboratory, and the TIGGE validation fields have been obtained via data archive portals at ECMWF.

## REFERENCES

Baker, N. L., and R. Daley, 2000: Observation and background adjoint sensitivity in the adaptive observation-targeting problem.

,*Quart. J. Roy. Meteor. Soc.***126**, 1431–1454, https://doi.org/10.1002/qj.49712656511.Beck, A., and L. Tetruashvili, 2013: On the convergence of block coordinate descent type methods.

,*SIAM J. Optim.***23**, 2037–2060, https://doi.org/10.1137/120887679.Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble.

,*Bull. Amer. Meteor. Soc.***91**, 1059–1072, https://doi.org/10.1175/2010BAMS2853.1.Buehner, M., P. Du, and J. Bedard, 2018: A new approach for estimating the observation impact in ensemble–variational data assimilation.

,*Mon. Wea. Rev.***146**, 447–465, https://doi.org/10.1175/MWR-D-17-0252.1.Cardinali, C., 2009: Monitoring the observation impact on the short-range forecast.

,*Quart. J. Roy. Meteor. Soc.***135**, 239–250, https://doi.org/10.1002/qj.366.Cardinali, C., 2018: Forecast sensitivity observation impact with an observation-only based objective function.

,*Quart. J. Roy. Meteor. Soc.***144**, 2089–2098, https://doi.org/10.1002/qj.3305.Cardinali, C., S. Pezzulli, and E. Andersson, 2004: Influence-matrix diagnostic of a data assimilation system.

,*Quart. J. Roy. Meteor. Soc.***130**, 2767–2786, https://doi.org/10.1256/QJ.03.205.Chen, T.-C., and E. Kalnay, 2019: Proactive quality control: Observing system simulation experiments with the Lorenz ’96 model.

,*Mon. Wea. Rev.***147**, 53–67, https://doi.org/10.1175/MWR-D-18-0138.1.Chen, T.-C., and E. Kalnay, 2020: Proactive quality control: Observing system experiments using the NCEP Global Forecast System.

,*Mon. Wea. Rev.***148**, 3911–3931, https://doi.org/10.1175/MWR-D-20-0001.1.Daescu, D. N., 2008: On the sensitivity equations of four-dimensional variational (4D-Var) data assimilation.

,*Mon. Wea. Rev.***136**, 3050–3065, https://doi.org/10.1175/2007MWR2382.1.Daescu, D. N., and R. Todling, 2009: Adjoint estimation of the variation in model functional output due to the assimilation of data.

,*Mon. Wea. Rev.***137**, 1705–1716, https://doi.org/10.1175/2008MWR2659.1.Daescu, D. N., and R. Todling, 2010: Adjoint sensitivity of the model forecast to data assimilation system error covariance parameters.

,*Quart. J. Roy. Meteor. Soc.***136**, 2000–2012, https://doi.org/10.1002/qj.693.Daescu, D. N., and R. H. Langland, 2013: Error covariance sensitivity and impact estimation with adjoint 4D-Var: Theoretical aspects and first applications to NAVDAS-AR.

,*Quart. J. Roy. Meteor. Soc.***139**, 226–241, https://doi.org/10.1002/qj.1943.Daescu, D. N., and R. H. Langland, 2016: Innovation-weight parametrization in data assimilation: Formulation and analysis with NAVDAS-AR/NAVGEM.

,*IFAC-PapersOnLine***49**, 176–181, https://doi.org/10.1016/j.ifacol.2016.10.159.Dee, D. P., 2005: Bias and data assimilation.

,*Quart. J. Roy. Meteor. Soc.***131**, 3323–3343, https://doi.org/10.1256/qj.05.137.Diniz, F. L. R., and R. Todling, 2020: Assessing the impact of observations in a multi-year reanalysis.

,*Quart. J. Roy. Meteor. Soc.***146**, 724–747, https://doi.org/10.1002/qj.3705.Eyre, J. R., 2016: Observation bias correction schemes in data assimilation systems: A theoretical study of some of their properties.

,*Quart. J. Roy. Meteor. Soc.***142**, 2284–2291, https://doi.org/10.1002/qj.2819.Gelaro, R., Y. Zhu, and R. M. Errico, 2007: Examination of various-order adjoint-based approximations of observation impact.

*Meteor. Z.*,**16**, 685–692, https://doi.org/10.1127/0941-2948/2007/0248.Gelaro, R., R. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX observation impact intercomparison experiment.

,*Mon. Wea. Rev.***138**, 4009–4025, https://doi.org/10.1175/2010MWR3393.1.Hogan, T. F., and Coauthors, 2014: The Navy Global Environmental Model.

,*Oceanography***27**(3), 116–125, https://doi.org/10.5670/oceanog.2014.73.Hotta, D., T.-C. Chen, E. Kalnay, Y. Ota, and T. Miyoshi, 2017a: Proactive QC: A fully flow-dependent quality control scheme based on EFSO.

,*Mon. Wea. Rev.***145**, 3331–3354, https://doi.org/10.1175/MWR-D-16-0290.1.Hotta, D., E. Kalnay, Y. Ota, and T. Miyoshi, 2017b: EFSR: Ensemble forecast sensitivity to observation error covariance.

,*Mon. Wea. Rev.***145**, 5015–5031, https://doi.org/10.1175/MWR-D-17-0122.1.Ishibashi, T., 2018: Adjoint-based observation impact estimation with direct verification using forward calculation.

,*Mon. Wea. Rev.***146**, 2837–2858, https://doi.org/10.1175/MWR-D-18-0037.1.Kalnay, E., Y. Ota, T. Miyoshi, and J. Liu, 2012: A simpler formulation of forecast sensitivity to observations: Application to ensemble Kalman filters.

*Tellus*,**64A**, 18462, https://doi.org/10.3402/tellusa.v64i0.18462.Kim, S.-M., and H. M. Kim, 2018: Effect of observation error variance adjustment on numerical weather prediction using forecast sensitivity to error covariance parameters.

*Tellus*,**70A**, 1–16, https://doi.org/10.1080/16000870.2018.1492839.Kim, S.-M., and H. M. Kim, 2019: Forecast sensitivity observation impact in the 4DVAR and hybrid-4DVAR data assimilation systems.

,*J. Atmos. Oceanic Technol.***36**, 1563–1575, https://doi.org/10.1175/JTECH-D-18-0240.1.Kotsuki, S., K. Kurosawa, and T. Miyoshi, 2019: On the properties of ensemble forecast sensitivity to observations.

,*Quart. J. Roy. Meteor. Soc.***145**, 1897–1914, https://doi.org/10.1002/qj.3534.Langland, R., and N. Baker, 2004: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system.

*Tellus*,**56**, 189–201, https://doi.org/10.3402/tellusa.v56i3.14413.Le Dimet, F.-X., H.-E. Ngodock, B. Luong, and J. Verron, 1997: Sensitivity analysis in variational data assimilation.

,*J. Meteor. Soc. Japan***75**, 245–255, https://doi.org/10.2151/jmsj1965.75.1B_245.Liu, J., and E. Kalnay, 2008: Estimating observation impact without adjoint model in an ensemble Kalman filter.

,*Quart. J. Roy. Meteor. Soc.***134**, 1327–1335, https://doi.org/10.1002/qj.280.Lorenc, A., and R. Marriott, 2014: Forecast sensitivity to observations in the Met Office global numerical weather prediction system.

,*Quart. J. Roy. Meteor. Soc.***140**, 209–224, https://doi.org/10.1002/qj.2122.Lupu, C., P. Gauthier, and S. Laroche, 2011: Evaluation of the impact of observations on analyses in 3D- and 4D-Var based on information content.

,*Mon. Wea. Rev.***139**, 726–737, https://doi.org/10.1175/2010MWR3404.1.Lupu, C., C. Cardinali, and A. P. McNally, 2015: Adjoint-based forecast sensitivity applied to observation-error variance tuning.

,*Quart. J. Roy. Meteor. Soc.***141**, 3157–3165, https://doi.org/10.1002/qj.2599.Moulay, E., V. Léchappé, and F. Plestan, 2019: Properties of the sign gradient descent algorithms.

,*Inf. Sci.***492**, 29–39, https://doi.org/10.1016/j.ins.2019.04.012.Ota, Y., J. C. Derber, E. Kalnay, and T. Miyoshi, 2013: Ensemble-based observation impact estimates using the NCEP GFS.

*Tellus*,**65A**, 20038, https://doi.org/10.3402/tellusa.v65i0.20038.Privé, N., R. M. Errico, R. Todling, and A. El Akkraoui, 2021: Evaluation of adjoint-based observation impacts as a function of forecast length using an observing system simulation experiment.

,*Quart. J. Roy. Meteor. Soc.***147**, 121–138, https://doi.org/10.1002/qj.3909.Rosmond, T., 1997: A technical description of the NRL adjoint modeling system. NRL Rep. NRL/MR/7532/97/7230, 62 pp., https://doi.org/10.21236/ADA330960.

Rosmond, T., and L. Xu, 2006: Development of NAVDAS-AR: Non-linear formulation and outer loop tests.

,*Tellus***58A**, 45–58, https://doi.org/10.1111/j.1600-0870.2006.00148.x.Ruston, B., and S. Healy, 2021: Forecast impact of FORMOSAT-7/COSMIC-2 GNSS radio occultation measurements.

,*Atmos. Sci. Lett.***22**, e1019, https://doi.org/10.1002/asl.1019.Todling, R., 2013: Comparing two approaches for assessing observation impact.

,*Mon. Wea. Rev.***141**, 1484–1505, https://doi.org/10.1175/MWR-D-12-00100.1.Todling, R., and A. El Akkraoui, 2018: The GMAO hybrid ensemble-variational atmospheric data assimilation system: Version 2.0. NASA Tech. Rep. Series NASA/TM-2018-104606, Vol. 50, 167 pp.

Trémolet, Y., 2008: Computation of observation sensitivity and observation impact in incremental variational data assimilation.

,*Tellus***60A**, 964–978, https://doi.org/10.1111/j.1600-0870.2008.00349.x.Wright, S. J., 2015: Coordinate descent algorithms.

,*Math. Program.***151**, 3–34, https://doi.org/10.1007/s10107-015-0892-3.Xu, L., T. Rosmond, and R. Daley, 2005: Development of NAVDAS-AR: Formulation and initial tests of the linear problem.

*Tellus*,**57A**, 546–559, https://doi.org/10.3402/tellusa.v57i4.14710.