An FSO-Based Optimization Framework for Improved Observation Performance: Theoretical Formulation and Experiments with NAVDAS-AR/NAVGEM

Dacian N. Daescu aPortland State University, Portland, Oregon

Search for other papers by Dacian N. Daescu in
Current site
Google Scholar
PubMed
Close
and
Rolf H. Langland bNaval Research Laboratory, Monterey, California

Search for other papers by Rolf H. Langland in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

The forecast sensitivity to observations (FSO) is embedded into a new optimization framework for improving the observation performance in atmospheric data assimilation. Key ingredients are introduced as follows: the innovation-weight parameterization of the analysis equation, the FSO-based evaluation of the forecast error gradient to parameters, a line search approach to optimization, and an efficient mechanism for step length specification. This methodology is tested in preliminary numerical experiments with the Naval Research Laboratory Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR) and the U.S. Navy’s Global Environmental Model (NAVGEM) at a T425L60 resolution. The experimental setup relies on a verification state produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) to estimate the analysis and short-range forecast errors. Parameter tuning is implemented in a training stage valid for 1–14 April 2018 and aimed at improving the use of assimilated observations in reducing the initial-condition errors. Assessment is carried out for 15 April–31 May 2018 to investigate the performance of the weighted assimilation system in reducing the errors in analyses and 24-h model forecasts. In average, as compared with the control run and verified against the ECMWF analyses, the weighted approach provided 17.4% reduction in analysis errors and 3.1% reduction in 24-h forecast errors, measured in a dry total energy norm. Observation impacts are calculated to assess the use of various observation types in reducing the analysis and forecast errors. In particular, assimilation of satellite wind data is significantly improved through the innovation-weighting procedure.

Significance Statement

A new methodology is introduced to improve the information content of observations in numerical weather prediction (NWP). The computational procedure relies on observation sensitivity tools developed at all major NWP centers, and therefore, it appeals to a large audience for implementation and testing in practical applications. Our approach retains all available observations and provides a judicious optimization-based guidance to identify system deficiencies and improve the weighting assigned to various observing system components. The practical ability to implement this methodology is demonstrated in a computational environment that features all elements necessary for NWP applications. Preliminary results show that proper specification of the innovation weights can significantly improve the observation performance in reducing both the analysis errors and short-range forecast errors.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Dacian N. Daescu, daescu@pdx.edu

Abstract

The forecast sensitivity to observations (FSO) is embedded into a new optimization framework for improving the observation performance in atmospheric data assimilation. Key ingredients are introduced as follows: the innovation-weight parameterization of the analysis equation, the FSO-based evaluation of the forecast error gradient to parameters, a line search approach to optimization, and an efficient mechanism for step length specification. This methodology is tested in preliminary numerical experiments with the Naval Research Laboratory Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR) and the U.S. Navy’s Global Environmental Model (NAVGEM) at a T425L60 resolution. The experimental setup relies on a verification state produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) to estimate the analysis and short-range forecast errors. Parameter tuning is implemented in a training stage valid for 1–14 April 2018 and aimed at improving the use of assimilated observations in reducing the initial-condition errors. Assessment is carried out for 15 April–31 May 2018 to investigate the performance of the weighted assimilation system in reducing the errors in analyses and 24-h model forecasts. In average, as compared with the control run and verified against the ECMWF analyses, the weighted approach provided 17.4% reduction in analysis errors and 3.1% reduction in 24-h forecast errors, measured in a dry total energy norm. Observation impacts are calculated to assess the use of various observation types in reducing the analysis and forecast errors. In particular, assimilation of satellite wind data is significantly improved through the innovation-weighting procedure.

Significance Statement

A new methodology is introduced to improve the information content of observations in numerical weather prediction (NWP). The computational procedure relies on observation sensitivity tools developed at all major NWP centers, and therefore, it appeals to a large audience for implementation and testing in practical applications. Our approach retains all available observations and provides a judicious optimization-based guidance to identify system deficiencies and improve the weighting assigned to various observing system components. The practical ability to implement this methodology is demonstrated in a computational environment that features all elements necessary for NWP applications. Preliminary results show that proper specification of the innovation weights can significantly improve the observation performance in reducing both the analysis errors and short-range forecast errors.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Dacian N. Daescu, daescu@pdx.edu

1. Introduction

Rooted in the general theory of sensitivity analysis in data assimilation (Le Dimet et al. 1997), the evaluation of the model forecast sensitivity to observations (FSO) provides an efficient tool for assessing the observing system performance in numerical weather prediction (NWP). The adjoint-based derivation of FSO was put forward by Baker and Daley (2000) and further extended by Langland and Baker (2004) to provide an all-at-once assessment of the observation impact on reducing a functional aspect of the model forecast error, including the impact of various observation types categorized by their distribution in the time-space domain. Forecast sensitivity and observation impact (FSOI) methodologies have been formulated and implemented for variational, ensemble, and hybrid data assimilation systems (DAS) and are now incorporated in routine observation monitoring activities at operational NWP centers (Liu and Kalnay 2008; Cardinali 2009; Gelaro et al. 2010; Kalnay et al. 2012; Todling 2013; Ota et al. 2013; Lorenc and Marriott 2014; Buehner et al. 2018; Ishibashi 2018; Kim and Kim 2019; Ruston and Healy 2021). FSOI tools are used along with observing system experiments (OSEs) in synergistic efforts to assess the interaction between various data types in the DAS and their role in improving the model forecasts.

The experience gained from FSOI studies is that in practice, only a small majority of the assimilated observations improve the forecasts (Gelaro et al. 2010; Lorenc and Marriott 2014; Kotsuki et al. 2019). However, caution must be exercised in the design of a data thinning strategy that is effective in reducing the forecast errors. Research to directly embed sensitivity information into a feedback mechanism for improving the observation performance is receiving an increased interest. The FSO has been extended to include error covariance parameters (Daescu 2008) and provide guidance for tuning the observation error covariance representation in the DAS (Daescu and Langland 2013; Lupu et al. 2015; Hotta et al. 2017b; Kim and Kim 2018). A method to flow dependent proactive quality control (PQC) based on FSOI guidance was developed by Hotta et al. (2017a) using the ensemble forecast sensitivity to observations (EFSO) approach of Kalnay et al. (2012) and positive outcomes from PQC through data denial OSEs have been obtained by Chen and Kalnay (2019, 2020).

Our work seeks to improve the observation performance through an improved weighting of the observed-minus-forecast (omf) residuals (innovation vector), without removing observations from the DAS. A new FSO-based optimization methodology is developed to identify and correct deficiencies in the assigned innovation weights through an iterative gradient-based optimization process. By performing parameter updates carried over a time period and effective in reducing a forecast error aspect, the expected outcome is to correct systematic deficiencies in the way information is processed in the DAS and reach a configuration that will sustain improved performance. This methodology is implemented and tested in numerical experiments performed with the Naval Research Laboratory (NRL) Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR; Xu et al. 2005; Rosmond and Xu 2006) and the U.S. Navy’s Global Environmental Model (NAVGEM; Hogan et al. 2014). The article is organized as follows. Section 2 outlines the FSO and FSOI evaluation in a formulation that is relevant to our study. In section 3, key ingredients of a general theoretical formulation are introduced and discussed as follows: specification of the error functional, the parameterization of the analysis equation, the FSO-based identification of a descent direction for gradient-based optimization, and a computationally efficient procedure for step length selection. Section 4 presents the experimental setup, the parameter tuning process, and a detailed analysis of the validation results. Summary and further research directions are in section 5.

2. Outline of adjoint-based FSO and FSOI

Given a prior estimate (background forecast) xbn to the initial conditions of an atmospheric model and observational data yp, consider the analysis equation
xa=xb+Kδy,
where δy = yh(xb) is the innovation vector, h:np is the (nonlinear) observation operator, and the gain matrix Kn×p defines the assimilation algorithm. Formally, (1) represents the analysis produced by an ensemble-based DAS or a variational DAS implementing a single outer loop iteration, e.g., in the four-dimensional variational system (4D-Var) NAVDAS-AR used in this study,
K=BHbT(HbBHbT+R)1n×p,
where Bn×n and Rp×p represent in the DAS, respectively, the background error covariance and the observation error covariance, and Hbp×n is the Jacobian matrix of h evaluated at xb.
The FSO and FSOI are evaluated for a selected functional aspect of the short-range forecast errors. Given a generic initial condition x0 valid at the analysis time t0 and a forecast lead time τ (e.g., τ = 24 h), a measure of the forecast error at tf = t0 + τ is typically specified as
e(x0)=x0fxυE2=(x0fxυ)TE(x0fxυ),
where
x0f=Mt0,tf(x0)
denotes the nonlinear model forecast initiated at t0 from x0 and valid at tf, xυ is the verifying analysis at tf, and E is a symmetric positive definite matrix that defines the norm in the state space. A total energy norm is often used in the error measure (3) and observation-space metrics have been also considered (Todling 2013; Cardinali 2018). The gradient of e with respect to the initial condition x0 is
g0=defx0e(x0)=2M0TE(x0fxυ)n,
where M0 denotes the Jacobian matrix (tangent linear model) of the nonlinear model Mt0,tf evaluated at x0 and M0T is its transpose (adjoint model). The errors in the analysis forecast xaf and in the background forecast xbf are evaluated from (3) and (4) at x0 = xa and x0 = xb, respectively,
e(xa)=xafxυE2,e(xb)=xbfxυE2,
and (5) is used to provide the corresponding gradients,
ga=defex0(xa)=2MaTE(xafxυ)n,
gb=defx0e(xb)=2MbTE(xbfxυ)n.

The particular case when a verification state is available at the initial time, tf = t0, corresponds to x0f=x0 and M0 is the identity matrix. In this case, e(xa) and e(xb) measure, respectively, the error in the analysis state and the error in the background state.

a. FSO evaluation

By viewing the analysis (1) as a function of observations, xa = xa(y), the FSO is defined as the gradient of the functional e[xa(y)] with respect to y. From (1) and (7), with chain rule differentiation the FSO is expressed as
ye(xa)=yxax0e(xa)=KTgap.
The FSO vector may be used in conjunction with the observed-minus-analysis (oma) residual yh(xa) to assess the forecast sensitivity with respect to weight parameters in the observation error covariance representation
Ri(sio)=sioRi,iI.
The parameterization (10) corresponds to a block diagonal specification of the observation error covariance matrix, R=diag(Ri),Ripi×pi for iI, and sio>0 are scalar coefficients used to adjust the weight given in the DAS to various observation data components yipi,iI. As shown by Daescu and Todling (2010), the sensitivity of the forecast error e(xa) to the observation error covariance weight parameters (FSR) in the reference DAS specification sio=1,iI, is expressed as
e(xa)sio=[yihi(xa)]Tyie(xa),iI.

This approach was implemented for tuning the covariance model R in a 4D-Var DAS by Lupu et al. (2015) and Kim and Kim (2018), and the ensemble-based formulation to FSR evaluation was developed by Hotta et al. (2017b).

b. FSOI evaluation

The impact of all observations in the DAS is quantified by the change in the e measure,
δe=e(xa)e(xb).
As shown by Langland and Baker (2004), an observation-space approximation δe˜ to (12) may be expressed as
δeδe˜=defδyTg¯a,bo,
with the observation space vector g¯a,bop defined as
g¯a,bo=KT[12ga+12gb].
The observation impact (FSOI) associated to any observation component yipi is evaluated as
δe˜i=δyiT[g¯a,bo]i.

Data components with δe˜i<0 contribute to the forecast error norm reduction (beneficial), whereas data components with δe˜i>0 provide an increase in the forecast error norm (detrimental). Essentially, all FSOI measures rely on an approximation to δe expressed as (13) and are distinguished by their specification and evaluation of the weight vector (14). Analysis of FSOI approximation properties, derivation of higher-order measures, and extensions to fully nonlinear variational schemes are provided in references Gelaro et al. (2007), Trémolet (2008), and Daescu and Todling (2009).

3. An FSO-based optimization framework

A practical difficulty in the implementation of iterative procedures for tuning covariance parameterizations such as (10) is given by the nonlinear dependence of the analysis state defined by (1) and (2) on the input parameters. Issues related with the nonlinear dependence of the forecasts on the tuning parameters are compounded when first-order sensitivity methods are used to assess the information weighting in the DAS. The optimization framework developed in this study relies on a linear innovation-weight parameterization of the analysis state that was first introduced in our work Daescu and Langland (2016). This approach is further developed here and embedded into an iterative process to identify and correct deficiencies in the weights associated to various observation input components based on guidance provided by a short-range forecast error measure. The parametric representation of the analysis is expressed as
xa(s)=xb+K[sδy],
where the parameter vector is sRm with entries denoted sj, j = 1: m. The parameter dimension, 1 ≤ mp, is specified by experiment criteria to partition the data yp into components yjRpj,j=1:m with p1+p2+pm=p. Practical examples include data partitions by observation type, observed variable, and/or location in the time-space domain. In (16), ∘ denotes a generalized elementwise (Hadamard) product between the vector sm and the partitioned innovation vector δyp, defined as
sδy=[s1δy1s2δy2smδym]p.
Particular cases to (16) are m = 1 when the innovation vector is scaled through a single scalar parameter y and m = p when a scalar parameter is associated to each observation in the DAS. In the latter case, (17) becomes the standard elementwise vector product, siyi, i = 1: p. It is noticed that for any specification of the weight parameter s, the analysis (16) is obtained by minimizing a quadratic cost functional (single outer loop iteration) for the analysis increment δx = xxb,
J(δx)=12δxTB1δx+12[Hbδxsδy]TR1[Hbδxsδy].

The reference analysis (1) corresponds to all weights set to one in (16)/(18), xa = xa(1m) where 1mm denotes the vector with all entries equal to one; henceforth, this DAS configuration is referred to as the control DAS.

Given an analysis (16) produced with weight specification s, possibly different from the control DAS, we seek a correction δs such that the updated weights snew will reduce the error measure defined by (3) and (4),
snew=s+δs,e[xa(snew)]<e[xa(s)].
The difference
δenew=e[xa(snew)]e[xa(s)]<0
represents the improved observation performance on reducing the forecast errors induced by the parameter correction δs. When the update (19) is applied at s = 1m, the goal is to improve the analysis forecast produced by the control DAS,
snew=1m+δs,e[xa(snew)]<e(xa).
For any specification s1m, the goal is to improve a parameter configuration that has already been updated in a previous analysis, within an iterative parameter tuning process performed over several data assimilation and forecast cycles. In any situation, the corrected analysis xa(snew) is used to produce the background to the next analysis cycle, e.g., for a 6-h assimilation window
xb|t0+6h=Mt0,t0+6h[xa|t0(snew)].
In the context of gradient-based optimization, the corrected weights snew may be expressed as
δs=αp,snew=s+δs=s+αp,
and key ingredients for an effective update (23) are the computation of a descent direction pm,
pTse[xa(s)]<0
and specification of an appropriate step length α > 0. Before addressing (23) and (24), we extend the FSO (9) and FSOI (15) to the weighted DAS representation (16).

a. FSO and FSOI in the weighted DAS representation

Notice that the observation partitioning criteria implicitly defines a partition of the DAS operator K as
K=[K1|K2||Km],Kjn×pj,j=1:m,
such that (16) is mathematically equivalent with
xa(s)=xb+KSδy=xb+j=1msjKjδyj.
In (26), Sp×p is a diagonal matrix with the vector s1p on its diagonal, where 1pp denotes the vector with all entries set to one. For comparison with (9), in a weighted system (16)/(26) the FSO is expressed as
ye[xa(s)]=STKTga,s=s[KTga,s],
where
ga,s=defx0e[xa(s)]
is the forecast error gradient to initial conditions at xa(s). The evaluation of (27) is performed with the same computational tools of the control DAS to evaluate the observation space vector
ga,so=defKTga,sp
and followed by the weight multiplication,
ye[xa(s)]=sga,sop.
The FSO (9) in the control DAS is a particular case of (30), s = 1m. By analogy with (14) and (15), the FSOI calculation associated with the weighted representation (16) is
δe˜j(s)=sjδyjT[g¯s,bo]j,j=1:m,
where g¯s,bo is defined as in (14) with ga replaced by ga,s. Therefore, as compared with the control DAS, the software development efforts and the added computational cost of implementing and evaluating the weighted formulations (16)/(26), (30), and (31) are negligible.

b. FSI

Identification of a descent direction (24) is achieved by evaluating the gradient se[xa(s)]m, which is the forecast error sensitivity to innovation weights (FSI). From (26), the Jacobian matrix of xa(s) to parameters is
xa(s)s=[K1δy1K2δy2Kmδym]n×m
and chain rule differentiation is used to express the FSI:
se[xa(s)]=[xa(s)s]Tga,s=(32)[δy1TK1Tδy2TK2TδymTKmT]ga,s=(29)[δy1T(ga,so)1δy2T(ga,so)2δymT(ga,so)m]m.
It is noticed that the FSI (33) is expressed in terms of the unweighted innovations. From (30) and (33), we establish the relationship between the FSO and FSI in the weighted analysis (16):
sje[xa(s)]sj=δyjTyje[xa(s)],j=1:m.

Each gradient component in (33) represents the forecast sensitivity to the corresponding innovation-weight parameter: a positive value e[xa(s)]/sj>0 indicates that a reduced weight sjnew<sj to δyj is of potential benefit to the forecasts, whereas a negative value ∂e[xa(s)]/∂sj < 0 indicates that an increased weight sjnew>sj to δyj is of potential benefit to the forecasts.

The FSO-developed software tools allow the simultaneous evaluation of the sensitivities FSR (11) and FSI (33): the FSR relies on FSO and oma residual yh(xa), whereas the FSI relies on FSO and omf residual yh(xb). The correlation between the FSR and the FSI guidance is analyzed in our work Daescu and Langland (2016). A practical advantage of the FSI parameterization is that the analysis (16) is linearly related with the weight parameters. This property facilitates the search for improved parameter estimates through quadratic approximations to the forecast error measure, as explained below.

1) The FSI guidance for the analysis increment

The particular case m = 1 corresponds to a single scalar parameter s and (16) is represented as
xa(s)=xb+sKδy.
The parameterization (35) was introduced in Daescu and Todling (2009) for analyzing the approximation properties of the FSOI (15). Here, we seek to provide a diagnosis and improve the scaling factor to the analysis increment,
δxa=Kδy,δxa(s)=sδxa,
based on FSI guidance from the parametric forecast error functional
e^(s)=defe[xa(s)]=xaf(s)xυE2,
where xaf(s) is the forecast produced by the analysis xa(s),
xaf(s)=(35)Mt0,tf(xb+sδxa).
The parameter values s = 0 and 1 correspond to the background state xa(0)=xb and to the analysis state xa(1)=xa, respectively, and the observation impact (12) is represented as δe=e^(1)e^(0). From (35) and (36) it is noticed that xa(s)/s=Kδy=δxa such that the derivative e^(s) is evaluated as
e^(s)=[xa(s)s]Tga,s=δxaTga,s.

The first-order optimality condition is e^(s)=0. When evaluated in the control DAS at s = 1, a negative sensitivity e^(1)<0 indicates that s > 1 is a descent direction, i.e., δxa is “too short,” whereas a positive sensitivity e^(1)>0 indicates that s < 1 is a descent direction, i.e., δxa is “too long.” Strategies for estimating the optimal scaling to the analysis increment are presented below. In the following section, we explain that this process is also highly relevant to the problem of finding an appropriate step length α along a generic direction p in the parameter update (23).

Approximation to the optimal scaling s
For a short-range forecast error measure, the experience gained from FSOI studies is that the forecast model is nearly linear in the time interval from t0 to tf (Gelaro et al. 2007). Under this assumption, an estimate s^ to the optimal scaling s for the analysis increment may be obtained from a quadratic approximation to the nonlinear functional (37),
e^(s)q(s)=As2+Bs+C,ss^=B2A.
As illustrated in Fig. 1a, approximation by quadratic interpolation may be achieved by fitting the function values e^(0)=e(xb),e^(1)=e(xa) and either the derivative e^(1)=δxaTga to obtain
qa(0)=e(xb),qa(1)=e(xa),qa(1)=e^(1)
or the derivative e^(0)=δxaTgb to obtain
qb(0)=e(xb),qb(1)=e(xa),qb(0)=e^(0).
Fig. 1.
Fig. 1.

(a) Approximation to the nonlinear forecast error functional e[xa(s)] by the quadratic functions qa(s) and qb(s) and (b) the secant approximation to the parametric forecast state xaf(s) and the approximation to the optimal scaling by projection on the direction δxaf.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

A derivative free estimate may be obtained by approximating the forecast state xf(s) with the secant line passing through xbf at s = 0 and xaf at s = 1,
xaf(s)x^af(s)=defxbf+sδxaf,
where
δxaf=xafxbf
denotes the forecast increment. This process is illustrated in Fig. 1b. By replacing (43) in (37), the nonlinear function e^(s) is approximated by the quadratic
qab(s)=xbf+sδxafxυE2.
Minimization to (45) is achieved by projecting xυxbf on the forecast increment δxaf, as illustrated in Fig. 1b, and corresponds to the parameter value
s^=(xbfxυ,δxaf)EδxafE2.
Noticing the identity1
2(xbfxυ,δxaf)E=δxafE2+xbfxυE2xafxυE2,
the estimate (46) is expressed in terms of the E norms as
s^=12δxafE2+xbfxυE2xafxυE2δxafE2=(12)12(1δeδxafE2).
From (47) we derive a first-order consistency diagnostic to the optimal scaling of the analysis increment δxa in the control DAS expressed in terms of the observation impact δe and the forecast increment δxaf,
s^=1ifandonlyifδe=δxafE2.

For completeness, Table 1 provides the coefficients (40) for each of the quadratics qa, qb, and qab defined in (41), (42), and (45), respectively, and the estimated scaling factor s^ to the analysis increment that minimizes the parametric error functional (37). For a linear forecast model, these expressions are mathematically equivalent and provide the optimal value, s^=s. In general, distinct estimates are produced by each quadratic and their accuracy is determined by nonlinearities in the model forecast.

Table 1

Quadratic approximations to the parametric forecast error measure and the associated estimates to the optimal scaling of the analysis increment.

Table 1

2) Parameter updates along a descent direction p

The FSI (33) identifies the steepest descent direction p = −∇se[xa(s)] associated with the weight specification s in the analysis (16). Aside from the steepest descent iteration, various strategies may be considered in the practical implementation of the parameter tuning procedure. For example, a block coordinate descent approach (Beck and Tetruashvili 2013; Wright 2015) may be used to update selected parameter components sj, e.g., the weights assigned to innovation components of increased sensitivity,
sjnew=sjαe[xa(s)]sj.
Sign gradient descent algorithms are receiving an increased interest, particularly in machine learning applications (Moulay et al. 2019). The sign gradient descent iteration is expressed as
snew=sαsgn{se[xa(s)]},
where the sgn operator is defined as
sgn(x)={1ifx<00ifx=01ifx>0
and is applied elementwise to the gradient vector. Iteration (50) is particularly suitable when parameter updates are performed based on tendencies in the FSI guidance analyzed over several assimilation cycles and in practical situations when only an approximate value of the gradient (33) is available, e.g., due to errors induced by simplified versions of the adjoint model and the adjoint DAS operator.
Step length specification
Proper selection of the step length α > 0 is essential in determining the performance of the corrected weights (23): a cautiously small step may not produce a substantial forecast improvement, whereas a large step may prove to be detrimental. In general, a correction applied to the weight parameters along a direction p requires the computational overhead of performing one additional analysis. For a trial step length specification α0 > 0, we express the trial analysis as
xa(s+α0p)=xb+K[(s+α0p)δy].
An additional model forecast xaf(s+α0p) must be produced to verify the analysis (52). If the outcomes of the trial experiment are not satisfactory, a correction may be performed based on a computationally efficient procedure for estimating the optimal step length α, suitable for practical implementation. The key to the process is the linear relationship
K[(s+αp)δy]=K(sδy)+αK(pδy).
Using (53), the trial analysis (52) is expressed in terms of xa(s) in (16) as
xa(s+α0p)=xa(s)+α0K(pδy)
and the analysis increment associated with (54) is
δxa(s,p;α0)=xa(s+α0p)xa(s)=α0K(pδy).
The analysis produced by taking any other step α along the direction p is expressed using (53) as
xa(s+αp)=xa(s)+αK(pδy).
By replacing (55) in (56), we obtain
xa(s+αp)=xa(s)+αα0δxa(s,p;α0).
The significance of (35) and (36) to (57) is evident: we view xa(s) and xa(s + α0p) in (57) as the background state xb and the analysis state xa = xa(1) in (35), respectively, and the step ratio α/α0 as the scaling factor to the analysis increment. In this context, a quadratic approximation such as (42) or (45) may be used to provide an improved step length estimate. For example, the qab approximation (47) to the optimal step α in the direction p is expressed as
α^=α02{1e[xa(s+α0p)]e[xa(s)]δxaf(s,p;α0)E2},
where δxaf(s,p;α0)=xaf(s+α0p)xaf(s) is the increment in the forecasts produced by the trial analysis (52) and the reference analysis xa(s) in (16). The updated weights (23) are defined as
snew=s+α^p,
and the corrected analysis is evaluated as
xa(snew)=xa(s)+α^α0δxa(s,p;α0).

It is noticed that the computational overhead required to implement the corrected analysis (60), as compared with the trial analysis (52), is roughly given by the model forecast xaf(s+α0p), and therefore, (58)(60) provide an efficient mechanism to improve the outcomes of the trial experiment. For linear dynamics, the estimates derived in this section are optimal, α^=α. The sequence of computational steps required to perform the correction (23) to the assigned parameter weights, including the enhancement (60), is outlined in Fig. 2.

Fig. 2.
Fig. 2.

The computational steps required to perform the correction to the assigned innovation-weight parameters within a data assimilation cycle.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The theoretical concepts developed in this section provide the main ingredients to the formulation of a gradient-based parameter tuning procedure. We emphasize that the practical implementation must be tailored to the characteristics of the data assimilation and forecast system and the specific goals of the application at hand.

4. Experiments with NAVDAS-AR/NAVGEM

The practical applicability of the theoretical framework developed in this work is illustrated in numerical experiments performed with the 4D-Var system NAVDAS-AR and the NAVGEM forecasting model at a resolution of T425/L60 (Hogan et al. 2014). The vertical coordinate is a hybrid sigma-pressure coordinate system with 60 vertical levels (L60) and a model top of 0.04 hPa. The adjoint-DAS tools developed at NRL facilitated all FSI computations and have been extended in our work to incorporate the innovation-weight parameterization. The specific goals of the FSI analysis and parameter tuning procedure are to identify and address systematic deficiencies in the way the information provided by various observing system components is processed in NAVDAS-AR and achieve a weighting configuration capable of sustaining an improved performance over an extended time period, suitable for use in an operational environment. Not all of the theoretical features of the adaptive framework outlined in Fig. 2 have been exercised and in particular, we have not attempted to optimize parameter values at each assimilation cycle.

The experiments are valid for the time period 1 April–31 May 2018 and include all 6-h analyses and the associated 24-h forecasts produced by NAVDAS-AR/NAVGEM. The verification fields have been specified from the THORPEX Interactive Grand Global Ensemble (TIGGE; Bougeault et al. 2010) datasets produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) model, retrieved at 0.5° resolution with 18 vertical pressure levels and interpolated to the NAVGEM Gaussian grid at T119 spatial resolution and 60 vertical levels in sigma-pressure coordinates.

The experimental setup has two distinctive stages: parameter tuning and validation. Tuning of the innovation-weight parameters in NAVDAS-AR is performed for the time period of 1–14 April 2018 and validation of the tuned system is performed for 15 April–31 May 2018.

The parameter tuning stage is directly aimed at improving the DAS performance on reducing the initial-condition errors and implemented with a verification state specified at the initial time t0. In this particular context, the FSI represents the sensitivity of the analysis error with respect to the innovation weights. Throughout the tuning stage, a systematic assessment of the weighting procedure has been performed over time intervals of increased length, ranging from 24 to 120 h. In this process, adjustments to the weight parameterization and the assigned parameter values have been implemented, with the FSI (33) reevaluated in the most recent weight configuration. The performance of the weighted NAVDAS-AR/NAVGEM system is assessed during the validation stage in terms of the reduction in the analysis errors and the 24-h model forecast errors, as compared with the control DAS. The specification of the verification fields at the initial-time t0 is a distinctive feature from traditional FSO/FSOI studies that rely on self-analysis fields and relates directly to the way information from observations is processed in the DAS. This experimental setup alleviates ambiguities in the parameter tuning associated with the model forecast errors (biases), forecast lead time, and self-analyses (Gelaro et al. 2007; Privé et al. 2021). The FSOI analysis is performed for both initial condition errors and the 24-h forecast errors to provide further insight on the role of observations at various stages of the assimilation and forecast process. In all computations, the error measure is taken as a dry total energy norm (Rosmond 1997) truncated to NAVGEM vertical levels 7–60 (atmospheric region below ∼0.50 hPa). The interpretation of the numerical results must take into account the limitations associated with the energy norm selection that emphasizes the tropospheric region of the atmosphere (Todling 2013; Diniz and Todling 2020). Notwithstanding these limitations, the energy norm has been the FSOI metric used most commonly in the NWP community, and it is also the metric with which we have most-extensive baseline of results for comparison.

a. Training stage: FSI guidance and parameter tuning

NAVDAS-AR assimilates observations of atmospheric temperature (T), wind longitudinal (u) and latitudinal (υ) components, wind speed (ws), relative humidity (rh), surface pressure (p), radiance, and bending angle. A summary of the observation types and the associated observed variables is given in Table 2. Over the time period of the study, only clear-sky radiances have been assimilated and with the Cross-track Infrared Sounder (CrIS) data at nominal spectral resolution.

Table 2

Observation types and observed parameters in NAVDAS-AR. The third column provides the default values assigned to the innovation weights in EXP, and the fourth column provides additional parameter corrections performed in the tuning process.

Table 2

Insight on the potential deficiencies in the weighting of innovations was obtained from a comprehensive analysis of the FSI guidance in the control DAS (CTL) and only a few illustrative examples are discussed below. The average FSI values in CTL valid at 0000 UTC are shown for various observation types in Fig. 3a. Positive FSI values indicate that in general, innovations are overweighted and significantly increased sensitivity values are associated with SATWND data,2 as compared with other observation types. The vertical profile of the FSI for SATWND is shown in Fig. 4 and provides a consistent guidance of overweighted innovations throughout the vertical dimension and with larger sensitivities associated to upper-level wind data. Information on the fraction of observations from total SATWND number of observations (nobs) is also included in Fig. 4, e.g., observations assimilated above the 500-hPa level represent in average ∼64% of SATWND observations assimilated at 0000 UTC. In Fig. 3a, radiosondes (RAOBS) display the second largest FSI guidance as an overweighted data type. However, the vertical profile of the FSI analysis (not shown) indicated that relatively large values are associated mainly with the data collected in the upper levels of the atmosphere, particularly RAOBS winds above 100 hPa, where a reduction in the assigned innovation weight parameters was deemed to improve the analysis errors. By contrast, the FSI profile indicated that in general, RAOBS temperature innovations should receive an increased weight in the atmospheric region below 525 hPa. The analysis of radiance data revealed systematic tendencies in the FSI guidance associated with various instruments, closely determined by instrument channels and the sign (positive or negative) of the innovations, an indication of biases in innovations and potential issues in the variational bias correction.

Fig. 3.
Fig. 3.

The FSI values for various observation types in the DAS: (a) control and (b) weighted experiment. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Fig. 4.
Fig. 4.

Vertical profile of FSI values for SATWND data. Displayed are average values in CTL and EXP valid at 0000 UTC with observations binned in 50-hPa pressure intervals.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

These examples illustrate the use of the FSI information as a diagnostic tool as well as challenges in the process of identifying a parameterization effective in improving the DAS performance. The practical approach taken was to assign a weight coefficient to each data type and analyze tendencies in the FSI guidance over the training period. Valuable insight on the initial weight specification was obtained from the quadratic approximation (45) and the estimates (47) evaluated in various norm components. This capability is further illustrated and discussed in the validation experiments (see Figs. 6 and 7). A first trial experiment performed with a parameter value of s = 0.5 proved that the quadratic (45) provided an accurate a priori representation of the experiment’s outcomes for all components of the energy norm. Results from the FSI analysis in the trial experiment are shown in Fig. 5. Specifically, the scaling factor of s = 0.5 applied to the analysis increment is significantly reducing the magnitude of the sensitivities to SATWND (the dominating FSI component) and GPS-RO observations; however, as indicated by the negative FSI values, this DAS configuration underweights the information from most observation types and in particular, RAOBS and MDCRS observations and radiances assimilated from the AMSU-A, ATMS, AIRS, and IASI instruments. These results indicate that a systematic assessment of the FSI guidance is needed to improve the performance of various observation types by tuning a multidimensional weight parameter.

Fig. 5.
Fig. 5.

The FSI values for various observation types in the trial experiment with a scaling factor of s = 0.5 applied to the analysis increment. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The experience gained from the training stage was that an effective tuning of the weights associated with conventional data, satellite-derived winds, and GPS-RO observations can be achieved using only a few parameters. We adopted a scaled version of the sign gradient descent update (50) based on the FSI guidance collected over several assimilation cycles of the training period and successive parameter corrections have been applied at time intervals distributed as follows: 2, 3, 4, 5, 7, 10, 14 April. An initial step length of α = 0.5 was taken for SATWND and GPS-RO observations and a more cautious initial step length of α = 0.25 was specified for conventional observations. The default values assigned to the innovation weights for various observation types are provided in Table 2 together with a summary of the additional corrections performed throughout the training stage based on guidance from the recalculated vertical distribution of the FSI. In particular, weight corrections have been applied to SATWND data where the assigned weight values ranged from 0.35 to 0.45 in the region below 200 hPa to 0.5–0.7 in the region above 200 hPa. RAOBS temperature observations received an increased weight of s = 1.5 in the region below 525 hPa, whereas all RAOBS observations in the region above 100 hPa received a weight of s = 0.5. To facilitate the comparison with CTL, the FSI values reevaluated in the final weighted configuration of the experiment (EXP) are shown for each observation type in Fig. 3b and included in Fig. 4 for the vertical profile of the SATWND data.

Assigning appropriate weight coefficients to radiance data proved to be a difficult task and in particular, the interaction between the flow dependent variational bias correction (VarBC) applied to radiances in NAVDAS-AR and the predefined innovation weighting, determined from a prior time period, requires further investigation. Several configurations have been tested and the final version adopted was a parameterization with instrument channels weighted by the tendencies in the sign associated with omf (positive or negative) and five geographical regions: polar, middle latitudes, and tropics. It is noticed that a weight adjustment s = 1 + δs to the innovation component δy may be interpreted as a correction δsδy and by analogy, an FSO-based approach may be used to adjust the weight assigned to the bias estimates produced by VarBC. Since VarBC computes bias corrections in an adaptive fashion, as part of the analysis scheme (Dee 2005; Eyre 2016), further research is needed to investigate the potential use of a flow dependent FSO-based procedure to improve the VarBC applied to radiances in a formulation suitable for operational implementation.

b. Validation results

The ability of the weighted configuration EXP to improve the assimilation performance is investigated in validation experiments over the time period 15 April–31 May 2018 with data collected at all 6-h analysis cycles. Performance is assessed in terms of the error reduction in analyses (e0), the 6-h background forecasts (e6), and the associated 24-h (e24) and 30-h (e30) forecasts valid at t0 + 24 h. Results are analyzed in the dry total energy norm and individually for the vorticity, divergence, and temperature components. Bias reduction is investigated for analyses and 24-h forecasts and the FSOI assessment is calculated at t0 and t0 + 24 h for various observation types. Time series analysis of errors is used to investigate the ability of the EXP configuration to sustain an improved performance. Comparative results of the background fit to radiosonde observations are provided for CTL and EXP. The validation experiments have been initiated at 0600 UTC 15 April 2018 assimilation cycle when the 6-h forecast produced by the CTL analysis valid at 0000 UTC 15 April 2018 was used to provide the background state to both CTL and EXP.

Diagnosis of the optimal scaling to the analysis increment is an essential stage of the tuning process. Insight on the suboptimal observation performance with the default-weighted innovations (CTL) and the performance gain achieved in EXP through the innovation-weighting procedure may be obtained from the parametric representation to the analysis Eq. (35). A representative example is provided in Fig. 6 where the initial condition errors associated with the CTL and EXP in the first analysis cycle of the validation period, at 0600 UTC 15 April 2018, are shown as a function of the scaling factor applied to the analysis increment based on the quadratic approximation measure (45). The results in Fig. 6a indicate that in a single analysis cycle, a substantial reduction in the total energy error may be achieved by adjusting the δxa scaling factor to a value of s ∼ 0.5.

Fig. 6.
Fig. 6.

The parametric representation of the initial condition error in CTL and EXP as a function of the scaling factor to the analysis increment. Results valid at 0600 UTC 15 Apr 2018 are displayed for errors measured in (a) the total energy norm and (b) the temperature component of the total energy norm.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

As outlined in Table 2, EXP assigns a default weight of s = 0.45 to the majority of innovations associated with the wind observations and we emphasize that the kinetic energy (divergence and vorticity components) represents a large fraction of the total energy norm (in our experiments ∼69%). The additional gains in the observation performance achieved from the tuning applied in EXP, as compared with the δxa scaling of s = 0.5, are best represented by the error reduction in the temperature component of the total energy norm. To illustrate this aspect, the parametric representation of the initial condition error associated with the temperature component is shown in Fig. 6b.

Estimates (47) to the optimal scaling factor s^ have been calculated in CTL and EXP throughout the validation period for various components of the total energy norm. The 28-cycle moving average of the estimated optimal scaling factor s^ evaluated in CTL and reevaluated in EXP is provided in Fig. 7 for the total energy norm and its vorticity and temperature components. It is noticed that the estimated optimal weights are specific to each error measure and this important practical aspect must be taken into account in the design of a parameter tuning procedure for improving the observation performance. The diagnosis for optimal scaling corresponds to an estimate (47) of s^=1 and the results indicate systematic overweighting of innovations in CTL and the EXP’s ability of sustaining an improved weighting with respect to various norm components.

Fig. 7.
Fig. 7.

Estimated optimal scaling to the analysis increment for errors measured in the total energy norm and its vorticity and temperature components. Displayed are 28-cycle simple moving average values calculated over the validation period in CTL (solid line) and in EXP (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The information derived from the FSI analysis complements other diagnostics for assessing the tuning in the DAS. In particular, as shown by Eyre (2016), an approximation to the trace of the observation influence matrix scaled by the number of observations p may be obtained as
Tr(HK)/p1[E(Jao)/E(Jbo)]1/2,
where E denotes the statistical expectation operator and Jao and Jbo denote the observation component of the cost functional in the DAS evaluated at xa and xb, respectively. A time series of the values 1(Jao/Jbo)1/2 evaluated at the 0000 UTC analysis cycles over the validation period in CTL and in the innovation-weighted configuration of the EXP is shown in Fig. 8. It is noticed that both CTL and EXP configurations produced an average value of ∼0.2, which is consistent with the estimates derived in the Met Office operational global 4D-Var DAS (Eyre 2016) and in the NASA Global Modeling and Assimilation Office hybrid En-Var DAS (Todling and El Akkraoui 2018).
Fig. 8.
Fig. 8.

Time series of the estimated value Tr(HK)/p in CTL and EXP at various 0000 UTC analysis cycles of the validation period. The legend provides the corresponding mean value and standard deviation.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

1) Error reduction in the e norm

Table 3 provides a summary of the average percentage reduction in the errors e0, e6, e24, and e30 produced by the EXP, as compared with CTL based on data collected from all analysis/forecast cycles over the validation period, measured in the total energy norm and the vorticity, divergence, and temperature components. The results indicate a substantially increased performance of the EXP in improving the initial condition errors (e0), ranging from an average reduction of 12.6% in the temperature component to an average reduction of 21.7% in the vorticity component. The EXP also provided noticeable improvements in all components of the e24 and e30 errors and in particular, the average reduction achieved in the total energy norm error was of 3.1% and 1.8%, respectively. A detailed analysis of the error reduction throughout the validation period is provided below.

Table 3

Average percentage reduction in the state errors produced by EXP as compared with CTL, measured in various norm components.

Table 3

(i) Performance on the analysis and background errors

The time series of the total energy errors e0 and e6 associated with CTL and EXP during the first week (28 analysis cycles) of the validation period are shown in Fig. 9a and the 28-cycle moving average of the errors throughout the validation period is shown in Fig. 9b. Remarkable in Fig. 9a is the fast rate of decay of the errors in EXP at the beginning of the validation period when the experiment corrects the initial state received from CTL, an indication that the benefit from the weighting procedure is compounded over several assimilation cycles. Also noticeable in Fig. 9a is the underperformance of observations in the CTL configuration in reducing the initial condition error at the 0600 and 1800 UTC analysis cycles. The 28-cycle moving average values of the e0 and e6 errors in the vorticity, divergence, and temperature components are shown in Fig. 10. Throughout the validation period, the EXP provided a significant reduction in both e0 and e6 errors. In particular, as shown in Fig. 10, the errors in all components of the 6-h forecasts produced by EXP are significantly lower than the e0 errors in CTL, indicating the potential for a 6-h gain in the forecast lead time. A comparison of the vertical distribution of the globally averaged e0 error components for vorticity, divergence, and temperature in CTL and EXP is shown in Fig. 11 together with the associated average percentage error reduction. The EXP produced a substantial reduction in the analysis errors at most vertical levels and in particular, a consistently large improvement was achieved between NAVGEM vertical levels 30–45 (middle-tropospheric region) where the error reduction in each norm component is in the range of 10%–20% or higher.

Fig. 9.
Fig. 9.

Time series of the e0 and e6 errors associated with CTL and EXP, measured in a dry total energy norm (J kg−1). Displayed are (a) errors for the first 28 assimilation cycles and (b) the 28-cycle simple moving average values over the validation time period.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Fig. 10.
Fig. 10.

Time series of the e0 and e6 errors associated with CTL and EXP for the (a) vorticity, (b) divergence, and (c) temperature components of the total energy norm (J kg−1). Displayed are the 28-cycle simple moving average values over the validation time period.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Fig. 11.
Fig. 11.

Vertical distribution of the globally averaged e0 errors in CTL and EXP for the vorticity, divergence, and temperature components of the total energy norm (J kg−1). The panels on the right display the corresponding average percentage reduction of the errors produced by EXP, as compared with CTL.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

(ii) Performance on the model forecast errors

In average, EXP produced improved e24 and e30 errors in all norm components, as compared with CTL, and with increased percentage reductions in the 24-h forecast errors. The 28-cycle moving average values of the errors in analysis forecasts (e24) and background forecasts (e30) produced by CTL and EXP are shown in Fig. 12. A large improvement was achieved in the divergence component of the forecast errors where the average EXP e30 errors are close to the average CTL e24 errors. However, the 6-h gain in the forecast time lead achieved by EXP in the e6 errors has not been materialized in the e30 errors of the vorticity and temperature components. In particular, as noted in Table 3 and shown in Figs. 10 and 12, whereas the initial condition errors in the vorticity component exhibit a large percentage reduction in EXP, consistent throughout the validation period, the associated percentage reduction in the 24- and 30-h forecasts is significantly lower. Possible factors include the accumulated errors in the model forecast and the fast growth of initial-condition errors that is not properly reflected in the e0 and e6 measures. The vertical profile of the globally averaged e24 errors in CTL and EXP is shown in Fig. 13 together with the corresponding average percentage error reduction achieved in EXP. In general, EXP provided error improvements in all norm components and at the majority of model levels. However, by comparison with the vertical distribution of the e0 errors in Fig. 11, some components of the e24 errors are increased in EXP at various model levels as noticed in particular, for the vorticity component near the surface.

Fig. 12.
Fig. 12.

As in Fig. 10, for the e24 and e30 errors.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

Fig. 13.
Fig. 13.

As in Fig. 11, for the e24 errors.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

2) Bias analysis

Biases in the analyses and forecasts produced by CTL and EXP have been investigated to assess whether the error reduction in EXP produced an increase in bias. The time series of the biases, evaluated against ECMWF verification fields at each vertical level over the global horizontal domain, has been analyzed in CTL and EXP and in general, biases have been reduced in the EXP analyses at the majority of vertical levels. The root-mean-square (rms) values associated with the time series of biases in the analyses and in the 24-h forecasts have been evaluated for the T, u, υ variables and the vertical distribution of the difference rms(bias_EXP) − rms(bias_CTL), is displayed in Fig. 14. Negative values associated with the analyses of T, u, υ variables at the majority of vertical levels show that in general, rms(bias_EXP) values are lower than rms(bias_CTL) values and provide an indication of the ability of the weighting procedure to improve the bias in analyses. For the 24-h forecasts, temperature biases have been in general improved at most vertical levels and only small differences are associated with the biases in the wind components.

Fig. 14.
Fig. 14.

Vertical distribution of the difference between the rms values of the time series of biases in EXP and the rms values of the time series of biases in CTL. Displayed are differences in rms values associated to biases in analyses (solid line) and to biases in the 24-h forecasts (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

3) Background fit to radiosonde observations

A valuable indication of the improved weighting may be obtained from the statistical analysis of the omf residuals (innovations) in CTL and EXP. In Fig. 15 we provide the vertical profile of the rms values of the omf for radiosonde (RAOBS) temperature and zonal wind observations in CTL and EXP, evaluated at 0000 UTC analyses over the validation period. For temperature observations, the rms values are shown in Fig. 15a and the normalized values rms(EXP)/rms(CTL) are shown in Fig. 15b. For zonal wind observations, the rms values are shown in Fig. 15c and the normalized values rms(EXP)/rms(CTL) are shown in Fig. 15d. The normalized rms values in Fig. 15b indicate that in general, the EXP provided an improved background fit to RAOBS temperature data, with largest relative improvement in the atmospheric region between 500 and 800 hPa. The normalized rms values in Fig. 15d indicate that in general, the EXP also provided an improved background fit to RAOBS data for the zonal wind component, although an increase in the EXP rms values is noticed in the region below 950 hPa. Similar results (not shown) have been obtained for the background fit to RAOBS data for the meridional wind component.

Fig. 15.
Fig. 15.

Vertical profile of rms values of the innovations (omf) associated with radiosonde observations in CTL and EXP. Displayed are values valid at 0000 UTC for (a) temperature and (c) zonal wind observations. (b),(d) The corresponding normalized values rms(EXP)/rms(CTL) are shown.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

4) Observation impact assessment

As noticed in Fig. 9, the innovation-weighting procedure applied in EXP has significantly reduced both the initial condition errors and the 6-h forecast errors, as compared with CTL. Larger observation impacts δe0 = e0–e6 are achieved in EXP with analysis increments δxa of smaller magnitude, as compared with CTL. This aspect is shown in Fig. 16 where the 28-cycle simple moving average values of the magnitude of δxa calculated in CTL and EXP are displayed throughout the validation period. Therefore, the analysis and background forecasts are closer in EXP, as compared with CTL. In general, an improved data assimilation and forecast system will be characterized by smaller analysis and forecast errors and reduced observation impacts. As compared with EXP, in the CTL configuration the reduced observation impacts δe0 are associated with larger analysis increments and provide further indication of suboptimal observation performance due to overweighted innovations. The interpretation of the results must account for the fact that our experiments rely on ECMWF verification fields that are independent from the analyses and forecasts produced by NAVDAS-AR/NAVGEM.

Fig. 16.
Fig. 16.

The magnitude of the analysis increment measured in the total energy norm and its vorticity and temperature components. Displayed are the 28-cycle simple moving average values over the validation time period calculated in CTL (solid line) and in EXP (dashed line).

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

FSOI has been performed in CTL and EXP to assess the impact of various observation types in reducing the initial-condition errors (δe0) and in reducing the 24-h forecast errors (δe24). The FSOI evaluation is implemented in CTL based on (15) and in EXP based on the weighted formulation (31). Comparative results of observation impact on δe0 in CTL and EXP are provided in Fig. 17. The dominant feature in Fig. 17 is the large detrimental impact on the initial condition errors associated with SATWND data in CTL, which is due to significantly overweighted innovations. By comparison, as a result of the improved specification of the innovation weights, in EXP the SATWND data are the second largest contributor to the error reduction, only surpassed by RAOBS. Also notable in Fig. 17 is the improved use of GPS-RO and LEOGEO observations in EXP, as compared with CTL.

Fig. 17.
Fig. 17.

Observation impact (FSOI) on reducing the initial condition errors: (a) CTL and (b) EXP.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

The FSOI assessment for δe24 in CTL and EXP is provided in Fig. 18. A major distinction between the CTL results for δe0 and δe24 is the beneficial impact of SATWND data on the 24-h forecast errors. By comparison with CTL, the FSOI analysis in EXP identifies SATWND as the largest contributor to the reduction of 24-h forecast errors and with a significantly larger impact value as compared with CTL. In general, other observation types exhibit only small variations in their 24-h forecast error impacts between CTL and EXP.

Fig. 18.
Fig. 18.

Observation impact (FSOI) on reducing the 24-h forecast errors: (a) CTL and (b) EXP.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

FSOI analysis for SATWND data

The weighting applied to SATWND data in EXP increased the average fraction of observations with a beneficial impact on the error reduction, from 48.9% in CTL to 52.3% in EXP for δe0 and from 50.1% in CTL to 51% in EXP for δe24. A comparison between the vertical profiles of the FSOI analysis for SATWND data in CTL and EXP is provided in Fig. 19. The results obtained in CTL are shown in Figs. 19a and 19c and indicate that, in general, the SATWND data produced an increase in the initial condition errors; however, the upper-level observations contributed to the 24-h forecast error reduction. The EXP results are shown in Figs. 19b and 19d and indicate that the weighting procedure significantly improved the use of SATWND data both for δe0 and δe24 errors, at a large majority of the vertical levels.

Fig. 19.
Fig. 19.

Vertical profile of SATWND data impact on initial condition errors (a) CTL and (b) EXP and on 24-h forecast errors (c) CTL and (d) EXP. Displayed are average values valid at 0000 UTC.

Citation: Monthly Weather Review 150, 6; 10.1175/MWR-D-21-0305.1

In the interpretation of the δe0 and δe24 results, it is noticed that the observation impact on forecasts errors is closely determined by the growth of the initial-condition errors, which is not represented in the δe0 measure, and therefore, observations with a small impact on δe0 may in fact provide a much larger impact on reducing the forecast errors. Our results corroborate the findings of Privé et al. (2021) where observation impacts evaluated for forecast errors in the range of 6–48 h have shown significant dependencies on the specified forecast lead time, including sign variations. Our results also indicate that attempts to address deficiencies in the DAS based on the FSOI assessment of forecast errors may lead to ambiguous results. A judicious assessment of the observation performance must include complementary diagnostics for quantifying the observation influence on the analysis such as observation-space diagnostics and estimates based on the degrees of freedom for signal (Cardinali et al. 2004; Lupu et al. 2011).

5. Summary and further research directions

This work presents a new methodology for embedding FSO information into a feedback mechanism aimed at improving the observation performance in data assimilation. The main ingredients are the innovation-weight parameterization of the analysis equation, the FSO-based evaluation of the gradient (FSI) to identify a descent direction in the parameter space, and a computationally efficient mechanism for step length selection guided by a quadratic approximation to the parametric forecast error measure. Unlike data denial approaches, our methodology retains all observations assimilated in the DAS and improvements are sought through diagnosis and adaptive adjustments to innovation weight parameters. This approach may be implemented and tested in a variety of data assimilation systems where FSO tools have been developed.

The specification of the validation state and the selection of the error metric are key ingredients for the procedure to be effective in improving the DAS performance. Feasible selections of the validation fields include high-resolution reanalyses valid at the initial time and aimed directly at improving the initial-condition to the model forecasts, observation space metrics or, used in a cautious fashion, self-analyses with a time-varying forecast lead time. The total energy norm adopted in this study facilitated the FSI calculations based on the software tools developed at NRL for routine FSO/FSOI monitoring. Covariance metrics provide an improved quantification of the errors and computationally feasible implementations can be obtained from ensemble forecast estimates.

Diagnosis of suboptimal observation performance and identification of high-impact parameters require a systematic analysis of tendencies in the FSI associated with various data types and geographical regions. High impact innovation-weight parameters identified through FSI are then iteratively optimized over a time period spanning various assimilation cycles. The dimension of the parameter vector of weights may be adaptively refined during the training stage; however, it is emphasized that short-term gains that may be achieved from tuning a high-dimensional parameterization do not entail an improved long-term performance. The practical goal is to identify and correct a relatively small number of key parameters that capture systematic deficiencies in the way information is processed in the DAS and have a potentially large impact on improving the observation performance. As shown in our experiments, this process may be initiated through diagnosis and tuning of the scaling applied to the analysis increment and followed by additional refinements in the specified parameterization. The practical ability to implement this methodology for NWP applications has been demonstrated in preliminary numerical experiments with the 4D-Var NAVDAS-AR system and the NAVGEM forecasting model. It is emphasized that the availability of the verification fields at the initial time, here produced by ECMWF, facilitated the parameter tuning procedure implemented in NAVDAS-AR. Further research, testing, and validation experiments are necessary to develop an approach suitable for operational implementation.

Theoretically, the parameterization (16)/(18) may be extended to a nonlinear analysis produced through multiple outer loop iterations and FSI calculations may be achieved based on the FSO evaluation presented by Trémolet (2008). As new software capabilities are developed, further applications are envisaged for adaptive tuning of innovation-weight parameters and in particular, a flow-dependent implementation for improved assimilation of radiance data. The extension of the PQC framework developed by Hotta et al. (2017a) to the FSI-based optimal weight formulation could provide a feasible approach to advance research in this area. Flow-dependent adaptive weighting of innovations based on a short-range forecast error guidance will also be investigated for goal-oriented applications such as targeting an extreme weather event.

1

This is the vector identity −2(u, v) = ǁvǁ2 + ǁuǁ2 − ǁu + vǁ2 applied to u=xbfxυ and v=xafxbf.

2

SATWND incorporates observations from NOAA GOES-15 and GOES-16, EUMETSAT Meteosat-8Meteosat-11, JMA Himawari-8, and INM INSAT satellites.

Acknowledgments.

We are grateful to Dr. Ricardo Todling at NASA Global Modeling and Assimilation Office for many discussions and insightful comments throughout the development of this work. We thank two anonymous reviewers for their constructive criticism that helped to improve the presentation. This research is supported by the Office of Naval Research (ONR) through the NRL Base Program PE 62435N. The first author was supported by the NRL BAA 75-18-01 under Award N00173-19-2-C003. This work is based on TIGGE data obtained from the ECMWF Data Server. The THORPEX Interactive Grand Global Ensemble (TIGGE) is an initiative of the World Weather Research Programme (WWRP).

Data availability statement.

The NAVDAS-AR analyses fields used in this research are archived at the Naval Research Laboratory, and the TIGGE validation fields have been obtained via data archive portals at ECMWF.

REFERENCES

  • Baker, N. L., and R. Daley, 2000: Observation and background adjoint sensitivity in the adaptive observation-targeting problem. Quart. J. Roy. Meteor. Soc., 126, 14311454, https://doi.org/10.1002/qj.49712656511.

    • Search Google Scholar
    • Export Citation
  • Beck, A., and L. Tetruashvili, 2013: On the convergence of block coordinate descent type methods. SIAM J. Optim., 23, 20372060, https://doi.org/10.1137/120887679.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble. Bull. Amer. Meteor. Soc., 91, 10591072, https://doi.org/10.1175/2010BAMS2853.1.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., P. Du, and J. Bedard, 2018: A new approach for estimating the observation impact in ensemble–variational data assimilation. Mon. Wea. Rev., 146, 447465, https://doi.org/10.1175/MWR-D-17-0252.1.

    • Search Google Scholar
    • Export Citation
  • Cardinali, C., 2009: Monitoring the observation impact on the short-range forecast. Quart. J. Roy. Meteor. Soc., 135, 239250, https://doi.org/10.1002/qj.366.

    • Search Google Scholar
    • Export Citation
  • Cardinali, C., 2018: Forecast sensitivity observation impact with an observation-only based objective function. Quart. J. Roy. Meteor. Soc., 144, 20892098, https://doi.org/10.1002/qj.3305.

    • Search Google Scholar
    • Export Citation
  • Cardinali, C., S. Pezzulli, and E. Andersson, 2004: Influence-matrix diagnostic of a data assimilation system. Quart. J. Roy. Meteor. Soc., 130, 27672786, https://doi.org/10.1256/QJ.03.205.

    • Search Google Scholar
    • Export Citation
  • Chen, T.-C., and E. Kalnay, 2019: Proactive quality control: Observing system simulation experiments with the Lorenz ’96 model. Mon. Wea. Rev., 147, 5367, https://doi.org/10.1175/MWR-D-18-0138.1.

    • Search Google Scholar
    • Export Citation
  • Chen, T.-C., and E. Kalnay, 2020: Proactive quality control: Observing system experiments using the NCEP Global Forecast System. Mon. Wea. Rev., 148, 39113931, https://doi.org/10.1175/MWR-D-20-0001.1.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., 2008: On the sensitivity equations of four-dimensional variational (4D-Var) data assimilation. Mon. Wea. Rev., 136, 30503065, https://doi.org/10.1175/2007MWR2382.1.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. Todling, 2009: Adjoint estimation of the variation in model functional output due to the assimilation of data. Mon. Wea. Rev., 137, 17051716, https://doi.org/10.1175/2008MWR2659.1.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. Todling, 2010: Adjoint sensitivity of the model forecast to data assimilation system error covariance parameters. Quart. J. Roy. Meteor. Soc., 136, 20002012, https://doi.org/10.1002/qj.693.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. H. Langland, 2013: Error covariance sensitivity and impact estimation with adjoint 4D-Var: Theoretical aspects and first applications to NAVDAS-AR. Quart. J. Roy. Meteor. Soc., 139, 226241, https://doi.org/10.1002/qj.1943.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. H. Langland, 2016: Innovation-weight parametrization in data assimilation: Formulation and analysis with NAVDAS-AR/NAVGEM. IFAC-PapersOnLine, 49, 176181, https://doi.org/10.1016/j.ifacol.2016.10.159.

    • Search Google Scholar
    • Export Citation
  • Dee, D. P., 2005: Bias and data assimilation. Quart. J. Roy. Meteor. Soc., 131, 33233343, https://doi.org/10.1256/qj.05.137.

  • Diniz, F. L. R., and R. Todling, 2020: Assessing the impact of observations in a multi-year reanalysis. Quart. J. Roy. Meteor. Soc., 146, 724747, https://doi.org/10.1002/qj.3705.

    • Search Google Scholar
    • Export Citation
  • Eyre, J. R., 2016: Observation bias correction schemes in data assimilation systems: A theoretical study of some of their properties. Quart. J. Roy. Meteor. Soc., 142, 22842291, https://doi.org/10.1002/qj.2819.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., Y. Zhu, and R. M. Errico, 2007: Examination of various-order adjoint-based approximations of observation impact. Meteor. Z., 16, 685–692, https://doi.org/10.1127/0941-2948/2007/0248.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., R. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX observation impact intercomparison experiment. Mon. Wea. Rev., 138, 40094025, https://doi.org/10.1175/2010MWR3393.1.

    • Search Google Scholar
    • Export Citation
  • Hogan, T. F., and Coauthors, 2014: The Navy Global Environmental Model. Oceanography, 27 (3), 116125, https://doi.org/10.5670/oceanog.2014.73.

    • Search Google Scholar
    • Export Citation
  • Hotta, D., T.-C. Chen, E. Kalnay, Y. Ota, and T. Miyoshi, 2017a: Proactive QC: A fully flow-dependent quality control scheme based on EFSO. Mon. Wea. Rev., 145, 33313354, https://doi.org/10.1175/MWR-D-16-0290.1.

    • Search Google Scholar
    • Export Citation
  • Hotta, D., E. Kalnay, Y. Ota, and T. Miyoshi, 2017b: EFSR: Ensemble forecast sensitivity to observation error covariance. Mon. Wea. Rev., 145, 50155031, https://doi.org/10.1175/MWR-D-17-0122.1.

    • Search Google Scholar
    • Export Citation
  • Ishibashi, T., 2018: Adjoint-based observation impact estimation with direct verification using forward calculation. Mon. Wea. Rev., 146, 28372858, https://doi.org/10.1175/MWR-D-18-0037.1.

    • Search Google Scholar
    • Export Citation
  • Kalnay, E., Y. Ota, T. Miyoshi, and J. Liu, 2012: A simpler formulation of forecast sensitivity to observations: Application to ensemble Kalman filters. Tellus, 64A, 18462, https://doi.org/10.3402/tellusa.v64i0.18462.

    • Search Google Scholar
    • Export Citation
  • Kim, S.-M., and H. M. Kim, 2018: Effect of observation error variance adjustment on numerical weather prediction using forecast sensitivity to error covariance parameters. Tellus, 70A, 116, https://doi.org/10.1080/16000870.2018.1492839.

    • Search Google Scholar
    • Export Citation
  • Kim, S.-M., and H. M. Kim, 2019: Forecast sensitivity observation impact in the 4DVAR and hybrid-4DVAR data assimilation systems. J. Atmos. Oceanic Technol., 36, 15631575, https://doi.org/10.1175/JTECH-D-18-0240.1.

    • Search Google Scholar
    • Export Citation
  • Kotsuki, S., K. Kurosawa, and T. Miyoshi, 2019: On the properties of ensemble forecast sensitivity to observations. Quart. J. Roy. Meteor. Soc., 145, 18971914, https://doi.org/10.1002/qj.3534.

    • Search Google Scholar
    • Export Citation
  • Langland, R., and N. Baker, 2004: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56, 189201, https://doi.org/10.3402/tellusa.v56i3.14413.

    • Search Google Scholar
    • Export Citation
  • Le Dimet, F.-X., H.-E. Ngodock, B. Luong, and J. Verron, 1997: Sensitivity analysis in variational data assimilation. J. Meteor. Soc. Japan, 75, 245255, https://doi.org/10.2151/jmsj1965.75.1B_245.

    • Search Google Scholar
    • Export Citation
  • Liu, J., and E. Kalnay, 2008: Estimating observation impact without adjoint model in an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 134, 13271335, https://doi.org/10.1002/qj.280.

    • Search Google Scholar
    • Export Citation
  • Lorenc, A., and R. Marriott, 2014: Forecast sensitivity to observations in the Met Office global numerical weather prediction system. Quart. J. Roy. Meteor. Soc., 140, 209224, https://doi.org/10.1002/qj.2122.

    • Search Google Scholar
    • Export Citation
  • Lupu, C., P. Gauthier, and S. Laroche, 2011: Evaluation of the impact of observations on analyses in 3D- and 4D-Var based on information content. Mon. Wea. Rev., 139, 726737, https://doi.org/10.1175/2010MWR3404.1.

    • Search Google Scholar
    • Export Citation
  • Lupu, C., C. Cardinali, and A. P. McNally, 2015: Adjoint-based forecast sensitivity applied to observation-error variance tuning. Quart. J. Roy. Meteor. Soc., 141, 31573165, https://doi.org/10.1002/qj.2599.

    • Search Google Scholar
    • Export Citation
  • Moulay, E., V. Léchappé, and F. Plestan, 2019: Properties of the sign gradient descent algorithms. Inf. Sci., 492, 2939, https://doi.org/10.1016/j.ins.2019.04.012.

    • Search Google Scholar
    • Export Citation
  • Ota, Y., J. C. Derber, E. Kalnay, and T. Miyoshi, 2013: Ensemble-based observation impact estimates using the NCEP GFS. Tellus, 65A, 20038, https://doi.org/10.3402/tellusa.v65i0.20038.

    • Search Google Scholar
    • Export Citation
  • Privé, N., R. M. Errico, R. Todling, and A. El Akkraoui, 2021: Evaluation of adjoint-based observation impacts as a function of forecast length using an observing system simulation experiment. Quart. J. Roy. Meteor. Soc., 147, 121138, https://doi.org/10.1002/qj.3909.

    • Search Google Scholar
    • Export Citation
  • Rosmond, T., 1997: A technical description of the NRL adjoint modeling system. NRL Rep. NRL/MR/7532/97/7230, 62 pp., https://doi.org/10.21236/ADA330960.

  • Rosmond, T., and L. Xu, 2006: Development of NAVDAS-AR: Non-linear formulation and outer loop tests. Tellus, 58A, 4558, https://doi.org/10.1111/j.1600-0870.2006.00148.x.

    • Search Google Scholar
    • Export Citation
  • Ruston, B., and S. Healy, 2021: Forecast impact of FORMOSAT-7/COSMIC-2 GNSS radio occultation measurements. Atmos. Sci. Lett., 22, e1019, https://doi.org/10.1002/asl.1019.

    • Search Google Scholar
    • Export Citation
  • Todling, R., 2013: Comparing two approaches for assessing observation impact. Mon. Wea. Rev., 141, 14841505, https://doi.org/10.1175/MWR-D-12-00100.1.

    • Search Google Scholar
    • Export Citation
  • Todling, R., and A. El Akkraoui, 2018: The GMAO hybrid ensemble-variational atmospheric data assimilation system: Version 2.0. NASA Tech. Rep. Series NASA/TM-2018-104606, Vol. 50, 167 pp.

    • Search Google Scholar
    • Export Citation
  • Trémolet, Y., 2008: Computation of observation sensitivity and observation impact in incremental variational data assimilation. Tellus, 60A, 964978, https://doi.org/10.1111/j.1600-0870.2008.00349.x.

    • Search Google Scholar
    • Export Citation
  • Wright, S. J., 2015: Coordinate descent algorithms. Math. Program., 151, 334, https://doi.org/10.1007/s10107-015-0892-3.

  • Xu, L., T. Rosmond, and R. Daley, 2005: Development of NAVDAS-AR: Formulation and initial tests of the linear problem. Tellus, 57A, 546559, https://doi.org/10.3402/tellusa.v57i4.14710.

    • Search Google Scholar
    • Export Citation
Save
  • Baker, N. L., and R. Daley, 2000: Observation and background adjoint sensitivity in the adaptive observation-targeting problem. Quart. J. Roy. Meteor. Soc., 126, 14311454, https://doi.org/10.1002/qj.49712656511.

    • Search Google Scholar
    • Export Citation
  • Beck, A., and L. Tetruashvili, 2013: On the convergence of block coordinate descent type methods. SIAM J. Optim., 23, 20372060, https://doi.org/10.1137/120887679.

    • Search Google Scholar
    • Export Citation
  • Bougeault, P., and Coauthors, 2010: The THORPEX Interactive Grand Global Ensemble. Bull. Amer. Meteor. Soc., 91, 10591072, https://doi.org/10.1175/2010BAMS2853.1.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., P. Du, and J. Bedard, 2018: A new approach for estimating the observation impact in ensemble–variational data assimilation. Mon. Wea. Rev., 146, 447465, https://doi.org/10.1175/MWR-D-17-0252.1.

    • Search Google Scholar
    • Export Citation
  • Cardinali, C., 2009: Monitoring the observation impact on the short-range forecast. Quart. J. Roy. Meteor. Soc., 135, 239250, https://doi.org/10.1002/qj.366.

    • Search Google Scholar
    • Export Citation
  • Cardinali, C., 2018: Forecast sensitivity observation impact with an observation-only based objective function. Quart. J. Roy. Meteor. Soc., 144, 20892098, https://doi.org/10.1002/qj.3305.

    • Search Google Scholar
    • Export Citation
  • Cardinali, C., S. Pezzulli, and E. Andersson, 2004: Influence-matrix diagnostic of a data assimilation system. Quart. J. Roy. Meteor. Soc., 130, 27672786, https://doi.org/10.1256/QJ.03.205.

    • Search Google Scholar
    • Export Citation
  • Chen, T.-C., and E. Kalnay, 2019: Proactive quality control: Observing system simulation experiments with the Lorenz ’96 model. Mon. Wea. Rev., 147, 5367, https://doi.org/10.1175/MWR-D-18-0138.1.

    • Search Google Scholar
    • Export Citation
  • Chen, T.-C., and E. Kalnay, 2020: Proactive quality control: Observing system experiments using the NCEP Global Forecast System. Mon. Wea. Rev., 148, 39113931, https://doi.org/10.1175/MWR-D-20-0001.1.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., 2008: On the sensitivity equations of four-dimensional variational (4D-Var) data assimilation. Mon. Wea. Rev., 136, 30503065, https://doi.org/10.1175/2007MWR2382.1.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. Todling, 2009: Adjoint estimation of the variation in model functional output due to the assimilation of data. Mon. Wea. Rev., 137, 17051716, https://doi.org/10.1175/2008MWR2659.1.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. Todling, 2010: Adjoint sensitivity of the model forecast to data assimilation system error covariance parameters. Quart. J. Roy. Meteor. Soc., 136, 20002012, https://doi.org/10.1002/qj.693.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. H. Langland, 2013: Error covariance sensitivity and impact estimation with adjoint 4D-Var: Theoretical aspects and first applications to NAVDAS-AR. Quart. J. Roy. Meteor. Soc., 139, 226241, https://doi.org/10.1002/qj.1943.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. H. Langland, 2016: Innovation-weight parametrization in data assimilation: Formulation and analysis with NAVDAS-AR/NAVGEM. IFAC-PapersOnLine, 49, 176181, https://doi.org/10.1016/j.ifacol.2016.10.159.

    • Search Google Scholar
    • Export Citation
  • Dee, D. P., 2005: Bias and data assimilation. Quart. J. Roy. Meteor. Soc., 131, 33233343, https://doi.org/10.1256/qj.05.137.

  • Diniz, F. L. R., and R. Todling, 2020: Assessing the impact of observations in a multi-year reanalysis. Quart. J. Roy. Meteor. Soc., 146, 724747, https://doi.org/10.1002/qj.3705.

    • Search Google Scholar
    • Export Citation
  • Eyre, J. R., 2016: Observation bias correction schemes in data assimilation systems: A theoretical study of some of their properties. Quart. J. Roy. Meteor. Soc., 142, 22842291, https://doi.org/10.1002/qj.2819.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., Y. Zhu, and R. M. Errico, 2007: Examination of various-order adjoint-based approximations of observation impact. Meteor. Z., 16, 685–692, https://doi.org/10.1127/0941-2948/2007/0248.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., R. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX observation impact intercomparison experiment. Mon. Wea. Rev., 138, 40094025, https://doi.org/10.1175/2010MWR3393.1.

    • Search Google Scholar
    • Export Citation
  • Hogan, T. F., and Coauthors, 2014: The Navy Global Environmental Model. Oceanography, 27 (3), 116125, https://doi.org/10.5670/oceanog.2014.73.

    • Search Google Scholar
    • Export Citation
  • Hotta, D., T.-C. Chen, E. Kalnay, Y. Ota, and T. Miyoshi, 2017a: Proactive QC: A fully flow-dependent quality control scheme based on EFSO. Mon. Wea. Rev., 145, 33313354, https://doi.org/10.1175/MWR-D-16-0290.1.

    • Search Google Scholar
    • Export Citation
  • Hotta, D., E. Kalnay, Y. Ota, and T. Miyoshi, 2017b: EFSR: Ensemble forecast sensitivity to observation error covariance. Mon. Wea. Rev., 145, 50155031, https://doi.org/10.1175/MWR-D-17-0122.1.

    • Search Google Scholar
    • Export Citation
  • Ishibashi, T., 2018: Adjoint-based observation impact estimation with direct verification using forward calculation. Mon. Wea. Rev., 146, 28372858, https://doi.org/10.1175/MWR-D-18-0037.1.

    • Search Google Scholar
    • Export Citation
  • Kalnay, E., Y. Ota, T. Miyoshi, and J. Liu, 2012: A simpler formulation of forecast sensitivity to observations: Application to ensemble Kalman filters. Tellus, 64A, 18462, https://doi.org/10.3402/tellusa.v64i0.18462.

    • Search Google Scholar
    • Export Citation
  • Kim, S.-M., and H. M. Kim, 2018: Effect of observation error variance adjustment on numerical weather prediction using forecast sensitivity to error covariance parameters. Tellus, 70A, 116, https://doi.org/10.1080/16000870.2018.1492839.

    • Search Google Scholar
    • Export Citation
  • Kim, S.-M., and H. M. Kim, 2019: Forecast sensitivity observation impact in the 4DVAR and hybrid-4DVAR data assimilation systems. J. Atmos. Oceanic Technol., 36, 15631575, https://doi.org/10.1175/JTECH-D-18-0240.1.

    • Search Google Scholar
    • Export Citation
  • Kotsuki, S., K. Kurosawa, and T. Miyoshi, 2019: On the properties of ensemble forecast sensitivity to observations. Quart. J. Roy. Meteor. Soc., 145, 18971914, https://doi.org/10.1002/qj.3534.

    • Search Google Scholar
    • Export Citation
  • Langland, R., and N. Baker, 2004: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56, 189201, https://doi.org/10.3402/tellusa.v56i3.14413.

    • Search Google Scholar
    • Export Citation
  • Le Dimet, F.-X., H.-E. Ngodock, B. Luong, and J. Verron, 1997: Sensitivity analysis in variational data assimilation. J. Meteor. Soc. Japan, 75, 245255, https://doi.org/10.2151/jmsj1965.75.1B_245.

    • Search Google Scholar
    • Export Citation
  • Liu, J., and E. Kalnay, 2008: Estimating observation impact without adjoint model in an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 134, 13271335, https://doi.org/10.1002/qj.280.

    • Search Google Scholar
    • Export Citation
  • Lorenc, A., and R. Marriott, 2014: Forecast sensitivity to observations in the Met Office global numerical weather prediction system. Quart. J. Roy. Meteor. Soc., 140, 209224, https://doi.org/10.1002/qj.2122.

    • Search Google Scholar
    • Export Citation
  • Lupu, C., P. Gauthier, and S. Laroche, 2011: Evaluation of the impact of observations on analyses in 3D- and 4D-Var based on information content. Mon. Wea. Rev., 139, 726737, https://doi.org/10.1175/2010MWR3404.1.

    • Search Google Scholar
    • Export Citation
  • Lupu, C., C. Cardinali, and A. P. McNally, 2015: Adjoint-based forecast sensitivity applied to observation-error variance tuning. Quart. J. Roy. Meteor. Soc., 141, 31573165, https://doi.org/10.1002/qj.2599.

    • Search Google Scholar
    • Export Citation
  • Moulay, E., V. Léchappé, and F. Plestan, 2019: Properties of the sign gradient descent algorithms. Inf. Sci., 492, 2939, https://doi.org/10.1016/j.ins.2019.04.012.

    • Search Google Scholar
    • Export Citation
  • Ota, Y., J. C. Derber, E. Kalnay, and T. Miyoshi, 2013: Ensemble-based observation impact estimates using the NCEP GFS. Tellus, 65A, 20038, https://doi.org/10.3402/tellusa.v65i0.20038.

    • Search Google Scholar
    • Export Citation
  • Privé, N., R. M. Errico, R. Todling, and A. El Akkraoui, 2021: Evaluation of adjoint-based observation impacts as a function of forecast length using an observing system simulation experiment. Quart. J. Roy. Meteor. Soc., 147, 121138, https://doi.org/10.1002/qj.3909.

    • Search Google Scholar
    • Export Citation
  • Rosmond, T., 1997: A technical description of the NRL adjoint modeling system. NRL Rep. NRL/MR/7532/97/7230, 62 pp., https://doi.org/10.21236/ADA330960.

  • Rosmond, T., and L. Xu, 2006: Development of NAVDAS-AR: Non-linear formulation and outer loop tests. Tellus, 58A, 4558, https://doi.org/10.1111/j.1600-0870.2006.00148.x.

    • Search Google Scholar
    • Export Citation
  • Ruston, B., and S. Healy, 2021: Forecast impact of FORMOSAT-7/COSMIC-2 GNSS radio occultation measurements. Atmos. Sci. Lett., 22, e1019, https://doi.org/10.1002/asl.1019.

    • Search Google Scholar
    • Export Citation
  • Todling, R., 2013: Comparing two approaches for assessing observation impact. Mon. Wea. Rev., 141, 14841505, https://doi.org/10.1175/MWR-D-12-00100.1.

    • Search Google Scholar
    • Export Citation
  • Todling, R., and A. El Akkraoui, 2018: The GMAO hybrid ensemble-variational atmospheric data assimilation system: Version 2.0. NASA Tech. Rep. Series NASA/TM-2018-104606, Vol. 50, 167 pp.

    • Search Google Scholar
    • Export Citation
  • Trémolet, Y., 2008: Computation of observation sensitivity and observation impact in incremental variational data assimilation. Tellus, 60A, 964978, https://doi.org/10.1111/j.1600-0870.2008.00349.x.

    • Search Google Scholar
    • Export Citation
  • Wright, S. J., 2015: Coordinate descent algorithms. Math. Program., 151, 334, https://doi.org/10.1007/s10107-015-0892-3.

  • Xu, L., T. Rosmond, and R. Daley, 2005: Development of NAVDAS-AR: Formulation and initial tests of the linear problem. Tellus, 57A, 546559, https://doi.org/10.3402/tellusa.v57i4.14710.

    • Search Google Scholar
    • Export Citation
  • Fig. 1.

    (a) Approximation to the nonlinear forecast error functional e[xa(s)] by the quadratic functions qa(s) and qb(s) and (b) the secant approximation to the parametric forecast state xaf(s) and the approximation to the optimal scaling by projection on the direction δxaf.

  • Fig. 2.

    The computational steps required to perform the correction to the assigned innovation-weight parameters within a data assimilation cycle.

  • Fig. 3.

    The FSI values for various observation types in the DAS: (a) control and (b) weighted experiment. Displayed are average values valid at 0000 UTC.

  • Fig. 4.

    Vertical profile of FSI values for SATWND data. Displayed are average values in CTL and EXP valid at 0000 UTC with observations binned in 50-hPa pressure intervals.

  • Fig. 5.

    The FSI values for various observation types in the trial experiment with a scaling factor of s = 0.5 applied to the analysis increment. Displayed are average values valid at 0000 UTC.

  • Fig. 6.

    The parametric representation of the initial condition error in CTL and EXP as a function of the scaling factor to the analysis increment. Results valid at 0600 UTC 15 Apr 2018 are displayed for errors measured in (a) the total energy norm and (b) the temperature component of the total energy norm.

  • Fig. 7.

    Estimated optimal scaling to the analysis increment for errors measured in the total energy norm and its vorticity and temperature components. Displayed are 28-cycle simple moving average values calculated over the validation period in CTL (solid line) and in EXP (dashed line).

  • Fig. 8.

    Time series of the estimated value Tr(HK)/p in CTL and EXP at various 0000 UTC analysis cycles of the validation period. The legend provides the corresponding mean value and standard deviation.

  • Fig. 9.

    Time series of the e0 and e6 errors associated with CTL and EXP, measured in a dry total energy norm (J kg−1). Displayed are (a) errors for the first 28 assimilation cycles and (b) the 28-cycle simple moving average values over the validation time period.

  • Fig. 10.

    Time series of the e0 and e6 errors associated with CTL and EXP for the (a) vorticity, (b) divergence, and (c) temperature components of the total energy norm (J kg−1). Displayed are the 28-cycle simple moving average values over the validation time period.

  • Fig. 11.

    Vertical distribution of the globally averaged e0 errors in CTL and EXP for the vorticity, divergence, and temperature components of the total energy norm (J kg−1). The panels on the right display the corresponding average percentage reduction of the errors produced by EXP, as compared with CTL.