## 1. Introduction

Data assimilation of observations in both time and space improves a numerical weather prediction (NWP) forecast, in an average sense. However, it is important to determine the value added to a forecast from a specific subset of observations. In this way, we can investigate by instrument type, observation type, and location, which observations are the most impactful on a forecast. Additionally, we can avoid using observations that have negative impacts on a forecast. Evaluating the usefulness of observations is particularly important for operational NWP centers that operate under limited budgets and need to weigh the costs and benefits of adding more observations to an already large observational dataset.

There are a few basic approaches to quantifying the impact that assimilated observations have on a forecast. The first is the straightforward data-denial method, where parallel sets of analysis and forecast experiments are conducted with a “control” experiment assimilating all observations and experiments withholding subsets of observations (e.g., Zapotocny et al. 2002, 2007; Benjamin et al. 2010; Kutty and Wang 2015, manuscript submitted to *Adv. Meteor.*). This approach is popular; however, it is computationally expensive because of the number of experiments required. A second approach is an adjoint-based approach, first explored by Langland and Baker (2004). Different from data denial, this method can provide observation impact estimates for all observations simultaneously, without the need for separate denial experiments. The adjoint method has been applied successfully as an important diagnostic tool (e.g., Cardinali 2009; Langland et al. 2009; Gelaro et al. 2010; Weissmann et al. 2012; Hamill et al. 2013). However, adjoints are generally difficult to create, and because of the tangent linear assumption their application is limited to shorter forecast lengths.

A third approach to evaluating observation impact on a forecast is the ensemble-based method. An approach analogous to the adjoint method of Langland and Baker (2004) was proposed by Liu and Kalnay (2008) with a minor correction in Li et al. (2010). Kunii et al. (2012) successfully applied their method by evaluating the impact of real observations in a forecast of Typhoon Sinlaku, using the Weather Research and Forecasting (WRF) Model together with the local ensemble transform Kalman filter (LETKF; Hunt et al. 2007). Kalnay et al. (2012) derived a simpler formulation that makes fewer approximations; it is more general and computationally efficient because it relies on readily available ensemble Kalman filter (EnKF) products and can be used with any deterministic EnKF method. Ota et al. (2013) successfully applied the Kalnay et al. (2012) formulation to the National Centers for Environmental Prediction (NCEP) Global Forecast System EnKF (GFS/EnKF; Whitaker et al. 2008), which is now part of the GFS hybrid data assimilation system (Wang et al. 2013).

The Kalnay et al. (2012) observation impact metric is appealing because ensemble perturbations take the place of the adjoint model in estimating sensitivities. However as with any ensemble method, it suffers from sampling error, which occurs when the number of ensemble members is small compared to the degrees of freedom in a model and observing system—predominantly the case in ensemble NWP. Sampling error results in spurious correlations and can lead to filter divergence in deterministic EnKF assimilation. Houtekamer and Mitchell (1998) showed that the effects of sampling error can be suppressed by excluding distant observations from influencing the analysis at a given grid point. They experimented filtering covariance estimates using a distance-dependent correlation function, referred to as *covariance localization* (Houtekamer and Mitchell 2001). Since then, much research has been done in developing localization methods to improve EnKF analyses using limited ensembles (e.g., Hamill et al. 2001; Houtekamer and Mitchell 2005; Anderson 2007; Kepert 2011; Anderson 2012; Anderson and Lei 2013; Holland and Wang 2013).

In many cases, localization may be nontrivial, such as the case for applying localization to observations with complex forward operators (e.g., radar, satellite observations), or when localizing the impact of observations on variables not directly linked to the observations (e.g., localizing the impact of a temperature observation on the analysis of wind). Several adaptive localization methods have been developed to account for these issues. Anderson (2007) employed a Monte Carlo statistical method known as the group filter (GF) for evaluating sampling error using groups of ensemble members and calculating a weighting coefficient that minimizes root-mean-square (RMS) differences between group regression coefficients. Studies such as Chen and Oliver (2010), Bishop and Hodyss (2009a,b), and Anderson (2012) compute localization based on correlations between an observation and a state variable. More recently, Anderson and Lei (2013) developed an empirical localization function (ELF), which computes localizations from the output of an observing system simulation experiment in sets of pairs of observation and state variables binned by distance. Lei and Anderson (2014) compared the ELF to the GF method of Anderson (2007). In an ideal simulation where the true covariances are known, the ELF and GF show similar results, especially for larger ensembles. The ELF shows benefits over the GF in cases where there are biases in the spurious covariances and in the ability to automatically inflate covariances. However, extension of the ELF to real cases poses challenges, particularly because the ELF relies on the knowledge of a true state (Anderson and Lei 2013).

The problem of sampling error caused by small ensembles is a more serious issue for the ensemble-based observation impact metric. Localization can be applied to alleviate sampling error; however, a time-forecast component is added to the localization problem, such that a straightforward application of fixed localization techniques would not guarantee accurate impact estimates. To partially address the issue, Kalnay et al. (2012) proposed two methods of moving localization: 1) using a model-forecast nonlinear incremental evolution of the localization function, and 2) advecting the localization center using the climatological group velocity of dominant wavenumbers. Ota et al. (2013) applied a similar advected localization method, using the average forecast horizontal wind at each model vertical level. Both studies showed improvement relative to fixed GC localization; however, possible limitations exist with each method. The nonlinear evolution of the localization is computationally prohibitive for real NWP systems because a forecast is required for every observation. The advection methods are simpler to implement; however, they assume that the optimal localization is tied to the mean flow of the model and does not change in magnitude, size, or shape.

This study explores a dynamic localization method for ensemble-based observation impact estimate. There are two main purposes of this study. The first is to simply learn what a “proper” localization function looks like for the impact estimate, and how it evolves with increasing forecast time. Kalnay et al. (2012) and Ota et al. (2013) only considered moving the localization function, but it is possible that the shape and magnitude also need to evolve with forecast component, something that an adaptive method will be able to automatically determine. The second purpose is to test the potential effectiveness of an adaptive method on the observation impact metric. Adaptive localizations generally have been shown to provide more accurate assimilations at increased computational cost relative to fixed localization; in this study it is explored whether the application of such adaptive methods initially developed for assimilation can be extended and applied successfully to the ensemble impact metric. Briefly, the dynamic localization function used here is obtained from confidence factors derived using groups of ensembles, first proposed for EnKF by Anderson (2007). The focus of this study is to extend the GF concept in the context of the observation impact estimate.

In section 2, the observation impact metric is described along with the dynamic localization method, referred throughout as regression confidence factor (RCF) localization. Section 3 describes the experiment setup. As an initial test of the method on observation impact estimates, an isentropic two-layer primitive equation model (Zou et al. 1993) under the perfect-model assumption is adopted. This model is coupled with the LETKF data assimilation system following Holland and Wang (2013). The RCF calculation settings are also explained in section 3. The resulting RCF localization functions are shown and applied in section 4 for both single-observation and full-domain observation assimilation experiments. The accuracy of the ensemble observation impact estimate using the dynamic localization is compared with that of using fixed Gaspari–Cohn [GC; see Eq. (4.10) in Gaspari and Cohn (1999)] localization for different observation locations (tropical vs midlatitude), forecast length, and differing state variables. The goal of comparison to static GC is to first determine if the adaptive method is better for the ensemble impact metric. More importantly, the GC function is used as a tool to provide a baseline for comparisons with RCF localization experiments, to help provide context for qualitative discussion. From this comparison, an important relationship between localization used during assimilation and for the impact estimate was discovered and is discussed in section 4. Another potential use for adaptive methods is in the ability to automatically tune GC localization. This concept is introduced and tested in section 4. A summary and discussion are given in section 5.

## 2. Methods

### a. The observation impact metric

*t|*0

*”*can be read as “valid at time

*t*, initialized from analysis at time 0”). A cost function

*J*is defined in Langland and Baker (2004) to be the actual forecast error reduction (i.e., the difference in squared error between two adjacent forecasts):

*t*= 0 and

*t*= −

*n*, respectively. Here

*t*; in the absence of the true state, a verifying analysis can be used. The differences in forecast errors

*t*= 0, so (1) represents the impact of assimilating observations on a forecast [see Fig. 1 in Langland and Baker (2004)]. When

*J*is negative (positive), the magnitude of error in

*positive*(

*negative*)

*impact*.

*k*is the ensemble size, and

*m*×

*k*analysis and forecast perturbation matrices, respectively (

*m*= number of model state variables,

*k*= number of ensemble members). Despite the use of the tangent linear model approximation to obtain (2), each column in

*M*( ), such that the

*i*th column is

*t*. Localization of (2) is applied to this

*p*×

*m*matrix (

*p =*number of observations). Denoting localization matrix

_{,}the observation impact estimate modulated by the localization function becomes

*m*×

*p*matrix, meaning that every grid point–observation pair can have a unique localization weight. Since localization in (3) is applied to

### b. The RCF method of computing localization for observation impact estimate

*g*groups of

*k*ensembles are available in an assimilation system. When computing the linear regression between the state variables and observations, there are

*g*samples of the regression coefficient

*β.*A weighting factor

*α*is defined to minimize the expected RMS differences between all possible combinations of sample

*β*pairs. So,

*α*is chosen to minimize

*α*

_{min}:

*Q*is the ratio of the sample standard deviation to the absolute value of the sample mean of the group

*β*s. The optimal weighting factor

*α*

_{min}is also known as the RCF. A unique

*α*

_{min}can be calculated for each observation–state pair. The set of RCFs for a given observation and all state variables is called a

*regression confidence envelope*and can be used directly as a localization function (Anderson 2007).

*β*represents the regression between observations

*β*is the covariance between

*l*and state variable

*j*,

*β*s according to (6), so there is an inherent limit to the forecast length at which it can be successfully applied.

## 3. Experiment design

### a. The assimilation and forecast system

To evaluate and explore methods to improve the ensemble-based observation impact, experiments with a simplified primitive equation model and simulated observations were done. The dry, global, two-layer primitive equation spectral model of Zou et al. (1993) was chosen, which has been used in several studies of perfect- and imperfect-model ensemble-based data assimilation experiments (e.g., Wang et al. 2007, 2009; Holland and Wang 2013). It is useful because of its low computational demands allowing for many experiments to be conducted. The model variables include two vertical layers of vorticity, divergence, and layer thickness coefficients. The layer thicknesses, Δπ_{1} and Δπ_{2}, are described in terms of π (i.e., the Exner function). The model includes simple parameterization schemes for radiative heating and surface drag, with zonal wavenumber-2 terrain. A fourth-order Runge–Kutta scheme is used for forward integration.

The model was run using the same parameters as in Holland and Wang (2013). To isolate the impact of sampling error, the experiments were conducted in a perfect-model context. A model run with daily output over thousands of days of integration at T31 resolution served as the truth. An initial ensemble was generated by a random draw of the truth states. The assimilation-forecast cycles were run at the T31 resolution for 1000 cycles at 1 day (*n* = 24 h) intervals because of the long error doubling time of 3.78 days (Hamill and Whitaker 2005). The first 100 cycles were discarded from the RCF and impact calculations to allow the system to stabilize. Observations in interface height (i.e., the height between layer 1 and layer 2) were generated from the truth by adding errors drawn from a distribution with zero mean and fixed standard deviation of 250 m, as in Wang et al. (2007). There are 362 equally spaced observations total. These observations were assimilated into the LETKF following the settings of Holland and Wang (2013) with multiplicative and additive inflation. The GC localization was applied during assimilation with a cutoff radius of 8000 km, which was optimally tuned but, more importantly, provided a stable forecast-assimilation system for the full 1000 cycles.

### b. Settings for impact estimate experiments using RCF

Figure 1 shows a flowchart of RCF computations. An initial set of ensembles is randomly split into *g* groups of *k* ensembles each prior to assimilation. Cycled LETKF analyses are done for each group of *k* ensembles separately. Following analysis, an ensemble forecast is run to some valid time *t*. For each group, a regression coefficient *β* is then calculated according to (6), from which RCF is calculated according to (5) and saved for each assimilation cycle. Each grid point–observation pair has a unique RCF. Since the RCF from each cycle is noisy because of a small number of groups used in the calculation, particularly for locations far away from an observation, the next step is to take the mean of the RCFs across all cycles to dampen out those effects. This results in a “lookup table” of RCF functions for every observation. There are up to 256 ensemble members total, which are grouped randomly prior to assimilation. Initial experiments tested the sensitivity of resulting RCF functions with varying number of groups (2, 4, 8, and 16) using 16-member groups, and varying the number of ensembles per group (8, 16, 32, and 64) using 4 groups. For the subsequent impact experiments, RCF functions were calculated using *g =* 4 groups of *k =* 16 ensemble members each (64 ensemble members total) and applied as localization functions for ensemble impact estimates using a 16-member ensemble.

Impact experiments were conducted varying the forecast valid time *t* from 0 to 4 days. At *t* = 0, no forecast is run so the method gives RCF functions for the impact of the observations on the *analysis*. RCF functions were calculated first for model interface height, *z*_{int}, and then in terms of the state variables: Δπ_{1}, Δπ_{2}, *u*_{1}, *u*_{2}, *υ*_{1}, and *υ*_{2}. RCF functions for the latter are considered cross-variable impacts, which will serve to examine the effects of using the new localization method to estimate the impact of observations on unobserved variables.

Once the RCF functions were calculated, they were applied directly to the impact metric in (3) using one of the 16 ensemble group assimilations and compared with ensemble estimates with no localization and using a static GC localization. The results were validated against the *actual impact*, which is the impact calculated directly from (1) using the truth. Additionally, sets of single-observation experiments were conducted to better understand the results qualitatively, to aid in viewing specifically the time- and cross-variable components of the localization and the impact estimate.

## 4. Results

### a. RCF localization

Prior to application of the dynamic RCF functions to the ensemble impact metric, we first examine the structure of the localization functions outputted from the procedure described in Fig. 1. At the analysis time (Fig. 2a), RCF appears to have a Gaussian-like shape to it, though there are some differences such as narrow peaks and heavier tails. The effect of averaging over 900 cycles has smoothed the functions, though some noisiness still remains particularly in the tails of the distributions. Increasing the number of ensemble members per group increases the width of the RCF function, mostly greater than 30° in longitude away from the observation. The increased RCF width is because larger ensembles are less prone to noise from spurious correlations until greater distances from the observation. This result is consistent with results of Anderson (2007, his Fig. 4), as well as studies examining changes in optimal GC length scale with changing ensemble size (e.g., Hamill et al. 2001).

RCF functions for a 2-day forecast are shown in Fig. 2b. The time-forecast dependency causes the main signal to dampen and shift downstream from the observation location. The diminished magnitude suggests less confidence in ensemble covariances at longer forecast lead times. With increasing ensemble member size per group, there is an increase in the strength of the RCF function across all longitudes. Each ensemble size is able to capture the same time-dependent shift away from the observation and generally the same shape. The ensemble size of 16 chosen for various impact experiments in the following sections has a maximum signal of about 0.55 for the 2-day forecast, in contrast to the maximum of 1.0 for the analysis RCF function in Fig. 2a.

Figure 3 examines the sensitivity of changing the number of groups used in the RCF computation. The differences in the number of groups results in small differences in the RCF functions for both the analysis and 2-day forecast. This insensitivity can be attributed to the process of taking a long-term average of RCF functions over all LETKF cycles. Interestingly, just two groups would be sufficient in capturing the general shape of the RCF function. The rest of the results consider RCF functions computed from four groups of 16 ensemble members per group.

Each observation within the domain has a unique RCF function, and because the method reveals dynamical features of the model not all observations have a Gaussian-like spatial correlation. The seven observations in Fig. 4 each show zonally stretched RCF distributions, consistent with the predominantly zonal flow of the model. Midlatitude observations (observations 1, 2, 3, 6, and 7) tend to have a more Gaussian-like appearance, though some have a triple-peaked structure (observations 1 and 6) associated with the strongest westerly flow. The distance between the peaks of about 40° in longitude is likely a representation of underlying Rossby waves. For example, for an observation placed at a trough these additional peaks represent the adjacent ridges associated with a trough. As observations get closer to the tropics (e.g., observations 4 and 5), RCF begins to take a different, sometimes complex shape, including stretching eastward along the equator upstream of the main flow.

RCF functions from Fig. 4 are valid for the analysis time of model interface height. As illustrated in Fig. 5, RCF functions also reveal the time-forecast and cross-variable dynamics of the model. With increasing time from analysis to 3-day forecast, the RCF function in interface height (Figs. 5a,d,g) shifts downstream of the observation, expands in area, and reduces in magnitude. The shift in maximum amplitude is approximately 10° per day between analysis and 2-day forecast, which is roughly 1000 km day^{−1}. This is consistent with results of Torn and Hakim (2008) who found a 1000-km distance in their composite sensitivity 24-h patterns over Washington State. In terms of layer two zonal wind *u*_{2} (Fig. 5b) and meridional wind *υ*_{2} (Fig. 5c), the RCF functions exhibit dual peak dynamical structures. These RCF functions together mimic the shape of geostrophic adjustment correlations (e.g., Schlatter 1975) for both analysis and 1-day forecast. As forecast time increases to 2 days and longer, the RCF functions for cross variables smooth out and lose definition in dynamical linkage, though they still show the time-forecast dependency. It is possible that the small 16-member ensemble is unable to resolve cross-variable correlations beyond 2 days because of the weaker correlations, or other dynamical processes beyond advection occur past the 2-day lead time forecast.

### b. Single-observation impact experiment

A single-observation experiment was first conducted to examine the qualitative results of impact estimates with differing localizations, compared to the actual forecast error reduction. For the experiment, a 1-day forecast of one of the all-observation LETKF analyses was chosen as the background for a single-observation analysis. This single interface height observation has an observation innovation of +500 m. Deterministic forecasts initialized from the ensemble mean analysis and background were run to calculate actual impact according to (1). Ensemble forecasts of the single-observation analysis were run to calculate the ensemble-estimated impact according to (3), with differing choices of localization. Two localizations were tested: the same optimal GC localization (8000 km) as was used during assimilation, and the dynamic RCF functions shown in Fig. 5.

Results for one single-observation experiment are shown in Fig. 6 for analysis, 2-day, and 4-day impact of model interface height. Note again that negative values (filled blue) imply positive impact. The actual impact at analysis (Fig. 6a) follows flow-dependent structures of the background. Initially, three main centers are present at magnitudes above 500 m^{2}. The ensemble estimates (Figs. 6b,c) are qualitatively similar, as both localizations capture these three main centers well.

The actual impact on 2- and 4-day forecasts (Figs. 6e,i) shows many more impact centers due to forecast error growth. These impact areas propagate along the main westerly waves predominantly located within the tightly packed interface height contours. Actual impact centers span a much greater zonal distance and can be seen at distances exceeding 8000 km from the observation. The ensemble impact estimate using static GC localization (Figs. 6f,j) cannot capture these far away impact centers because of the limited length scale. Moreover, the magnitude of the estimated impacts of areas closest to the observation is much stronger than those shown in the actual impact, indicating the GC localization weight is too large there. Conversely, the largest area of actual positive impact in the analysis (60°N, 75°W) has now advected 30° and 75° eastward near the edge of the GC localization function in the 2- and 4-day forecasts, respectively, where the localization weight is nearly zero. As a result, the GC localized impact value is underestimated compared to actual impact. These effects combined results in a global RMSE that is about the same or greater than the RMS of the actual impact fields. The ensemble impact using dynamic RCF localization (Figs. 6g,k) shows improved estimates, and thus, is able to match the magnitudes of each center more closely with the actual error reduction. The localization functions spans a much greater distance than the GC function, with centers shifted downstream of the observation. Overall, the RMSE is much lower at less than half of global RMS of the actual impact.

This one case demonstrates both the qualitative nature of actual impact varying with forecast length, and how the RCF can outperform static GC localization and lead to improved estimates from the evolving RCF functions. In the next section, verification of the all-observation experiment is discussed to see if the RCF localization shows added overall skill in the case of homogeneous observation coverage.

### c. All-observation impact experiments

*N*is the number of cycles in time considered (900 for this study). This SS is equivalent to that used in Kalnay et al. (2012) where the reference RMSE is equivalent to the time-mean RMS values of the actual forecast error reduction.

Figure 7 shows the time-mean SS averaged over the globe. With increasing forecast time, for observation impact estimation of *z*_{int}, differences in SS emerge between GC and RCF localizations, with RCF localization showing increasingly higher skill. While the GC experiment approaches the no-skill line (0.0) by day 4, the RCF experiment has a skill around 0.4, still higher than the skill of the GC experiment at day 2. At the analysis time, the SS of the fixed GC localization experiment is nearly the same as the RCF localization in impact estimates of model interface height. In terms of indirectly observed layer-2 meridional wind (Fig. 7b), at analysis time the RCF experiment shows slightly lower skill than the GC experiment, but with increasing forecast time the RCF experiment becomes increasingly skillful relative to the GC experiment. The slight degradation in skill at the analysis time can be attributed to the inconsistency between the localization used in data assimilation and the localization used for the observation impact estimate. This issue is discussed further in section 4d.

In addition to SS using time-mean RMSE, Fig. 8 considers SS where global-mean RMSE is calculated in (6), and *N* is now the number of grid points (4608). The result is a unique global SS for each cycle in the experiment. The bar chart in Fig. 8a shows the percentage of the 900 cycles that show positive global skill for interface height impact. At the analysis time and 1-day forecast, both RCF and GC localization have a high percentage of cycles showing positive global SS. But as in Fig. 7, at 2-day forecast and beyond the differences become larger. The RCF function still shows positive skill more than three-quarters of the time at the 4-day forecast length, which is still better than even the GC function at 2-day forecast impacts. Of course, positive skill may not necessarily mean much when the skill is close to zero, so Fig. 8b shows instead percentages of cycles with SS exceeding 0.5. With the increased threshold, at 2 days and beyond the RCF still outperforms the GC by more than 20%. By the 4-day forecast, with a stricter threshold the GC function is not any more favorable than a simple top-hat localization with the same length scale, whereas the RCF shows high skill for nearly a quarter of the cycles considered.

The previous results for GC localization operate under the assumption that the same localization applied during assimilation is applied toward the impact estimate. What if, instead, we were allowed to optimally tune the localization length of the GC function for each forecast length? The GC tuning of impact estimates is shown in Fig. 9 for the analysis, 1-day, and 4-day forecast lengths. At the analysis, it is verified that the same tuning (8000 km) used at assimilation also provides optimal ensemble impact estimates. With forecasts, potential improvement by tuning the GC width is relatively minor, especially compared to the improvement that the RCF localizations show (dots in Fig. 9). A similar set of experiments varying the GC radius was conducted in Sommer and Weissmann (2014), and they too concluded that simple adjustment of the GC radius did not lead to a large improvement in accuracy of the ensemble impact method.

Finally, we examine the effect of RCF localization varying by latitude, since RCF functions at the midlatitudes and equator are very different in shape (Fig. 4). Here the pattern or map correlation is considered, using the Pearson correlation calculated between the estimated ensemble and the actual impact at the same location over all 900 cycles. The result is a map of correlation values, and in Fig. 10 the zonal average of this correlation map is shown. For 1-day forecast impact, both fixed GC and RCF show correlations well above the no-localization case, as expected due to sampling error in the raw ensemble correlations. RCF shows slight improvement relative to GC at midlatitudes (30°–70°) and the same or slight degradation elsewhere. At the 3-day impact (Fig. 10b), the RCF shows improvement over the fixed GC localization for most latitudes with the largest improvement at the midlatitudes. On the other hand, GC shows low skill comparable to no localization, an indication that while localization is needed, the GC localization is too restricted and centered in the wrong spot. A distinct shape is present at both 1- and 3-day lead times, showing high values at midlatitudes with a large dip within 30° of the equator. The improvement of GC and RCF relative to no localization is much smaller in the tropics than the midlatitudes. In the next section we further investigate this issue.

### d. Relationship between localizations for data assimilation and for observation impact estimate

The improvement in skill in the 1–4-day forecast range is due to the time-shift dependency. Two issues with the RCF localization applied to the impact estimate were noted. The first is why RCF did not show improved performance at the tropics, as seen in Fig. 10. The second, as seen in Fig. 7, is that the cross-variable dynamical linkage revealed within RCF did not offer any improvement on skill at analysis time. Both issues are related to the fact that localization used during assimilation was not consistent with the localization that the RCF suggested for the observation impact estimate.

First, it is important to note the uniqueness of the equatorially stretched RCF functions in the tropics (Fig. 11). The two-layer model used here, while able to well represent realistic midlatitude baroclinic instability, is not a realistic representation for the tropics due in part to the simplifying exclusion of moisture. Hendon and Hartmann (1985) analyzed the variability of a similar dry two-layer model and noted that the tropics are dominated by internal normal modes consistent with Matsuno (1966). Additionally, these waves are equatorially trapped, with minimal activity propagating from the tropics to the midlatitudes. The RCFs of an equatorial observation are also equatorially trapped and show east- and westward-propagating components (Fig. 11c). So tropical RCF functions represent the internal normal mode dynamics of the model used in this study.

*δ*

**y**. Assuming that localization

^{a}

^{T}

^{−1}= (

*k*− 1)

^{−1}

^{a}

^{T}

^{−1}into

*δ*

**y**(where

**ρ**_{I}so that

The implicit relationship between

### e. All-observation experiment using RCF localization during assimilation

The inherent relationship between _{}. This experiment will be referred to as “RCF-assim.” Impact estimates are calculated with the RCF functions applied as localization and verified against the actual forecast error reduction, as in previous experiments. Pattern correlation and SS is compared to the results of the previous experiments using GC localization during assimilation.

The SS in Fig. 7 shows that RCF-assim yields improved skill compared to the previous application of RCF to the GC assimilation, particularly for shorter (0–2 day) forecasts. This improvement converges to the level of skill from the previous RCF experiment at longer lead times for interface height. In terms of cross-variable *υ*_{2} impact, RCF-assim shows higher skill than the previous RCF experiment for 0–2-day forecasts, but actually becomes less skillful for 3–4 days. It is possibly an indication that RCF should also be recalculated from the new RCF assimilation. Comparing pattern correlations of RCF-assim to the previous RCF experiment, there is clear improvement at all latitudes, including significant improvement in the tropics at both 1- and 3-day forecasts (Fig. 10). The results of RCF-assim confirm that the best use of adaptive localization methods for the ensemble impact metric occurs when the same adaptive method is used for localization during the assimilation.

### f. Using RCF to design automatically tuned elliptical GC localization functions

Another potential use of adaptive methods such as the RCF is its derived shifting, magnitude change, and area coverage of the localization that could be used in the future to automatically tune a GC function for observation impact localization. For example, Fig. 13 shows four potential tuning parameters that may be calculated from RCF and how they evolve with increasing forecast time. Given predominantly zonal flow, each parameter was tuned for observations binned by latitude. One can see, for example, the differences in how far to advect a GC localization away from an observation due to differences in the zonal wind strength at each latitude (Fig. 13a) and the similarities at each latitude in reduction of maximum magnitude with increasing forecast time (Fig. 13b). It can also be seen that localization area should expand with increasing forecast time, but at different amounts depending on latitude of the observation (Fig. 13c). Complimentary to that, the localization function should have more of a two dimensional elliptical shape, since the zonal extent is greater than the meridional extent (Fig. 13d).

One question is how to apply these parameters with the preexisting definition of the GC function from Gaspari and Cohn (1999) to create an elliptically shaped localization. In the homogeneous or “circular” case, the GC function has the same cutoff distance in all directions; however, in the elliptical case, the cutoff distance changes based on the angle relative to the major and minor axes of the ellipse.

The steps to defining an elliptical GC localization for the impact metric are as follows:

Calculate parameters from the RCF functions as in Fig. 13. Parameters needed are the zonal span, meridional span, maximum magnitude, and offset distance from observation.

Use offset distance to define the center of the ellipse for a given observation and forecast time.

At each grid point, calculate the angle

*θ*relative to the center of the ellipse using*θ*= arctan(Δ*y*/Δ*x*), where Δ*x*and Δ*y*are the zonal and meridional component distances, respectively, between the grid point and the center of the ellipse.- The parametric form of an ellipse is given by
*x*=*a*cos*t*;*y*=*b*sin*t*, where*a*and*b*are major and minor axis distances, respectively, and*t*is the parametric angle. In this case,*a*and*b*are simply half of the zonal and meridional spans. Convert from polar angle*θ*to parametric angle*t*using - The cutoff distance for the GC functions as a function of angle is given by the distance formula:
Once the cutoff radius

*c*is determined, the GC function can be applied directly. To adjust for reducing magnitude with forecast time, multiply the output weights from the GC function by the maximum magnitude parameter. Figure 14 displays the resulting elliptical GC localization functions using the parameters from Fig. 13.

This new elliptical GC localization function was tested with the single-observation impact experiment. As shown in the rightmost column of Fig. 6, the estimates are similar in performance to the RCF in terms of overall structure and RMSE. More precise definitions and applications of these derived parameters may yield better results, but are beyond the scope of this paper. Here it is presented as a proof-of-concept to demonstrate how such parameters can be calculated from RCF and applied to tune the GC function.

## 5. Conclusions and discussion

The real-world application of the ensemble-based observational impact estimate needs to consider proper implementation of localization to reduce errors due to limited ensemble members. Such localization needs to consider time-forecast dependency in addition to spatial and cross-variable dependencies. Kalnay et al. (2012) and Ota et al. (2013) have shown that initial attempts to “advect” the localization function downstream from an observation leads to more reliable and accurate impact estimates. Here we examined an alternative method, an adaptive localization that varies by location and state variable according to the underlying model dynamics. The method is based on confidence factors of grouped ensemble regression coefficients, first proposed for EnKF assimilation (Anderson 2007). In this study, the method was extended to work for the ensemble-based observation impact estimate. An envelope of *regression confidence factors* (RCFs) for each observation and all grid points was used directly as localization. The purpose of the study was to explore the potential effectiveness of this adaptive method on the observation impact estimate and to learn more about how to properly localize the ensemble-based impact metric.

Results of the technique within a simple two-layer isentropic model showed the ability of the RCF method to reveal underlying dynamics between variables and the time-forecast component. Applying RCF functions for impact estimates showed overall improvement when verified against the actual forecast error reduction, especially at longer forecast lead times compared to using fixed GC localization. Single-observation experiments displayed the evolving structures of the actual forecast error reduction with increasing forecast time and showed that the dynamic localization was able to simulate the evolved error reduction much more closely than the static localization especially at longer forecast lead times. For an LETKF using all 362 observations, skill scores of the observation impact estimate applying RCF localization significantly improved upon those using fixed GC for increasing forecast lead times. For example, the skill of the estimates of the impact using RCF on a 4-day forecast beats the skill of the estimate using static GC at the 2-day forecast, essentially doubling the forecast length of accurate impact estimates. This improvement was attributed mainly to the time dependency of RCF shifting downstream, diminishing in magnitude, and expanding in areal coverage.

Further diagnostics also found that optimal use of localization from dynamical methods relies on the assumption that consistent localization was used during the assimilation as well. In other words, there is a fundamental relationship between the localization applied at assimilation time and the localization used for the impact estimate. Therefore, further improvement in skill of using the RCF in observation impact estimate is possible if a consistent adaptive localization was used at the assimilation time in place of the static GC function. This was confirmed in an additional experiment using RCF localization both during assimilation and for the impact estimates. Skill scores were higher and significant improvement was made in the estimates at the topics.

Another potential utility of adaptive methods such as the RCF is to automatically tune an elliptical GC function for observation impact localization. As a proof-of-concept, the application of RCF-derived parameters to tune the GC function shows promising results that were comparable to the RCF experiment for a single observation test.

For real-time application, a drawback of the RCF technique is computational cost of additional groups of ensemble members. To partially alleviate this issue, one could use increased ensembles only during a “training period” for a given weather regime to create RCF lookup tables offline, similar to the suggestion of Anderson (2007). Additionally, Fig. 3 suggests that just two groups can yield RCF functions consistent with RCF using more groups, due to the averaging process over many cycles to reduce noise. Figure 3 also suggests another idea, which is to split available ensembles into subgroups without increasing the total number of ensemble members and then artificially “inflate” the resulting RCF functions, because with increasing groups the RCF functions retain similar shape and shift but differ only in magnitude. Alternatively, dividing existing ensembles into subgroups the RCF method can still be utilized to derive parameters for automatically tuning localizations.

More challenges may be found for application to complex models, especially down to convective scales, which have many different kinds of day-to-day extreme weather patterns that we would not want the averaging process to smooth out. One idea to address this issue is an “online smoothing” process, or averaging over composites of weather patterns for a given day. Another idea is to use a different adaptive approach such as the ensemble correlations raised to a power (ECO-RAP) technique of Bishop and Hodyss (2009a,b), which can be applied to day-to-day ensemble correlations to automatically localize sampling error. The current study extends the RCF method for an observation impact study and explored its effectiveness compared to fixed GC. Comparing the RCF method with other adaptive methods at convective scales will be a topic in future work.

## Acknowledgments

The authors thank the Oklahoma Supercomputing Center for Education and Research for providing the computational resources that were used for simulations throughout this study. The research was primarily supported through the Department of Commerce and NOAA Grant NA110AR4320072 and NASA NIP Grant NNX10AQ78G. The authors would also like to thank Brian Holland for providing his EnKF system for our use and Govindan Kutty for providing the subroutine to calculate the observation impact. Comments from three anonymous reviewers and the editor were very helpful in improving the manuscript and are greatly appreciated.

## REFERENCES

Anderson, J. L., 2007: Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter.

,*Physica D***230**, 99–111, doi:10.1016/j.physd.2006.02.011.Anderson, J. L., 2012: Localization and sampling error correction in ensemble Kalman filter data assimilation.

,*Mon. Wea. Rev.***140**, 2359–2371, doi:10.1175/MWR-D-11-00013.1.Anderson, J. L., and L. Lei, 2013: Empirical localization of observation impact in ensemble Kalman filters.

,*Mon. Wea. Rev.***141**, 4140–4153, doi:10.1175/MWR-D-12-00330.1.Benjamin, S. G., B. D. Jamison, W. R. Moninger, S. R. Sahm, B. E. Schwartz, and T. W. Schlatter, 2010: Relative short-range forecast impact from aircraft, profiler, radiosonde, VAD, GPS-PW, METAR, and mesonet observations via the RUC hourly assimilation cycle.

,*Mon. Wea. Rev.***138**, 1319–1343, doi:10.1175/2009MWR3097.1.Bishop, C. H., and D. Hodyss, 2009a: Ensemble covariances adaptively localized with ECO-RAP. Part 1: Tests on simple error models.

,*Tellus***61A**, 84–96, doi:10.1111/j.1600-0870.2008.00371.x.Bishop, C. H., and D. Hodyss, 2009b: Ensemble covariances adaptively localized with ECO-RAP. Part 2: A strategy for the atmosphere.

,*Tellus***61A**, 97–111, doi:10.1111/j.1600-0870.2008.00372.x.Cardinali, C., 2009: Monitoring the observation impact on the short-range forecast.

,*Quart. J. Roy. Meteor. Soc.***135**, 239–250, doi:10.1002/qj.366.Chen, Y., and D. Oliver, 2010: Cross-covariances and localization for EnKF in multiphase flow data assimilation.

,*Comput. Geosci.***14**, 579–601, doi:10.1007/s10596-009-9174-6.Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions.

,*Quart. J. Roy. Meteor. Soc.***125**, 723–757, doi:10.1002/qj.49712555417.Gelaro, R., R. H. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX Observation Impact Intercomparison Experiment.

,*Mon. Wea. Rev.***138**, 4009–4025, doi:10.1175/2010MWR3393.1.Hamill, T. M., and J. S. Whitaker, 2005: Accounting for the error due to unresolved scales in ensemble data assimilation: A comparison of different approaches.

,*Mon. Wea. Rev.***133**, 3132–3147, doi:10.1175/MWR3020.1.Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter.

,*Mon. Wea. Rev.***129**, 2776–2790, doi:10.1175/1520-0493(2001)129<2776:DDFOBE>2.0.CO;2.Hamill, T. M., F. Yang, C. Cardinali, and S. J. Majumdar, 2013: Impact of targeted winter storm reconnaissance dropwindsonde data on midlatitude numerical weather predictions.

,*Mon. Wea. Rev.***141**, 2058–2065, doi:10.1175/MWR-D-12-00309.1.Hendon, H. H., and D. L. Hartmann, 1985: Variability in a nonlinear model of the atmosphere with zonally symmetric forcing.

,*J. Atmos. Sci.***42**, 2783–2797, doi:10.1175/1520-0469(1985)042<2783:VIANMO>2.0.CO;2.Holland, B., and X. Wang, 2013: Effects of sequential or simultaneous assimilation of observations and localization methods on the performance of the ensemble Kalman filter.

,*Quart. J. Roy. Meteor. Soc.***139**, 758–770, doi:10.1002/qj.2006.Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique.

,*Mon. Wea. Rev.***126**, 796–811, doi:10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation.

,*Mon. Wea. Rev.***129**, 123–137, doi:10.1175/1520-0493(2001)129<0123:ASEKFF>2.0.CO;2.Houtekamer, P. L., and H. L. Mitchell, 2005: Ensemble Kalman filtering.

,*Quart. J. Roy. Meteor. Soc.***131**, 3269–3289, doi:10.1256/qj.05.135.Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter.

,*Physica D***230**, 112–126, doi:10.1016/j.physd.2006.11.008.Kalnay, E., Y. Ota, T. Miyoshi, and J. Liu, 2012: A simpler formulation of forecast sensitivity to observations: Application to ensemble Kalman filters.

,*Tellus***64A,**18462, doi:10.3402/tellusa.v64i0.18462.Kepert, J. D., 2011: Balance-aware covariance localisation for atmospheric and oceanic ensemble Kalman filters.

,*Comput. Geosci.***15**, 239–250, doi:10.1007/s10596-010-9188-0.Kunii, M., T. Miyoshi, and E. Kalnay, 2012: Estimating the impact of real observations in regional numerical weather prediction using an ensemble Kalman filter.

,*Mon. Wea. Rev.***140**, 1975–1987, doi:10.1175/MWR-D-11-00205.1.Langland, R. H., and N. L. Baker, 2004: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system.

,*Tellus***56A**, 189–201, doi:10.1111/j.1600-0870.2004.00056.x.Langland, R. H., C. Velden, P. M. Pauley, and H. Berger, 2009: Impact of satellite-derived rapid-scan wind observations on numerical model forecasts of Hurricane Katrina.

,*Mon. Wea. Rev.***137**, 1615–1622, doi:10.1175/2008MWR2627.1.Lei, L., and J. L. Anderson, 2014: Comparisons of empirical localization techniques for serial ensemble Kalman filters in a simple atmospheric general circulation model.

,*Mon. Wea. Rev.***142**, 739–754, doi:10.1175/MWR-D-13-00152.1.Li, H., J. Liu, and E. Kalnay, 2010: Correction of ‘Estimating observation impact without adjoint model in an ensemble Kalman filter.’

,*Quart. J. Roy. Meteor. Soc.***136**, 1652–1654, doi:10.1002/qj.658.Liu, J., and E. Kalnay, 2008: Estimating observation impact without adjoint model in an ensemble Kalman filter.

,*Quart. J. Roy. Meteor. Soc.***134**, 1327–1335, doi:10.1002/qj.280.Matsuno, T., 1966: Quasigeostrophic motions in the equatorial area.

,*J. Meteor. Soc. Japan***44**, 25–43.Ota, Y., J. C. Derber, E. Kalnay, and T. Miyoshi, 2013: Ensemble-based observation impact estimates using the NCEP GFS.

,*Tellus***65A,**20038, doi:10.3402/tellusa.v65i0.20038.Schlatter, T. W., 1975: Some experiments with a multivariate statistical objective analysis scheme.

,*Mon. Wea. Rev.***103**, 246–257, doi:10.1175/1520-0493(1975)103<0246:SEWAMS>2.0.CO;2.Sommer, M., and M. Weissmann, 2014: Observation impact in a convective-scale localized ensemble transform Kalman filter.

*Quart. J. Roy. Meteor. Soc.,***140,**2672–2679, doi:10.1002/qj.2343.Torn, R. D., and G. J. Hakim, 2008: Ensemble-based sensitivity analysis.

,*Mon. Wea. Rev.***136**, 663–677, doi:10.1175/2007MWR2132.1.Wang, X., T. M. Hamill, J. S. Whitaker, and C. H. Bishop, 2007: A comparison of hybrid ensemble transform Kalman filter–optimum interpolation and ensemble square root filter analysis schemes.

,*Mon. Wea. Rev.***135**, 1055–1076, doi:10.1175/MWR3307.1.Wang, X., T. M. Hamill, J. S. Whitaker, and C. H. Bishop, 2009: A comparison of the hybrid and EnSRF analysis schemes in the presence of model errors due to unresolved scales.

,*Mon. Wea. Rev.***137**, 3219–3232, doi:10.1175/2009MWR2923.1.Wang, X., D. Parrish, D. Kleist, and J. Whitaker, 2013: GSI 3DVar-based ensemble–variational hybrid data assimilation for NCEP Global Forecast System: Single-resolution experiments.

,*Mon. Wea. Rev.***141**, 4098–4117, doi:10.1175/MWR-D-12-00141.1.Weissmann, M., R. H. Langland, C. Cardinali, P. M. Pauley, and S. Rahm, 2012: Influence of airborne Doppler wind lidar profiles near Typhoon Sinlaku on ECMWF and NOGAPS forecasts.

,*Quart. J. Roy. Meteor. Soc.***138**, 118–130, doi:10.1002/qj.896.Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP Global Forecast System.

,*Mon. Wea. Rev.***136**, 463–482, doi:10.1175/2007MWR2018.1.Zapotocny, T. H., W. P. Menzel, J. P. Nelson, and J. A. Jung, 2002: An impact study of five remotely sensed and five in situ data types in the Eta Data Assimilation System.

,*Wea. Forecasting***17**, 263–285, doi:10.1175/1520-0434(2002)017<0263:AISOFR>2.0.CO;2.Zapotocny, T. H., J. A. Jung, J. F. Le Marshall, and R. E. Treadon, 2007: A two-season impact study of satellite and in situ data in the NCEP Global Data Assimilation System.

,*Wea. Forecasting***22**, 887–909, doi:10.1175/WAF1025.1.Zou, X., A. Barcilon, I. M. Navon, J. Whitaker, and D. G. Cacuci, 1993: An adjoint sensitivity study of blocking in a two-layer isentropic model.

,*Mon. Wea. Rev.***121**, 2833–2857, doi:10.1175/1520-0493(1993)121<2833:AASSOB>2.0.CO;2.