Editorial Type:
Article
Article Type:
Research Article

The Gaussian Approach to Adaptive Covariance Inflation and Its Implementation with the Local Ensemble Transform Kalman Filter

Takemasa Miyoshi Department of Atmospheric and Oceanic Science, University of Maryland, College Park, College Park, Maryland

Search for other papers by Takemasa Miyoshi in
Current site
Google Scholar
PubMed Close
Print Publication:
01 May 2011
DOI:
https://doi.org/10.1175/2010MWR3570.1
Page(s):
1519–1535
Full access

Abstract

In ensemble Kalman filters, the underestimation of forecast error variance due to limited ensemble size and other sources of imperfection is commonly treated by empirical covariance inflation. To avoid manual optimization of multiplicative inflation parameters, previous studies proposed adaptive inflation approaches using observations. Anderson applied Bayesian estimation theory to the probability density function of inflation parameters. Alternatively, Li et al. used the innovation statistics of Desroziers et al. and applied a Kalman filter analysis update to the inflation parameters based on the Gaussian assumption. In this study, Li et al.’s Gaussian approach is advanced to include the variance of the estimated inflation as derived from the central limit theorem. It is shown that the Gaussian approach is an accurate approximation of Anderson’s general Bayesian approach. An advanced implementation of the Gaussian approach with the local ensemble transform Kalman filter is proposed, where the adaptive inflation parameters are computed simultaneously with the ensemble transform matrix at each grid point. The spatially and temporally varying adaptive inflation technique is implemented with the Lorenz 40-variable model and a low-resolution atmospheric general circulation model; numerical experiments show promising results both with and without model errors.

Corresponding author address: Takemasa Miyoshi, Department of Atmospheric and Oceanic Science, University of Maryland, College Park, College Park, MD 20742. E-mail: miyoshi@atmos.umd.edu

Abstract

In ensemble Kalman filters, the underestimation of forecast error variance due to limited ensemble size and other sources of imperfection is commonly treated by empirical covariance inflation. To avoid manual optimization of multiplicative inflation parameters, previous studies proposed adaptive inflation approaches using observations. Anderson applied Bayesian estimation theory to the probability density function of inflation parameters. Alternatively, Li et al. used the innovation statistics of Desroziers et al. and applied a Kalman filter analysis update to the inflation parameters based on the Gaussian assumption. In this study, Li et al.’s Gaussian approach is advanced to include the variance of the estimated inflation as derived from the central limit theorem. It is shown that the Gaussian approach is an accurate approximation of Anderson’s general Bayesian approach. An advanced implementation of the Gaussian approach with the local ensemble transform Kalman filter is proposed, where the adaptive inflation parameters are computed simultaneously with the ensemble transform matrix at each grid point. The spatially and temporally varying adaptive inflation technique is implemented with the Lorenz 40-variable model and a low-resolution atmospheric general circulation model; numerical experiments show promising results both with and without model errors.

Corresponding author address: Takemasa Miyoshi, Department of Atmospheric and Oceanic Science, University of Maryland, College Park, College Park, MD 20742. E-mail: miyoshi@atmos.umd.edu
Keywords: Kalman filters; Ensembles; Forecasting; Bayesian methods

1. Introduction

After Evensen (1994) developed the first ensemble Kalman filter (EnKF) for a quasigeostrophic model, many studies have been published advancing the EnKF in geophysical applications. Various EnKF algorithms have been proposed such as the ensemble adjustment Kalman filter (EAKF; Anderson 2001), the ensemble transform Kalman filter (ETKF; Bishop et al. 2001), the serial ensemble square root filter (serial EnSRF; Whitaker and Hamill 2002), and the local ensemble transform Kalman filter (LETKF; Hunt et al. 2007). EnKF methods have been applied to atmospheric models for advancing data assimilation and ensemble prediction (e.g., Houtekamer and Mitchell 1998, 2001; Szunyogh et al. 2005, 2008; Miyoshi and Yamane 2007). Recent applications to operational numerical weather prediction (NWP) systems have obtained promising results (e.g., Houtekamer and Mitchell 2005; Houtekamer et al. 2005; Bowler et al. 2008; Whitaker et al. 2008; Bonavita et al. 2008, 2010; Miyoshi et al. 2010). The Canadian Meteorological Centre (CMC) has used the EnKF to provide the initial conditions for the operational ensemble prediction system (EPS) since January 2005, and the Met Office started using an ETKF operationally in the summer of 2005.

The main difficulties for the EnKF in geophysical applications include variance underestimation, rank deficiency, and sampling error. It is known that the background error variance is usually underestimated because of various sources of imperfections such as limited ensemble size and model errors. Moreover, the limited ensemble size can cause significant rank deficiency and sampling errors in the generally high-dimensional background error covariance matrix. Therefore, the naive application of the EnKF to realistic systems usually results in filter divergence (i.e., the analysis diverges from the nature state while the ensemble spread remains small). These problems are treated by empirical covariance inflation and localization techniques. The ensemble perturbations are “inflated” to avoid underestimation of the error variance (Anderson and Anderson 1999). Off-diagonal components of the error covariance matrix are reduced according to how “far” they are from the diagonal terms, so that the “localized” covariance matrix has a higher rank and less sampling error (e.g., Houtekamer and Mitchell 2001; Hamill et al. 2001). With the perfect model, using more members and subensembles (Houtekamer and Mitchell 1998; Mitchell and Houtekamer 2009) could be an alternative to covariance inflation, as Houtekamer et al. (2009) showed that the ensemble spread almost perfectly estimated the ensemble mean error using 4 subensembles of 96 members (see their Fig. 3).

Many studies have shown that EnKF is generally sensitive to the choice of covariance inflation and localization methods and parameters, thus, it is important to optimize them (e.g., Houtekamer and Mitchell 1998; Ott et al. 2004; Szunyogh et al. 2005). However, optimization by trial and error is expensive and even prohibitive if we consider spatial and temporal variations of the parameters in realistic systems. To avoid manual optimization of localization functions, Anderson (2007a) and Bishop and Hodyss (2009a,b) proposed adaptive localization approaches. As for inflation, several kinds of inflation methods have been proposed. Many studies have adopted simple multiplicative inflation, where the forecast ensemble perturbations are multiplied by a factor slightly larger than 1. As better alternatives to account for the inhomogeneity of covariance underestimation, previous studies adopted additive inflation (e.g., Whitaker et al. 2008; Houtekamer et al. 2009), where random perturbations are added to the analysis ensemble members, or the relaxation-to-prior method (Zhang et al. 2004), where the forecast ensemble perturbations are combined with the analysis ensemble perturbations. Both of these techniques require manual optimization of their parameters: the amplitude and source of additive perturbations, and the relaxation weighting factor, respectively. Alternatively, Anderson (2007b, hereafter A07b), Anderson (2009, hereafter A09), and Li et al. (2009, hereafter LKM09) proposed adaptive inflation approaches, where multiplicative inflation parameters are estimated adaptively. This study focuses on adaptive inflation approaches.

Both A07b–A09 and LKM09 used observation data to estimate inflation parameters adaptively. A07b–A09 applied Bayesian estimation theory to the probability density function (PDF) of the inflation parameters. Alternatively, LKM09 used the innovation statistics of Desroziers et al. (2005) and applied a Kalman filter analysis update to the inflation parameters based on the Gaussian assumption. Although the general A07b–A09 approach requires solving a cubic equation or a quadratic equation (see appendix A of A07b–A09), the LKM09 approach is simpler because of the Gaussian assumption. This study compares the A07b–A09 and LKM09 approaches, and reveals the accuracy of the Gaussian approximation to the Bayesian estimate. Here, it is important to consider the appropriate variance of the inflation estimates in LKM09, which eliminates the need for forcing the upper and lower bounds of the estimated inflation parameters. Next, this study proposes an advanced implementation of the LKM09 approach with the LETKF, where the adaptive inflation parameters are computed simultaneously with the ensemble transform matrix at each grid point. This implementation naturally allows spatially and temporally varying adaptive inflation as in A09 and Bonavita et al. (2008, 2010).

A09 proposed spatially varying adaptive inflation that considers covariance localization, based on serial EnKF algorithms (e.g., EAKF, ETKF, and serial EnSRF) in which observations are treated serially. A09 used sample correlations of the ensemble between the observation space and the model space for converting the inflation estimates in the observation space to those in the model space. Here, covariance localization played a role in reducing sampling noise in the sample correlations. This A09 approach has been applied to the Data Assimilation Research Test bed (DART; Anderson et al. 2009) system, which has been used for realistic cases (e.g., Torn 2010a,b). However, the A09 approach is not directly applicable to the LETKF since the LETKF treats multiple observations simultaneously (Ott et al. 2004; Hunt et al. 2007).

Section 2 describes the Bayesian approach (A07b and A09), and section 3 describes the Gaussian approach (LKM09) with an additional derivation of the variance of the estimated inflation parameter. In section 4, the Bayesian and Gaussian approaches are compared. Then, section 5 describes an efficient implementation of the Gaussian approach with the LETKF. Sections 6 and 7 present numerical experiments with a low-order model and a low-resolution atmospheric general circulation model (AGCM), respectively. Here, experiments are performed under the perfect model assumption as well as with model errors. Finally, section 8 provides a summary and discussion.

2. Bayesian approach

Following A07b–A09, Bayes’s rule gives the PDF of inflation parameter α for a particular model variable, provided a set of independent scalar observations y:
e1
[cf. Eq. (5.5) of A07b and Eq. (6) of A09]. Subscripts denote the observation index in chronological order. The denominator is considered a normalization factor for the PDF. If the p newest observations are taken at the same time, applying Eq. (1) serially p times gives
e2
where, for simplicity, denote the posterior PDF Pr(αi|yi, yi−1, … , y0) and the prior PDF Pr(αi|yi−p, … , y0), respectively, after the analysis (a) and background (b).
The observation PDF is assumed to be Gaussian:
e3
[cf. Eq. (5.8) of A07b and Eq. (11) of A09]. Here, xi is an n-by-1 column vector of the prior estimate of the state, Hi is a 1-by-n vector of the observation operator for the ith observation yi, is an n-by-n matrix of the underestimated covariance matrix of xi, and Ri is the scalar variance of yi. Since the p observations are taken at the same time, the prior state estimate is the same for all p observations: xi = xi−1 = … = xi−p+1.
The prior PDF of the inflation parameter is assumed to be a Gaussian PDF with mean and variance :
e4
[cf. Eq. (5.4) of A07b and Eq. (5) of A09]. Here, N(μ, υ) denotes the Gaussian function with mean μ and variance υ.

Substituting Eqs. (3) and (4) into Eq. (2) gives the posterior PDF [cf. Eq. (5.9) of A07b and Eq. (12) of A09 in the case of p = 1]. Equation (3) is a Gaussian function with respect to xi and yi, but not for αi; consequently, the posterior PDF is non-Gaussian (A07b–A09). The maximum likelihood estimate of Eq. (1) requires solving a cubic equation or a quadratic equation (see appendix A of A07b–A09).

3. Gaussian approach

LKM09 proposed to apply the Kalman filter analysis update to the adaptive inflation parameter, assuming a Gaussian PDF for the newly available information. In this Gaussian approach, the posterior PDF is given by
e5
where is the estimated inflation parameter from the newest p observations, and is the error variance of . Since the prior PDF is also assumed to be Gaussian [Eq. (4)], the maximum likelihood estimate, or equivalently the minimum variance estimate with the Gaussian assumption, of the posterior is given by the Kalman filter analysis update:
e6
This is a simplification of the Bayesian approach due to the Gaussian assumption.
To obtain from p observations, LKM09 proposed to use the following statistical relationship derived by Daley (1992) and Desroziers et al. (2005):
e7
Here, d = y − Hx is the p-dimensional innovation vector, H is the observation operator, and is the p-by-p observation error covariance matrix. Since the diagonal terms of the covariance matrix are underestimated, the inflated covariance should satisfy Eq. (7). This condition yields the inflation parameter to be
e8
where ∘ denotes element-wise multiplication for the normalization of multiple observations. One may consider a more sophisticated normalization such that more accurate observations have more weight in the trace.
For the variance of the estimate , LKM09 gave as a constant tuning parameter. However, we can calculate the variance of Eq. (8) using the central limit theorem, which gives the variance of sample estimates. In the case of an estimate of the mean value of random samples drawn from a population with variance υ, the sample mean has the variance of υ/p, where p is the sample size. Hence, the sample estimate is expected to be more accurate when the sample size p is large (i.e., the law of large numbers). Similarly, the sample variance, instead of the sample mean, has a variance of 2υ2/p. In the present case, the innovations d are considered as random samples drawn from a population with zero mean and a covariance of . Since Eq. (8) contains the sample variance of d, we can calculate the variance of Eq. (8) using the central limit theorem, which yields the following:
e9
As our best estimate of the unknown true value α, in practice we use the prior estimate in Eq. (9).

4. Comparison of the Bayesian and Gaussian approaches

The difference between the Bayesian and Gaussian approaches is summarized by comparing Eqs. (2) and (5). Namely, the difference exists in the PDF from p observations:
e10
e11
To visualize the difference, the posterior and observation PDFs are computed numerically in the simple case of a scalar variable with underestimated error variance P and of p observations with error variance R. The innovation d is simulated by
e12
where η(μ, υ) denotes a random number drawn from a Gaussian PDF with mean μ and variance υ. Since P is the underestimated variance, the innovation is generated using the true variance of αtrueP. In the following simulations, the parameters are chosen to be P = 4.0, R = 9.0, and αtrue = 1.5. The prior estimate and the number of observations are fixed at and p = 100, respectively. The prior variance needs to be specified to compute the PDFs.

The first case employs a small but realistic value for the variance of the prior estimate , as in practice the inflation parameter is typically tuned to the nearest integer percent (i.e., the order of 0.01). Then, the posterior PDFs of Eqs. (2) and (5) are almost identical (thick curves of Fig. 1a, similar to the solid curve of Fig. 1a of A07b). Here, the PDFs are normalized for better visualization, so that the maximum values become 1. The increments from the prior PDF show a negligible difference (Fig. 1b, similar to Fig. 1b of A07b). If we look at the observation PDFs (the thin curves of Fig. 1a, similar to the dashed curve of Fig. 1a of A07b), the Bayesian PDF of Eq. (10) shows a slightly larger variance than the Gaussian PDF of Eq. (11), while the maximum likelihood estimates almost coincide. If we use αtrue = 1.5 in Eq. (9), the observation variance becomes very similar. In the neighborhood of the prior estimate , the slope and curvature of both observation PDFs are very similar, which justifies the negligible difference in the posterior PDFs.

Fig. 1.
Fig. 1.

Normalized PDFs from simple simulations with a realistic prior inflation variance of : (a) posterior PDFs (thick) and observation PDFs (thin), and (b) changes of PDFs from prior to posterior. Solid and dashed curves show PDFs of the Bayesian and Gaussian approaches, respectively. Since both approaches have nearly identical posterior PDFs, the dashed curves are obscured except for the thin dashed curve of (a).

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 2.
Fig. 2.

As in Fig. 1a, but with a large prior inflation variance of .

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 3.
Fig. 3.

As in Fig. 2, but using the true inflation in Eq. (9).

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

To highlight the impact of the non-Gaussianity of the Bayesian posterior PDF, the second case employs a large prior variance of , 100 times the standard deviation used in the previous experiment. The prior variance is now similar to the observation variance. Then, the posterior PDFs [Eqs. (2) and (5)] show clear differences, with slightly different maximum likelihood estimates (Fig. 2). Since the Gaussian approach shows a smaller observation variance, the maximum likelihood estimate of the posterior is closer to the observation than that of the Bayesian approach. Yet, the Gaussian approach approximates the Bayesian approach quite accurately. If we use αtrue = 1.5 in Eq. (9), the observation variance becomes very similar, although the Bayesian approach shows a slightly skewed PDF (Fig. 3). As a result, the maximum likelihood estimates of the posterior PDFs become very similar. Thus, the Gaussian approach best approximates the Bayesian approach when the prior estimate is closer to the true value.

Generally the non-Gaussianity of the PDFs used in the Bayesian approach is very weak, thus the Gaussian approach is an accurate approximation. The difference between the Bayesian and Gaussian approaches becomes more apparent when the prior variance and the observation variance are similar. The number of observations has an impact on the observation variance; more observations yield a smaller observation variance. Since the prior variance is typically on the order of , the observation variance becomes similar to this when the number of observations approaches p = O(105) based on Eq. (9). This could be the case in realistic NWP if we estimate a single inflation parameter using all global observations taken at a single assimilation time. If we allow spatial variation to the inflation values, the local observations are at most p = O(103), and the Gaussian approach would give an essentially identical solution to the Bayesian approach.

5. Implementation with the LETKF

LKM09 implemented the Gaussian approach with the LETKF. In this study, the LKM09 implementation is enhanced, so that the new implementation computes the inflation estimates simultaneously with the ensemble transform matrix. The major advancement from the LKM09 implementation comes from additional considerations of covariance localization and the observation variance of inflation parameters [Eq. (9)]. Covariance localization allows smooth spatial transitions of the local inflation estimates. Besides, Eq. (9) plays an essential role in smooth temporal transitions, particularly when the number of observations in local areas varies in space and time. Although the LKM09 implementation requires enforcing upper and lower bounds for the estimated inflation parameters, this is no longer needed because of the consideration of Eq. (9).

The proposed adaptive inflation algorithm solves the Kalman filter analysis update in Eq. (6) following LKM09. The first-guess mean is usually from the previous analysis, and the variance is prescribed as a tuning parameter. Here may be relaxed to a preset value from the previous analysis, especially when the observing network varies in time (A09). The inflation estimates with variance are computed by Eqs. (8) and (9). To include the impact of covariance localization, Eqs. (8) and (9) are modified and are computed simultaneously with the ensemble transform matrix at each grid point.

In the LETKF, the m-by-m ensemble transform matrix is computed for each lth subset of the state vector, where m denotes the ensemble size. Each l has its own local subset of global observations. Here, the subsets of observations are overlapping, but the subsets of the state vector are not necessarily overlapping, which is an advantage of the LETKF over the local ensemble Kalman filter (LEKF; Ott et al. 2004; Miyoshi et al. 2007). Usually the same local observation subset is chosen for all variables at the same grid point, in which case l is the gridpoint index. Then, the analysis ensemble members for the lth subset of the state vector are computed by
e13
Here, denotes an n-by-m matrix with each column consisting of the lth subset of the analysis (a) or forecast (f) state vector of each ensemble member, where n is the number of state-vector components in the lth subset. Here δ indicates ensemble perturbations (i.e., is an n-by-m matrix with each column consisting of the lth subset of forecast ensemble perturbations). The overbar denotes the ensemble mean, so that is an n-by-1 column vector of the lth subset of the forecast ensemble mean state. The denotes the n-by-m matrix with every element being a 1. The contains the terms from the analysis increment for the ensemble mean and from the update of the ensemble perturbations. The explicit formulation of is described by Hunt et al. (2007), but in this study, it is sufficient to note that is given as a function of the ensemble perturbations in the observation space , the observation error covariance matrix , and the observation increments (dl). The observation error covariance matrix is modified for each l to consider covariance localization, by multiplying the observation error variance by the inverse of the localization weighting function, so that far observations have less weight (Hunt et al. 2007). This has a similar effect as covariance localization of ρ ∘ , but is not equivalent (Miyoshi and Yamane 2007; Campbell et al. 2010; Greybush et al. 2011). Therefore, we use a different notation and replace the observation error covariance matrix with to include observation-space covariance localization in the LETKF. Here, contains the localization weights for each observation, which are values between 0 and 1.
When we compute the ensemble transform matrix in the LETKF, we can also compute the inflation factor simultaneously for each l by modifying Eq. (8) as follows:
e14
Here, is the sample covariance by the forecast ensemble. Covariance localization acts as a spatial smoother to the inflation estimate in the same way it does to . Since “closer” observations have more weight, localization helps to smooth transitions among different l in nearby regions. As with , covariance localization is considered in Eq. (9) to obtain as
e15
Here, the effective number of observations is estimated using the localization weight . Intuitively, a far observation with a smaller localization weight, say 0.2, is counted as 0.2 observations; the “closest” observation with the localization weight of 1 is counted as 1 observation. Equation (15) requires only small additional computational cost since components are shared with Eq. (14).

Multiplicative inflation is usually applied to the background ensemble perturbations (i.e., ). Hunt et al. (2007) took advantage of the LETKF formulation and proposed an equivalent, but slightly more efficient implementation, in which multiplicative inflation is embedded in the transform matrix . This study employs this efficient implementation.

6. Experiments with a low-order model

a. Perfect-model experiments

For the first assessment of the proposed adaptive inflation method, experiments are performed with the Lorenz 40-variable model (Lorenz 1996; Lorenz and Emanuel 1998). The model parameters are chosen to be identical to those of Lorenz and Emanuel (1998), except that the model time step has been shortened from 0.05 nondimensional units (corresponding to 6 h) to 0.005 so that the model is integrated more frequently than once per assimilation cycle. Following Lorenz and Emanuel (1998), observations are given only at the first 20 “land” grid points: J = 1, … , 20, where J denotes the grid index. The other 20 “ocean” grid points (J = 21, … , 40) do not have observations. A nature run is generated after a 100-yr spinup from a field consisting of 40 random numbers. Then, random numbers drawn from a Gaussian PDF with a standard deviation of 1.0 are added to the nature run at J = 1, … , 20 to generate 20 observations every 6 h. The simulated observation data are assimilated using the LETKF with 10 ensemble members for 110 yr (i.e., 158 400 assimilation steps). Time averages are taken for the latter 100 yr after spinning up for 10 yr. The covariance localization parameter of the LETKF, which is defined as the one standard deviation radius of the Gaussian function, is fixed at 3.0 grid points throughout this study. The corresponding radius of influence is times the one standard deviation radius (Gaspari and Cohn 1999; Hamill et al. 2001; Miyoshi and Yamane 2007), so that the present localization setting considers observations up to 10 grid points away.

First, perfect-model identical twin experiments are performed, where the same forecast model for generating the nature run is used for data assimilation experiments. Experiments are performed with 1.5% fixed covariance inflation and adaptive inflation using four different prior inflation variance settings: , 0.022, 0.042, 0.082. The 1.5% inflation is a manually optimized value to yield the minimum global analysis root-mean-square errors (RMSEs) among 1.0%, 1.5%, 2%, 2.5%, and 3% fixed inflation settings. Computation time is not significantly increased as a result of the adaptive inflation scheme.

The LETKF performs robustly with both fixed and adaptive inflation schemes. Since the prior inflation variance corresponds to the time-smoothing strength, a larger value yields more temporal fluctuations, whereas a smaller value makes the spinup slower (Fig. 4). With a small prior variance of , it takes nearly 40 yr to spin up the adaptive inflation, but thereafter the time series is almost constant. By contrast, a large prior variance of produces a noisy time series with almost instantaneous spinup. The results agree well with those of A07b that showed similar time series using the Lorenz 40-variable model but with different experimental settings (Fig. 9 of A07b). Yet, over land with dense observations, the time-mean inflation estimates as well as the analysis RMSE and ensemble spread seems to be robust among different prior variance settings (Table 1 and Fig. 5). By contrast, ocean grid points show relatively large sensitivity to the prior variance, probably because stronger adaptation to individual observations due to larger prior variance may lead to increased variance over nonobserved areas. In addition, the RMSE tends to be larger than the ensemble spread over the ocean grid points, particularly when the prior variance is set to be smaller. A09 showed a similar mismatch between the RMSE and ensemble spread over nonobserved regions (Fig. 3 of A09). To investigate the impact of adaptive inflation, the relative improvement of RMSE is defined by
e16
where RMSECtrl and RMSETest denote the RMSEs of a baseline control experiment and a test experiment, respectively. Since smaller RMSE corresponds to more accuracy, positive values correspond to improvement. Here, the 1.5% fixed inflation experiment is used as the baseline, and the relative improvements due to adaptive inflation are computed. Results indicate that adaptive inflation improves the analysis accuracy if we choose a proper prior variance setting (Fig. 6). Although is the optimal choice, the results are relatively robust with the wide range of prior variance settings.
Fig. 4.
Fig. 4.

Time series of the global-mean adaptive inflation parameters for two Lorenz model experiments under the perfect-model assumption with four prior inflation variance settings as shown in the legend.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 5.
Fig. 5.

The 100-yr-mean analysis RMSE and ensemble spread (SPREAD) of the Lorenz model experiments with 1.5% fixed inflation and adaptive inflation using 4 prior inflation variance settings as shown in the legend, averaged over all grid points (global, J = 1, … , 40), observed “land” grid points (land, J = 1, … , 20), and unobserved “ocean” grid points (ocean, J = 21, … , 40).

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 6.
Fig. 6.

Improvements (%) of analysis RMSE, as shown in Fig. 5, of the adaptive inflation experiments relative to the 1.5% fixed inflation experiment.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Table 1.

Time-mean adaptive inflation values averaged over land and ocean grid points for the Lorenz 40-variable model.

Table 1.

b. Experiments with model errors

To investigate how the adaptive inflation method performs with model errors, the forcing parameter F of the Lorenz model is modified, which is a simple and widely used approach for including model errors in data assimilation experiments (e.g., A07b). Here, the same simulated observations are used, which is based on the nature run using the default forcing parameter of F = 8. However, the LETKF experiments employ a modified forcing parameter of F = 10, so that the forecast model is no longer perfect. Experiments are performed with 9% fixed covariance inflation and adaptive inflation using three different prior inflation variance settings: , 0.022, 0.042. The 9% inflation is again manually tuned to yield the minimum global analysis RMSE among 8%, 9%, and 10% inflation values. Here is removed since it showed a very noisy time series and the worst analysis accuracy in the perfect-model experiments.

The LETKF performs robustly with model errors. The adaptive inflation parameters are significantly larger, ∼20% global average in contrast to ∼4% of the perfect-model case (Table 2 and Fig. 7). The spinup times and temporal fluctuations are very similar to the perfect-model results. The contrast of the adaptive inflation values between the land and ocean grid points is much greater than for the perfect-model results (Tables 1 and 2). The model errors cause larger observation-minus-forecast innovations, which result in larger inflation estimates over land. However, the ocean grid points have no direct observations, which means there is less reduction of the ensemble spread in the analysis update and hence small inflation. The greater contrast between the land and ocean makes the globally constant fixed inflation even worse; the RMSE comparison shows clear improvement because of adaptive inflation both over land and the ocean (Fig. 8). With the optimal 9% fixed inflation, the ensemble spread significantly underestimates the RMSE over land, and the opposite is the case over the ocean. This problem is greatly reduced with the adaptive inflation, although not perfectly.

Fig. 7.
Fig. 7.

As in Fig. 4, but in the case with model errors. The dashed line is the same as the black line in Fig. 4, and is a reference to the perfect-model case.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 8.
Fig. 8.

As in Fig. 5, but for the case with model errors.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Table 2.

As in Table 1, but in the case with model error.

Table 2.

7. Experiments with an AGCM

a. Perfect-model experiments

The proposed adaptive inflation method is implemented and assessed with a low-resolution AGCM known as the Simplified Parameterization, Primitive-Equation Dynamics model (SPEEDY; Molteni 2003). The SPEEDY model employs the primitive equation spectral dynamical core with truncation at 30 wavenumbers and 7 vertical levels. Simplified physical schemes include essential processes such as longwave and shortwave radiation, cloud processes (large-scale condensation and convective process), and boundary layer turbulences. After Miyoshi (2005) adapted the SPEEDY model for data assimilation experiments, this model has been widely used in many studies to test new ideas of data assimilation (e.g., Harlim and Hunt 2007; Fertig et al. 2007, 2009; LKM09).

First, perfect-model identical twin experiments are performed. The nature run is generated after a 1-yr spinup from the standard atmosphere at rest. The observation data are simulated by adding to the nature run random numbers drawn from Gaussian PDFs with standard deviations of 1.0 m s−1 for zonal and meridional wind components, 1.0 K for temperature, 0.001 kg kg−1 for specific humidity, and 1.0 hPa for surface pressure. Observations are taken every 6 h at all 7 vertical levels at given horizontal stations of either a radiosonde-like network or a regular network (Fig. 9). Although the regular network appears to be homogeneous in the longitude–latitude coordinate system, the observations are geographically denser in the higher latitudes and extend farther south than north. The simulated observation data are assimilated using the LETKF with 20 ensemble members for a year starting from 1 January. Time averages are taken for the final 2 months (i.e., November and December). Experiments are performed with 3% fixed covariance inflation and adaptive inflation with three different prior inflation variance settings: , 0.042, 0.062. The 3% inflation is the manually optimized value to yield the minimum global analysis RMSE among 2%, 3%, 4%, and 5% fixed inflation settings. For the adaptive inflation experiments, 3% inflation is chosen as the initial prior value, unless otherwise noted. The localization parameters are fixed at 500 km in the horizontal and at 0.1 lnp in the vertical coordinate throughout this study; the corresponding radii of influence are approximately 1825 km in the horizontal and 0.365 lnp coordinate in the vertical. The chosen values may not be perfectly optimal, but they result in a reasonable performance of the LETKF. (The complete LETKF system is available online at http://code.google.com/p/miyoshi/.) Hemispheric averages are computed as area averages between 20° and the Poles throughout this study.

Fig. 9.
Fig. 9.

Horizontal maps of (a) a radiosonde-like observing network (416 stations) and (b) a regular observing network (448 stations). Dots indicate observing stations.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

With the radiosonde-like network (Fig. 9a), the time series of RMSE and ensemble spread indicate stable performance of the LETKF (Fig. 10). As a result of the sparser observations in the SH, RMSE and ensemble spread are significantly larger in the SH than in the NH. A similar separation of hemispheric averages is found in the time series of adaptive inflation estimates (Fig. 11). Adaptive inflation parameters are significantly larger in the densely observed NH than those in the SH. Larger prior inflation variance corresponds to a faster response, which is found in the timing and magnitude of the initial inflation increase. Namely, shows a fast and large increase, whereas a slow and small increase, and in the middle. The inflation parameters decrease after the initial increase, during which larger prior inflation variance corresponds to faster and greater response. Overall, adaptive inflation provides a stable performance with this range of prior inflation variance. After about 6 months, all experiments show similar inflation values. However, the inflation estimates do not perfectly converge to a single value, probably because of the different responses to temporal variations, or possibly because multiple stable states may exist. Namely, small (large) inflation in a region could cause large (small) inflation in downstream regions, thus the stable solution of inflation fields may not be unique. Detailed investigations on this issue are beyond the scope of this study.

Fig. 10.
Fig. 10.

Time series of the hemispheric-mean zonal-wind analysis RMSE (m s−1, solid) and ensemble spread (m s−1, dotted) at the lowest model level for the SPEEDY model experiments with 3% fixed inflation (black for RMSE, blue for spread) and adaptive inflation with the prior inflation variance of (red for RMSE, orange for spread). The RMSE and ensemble spread are smaller in the (bottom curves) NH than the (top curves) SH.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 11.
Fig. 11.

Time series of the hemispheric-mean adaptive inflation parameters at the lowest model level for the SPEEDY model experiments with three prior inflation variance settings as shown in the legend. Thick (thin) curves indicate the NH (SH).

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

The RMSE indicates general improvements due to adaptive inflation, although the analysis accuracy is considerably sensitive to the choice of the prior inflation variance settings (Fig. 12). The provides the most accurate analysis, significantly better than the fixed inflation case in both hemispheres. The performs similarly in the SH, but with significant degradation in the NH. Yet, outperforms the 3% fixed inflation case in both hemispheres. The is degraded in the SH and shows comparable analysis accuracy to the fixed inflation case in both hemispheres. Therefore, we may need a certain level of manual optimization of , which is only a global constant. Although it was prohibitive to optimize the spatially varying inflation parameters directly, it would be feasible to find a reasonable value of the global constant . Hereafter, is fixed at 0.042.

Fig. 12.
Fig. 12.

Hemispheric-mean zonal-wind analysis RMSE (m s−1) in the NH (solid) and the SH (dotted) for the SPEEDY model experiments, averaged over 2 months in November and December. Colors, as shown in the legend, refer to experiments with 3% fixed inflation and 3 prior inflation variance settings.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

The horizontal pattern of adaptive inflation generally agrees with the observing density pattern (Fig. 13), although there are locations with deflation (i.e., inflation parameter values less than 1.0). These locations tend to be located downstream of no-observation areas. Inflation of the no-observation areas causes the ensemble spread to be overestimated in the downstream areas, which is compensated by the deflation. Since the initial prior inflation value is not updated in the no-observation areas, the initial prior inflation setting plays a fatal role in those regions. Additional experiments are performed with different initial prior inflation settings of 0%, 1%, and 2%, showing impact not only on the deflation areas but also on the RMSE. A smaller initial prior inflation value yields less deflation areas. As for the RMSE, the initial prior inflation of 3% gives the minimum analysis RMSE in both hemispheres (Fig. 14). Although Fig. 13 shows a snapshot at the end of the year, the temporal variation is generally small at each grid point, so a similar pattern appears at any single assimilation time.

Fig. 13.
Fig. 13.

Horizontal map of adaptive inflation at the bottom model level when at 1800 UTC 31 Dec after a year of data assimilation.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 14.
Fig. 14.

As in Fig. 12, but with 4 initial prior inflation settings as shown in the legend. The prior inflation variance is fixed at .

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

The zonal-mean vertical structure (Fig. 15b) indicates large inflation in the NH except for the upper levels. The metric defined by Eq. (16) shows significant improvement almost everywhere due to adaptive inflation (Fig. 15a). We see about 20% reduction of RMSE in most areas, and the improvement reaches 30% particularly in the NH. In addition, the ensemble spread is generally improved by the adaptive inflation (Figs. 15c,d). With fixed inflation the ensemble spread of zonal wind component overestimates the RMSE by about 1 m s−1 at the top level and underestimates it by about 0.5 m s−1 near the polar regions (Fig. 15c), which are significantly improved by adaptive inflation (Fig. 15d). Yet, there is a significant mismatch between the RMSE and ensemble spread, which may be related to the convergence of the inflation estimates. The additional computational time due to adaptive inflation was negligible.

Fig. 15.
Fig. 15.

Zonal-mean structures for U-wind component in the case of a radiosonde-like observing network: (a) relative improvement (%) of analysis RMSE due to adaptive inflation, (b) adaptive inflation values, (c) ensemble spread minus RMSE (m s−1) for 3% fixed inflation, and (d) ensemble spread minus RMSE (m s−1) for adaptive inflation.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Finally, Fig. 16 shows the results for the regular observing network (Fig. 9b). Adaptive inflation ( and 3% initial prior inflation) outperforms fixed inflation by about 25% in the lower and middle troposphere in both hemispheres. The SH shows smaller RMSE than the NH, which is due to generally smaller RMSE in the summer hemisphere. A similar figure averaged over boreal summer months indicates larger RMSE in the SH. At the top two levels, higher than about the 200-hPa level, the ensemble spread tends to underestimate (overestimate) the RMSE with adaptive inflation (3% fixed inflation). Since 3% initial prior inflation is chosen in adaptive inflation, the ensemble spread overestimates the RMSE initially. The overestimation initiates adaptive deflation after mid-April (Fig. 17), which causes the spread to underestimate the RMSE. The underestimation then initiates increasing the adaptive inflation estimates, and this is still undertransition even after the 1-yr experiment. This transient process is primarily determined by the information from observations (i.e., the number and accuracy of observations). With a given observing network, the transient process is sensitive to the two parameters of adaptive inflation: the prior inflation variance and the initial prior inflation value. Moreover, since the persistence evolution is assumed for inflation, adaptive inflation yields always delayed response. Considering a tendency or a “forecast” of prior inflation to avoid such delays may be important for further improvements, which is beyond the scope of this study. As an example of inflation forecast, A09 describes the idea of damping the inflation value toward 1 as a function of time, and Torn (2010b) obtained its successful performance.

Fig. 16.
Fig. 16.

Hemispheric-mean zonal-wind analysis RMSE (m s−1, solid) and corresponding ensemble spread (m s−1, dotted) in the (a) NH and (b) SH for the SPEEDY model experiments, averaged over 2 months in November and December. Thin and thick curves correspond to 3% fixed inflation and adaptive inflation, respectively.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 17.
Fig. 17.

Time series of the hemispheric-mean adaptive inflation parameters at the second-from-the-top model level for the SPEEDY model experiments with the regular observing network. Solid (dotted) curves indicate the NH (SH).

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

b. Experiments with model errors

To investigate the performance of the adaptive inflation scheme with model errors, imperfect model experiments are performed by perturbing the parameters of the SPEEDY model. Following J. Ruiz (2010, personal communication), the parameters of the convective parameterization are modified (Table 3). Moreover, diffusion and wind-drag time-scale parameters are increased by 20%. Although these modified parameters are used in the LETKF, the nature run and simulated observations are the same as in the previous experiments.

Table 3.

SPEEDY model parameters for the perfect and imperfect model experiments.

Table 3.

Although the model error introduced by this approach is not very large, it is significant enough to require larger covariance inflation. In experiments with model errors, covariance inflation accounts for the role of not only the limited ensemble size but also model errors in underestimating forecast error variance. Here 6% is the manually tuned globally constant inflation to yield the minimum global analysis RMSE, in contrast to the 3% inflation used in the perfect-model case. Here, as suggested by the perfect-model experiments, the top level requires smaller inflation (1%) to avoid filter divergence. Similarly, adaptive inflation yields larger inflation in the NH (Fig. 18). The RMSE shows slight but consistent advantages of adaptive inflation. Moreover, notable advantages are found in the ensemble spread, as it better represents the RMSE (Fig. 19). Overall, adaptive inflation provides promising results with model errors.

Fig. 18.
Fig. 18.

Time series of the hemispheric-mean adaptive inflation parameters at the lowest model level for the perfect-model (dashed) and imperfect-model (solid) SPEEDY model experiments. Thick (thin) curves indicate the NH (SH).

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

Fig. 19.
Fig. 19.

As in Fig. 16, but for the experiments with model errors and the radiosonde-like observing network. Thin and thick curves correspond to 6% fixed inflation and adaptive inflation, respectively.

Citation: Monthly Weather Review 139, 5; 10.1175/2010MWR3570.1

8. Summary and discussion

This study compared two approaches to adaptive covariance inflation in the EnKF: A07b/A09’s Bayesian approach and LKM09’s Gaussian approach. The Gaussian approach provides a reasonable approximation to the more general Bayesian approach, especially when the prior inflation variance is smaller than the variance from observations. Then, the LKM09 implementation of the Gaussian approach with the LETKF was advanced so that the inflation parameter αl is computed simultaneously with the transform matrix at each grid point l by considering the covariance localization function. This allows stable estimates of spatially varying inflation parameters. The numerical experiments with the Lorenz 40-variable model and the SPEEDY model indicated generally promising results both with and without model errors, although further consideration of a time tendency of prior inflation may be important for more improvements. The additional computational time due to adaptive inflation was negligible.

This method showed considerable sensitivity to the choice of the prior inflation variance . The SPEEDY model results were generally sensitive to . Although it was prohibitive to optimize the spatially varying inflation parameters directly, it would be feasible to find a reasonable value of the global constant . The Lorenz model results were generally more robust to the choice of , but the results were considerably sensitive in the sparsely observed regions (i.e., the ocean grid points). A09 performed similar Lorenz model experiments with the Bayesian approach and mentioned that “results show small sensitivity to the fixed value of the inflation standard deviation between 0.01 and 0.5,” but A09 did not elaborate the different responses between the densely and sparsely observed regions. If the prior inflation PDF is assumed to be a Gaussian PDF significantly narrower than the observation PDF, the non-Gaussianity is negligible (section 4). With a large , the non-Gaussianity may have an impact. Thorough investigations on the sensitivity to the prior inflation variance remain to be a subject of possible future research.

This method yielded significant mismatch between the RMSE and ensemble spread, particularly over sparsely observed areas. Since this method is based on innovation statistics and utilizes observation data for estimating inflation parameters, it is natural to have difficulties with fewer observations. If no observation exists within the radius of influence around a grid point, the inflation value at the grid point is not updated and holds the initial value. Sparser observations yield slower adaptation of inflation estimates. Moreover, sampling errors in the forecast error covariance between distant locations may lead to spurious adjustments to the inflation values due to distant observations, though covariance localization reduces the impact. These may contribute to the mismatch between the RMSE and ensemble spread, although more investigations are necessary for a better understanding.

With this method, inflation is adaptively estimated every time when the ensemble transform matrix is computed. Kang (2009) and Kang et al. (2011) found that covariance localization between variables played an essential role in reducing sampling errors in data assimilation of carbon dioxide and the determination of surface carbon fluxes. With “variable localization,” the ensemble transform matrix is computed for each variable separately, and the proposed inflation method is naturally compatible with such an approach. When some variables are only weakly affected by observations, variable localization is effective to avoid applying a large inflation factor to such variables.

This study dealt only with fixed observation locations. In addition, this study simulated model errors by modifying the model parameters. In reality, however, the observing network changes significantly in time, and models have significant systematic errors more complex than those generated by just changing the parameters. In this regard, Y. Ota (2010, personal communication) of the Japan Meteorological Agency (JMA) performed LETKF experiments with the operational global NWP system to assimilate real observations including satellite radiances, and found that the present adaptive inflation significantly improved most verification scores. Moreover, Miyoshi and Kunii (2011) assimilated real observations with a mesoscale model known as the Weather Research and Forecasting (WRF) model and obtained promising results for this adaptive inflation method. Furthermore, S. Penny (2010, personal communication) applied the LETKF to an ocean general circulation model known as the Modular Ocean Model (MOM) and found that the adaptive inflation played an essential role in assimilating sparse and irregular ocean observations. These new results, which will be reported separately, verify that the proposed method is promising in realistic applications.

Throughout this paper, the observation error statistics are assumed to be perfectly known, which is invalid in practice. LKM09 proposed a similar Gaussian approach to adaptively estimate the observation error variance. It will be an important future study to apply both the adaptive estimations of observation error variance and covariance inflation.

Acknowledgments

The author is grateful to Eugenia Kalnay, Hong Li, Ji-Sun Kang, Steve Greybush, and other members of the Chaos/Weather group of the University of Maryland, and to Fuqing Zhang of The Pennsylvania State University, Jeff Anderson of the National Center for Atmospheric Research (NCAR), Junjie Liu of the University of California at Berkeley, Massimo Bonavita of the European Centre for Medium-Range Weather Forecasts (ECMWF), Lucio Torrisi of Italian Meteorological Service, and Yoichiro Ota of JMA for insightful discussions. Juan Ruiz of University of Buenos Aires provided very useful suggestions on the SPEEDY model physics parameters for introducing model errors. The SPEEDY model was kindly provided by Franco Molteni and Fred Kucharski. The author thanks anonymous reviewers for very constructive comments to improve this study. This study was supported by the Office of Naval Research (ONR) Grant N000141010149 under the National Oceanographic Partnership Program (NOPP), by the Department of Energy (DOE) Grant DEFG0207ER64437, and by NASA Grants NNX07AM97G and NNX08AD40G.

REFERENCES

  • Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884–2903.

  • Anderson, J. L., 2007a: Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter. Physica D, 230, 99–111.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., 2007b: An adaptive covariance inflation error correction algorithm for ensemble filters. Tellus, 59A, 210–224.

  • Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72–83.

  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 2741–2758.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 1283–1296.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., and D. Hodyss, 2009a: Ensemble covariances adaptively localized with ECO-RAP. Part 1: Tests on simple error models. Tellus, 61A, 84–96.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., and D. Hodyss, 2009b: Ensemble covariances adaptively localized with ECO-RAP. Part 2: A strategy for the atmosphere. Tellus, 61A, 97–111.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420–436.

    • Search Google Scholar
    • Export Citation

  • Bonavita, M., L. Torrisi, and F. Marcucci, 2008: The ensemble Kalman filter in an operational regional NWP system: Preliminary results with real observations. Quart. J. Roy. Meteor. Soc., 134, 1733–1744.

    • Search Google Scholar
    • Export Citation

  • Bonavita, M., L. Torrisi, and F. Marcucci, 2010: Ensemble data assimilation with the CNMCA regional forecasting system. Quart. J. Roy. Meteor. Soc., 136, 1132–1145.

    • Search Google Scholar
    • Export Citation

  • Bowler, N. E., A. Arribas, K. R. Mylne, K. B. Robertson, and S. E. Beare, 2008: The MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134, 703–722.

    • Search Google Scholar
    • Export Citation

  • Campbell, W. F., C. H. Bishop, and D. Hodyss, 2010: Vertical covariance localization for satellite radiances in ensemble Kalman filters. Mon. Wea. Rev., 138, 282–290.

    • Search Google Scholar
    • Export Citation

  • Daley, R., 1992: Estimating model-error covariances for application to atmospheric data assimilation. Mon. Wea. Rev., 120, 1735–1746.

  • Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 3385–3396.

    • Search Google Scholar
    • Export Citation

  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99 (C5), 10 143–10 162.

    • Search Google Scholar
    • Export Citation

  • Fertig, E. J., B. R. Hunt, E. Ott, and I. Szunyogh, 2007: Assimilating non-local observations with a local ensemble Kalman filter. Tellus, 59A, 719–730.

    • Search Google Scholar
    • Export Citation

  • Fertig, E. J., and Coauthors, 2009: Observation bias correction with an ensemble Kalman filter. Tellus, 61A, 210–226.

  • Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723–757.

    • Search Google Scholar
    • Export Citation

  • Greybush, S. J., E. Kalnay, T. Miyoshi, K. Ide, and B. R. Hunt, 2011: Balance and ensemble Kalman filter localization techniques. Mon. Wea. Rev., 139, 511–522.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 2776–2790.

    • Search Google Scholar
    • Export Citation

  • Harlim, J., and B. R. Hunt, 2007: Four-dimensional local ensemble transform Kalman filter: Numerical experiments with a global circulation model. Tellus, 59A, 731–748.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796–811.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123–137.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 2005: Ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 131, 3269–3289.

  • Houtekamer, P. L., H. L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133, 604–620.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., H. K. Mitchell, and X. Deng, 2009: Model error representation in an operational ensemble Kalman filter. Mon. Wea. Rev., 137, 2126–2143.

    • Search Google Scholar
    • Export Citation

  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112–126.

    • Search Google Scholar
    • Export Citation

  • Kang, J.-S., 2009: Carbon cycle data assimilation using a coupled atmosphere-vegetation model and the local ensemble transform Kalman filter. Ph.D. dissertation, University of Maryland, College Park, 164 pp.

    • Search Google Scholar
    • Export Citation

  • Kang, J.-S., E. Kalnay, J. Liu, I. Fung, T. Miyoshi, and K. Ide, 2011: “Variable localization” in an Ensemble Kalman Filter: Application to the carbon cycle data assimilation. J. Geophys. Res., in press.

    • Search Google Scholar
    • Export Citation

  • Li, H., E. Kalnay, and T. Miyoshi, 2009: Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 523–533.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E., 1996: Predictability: A problem partly solved. Proc. ECMWF Seminar on Predictability, Vol. 1, Reading, United Kingdom, ECMWF, 1–18.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E., and K. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399–414.

    • Search Google Scholar
    • Export Citation

  • Mitchell, H. L., and P. L. Houtekamer, 2009: Ensemble Kalman filter configurations and their performance with the logistic map. Mon. Wea. Rev., 137, 4325–4343.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., 2005: Ensemble Kalman filter experiments with a primitive-equation global model. Ph.D. dissertation, University of Maryland, College Park, 197 pp.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., and S. Yamane, 2007: Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution. Mon. Wea. Rev., 135, 3841–3861.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., S. Yamane, and T. Enomoto, 2007: Localizing the error covariance by physical distances within a local ensemble transform Kalman filter (LETKF). SOLA, 3, 89–92, doi:10.2151/sola.2007-023.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., Y. Sato, and T. Kadowaki, 2010: Ensemble Kalman filter and 4D-Var intercomparison with the Japanese operational global analysis and prediction system. Mon. Wea. Rev., 138, 2846–2866.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., and M. Kunii, 2011: The local ensemble transform Kalman filter with the Weather Research and Forecasting model: Experiments with real observations. Pure Appl. Geophys., in press.

    • Search Google Scholar
    • Export Citation

  • Molteni, F., 2003: Atmospheric simulations using a GCM with simplified physical parametrizations. I: Model climatology and variability in multi-decadal experiments. Climate Dyn., 20, 175–191.

    • Search Google Scholar
    • Export Citation

  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415–428.

  • Szunyogh, I., E. J. Kostelich, G. Gyarmati, D. J. Patil, B. R. Hunt, E. Kalnay, E. Ott, and J. A. Yorke, 2005: Assessing a local ensemble Kalman filter: Perfect model experiments with the National Centers for Environmental Prediction global model. Tellus, 57A, 528–545.

    • Search Google Scholar
    • Export Citation

  • Szunyogh, I., E. J. Kostelich, G. Gyarmati, E. Kalnay, B. R. Hunt, E. Ott, E. Satterfield, and J. A. Yorke, 2008: A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A, 113–130.

    • Search Google Scholar
    • Export Citation

  • Torn, R. D., 2010a: Ensemble-based sensitivity analysis applied to African Easterly Waves. Wea. Forecasting, 25, 61–78.

  • Torn, R. D., 2010b: Performance of a mesoscale ensemble Kalman filter (EnKF) during the NOAA high-resolution hurricane test. Mon. Wea. Rev., 138, 4375–4392.

    • Search Google Scholar
    • Export Citation

  • Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1924.

  • Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP Global Forecast System. Mon. Wea. Rev., 136, 463–482.

    • Search Google Scholar
    • Export Citation

  • Zhang, F., C. Snyder, and J. Sun, 2004: Impacts of initial estimate and observation availability on convective-scale data assimilation with ensemble Kalman filter. Mon. Wea. Rev., 132, 1238–1253.

    • Search Google Scholar
    • Export Citation
Save
Cover Monthly Weather Review
Monthly Weather Review

  • Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884–2903.

  • Anderson, J. L., 2007a: Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter. Physica D, 230, 99–111.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., 2007b: An adaptive covariance inflation error correction algorithm for ensemble filters. Tellus, 59A, 210–224.

  • Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72–83.

  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 2741–2758.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 1283–1296.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., and D. Hodyss, 2009a: Ensemble covariances adaptively localized with ECO-RAP. Part 1: Tests on simple error models. Tellus, 61A, 84–96.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., and D. Hodyss, 2009b: Ensemble covariances adaptively localized with ECO-RAP. Part 2: A strategy for the atmosphere. Tellus, 61A, 97–111.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420–436.

    • Search Google Scholar
    • Export Citation

  • Bonavita, M., L. Torrisi, and F. Marcucci, 2008: The ensemble Kalman filter in an operational regional NWP system: Preliminary results with real observations. Quart. J. Roy. Meteor. Soc., 134, 1733–1744.

    • Search Google Scholar
    • Export Citation

  • Bonavita, M., L. Torrisi, and F. Marcucci, 2010: Ensemble data assimilation with the CNMCA regional forecasting system. Quart. J. Roy. Meteor. Soc., 136, 1132–1145.

    • Search Google Scholar
    • Export Citation

  • Bowler, N. E., A. Arribas, K. R. Mylne, K. B. Robertson, and S. E. Beare, 2008: The MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134, 703–722.

    • Search Google Scholar
    • Export Citation

  • Campbell, W. F., C. H. Bishop, and D. Hodyss, 2010: Vertical covariance localization for satellite radiances in ensemble Kalman filters. Mon. Wea. Rev., 138, 282–290.

    • Search Google Scholar
    • Export Citation

  • Daley, R., 1992: Estimating model-error covariances for application to atmospheric data assimilation. Mon. Wea. Rev., 120, 1735–1746.

  • Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 3385–3396.

    • Search Google Scholar
    • Export Citation

  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99 (C5), 10 143–10 162.

    • Search Google Scholar
    • Export Citation

  • Fertig, E. J., B. R. Hunt, E. Ott, and I. Szunyogh, 2007: Assimilating non-local observations with a local ensemble Kalman filter. Tellus, 59A, 719–730.

    • Search Google Scholar
    • Export Citation

  • Fertig, E. J., and Coauthors, 2009: Observation bias correction with an ensemble Kalman filter. Tellus, 61A, 210–226.

  • Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723–757.

    • Search Google Scholar
    • Export Citation

  • Greybush, S. J., E. Kalnay, T. Miyoshi, K. Ide, and B. R. Hunt, 2011: Balance and ensemble Kalman filter localization techniques. Mon. Wea. Rev., 139, 511–522.

    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 2776–2790.

    • Search Google Scholar
    • Export Citation

  • Harlim, J., and B. R. Hunt, 2007: Four-dimensional local ensemble transform Kalman filter: Numerical experiments with a global circulation model. Tellus, 59A, 731–748.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796–811.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123–137.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 2005: Ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 131, 3269–3289.

  • Houtekamer, P. L., H. L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133, 604–620.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., H. K. Mitchell, and X. Deng, 2009: Model error representation in an operational ensemble Kalman filter. Mon. Wea. Rev., 137, 2126–2143.

    • Search Google Scholar
    • Export Citation

  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112–126.

    • Search Google Scholar
    • Export Citation

  • Kang, J.-S., 2009: Carbon cycle data assimilation using a coupled atmosphere-vegetation model and the local ensemble transform Kalman filter. Ph.D. dissertation, University of Maryland, College Park, 164 pp.

    • Search Google Scholar
    • Export Citation

  • Kang, J.-S., E. Kalnay, J. Liu, I. Fung, T. Miyoshi, and K. Ide, 2011: “Variable localization” in an Ensemble Kalman Filter: Application to the carbon cycle data assimilation. J. Geophys. Res., in press.

    • Search Google Scholar
    • Export Citation

  • Li, H., E. Kalnay, and T. Miyoshi, 2009: Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 523–533.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E., 1996: Predictability: A problem partly solved. Proc. ECMWF Seminar on Predictability, Vol. 1, Reading, United Kingdom, ECMWF, 1–18.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E., and K. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399–414.

    • Search Google Scholar
    • Export Citation

  • Mitchell, H. L., and P. L. Houtekamer, 2009: Ensemble Kalman filter configurations and their performance with the logistic map. Mon. Wea. Rev., 137, 4325–4343.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., 2005: Ensemble Kalman filter experiments with a primitive-equation global model. Ph.D. dissertation, University of Maryland, College Park, 197 pp.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., and S. Yamane, 2007: Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution. Mon. Wea. Rev., 135, 3841–3861.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., S. Yamane, and T. Enomoto, 2007: Localizing the error covariance by physical distances within a local ensemble transform Kalman filter (LETKF). SOLA, 3, 89–92, doi:10.2151/sola.2007-023.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., Y. Sato, and T. Kadowaki, 2010: Ensemble Kalman filter and 4D-Var intercomparison with the Japanese operational global analysis and prediction system. Mon. Wea. Rev., 138, 2846–2866.

    • Search Google Scholar
    • Export Citation

  • Miyoshi, T., and M. Kunii, 2011: The local ensemble transform Kalman filter with the Weather Research and Forecasting model: Experiments with real observations. Pure Appl. Geophys., in press.

    • Search Google Scholar
    • Export Citation

  • Molteni, F., 2003: Atmospheric simulations using a GCM with simplified physical parametrizations. I: Model climatology and variability in multi-decadal experiments. Climate Dyn., 20, 175–191.

    • Search Google Scholar
    • Export Citation

  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415–428.

  • Szunyogh, I., E. J. Kostelich, G. Gyarmati, D. J. Patil, B. R. Hunt, E. Kalnay, E. Ott, and J. A. Yorke, 2005: Assessing a local ensemble Kalman filter: Perfect model experiments with the National Centers for Environmental Prediction global model. Tellus, 57A, 528–545.

    • Search Google Scholar
    • Export Citation

  • Szunyogh, I., E. J. Kostelich, G. Gyarmati, E. Kalnay, B. R. Hunt, E. Ott, E. Satterfield, and J. A. Yorke, 2008: A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A, 113–130.

    • Search Google Scholar
    • Export Citation

  • Torn, R. D., 2010a: Ensemble-based sensitivity analysis applied to African Easterly Waves. Wea. Forecasting, 25, 61–78.

  • Torn, R. D., 2010b: Performance of a mesoscale ensemble Kalman filter (EnKF) during the NOAA high-resolution hurricane test. Mon. Wea. Rev., 138, 4375–4392.

    • Search Google Scholar
    • Export Citation

  • Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1924.

  • Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP Global Forecast System. Mon. Wea. Rev., 136, 463–482.

    • Search Google Scholar
    • Export Citation

  • Zhang, F., C. Snyder, and J. Sun, 2004: Impacts of initial estimate and observation availability on convective-scale data assimilation with ensemble Kalman filter. Mon. Wea. Rev., 132, 1238–1253.

    • Search Google Scholar
    • Export Citation
  • <sc>Fig</sc>. 1.
    View in gallery
    Fig. 1.

    Normalized PDFs from simple simulations with a realistic prior inflation variance of : (a) posterior PDFs (thick) and observation PDFs (thin), and (b) changes of PDFs from prior to posterior. Solid and dashed curves show PDFs of the Bayesian and Gaussian approaches, respectively. Since both approaches have nearly identical posterior PDFs, the dashed curves are obscured except for the thin dashed curve of (a).

  • <sc>Fig</sc>. 2.
    View in gallery
    Fig. 2.

    As in Fig. 1a, but with a large prior inflation variance of .

  • <sc>Fig</sc>. 3.
    View in gallery
    Fig. 3.

    As in Fig. 2, but using the true inflation in Eq. (9).

  • <sc>Fig</sc>. 4.
    View in gallery
    Fig. 4.

    Time series of the global-mean adaptive inflation parameters for two Lorenz model experiments under the perfect-model assumption with four prior inflation variance settings as shown in the legend.

  • <sc>Fig</sc>. 5.
    View in gallery
    Fig. 5.

    The 100-yr-mean analysis RMSE and ensemble spread (SPREAD) of the Lorenz model experiments with 1.5% fixed inflation and adaptive inflation using 4 prior inflation variance settings as shown in the legend, averaged over all grid points (global, J = 1, … , 40), observed “land” grid points (land, J = 1, … , 20), and unobserved “ocean” grid points (ocean, J = 21, … , 40).

  • <sc>Fig</sc>. 6.
    View in gallery
    Fig. 6.

    Improvements (%) of analysis RMSE, as shown in Fig. 5, of the adaptive inflation experiments relative to the 1.5% fixed inflation experiment.

  • <sc>Fig</sc>. 7.
    View in gallery
    Fig. 7.

    As in Fig. 4, but in the case with model errors. The dashed line is the same as the black line in Fig. 4, and is a reference to the perfect-model case.