Browse
Abstract
The spread of an ensemble of weather predictions initialized from an ensemble Kalman filter may grow slowly relative to other methods for initializing ensemble predictions, degrading its skill. Several possible causes of the slow spread growth were evaluated in perfect- and imperfect-model experiments with a two-layer primitive equation spectral model of the atmosphere. The causes examined were the covariance localization, the additive noise used to stabilize the assimilation method and parameterize the system error, and the model error itself. In these experiments, the flow-independent additive noise was the biggest factor in constraining spread growth. Preevolving additive noise perturbations were tested as a way to make the additive noise more flow dependent. This modestly improved the data assimilation and ensemble predictions, both in the two-layer model results and in a brief test of the assimilation of real observations into a global multilevel spectral primitive equation model. More generally, these results suggest that methods for treating model error in ensemble Kalman filters that greatly reduce the flow dependency of the background-error covariances may increase the filter analysis error and decrease the rate of forecast spread growth.
Abstract
The spread of an ensemble of weather predictions initialized from an ensemble Kalman filter may grow slowly relative to other methods for initializing ensemble predictions, degrading its skill. Several possible causes of the slow spread growth were evaluated in perfect- and imperfect-model experiments with a two-layer primitive equation spectral model of the atmosphere. The causes examined were the covariance localization, the additive noise used to stabilize the assimilation method and parameterize the system error, and the model error itself. In these experiments, the flow-independent additive noise was the biggest factor in constraining spread growth. Preevolving additive noise perturbations were tested as a way to make the additive noise more flow dependent. This modestly improved the data assimilation and ensemble predictions, both in the two-layer model results and in a brief test of the assimilation of real observations into a global multilevel spectral primitive equation model. More generally, these results suggest that methods for treating model error in ensemble Kalman filters that greatly reduce the flow dependency of the background-error covariances may increase the filter analysis error and decrease the rate of forecast spread growth.
Abstract
The use of local spatial averaging to estimate and validate background error covariances has received increasing attention recently, in particular in the context of variational data assimilation for global numerical weather prediction. First, theoretical and experimental results are presented to examine spatial structures of sampling noise and signal in ensemble-based variance fields in this context. They indicate that sampling noise tends to be relatively small scale, compared to the signal of interest. This difference in spatial structure motivates the use of spatial averaging techniques.
Based on the usual linear estimation theory, it is shown how this information can be taken into account in order to calculate and apply an objective spatial filter. This kind of approach can also be used to compare and validate ensemble-based variances with innovation-based variances. The use of spatial averaging is even more important for innovation-based variances because local innovations correspond to single error realizations.
Similar ideas can be considered for the estimation of correlation functions. The spatial structures of sampling noise and signal in correlation length scale fields suggest that space-averaging techniques could also be applied to correlation functions. The use of wavelets for this purpose is presented in particular. Connections with related approaches in different contexts such as ensemble Kalman filters and probabilistic forecasting are also discussed.
Abstract
The use of local spatial averaging to estimate and validate background error covariances has received increasing attention recently, in particular in the context of variational data assimilation for global numerical weather prediction. First, theoretical and experimental results are presented to examine spatial structures of sampling noise and signal in ensemble-based variance fields in this context. They indicate that sampling noise tends to be relatively small scale, compared to the signal of interest. This difference in spatial structure motivates the use of spatial averaging techniques.
Based on the usual linear estimation theory, it is shown how this information can be taken into account in order to calculate and apply an objective spatial filter. This kind of approach can also be used to compare and validate ensemble-based variances with innovation-based variances. The use of spatial averaging is even more important for innovation-based variances because local innovations correspond to single error realizations.
Similar ideas can be considered for the estimation of correlation functions. The spatial structures of sampling noise and signal in correlation length scale fields suggest that space-averaging techniques could also be applied to correlation functions. The use of wavelets for this purpose is presented in particular. Connections with related approaches in different contexts such as ensemble Kalman filters and probabilistic forecasting are also discussed.
Abstract
This study examines the sensitivity of global ensemble forecasts to the use of different approaches for specifying both the initial ensemble mean and perturbations. The current operational ensemble prediction system of the Meteorological Service of Canada uses the ensemble Kalman filter (EnKF) to define both the ensemble mean and perturbations. To evaluate the impact of different approaches for obtaining the initial ensemble perturbations, the operational EnKF approach is compared with using either no initial perturbations or perturbations obtained using singular vectors (SVs). The SVs are computed using the (dry) total-energy norm with a 48-h optimization time interval. Random linear combinations of 60 SVs are computed for each of three regions. Next, the impact of replacing the initial ensemble mean, currently the EnKF ensemble mean analysis, with the higher-resolution operational four-dimensional variational data assimilation (4D-Var) analysis is evaluated. For this comparison, perturbations are provided by the EnKF. All experiments are performed over two-month periods during both the boreal summer and winter using a system very similar to the global ensemble prediction system that became operational on 10 July 2007. Relative to the operational configuration that relies on the EnKF, the use of SVs to compute initial perturbations produces small, but statistically significant differences in probabilistic forecast scores in favor of the EnKF both in the tropics and, for a limited set of forecast lead times, in the summer hemisphere extratropics, whereas the results are very similar in the winter hemisphere extratropics. Both approaches lead to significantly better ensemble forecasts than with no initial perturbations, though results are quite similar in the tropics when using SVs and no perturbations. The use of an initial-time norm that does not include information on analysis uncertainty and the lack of linearized moist processes in the calculation of the SVs are two factors that limit the quality of the resulting SV-based ensemble forecasts. Relative to the operational configuration, use of the 4D-Var analysis to specify the initial ensemble mean results in improved probabilistic forecast scores during the boreal summer period in the southern extratropics and tropics, but a near-neutral impact otherwise.
Abstract
This study examines the sensitivity of global ensemble forecasts to the use of different approaches for specifying both the initial ensemble mean and perturbations. The current operational ensemble prediction system of the Meteorological Service of Canada uses the ensemble Kalman filter (EnKF) to define both the ensemble mean and perturbations. To evaluate the impact of different approaches for obtaining the initial ensemble perturbations, the operational EnKF approach is compared with using either no initial perturbations or perturbations obtained using singular vectors (SVs). The SVs are computed using the (dry) total-energy norm with a 48-h optimization time interval. Random linear combinations of 60 SVs are computed for each of three regions. Next, the impact of replacing the initial ensemble mean, currently the EnKF ensemble mean analysis, with the higher-resolution operational four-dimensional variational data assimilation (4D-Var) analysis is evaluated. For this comparison, perturbations are provided by the EnKF. All experiments are performed over two-month periods during both the boreal summer and winter using a system very similar to the global ensemble prediction system that became operational on 10 July 2007. Relative to the operational configuration that relies on the EnKF, the use of SVs to compute initial perturbations produces small, but statistically significant differences in probabilistic forecast scores in favor of the EnKF both in the tropics and, for a limited set of forecast lead times, in the summer hemisphere extratropics, whereas the results are very similar in the winter hemisphere extratropics. Both approaches lead to significantly better ensemble forecasts than with no initial perturbations, though results are quite similar in the tropics when using SVs and no perturbations. The use of an initial-time norm that does not include information on analysis uncertainty and the lack of linearized moist processes in the calculation of the SVs are two factors that limit the quality of the resulting SV-based ensemble forecasts. Relative to the operational configuration, use of the 4D-Var analysis to specify the initial ensemble mean results in improved probabilistic forecast scores during the boreal summer period in the southern extratropics and tropics, but a near-neutral impact otherwise.
Abstract
In data assimilation, observations are combined with the dynamics to get an estimate of the actual state of a natural system. The knowledge of the dynamics, under the form of a model, is unavoidably incomplete and model error affects the prediction accuracy together with the error in the initial condition. The variational assimilation theory provides a framework to deal with model error along with the uncertainties coming from other sources entering the state estimation. Nevertheless, even if the problem is formulated as Gaussian, accounting for model error requires the estimation of its covariances and correlations, which are difficult to estimate in practice, in particular because of the large system dimension and the lack of enough observations. Model error has been therefore either neglected or assumed to be an uncorrelated noise. In the present work, an approach to account for a deterministic model error in the variational assimilation is presented. Equations for its correlations are first derived along with an approximation suitable for practical applications. Based on these considerations, a new four-dimensional variational data assimilation (4DVar) weak-constraint algorithm is formulated and tested in the context of a linear unstable system and of the three-component Lorenz model, which has chaotic dynamics. The results demonstrate that this approach is superior in skill to both the strong-constraint and a weak-constraint variational assimilation that employs the uncorrelated noise model error assumption.
Abstract
In data assimilation, observations are combined with the dynamics to get an estimate of the actual state of a natural system. The knowledge of the dynamics, under the form of a model, is unavoidably incomplete and model error affects the prediction accuracy together with the error in the initial condition. The variational assimilation theory provides a framework to deal with model error along with the uncertainties coming from other sources entering the state estimation. Nevertheless, even if the problem is formulated as Gaussian, accounting for model error requires the estimation of its covariances and correlations, which are difficult to estimate in practice, in particular because of the large system dimension and the lack of enough observations. Model error has been therefore either neglected or assumed to be an uncorrelated noise. In the present work, an approach to account for a deterministic model error in the variational assimilation is presented. Equations for its correlations are first derived along with an approximation suitable for practical applications. Based on these considerations, a new four-dimensional variational data assimilation (4DVar) weak-constraint algorithm is formulated and tested in the context of a linear unstable system and of the three-component Lorenz model, which has chaotic dynamics. The results demonstrate that this approach is superior in skill to both the strong-constraint and a weak-constraint variational assimilation that employs the uncorrelated noise model error assumption.
Abstract
This review discusses recent advances in geophysical data assimilation beyond Gaussian statistical modeling, in the fields of meteorology, oceanography, as well as atmospheric chemistry. The non-Gaussian features are stressed rather than the nonlinearity of the dynamical models, although both aspects are entangled. Ideas recently proposed to deal with these non-Gaussian issues, in order to improve the state or parameter estimation, are emphasized.
The general Bayesian solution to the estimation problem and the techniques to solve it are first presented, as well as the obstacles that hinder their use in high-dimensional and complex systems. Approximations to the Bayesian solution relying on Gaussian, or on second-order moment closure, have been wholly adopted in geophysical data assimilation (e.g., Kalman filters and quadratic variational solutions). Yet, nonlinear and non-Gaussian effects remain. They essentially originate in the nonlinear models and in the non-Gaussian priors. How these effects are handled within algorithms based on Gaussian assumptions is then described. Statistical tools that can diagnose them and measure deviations from Gaussianity are recalled.
The following advanced techniques that seek to handle the estimation problem beyond Gaussianity are reviewed: maximum entropy filter, Gaussian anamorphosis, non-Gaussian priors, particle filter with an ensemble Kalman filter as a proposal distribution, maximum entropy on the mean, or strictly Bayesian inferences for large linear models, etc. Several ideas are illustrated with recent or original examples that possess some features of high-dimensional systems. Many of the new approaches are well understood only in special cases and have difficulties that remain to be circumvented. Some of the suggested approaches are quite promising, and sometimes already successful for moderately large though specific geophysical applications. Hints are given as to where progress might come from.
Abstract
This review discusses recent advances in geophysical data assimilation beyond Gaussian statistical modeling, in the fields of meteorology, oceanography, as well as atmospheric chemistry. The non-Gaussian features are stressed rather than the nonlinearity of the dynamical models, although both aspects are entangled. Ideas recently proposed to deal with these non-Gaussian issues, in order to improve the state or parameter estimation, are emphasized.
The general Bayesian solution to the estimation problem and the techniques to solve it are first presented, as well as the obstacles that hinder their use in high-dimensional and complex systems. Approximations to the Bayesian solution relying on Gaussian, or on second-order moment closure, have been wholly adopted in geophysical data assimilation (e.g., Kalman filters and quadratic variational solutions). Yet, nonlinear and non-Gaussian effects remain. They essentially originate in the nonlinear models and in the non-Gaussian priors. How these effects are handled within algorithms based on Gaussian assumptions is then described. Statistical tools that can diagnose them and measure deviations from Gaussianity are recalled.
The following advanced techniques that seek to handle the estimation problem beyond Gaussianity are reviewed: maximum entropy filter, Gaussian anamorphosis, non-Gaussian priors, particle filter with an ensemble Kalman filter as a proposal distribution, maximum entropy on the mean, or strictly Bayesian inferences for large linear models, etc. Several ideas are illustrated with recent or original examples that possess some features of high-dimensional systems. Many of the new approaches are well understood only in special cases and have difficulties that remain to be circumvented. Some of the suggested approaches are quite promising, and sometimes already successful for moderately large though specific geophysical applications. Hints are given as to where progress might come from.
Abstract
The local ensemble transform Kalman filter (LETKF) is implemented and assessed with the experimental operational system at the Japanese Meteorological Agency (JMA). This paper describes the details of the LETKF system and verification of deterministic forecast skill. JMA has been operating a four-dimensional variational data assimilation (4D-Var) system for global numerical weather prediction since 2005. The main purpose of this study is to make a reasonable comparison between the LETKF and the operational 4D-Var.
Several forecast–analysis cycle experiments are performed to find sensitivity to the parameters of the LETKF. The difference between additive and multiplicative error covariance inflation schemes is investigated. Moreover, an adaptive bias correction method for satellite radiance observations is proposed and implemented, so that the LETKF is equipped with functionality similar to the variational bias correction used in the operational 4D-Var. Finally, the LETKF is compared with the operational 4D-Var. Although forecast verification scores of the two systems relative to each system’s own analyses and to radiosonde observations show some disagreement, the overall conclusion indicates that the LETKF and 4D-Var have essentially comparable performance. The LETKF shows larger temperature bias in the lower troposphere mainly over the ocean, which is related to a well-known JMA model bias that plays an important role in the significant degradation of the forecast scores in the SH. The LETKF suffers less of a performance degradation than 4D-Var in the absence of satellite radiance assimilation. This suggests that better treatment of satellite radiances would be important in future developments toward operational use of the LETKF. Developing both LETKF and 4D-Var at JMA has shown significant benefits by the synergistic effect and is the recommended strategy for the moment.
Abstract
The local ensemble transform Kalman filter (LETKF) is implemented and assessed with the experimental operational system at the Japanese Meteorological Agency (JMA). This paper describes the details of the LETKF system and verification of deterministic forecast skill. JMA has been operating a four-dimensional variational data assimilation (4D-Var) system for global numerical weather prediction since 2005. The main purpose of this study is to make a reasonable comparison between the LETKF and the operational 4D-Var.
Several forecast–analysis cycle experiments are performed to find sensitivity to the parameters of the LETKF. The difference between additive and multiplicative error covariance inflation schemes is investigated. Moreover, an adaptive bias correction method for satellite radiance observations is proposed and implemented, so that the LETKF is equipped with functionality similar to the variational bias correction used in the operational 4D-Var. Finally, the LETKF is compared with the operational 4D-Var. Although forecast verification scores of the two systems relative to each system’s own analyses and to radiosonde observations show some disagreement, the overall conclusion indicates that the LETKF and 4D-Var have essentially comparable performance. The LETKF shows larger temperature bias in the lower troposphere mainly over the ocean, which is related to a well-known JMA model bias that plays an important role in the significant degradation of the forecast scores in the SH. The LETKF suffers less of a performance degradation than 4D-Var in the absence of satellite radiance assimilation. This suggests that better treatment of satellite radiances would be important in future developments toward operational use of the LETKF. Developing both LETKF and 4D-Var at JMA has shown significant benefits by the synergistic effect and is the recommended strategy for the moment.
Abstract
An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of global deterministic NWP. In an EnKF experiment having the same spatial resolution as the inner loop in the four-dimensional variational data assimilation system (4D-Var), the mean of each analysis ensemble is used to initialize the higher-resolution deterministic forecasts. Five different variational data assimilation experiments are also conducted. These include both 4D-Var and 3D-Var (with first guess at appropriate time) experiments using either (i) prescribed background-error covariances similar to those used operationally, which are static in time and include horizontally homogeneous and isotropic correlations; or (ii) flow-dependent covariances computed from the EnKF background ensembles with spatial covariance localization applied. The fifth variational data assimilation experiment is a new approach called the Ensemble-4D-Var (En-4D-Var). This approach uses 4D flow-dependent background-error covariances estimated from EnKF ensembles to produce a 4D analysis without the need for tangent-linear or adjoint versions of the forecast model. In this first part of a two-part paper, results from a series of idealized assimilation experiments are presented. In these experiments, only a single observation or vertical profile of observations is assimilated to explore the impact of various fundamental differences among the EnKF and the various variational data assimilation approaches considered. In particular, differences in the application of covariance localization in the EnKF and variational approaches are shown to have a significant impact on the assimilation of satellite radiance observations. The results also demonstrate that 4D-Var and the EnKF can both produce similar 4D background-error covariances within a 6-h assimilation window. In the second part, results from medium-range deterministic forecasts for the study period of February 2007 are presented for the EnKF and the five variational data assimilation approaches considered.
Abstract
An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of global deterministic NWP. In an EnKF experiment having the same spatial resolution as the inner loop in the four-dimensional variational data assimilation system (4D-Var), the mean of each analysis ensemble is used to initialize the higher-resolution deterministic forecasts. Five different variational data assimilation experiments are also conducted. These include both 4D-Var and 3D-Var (with first guess at appropriate time) experiments using either (i) prescribed background-error covariances similar to those used operationally, which are static in time and include horizontally homogeneous and isotropic correlations; or (ii) flow-dependent covariances computed from the EnKF background ensembles with spatial covariance localization applied. The fifth variational data assimilation experiment is a new approach called the Ensemble-4D-Var (En-4D-Var). This approach uses 4D flow-dependent background-error covariances estimated from EnKF ensembles to produce a 4D analysis without the need for tangent-linear or adjoint versions of the forecast model. In this first part of a two-part paper, results from a series of idealized assimilation experiments are presented. In these experiments, only a single observation or vertical profile of observations is assimilated to explore the impact of various fundamental differences among the EnKF and the various variational data assimilation approaches considered. In particular, differences in the application of covariance localization in the EnKF and variational approaches are shown to have a significant impact on the assimilation of satellite radiance observations. The results also demonstrate that 4D-Var and the EnKF can both produce similar 4D background-error covariances within a 6-h assimilation window. In the second part, results from medium-range deterministic forecasts for the study period of February 2007 are presented for the EnKF and the five variational data assimilation approaches considered.
Abstract
An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of producing global deterministic numerical weather forecasts. Five different variational data assimilation approaches are considered including four-dimensional variational data assimilation (4D-Var) and three-dimensional variational data assimilation (3D-Var) with first guess at the appropriate time (3D-FGAT). Also included among these is a new approach, called Ensemble-4D-Var (En-4D-Var), that uses 4D ensemble background-error covariances from the EnKF. A description of the experimental configurations and results from single-observation experiments are presented in the first part of this two-part paper. The present paper focuses on results from medium-range deterministic forecasts initialized with analyses from the EnKF and the five variational data assimilation approaches for the period of February 2007. All experiments assimilate exactly the same full set of meteorological observations and use the same configuration of the forecast model to produce global deterministic medium-range forecasts.
The quality of forecasts in the short (medium) range obtained by using the EnKF ensemble mean analysis is slightly degraded (improved) in the extratropics relative to using the 4D-Var analysis with background-error covariances similar to those used operationally. The use of the EnKF flow-dependent error covariances in the variational system (4D-Var or 3D-FGAT) leads to large (modest) forecast improvements in the southern extratropics (tropics) as compared with using covariances similar to the operational system (a gain of up to 9 h at day 5). The En-4D-Var approach leads to (i) either improved or similar forecast quality when compared with the 4D-Var experiment similar to the currently operational system, (ii) slightly worse forecast quality when compared with the 4D-Var experiment with EnKF error covariances, and (iii) generally similar forecast quality when compared with the EnKF experiment.
Abstract
An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of producing global deterministic numerical weather forecasts. Five different variational data assimilation approaches are considered including four-dimensional variational data assimilation (4D-Var) and three-dimensional variational data assimilation (3D-Var) with first guess at the appropriate time (3D-FGAT). Also included among these is a new approach, called Ensemble-4D-Var (En-4D-Var), that uses 4D ensemble background-error covariances from the EnKF. A description of the experimental configurations and results from single-observation experiments are presented in the first part of this two-part paper. The present paper focuses on results from medium-range deterministic forecasts initialized with analyses from the EnKF and the five variational data assimilation approaches for the period of February 2007. All experiments assimilate exactly the same full set of meteorological observations and use the same configuration of the forecast model to produce global deterministic medium-range forecasts.
The quality of forecasts in the short (medium) range obtained by using the EnKF ensemble mean analysis is slightly degraded (improved) in the extratropics relative to using the 4D-Var analysis with background-error covariances similar to those used operationally. The use of the EnKF flow-dependent error covariances in the variational system (4D-Var or 3D-FGAT) leads to large (modest) forecast improvements in the southern extratropics (tropics) as compared with using covariances similar to the operational system (a gain of up to 9 h at day 5). The En-4D-Var approach leads to (i) either improved or similar forecast quality when compared with the 4D-Var experiment similar to the currently operational system, (ii) slightly worse forecast quality when compared with the 4D-Var experiment with EnKF error covariances, and (iii) generally similar forecast quality when compared with the EnKF experiment.
Abstract
The differences in the balance characteristics between dry and precipitation areas in estimated short-term forecast error fields are investigated. The motivation is to see if dry and precipitation areas need to be treated differently in atmospheric data assimilation systems. Using an ensemble of lagged forecast differences, it is shown that perturbations are, on average, farther away from geostrophic balance over precipitation areas than over dry areas and that the deviation from geostrophic balance is proportional to the intensity of precipitation. Following these results, the authors investigate whether some improvements in the coupling between mass and rotational wind increments over precipitation areas can be achieved by using only the precipitation points within an ensemble of estimated forecast errors to construct a so-called diabatic balance operator by linear regression. Comparisons with a traditional approach to construct balance operators by linear regression show that the new approach leads to a gradually significant improvement (related to the intensity of the diabatic processes) of the accuracy of the coupling over precipitation areas as judged from an ensemble of lagged forecast differences. Results from a series of simplified data assimilation experiments show that the new balance operators can produce analysis increments that are substantially different from those associated with the traditional balance operator, particularly for observations located in the lower atmosphere. Issues concerning the implementation of this new approach in a full-fledged analysis system are briefly discussed but their investigations are left for a following study.
Abstract
The differences in the balance characteristics between dry and precipitation areas in estimated short-term forecast error fields are investigated. The motivation is to see if dry and precipitation areas need to be treated differently in atmospheric data assimilation systems. Using an ensemble of lagged forecast differences, it is shown that perturbations are, on average, farther away from geostrophic balance over precipitation areas than over dry areas and that the deviation from geostrophic balance is proportional to the intensity of precipitation. Following these results, the authors investigate whether some improvements in the coupling between mass and rotational wind increments over precipitation areas can be achieved by using only the precipitation points within an ensemble of estimated forecast errors to construct a so-called diabatic balance operator by linear regression. Comparisons with a traditional approach to construct balance operators by linear regression show that the new approach leads to a gradually significant improvement (related to the intensity of the diabatic processes) of the accuracy of the coupling over precipitation areas as judged from an ensemble of lagged forecast differences. Results from a series of simplified data assimilation experiments show that the new balance operators can produce analysis increments that are substantially different from those associated with the traditional balance operator, particularly for observations located in the lower atmosphere. Issues concerning the implementation of this new approach in a full-fledged analysis system are briefly discussed but their investigations are left for a following study.
