Search Results
You are looking at 1 - 10 of 24 items for
- Author or Editor: Ibrahim Hoteit x
- Refine by Access: Content accessible to me x
Abstract
A robust ensemble filtering scheme based on the H ∞ filtering theory is proposed. The optimal H ∞ filter is derived by minimizing the supremum (or maximum) of a predefined cost function, a criterion different from the minimum variance used in the Kalman filter. By design, the H ∞ filter is more robust than the Kalman filter, in the sense that the estimation error in the H ∞ filter in general has a finite growth rate with respect to the uncertainties in assimilation, except for a special case that corresponds to the Kalman filter.
The original form of the H ∞ filter contains global constraints in time, which may be inconvenient for sequential data assimilation problems. Therefore a variant is introduced that solves some time-local constraints instead, and hence it is called the time-local H ∞ filter (TLHF). By analogy to the ensemble Kalman filter (EnKF), the concept of ensemble time-local H ∞ filter (EnTLHF) is also proposed. The general form of the EnTLHF is outlined, and some of its special cases are discussed. In particular, it is shown that an EnKF with certain covariance inflation is essentially an EnTLHF. In this sense, the EnTLHF provides a general framework for conducting covariance inflation in the EnKF-based methods. Some numerical examples are used to assess the relative robustness of the TLHF–EnTLHF in comparison with the corresponding KF–EnKF method.
Abstract
A robust ensemble filtering scheme based on the H ∞ filtering theory is proposed. The optimal H ∞ filter is derived by minimizing the supremum (or maximum) of a predefined cost function, a criterion different from the minimum variance used in the Kalman filter. By design, the H ∞ filter is more robust than the Kalman filter, in the sense that the estimation error in the H ∞ filter in general has a finite growth rate with respect to the uncertainties in assimilation, except for a special case that corresponds to the Kalman filter.
The original form of the H ∞ filter contains global constraints in time, which may be inconvenient for sequential data assimilation problems. Therefore a variant is introduced that solves some time-local constraints instead, and hence it is called the time-local H ∞ filter (TLHF). By analogy to the ensemble Kalman filter (EnKF), the concept of ensemble time-local H ∞ filter (EnTLHF) is also proposed. The general form of the EnTLHF is outlined, and some of its special cases are discussed. In particular, it is shown that an EnKF with certain covariance inflation is essentially an EnTLHF. In this sense, the EnTLHF provides a general framework for conducting covariance inflation in the EnKF-based methods. Some numerical examples are used to assess the relative robustness of the TLHF–EnTLHF in comparison with the corresponding KF–EnKF method.
Abstract
This article examines the influence of covariance inflation on the distance between the measured observation and the simulated (or predicted) observation with respect to the state estimate. In order for the aforementioned distance to be bounded in a certain interval, some sufficient conditions are derived, indicating that the covariance inflation factor should be bounded in a certain interval, and that the inflation bounds are related to the maximum and minimum eigenvalues of certain matrices. Implications of these analytic results are discussed, and a numerical experiment is presented to verify the validity of the analysis conducted.
Abstract
This article examines the influence of covariance inflation on the distance between the measured observation and the simulated (or predicted) observation with respect to the state estimate. In order for the aforementioned distance to be bounded in a certain interval, some sufficient conditions are derived, indicating that the covariance inflation factor should be bounded in a certain interval, and that the inflation bounds are related to the maximum and minimum eigenvalues of certain matrices. Implications of these analytic results are discussed, and a numerical experiment is presented to verify the validity of the analysis conducted.
Abstract
This study considers the data assimilation problem in coupled systems, which consists of two components (subsystems) interacting with each other through certain coupling terms. A straightforward way to tackle the assimilation problem in such systems is to concatenate the states of the subsystems into one augmented state vector, so that a standard ensemble Kalman filter (EnKF) can be directly applied. This work presents a divided state-space estimation strategy, in which data assimilation is carried out with respect to each individual subsystem, involving quantities from the subsystem itself and correlated quantities from other coupled subsystems. On top of the divided state-space estimation strategy, the authors also consider the possibility of running the subsystems separately. Combining these two ideas, a few variants of the EnKF are derived. The introduction of these variants is mainly inspired by the current status and challenges in coupled data assimilation problems and thus might be of interest from a practical point of view. Numerical experiments with a multiscale Lorenz 96 model are conducted to evaluate the performance of these variants against that of the conventional EnKF. In addition, specific for coupled data assimilation problems, two prototypes of extensions of the presented methods are also developed in order to achieve a trade-off between efficiency and accuracy.
Abstract
This study considers the data assimilation problem in coupled systems, which consists of two components (subsystems) interacting with each other through certain coupling terms. A straightforward way to tackle the assimilation problem in such systems is to concatenate the states of the subsystems into one augmented state vector, so that a standard ensemble Kalman filter (EnKF) can be directly applied. This work presents a divided state-space estimation strategy, in which data assimilation is carried out with respect to each individual subsystem, involving quantities from the subsystem itself and correlated quantities from other coupled subsystems. On top of the divided state-space estimation strategy, the authors also consider the possibility of running the subsystems separately. Combining these two ideas, a few variants of the EnKF are derived. The introduction of these variants is mainly inspired by the current status and challenges in coupled data assimilation problems and thus might be of interest from a practical point of view. Numerical experiments with a multiscale Lorenz 96 model are conducted to evaluate the performance of these variants against that of the conventional EnKF. In addition, specific for coupled data assimilation problems, two prototypes of extensions of the presented methods are also developed in order to achieve a trade-off between efficiency and accuracy.
Abstract
The ensemble Kalman filter (EnKF) is an efficient algorithm for many data assimilation problems. In certain circumstances, however, divergence of the EnKF might be spotted. In previous studies, the authors proposed an observation-space-based strategy, called residual nudging, to improve the stability of the EnKF when dealing with linear observation operators. The main idea behind residual nudging is to monitor and, if necessary, adjust the distances (misfits) between the real observations and the simulated ones of the state estimates, in the hope that by doing so one may be able to obtain better estimation accuracy.
In the present study, residual nudging is extended and modified in order to handle nonlinear observation operators. Such extension and modification result in an iterative filtering framework that, under suitable conditions, is able to achieve the objective of residual nudging for data assimilation problems with nonlinear observation operators. The 40-dimensional Lorenz-96 model is used to illustrate the performance of the iterative filter. Numerical results show that, while a normal EnKF may diverge with nonlinear observation operators, the proposed iterative filter remains stable and leads to reasonable estimation accuracy under various experimental settings.
Abstract
The ensemble Kalman filter (EnKF) is an efficient algorithm for many data assimilation problems. In certain circumstances, however, divergence of the EnKF might be spotted. In previous studies, the authors proposed an observation-space-based strategy, called residual nudging, to improve the stability of the EnKF when dealing with linear observation operators. The main idea behind residual nudging is to monitor and, if necessary, adjust the distances (misfits) between the real observations and the simulated ones of the state estimates, in the hope that by doing so one may be able to obtain better estimation accuracy.
In the present study, residual nudging is extended and modified in order to handle nonlinear observation operators. Such extension and modification result in an iterative filtering framework that, under suitable conditions, is able to achieve the objective of residual nudging for data assimilation problems with nonlinear observation operators. The 40-dimensional Lorenz-96 model is used to illustrate the performance of the iterative filter. Numerical results show that, while a normal EnKF may diverge with nonlinear observation operators, the proposed iterative filter remains stable and leads to reasonable estimation accuracy under various experimental settings.
Abstract
This work addresses the state–parameter filtering problem for dynamical systems with relatively large-dimensional state and low-dimensional parameters’ vector. A Bayesian filtering algorithm combining the strengths of the particle filter (PF) and the ensemble Kalman filter (EnKF) is proposed. At each assimilation cycle of the proposed EnKF–PF, the PF is first used to sample the parameters’ ensemble followed by the EnKF to compute the state ensemble conditional on the resulting parameters’ ensemble. The proposed scheme is expected to be more efficient than the traditional state augmentation techniques, which suffer from the curse of dimensionality and inconsistency that is particularly pronounced when the state is a strongly nonlinear function of the parameters. In the new scheme, the EnKF and PF interact via their ensembles’ members, in contrast with the recently introduced two-stage EnKF–PF (TS–EnKF–PF), which exchanges point estimates between EnKF and PF while requiring almost double the computational load. Numerical experiments are conducted with the Lorenz-96 model to assess the behavior of the proposed filter and to evaluate its performances against the joint PF, joint EnKF, and TS–EnKF–PF. Numerical results suggest that the EnKF–PF performs best in all tested scenarios. It was further found to be more robust, successfully estimating both state and parameters in different sensitivity experiments.
Abstract
This work addresses the state–parameter filtering problem for dynamical systems with relatively large-dimensional state and low-dimensional parameters’ vector. A Bayesian filtering algorithm combining the strengths of the particle filter (PF) and the ensemble Kalman filter (EnKF) is proposed. At each assimilation cycle of the proposed EnKF–PF, the PF is first used to sample the parameters’ ensemble followed by the EnKF to compute the state ensemble conditional on the resulting parameters’ ensemble. The proposed scheme is expected to be more efficient than the traditional state augmentation techniques, which suffer from the curse of dimensionality and inconsistency that is particularly pronounced when the state is a strongly nonlinear function of the parameters. In the new scheme, the EnKF and PF interact via their ensembles’ members, in contrast with the recently introduced two-stage EnKF–PF (TS–EnKF–PF), which exchanges point estimates between EnKF and PF while requiring almost double the computational load. Numerical experiments are conducted with the Lorenz-96 model to assess the behavior of the proposed filter and to evaluate its performances against the joint PF, joint EnKF, and TS–EnKF–PF. Numerical results suggest that the EnKF–PF performs best in all tested scenarios. It was further found to be more robust, successfully estimating both state and parameters in different sensitivity experiments.
Abstract
The Bayesian filtering problem for data assimilation is considered following the kernel-based ensemble Gaussian mixture filtering (EnGMF) approach introduced by Anderson and Anderson. In this approach, the posterior distribution of the system state is propagated with the model using the ensemble Monte Carlo method, providing a forecast ensemble that is then used to construct a prior Gaussian mixture (GM) based on the kernel density estimator. This results in two update steps: a Kalman filter (KF)-like update of the ensemble members and a particle filter (PF)-like update of the weights, followed by a resampling step to start a new forecast cycle. After formulating EnGMF for any observational operator, the influence of the bandwidth parameter of the kernel function on the covariance of the posterior distribution is analyzed. Then the focus is on two aspects: (i) the efficient implementation of EnGMF with (relatively) small ensembles, where a new deterministic resampling strategy is proposed preserving the first two moments of the posterior GM to limit the sampling error; and (ii) the analysis of the effect of the bandwidth parameter on contributions of KF and PF updates and on the weights variance. Numerical results using the Lorenz-96 model are presented to assess the behavior of EnGMF with deterministic resampling, study its sensitivity to different parameters and settings, and evaluate its performance against ensemble KFs. The proposed EnGMF approach with deterministic resampling suggests improved estimates in all tested scenarios, and is shown to require less localization and to be less sensitive to the choice of filtering parameters.
Abstract
The Bayesian filtering problem for data assimilation is considered following the kernel-based ensemble Gaussian mixture filtering (EnGMF) approach introduced by Anderson and Anderson. In this approach, the posterior distribution of the system state is propagated with the model using the ensemble Monte Carlo method, providing a forecast ensemble that is then used to construct a prior Gaussian mixture (GM) based on the kernel density estimator. This results in two update steps: a Kalman filter (KF)-like update of the ensemble members and a particle filter (PF)-like update of the weights, followed by a resampling step to start a new forecast cycle. After formulating EnGMF for any observational operator, the influence of the bandwidth parameter of the kernel function on the covariance of the posterior distribution is analyzed. Then the focus is on two aspects: (i) the efficient implementation of EnGMF with (relatively) small ensembles, where a new deterministic resampling strategy is proposed preserving the first two moments of the posterior GM to limit the sampling error; and (ii) the analysis of the effect of the bandwidth parameter on contributions of KF and PF updates and on the weights variance. Numerical results using the Lorenz-96 model are presented to assess the behavior of EnGMF with deterministic resampling, study its sensitivity to different parameters and settings, and evaluate its performance against ensemble KFs. The proposed EnGMF approach with deterministic resampling suggests improved estimates in all tested scenarios, and is shown to require less localization and to be less sensitive to the choice of filtering parameters.
Abstract
This paper investigates an approximation scheme of the optimal nonlinear Bayesian filter based on the Gaussian mixture representation of the state probability distribution function. The resulting filter is similar to the particle filter, but is different from it in that the standard weight-type correction in the particle filter is complemented by the Kalman-type correction with the associated covariance matrices in the Gaussian mixture. The authors show that this filter is an algorithm in between the Kalman filter and the particle filter, and therefore is referred to as the particle Kalman filter (PKF).
In the PKF, the solution of a nonlinear filtering problem is expressed as the weighted average of an “ensemble of Kalman filters” operating in parallel. Running an ensemble of Kalman filters is, however, computationally prohibitive for realistic atmospheric and oceanic data assimilation problems. For this reason, the authors consider the construction of the PKF through an “ensemble” of ensemble Kalman filters (EnKFs) instead, and call the implementation the particle EnKF (PEnKF). It is shown that different types of the EnKFs can be considered as special cases of the PEnKF. Similar to the situation in the particle filter, the authors also introduce a resampling step to the PEnKF in order to reduce the risk of weights collapse and improve the performance of the filter. Numerical experiments with the strongly nonlinear Lorenz-96 model are presented and discussed.
Abstract
This paper investigates an approximation scheme of the optimal nonlinear Bayesian filter based on the Gaussian mixture representation of the state probability distribution function. The resulting filter is similar to the particle filter, but is different from it in that the standard weight-type correction in the particle filter is complemented by the Kalman-type correction with the associated covariance matrices in the Gaussian mixture. The authors show that this filter is an algorithm in between the Kalman filter and the particle filter, and therefore is referred to as the particle Kalman filter (PKF).
In the PKF, the solution of a nonlinear filtering problem is expressed as the weighted average of an “ensemble of Kalman filters” operating in parallel. Running an ensemble of Kalman filters is, however, computationally prohibitive for realistic atmospheric and oceanic data assimilation problems. For this reason, the authors consider the construction of the PKF through an “ensemble” of ensemble Kalman filters (EnKFs) instead, and call the implementation the particle EnKF (PEnKF). It is shown that different types of the EnKFs can be considered as special cases of the PEnKF. Similar to the situation in the particle filter, the authors also introduce a resampling step to the PEnKF in order to reduce the risk of weights collapse and improve the performance of the filter. Numerical experiments with the strongly nonlinear Lorenz-96 model are presented and discussed.
Abstract
The Red Sea is a narrow, elongated basin that is more than 2000 km long. This deceivingly simple structure offers very interesting challenges for wind and wave modeling, not easily, if ever, found elsewhere. Using standard meteorological products and local wind and wave models, this study explores how well the general and unusual wind and wave patterns of the Red Sea could be reproduced. The authors obtain the best results using two rather opposite approaches: the high-resolution Weather Research Forecasting (WRF) local model and the slightly enhanced surface winds from the global European Centre for Medium-Range Weather Forecasts model. The reasons why these two approaches produce the best results and the implications on wave modeling in the Red Sea are discussed. The unusual wind and wave patterns in the Red Sea suggest that the currently available wave model source functions may not properly represent the evolution of local fields. However, within limits, the WAVEWATCH III wave model, based on Janssen’s and also Ardhuin’s wave model physics, provides very reasonable results in many cases. The authors also discuss these findings and outline related future work.
Abstract
The Red Sea is a narrow, elongated basin that is more than 2000 km long. This deceivingly simple structure offers very interesting challenges for wind and wave modeling, not easily, if ever, found elsewhere. Using standard meteorological products and local wind and wave models, this study explores how well the general and unusual wind and wave patterns of the Red Sea could be reproduced. The authors obtain the best results using two rather opposite approaches: the high-resolution Weather Research Forecasting (WRF) local model and the slightly enhanced surface winds from the global European Centre for Medium-Range Weather Forecasts model. The reasons why these two approaches produce the best results and the implications on wave modeling in the Red Sea are discussed. The unusual wind and wave patterns in the Red Sea suggest that the currently available wave model source functions may not properly represent the evolution of local fields. However, within limits, the WAVEWATCH III wave model, based on Janssen’s and also Ardhuin’s wave model physics, provides very reasonable results in many cases. The authors also discuss these findings and outline related future work.
Abstract
A new approach is proposed to address the background covariance limitations arising from undersampled ensembles and unaccounted model errors in the ensemble Kalman filter (EnKF). The method enhances the representativeness of the EnKF ensemble by augmenting it with new members chosen adaptively to add missing information that prevents the EnKF from fully fitting the data to the ensemble. The vectors to be added are obtained by back projecting the residuals of the observation misfits from the EnKF analysis step onto the state space. The back projection is done using an optimal interpolation (OI) scheme based on an estimated covariance of the subspace missing from the ensemble. In the experiments reported here, the OI uses a preselected stationary background covariance matrix, as in the hybrid EnKF–three-dimensional variational data assimilation (3DVAR) approach, but the resulting correction is included as a new ensemble member instead of being added to all existing ensemble members.
The adaptive approach is tested with the Lorenz-96 model. The hybrid EnKF–3DVAR is used as a benchmark to evaluate the performance of the adaptive approach. Assimilation experiments suggest that the new adaptive scheme significantly improves the EnKF behavior when it suffers from small size ensembles and neglected model errors. It was further found to be competitive with the hybrid EnKF–3DVAR approach, depending on ensemble size and data coverage.
Abstract
A new approach is proposed to address the background covariance limitations arising from undersampled ensembles and unaccounted model errors in the ensemble Kalman filter (EnKF). The method enhances the representativeness of the EnKF ensemble by augmenting it with new members chosen adaptively to add missing information that prevents the EnKF from fully fitting the data to the ensemble. The vectors to be added are obtained by back projecting the residuals of the observation misfits from the EnKF analysis step onto the state space. The back projection is done using an optimal interpolation (OI) scheme based on an estimated covariance of the subspace missing from the ensemble. In the experiments reported here, the OI uses a preselected stationary background covariance matrix, as in the hybrid EnKF–three-dimensional variational data assimilation (3DVAR) approach, but the resulting correction is included as a new ensemble member instead of being added to all existing ensemble members.
The adaptive approach is tested with the Lorenz-96 model. The hybrid EnKF–3DVAR is used as a benchmark to evaluate the performance of the adaptive approach. Assimilation experiments suggest that the new adaptive scheme significantly improves the EnKF behavior when it suffers from small size ensembles and neglected model errors. It was further found to be competitive with the hybrid EnKF–3DVAR approach, depending on ensemble size and data coverage.
Abstract
Current ensemble-based Kalman filter (EnKF) algorithms are not robust to gross observation errors caused by technical or human errors during the data collection process. In this paper, the authors consider two types of gross observational errors, additive statistical outliers and innovation outliers, and introduce a method to make EnKF robust to gross observation errors. Using both a one-dimensional linear system of dynamics and a 40-variable Lorenz model, the performance of the proposed robust ensemble Kalman filter (REnKF) was tested and it was found that the new approach greatly improves the performance of the filter in the presence of gross observation errors and leads to only a modest loss of accuracy with clean, outlier-free, observations.
Abstract
Current ensemble-based Kalman filter (EnKF) algorithms are not robust to gross observation errors caused by technical or human errors during the data collection process. In this paper, the authors consider two types of gross observational errors, additive statistical outliers and innovation outliers, and introduce a method to make EnKF robust to gross observation errors. Using both a one-dimensional linear system of dynamics and a 40-variable Lorenz model, the performance of the proposed robust ensemble Kalman filter (REnKF) was tested and it was found that the new approach greatly improves the performance of the filter in the presence of gross observation errors and leads to only a modest loss of accuracy with clean, outlier-free, observations.
