Large Sample Properties of the Adaptive Gaussian Mixture Filter

Andreas S. Stordal IRIS, Stavanger, Norway

Search for other papers by Andreas S. Stordal in
Current site
Google Scholar
PubMed
Close
and
Hans A. Karlsen Department of Mathematics, University of Bergen, Bergen, Norway

Search for other papers by Hans A. Karlsen in
Current site
Google Scholar
PubMed
Close
Restricted access

We are aware of a technical issue preventing figures and tables from showing in some newly published articles in the full-text HTML view.
While we are resolving the problem, please use the online PDF version of these articles to view figures and tables.

Abstract

In high-dimensional dynamic systems, standard Monte Carlo techniques that asymptotically reproduce the posterior distribution are computationally too expensive. Alternative sampling strategies are usually applied and among these the ensemble Kalman filter (EnKF) is perhaps the most popular. However, the EnKF suffers from severe bias if the model under consideration is far from linear. Another class of sequential Monte Carlo methods is kernel-based Gaussian mixture filters, which reduce the bias but maintain the robustness of the EnKF. Although many hybrid methods have been introduced in recent years, not many have been analyzed theoretically. Here it is shown that the recently proposed adaptive Gaussian mixture filter can be formulated in a rigorous Bayesian framework and that the algorithm can be generalized to a broader class of interpolated kernel filters. Two parameters—the bandwidth of the kernel and a weight interpolation factor—determine the filter performance. The new formulation of the filter includes particle filters, EnKF, and kernel-based Gaussian mixture filters as special cases. Techniques from particle filter literature are used to calculate the asymptotic bias of the filter as a function of the parameters and to derive a central limit theorem. The asymptotic theory is then used to determine the parameters as a function of the sample size in a robust way such that the error norm vanishes asymptotically, whereas the normalized error is sample independent and bounded. The parameter choice is tested on the Lorenz 63 model, where it is shown that the error is smaller or equal to the EnKF and the optimal particle filter for a varying sample size.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Andreas S. Stordal, andreas.s.stordal@iris.no

Abstract

In high-dimensional dynamic systems, standard Monte Carlo techniques that asymptotically reproduce the posterior distribution are computationally too expensive. Alternative sampling strategies are usually applied and among these the ensemble Kalman filter (EnKF) is perhaps the most popular. However, the EnKF suffers from severe bias if the model under consideration is far from linear. Another class of sequential Monte Carlo methods is kernel-based Gaussian mixture filters, which reduce the bias but maintain the robustness of the EnKF. Although many hybrid methods have been introduced in recent years, not many have been analyzed theoretically. Here it is shown that the recently proposed adaptive Gaussian mixture filter can be formulated in a rigorous Bayesian framework and that the algorithm can be generalized to a broader class of interpolated kernel filters. Two parameters—the bandwidth of the kernel and a weight interpolation factor—determine the filter performance. The new formulation of the filter includes particle filters, EnKF, and kernel-based Gaussian mixture filters as special cases. Techniques from particle filter literature are used to calculate the asymptotic bias of the filter as a function of the parameters and to derive a central limit theorem. The asymptotic theory is then used to determine the parameters as a function of the sample size in a robust way such that the error norm vanishes asymptotically, whereas the normalized error is sample independent and bounded. The parameter choice is tested on the Lorenz 63 model, where it is shown that the error is smaller or equal to the EnKF and the optimal particle filter for a varying sample size.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Andreas S. Stordal, andreas.s.stordal@iris.no
Save
  • Aanonsen, S. I., G. Nævdal, D. S. Oliver, A. C. Reynolds, and B. Vallès, 2009: The ensemble Kalman filter in reservoir engineering—A review. Soc. Pet. Eng. J., 14, 393412, doi:10.2118/117274-PA.

    • Search Google Scholar
    • Export Citation
  • Andrieu, C., A. Doucet, and R. Holenstein, 2010: Particle Markov chain Monte Carlo methods. J. Roy. Stat. Soc., 72B, 269342, doi:10.1111/j.1467-9868.2009.00736.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Apte, A., M. Hairer, A. M. Stuart, and J. Voss, 2007: Sampling the posterior: An approach to non-Gaussian data assimilation. Physica D, 230, 5064, doi:10.1016/j.physd.2006.06.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bengtsson, T., C. Snyder, and D. Nychka, 2003: Toward a nonlinear ensemble filter for high-dimensional systems. J. Geophys. Res., 108, 8775, doi:10.1029/2002JD002900.

    • Search Google Scholar
    • Export Citation
  • Bengtsson, T., P. Bickel, and B. Li, 2008: Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems. Probability and Statistics: Essays in Honor of David A. Freedman, D. A. Nolan and T. Speed, Eds., Institute of Mathematical Statistics Collections, Vol. 2, Institute of Mathematical Statistics, 316334, doi:10.1214/193940307000000518.

    • Search Google Scholar
    • Export Citation
  • Bertino, L., G. Evensen, and H. Wackernagel, 2003: Sequential data assimilation techniques in oceanography. Int. Stat. Rev., 71, 223241, doi:10.1111/j.1751-5823.2003.tb00194.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Beskos, A., D. Crisan, and A. Jasra, 2014: On the stability of sequential Monte Carlo methods in high dimensions. Ann. Appl. Probab., 24, 13961445, doi:10.1214/13-AAP951.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bray, M., E. Koller-Meier, and L. Van Gool, 2007: Smart particle filtering for high-dimensional tracking. Comput. Vision Image Understanding, 106, 116129, doi:10.1016/j.cviu.2005.09.013.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chang, Y., A. S. Stordal, and R. Valestrand, 2016: Integrated work flow of preserving facies realism in history matching: Application to the Brugge field. Soc. Pet. Eng. J., 21, 14131424, doi:10.2118/179732-PA.

    • Search Google Scholar
    • Export Citation
  • Chen, R., and J. S. Liu, 2000: Mixture Kalman filters. J. Roy. Stat. Soc., 62B, 493508, doi:10.1111/1467-9868.00246.

  • Csilléry, K., M. G. Blum, O. E. Gaggiotti, and O. François, 2010: Approximate Bayesian Computation (ABC) in practice. Trends Ecol. Evol., 25, 410418, doi:10.1016/j.tree.2010.04.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Del Moral, P., and J. Jacod, 2001: Interacting particle filtering with discrete observations. Sequential Monte Carlo Methods in Practice, A. Doucet, N. de Freitas, and N. Gordon, Eds., Statistics for Engineering and Information Science, Springer, 43–75.

    • Crossref
    • Export Citation
  • Del Moral, P., and J. Jacod, 2002: The Monte-Carlo method for filtering with discrete-time observations: Central limit theorems. Numerical Methods and Stochastics, T. J. Lyons and T. Stephenson Salisbury, Eds., Fields Institute Communications, Vol. 34, American Mathematical Society, 29–53, doi:10.1093/acprof:oso/9780198506485.001.0001.

    • Crossref
    • Export Citation
  • Doucet, A., N. de Freitas, K. Murphy, and S. Russell, 2000a: Rao-Blackwellised particle filtering for dynamic Bayesian networks. UAI’00: Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, C. Boutilier and M. Goldszmidt, Eds., Morgan Kaufmann Publishers Inc., 176183.

  • Doucet, A., S. Godsill, and C. Andrieu, 2000b: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput., 10, 197208, doi:10.1023/A:1008935410038.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doucet, A., N. de Freitas, and N. Gordon, 2001: An introduction to sequential Monte Carlo methods. Sequential Monte Carlo Methods in Practice, A. Doucet, N. de Freitas, and N. Gordon, Eds., Statistics for Engineering and Information Science, Springer, 3–14.

    • Crossref
    • Export Citation
  • Evensen, G., 2007: Data Assimilation: The Ensemble Kalman Filter. Springer, 279 pp.

  • Eyink, G. L., and S. Kim, 2006: A maximum entropy method for particle filtering. J. Stat. Phys., 123, 10711128, doi:10.1007/s10955-006-9124-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frei, M., and H. R. Künsch, 2012: Sequential state and observation noise covariance estimation using combined ensemble Kalman and particle filters. Mon. Wea. Rev., 140, 14761495, doi:10.1175/MWR-D-10-05088.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frei, M., and H. R. Künsch, 2013: Bridging the ensemble Kalman and particle filters. Biometrika, 100, 781800, doi:10.1093/biomet/ast020.

  • Gneiting, T., A. E. Raftery, A. H. Westveld, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133, 10981118, doi:10.1175/MWR2904.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Godsill, S., and T. Clapp, 2001: Improvement strategies for Monte Carlo particle filters. Sequential Monte Carlo Methods in Practice, A. Doucet, N. de Freitas, and N. Gordon, Eds., Statistics for Engineering and Information Science, Springer, 139–158.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gordon, N., 1994: Bayesian methods for tracking. Ph.D. thesis, Imperial College London.

  • Gustafsson, F., F. Gunnarsson, N. Bergman, U. Forssell, J. Jansson, R. Karlsson, and P.-J. Nordlund, 2002: Particle filters for positioning, navigation, and tracking. IEEE Trans. Signal Processing, 50, 425437, doi:10.1109/78.978396.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hendricks Franssen, H., and W. Kinzelbach, 2008: Real-time groundwater flow modeling with the ensemble Kalman filter: Joint estimation of states and parameters and the filter inbreeding problem. Water Resour. Res., 44, W09408, doi:10.1029/2007WR006505.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoteit, I., D. T. Pham, G. Triantafyllou, and G. Korres, 2008: A new approximate solution of the optimal nonlinear filter for data assimilation in meteorology and oceanography. Mon. Wea. Rev., 136, 317334, doi:10.1175/2007MWR1927.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., H. L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133, 604620, doi:10.1175/MWR-2864.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Keppenne, C. L., and M. M. Rienecker, 2002: Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model. Mon. Wea. Rev., 130, 29512965, doi:10.1175/1520-0493(2002)130<2951:ITOAMP>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kong, A., J. Liu, and W. Wong, 1994: Sequential imputations and Bayesian missing data problems. J. Amer. Stat. Assoc., 89, 278288, doi:10.1080/01621459.1994.10476469.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Künsch, H. R., 2005: Recursive Monte Carlo filters: Algorithms and theoretical analysis. Ann. Stat., 33, 19832021, doi:10.1214/009053605000000426.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Le Gland, F., V. Monbet, and V.-D. Tran, 2009: Large sample asymptotics for the ensemble Kalman filter. INRIA Research Rep. RR-7014, 25 pp. [Available online at http://hal.inria.fr/inria-00409060/PDF/RR-7014.pdf.]

  • Lei, J., and P. Bickel, 2011: A moment matching ensemble filter for nonlinear non-Gaussian data assimilation. Mon. Wea. Rev., 139, 39643973, doi:10.1175/2011MWR3553.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, B., B. Ait-El-Fquih, and I. Hoteit, 2016: Efficient kernel-based ensemble Gaussian mixture filtering. Mon. Wea. Rev., 144, 781800, doi:10.1175/MWR-D-14-00292.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWP—A comparison with 4D-Var. Quart. J. Roy. Meteor. Soc., 129, 31833203, doi:10.1256/qj.02.132.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130141, doi:10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mandel, J., and J. Beezley, 2009: An ensemble Kalman-particle predictor-corrector filter for non-Gaussian data assimilation. Computational Science—ICCS 2009, G. Allen et al., Eds., Lecture Notes in Computer Science, Vol. 5545, Springer, 470–478, doi:10.1007/978-3-642-01973-9_53.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meyn, S., and R. L. Tweedie, 2009: Markov Chains and Stochastic Stability. 2nd ed. Cambridge University Press, 594 pp., doi:10.1017/CBO9780511626630.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122, 73119, doi:10.1002/qj.49712252905.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Musso, C., N. Oudjane, and F. Le Gland, 2001: Improving regularised particle filters. Sequential Monte Carlo Methods in Practice, A. Doucet, N. de Freitas, and N. Gordon, Eds., Statistics for Engineering and Information Science, Springer, 247–271.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Papadakis, N., E. Mémin, A. Cuzol, and N. Gengembre, 2010: Data assimilation with the weighted ensemble Kalman filter. Tellus, 62A, 673697, doi:10.3402/tellusa.v62i5.15716.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pitt, M. K., and N. Shephard, 1999: Filtering via simulation: Auxiliary particle filters. J. Amer. Stat. Assoc., 94, 590599, doi:10.1080/01621459.1999.10474153.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reich, S., 2012: A Gaussian-mixture ensemble transform filter. Quart. J. Roy. Meteor. Soc., 138, 222233, doi:10.1002/qj.898.

  • Reichle, R. H., D. B. McLaughlin, and D. Entekhabi, 2002: Hydrologic data assimilation with the ensemble Kalman filter. Mon. Wea. Rev., 130, 103114, doi:10.1175/1520-0493(2002)130<0103:HDAWTE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Robert, C. P., and G. Casella, 2004: Monte Carlo Statistical Methods. 2nd ed. Springer Texts in Statistics, Springer-Verlag, 649 pp., doi:10.1007/978-1-4757-4145-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sætrom, J., and H. Omre, 2013: Uncertainty quantification in the ensemble Kalman filter. Scand. J. Stat., 40, 868885, doi:10.1111/sjos.12039.

  • Sakov, P., and P. R. Oke, 2008: Implications of the form of the ensemble transformation in the ensemble square root filters. Mon. Wea. Rev., 136, 10421053, doi:10.1175/2007MWR2021.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sebacher, B., A. Stordal, and R. Hanea, 2015: Bridging multipoint statistics and truncated Gaussian fields for improved estimation of channelized reservoirs with ensemble methods. Comput. Geosci., 19, 341369, doi:10.1007/s10596-014-9466-3.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Silverman, B. W., 1986: Density Estimation for Statistics and Data Analysis. Chapman and Hall, 176 pp.

  • Sondergaard, T., and P. F. Lermusiaux, 2013: Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations. Part I: Theory and scheme. Mon. Wea. Rev., 141, 17371760, doi:10.1175/MWR-D-11-00295.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stordal, A. S., 2015: Iterative Bayesian inversion with Gaussian mixtures: Finite sample implementation and large sample asymptotics. Comput. Geosci., 19, 115, doi:10.1007/s10596-014-9444-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stordal, A. S., and R. Lorentzen, 2014: An iterative version of the adaptive Gaussian mixture filter. Comput. Geosci., 18, 579595, doi:10.1007/s10596-014-9402-6.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stordal, A. S., and A. H. Elsheikh, 2015: Iterative ensemble smoothers in the annealed importance sampling framework. Adv. Water Resour., 86, 231239, doi:10.1016/j.advwatres.2015.09.030.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stordal, A. S., H. A. Karlsen, G. Nævdal, H. J. Skaug, and B. Vallès, 2011: Bridging the ensemble Kalman filter and particle filters: The adaptive Gaussian mixture filter. Comput. Geosci., 15, 293305, doi:10.1007/s10596-010-9207-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stordal, A. S., H. A. Karlsen, G. Nævdal, D. S. Oliver, and H. J. Skaug, 2012: Filtering with state space localized Kalman gain. Physica D, 241, 11231135, doi:10.1016/j.physd.2012.03.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tagade, P., H. Seybold, and S. Ravela, 2014: Mixture ensembles for data assimilation in dynamic data-driven environmental systems. Procedia Comput. Sci., 29, 12661276, doi:10.1016/j.procs.2014.05.114.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125, 32973319, doi:10.1175/1520-0493(1997)125<3297:EFANAT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Valestrand, R., G. Nævdal, and A. Stordal, 2012a: Evaluation of EnKF and variants on the PUNQ-S3 case. Oil Gas Sci. Technol., 67, 841845, doi:10.2516/ogst/2012044.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Valestrand, R., G. Nævdal, and A. Stordal, 2012b: Refined adaptive Gaussian mixture filter—Application on a real field case. Proc. SPE EUROPEC/74th European Association of Geoscientists and Engineers (EAGE) Conf. and Exhibition, Copenhagen, Denmark, Society of Petroleum Engineers, SPE-154479-MS, doi:10.2118/154479-MS.

    • Crossref
    • Export Citation
  • van Leeuwen, P. J., 2009: Particle filtering in geophysical systems. Mon. Wea. Rev., 137, 40894114, doi:10.1175/2009MWR2835.1.

  • van Leeuwen, P. J., 2010: Nonlinear data assimilation in geosciences: An extremely efficient particle filter. Quart. J. Roy. Meteor. Soc., 136, 19911999, doi:10.1002/qj.699.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Verlaan, M., A. Zijderveld, H. de Vries, and J. Kroos, 2005: Operational storm surge forecasting in the Netherlands: Developments in the last decade. Philos. Trans. Roy. Soc. London, 363A, 14411453, doi:10.1098/rsta.2005.1578.

    • Search Google Scholar
    • Export Citation
  • Vlassis, N., B. Terwijn, and B. Kröse, 2002: Auxiliary particle filter robot localization from high-dimensional sensor observations. Proceedings: 2002 IEEE International Conference on Robotics and Automation, Vol. 1, IEEE, 7–12, 10.1109/ROBOT.2002.1013331.

    • Search Google Scholar
    • Export Citation
  • Xiong, X., I. M. Navon, and B. Uzunoglu, 2006: A note on the particle filter with posterior Gaussian resampling. Tellus, 58A, 456460, doi:10.1111/j.1600-0870.2006.00185.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 477 220 62
PDF Downloads 195 46 4