Performance Bounds for Particle Filters Using the Optimal Proposal

Chris Snyder National Center for Atmospheric Research,* Boulder, Colorado

Search for other papers by Chris Snyder in
Current site
Google Scholar
PubMed
Close
,
Thomas Bengtsson Genentech, San Francisco, California

Search for other papers by Thomas Bengtsson in
Current site
Google Scholar
PubMed
Close
, and
Mathias Morzfeld Department of Mathematics, University of California, Berkeley, Berkeley, California

Search for other papers by Mathias Morzfeld in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Particle filters may suffer from degeneracy of the particle weights. For the simplest “bootstrap” filter, it is known that avoiding degeneracy in large systems requires that the ensemble size must increase exponentially with the variance of the observation log-likelihood. The present article shows first that a similar result applies to particle filters using sequential importance sampling and the optimal proposal distribution and, second, that the optimal proposal yields minimal degeneracy when compared to any other proposal distribution that depends only on the previous state and the most recent observations. Thus, the optimal proposal provides performance bounds for filters using sequential importance sampling and any such proposal. An example with independent and identically distributed degrees of freedom illustrates both the need for exponentially large ensemble size with the optimal proposal as the system dimension increases and the potentially dramatic advantages of the optimal proposal relative to simpler proposals. Those advantages depend crucially on the magnitude of the system noise.

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

Corresponding author address: Chris Snyder, NCAR, P.O. Box 3000, Boulder, CO 80307. E-mail: chriss@ucar.edu

Abstract

Particle filters may suffer from degeneracy of the particle weights. For the simplest “bootstrap” filter, it is known that avoiding degeneracy in large systems requires that the ensemble size must increase exponentially with the variance of the observation log-likelihood. The present article shows first that a similar result applies to particle filters using sequential importance sampling and the optimal proposal distribution and, second, that the optimal proposal yields minimal degeneracy when compared to any other proposal distribution that depends only on the previous state and the most recent observations. Thus, the optimal proposal provides performance bounds for filters using sequential importance sampling and any such proposal. An example with independent and identically distributed degrees of freedom illustrates both the need for exponentially large ensemble size with the optimal proposal as the system dimension increases and the potentially dramatic advantages of the optimal proposal relative to simpler proposals. Those advantages depend crucially on the magnitude of the system noise.

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

Corresponding author address: Chris Snyder, NCAR, P.O. Box 3000, Boulder, CO 80307. E-mail: chriss@ucar.edu
Save
  • Ades, M., and P. J. Van Leeuwen, 2015: The equivalent-weights particle filter in a high-dimensional system. Quart. J. Roy. Meteor. Soc., 141, 484503, doi:10.1002/qj.2370.

    • Search Google Scholar
    • Export Citation
  • Arulampalam, M., S. Maskell, N. Gordon, and T. Clapp, 2002: A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process., 50, 174188, doi:10.1109/78.978374.

    • Search Google Scholar
    • Export Citation
  • Bengtsson, T., C. Snyder, and D. Nychka, 2003: Toward a nonlinear ensemble filter for high-dimensional systems. J. Geophys. Res., 108, 8775, doi:10.1029/2002JD002900.

    • Search Google Scholar
    • Export Citation
  • Bengtsson, T., P. Bickel, and B. Li, 2008: Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems. Probability and Statistics: Essays in Honor of David A. Freedman, D. Nolan and T. Speeds, Eds., Vol. 2, Institute of Mathematical Statistics, 316–334, doi:10.1214/193940307000000518.

  • Bickel, P., B. Li, and T. Bengtsson, 2008: Sharp failure rates for the bootstrap particle filter in high dimensions. Pushing the Limits of Contemporarly Statistics: Contributions in Honor of Jayanta K. Ghosh, B. Clarke and S. Ghosal, Eds., Vol. 3, Institute of Mathematical Statistics, 318–329, doi:10.1214/074921708000000228.

  • Bocquet, M., C. A. Pires, and L. Wu, 2010: Beyond Gaussian statistical modeling in geophysical data assimilation. Mon. Wea. Rev., 138, 29973023, doi:10.1175/2010MWR3164.1.

    • Search Google Scholar
    • Export Citation
  • Boffetta, G., A. Celani, A. Crisanti, and A. Vulpiani, 1997: Predictability in two-dimensional decaying turbulence. Phys. Fluids, 9, 724734, doi:10.1063/1.869227.

    • Search Google Scholar
    • Export Citation
  • Chorin, A. J., and X. Tu, 2009: Implicit sampling for particle filters. Proc. Natl. Acad. Sci. USA, 106, 17 24917 254, doi:10.1073/pnas.0909196106.

    • Search Google Scholar
    • Export Citation
  • Chorin, A. J., and M. Morzfeld, 2013: Conditions for successful data assimilation. J. Geophys. Res. Atmos., 118, 11 52211 533, doi:10.1002/2013JD019838.

    • Search Google Scholar
    • Export Citation
  • Chorin, A. J., M. Morzfeld, and X. Tu, 2013: A survey of implicit particle filters for data assimilation. State Space Models: Applications in Economics and Finance, Y. Zeng and S. Wu, Eds., Springer, 63–88.

  • Doucet, A., S. Godsill, and C. Andrieu, 2000: Sequential Monte-Carlo methods for Bayesian filtering. Stat. Comput., 10, 197208, doi:10.1023/A:1008935410038.

    • Search Google Scholar
    • Export Citation
  • Doucet, A., M. Briers, and S. Sénécal, 2006: Efficient block sampling strategies for sequential Monte Carlo methods. J. Comput. Graph. Stat., 15, 693711, doi:10.1198/106186006X142744.

    • Search Google Scholar
    • Export Citation
  • Gordon, N. J., D. J. Salmond, and A. F. M. Smith, 1993: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc.-F Radar Signal Process., 140, 107113.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 27762790, doi:10.1175/1520-0493(2001)129<2776:DDFOBE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796811, doi:10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123137, doi:10.1175/1520-0493(2001)129<0123:ASEKFF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112126, doi:10.1016/j.physd.2006.11.008.

    • Search Google Scholar
    • Export Citation
  • Klaas, M., A. Doucet, and N. D. Frietas, 2005: Towards practical Monte Carlo: The marginal particle filter. Proc. 21st Annual Conf. on Uncertainty in Artificial Intelligence, Arlington, VA, Association for Uncertainty in Artificial Intelligence, 308315.

  • Kong, A., J. Liu, and W. Wong, 1994: Sequential imputations and Bayesian missing data problems. J. Amer. Stat. Assoc., 89, 278288, doi:10.1080/01621459.1994.10476469.

    • Search Google Scholar
    • Export Citation
  • Lin, M., R. Chen, and J. S. Liu, 2013: Lookahead strategies for sequential Monte Carlo. Stat. Sci., 28, 6994, doi:10.1214/12-STS401.

  • Liu, J. S., and R. Chen, 1998: Sequential Monte Carlo methods for dynamic systems. J. Amer. Stat. Assoc., 93, 10321044, doi:10.1080/01621459.1998.10473765.

    • Search Google Scholar
    • Export Citation
  • Metref, S., E. Cosme, C. Snyder, and P. Brasseur, 2014: A non-Gaussian analysis scheme using rank histograms for ensemble data assimilation. Nonlinear Processes Geophys., 21, 869885, doi:10.5194/npg-21-869-2014.

    • Search Google Scholar
    • Export Citation
  • Morzfeld, M., X. Tu, E. Atkins, and A. J. Chorin, 2012: A random map implementation of implicit filters. J. Comput. Phys., 231, 20492066, doi:10.1016/j.jcp.2011.11.022.

    • Search Google Scholar
    • Export Citation
  • Nakano, S., 2014: Hybrid algorithm of ensemble transform and importance sampling for assimilation of non-Gaussian observations. Tellus, 66A, 21429, doi:10.3402/tellusa.v66.21429.

    • Search Google Scholar
    • Export Citation
  • Papadakis, N., E. Mémin, A. Cuzol, and N. Gengembre, 2010: Data assimilation with the weighted ensemble Kalman filter. Tellus, 62A, 673697, doi: 10.1111/j.1600-0870.2010.00461.x.

    • Search Google Scholar
    • Export Citation
  • Parlett, B. N., 1998: The Symmetric Eigenvalue Problem. SIAM, xxiv + 391 pp.

  • Rabier, F., A. McNally, E. Andersson, P. Courtier, P. Undén, J. Eyre, A. Hollingsworth, and F. Bouttier, 1998: The ECMWF implementation of three-dimensional variational assimilation (3D-Var). Part II: Structure functions. Quart. J. Roy. Meteor. Soc., 124, 18091829, doi:10.1002/qj.49712455003.

    • Search Google Scholar
    • Export Citation
  • Reich, S., 2013: A nonparametric ensemble transform method for Bayesian inference. SIAM J. Sci. Comput., 35, A2013A2024, doi:10.1137/130907367.

    • Search Google Scholar
    • Export Citation
  • Rotunno, R., and C. Snyder, 2008: A generalization of Lorenz’s model for the predictability of flows with many scales of motion. J. Atmos. Sci., 65, 10631076, doi:10.1175/2007JAS2449.1.

    • Search Google Scholar
    • Export Citation
  • Snyder, C., 2012: Particle filters, the “optimal” proposal and high-dimensional systems. ECMWF Seminar on Data Assimilation for Atmosphere and Ocean, Shinfield, United Kingdom, ECMWF, 161–170.

  • Snyder, C., T. Bengtsson, P. Bickel, and J. Anderson, 2008: Obstacles to high-dimensional particle filtering. Mon. Wea. Rev., 136, 46294640, doi:10.1175/2008MWR2529.1.

    • Search Google Scholar
    • Export Citation
  • van Leeuwen, P. J., 2009: Particle filtering in geophysical systems. Mon. Wea. Rev., 137, 40894114, doi:10.1175/2009MWR2835.1.

  • van Leeuwen, P. J., 2010: Nonlinear data assimilation in geosciences: An extremely efficient particle filter. Quart. J. Roy. Meteor. Soc., 136, 19911999, doi:10.1002/qj.699.

    • Search Google Scholar
    • Export Citation
  • Weir, B., R. N. Miller, and Y. H. Spitz, 2013: A potential implicit particle method for high-dimensional systems. Nonlinear Processes Geophys., 20, 10471060, doi:10.5194/npg-20-1047-2013.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1013 231 6
PDF Downloads 651 173 4