• Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 28842903.

  • Anderson, J. L., 2003: A local least squares framework for ensemble filtering. Mon. Wea. Rev., 131, 634642.

  • Anderson, J. L., 2007: Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter. Physica D, 230, 99111.

    • Search Google Scholar
    • Export Citation
  • Bengtsson, T., C. Snyder, and D. Nychka, 2003: Toward a nonlinear ensemble filter for high-dimensional systems. J. Geophys. Res., 108 (D24), 8775, doi:10.1029/2002JD002900.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., B. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transformation Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420436.

    • Search Google Scholar
    • Export Citation
  • Brusdal, K., J. M. Brankart, G. Halberstadt, G. Evensen, P. Brasseur, P. J. van Leeuwen, E. Dombrowsky, and J. Verron, 2003: A demonstration of ensemble-based assimilation methods with a layered OGCM from the perspective of operational ocean forecasting systems. J. Mar. Syst., 40–41, 253289.

    • Search Google Scholar
    • Export Citation
  • Burgers, G., P. J. van Leeuwen, and G. Evensen, 1998: Analysis scheme in the ensemble Kalman filter. Mon. Wea. Rev., 126, 17191724.

  • Chorin, A., and X. Tu, 2009: Implicit sampling for particle filters. Proc. Natl. Acad. Sci. USA, 106, 17 24917 254.

  • Evensen, G., 1994: Sequential data assimilation with a non-linear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99 (C5), 10 14310 162.

    • Search Google Scholar
    • Export Citation
  • Evensen, G., 2003: The ensemble Kalman filter: Theoretical formulation and practical implementation. Ocean Dyn., 53, 343367.

  • Evensen, G., 2007: Data Assimilation: The Ensemble Kalman Filter. Springer, 279 pp.

  • Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723757.

    • Search Google Scholar
    • Export Citation
  • Gordon, N., D. Salmond, and A. Smith, 1993: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc.-F, 140, 107113.

    • Search Google Scholar
    • Export Citation
  • Hammersley, J. M., and D. C. Handscomb, 1965: Monte Carlo Methods. Methuen & Co., 178 pp.

  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796811.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123137.

    • Search Google Scholar
    • Export Citation
  • Künsch, H. R., 2005: Recursive Monte Carlo filters: Algorithms and theoretical analysis. Ann. Stat., 33, 19832021.

  • Lawson, W. G., and J. A. Hansen, 2004: Implications of stochastic and deterministic filters as ensemble-based data assimilation methods in varying regimes of error growth. Mon. Wea. Rev., 132, 19661981.

    • Search Google Scholar
    • Export Citation
  • Lei, J., P. Bickel, and C. Snyder, 2010: Comparison of ensemble Kalman filters under non-Gaussianity. Mon. Wea. Rev., 138, 12931306.

  • Lorenz, E. N., 1996: Predictability: A problem partly solved. Proc. Seminar on Predictability, Reading, Berkshire, United Kingdom, European Centre for Medium-Range Weather Forecasts, 1–18.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 2005: Designing chaotic models. J. Atmos. Sci., 62, 15741587.

  • Mitchell, H. L., and P. L. Houtekamer, 2009: Ensemble Kalman filter configurations and their performance with the logistic map. Mon. Wea. Rev., 137, 43254343.

    • Search Google Scholar
    • Export Citation
  • Musso, C., N. Oudjane, and F. Le Gland, 2001: Improving regularized particle filters. Sequential Monte Carlo Methods in Practice, A. Doucet, N. de Freitas, and N. Gordon, Eds., Springer-Verlag, 247–271.

    • Search Google Scholar
    • Export Citation
  • Nakano, S., G. Ueno, and T. Higuchi, 2007: Merging particle filter for sequential data assimilation. Nonlinear Processes Geophys., 14, 395408.

    • Search Google Scholar
    • Export Citation
  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415428.

  • Pham, D. T., 2001: Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Wea. Rev., 129, 11941207.

  • Sacher, W., and P. Bartello, 2008: Sampling errors in ensemble Kalman filtering. Part I: Theory. Mon. Wea. Rev., 136, 30353049.

  • Sakov, P., and P. R. Oke, 2008a: A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters. Tellus, 60A, 361371.

    • Search Google Scholar
    • Export Citation
  • Sakov, P., and P. R. Oke, 2008b: Implications of the form of the ensemble transformation in the ensemble square root filters. Mon. Wea. Rev., 136, 10421053.

    • Search Google Scholar
    • Export Citation
  • Snyder, C., T. Bengtsson, P. Bickel, and J. Anderson, 2008: Obstacles to high-dimensional particle filtering. Mon. Wea. Rev., 136, 46294640.

    • Search Google Scholar
    • Export Citation
  • van Leeuwen, P. J., 2003: A variance-minimizing filter for large-scale applications. Mon. Wea. Rev., 131, 20712084.

  • van Leeuwen, P. J., 2009: Particle filtering in geophysical systems. Mon. Wea. Rev., 137, 40894114.

  • van Leeuwen, P. J., 2010: Nonlinear data assimilation in geosciences: An extremely efficient particle filter. Quart. J. Roy. Meteor. Soc., 136, 19911999.

    • Search Google Scholar
    • Export Citation
  • Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 19131924.

  • Xiong, X., I. M. Navon, and B. Uzunoglu, 2006: A note on the particle filter with posterior Gaussian resampling. Tellus, 58A, 456460.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 4 4 4
PDF Downloads 3 3 3

A Moment Matching Ensemble Filter for Nonlinear Non-Gaussian Data Assimilation

View More View Less
  • 1 Department of Statistics, University of California, Berkeley, Berkeley, California
Restricted access

Abstract

The ensemble Kalman filter is now an important component of ensemble forecasting. While using the linear relationship between the observation and state variables makes it applicable for large systems, relying on linearity introduces nonnegligible bias since the true distribution will never be Gaussian. This paper analyzes the bias of the ensemble Kalman filter from a statistical perspective and proposes a debiasing method called the nonlinear ensemble adjustment filter. This new filter transforms the forecast ensemble in a statistically principled manner so that the updated ensemble has the desired mean and variance. It is also easily localizable and, hence, potentially useful for large systems. Its performance is demonstrated and compared with other Kalman filter and particle filter variants through various experiments on the Lorenz-63 and Lorenz-96 systems. The results show that the new filter is stable and accurate for challenging situations such as nonlinear, high-dimensional systems with sparse observations.

Corresponding author address: Jing Lei, Department of Statistics, UC Berkeley, 367 Evans Hall, Berkeley, CA 94720. E-mail: leij09@gmail.com

Abstract

The ensemble Kalman filter is now an important component of ensemble forecasting. While using the linear relationship between the observation and state variables makes it applicable for large systems, relying on linearity introduces nonnegligible bias since the true distribution will never be Gaussian. This paper analyzes the bias of the ensemble Kalman filter from a statistical perspective and proposes a debiasing method called the nonlinear ensemble adjustment filter. This new filter transforms the forecast ensemble in a statistically principled manner so that the updated ensemble has the desired mean and variance. It is also easily localizable and, hence, potentially useful for large systems. Its performance is demonstrated and compared with other Kalman filter and particle filter variants through various experiments on the Lorenz-63 and Lorenz-96 systems. The results show that the new filter is stable and accurate for challenging situations such as nonlinear, high-dimensional systems with sparse observations.

Corresponding author address: Jing Lei, Department of Statistics, UC Berkeley, 367 Evans Hall, Berkeley, CA 94720. E-mail: leij09@gmail.com
Save