• Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129 , 28842903.

  • Anderson, J. L., , and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127 , 27412758.

    • Search Google Scholar
    • Export Citation
  • Burgers, G., , P. J. van Leeuwen, , and G. Evensen, 1998: On the analysis scheme of the ensemble Kalman filter. Mon. Wea. Rev., 126 , 17191724.

    • Search Google Scholar
    • Export Citation
  • de Ruijter, W. P. M., , A. Biastoch, , S. S. Drijfhout, , J. R. E. Lutjeharms, , R. P. Matano, , T. Pichevin, , P. J. van Leeuwen, , and W. Weijer, 1999a: Indian–Atlantic interocean exchange: Dynamics, estimation and impact. J. Geophys. Res., 104 , 2088520910.

    • Search Google Scholar
    • Export Citation
  • de Ruijter, W. P. M., , P. J. van Leeuwen, , and J. R. E. Lutjeharms, 1999b: Generation and evolution of Natal pulses: Solitary meanders in the Agulhas Current. J. Phys. Oceanogr., 29 , 30433055.

    • Search Google Scholar
    • Export Citation
  • Doucet, A., , N. de Freitas, , and N. J. Gordon, Eds.,. 2001: Sequential Monte Carlo Methods in Practice. Springer, 581 pp.

  • Drazin, P. G., , and R. S. Johnson, 1989: Solitons: An Introduction. Cambridge University Press, 226 pp.

  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99 (C5) 1014310162.

    • Search Google Scholar
    • Export Citation
  • Evensen, G., , and P. J. van Leeuwen, 2000: An ensemble Kalman smoother for nonlinear dynamics. Mon. Wea. Rev., 128 , 18521867.

  • Gordon, N. J., , D. N. Salmond, , and A. F. M. Smith, 1993: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEEE Proc., 140F , 107113.

    • Search Google Scholar
    • Export Citation
  • Heemink, A. W., , M. Verlaan, , and A. J. Segers, 2001: Variance reduced ensemble Kalman filtering. Mon. Wea. Rev., 129 , 17181728.

  • Houtekamer, P. L., , and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126 , 796811.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., , and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129 , 123137.

    • Search Google Scholar
    • Export Citation
  • Jazwinski, A. H., 1970: Stochastic Processes and Filtering Theory. Academic Press, 376 pp.

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20 , 130141.

  • Lutjeharms, J. R. E., , and H. R. Roberts, 1988: The Natal pulse: An extreme transient on the Agulhas Current. J. Geophys. Res., 93 , 631645.

    • Search Google Scholar
    • Export Citation
  • Miller, R. N., , E. F. Carter, , and S. T. Blue, 1999: Data assimilation into nonlinear stochastic models using the ensemble Kalman filter with a quasi-geostrophic model. Tellus, 51A , 167194.

    • Search Google Scholar
    • Export Citation
  • Pham, T. P., 2001: Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Wea. Rev., 129 , 11941207.

  • Rubin, D. B., 1988: Using the SIR algorithm to simulate posterior distributions. Bayesian Statistics 3, J. M. Bernado et al., Eds., Oxford University Press, 395–402.

    • Search Google Scholar
    • Export Citation
  • Schouten, M. W., , W. P. M. de Ruijter, , and P. J. van Leeuwen, 2000: Translation, decay and splitting of Agulhas rings in the southeastern Atlantic Ocean. J. Geophys. Res., 105 , 2191321925.

    • Search Google Scholar
    • Export Citation
  • Shannon, C. E., 1948: A mathematical theory of communication. Bell Syst. Tech. J., 27 , 379423.

  • Silverman, B. W., 1986: Density Estimation for Statistics and Data Analysis. Chapman and Hall, 175 pp.

  • van der Vaart, P. C. F., , and W. P. M. de Ruijter, 2001: Stability of western boundary currents with an application to pulse-like behavior of the Agulhas Current. J. Phys. Oceanogr., 31 , 26252644.

    • Search Google Scholar
    • Export Citation
  • van Leeuwen, P. J., 1999: The time-mean circulation in the Agulhas region determined with the ensemble smoother. J. Geophys. Res., 104 , 13931404.

    • Search Google Scholar
    • Export Citation
  • van Leeuwen, P. J., 2001: An ensemble smoother with error estimates. Mon. Wea. Rev., 129 , 709728.

  • van Leeuwen, P. J., , and G. Evensen, 1996: Data assimilation and inverse problems in a probabilistic formulation. Mon. Wea. Rev., 124 , 28982913.

    • Search Google Scholar
    • Export Citation
  • van Leeuwen, P. J., , W. P. M. de Ruijter, , and J. R. E. Lutjeharms, 2000: Natal pulses and the formation of Agulhas Rings. J. Geophys. Res., 105 , 64256436.

    • Search Google Scholar
    • Export Citation
  • Verlaan, M., , and A. W. Heemink, 1997: Tidal flow forecasting using reduced rank square root filters. Stochastic Hydrol. Hydraul., 11 , 349368.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 131 131 33
PDF Downloads 75 75 7

A Variance-Minimizing Filter for Large-Scale Applications

View More View Less
  • 1 IMAU, Utrecht University, Utrecht, Netherlands
© Get Permissions Rent on DeepDyve
Restricted access

Abstract

A truly variance-minimizing filter is introduced and its performance is demonstrated with the Korteweg–DeVries (KdV) equation and with a multilayer quasigeostrophic model of the ocean area around South Africa. It is recalled that Kalman-like filters are not variance minimizing for nonlinear model dynamics and that four-dimensional variational data assimilation (4DVAR)-like methods relying on perfect model dynamics have difficulty with providing error estimates. The new method does not have these drawbacks. In fact, it combines advantages from both methods in that it does provide error estimates while automatically having balanced states after analysis, without extra computations. It is based on ensemble or Monte Carlo integrations to simulate the probability density of the model evolution. When observations are available, the so-called importance resampling algorithm is applied. From Bayes's theorem it follows that each ensemble member receives a new weight dependent on its “distance” to the observations. Because the weights are strongly varying, a resampling of the ensemble is necessary. This resampling is done such that members with high weights are duplicated according to their weights, while low-weight members are largely ignored. In passing, it is noted that data assimilation is not an inverse problem by nature, although it can be formulated that way. Also, it is shown that the posterior variance can be larger than the prior if the usual Gaussian framework is set aside. However, in the examples presented here, the entropy of the probability densities is decreasing. The application to the ocean area around South Africa, governed by strongly nonlinear dynamics, shows that the method is working satisfactorily. The strong and weak points of the method are discussed and possible improvements are proposed.

Corresponding author address: P. J. van Leeuwen, IMAU, Utrecht University, P.O. Box 80005, 3508 TA Utrecht, Netherlands. Email: leeuwen@phys.uu.nl

Abstract

A truly variance-minimizing filter is introduced and its performance is demonstrated with the Korteweg–DeVries (KdV) equation and with a multilayer quasigeostrophic model of the ocean area around South Africa. It is recalled that Kalman-like filters are not variance minimizing for nonlinear model dynamics and that four-dimensional variational data assimilation (4DVAR)-like methods relying on perfect model dynamics have difficulty with providing error estimates. The new method does not have these drawbacks. In fact, it combines advantages from both methods in that it does provide error estimates while automatically having balanced states after analysis, without extra computations. It is based on ensemble or Monte Carlo integrations to simulate the probability density of the model evolution. When observations are available, the so-called importance resampling algorithm is applied. From Bayes's theorem it follows that each ensemble member receives a new weight dependent on its “distance” to the observations. Because the weights are strongly varying, a resampling of the ensemble is necessary. This resampling is done such that members with high weights are duplicated according to their weights, while low-weight members are largely ignored. In passing, it is noted that data assimilation is not an inverse problem by nature, although it can be formulated that way. Also, it is shown that the posterior variance can be larger than the prior if the usual Gaussian framework is set aside. However, in the examples presented here, the entropy of the probability densities is decreasing. The application to the ocean area around South Africa, governed by strongly nonlinear dynamics, shows that the method is working satisfactorily. The strong and weak points of the method are discussed and possible improvements are proposed.

Corresponding author address: P. J. van Leeuwen, IMAU, Utrecht University, P.O. Box 80005, 3508 TA Utrecht, Netherlands. Email: leeuwen@phys.uu.nl

Save