Editorial Type:
Article
Article Type:
Research Article

Data Assimilation with Gaussian Mixture Models Using the Dynamically Orthogonal Field Equations. Part I: Theory and Scheme

Thomas Sondergaard Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts

Search for other papers by Thomas Sondergaard in
Current site
Google Scholar
PubMed Close
and
Pierre F. J. Lermusiaux Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts

Search for other papers by Pierre F. J. Lermusiaux in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Jun 2013
DOI:
https://doi.org/10.1175/MWR-D-11-00295.1
Page(s):
1737–1760
Restricted access

Abstract

This work introduces and derives an efficient, data-driven assimilation scheme, focused on a time-dependent stochastic subspace that respects nonlinear dynamics and captures non-Gaussian statistics as it occurs. The motivation is to obtain a filter that is applicable to realistic geophysical applications, but that also rigorously utilizes the governing dynamical equations with information theory and learning theory for efficient Bayesian data assimilation. Building on the foundations of classical filters, the underlying theory and algorithmic implementation of the new filter are developed and derived. The stochastic Dynamically Orthogonal (DO) field equations and their adaptive stochastic subspace are employed to predict prior probabilities for the full dynamical state, effectively approximating the Fokker–Planck equation. At assimilation times, the DO realizations are fit to semiparametric Gaussian Mixture Models (GMMs) using the Expectation-Maximization algorithm and the Bayesian Information Criterion. Bayes’s law is then efficiently carried out analytically within the evolving stochastic subspace. The resulting GMM-DO filter is illustrated in a very simple example. Variations of the GMM-DO filter are also provided along with comparisons with related schemes.

Corresponding author address: Pierre F. J. Lermusiaux, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139. E-mail: pierrel@mit.edu

Abstract

This work introduces and derives an efficient, data-driven assimilation scheme, focused on a time-dependent stochastic subspace that respects nonlinear dynamics and captures non-Gaussian statistics as it occurs. The motivation is to obtain a filter that is applicable to realistic geophysical applications, but that also rigorously utilizes the governing dynamical equations with information theory and learning theory for efficient Bayesian data assimilation. Building on the foundations of classical filters, the underlying theory and algorithmic implementation of the new filter are developed and derived. The stochastic Dynamically Orthogonal (DO) field equations and their adaptive stochastic subspace are employed to predict prior probabilities for the full dynamical state, effectively approximating the Fokker–Planck equation. At assimilation times, the DO realizations are fit to semiparametric Gaussian Mixture Models (GMMs) using the Expectation-Maximization algorithm and the Bayesian Information Criterion. Bayes’s law is then efficiently carried out analytically within the evolving stochastic subspace. The resulting GMM-DO filter is illustrated in a very simple example. Variations of the GMM-DO filter are also provided along with comparisons with related schemes.

Corresponding author address: Pierre F. J. Lermusiaux, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139. E-mail: pierrel@mit.edu
Save
Cover Monthly Weather Review
Monthly Weather Review

  • Alspach, D. L., and H. W. Sorenson, 1972: Nonlinear Bayesian estimation using Gaussian sum approximations. IEEE Trans. Autom. Control, 17, 438–448.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 2741–2758.

    • Search Google Scholar
    • Export Citation

  • Auclair, F., P. Marsaleix, and P. D. Mey, 2003: Space-time structure and dynamics of the forecast error in a coastal circulation model of the Gulf of Lions. Dyn. Atmos. Oceans, 36, 309–346.

    • Search Google Scholar
    • Export Citation

  • Bengtsson, T., C. Snyder, and D. Nychka, 2003: Toward a nonlinear ensemble filter for high-dimensional systems. J. Geophys. Res., 108, 8775, doi:10.1029/2002JD002900.

    • Search Google Scholar
    • Export Citation

  • Bennett, A., 1992: Inverse Methods in Physical Oceanography. Cambridge University Press, 346 pp.

  • Bennett, A., 2002: Inverse Modeling of the Ocean and Atmosphere. Cambridge University Press, 234 pp.

  • Bertsekas, D. P., and J. N. Tsitsiklis, 2008: Introduction to Probability. 2nd ed. Athena Scientific, 544 pp.

  • Bishop, C. M., 2006: Pattern Recognition and Machine Learning. Springer, 738 pp.

  • Bocquet, M., C. A. Pires, and L. Wu, 2010: Beyond Gaussian statistical modeling in geophysical data assimilation. Mon. Wea. Rev., 138, 2997–3023.

    • Search Google Scholar
    • Export Citation

  • Casella, G., and R. L. Berger, 2001: Statistical Inference. 2nd ed. Duxbury, 660 pp.

  • Chen, R., and J. S. Liu, 2000: Mixture Kalman filters. J. Roy. Stat. Soc., 62B, 493–508.

  • Cover, T. M., and J. A. Thomas, 2006: Elements of Information Theory. Wiley-Interscience, 748 pp.

  • Commission on Physical Sciences, Mathematics, and Applications, 1993: Statistics and Physical Oceanography. The National Academies Press, 62 pp.

  • Daley, R., 1991: Atmospheric Data Analysis. Cambridge University Press, 457 pp.

  • Dee, D. P., and A. M. D. Silva, 2003: The choice of variable for atmospheric moisture analysis. Mon. Wea. Rev., 131, 155–171.

  • Dempster, A. P., N. M. Laird, and D. B. Rubin, 1977: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc., 39B, 1–38.

    • Search Google Scholar
    • Export Citation

  • Dimet, F. X. L., and O. Talagrand, 1986: Variational algorithms for analysis and assimilation of meteorological observations. Tellus, 38A, 97–110.

    • Search Google Scholar
    • Export Citation

  • Doucet, A., N. de Freitas, and N. Gordon, 2001: Sequential Monte-Carlo Methods in Practice. Springer-Verlag, 612 pp.

  • Dovera, L., and E. D. Rossa, 2011: Multimodal ensemble Kalman filtering using Gaussian mixture models. Comput. Geosci., 15,307–323.

  • Duda, R. O., P. E. Hart, and D. G. Stork, 2001: Pattern Classification. 2nd ed. Wiley-Interscience, 654 pp.

  • Eisenberger, I., 1964: Genesis of bimodal distributions. Technometrics, 6, 357–363.

  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte-Carlo methods to forecast error statistics. J. Geophys. Res., 99 (C5), 10 143–10 162.

    • Search Google Scholar
    • Export Citation

  • Evensen, G., 2007: Data Assimilation: The Ensemble Kalman Filter. Springer, 279 pp.

  • Eyink, G. L., and S. Kim, 2006: A maximum entropy method for particle filtering. J. Stat. Phys., 123, 1071–1128.

  • Frei, M., and H. R. Kunsch, 2013: Mixture ensemble Kalman filters. Comput. Stat. Data Anal., 58, 127–138.

  • Gelb, A., 1974: Applied Optimal Estimation. MIT Press, 374 pp.

  • Ghanem, R., and P. Spanos, 1991: Stochastic Finite Elements: A Spectral Approach. Springer-Verlag, 214 pp.

  • Ghil, M., and P. Malanotte-Rizzoli, 1991: Data assimilation in meteorology and oceanography. Advances in Geophysics, Vol. 33, Academic Press, 141–266.

  • Holmes, P., J. Lumley, and G. Berkooz, 1996: Turbulence, Coherent Structures, Dynamical Systems, and Symmetry. Cambridge University Press, 420 pp.

  • Hoteit, I., D. T. Pham, G. Triantafyllou, and G. Korres, 2008: A new approximate solution of the optimal nonlinear filter for data assimilation in meteorology and oceanography. Mon. Wea. Rev., 136, 317–334.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., H. L. Mitchell, and L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796–811.

    • Search Google Scholar
    • Export Citation

  • Ide, K., P. Courtier, M. Ghil, and A. Lorenc, 1997: Unified notation for data assimilation: Operational, sequential and variational. J.Meteor. Soc. Japan, 75, 181–189.

    • Search Google Scholar
    • Export Citation

  • Jazwinski, A. H., 1970: Stochastic Processes and Filtering Theory. Academic Press, 376 pp.

  • Kalman, R. E., 1960: A new approach to linear filtering and prediction problems. Trans. ASME, 82D, 35–45.

  • Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, 341 pp.

  • Kim, S., G. L. Eyink, J. M. Restrepo, F. J. Alexander, and G. Johnson, 2009: Ensemble filtering for nonlinear dynamics. Mon. Wea. Rev., 131, 2586–2594.

    • Search Google Scholar
    • Export Citation

  • Kotecha, J. H., and P. A. Djuric, 2003: Gaussian particle filtering. IEEE Trans. Signal Process., 51, 2592–2601.

  • Krause, P., and J. M. Restrepo, 2009: The diffusion kernel filter applied to Lagrangian data assimilation. Mon. Wea. Rev., 137, 4386–4400.

    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., 1997: Data assimilation via error subspace statistical estimation. Ph.D. thesis, Division of Engineering and Applied Sciences, Harvard University, 402 pp.

  • Lermusiaux, P. F. J., 1999a: Data assimilation via error subspace statistical estimation. Part II: Middle Atlantic Bight shelfbreak front simulations. Mon. Wea. Rev., 127, 1408–1432.

    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., 1999b: Estimation and study of mesoscale variability in the Strait of Sicily. Dyn. Atmos. Oceans, 29, 255–303.

  • Lermusiaux, P. F. J., 2001: Evolving the subspace of the three-dimensional multiscale ocean variability: Massachusetts Bay. J. Mar. Syst., 29, 385–422.

    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., 2006: Uncertainty estimation and prediction for interdisciplinary ocean dynamics. J. Comput. Phys., 217, 176–199, doi:10.1016/j.jcp.2006.02.010.

    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., 2007: Adaptive modeling, adaptive data assimilation and adaptive sampling. Physica D, 230, 172–196.

  • Lermusiaux, P. F. J., and A. Robinson, 1999: Data assimilation via error subspace statistical estimation. Part I: Theory and scheme. Mon. Wea. Rev., 127, 1385–1407.

    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., C.-S. Chiu, and A. R. Robinson, 2002a: Modeling uncertainties in the prediction of the acoustic wavefield in a shelfbreak environment. Theoretical and Computational Acoustics, E.-C. Shang, Q. Li, and T. Gao, Eds., World Scientific Publishing Co., 191–200.

  • Lermusiaux, P. F. J., A. R. Robinson, P. J. Haley, and W. G. Leslie, 2002b: Advanced interdisciplinary data assimilation: Filtering and smoothing via error subspace statistical estimation. Proc. OCEANS 2002 MTS/IEEE Conf., Biloxi, MS, IEEE, 795–802.

  • Lermusiaux, P. F. J., and Coauthors, 2006: Quantifying uncertainties in ocean predictions. Oceanography, 19, 92–105.

  • Lions, J. L., 1971: Optimal Control of Systems Governed by Partial Differential Equations. Springer-Verlag, 396 pp.

  • Lorenz, E., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130–141.

  • MacKay, D. J. C., 2003: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 628 pp.

  • Malanotte-Rizzoli, P., 1996: Modern Approaches to Data Assimilation in Ocean Modeling. Elsevier, 455 pp.

  • McLachlan, G., and D. Peel, 2000: Finite Mixture Models. John Wiley & Sons, Inc., 419 pp.

  • Miller, R. N., M. Ghil, and F. Gauthiez, 1994: Advanced data assimilation in strongly nonlinear dynamical systems. J. Atmos. Sci., 51, 1037–1056.

    • Search Google Scholar
    • Export Citation

  • Moore, A. M., H. G. Arango, E. D. Lorenzo, B. D. Cornuelle, A. J. Miller, and D. J. Neilson, 2004: A comprehensive ocean prediction and analysis system based on the tangent linear and adjoint of a regional ocean model. Ocean Modell., 7, 227–258, doi:10.1016/j.ocemod.2003.11.001.

    • Search Google Scholar
    • Export Citation

  • Papoulis, A., 1965: Probability, Random Variables and Stochastic Processes. McGraw-Hill, 583 pp.

  • Pham, D. T., 2001: Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Wea. Rev., 129, 1194–1207.

  • Robinson, A. R., P. F. J. Lermusiaux, and N. Q. Sloan, 1998: Data assimilation. The Sea, K. H. Brink and A. R. Robinson, Eds., The Global Coastal Ocean: Processes and Methods, Vol. 10, John Wiley and Sons, 541–594.

  • Sapsis, T., 2010: Dynamically orthogonal field equations. Ph.D. thesis, Department of Mechanical Engineering, Massachusetts Institute of Technology, 188 pp.

  • Sapsis, T., and P. F. J. Lermusiaux, 2009: Dynamically orthogonal field equations for continuous stochastic dynamical systems. Physica D, 238, 2347–2360, doi:10.1016/j.physd.2009.09.017.

    • Search Google Scholar
    • Export Citation

  • Sapsis, T., and P. F. J. Lermusiaux, 2011: Dynamical criteria for the evolution of the stochastic dimensionality in flows with uncertainty. Physica D, 241, 60–76, doi:10.1016/j.physd.2011.10.001.

    • Search Google Scholar
    • Export Citation

  • Schwartz, G. E., 1978: Estimating the dimension of a model. Ann. Stat., 6, 461–464.

  • Silverman, B., 1992: Density Estimation for Statistics and Data Analysis. Chapman & Hall, 175 pp.

  • Smith, K. W., 2007: Cluster ensemble Kalman filter. Tellus, 59A, 749–757.

  • Sobczyk, K., 2001: Information dynamics: Premises, challenges and results. Mech. Syst. Signal Process., 15, 475–498.

  • Sondergaard, T., 2011: Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations. M.S. thesis, Department of Mechanical Engineering, Massachusetts Institute of Technology, 180 pp.

  • Sondergaard, T., and P. F. J. Lermusiaux, 2013: Data assimilation with Gaussian Mixture Models using the Dynamically Orthogonal field equations. Part II: Applications. Mon. Wea. Rev., 141, 1761–1785.

    • Search Google Scholar
    • Export Citation

  • Sura, P., 2010: On non-Gaussian SST variability in the Gulf Stream and other strong currents. Ocean Dyn., 60, 155–170.

  • Tarantola, A., 2005: Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM, 342 pp.

  • Tippett, M. K., J. L. Anderson, C. H. Bishop, T. M. Hamill, and J. S. Whitaker, 2003: Ensemble square root filters. Mon. Wea. Rev., 131, 1485–1490.

    • Search Google Scholar
    • Export Citation

  • Ueckermann, M. P., P. F. J. Lermusiaux, and T. P. Sapsis, 2013: Numerical schemes for dynamically orthogonal equations of stochastic fluid and ocean flows. J. Comput. Phys., 233, 272–294, doi:10.1016/j.jcp.2012.08.041.

    • Search Google Scholar
    • Export Citation

  • van Leeuwen, P. J., 2009: Particle filtering in geophysical systems. Mon. Wea. Rev., 137, 4089–4114.

  • Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1924.

  • Wunsch, C., 1996: The Ocean Circulation Inverse Problem. Cambridge University Press, 442 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 819 277 13
PDF Downloads 575 135 14