Editorial Type:
Article
Article Type:
Research Article

A Gaussian Mixture Model Smoother for Continuous Nonlinear Stochastic Dynamical Systems: Theory and Scheme

Tapovan Lolla Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts

Search for other papers by Tapovan Lolla in
Current site
Google Scholar
PubMed Close
and
Pierre F. J. Lermusiaux Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts

Search for other papers by Pierre F. J. Lermusiaux in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Jul 2017
DOI:
https://doi.org/10.1175/MWR-D-16-0064.1
Page(s):
2743–2761
Restricted access

Abstract

Retrospective inference through Bayesian smoothing is indispensable in geophysics, with crucial applications in ocean and numerical weather estimation, climate dynamics, and Earth system modeling. However, dealing with the high-dimensionality and nonlinearity of geophysical processes remains a major challenge in the development of Bayesian smoothers. Addressing this issue, a novel subspace smoothing methodology for high-dimensional stochastic fields governed by general nonlinear dynamics is obtained. Building on recent Bayesian filters and classic Kalman smoothers, the fundamental equations and forward–backward algorithms of new Gaussian Mixture Model (GMM) smoothers are derived, for both the full state space and dynamic subspace. For the latter, the stochastic Dynamically Orthogonal (DO) field equations and their time-evolving stochastic subspace are employed to predict the prior subspace probabilities. Bayesian inference, both forward and backward in time, is then analytically carried out in the dominant stochastic subspace, after fitting semiparametric GMMs to joint subspace realizations. The theoretical properties, varied forms, and computational costs of the new GMM smoother equations are presented and discussed.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author e-mail: Pierre F. J. Lermusiaux, pierrel@mit.edu

Abstract

Retrospective inference through Bayesian smoothing is indispensable in geophysics, with crucial applications in ocean and numerical weather estimation, climate dynamics, and Earth system modeling. However, dealing with the high-dimensionality and nonlinearity of geophysical processes remains a major challenge in the development of Bayesian smoothers. Addressing this issue, a novel subspace smoothing methodology for high-dimensional stochastic fields governed by general nonlinear dynamics is obtained. Building on recent Bayesian filters and classic Kalman smoothers, the fundamental equations and forward–backward algorithms of new Gaussian Mixture Model (GMM) smoothers are derived, for both the full state space and dynamic subspace. For the latter, the stochastic Dynamically Orthogonal (DO) field equations and their time-evolving stochastic subspace are employed to predict the prior subspace probabilities. Bayesian inference, both forward and backward in time, is then analytically carried out in the dominant stochastic subspace, after fitting semiparametric GMMs to joint subspace realizations. The theoretical properties, varied forms, and computational costs of the new GMM smoother equations are presented and discussed.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author e-mail: Pierre F. J. Lermusiaux, pierrel@mit.edu
Save
Cover Monthly Weather Review
Monthly Weather Review

  • Atkins, E., M. Morzfeld, and A. J. Chorin, 2013: Implicit particle methods and their connection with variational data assimilation. Mon. Wea. Rev., 141, 1786–1803, doi:10.1175/MWR-D-12-00145.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Barth, A., A. Alvera-Azcárate, K.-W. Gurgel, J. Staneva, A. Port, J.-M. Beckers, and E. V. Stanev, 2010: Ensemble perturbation smoother for optimizing tidal boundary conditions by assimilation of high-frequency radar surface currents application to the German Bight. Ocean Sci., 6, 161–178, doi:10.5194/os-6-161-2010.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bennett, A. F., 1992: Inverse Methods in Physical Oceanography. Cambridge University Press, 346 pp.

    • Crossref
    • Export Citation

  • Bilmes, J. A., 1998: A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models. International Computer Science Institute Tech. Rep. TR-97-021, 18 pp. [Available online at http://melodi.ee.washington.edu/people/bilmes/mypapers/em.pdf.]

  • Bocquet, M., 2005: Reconstruction of an atmospheric tracer source using the principle of maximum entropy. I: Theory. Quart. J. Roy. Meteor. Soc., 131, 2191–2208, doi:10.1256/qj.04.67.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bocquet, M., C. A. Pires, and L. Wu, 2010: Beyond Gaussian statistical modeling in geophysical data assimilation. Mon. Wea. Rev., 138, 2997–3023, doi:10.1175/2010MWR3164.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bouveyron, C., and C. Brunet-Saumard, 2014: Model-based clustering of high-dimensional data: A review. Comput. Stat. Data Anal., 71, 52–78, doi:10.1016/j.csda.2012.12.008.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bresler, Y., 1986: Two-filter formulae for discrete-time non-linear Bayesian smoothing. Int. J. Control, 43, 629–641, doi:10.1080/00207178608933489.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Briers, M., A. Doucet, and S. Maskell, 2010: Smoothing algorithms for state–space models. Ann. Inst. Stat. Math., 62, 61–89, doi:10.1007/s10463-009-0236-2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Choi, H.-L., and J. P. How, 2009: On the roles of smoothing in planning of informative paths. American Control Conf. 2009, St. Louis, MO, American Automatic Control Council, 2154–2159. [Available online at http://web.mit.edu/hanlimc/www/hl.docs/ChoiHow_ACC09.pdf.]

  • Choi, H.-L., and J. P. How, 2010: Continuous trajectory planning of mobile sensors for informative forecasting. Automatica, 46, 1266–1275, doi:10.1016/j.automatica.2010.05.004.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Cohn, S. E., N. Sivakumaran, and R. Todling, 1994: A fixed-lag Kalman smoother for retrospective data assimilation. Mon. Wea. Rev., 122, 2838–2867, doi:10.1175/1520-0493(1994)122<2838:AFLKSF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Cosme, E., J.-M. Brankart, J. Verron, P. Brasseur, and M. Krysta, 2010: Implementation of a reduced rank square-root smoother for high resolution ocean data assimilation. Ocean Modell., 33, 87–100, doi:10.1016/j.ocemod.2009.12.004.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Cosme, E., J. Verron, P. Brasseur, J. Blum, and D. Auroux, 2012: Smoothing problems in a Bayesian framework and their linear Gaussian solutions. Mon. Wea. Rev., 140, 683–695, doi:10.1175/MWR-D-10-05025.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Crassidis, J. L., and J. L. Junkins, 2011: Optimal Estimation of Dynamic Systems. 2nd ed. CRC Press, 733 pp.

    • Crossref
    • Export Citation

  • Daley, R., 1993: Atmospheric Data Analysis. 2nd ed. Cambridge Atmospheric and Space Science Series, Cambridge University Press, 472 pp.

  • Di Lorenzo, E., A. M. Moore, H. G. Arango, B. D. Cornuelle, A. J. Miller, B. Powell, B. S. Chua, and A. F. Bennett, 2007: Weak and strong constraint data assimilation in the inverse Regional Ocean Modeling System (ROMS): Development and application for a baroclinic coastal upwelling system. Ocean Modell., 16, 160–187, doi:10.1016/j.ocemod.2006.08.002.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Dimet, F.-X. L., and O. Talagrand, 1986: Variational algorithms for analysis and assimilation of meteorological observations: Theoretical aspects. Tellus, 38A, 97–110, doi:10.1111/j.1600-0870.1986.tb00459.x.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Doucet, A., and A. M. Johansen, 2009: A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of Nonlinear Filtering, D. Crisan and B. Rozovskiĭ, Eds., Oxford University Press, 656–704.

  • Evensen, G., and P. J. Van Leeuwen, 2000: An ensemble Kalman smoother for nonlinear dynamics. Mon. Wea. Rev., 128, 1852–1867, doi:10.1175/1520-0493(2000)128<1852:AEKSFN>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Feppon, F., and P. F. J. Lermusiaux, 2017: A geometric approach to dynamical model-order reduction. SIAM J. Sci. Comput., in press.

  • Gelb, A., 1974: Applied Optimal Estimation. The MIT Press, 374 pp.

  • Ghanem, R. G., and P. D. Spanos, 2003: Stochastic Finite Elements: A Spectral Approach. Courier Corporation, 222 pp.

  • Ghil, M., and P. Malanotte-Rizzoli, 1991: Data assimilation in meteorology and oceanography. Advances in Geophysics, Vol. 33, Academic Press, 141–266, doi:10.1016/S0065-2687(08)60442-2.

    • Crossref
    • Export Citation

  • Godsill, S. J., A. Doucet, and M. West, 2004: Monte Carlo smoothing for nonlinear time series. J. Amer. Stat. Assoc., 99, 156–168, doi:10.1198/016214504000000151.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hoteit, I., D.-T. Pham, G. Triantafyllou, and G. Korres, 2008: A new approximate solution of the optimal nonlinear filter for data assimilation in meteorology and oceanography. Mon. Wea. Rev., 136, 317–334, doi:10.1175/2007MWR1927.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hsieh, M. A., and G. Chirikjian, 2014: Distributed Autonomous Robotic Systems: The 11th International Symposium. Springer Tracts in Advanced Robotics, Vol. 104, Springer, 450 pp.

  • Jazwinski, A. H., 2007: Stochastic Processes and Filtering Theory. Dover Publications, 400 pp.

  • Kaess, M., S. Williams, V. Indelman, R. Roberts, J. J. Leonard, and F. Dellaert, 2012: Concurrent filtering and smoothing. 15th Int. Conf. on Information Fusion, Singapore, IEEE, 1300–1307.

  • Kalman, R. E., 1960: A new approach to linear filtering and prediction problems. J. Fluids Eng., 82, 35–45.

  • Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation, and Predictability. Cambridge University Press, 341 pp.

    • Crossref
    • Export Citation

  • Khare, S. P., J. L. Anderson, T. J. Hoar, and D. Nychka, 2008: An investigation into the application of an ensemble Kalman smoother to high-dimensional geophysical systems. Tellus, 60A, 97–112, doi:10.1111/j.1600-0870.2007.00281.x.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Kitagawa, G., 1987: Non-Gaussian state–space modeling of nonstationary time series. J. Amer. Stat. Assoc., 82, 1032–1041.

  • Kitagawa, G., 1994: The two-filter formula for smoothing and an implementation of the Gaussian-sum smoother. Ann. Inst. Stat. Math., 46, 605–623, doi:10.1007/BF00773470.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Klaas, M., M. Briers, N. De Freitas, A. Doucet, S. Maskell, and D. Lang, 2006: Fast particle smoothing: If I had a million particles. Proc. 23rd Int. Conf. on Machine Learning, Pittsburgh, PA, ACM, 481–488.

    • Crossref
    • Export Citation

  • Lee, D. J., and M. E. Campbell, 2015: Smoothing algorithm for nonlinear systems using Gaussian mixture models. J. Guid. Control Dyn., 38, 1438–1451, doi:10.2514/1.G000603.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., 1997: Error subspace data assimilation methods for ocean field estimation: Theory, validation and applications. Ph.D. thesis, Harvard University, 402 pp.

  • Lermusiaux, P. F. J., 1999: Estimation and study of mesoscale variability in the Strait of Sicily. Dyn. Atmos. Oceans, 29, 255–303, doi:10.1016/S0377-0265(99)00008-1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., 2006: Uncertainty estimation and prediction for interdisciplinary ocean dynamics. J. Comput. Phys., 217, 176–199, doi:10.1016/j.jcp.2006.02.010.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., 2007: Adaptive modeling, adaptive data assimilation and adaptive sampling. Physica D, 230, 172–196, doi:10.1016/j.physd.2007.02.014.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., and A. Robinson, 1999: Data assimilation via error subspace statistical estimation. Part I: Theory and schemes. Mon. Wea. Rev., 127, 1385–1407, doi:10.1175/1520-0493(1999)127<1385:DAVESS>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lermusiaux, P. F. J., A. R. Robinson, P. J. Haley, and W. G. Leslie, 2002: Advanced interdisciplinary data assimilation: Filtering and smoothing via error subspace statistical estimation. Proc. OCEANS 2002 MTS/IEEE Conf., Biloxi, MS, IEEE, 795–802.

  • Lolla, T., 2016: Path planning and adaptive sampling in the coastal ocean. Ph.D. thesis, Massachusetts Institute of Technology, 315 pp. [Available online at https://dspace.mit.edu/handle/1721.1/103438.]

  • Lolla, T., and P. F. J. Lermusiaux, 2017: A Gaussian mixture model smoother for continuous nonlinear stochastic dynamical systems: Applications. Mon. Wea. Rev., 145, 2763–2790, doi:10.1175/MWR-D-16-0065.1.

    • Crossref
    • Export Citation

  • McLachlan, G., and D. Peel, 2000: Finite Mixture Models. Wiley Series in Probability and Statistics, John Wiley and Sons, 419 pp.

    • Crossref
    • Export Citation

  • Miller, R. N., E. F. Carter, and S. T. Blue, 1999: Data assimilation into nonlinear stochastic models. Tellus, 51A, 167–194, doi:10.1034/j.1600-0870.1999.t01-2-00002.x.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Moore, A. M., H. G. Arango, E. Di Lorenzo, B. D. Cornuelle, A. J. Miller, and D. J. Neilson, 2004: A comprehensive ocean prediction and analysis system based on the tangent linear and adjoint of a regional ocean model. Ocean Modell., 7, 227–258, doi:10.1016/j.ocemod.2003.11.001.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Nerger, L., S. Schulte, and A. Bunse-Gerstner, 2014: On the influence of model nonlinearity and localization on ensemble Kalman smoothing. Quart. J. Roy. Meteor. Soc., 140, 2249–2259, doi:10.1002/qj.2293.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Raanes, P. N., 2016: On the ensemble Rauch–Tung–Striebel smoother and its equivalence to the ensemble Kalman smoother. Quart. J. Roy. Meteor. Soc., 142, 1259–1264, doi:10.1002/qj.2728.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Rauch, H. E., C. Striebel, and F. Tung, 1965: Maximum likelihood estimates of linear dynamic systems. AIAA J., 3, 1445–1450, doi:10.2514/3.3166.

  • Ravela, S., and D. McLaughlin, 2007: Fast ensemble smoothing. Ocean Dyn., 57, 123–134, doi:10.1007/s10236-006-0098-6.

  • Robinson, A. R., P. F. J. Lermusiaux, and N. Q. Sloan III, 1998: Data assimilation. The Global Coastal Ocean: Processes and Methods, K. H. Brink and A. R. Robinson, Eds., The Sea—Ideas and Observations on Progress in the Study of the Seas, Vol. 10, John Wiley and Sons, 541–594.

  • Sapsis, T. P., and P. F. J. Lermusiaux, 2009: Dynamically orthogonal field equations for continuous stochastic dynamical systems. Physica D, 238, 2347–2360, doi:10.1016/j.physd.2009.09.017.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Sapsis, T. P., and P. F. J. Lermusiaux, 2012: Dynamical criteria for the evolution of the stochastic dimensionality in flows with uncertainty. Physica D, 241, 60–76, doi:10.1016/j.physd.2011.10.001.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Särkkä, S., 2013: Bayesian Filtering and Smoothing. Institute of Mathematical Statistics Textbooks, Vol. 3, Cambridge University Press, 232 pp.

  • Sasaki, Y., 1970: Some basic formalisms in numerical variational analysis. Mon. Wea. Rev., 98, 875–883, doi:10.1175/1520-0493(1970)098<0875:SBFINV>2.3.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Skandrani, C., J.-M. Brankart, N. Ferry, J. Verron, P. Brasseur, and B. Barnier, 2009: Controlling atmospheric forcing parameters of global ocean models: Sequential assimilation of sea surface Mercator-Ocean reanalysis data. Ocean Sci., 5, 403–419, doi:10.5194/os-5-403-2009.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Sondergaard, T., and P. F. J. Lermusiaux, 2013a: Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations. Part I: Theory and scheme. Mon. Wea. Rev., 141, 1737–1760, doi:10.1175/MWR-D-11-00295.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Sondergaard, T., and P. F. J. Lermusiaux, 2013b: Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations. Part II: Applications. Mon. Wea. Rev., 141, 1761–1785, doi:10.1175/MWR-D-11-00296.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Stammer, D., and Coauthors, 2002: Global ocean circulation during 1992–1997, estimated from ocean observations and a general circulation model. J. Geophys. Res., 107, 3118, doi:10.1029/2001JC000888.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Tagade, P., H. Seybold, and S. Ravela, 2014: Mixture ensembles for data assimilation in dynamic data-driven environmental systems. Proc. Comput. Sci., 29, 1266–1276, doi:10.1016/j.procs.2014.05.114.

    • Crossref
    • Export Citation

  • Talagrand, O., 1997: Assimilation of observations, an introduction. J. Meteor. Soc. Japan, 75 (1B), 191–209.

  • Thrun, S., W. Burgard, and D. Fox, 2005: Probabilistic Robotics. The MIT Press, 647 pp.

  • Ueckermann, M. P., P. F. J. Lermusiaux, and T. P. Sapsis, 2013: Numerical schemes for dynamically orthogonal equations of stochastic fluid and ocean flows. J. Comput. Phys., 233, 272–294, doi:10.1016/j.jcp.2012.08.041.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Van Leeuwen, P. J., and G. Evensen, 1996: Data assimilation and inverse methods in terms of a probabilistic formulation. Mon. Wea. Rev., 124, 2898–2913, doi:10.1175/1520-0493(1996)124<2898:DAAIMI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Vo, B.-N., B.-T. Vo, and R. P. Mahler, 2012: Closed-form solutions to forward–backward smoothing. IEEE Trans. Signal Process., 60, 2–17, doi:10.1109/TSP.2011.2168519.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Weir, B., R. Miller, and Y. Spitz, 2013: A potential implicit particle method for high-dimensional systems. Nonlinear Processes Geophys., 20, 1047–1060, doi:10.5194/npg-20-1047-2013.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Willsky, A. S., 2002: Multiresolution Markov models for signal and image processing. Proc. IEEE, 90, 1396–1458, doi:10.1109/JPROC.2002.800717.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Wunsch, C., 1996: The Ocean Circulation Inverse Problem. Cambridge University Press, 442 pp.

    • Crossref
    • Export Citation

  • Wunsch, C., and P. Heimbach, 2007: Practical global oceanic state estimation. Physica D, 230, 197–208, doi:10.1016/j.physd.2006.09.040.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 541 132 11
PDF Downloads 379 91 9