Data Assimilation in the Low Noise Regime with Application to the Kuroshio

Eric Vanden-Eijnden Courant Institute, New York University, New York, New York

Search for other papers by Eric Vanden-Eijnden in
Current site
Google Scholar
PubMed
Close
and
Jonathan Weare University of Chicago, Chicago, Illinois

Search for other papers by Jonathan Weare in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Online data assimilation techniques such as ensemble Kalman filters and particle filters lose accuracy dramatically when presented with an unlikely observation. Such an observation may be caused by an unusually large measurement error or reflect a rare fluctuation in the dynamics of the system. Over a long enough span of time it becomes likely that one or several of these events will occur. Often they are signatures of the most interesting features of the underlying system and their prediction becomes the primary focus of the data assimilation procedure. The Kuroshio or Black Current that runs along the eastern coast of Japan is an example of such a system. It undergoes infrequent but dramatic changes of state between a small meander during which the current remains close to the coast of Japan, and a large meander during which it bulges away from the coast. Because of the important role that the Kuroshio plays in distributing heat and salinity in the surrounding region, prediction of these transitions is of acute interest. Here the authors focus on a regime in which both the stochastic forcing on the system and the observational noise are small. In this setting large deviation theory can be used to understand why standard filtering methods fail and guide the design of the more effective data assimilation techniques. Motivated by this analysis the authors propose several data assimilation strategies capable of efficiently handling rare events such as the transitions of the Kuroshio. These techniques are tested on a model of the Kuroshio and are shown to perform much better than standard filtering methods.

Corresponding author address: Jonathan Weare, University of Chicago, 5734 S. University Ave., Chicago, IL 60637. E-mail: weare@uchicago.edu

Abstract

Online data assimilation techniques such as ensemble Kalman filters and particle filters lose accuracy dramatically when presented with an unlikely observation. Such an observation may be caused by an unusually large measurement error or reflect a rare fluctuation in the dynamics of the system. Over a long enough span of time it becomes likely that one or several of these events will occur. Often they are signatures of the most interesting features of the underlying system and their prediction becomes the primary focus of the data assimilation procedure. The Kuroshio or Black Current that runs along the eastern coast of Japan is an example of such a system. It undergoes infrequent but dramatic changes of state between a small meander during which the current remains close to the coast of Japan, and a large meander during which it bulges away from the coast. Because of the important role that the Kuroshio plays in distributing heat and salinity in the surrounding region, prediction of these transitions is of acute interest. Here the authors focus on a regime in which both the stochastic forcing on the system and the observational noise are small. In this setting large deviation theory can be used to understand why standard filtering methods fail and guide the design of the more effective data assimilation techniques. Motivated by this analysis the authors propose several data assimilation strategies capable of efficiently handling rare events such as the transitions of the Kuroshio. These techniques are tested on a model of the Kuroshio and are shown to perform much better than standard filtering methods.

Corresponding author address: Jonathan Weare, University of Chicago, 5734 S. University Ave., Chicago, IL 60637. E-mail: weare@uchicago.edu
Save
  • Alexander, F., G. Eyink, and J. Restrepo, 2004: Accelerated Monte Carlo for optimal estimation of time series. J. Stat. Phys., 119, 13311345.

    • Search Google Scholar
    • Export Citation
  • Apte, A., C. Jones, and A. Stuart, 2008: A Bayesian approach to Lagrangian data assimilation. Tellus, 60A, 336347.

  • Bardi, M., and I. Capuzzo-Dolcetta, 2008: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, 588 pp.

  • Chao, S., 1984: Bimodality of the Kuroshio. J. Phys. Oceanogr., 14, 92103.

  • Chorin, A., and X. Tu, 2009: Implicit sampling for particle filters. Proc. Natl. Acad. Sci. USA, 106, 17 24917 254.

  • Chorin, A., M. Morzfeld, and X. Tu, 2010: Implicit filters for data assimilation. Commun. Appl. Math. Comput. Sci., 5, 221240.

  • Doucet, A., N. de Freitas, and N. Gordon, Eds., 2001: Sequential Monte Carlo Methods in Practice. Springer, 612 pp.

  • Dembo, A., and O. Zeitouni, 1993: Large Deviations Techniques and Applications. Jones and Bartlett, 346 pp.

  • Derber, J., 1989: A variational continuous assimilation technique. Mon. Wea. Rev., 117, 24372446.

  • Dupuis, P., and R. S. Ellis, 1997: A Weak Convergence Approach to the Theory of Large Deviations. Wiley, 504 pp.

  • Evensen, G., 2003: The ensemble Kalman filter: Theoretical formulation and practical implementation. Ocean Dyn., 53, 343367.

  • Fleming, W., and H. Soner, 2006: Controlled Markov Processes and Viscosity Solutions. Springer, 446 pp.

  • Freidlin, M., and A. Wentzell, 1984: Random Perturbations of Dynamical Systems. Springer-Verlag, 326 pp.

  • Gordon, N., D. Salmond, and A. Smith, 1993: Novel approach to nonlinear non-Gaussian Bayesian state estimation. IEE Proc. F, 51, 107113.

    • Search Google Scholar
    • Export Citation
  • Heymann, M., and E. Vanden-Eijnden, 2008: The geometric minimum action method: A least action principle on the space of curves. Commun. Pure Appl. Math., 61, 10521117.

    • Search Google Scholar
    • Export Citation
  • Hunt, B., E. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112126.

    • Search Google Scholar
    • Export Citation
  • Kalman, R., 1960: A new approach to linear filtering and prediction problems. J. Basic Eng., 82, 3545.

  • Karatzas, I., 1991: Brownian Motion and Stochastic Calculus. Springer-Verlag, 470 pp.

  • Kitagawa, G., 1996: Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Stat., 5, 125.

  • Law, K., and A. Stuart, 2012: Evaluating data assimilation algorithms. Mon. Wea. Rev.,140, 3757–3782.

  • Lei, J., and P. Bickel, 2009: Ensemble filtering for high dimensional nonlinear state space models. University of California, Berkeley, Rep. 779, 23 pp. [Available online at http://digitalassets.lib.berkeley.edu/sdtr/ucb/text/779.pdf.]

  • Lei, J., P. Bickel, and C. Snyder, 2010: Comparison of ensemble Kalman filters under non-Gaussianity. Mon. Wea. Rev., 138, 12931306.

  • Lewis, J. M., S. Lakshmivarahan, and S. Dhall, 2006: Dynamic Data Assimilation: A Least Squares Approach. Cambridge University Press, 680 pp.

  • Liu, J., 2002: Monte Carlo Strategies in Scientific Computing. Springer, 343 pp.

  • Miller, R., M. Ghil, and F. Gauthiez, 1994: Data assimilation in strongly nonlinear dynamical systems. J. Atmos. Sci., 51, 10371056.

  • Miller, R., E. Carter, and S. Blue, 1999a: Data assimilation into nonlinear stochastic models. Tellus, 51A, 167194.

  • Miller, R., E. Carter, and S. Blue, 1999b: Data assimilation into nonlinear stochastic models. Tellus, 51A, 167194.

  • Miyazawa, Y., T. Kagimoto, X. Guo, and H. Sakuma, 2008: The Kuroshio large meander formation in 2004 analyzed by an eddy-resolving ocean forecast system. J. Geophys. Res.,113, C10015, doi:10.1029/2007JC004226.

  • Nocedal, J., and S. Wright, 2006: Numerical Optimization. 2nd ed. Springer, 664 pp.

  • Ott, E., B. Hunt, I. Szunyogh, A. Zimin, E. Kostelich, M. Corazza, E. Kalnay, and D. Patil, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415428.

    • Search Google Scholar
    • Export Citation
  • Qiu, B., and W. Miao, 2000: Kuroshio path variations south of Japan: Bimodality as a self-sustained internal oscillation. J. Phys. Oceanogr., 30, 21242137.

    • Search Google Scholar
    • Export Citation
  • Stauffer, D., and J. Bao, 1993: Optimal determination of nudging coefficients using the adjoint equations. Tellus,45A, 358–369, doi:10.1034/j.1600-0870.1993.t01-4-00003.x.

  • Taft, B., 1972: Characteristics of the flow of the Kuroshio south of Japan. Kuroshio: Physical Aspects of the Japan Coast, K. Y. H. Stommel, Ed., University of Washington Press, 165–214.

  • van Leeuwen, P., 2010: Nonlinear data assimilation in geosciences: An extremely efficient particle filter. Quart. J. Roy. Meteor. Soc., 136B, 19911999.

    • Search Google Scholar
    • Export Citation
  • Vanden-Eijnden, E., and J. Weare, 2012: Rare event simulation for small noise diffusions. Commun. Pure Appl. Math., 65 (12 17701803.

  • Varadhan, S., 1985: Large Deviations and Applications. SIAM, 75 pp.

  • Weare, J., 2007: Bridge path sampling and filtering of stochastic and ordinary differential equations. Ph.D. thesis, University of California, Berkeley, 72 pp.

  • Weare, J., 2009: Particle filtering with path sampling and an application to a bimodal ocean current model. J. Comput. Phys., 228, 43124331.

    • Search Google Scholar
    • Export Citation
  • Weinan, E., W. Ren, and E. Vanden-Eijnden, 2004: Minimum action method for the study of rare events. Commun. Pure Appl. Math.,57 (5), 637–656, doi:10.1002/cpa.20005.

  • Yang, S., M. Corazza, A. Carrassi, E. Kalnay, and T. Miyoshi, 2009: Comparison of ensemble-based and variational-based data assimilation schemes in a quasi-geostrophic model. Mon. Wea. Rev., 137, 693709.

    • Search Google Scholar
    • Export Citation
  • Yoshida, K., 1961: On the variations of Kuroshio and cold water mass of Enshu Nada. Hydrogr. Bull., 67, 5457.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 216 74 7
PDF Downloads 106 37 5