Using the (3N)-Dimensional Generalized Lorenz Systems as a Testbed for Data Assimilation: The Ensemble Kalman Filter

Sungju Moon aSchool of Earth and Environmental Sciences, Seoul National University, Seoul, South Korea

Search for other papers by Sungju Moon in
Current site
Google Scholar
PubMed
Close
and
Jong-Jin Baik aSchool of Earth and Environmental Sciences, Seoul National University, Seoul, South Korea

Search for other papers by Jong-Jin Baik in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

The feasibility of using a (3N)-dimensional generalization of the Lorenz system in testing a traditional implementation of the ensemble Kalman filter is explored through numerical experiments. The generalization extends the Lorenz system, known as the Lorenz ’63 model, into a (3N)-dimensional nonlinear system for any positive integer N. Because the extension involves inclusion of additional wavenumber modes, raising the dimension allows the system to resolve smaller-scale motions, a unique characteristic of the present generalization that can be relevant to real modeling scenarios. Model imperfections are simulated by assuming a high-dimensional generalized Lorenz system as the true system and a generalized system of dimension less than or equal to the dimension of the true system as the model system. Different scenarios relevant to data assimilation practices are simulated by varying the dimensional differences between the model and true systems, ensemble size, and observation frequency and accuracy. It is suggested that the present generalization of the Lorenz system is an interesting and flexible tool for evaluating the effectiveness of data assimilation methods and a meaningful addition to the portfolio of testbed systems that includes the Lorenz ’63 and ’96 models, especially considering its relationship with the Lorenz ’63 model. The results presented in this study can serve as useful benchmarks for testing other data assimilation methods besides the ensemble Kalman filter.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jong-Jin Baik, jjbaik@snu.ac.kr

Abstract

The feasibility of using a (3N)-dimensional generalization of the Lorenz system in testing a traditional implementation of the ensemble Kalman filter is explored through numerical experiments. The generalization extends the Lorenz system, known as the Lorenz ’63 model, into a (3N)-dimensional nonlinear system for any positive integer N. Because the extension involves inclusion of additional wavenumber modes, raising the dimension allows the system to resolve smaller-scale motions, a unique characteristic of the present generalization that can be relevant to real modeling scenarios. Model imperfections are simulated by assuming a high-dimensional generalized Lorenz system as the true system and a generalized system of dimension less than or equal to the dimension of the true system as the model system. Different scenarios relevant to data assimilation practices are simulated by varying the dimensional differences between the model and true systems, ensemble size, and observation frequency and accuracy. It is suggested that the present generalization of the Lorenz system is an interesting and flexible tool for evaluating the effectiveness of data assimilation methods and a meaningful addition to the portfolio of testbed systems that includes the Lorenz ’63 and ’96 models, especially considering its relationship with the Lorenz ’63 model. The results presented in this study can serve as useful benchmarks for testing other data assimilation methods besides the ensemble Kalman filter.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jong-Jin Baik, jjbaik@snu.ac.kr
Save
  • Afraimovich, V. S., and N. N. Verichev, 1986: Stochastic synchronization of oscillation in dissipative systems. Radiophys. Quantum Electron., 29, 795803, https://doi.org/10.1007/BF01034476.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 28842903, https://doi.org/10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 2007: Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter. Physica D, 230, 99111, https://doi.org/10.1016/j.physd.2006.02.011.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 2012: Localization and sampling error correction in ensemble Kalman filter data assimilation. Mon. Wea. Rev., 140, 23592371, https://doi.org/10.1175/MWR-D-11-00013.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 27412758, https://doi.org/10.1175/1520-0493(1999)127<2741:AMCIOT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420436, https://doi.org/10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bocquet, M., and A. Farchi, 2019: On the consistency of the local ensemble square root Kalman filter perturbation update. Tellus, 71A, 1613142, https://doi.org/10.1080/16000870.2019.1613142.

    • Search Google Scholar
    • Export Citation
  • Bröcker, J., and I. G. Szendro, 2012: Sensitivity and out-of-sample error in continuous time data assimilation. Quart. J. Roy. Meteor. Soc., 138, 785801, https://doi.org/10.1002/qj.940.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Burgers, G., P. J. Van Leeuwen, and G. Evensen, 1998: Analysis scheme in the ensemble Kalman filter. Mon. Wea. Rev., 126, 17191724, https://doi.org/10.1175/1520-0493(1998)126<1719:ASITEK>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carrassi, A., M. Bocquet, A. Hannart, and M. Ghil, 2016: Estimating model evidence using data assimilation. Quart. J. Roy. Meteor. Soc., 143, 866880, https://doi.org/10.1002/qj.2972.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Elsheikh, A. H., I. Hoteit, and M. F. Wheeler, 2013: A nested sampling particle filter for nonlinear data assimilation. Quart. J. Roy. Meteor. Soc., 140, 16401653, https://doi.org/10.1002/qj.2245.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 143162, https://doi.org/10.1029/94JC00572.

    • Search Google Scholar
    • Export Citation
  • Evensen, G., 1997: Advanced data assimilation for strongly nonlinear dynamics. Mon. Wea. Rev., 125, 13421354, https://doi.org/10.1175/1520-0493(1997)125<1342:ADAFSN>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Evensen, G., 2003: The ensemble Kalman filter: Theoretical formulation and practical implementation. Ocean Dyn., 53, 343367, https://doi.org/10.1007/s10236-003-0036-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Evensen, G., 2006: Data Assimilation: The Ensemble Kalman Filter. Springer, 330 pp.

  • Evensen, G., and P. J. Van Leeuwen, 1996: Assimilation of Geosat altimeter data for the Agulhas Current using the ensemble Kalman filter with a quasigeostrophic model. Mon. Wea. Rev., 124, 8596, https://doi.org/10.1175/1520-0493(1996)124<0085:AOGADF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feng, J., R. Ding, D. Liu, and J. Li, 2014: The application of nonlinear local Lyapunov vectors to ensemble predictions in Lorenz systems. J. Atmos. Sci., 71, 35543567, https://doi.org/10.1175/JAS-D-13-0270.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goodliff, M., J. Amezcua, and P. J. Van Leeuwen, 2015: Comparing hybrid data assimilation methods on the Lorenz 1963 model with increasing non-linearity. Tellus, 67A, 26928, https://doi.org/10.3402/tellusa.v67.26928.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and J. S. Whitaker, 2005: Accounting for the error due to unresolved scales in ensemble data assimilation: A comparison of different approaches. Mon. Wea. Rev., 133, 31323147, https://doi.org/10.1175/MWR3020.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 27762790, https://doi.org/10.1175/1520-0493(2001)129<2776:DDFOBE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hamilton, F., T. Berry, and T. Sauer, 2019: Correcting observation model error in data assimilation. Chaos, 29, 053102, https://doi.org/10.1063/1.5087151.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Herrera, S., D. Pazó, J. Fernández, and M. A. Rodríguez, 2011: The role of large-scale spatial patterns in the chaotic amplification of perturbations in a Lorenz’96 model. Tellus, 63A, 978990, https://doi.org/10.1111/j.1600-0870.2011.00545.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796811, https://doi.org/10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123137, https://doi.org/10.1175/1520-0493(2001)129<0123:ASEKFF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., H. L. Mitchell, G. Pelerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133, 604620, https://doi.org/10.1175/MWR-2864.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112126, https://doi.org/10.1016/j.physd.2006.11.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and Predictability. 1st ed. Cambridge University Press, 341 pp.

    • Crossref
    • Export Citation
  • Kalnay, E., 2010: Ensemble Kalman filter: Current status and potential. Data Assimilation, W. Lahoz, B. Khattatov, and R. Menard, Eds., Springer-Verlag, 69–92.

    • Crossref
    • Export Citation
  • Lei, L., D. R. Stauffer, and A. Deng, 2012a: A hybrid nudging-ensemble Kalman filter approach to data assimilation in WRF/DART. Quart. J. Roy. Meteor. Soc., 138, 20662078, https://doi.org/10.1002/qj.1939.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lei, L., D. R. Stauffer, S. E. Haupt, and G. S. Young, 2012b: A hybrid nudging-ensemble Kalman filter approach to data assimilation. Part I: Application in the Lorenz system. Tellus, 64A, 18484, https://doi.org/10.3402/tellusa.v64i0.18484.

    • Search Google Scholar
    • Export Citation
  • Li, H., E. Kalnay, and T. Miyoshi, 2009: Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 523533, https://doi.org/10.1002/qj.371.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWP—A comparison with 4D-Var. Quart. J. Roy. Meteor. Soc., 129, 31833203, https://doi.org/10.1256/qj.02.132.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130141, https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1996: Predictability—A problem partly solved. Proc. ECMWF Seminar on Predictability, Vol. 1, Reading, United Kingdom, ECMWF, 1–18.

  • Lorenz, E. N., 2006: Predictability—A problem partly solved. Predictability of Weather and Climate, T. Palmer and R. Hagedorn, Eds., Cambridge University Press, 40–58.

    • Crossref
    • Export Citation
  • Lorenz, E. N., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399414, https://doi.org/10.1175/1520-0469(1998)055<0399:OSFSWO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moon, S., B.-S. Han, J. Park, J. M. Seo, and J.-J. Baik, 2017: Periodicity and chaos of high-order Lorenz systems. Int. J. Bifurcation Chaos, 27, 1750176, https://doi.org/10.1142/S0218127417501760.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moon, S., J. M. Seo, and J.-J. Baik, 2020: High-dimensional generalizations of the Lorenz system and implications for predictability. Phys. Scripta, 95, 085209, https://doi.org/10.1088/1402-4896/ab9d3e.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moon, S., J.-J. Baik, and J. M. Seo, 2021: Chaos synchronization in generalized Lorenz systems and an application to image encryption. Commun. Nonlinear Sci. Numer. Simul., 96, 105708, https://doi.org/10.1016/j.cnsns.2021.105708.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moore, E. H., 1920: On the reciprocal of the general algebraic matrix. Bull. Amer. Math. Soc., 26, 394395.

  • Orrell, D., L. Smith, J. Barkmeijer, and T. N. Palmer, 2001: Model error in weather forecasting. Nonlinear Processes Geophys., 8, 357371, https://doi.org/10.5194/npg-8-357-2001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Osinga, H. M., 2018: Understanding the geometry of dynamics: The stable manifold of the Lorenz system. J. Roy. Soc. N. Z., 48, 203214, https://doi.org/10.1080/03036758.2018.1434802.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Park, J., S. Moon, J. M. Seo, and J.-J. Baik, 2021: Systematic comparison between the generalized Lorenz equations and DNS in the two-dimensional Rayleigh–Bénard convection. Chaos, 31, 073119, https://doi.org/10.1063/5.0051482.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pecora, L. M., and T. L. Carroll, 1990: Synchronization in chaotic systems. Phys. Rev. Lett., 64, 821824, https://doi.org/10.1103/PhysRevLett.64.821.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Penrose, R., 1955: A generalized inverse for matrices. Math. Proc. Cambridge Philos. Soc., 51, 406413, https://doi.org/10.1017/S0305004100030401.

    • Search Google Scholar
    • Export Citation
  • Privé, N. C., and R. M. Errico, 2013: The role of model and initial condition error in numerical weather forecasting investigated with an observing system simulation experiment. Tellus, 65A, 21740, https://doi.org/10.3402/tellusa.v65i0.21740.

    • Search Google Scholar
    • Export Citation
  • Pu, Z., and J. Hacker, 2009: Ensemble-based Kalman filters in strongly nonlinear dynamics. Adv. Atmos. Sci., 26, 373380, https://doi.org/10.1007/s00376-009-0373-9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shen, B.-W., 2014: Nonlinear feedback in a five-dimensional Lorenz model. J. Atmos. Sci., 71, 17011723, https://doi.org/10.1175/JAS-D-13-0223.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shen, B.-W., 2015: Nonlinear feedback in a six-dimensional Lorenz model: Impact of an additional heating term. Nonlinear Processes Geophys., 22, 749764, https://doi.org/10.5194/npg-22-749-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shlyaeva, A., J. S. Whitaker, and C. Snyder, 2019: Model-space localization in serial ensemble filters. J. Adv. Model. Earth Syst., 11, 16271636, https://doi.org/10.1029/2018MS001514.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sparrow, C., 1982: The Lorenz Equations: Bifurcations, Chaos, and Strange Attractors. Springer-Verlag, 284 pp.

    • Crossref
    • Export Citation
  • Talagrand, O., and P. Courtier, 1987: Variational assimilation of meteorological observations with the adjoint vorticity equation. I: Theory. Quart. J. Roy. Meteor. Soc., 113, 13111328, https://doi.org/10.1002/qj.49711347812.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thépaut, J.-N., and P. Courtier, 1991: Four-dimensional variational data assimilation using the adjoint of a multilevel primitive-equation model. Quart. J. Roy. Meteor. Soc., 117, 12251254, https://doi.org/10.1002/qj.49711750206.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thompson, P. D., 1957: Uncertainty of initial state as a factor in the predictability of large scale atmospheric flow patterns. Tellus, 9, 275295, https://doi.org/10.1111/j.2153-3490.1957.tb01885.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tong, M., and M. Xue, 2005: Ensemble Kalman filter assimilation of Doppler radar data with a compressible nonhydrostatic model: OSS experiments. Mon. Wea. Rev., 133, 17891807, https://doi.org/10.1175/MWR2898.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tucker, W., 1999: The Lorenz attractor exists. C. R. Acad. Sci., Ser. 1 Math., 328, 11971202, https://doi.org/10.1016/S0764-4442(99)80439-X.

    • Search Google Scholar
    • Export Citation
  • Van Leeuwen, P. J., 1999: Comments on “Data assimilation using an ensemble Kalman filter technique.” Mon. Wea. Rev., 127, 13741377, https://doi.org/10.1175/1520-0493(1999)127<1374:CODAUA>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 19131924, https://doi.org/10.1175/1520-0493(2002)130<1913:EDAWPO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yang, S.-C., and Coauthors, 2006: Data assimilation as synchronization of truth and model: Experiments with the three-variable Lorenz system. J. Atmos. Sci., 63, 23402354, https://doi.org/10.1175/JAS3739.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yang, S.-C., E. Kalnay, and B. Hunt, 2012: Handling nonlinearity in an ensemble Kalman filter: Experiments with the three-variable Lorenz model. Mon. Wea. Rev., 140, 26282646, https://doi.org/10.1175/MWR-D-11-00313.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yeong, H. C., R. T. Beeson, N. S. Namachchivaya, and N. Perkowski, 2020: Particle filters with nudging in multiscale chaotic systems: With application to the Lorenz’96 atmospheric model. J. Nonlinear Sci., 30, 15191552, https://doi.org/10.1007/s00332-020-09616-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yoden, S., 2007: Atmospheric predictability. J. Meteor. Soc. Japan, 85B, 77102, https://doi.org/10.2151/jmsj.85B.77.

  • Zhang, Y., K. Ide, and E. Kalnay, 2015: Bred vectors of the Lorenz63 system. Adv. Atmos. Sci., 32, 15331538, https://doi.org/10.1007/s00376-015-4275-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 232 0 0
Full Text Views 244 151 10
PDF Downloads 244 122 6