The Hybrid Local Ensemble Transform Kalman Filter

Stephen G. Penny Department of Atmospheric and Oceanic Science, University of Maryland, College Park, and NOAA/NWS/NCEP, College Park, Maryland

Search for other papers by Stephen G. Penny in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Hybrid data assimilation methods combine elements of ensemble Kalman filters (EnKF) and variational methods. While most approaches have focused on augmenting an operational variational system with dynamic error covariance information from an ensemble, this study takes the opposite perspective of augmenting an operational EnKF with information from a simple 3D variational data assimilation (3D-Var) method. A class of hybrid methods is introduced that combines the gain matrices of the ensemble and variational methods, rather than linearly combining the respective background error covariances. A hybrid local ensemble transform Kalman filter (Hybrid-LETKF) is presented in two forms: 1) a traditionally motivated Hybrid/Covariance-LETKF that combines the background error covariance matrices of LETKF and 3D-Var, and 2) a simple-to-implement algorithm called the Hybrid/Mean-LETKF that falls into the new class of hybrid gain methods. Both forms improve analysis errors when using small ensemble sizes and low observation coverage versus either LETKF or 3D-Var used alone. The results imply that for small ensemble sizes, allowing a solution to be found outside of the space spanned by ensemble members provides robustness in both hybrid methods compared to LETKF alone. Finally, the simplicity of the Hybrid/Mean-LETKF design implies that this algorithm can be applied operationally while requiring only minor modifications to an existing operational 3D-Var system.

Corresponding author address: Stephen G. Penny, Dept. of Atmospheric and Oceanic Science, University of Maryland, College Park, 2431 Computer and Space Science Bldg., College Park, MD 20742. E-mail: Steve.Penny@noaa.gov

Abstract

Hybrid data assimilation methods combine elements of ensemble Kalman filters (EnKF) and variational methods. While most approaches have focused on augmenting an operational variational system with dynamic error covariance information from an ensemble, this study takes the opposite perspective of augmenting an operational EnKF with information from a simple 3D variational data assimilation (3D-Var) method. A class of hybrid methods is introduced that combines the gain matrices of the ensemble and variational methods, rather than linearly combining the respective background error covariances. A hybrid local ensemble transform Kalman filter (Hybrid-LETKF) is presented in two forms: 1) a traditionally motivated Hybrid/Covariance-LETKF that combines the background error covariance matrices of LETKF and 3D-Var, and 2) a simple-to-implement algorithm called the Hybrid/Mean-LETKF that falls into the new class of hybrid gain methods. Both forms improve analysis errors when using small ensemble sizes and low observation coverage versus either LETKF or 3D-Var used alone. The results imply that for small ensemble sizes, allowing a solution to be found outside of the space spanned by ensemble members provides robustness in both hybrid methods compared to LETKF alone. Finally, the simplicity of the Hybrid/Mean-LETKF design implies that this algorithm can be applied operationally while requiring only minor modifications to an existing operational 3D-Var system.

Corresponding author address: Stephen G. Penny, Dept. of Atmospheric and Oceanic Science, University of Maryland, College Park, 2431 Computer and Space Science Bldg., College Park, MD 20742. E-mail: Steve.Penny@noaa.gov
Save
  • Barker, D. M., 1999: The use of synoptic-dependent error structure in 3DVAR. Var Scientific Development Paper 25, Met Office Tech. Rep., 2 pp. [Available from Met Office, FitzRoy Rd., Exeter, Devon EX1 3PB, United Kingdom.]

  • Bishop, C. H., and E. A. Satterfield, 2013: Hidden error variance theory. Part I: Exposition and analytic model. Mon. Wea. Rev., 141, 14541468, doi:10.1175/MWR-D-12-00118.1.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420436, doi:10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Brett, C. E. A., K. F. Lam, K. J. H. Law, D. S. McCormick, M. R. Scott, and A. M. Stuart, 2013: Accuracy and stability of filters for dissipative PDEs. Physica D, 245, 3445, doi:10.1016/j.physd.2012.11.005.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., 2005: Ensemble-derived stationary and flow-dependent background error covariances: Evaluation in a quasi-operational NWP setting. Quart. J. Roy. Meteor. Soc., 131, 10131043, doi:10.1256/qj.04.15.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010a: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part I: Description and single-observation experiments. Mon. Wea. Rev., 138, 15501566, doi:10.1175/2009MWR3157.1.

    • Search Google Scholar
    • Export Citation
  • Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010b: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part II: One-month experiments with real observations. Mon. Wea. Rev., 138, 15671586, doi:10.1175/2009MWR3158.1.

    • Search Google Scholar
    • Export Citation
  • Derber, J., and A. Rosati, 1989: A global oceanic data assimilation system. J. Phys. Oceanogr., 19, 13331347, doi:10.1175/1520-0485(1989)019<1333:AGODAS>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Fehlberg, E., 1970: Classical Runge-Kutta fourth-order and lower order formulas with step size control and their application to heat conduction problems. Computing,6, 61–71.

  • Greybush, S. J., E. Kalnay, T. Miyoshi, K. Ide, and B. R. Hunt, 2011: Balance and ensemble Kalman filter localization techniques. Mon. Wea. Rev., 139, 511522, doi:10.1175/2010MWR3328.1.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., and C. Snyder, 2000: A hybrid ensemble Kalman filter-3D variational analysis scheme. Mon. Wea. Rev., 128, 29052919, doi:10.1175/1520-0493(2000)128<2905:AHEKFV>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Harlim, J., and A. J. Majda, 2010: Catastrophic filter divergence in filtering nonlinear dissipative systems. Commun. Math. Sci., 8, 2743, doi:10.4310/CMS.2010.v8.n1.a3.

    • Search Google Scholar
    • Export Citation
  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112126, doi:10.1016/j.physd.2006.11.008.

    • Search Google Scholar
    • Export Citation
  • Kalnay, E., 2003: Data assimilation. Atmospheric Modeling, Data Assimilation and Predictability,Cambridge University Press, 136–204.

  • Kalnay, E., and Z. Toth, 1994: Removing growing errors in the analysis. Preprints, 10th Conf. on Numerical Weather Prediction, Portland, OR, Amer. Meteor. Soc., 212215.

  • Karimi, A., and M. R. Paul, 2010: Extensive chaos in the Lorenz-96 model. Chaos, 20, 043105, doi:10.1063/1.3496397.

  • Kleist, D. T., 2012: An evaluation of hybrid variational-ensemble data assimilation for the NCEP GFS. Ph.D. dissertation, University of Maryland, College Park, 149 pp.

  • Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWP—A comparison with 4D-Var. Quart. J. Roy. Meteor. Soc., 129, 31833203, doi:10.1256/qj.02.132.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1996: Predictability—A problem partly solved. Proceedings of a Seminar Held at ECMWF on Predictability, ECMWF Seminar Proceedings, Vol. 1, ECMWF, 118.

  • Lorenz, E. N., 2005: Designing chaotic models. J. Atmos. Sci., 62, 15741587, doi:10.1175/JAS3430.1.

  • Lorenz, E. N., 2006: Regimes in simple systems. J. Atmos. Sci., 63, 20562073, doi:10.1175/JAS3727.1.

  • Messner, J., 2009: Probabilistic forecasting using analogs in the idealized Lorenz96 setting. M.S. thesis, Dept. of Meteorology and Geophysics, University of Innsbruck, 59 pp.

  • Orrell, D., 2003: Model error and predictability over different timescales in the Lorenz ’96 systems. J. Atmos. Sci., 60, 22192228, doi:10.1175/1520-0469(2003)060<2219:MEAPOD>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415428, doi:10.1111/j.1600-0870.2004.00076.x.

    • Search Google Scholar
    • Export Citation
  • Pazo, D., I. G. Szendro, J. M. Lopez, and M. A. Rodriguez, 2008: Structure of characteristic Lyapunov vectors in spatiotemporal chaos. Phys. Rev., 78E, 016209, doi:10.1103/PhysRevE.78.016209.

    • Search Google Scholar
    • Export Citation
  • Penny, S. G., 2011: Data assimilation of the global ocean using the 4D local ensemble transform Kalman filter (4D-LETKF) and the Modular Ocean Model. Ph.D. dissertation, University of Maryland, College Park, 141 pp.

  • Penny, S. G., E. Kalnay, J. A. Carton, B. R. Hunt, K. Ide, T. Miyoshi, and G. Chepurin, 2013: The running-in-place algorithm applied to a global ocean general circulation model. Nonlinear Processes Geophys.,20, 1031–1046, doi:10.5194/npg-20-1031-2013.

  • Trevisan, A., and L. Palatella, 2011: On the Kalman filter error covariance collapse into the unstable subspace. Nonlinear Processes Geophys., 18, 243250, doi:10.5194/npg-18-243-2011.

    • Search Google Scholar
    • Export Citation
  • Wang, X., 2010: Incorporating ensemble covariance in the gridpoint statistical interpolation variational minimization: A mathematical framework. Mon. Wea. Rev., 138, 29902995, doi:10.1175/2010MWR3245.1.

    • Search Google Scholar
    • Export Citation
  • Wang, X., T. M. Hamill, J. S. Whitaker, and C. H. Bishop, 2007a: A comparison of hybrid ensemble transform Kalman filter–optimum interpolation and ensemble square root filter analysis schemes. Mon. Wea. Rev., 135, 10551076, doi:10.1175/MWR3307.1.

    • Search Google Scholar
    • Export Citation
  • Wang, X., C. Snyder, and T. M. Hamill, 2007b: On the theoretical equivalence of differently proposed ensemble–3D-VAR hybrid analysis schemes. Mon. Wea. Rev., 135, 222227, doi:10.1175/MWR3282.1.

    • Search Google Scholar
    • Export Citation
  • Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008a: A hybrid ETKF–3DVAR data assimilation scheme for the WRF Model. Part I: Observing system simulation experiment. Mon. Wea. Rev., 136, 51165131, doi:10.1175/2008MWR2444.1.

    • Search Google Scholar
    • Export Citation
  • Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008b: A hybrid ETKF–3DVAR data assimilation scheme for the WRF Model. Part II: Real observation experiments. Mon. Wea. Rev., 136, 51325147, doi:10.1175/2008MWR2445.1.

    • Search Google Scholar
    • Export Citation
  • Wang, X., D. Parrish, D. Kleist, and J. Whitaker, 2013: GSI 3DVar-based ensemble–variational hybrid data assimilation for NCEP Global Forecast System: Single-resolution experiments. Mon. Wea. Rev., 141, 40984117, doi:10.1175/MWR-D-12-00141.1.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2005: Effects of stochastic parametrizations in the Lorenz ’96 system. Quart. J. Roy. Meteor. Soc., 131, 389407, doi:10.1256/qj.04.03.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 2006: Comparison of ensemble-MOS methods in the Lorenz ’96 setting. Meteor. Appl., 13, 243256, doi:10.1017/S1350482706002192.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 550 167 4
PDF Downloads 532 185 6