Editorial Type:
Article
Article Type:
Research Article

On Serial Observation Processing in Localized Ensemble Kalman Filters

Lars Nerger Alfred Wegener Institute, Helmholtz Center for Polar and Marine Research, Bremerhaven, Germany

Search for other papers by Lars Nerger in
Current site
Google Scholar
PubMed Close
Print Publication:
01 May 2015
Collections:
Sixth WMO Data Assimilation Symposium
DOI:
https://doi.org/10.1175/MWR-D-14-00182.1
Page(s):
1554–1567
Restricted access

Abstract

Ensemble square root filters can either assimilate all observations that are available at a given time at once, or assimilate the observations in batches or one at a time. For large-scale models, the filters are typically applied with a localized analysis step. This study demonstrates that the interaction of serial observation processing and localization can destabilize the analysis process, and it examines under which conditions the instability becomes significant. The instability results from a repeated inconsistent update of the state error covariance matrix that is caused by the localization. The inconsistency is present in all ensemble Kalman filters, except for the classical ensemble Kalman filter with perturbed observations. With serial observation processing, its effect is small in cases when the assimilation changes the ensemble of model states only slightly. However, when the assimilation has a strong effect on the state estimates, the interaction of localization and serial observation processing can significantly deteriorate the filter performance. In realistic large-scale applications, when the assimilation changes the states only slightly and when the distribution of the observations is irregular and changing over time, the instability is likely not significant.

Corresponding author address: Lars Nerger, Alfred Wegener Institute, Helmholtz Center for Polar and Marine Research, Am Handelshafen 12, 27570 Bremerhaven, Germany. E-mail: lars.nerger@awi.de

This article is included in the Sixth WMO Data Assimilation Symposium Special Collection.

Abstract

Ensemble square root filters can either assimilate all observations that are available at a given time at once, or assimilate the observations in batches or one at a time. For large-scale models, the filters are typically applied with a localized analysis step. This study demonstrates that the interaction of serial observation processing and localization can destabilize the analysis process, and it examines under which conditions the instability becomes significant. The instability results from a repeated inconsistent update of the state error covariance matrix that is caused by the localization. The inconsistency is present in all ensemble Kalman filters, except for the classical ensemble Kalman filter with perturbed observations. With serial observation processing, its effect is small in cases when the assimilation changes the ensemble of model states only slightly. However, when the assimilation has a strong effect on the state estimates, the interaction of localization and serial observation processing can significantly deteriorate the filter performance. In realistic large-scale applications, when the assimilation changes the states only slightly and when the distribution of the observations is irregular and changing over time, the instability is likely not significant.

Corresponding author address: Lars Nerger, Alfred Wegener Institute, Helmholtz Center for Polar and Marine Research, Am Handelshafen 12, 27570 Bremerhaven, Germany. E-mail: lars.nerger@awi.de

This article is included in the Sixth WMO Data Assimilation Symposium Special Collection.

Save
Cover Monthly Weather Review
Monthly Weather Review

  • Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884–2903, doi:10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., 2003: A local least squares framework for ensemble filtering. Mon. Wea. Rev., 131, 634–642, doi:10.1175/1520-0493(2003)131<0634:ALLSFF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., T. Hoar, K. Raeder, H. Liu, N. Collings, R. Torn, and A. Arellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 1283–1296, doi:10.1175/2009BAMS2618.1.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420–436, doi:10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Burgers, G., P. J. van Leeuwen, and G. Evensen, 1998: On the analysis scheme in the ensemble Kalman filter. Mon. Wea. Rev., 126, 1719–1724, doi:10.1175/1520-0493(1998)126<1719:ASITEK>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 10 143–10 162, doi:10.1029/94JC00572.

    • Search Google Scholar
    • Export Citation

  • Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723–757, doi:10.1002/qj.49712555417.

    • Search Google Scholar
    • Export Citation

  • Greybush, S. J., E. Kalnay, T. Miyoshi, K. Ide, and B. R. Hunt, 2011: Balance and ensemble Kalman filter localization techniques. Mon. Wea. Rev., 139, 511–522, doi:10.1175/2010MWR3328.1.

    • Search Google Scholar
    • Export Citation

  • Holland, B., and X. Wang, 2013: Effects of sequential or simultaneous assimilation of observations and localization methods on the performance of the ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 139, 758–770, doi:10.1002/qj.2006.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796–811, doi:10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123–137, doi:10.1175/1520-0493(2001)129<0123:ASEKFF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112–126, doi:10.1016/j.physd.2006.11.008.

    • Search Google Scholar
    • Export Citation

  • Janjić, T., L. Nerger, A. Albertella, J. Schröter, and S. Skachko, 2011: On domain localization in ensemble based Kalman filter algorithms. Mon. Wea. Rev., 139, 2046–2060, doi:10.1175/2011MWR3552.1.

    • Search Google Scholar
    • Export Citation

  • Lawson, W. G., and J. A. Hansen, 2004: Implications of stochastic and deterministic filters as ensemble-based data assimilation methods in varying regimes of error growth. Mon. Wea. Rev., 132, 1966–1981, doi:10.1175/1520-0493(2004)132<1966:IOSADF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E. N., 1996: Predictability—A problem partly solved. Proc. Seminar on Predictability, Reading, United Kingdom, ECMWF, 1–18.

  • Lorenz, E. N., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399–414, doi:10.1175/1520-0469(1998)055<0399:OSFSWO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Losa, S. N., S. Danilov, J. Schröter, T. Janjić, L. Nerger, and F. Janssen, 2014: Assimilating NOAA SST data into BSH operational circulation model for the North and Baltic Seas: Part 2. Sensitivity of the forecast’s skill to the prior model error statistics. J. Mar. Syst., 129, 259–270, doi:10.1016/j.jmarsys.2013.06.011.

    • Search Google Scholar
    • Export Citation

  • Maybeck, P. S., 1979: Stochastic Models, Estimation, and Control. Vol. 1. Academic Press, 444 pp.

  • Miyoshi, T., and S. Yamane, 2007: Local ensemble transform Kalman filter with an AGCM at a T159/L48 resolution. Mon. Wea. Rev., 135, 3841–3861, doi:10.1175/2007MWR1873.1.

    • Search Google Scholar
    • Export Citation

  • Nerger, L., and W. Hiller, 2013: Software for ensemble-based data assimilation systems —Implementation strategies and scalability. Comput. Geosci., 55, 110–118, doi:10.1016/j.cageo.2012.03.026.

    • Search Google Scholar
    • Export Citation

  • Nerger, L., W. Hiller, and J. Schröter, 2005: PDAF—The Parallel Data Assimilation Framework: Experiences with Kalman filtering. Use of High Performance Computing in Meteorology—Proceedings of the 11th ECMWF Workshop, W. Zwieflhofer and G. Mozdzynski, Eds., World Scientific, 63–83.

  • Nerger, L., T. Janjić, J. Schröter, and W. Hiller, 2012a: A regulated localization scheme for ensemble-based Kalman filters. Quart. J. Roy. Meteor. Soc., 138, 802–812, doi:10.1002/qj.945.

    • Search Google Scholar
    • Export Citation

  • Nerger, L., T. Janjić, J. Schröter, and W. Hiller, 2012b: A unification of ensemble square root Kalman filters. Mon. Wea. Rev., 140, 2335–2345, doi:10.1175/MWR-D-11-00102.1.

    • Search Google Scholar
    • Export Citation

  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data asimilation. Tellus, 56A, 415–428, doi:10.1111/j.1600-0870.2004.00076.x.

    • Search Google Scholar
    • Export Citation

  • Pham, D. T., 2001: Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Wea. Rev., 129, 1194–1207, doi:10.1175/1520-0493(2001)129<1194:SMFSDA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Pham, D. T., J. Verron, and L. Gourdeau, 1998a: Singular evolutive Kalman filters for data assimilation in oceanography. C. R. Acad. Sci., Ser. II,326 (4), 255–260.

    • Search Google Scholar
    • Export Citation

  • Pham, D. T., J. Verron, and M. C. Roubaud, 1998b: A singular evolutive extended Kalman filter for data assimilation in oceanography. J. Mar. Syst., 16, 323–340, doi:10.1016/S0924-7963(97)00109-7.

    • Search Google Scholar
    • Export Citation

  • Sakov, P., and P. R. Oke, 2008: Implications of the form of the ensemble transformation in the ensemble square root filters. Mon. Wea. Rev., 136, 1042–1053, doi:10.1175/2007MWR2021.1.

    • Search Google Scholar
    • Export Citation

  • Sakov, P., G. Evensen, and L. Bertino, 2010: Asynchronous data assimilation with the EnKF. Tellus, 62A, 24–29, doi:10.1111/j.1600-0870.2009.00417.x.

    • Search Google Scholar
    • Export Citation

  • Sakov, P., F. Counillon, L. Bertino, K. A. Lisaeter, P. R. Oke, and A. Korablev, 2012: TOPAZ4: An ocean-sea ice data assimilation system for the North Atlantic and Arctic. Ocean Sci., 8, 633–656, doi:10.5194/os-8-633-2012.

    • Search Google Scholar
    • Export Citation

  • Tippett, M. K., J. L. Anderson, C. H. Bishop, T. M. Hamill, and J. S. Whitaker, 2003: Ensemble square root filters. Mon. Wea. Rev., 131, 1485–1490, doi:10.1175/1520-0493(2003)131<1485:ESRF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Whitaker, J. S., and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1927, doi:10.1175/1520-0493(2002)130<1913:EDAWPO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP global forecast system. Mon. Wea. Rev., 136, 463–482, doi:10.1175/2007MWR2018.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 8751 8134 148
PDF Downloads 469 134 3