Editorial Type:
Article
Article Type:
Research Article

Accounting for Model Error from Unresolved Scales in Ensemble Kalman Filters by Stochastic Parameterization

Fei Lu Department of Mathematics, University of California, Berkeley, and Lawrence Berkeley National Laboratory, Berkeley, California

Search for other papers by Fei Lu in
Current site
Google Scholar
PubMed Close
,
Xuemin Tu Department of Mathematics, University of Kansas, Lawrence, Kansas

Search for other papers by Xuemin Tu in
Current site
Google Scholar
PubMed Close
, and
Alexandre J. Chorin Department of Mathematics, University of California, Berkeley, and Lawrence Berkeley National Laboratory, Berkeley, California

Search for other papers by Alexandre J. Chorin in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Sep 2017
DOI:
https://doi.org/10.1175/MWR-D-16-0478.1
Page(s):
3709–3723
Restricted access

Abstract

The use of discrete-time stochastic parameterization to account for model error due to unresolved scales in ensemble Kalman filters is investigated by numerical experiments. The parameterization quantifies the model error and produces an improved non-Markovian forecast model, which generates high quality forecast ensembles and improves filter performance. Results are compared with the methods of dealing with model error through covariance inflation and localization (IL), using as an example the two-layer Lorenz-96 system. The numerical results show that when the ensemble size is sufficiently large, the parameterization is more effective in accounting for the model error than IL; if the ensemble size is small, IL is needed to reduce sampling error, but the parameterization further improves the performance of the filter. This suggests that in real applications where the ensemble size is relatively small, the filter can achieve better performance than pure IL if stochastic parameterization methods are combined with IL.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Fei Lu, feilu@berkeley.edu

Abstract

The use of discrete-time stochastic parameterization to account for model error due to unresolved scales in ensemble Kalman filters is investigated by numerical experiments. The parameterization quantifies the model error and produces an improved non-Markovian forecast model, which generates high quality forecast ensembles and improves filter performance. Results are compared with the methods of dealing with model error through covariance inflation and localization (IL), using as an example the two-layer Lorenz-96 system. The numerical results show that when the ensemble size is sufficiently large, the parameterization is more effective in accounting for the model error than IL; if the ensemble size is small, IL is needed to reduce sampling error, but the parameterization further improves the performance of the filter. This suggests that in real applications where the ensemble size is relatively small, the filter can achieve better performance than pure IL if stochastic parameterization methods are combined with IL.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Fei Lu, feilu@berkeley.edu
Save
Cover Monthly Weather Review
Monthly Weather Review

  • Anderson, J. L., 2007a: An adaptive covariance inflation error correction algorithm for ensemble filters. Tellus, 59A, 210–224, doi:10.1111/j.1600-0870.2006.00216.x.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., 2007b: Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter. Physica D, 230, 99–111, doi:10.1016/j.physd.2006.02.011.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72–83, doi:10.1111/j.1600-0870.2008.00361.x.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., 2012: Localization and sampling error correction in ensemble Kalman filter data assimilation. Mon. Wea. Rev., 140, 2359–2371, doi:10.1175/MWR-D-11-00013.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 2741–2758, doi:10.1175/1520-0493(1999)127<2741:AMCIOT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Arnold, H. M., I. M. Moroz, and T. N. Palmer, 2013: Stochastic parametrization and model uncertainty in the Lorenz’96 system. Philos. Trans. Roy. Soc. London, 371A, doi:10.1098/rsta.2011.0479.

    • Search Google Scholar
    • Export Citation

  • Berry, T., and J. Harlim, 2014: Linear theory for filtering nonlinear multiscale systems with model error. Proc. Roy. Soc. London, 470A, doi:10.1098/rspa.2014.0168.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., and D. Hodyss, 2007: Flow-adaptive moderation of spurious ensemble correlations and its use in ensemble-based data assimilation. Quart. J. Roy. Meteor. Soc., 133, 2029–2044, doi:10.1002/qj.169.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Buizza, R., M. Miller, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 125, 2887–2908, doi:10.1002/qj.49712556006.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Burgers, G., P. J. van Leeuwen, and G. Evensen, 1998: Analysis scheme in the ensemble Kalman filter. Mon. Wea. Rev., 126, 1719–1724, doi:10.1175/1520-0493(1998)126<1719:ASITEK>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chekroun, M. D., D. Kondrashov, and M. Ghil, 2011: Predicting stochastic systems by noise sampling, and application to the El Niño-Southern Oscillation. Proc. Natl. Acad. Sci. USA, 108, 11 766–11 771, doi:10.1073/pnas.1015753108.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chorin, A. J., and O. H. Hald, 2013: Stochastic Tools in Mathematics and Science. 3rd ed. Texts in Applied Mathematics, Vol. 58, Springer-Verlag, 200 pp.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chorin, A. J., and F. Lu, 2015: Discrete approach to stochastic parametrization and dimension reduction in nonlinear dynamics. Proc. Natl. Acad. Sci. USA, 112, 9804–9809, doi:10.1073/pnas.1512080112.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chorin, A. J., O. H. Hald, and R. Kupferman, 2000: Optimal prediction and the Mori–Zwanzig representation of irreversible processes. Proc. Natl. Acad. Sci. USA, 97, 2968–2973, doi:10.1073/pnas.97.7.2968.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chorin, A. J., O. H. Hald, and R. Kupferman, 2002: Optimal prediction with memory. Physica D, 166, 239–257, doi:10.1016/S0167-2789(02)00446-3.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chorin, A. J., F. Lu, R. M. Miller, M. Morzfeld, and X. Tu, 2016: Sampling, feasibility, and priors in data assimilation. Discrete Contin. Dyn. Syst., 36A, 4227–4246, doi:10.3934/dcds.2016.36.8i.

    • Search Google Scholar
    • Export Citation

  • Crommelin, D., and E. Vanden-Eijnden, 2008: Subgrid-scale parameterization with conditional Markov chains. J. Atmos. Sci., 65, 2661–2675, doi:10.1175/2008JAS2566.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Danforth, C. M., and E. Kalnay, 2008: Using singular value decomposition to parameterize state-dependent model errors. J. Atmos. Sci., 65, 1467–1478, doi:10.1175/2007JAS2419.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Dee, D. P., and A. M. Da Silva, 1998: Data assimilation in the presence of forecast bias. Quart. J. Roy. Meteor. Soc., 124, 269–295, doi:10.1002/qj.49712454512.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ding, F., and T. Chen, 2005: Identification of Hammerstein nonlinear ARMAX systems. Automatica, 41, 1479–1489, doi:10.1016/j.automatica.2005.03.026.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Doucet, A., M. Briers, and S. Sénécal, 2006: Efficient block sampling strategies for sequential Monte Carlo methods. J. Comput. Graph. Stat., 15, 693–711, doi:10.1198/106186006X142744.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 10 143–10 162, doi:10.1029/94JC00572.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Evensen, G., and P. J. Van Leeuwen, 1996: Assimilation of Geosat altimeter data for the Agulhas current using the ensemble Kalman filter with a quasigeostrophic model. Mon. Wea. Rev., 124, 85–96, doi:10.1175/1520-0493(1996)124<0085:AOGADF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Evensen, G., and P. Van Leeuwen, 2000: An ensemble Kalman smoother for nonlinear dynamics. Mon. Wea. Rev., 128, 1852–1867, doi:10.1175/1520-0493(2000)128<1852:AEKSFN>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Fatkullin, I., and E. Vanden-Eijnden, 2004: A computational strategy for multi-scale systems with applications to Lorenz’96 model. J. Comput. Phys., 200, 605–638, doi:10.1016/j.jcp.2004.04.013.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Furrer, R., and T. Bengtsson, 2007: Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J. Multivariate Anal., 98, 227–255, doi:10.1016/j.jmva.2006.08.003.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723–757, doi:10.1002/qj.49712555417.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Gottwald, G. A., and J. Harlim, 2013: The role of additive and multiplicative noise in filtering complex dynamical systems. Proc. Roy. Soc. London, 469A, doi:10.1098/rspa.2013.0096.

    • Search Google Scholar
    • Export Citation

  • Gottwald, G. A., D. Crommelin, and C. Franzke, 2015: Stochastic climate theory. Nonlinear and Stochastic Climate Dynamics, C. L. E. Franzke and T. J. O’Kane, Eds., Cambridge University Press, 209–240.

    • Crossref
    • Export Citation

  • Hamill, T. M., and J. S. Whitaker, 2005: Accounting for the error due to unresolved scales in ensemble data assimilation: A comparison of different approaches. Mon. Wea. Rev., 133, 3132–3147, doi:10.1175/MWR3020.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 2776–2790, doi:10.1175/1520-0493(2001)129<2776:DDFOBE>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Harlim, J., 2016: Model error in data assimilation. Nonlinear and Stochastic Climate Dynamics, C. L. E. Franzke, and T. J. O’Kane, Eds., Cambridge University Press, 276–317.

    • Crossref
    • Export Citation

  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796–811, doi:10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., H. L. Mitchell, and X. Deng, 2009: Model error representation in an operational ensemble Kalman filter. Mon. Wea. Rev., 137, 2126–2143, doi:10.1175/2008MWR2737.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Kantas, N., A. Doucet, S. S. Singh, and J. M. Maciejowski, 2009: An overview of sequential Monte Carlo methods for parameter estimation in general state-space models. Proc. Symp. on System Identification, Saint-Malo, France, International Federation of Automatic Control. [Available online at http://wwwf.imperial.ac.uk/~nkantas/sysid09_final_normal_format.pdf.]

  • Kelly, D., K. Law, and A. Stuart, 2014: Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time. Nonlinearity, 27, 2579, doi:10.1088/0951-7715/27/10/2579.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Khare, S., J. Anderson, T. Hoar, and D. Nychka, 2008: An investigation into the application of an ensemble Kalman smoother to high-dimensional geophysical systems. Tellus, 60A, 97–112, doi:10.1111/j.1600-0870.2007.00281.x.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Kondrashov, D., M. D. Chekroun, and M. Ghil, 2015: Data-driven non-Markovian closure models. Physica D, 297, 33–55, doi:10.1016/j.physd.2014.12.005.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Kwasniok, F., 2012: Data-based stochastic subgrid-scale parametrization: An approach using cluster-weighted modeling. Philos. Trans. Roy. Soc. London, 370A, 1061–1086, doi:10.1098/rsta.2011.0384.

    • Search Google Scholar
    • Export Citation

  • Lei, J., P. Bickel, and C. Snyder, 2010: Comparison of ensemble Kalman filters under non-Gaussianity. Mon. Wea. Rev., 138, 1293–1306, doi:10.1175/2009MWR3133.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Li, H., E. Kalnay, T. Miyoshi, and C. M. Danforth, 2009: Accounting for model errors in ensemble data assimilation. Mon. Wea. Rev., 137, 3407–3419, doi:10.1175/2009MWR2766.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lorenz, E. N., 1996: Predictability: A problem partly solved. Proc. Seminar on Predictability, Reading, United Kingdom, ECMWF, 1–18.

  • Lu, F., K. K. Lin, and A. J. Chorin, 2016: Comparison of continuous and discrete-time data-based modeling for hypoelliptic systems. Commun. Appl. Math. Comput. Sci., 11, 187–216, doi:10.2140/camcos.2016.11.187.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lu, F., K. K. Lin, and A. J. Chorin, 2017: Data-based stochastic model reduction for the Kuramoto–Sivashinsky equation. Physica D, 340, 46–57, doi:10.1016/j.physd.2016.09.007.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Majda, A. J., and J. Harlim, 2013: Physics constrained nonlinear regression models for time series. Nonlinearity, 26, 201–217, doi:10.1088/0951-7715/26/1/201.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Meng, Z., and F. Zhang, 2007: Tests of an ensemble Kalman filter for mesoscale and regional-scale data assimilation. Part II: Imperfect model experiments. Mon. Wea. Rev., 135, 1403–1423, doi:10.1175/MWR3352.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Mitchell, H. L., and P. L. Houtekamer, 2000: An adaptive ensemble Kalman filter. Mon. Wea. Rev., 128, 416–433, doi:10.1175/1520-0493(2000)128<0416:AAEKF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Mitchell, L., and G. A. Gottwald, 2012: Data assimilation in slow–fast systems using homogenized climate models. J. Atmos. Sci., 69, 1359–1377, doi:10.1175/JAS-D-11-0145.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Mitchell, L., and A. Carrassi, 2015: Accounting for model error due to unresolved scales within ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 141, 1417–1428, doi:10.1002/qj.2451.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Palmer, T. N., 2001: A nonlinear dynamical perspective on model error: A proposal for non-local stochastic-dynamic parametrization in weather and climate prediction models. Quart. J. Roy. Meteor. Soc., 127, 279–304, doi:10.1002/qj.49712757202.

    • Search Google Scholar
    • Export Citation

  • Pavliotis, G. A., and A. Stuart, 2008: Multiscale Methods: Averaging and Homogenization. Texts in Applied Mathematics, Vol. 53, Springer-Verlag, 310 pp.

    • Search Google Scholar
    • Export Citation

  • Sakov, P., and L. Bertino, 2011: Relation between two common localisation methods for the EnKF. Comput. Geosci., 15, 225–237, doi:10.1007/s10596-010-9202-6.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Tong, X. T., A. J. Majda, and D. B. Kelly, 2015: Nonlinear stability of the ensemble Kalman filter with adaptive covariance inflation. arXiv.org, 34 pp. [Available online at https://arxiv.org/abs/1507.08319.]

  • Tong, X. T., A. J. Majda, and D. B. Kelly, 2016: Nonlinear stability and ergodicity of ensemble based Kalman filters. Nonlinearity, 29, 657, doi:10.1088/0951-7715/29/2/657.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Whitaker, J., and G. Compo, 2002: An ensemble Kalman smoother for reanalysis. Symp. on Observations, Data Assimilation, and Probabilistic Prediction, Orlando, FL, Amer. Meteor. Soc., 6.2. [Available online at https://ams.confex.com/ams/pdfpapers/28864.pdf.]

  • Wilks, D. S., 2005: Effects of stochastic parameterizations in the Lorenz’96 system. Quart. J. Roy. Meteor. Soc., 131, 389–407, doi:10.1256/qj.04.03.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Zhang, F., C. Snyder, and J. Sun, 2004: Impacts of initial estimate and observation availability on convective-scale data assimilation with an ensemble Kalman filter. Mon. Wea. Rev., 132, 1238–1253, doi:10.1175/1520-0493(2004)132<1238:IOIEAO>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Zwanzig, R., 1973: Nonlinear generalized Langevin equations. J. Stat. Phys., 9, 215–220, doi:10.1007/BF01008729.

  • Zwanzig, R., 2001: Nonequilibrium Statistical Mechanics. Oxford University Press, 240 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 3408 2694 133
PDF Downloads 192 35 3