Editorial Type:
Article
Article Type:
Research Article

Extending the Square Root Method to Account for Additive Forecast Noise in Ensemble Methods

Patrick Nima Raanes Nansen Environmental and Remote Sensing Center, Bergen, Norway, and Mathematical Institute, University of Oxford, Oxford, United Kingdom

Search for other papers by Patrick Nima Raanes in
Current site
Google Scholar
PubMed Close
,
Alberto Carrassi Nansen Environmental and Remote Sensing Center, Bergen, Norway

Search for other papers by Alberto Carrassi in
Current site
Google Scholar
PubMed Close
, and
Laurent Bertino Nansen Environmental and Remote Sensing Center, Bergen, Norway

Search for other papers by Laurent Bertino in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Oct 2015
Collections:
Sixth WMO Data Assimilation Symposium
DOI:
https://doi.org/10.1175/MWR-D-14-00375.1
Page(s):
3857–3873
Restricted access

Abstract

A square root approach is considered for the problem of accounting for model noise in the forecast step of the ensemble Kalman filter (EnKF) and related algorithms. The primary aim is to replace the method of simulated, pseudo-random additive so as to eliminate the associated sampling errors. The core method is based on the analysis step of ensemble square root filters, and consists in the deterministic computation of a transform matrix. The theoretical advantages regarding dynamical consistency are surveyed, applying equally well to the square root method in the analysis step. A fundamental problem due to the limited size of the ensemble subspace is discussed, and novel solutions that complement the core method are suggested and studied. Benchmarks from twin experiments with simple, low-order dynamics indicate improved performance over standard approaches such as additive, simulated noise, and multiplicative inflation.

Corresponding author address: Patrick Nima Raanes, Mathematical Institute, University of Oxford, Andrew Wiles Building, Woodstock Rd., Oxford OX2 6GG, United Kingdom. E-mail: patrick.raanes@maths.ox.ac.uk

This article is included in the Sixth WMO Data Assimilation Symposium Special Collection.

Abstract

A square root approach is considered for the problem of accounting for model noise in the forecast step of the ensemble Kalman filter (EnKF) and related algorithms. The primary aim is to replace the method of simulated, pseudo-random additive so as to eliminate the associated sampling errors. The core method is based on the analysis step of ensemble square root filters, and consists in the deterministic computation of a transform matrix. The theoretical advantages regarding dynamical consistency are surveyed, applying equally well to the square root method in the analysis step. A fundamental problem due to the limited size of the ensemble subspace is discussed, and novel solutions that complement the core method are suggested and studied. Benchmarks from twin experiments with simple, low-order dynamics indicate improved performance over standard approaches such as additive, simulated noise, and multiplicative inflation.

Corresponding author address: Patrick Nima Raanes, Mathematical Institute, University of Oxford, Andrew Wiles Building, Woodstock Rd., Oxford OX2 6GG, United Kingdom. E-mail: patrick.raanes@maths.ox.ac.uk

This article is included in the Sixth WMO Data Assimilation Symposium Special Collection.

Save
Cover Monthly Weather Review
Monthly Weather Review

  • Anderson, B. D. O., and J. B. Moore, 1979: Optimal Filtering. Prentice-Hall, 367 pp.

  • Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884–2903, doi:10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72–83, doi:10.1111/j.1600-0870.2008.00361.x.

    • Search Google Scholar
    • Export Citation

  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 2741–2758, doi:10.1175/1520-0493(1999)127<2741:AMCIOT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Arnold, H. M., I. M. Moroz, and T. N. Palmer, 2013: Stochastic parametrizations and model uncertainty in the Lorenz’96 system. Philos. Trans. Roy. Soc. London, A371, doi:10.1098/rsta.2011.0479.

    • Search Google Scholar
    • Export Citation

  • Ben-Israel, A., and T. N. E. Greville, 2003: Generalized Inverses: Theory and Applications. 2nd ed. Springer-Verlag, xvi + 420 pp.

  • Berry, T., and J. Harlim, 2014: Linear theory for filtering nonlinear multiscale systems with model error. Proc. Roy. Soc. London, A470, doi:10.1098/rspa.2014.0168.

    • Search Google Scholar
    • Export Citation

  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420–436, doi:10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Bocquet, M., 2011: Ensemble Kalman filtering without the intrinsic need for inflation. Nonlinear Processes Geophys., 18, 735–750, doi:10.5194/npg-18-735-2011.

    • Search Google Scholar
    • Export Citation

  • Burgers, G., P. J. van Leeuwen, and G. Evensen, 1998: Analysis scheme in the ensemble Kalman filter. Mon. Wea. Rev., 126, 1719–1724, doi:10.1175/1520-0493(1998)126<1719:ASITEK>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Chandrasekar, J., I. S. Kim, D. S. Bernstein, and A. J. Ridley, 2008: Reduced-rank unscented Kalman filtering using Cholesky-based decomposition. Int. J. Control, 81, 1779–1792, doi:10.1080/00207170801891724.

    • Search Google Scholar
    • Export Citation

  • Chen, Y., and D. S. Oliver, 2013: Levenberg–Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification. Comput. Geosci., 17, 689–703, doi:10.1007/s10596-013-9351-5.

    • Search Google Scholar
    • Export Citation

  • Daley, R., 1992: Estimating model-error covariances for application to atmospheric data assimilation. Mon. Wea. Rev., 120, 1735–1746, doi:10.1175/1520-0493(1992)120<1735:EMECFA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Dee, D. P., 1995: On-line estimation of error covariance parameters for atmospheric data assimilation. Mon. Wea. Rev., 123, 1128–1145, doi:10.1175/1520-0493(1995)123<1128:OLEOEC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Deng, Z., Y. Tang, and H. J. Freeland, 2011: Evaluation of several model error schemes in the EnKF assimilation: Applied to Argo profiles in the Pacific Ocean. J. Geophys. Res., 116, C09027, doi:10.1029/2011JC006942.

    • Search Google Scholar
    • Export Citation

  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 10 143–10 162, doi:10.1029/94JC00572.

    • Search Google Scholar
    • Export Citation

  • Evensen, G., 2003: The ensemble Kalman filter: Theoretical formulation and practical implementation. Ocean Dyn., 53, 343–367, doi:10.1007/s10236-003-0036-9.

    • Search Google Scholar
    • Export Citation

  • Evensen, G., 2009: The ensemble Kalman filter for combined state and parameter estimation. IEEE Control Syst., 29, 83–104, doi:10.1109/MCS.2009.932223.

    • Search Google Scholar
    • Export Citation

  • Evensen, G., and P. J. van Leeuwen, 1996: Assimilation of Geosat altimeter data for the Agulhas Current using the ensemble Kalman filter with a quasigeostrophic model. Mon. Wea. Rev., 124, 85–96, doi:10.1175/1520-0493(1996)124<0085:AOGADF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Golub, G. H., and C. F. Van Loan, 1996: Matrix Computations. 3rd ed. Johns Hopkins University Press, 728 pp.

  • Hamill, T. M., and J. S. Whitaker, 2005: Accounting for the error due to unresolved scales in ensemble data assimilation: A comparison of different approaches. Mon. Wea. Rev., 133, 3132–3147, doi:10.1175/MWR3020.1.

    • Search Google Scholar
    • Export Citation

  • Hanea, R. G., G. J. M. Velders, A. J. Segers, M. Verlaan, and A. W. Heemink, 2007: A hybrid Kalman filter algorithm for large-scale atmospheric chemistry data assimilation. Mon. Wea. Rev., 135, 140–151, doi:10.1175/MWR3269.1.

    • Search Google Scholar
    • Export Citation

  • Heemink, A. W., M. Verlaan, and A. J. Segers, 2001: Variance reduced ensemble Kalman filtering. Mon. Wea. Rev., 129, 1718–1728, doi:10.1175/1520-0493(2001)129<1718:VREKF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Hoang, H. S., R. Baraille, and O. Talagrand, 2005: On an adaptive filter for altimetric data assimilation and its application to a primitive equation model, MICOM. Tellus, 57A, 153–170, doi:10.1111/j.1600-0870.2005.00094.x.

    • Search Google Scholar
    • Export Citation

  • Horn, R. A., and C. R. Johnson, 2013: Matrix Analysis. 2nd ed. Cambridge University Press, xviii + 643 pp.

  • Hoteit, I., D.-T. Pham, and J. Blum, 2002: A simplified reduced order Kalman filtering and application to altimetric data assimilation in tropical Pacific. J. Mar. Syst., 36, 101–127, doi:10.1016/S0924-7963(02)00129-X.

    • Search Google Scholar
    • Export Citation

  • Hoteit, I., D.-T. Pham, M. Gharamti, and X. Luo, 2015: Mitigating observation perturbation sampling errors in the stochastic EnKF. Mon. Wea. Rev., 143, 2918–2936, doi:10.1175/MWR-D-14-00088.1.

    • Search Google Scholar
    • Export Citation

  • Houtekamer, P. L., H. L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133, 604–620, doi:10.1175/MWR-2864.1.

    • Search Google Scholar
    • Export Citation

  • Hunt, B. R., and Coauthors, 2004: Four-dimensional ensemble Kalman filtering. Tellus, 56A, 273–277, doi:10.1111/j.1600-0870.2004.00066.x.

    • Search Google Scholar
    • Export Citation

  • Hunt, B. R., E. J. Kostelich, and I. Szunyogh, 2007: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112–126, doi:10.1016/j.physd.2006.11.008.

    • Search Google Scholar
    • Export Citation

  • Jazwinski, A. H., 1970: Stochastic Processes and Filtering Theory. Academic Press, 376 pp.

  • Li, H., E. Kalnay, T. Miyoshi, and C. M. Danforth, 2009: Accounting for model errors in ensemble data assimilation. Mon. Wea. Rev., 137, 3407–3419, doi:10.1175/2009MWR2766.1.

    • Search Google Scholar
    • Export Citation

  • Livings, D. M., S. L. Dance, and N. K. Nichols, 2008: Unbiased ensemble square root filters. Physica D, 237, 1021–1028, doi:10.1016/j.physd.2008.01.005.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130–141, doi:10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Lorenz, E. N., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399–414, doi:10.1175/1520-0469(1998)055<0399:OSFSWO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • McLay, J. G., C. H. Bishop, and C. A. Reynolds, 2008: Evaluation of the ensemble transform analysis perturbation scheme at NRL. Mon. Wea. Rev., 136, 1093–1108, doi:10.1175/2007MWR2010.1.

    • Search Google Scholar
    • Export Citation

  • Mitchell, L., and A. Carrassi, 2015: Accounting for model error due to unresolved scales within ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 141, 1417–1428, doi:10.1002/qj.2451.

    • Search Google Scholar
    • Export Citation

  • Nakano, S., 2013: A prediction algorithm with a limited number of particles for state estimation of high-dimensional systems. 16th Int. Conf. on Information Fusion (FUSION), Istanbul, Turkey, IEEE,1356–1363.

  • Nicolis, C., 2004: Dynamics of model error: The role of unresolved scales revisited. J. Atmos. Sci., 61, 1740–1753, doi:10.1175/1520-0469(2004)061<1740:DOMETR>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Oliver, D. S., 2014: Minimization for conditional simulation: Relationship to optimal transport. J. Comput. Phys., 265, 1–15, doi:10.1016/j.jcp.2014.01.048.

    • Search Google Scholar
    • Export Citation

  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415–428, doi:10.1111/j.1600-0870.2004.00076.x.

    • Search Google Scholar
    • Export Citation

  • Pham, D. T., 2001: Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Wea. Rev., 129, 1194–1207, doi:10.1175/1520-0493(2001)129<1194:SMFSDA>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Reich, S., and C. J. Cotter, 2013: Ensemble filter techniques for intermittent data assimilation. Large Scale Inverse Problems: Computational Methods and Applications in the Earth Sciences, M. Cullen et al., Eds., Radon Series on Computational and Applied Mathematics, Vol. 13, De Gruyter, 91–134.

  • Rodgers, C. D., 2000: Inverse Methods for Atmospheric Sounding: Theory and Practice. World Scientific, 240 pp.

  • Sakov, P., and P. R. Oke, 2008a: A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters. Tellus, 60A, 361–371, doi:10.1111/j.1600-0870.2007.00299.x.

    • Search Google Scholar
    • Export Citation

  • Sakov, P., and P. R. Oke, 2008b: Implications of the form of the ensemble transformation in the ensemble square root filters. Mon. Wea. Rev., 136, 1042–1053, doi:10.1175/2007MWR2021.1.

    • Search Google Scholar
    • Export Citation

  • Shutts, G., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 131, 3079–3102, doi:10.1256/qj.04.106.

    • Search Google Scholar
    • Export Citation

  • Slingo, J., and T. Palmer, 2011: Uncertainty in weather and climate prediction. Philos. Trans. Roy. Soc. London, A369, 4751–4767, doi:10.1098/rsta.2011.0161.

    • Search Google Scholar
    • Export Citation

  • Tippett, M. K., J. L. Anderson, C. H. Bishop, T. M. Hamill, and J. S. Whitaker, 2003: Ensemble square root filters. Mon. Wea. Rev., 131, 1485–1490, doi:10.1175/1520-0493(2003)131<1485:ESRF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • van Leeuwen, P. J., 2009: Particle filtering in geophysical systems. Mon. Wea. Rev., 137, 4089–4114, doi:10.1175/2009MWR2835.1.

  • Verlaan, M., and A. W. Heemink, 1997: Tidal flow forecasting using reduced rank square root filters. Stochastic Hydrol. Hydraul., 11, 349–368, doi:10.1007/BF02427924.

    • Search Google Scholar
    • Export Citation

  • Wang, X., and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 1140–1158, doi:10.1175/1520-0469(2003)060<1140:ACOBAE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Wang, X., C. H. Bishop, and S. J. Julier, 2004: Which is better, an ensemble of positive–negative pairs or a centered spherical simplex ensemble? Mon. Wea. Rev., 132, 1590–1605, doi:10.1175/1520-0493(2004)132<1590:WIBAEO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Whitaker, J. S., and T. M. Hamill, 2012: Evaluating methods to account for system errors in ensemble data assimilation. Mon. Wea. Rev., 140, 3078–3089, doi:10.1175/MWR-D-11-00276.1.

    • Search Google Scholar
    • Export Citation

  • Whitaker, J. S., G. P. Compo, X. Wei, and T. M. Hamill, 2004: Reanalysis without radiosondes using ensemble data assimilation. Mon. Wea. Rev., 132, 1190–1200, doi:10.1175/1520-0493(2004)132<1190:RWRUED>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP global forecast system. Mon. Wea. Rev., 136, 463–482, doi:10.1175/2007MWR2018.1.

    • Search Google Scholar
    • Export Citation

  • Wunsch, C., 2006: Discrete Inverse and State Estimation Problems: With Geophysical Fluid Applications. Cambridge University Press, 384 pp.

    • Search Google Scholar
    • Export Citation

  • Zhang, F., C. Snyder, and J. Sun, 2004: Impacts of initial estimate and observation availability on convective-scale data assimilation with an ensemble Kalman filter. Mon. Wea. Rev., 132, 1238–1253, doi:10.1175/1520-0493(2004)132<1238:IOIEAO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Zupanski, D., and M. Zupanski, 2006: Model error estimation employing an ensemble data assimilation approach. Mon. Wea. Rev., 134, 1337–1354, doi:10.1175/MWR3125.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1738 355 9
PDF Downloads 264 92 0