• Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 28842903, doi:10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 2003: A local least squares framework for ensemble filtering. Mon. Wea. Rev., 131, 634642, doi:10.1175/1520-0493(2003)131<0634:ALLSFF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 2007: Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter. Physica D, 230, 99111, doi:10.1016/j.physd.2006.02.011.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 2009a: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 7283, doi:10.1111/j.1600-0870.2008.00361.x.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 2009b: Ensemble Kalman filters for large geophysical applications. IEEE Contr. Syst. Mag., 29, 6682, doi:10.1109/MCS.2009.932222.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., 2012: Localization and sampling error correction in ensemble Kalman filter data assimilation. Mon. Wea. Rev., 140, 23592371, doi:10.1175/MWR-D-11-00013.1.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 27412758, doi:10.1175/1520-0493(1999)127<2741:AMCIOT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., and N. Collins, 2007: Scalable implementations of ensemble filter algorithms for data assimilation. J. Atmos. Oceanic Technol., 24, 14521463, doi:10.1175/JTECH2049.1.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., and L. Lei, 2013: Empirical localization of observation impact in ensemble Kalman filters. Mon. Wea. Rev., 141, 41404153, doi:10.1175/MWR-D-12-00330.1.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., B. Wyman, S. Zhang, and T. Hoar, 2005: Assimilation of surface pressure observations using an ensemble filter in an idealized global atmospheric prediction system. J. Atmos. Sci., 62, 29252938, doi:10.1175/JAS3510.1.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Arellano, 2009: The Data Assimilation Research Testbed. Bull. Amer. Meteor. Soc., 90, 12831296, doi:10.1175/2009BAMS2618.1.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., and D. Hodyss, 2007: Flow adaptive moderation of spurious ensemble correlations and its use in ensemble based data assimilation. Quart. J. Roy. Meteor. Soc., 133, 20292044, doi:10.1002/qj.169.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., and D. Hodyss, 2009: Ensemble covariances adaptively localized with ECO-RAP. Part 1: Tests on simple error models. Tellus, 61A, 8496, doi:10.1111/j.1600-0870.2008.00371.x.

    • Search Google Scholar
    • Export Citation
  • Burgers, G., P. J. van Leeuwen, and G. Evensen, 1998: Analysis scheme in the ensemble Kalman filter. Mon. Wea. Rev., 126, 17191724, doi:10.1175/1520-0493(1998)126<1719:ASITEK>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Cavallo, S. M., R. D. Torn, C. Snyder, C. Davis, W. Wang, and J. Done, 2013: Evaluation of the Advanced Hurricane WRF data assimilation system for the 2009 Atlantic hurricane season. Mon. Wea. Rev., 141, 523541, doi:10.1175/MWR-D-12-00139.1.

    • Search Google Scholar
    • Export Citation
  • Chen, Y., and D. S. Oliver, 2010: Cross-covariances and localization for EnKF in multiphase flow data assimilation. Comput. Geosci., 14, 579601, doi:10.1007/s10596-009-9174-6.

    • Search Google Scholar
    • Export Citation
  • Emerick, A., and A. Reynolds, 2011: Combining sensitivities and prior information for covariance localization in the ensemble Kalman filter for petroleum reservoir applications. Comput. Geosci., 15, 251269, doi:10.1007/s10596-010-9198-y.

    • Search Google Scholar
    • Export Citation
  • Furrer, R., and T. Bengtsson, 2007: Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J. Multivar. Anal., 98, 227255, doi:10.1016/j.jmva.2006.08.003.

    • Search Google Scholar
    • Export Citation
  • Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723757, doi:10.1002/qj.49712555417.

    • Search Google Scholar
    • Export Citation
  • GFDL Global Atmospheric Model Development Team, 2004: The new GFDL Global Atmosphere and Land Model AM2–LM2: Evaluation with prescribed SST simulations. J. Climate, 17, 46414673, doi:10.1175/JCLI-3223.1.

    • Search Google Scholar
    • Export Citation
  • Greybush, S. J., E. Kalnay, T. Miyoshi, K. Ide, and B. R. Hunt, 2011: Balance and ensemble Kalman filter localization techniques. Mon. Wea. Rev., 139, 511522, doi:10.1175/2010MWR3328.1.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 27762790, doi:10.1175/1520-0493(2001)129<2776:DDFOBE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Held, I. M., and M. J. Suarez, 1994: A proposal for the intercomparison of the dynamical cores of atmospheric general circulation models. Bull. Amer. Meteor. Soc., 75, 18251830, doi:10.1175/1520-0477(1994)075<1825:APFTIO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796811, doi:10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., X. Deng, H. L. Mitchell, S.-J. Baek, and N. Gagnon, 2014: Higher resolution in an operational ensemble Kalman filter. Mon. Wea. Rev., 142, 11431162, doi:10.1175/MWR-D-13-00138.1.

    • Search Google Scholar
    • Export Citation
  • Karspeck, A. R., S. Yeager, G. Danabasoglu, T. Hoar, N. Collins, K. Raeder, J. Anderson, and J. Tribbia, 2013: An ensemble adjustment Kalman filter for the CCSM4 ocean component. J. Climate, 26, 73927413, doi:10.1175/JCLI-D-12-00402.1.

    • Search Google Scholar
    • Export Citation
  • Kepert, J. D., 2009: Covariance localisation and balance in an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 11571176, doi:10.1002/qj.443.

    • Search Google Scholar
    • Export Citation
  • Keppenne, C. L., M. M. Rienecker, J. P. Jacob, and R. Kovach, 2008: Error covariance modeling in the GMAO ocean ensemble Kalman filter. Mon. Wea. Rev., 136, 29642982, doi:10.1175/2007MWR2243.1.

    • Search Google Scholar
    • Export Citation
  • Lei, L., and J. L. Anderson, 2014a: Comparisons of empirical localization techniques for serial ensemble Kalman filters in a simple atmospheric general circulation model. Mon. Wea. Rev., 142, 739754, doi:10.1175/MWR-D-13-00152.1.

    • Search Google Scholar
    • Export Citation
  • Lei, L., and J. L. Anderson, 2014b: Empirical localization of observations for serial ensemble Kalman filter data assimilation in an atmospheric general circulation model. Mon. Wea. Rev., 142, 18351851, doi:10.1175/MWR-D-13-00288.1.

    • Search Google Scholar
    • Export Citation
  • Li, H., E. Kalnay, and T. Miyoshi, 2009a: Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 523533, doi:10.1002/qj.371.

    • Search Google Scholar
    • Export Citation
  • Li, H., E. Kalnay, T. Miyoshi, and C. M. Danforth, 2009b: Accounting for model errors in ensemble data assimilation. Mon. Wea. Rev., 137, 34073419, doi:10.1175/2009MWR2766.1.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399414, doi:10.1175/1520-0469(1998)055<0399:OSFSWO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Oke, P. R., P. Sakov, and S. P. Corney, 2007: Impacts of localisation in the EnKF and EnOI: Experiments with a small model. Ocean Dyn., 57, 3245, doi:10.1007/s10236-006-0088-8.

    • Search Google Scholar
    • Export Citation
  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415428, doi:10.1111/j.1600-0870.2004.00076.x.

    • Search Google Scholar
    • Export Citation
  • Reichle, R. H., D. B. McLaughlin, and D. Entekhabi, 2002: Hydrologic data assimilation with the ensemble Kalman filter. Mon. Wea. Rev., 130, 103114, doi:10.1175/1520-0493(2002)130<0103:HDAWTE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Rosolem, R., T. Hoar, A. Arellano, J. L. Anderson, W. J. Shuttleworth, X. Zeng, and T. E. Franz, 2014: Assimilation of near-surface cosmic-ray neutrons improves summertime soil moisture profile estimates at three distinct biomes in the USA. Hydrol. Earth Syst. Sci. Discuss., 11, 55155558, doi:10.5194/hessd-11-5515-2014.

    • Search Google Scholar
    • Export Citation
  • Shaman, J., and A. Karspeck, 2012: Forecasting seasonal outbreaks of influenza. Proc. Natl. Acad. Sci. USA, 109, 20 42520 430, doi:10.1073/pnas.1208772109.

    • Search Google Scholar
    • Export Citation
  • Tippett, M. K., J. L. Anderson, C. H. Bishop, T. M. Hamill, and J. S. Whitaker, 2003: Ensemble square root filters. Mon. Wea. Rev., 131, 14851490, doi:10.1175/1520-0493(2003)131<1485:ESRF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Whitaker, J. S., and T. M. Hamill, 2012: Evaluating methods to account for system errors in ensemble data assimilation. Mon. Wea. Rev., 140, 30783089, doi:10.1175/MWR-D-11-00276.1.

    • Search Google Scholar
    • Export Citation
  • Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: Ensemble data assimilation with the NCEP Global Forecast System. Mon. Wea. Rev., 136, 463482, doi:10.1175/2007MWR2018.1.

    • Search Google Scholar
    • Export Citation
  • Zhang, F., C. Snyder, and J. Sun, 2004: Impacts of initial estimate and observation availability on convective-scale data assimilation with an ensemble Kalman filter. Mon. Wea. Rev., 132, 12381253, doi:10.1175/1520-0493(2004)132<1238:IOIEAO>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Zhang, Y., and D. S. Oliver, 2010: Improving the ensemble estimate of the Kalman gain by bootstrap sampling. Math. Geosci., 42, 327345, doi:10.1007/s11004-010-9267-8.

    • Search Google Scholar
    • Export Citation
  • Zhu, K., Y. Pan, M. Xue, X. Wang, J. S. Whitaker, S. G. Benjamin, S. S. Weygandt, and M. Hu, 2013: A regional GSI-based ensemble Kalman filter data assimilation system for the rapid refresh configuration: Testing at reduced resolution. Mon. Wea. Rev., 141, 41184139, doi:10.1175/MWR-D-13-00039.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 4 4 4
PDF Downloads 2 2 2

Reducing Correlation Sampling Error in Ensemble Kalman Filter Data Assimilation

View More View Less
  • 1 Data Assimilation Research Section, National Center for Atmospheric Research,* Boulder, Colorado
Restricted access

Abstract

Ensemble Kalman filters are widely used for data assimilation in large geophysical models. Good results with affordable ensemble sizes require enhancements to the basic algorithms to deal with insufficient ensemble variance and spurious ensemble correlations between observations and state variables. These challenges are often dealt with by using inflation and localization algorithms. A new method for understanding and reducing some ensemble filter errors is introduced and tested. The method assumes that sampling error due to small ensemble size is the primary source of error. Sampling error in the ensemble correlations between observations and state variables is reduced by estimating the distribution of correlations as part of the ensemble filter algorithm. This correlation error reduction (CER) algorithm can produce high-quality ensemble assimilations in low-order models without using any a priori localization like a specified localization function. The method is also applied in an observing system simulation experiment with a very coarse resolution dry atmospheric general circulation model. This demonstrates that the algorithm provides insight into the need for localization in large geophysical applications, suggesting that sampling error may be a primary cause in some cases.

Corresponding author address: Jeffrey L. Anderson, NCAR, 1850 Table Mesa Dr., Boulder, CO 80305. E-mail: jla@ucar.edu

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

Abstract

Ensemble Kalman filters are widely used for data assimilation in large geophysical models. Good results with affordable ensemble sizes require enhancements to the basic algorithms to deal with insufficient ensemble variance and spurious ensemble correlations between observations and state variables. These challenges are often dealt with by using inflation and localization algorithms. A new method for understanding and reducing some ensemble filter errors is introduced and tested. The method assumes that sampling error due to small ensemble size is the primary source of error. Sampling error in the ensemble correlations between observations and state variables is reduced by estimating the distribution of correlations as part of the ensemble filter algorithm. This correlation error reduction (CER) algorithm can produce high-quality ensemble assimilations in low-order models without using any a priori localization like a specified localization function. The method is also applied in an observing system simulation experiment with a very coarse resolution dry atmospheric general circulation model. This demonstrates that the algorithm provides insight into the need for localization in large geophysical applications, suggesting that sampling error may be a primary cause in some cases.

Corresponding author address: Jeffrey L. Anderson, NCAR, 1850 Table Mesa Dr., Boulder, CO 80305. E-mail: jla@ucar.edu

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

Save