• Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 28842903.

  • Anderson, J. L., 2003: A local least squares framework for ensemble filtering. Mon. Wea. Rev., 131, 634642.

  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 27412758.

    • Search Google Scholar
    • Export Citation
  • Bennett, A. F., L. M. Leslie, C. R. Hagelberg, and P. E. Powers, 1993: Tropical cyclone prediction using a barotropic model initialized by a generalized inverse method. Mon. Wea. Rev., 121, 17141729.

    • Search Google Scholar
    • Export Citation
  • Bennett, A. F., B. S. Chua, and L. M. Leslie, 1996: Generalized inversion of a global numerical weather prediction model. Meteor. Atmos. Phys., 60, 165178.

    • Search Google Scholar
    • Export Citation
  • Bernardo, J. M., 1979: Expected information as expected utility. Ann. Stat., 7, 686690.

  • Brocker, J., and L. A. Smith, 2007: Scoring probabilistic forecasts: On the importance of being proper. Wea. Forecasting, 22, 382388.

    • Search Google Scholar
    • Export Citation
  • Brocker, J., and L. A. Smith, 2008: From ensemble forecasts to predictive distribution functions. Tellus, 60, 663678.

  • Buizza, R., M. Miller, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 28872908.

    • Search Google Scholar
    • Export Citation
  • Courtier, P., J. N. Thepaut, and A. Hollingsworth, 1994: A strategy for operational implementation of 4DVAR, using an incremental approach. Quart. J. Roy. Meteor. Soc., 120, 13671387.

    • Search Google Scholar
    • Export Citation
  • Du, H., 2009: Combining statistical methods with dynamical insight to improve nonlinear estimation. Ph.D. dissertation, London School of Economics and Political Science, 190 pp.

  • Du, H., and L. A. Smith, 2014: Pseudo-orbit data assimilation. Part I: The perfect model scenario. J. Atmos. Sci.,71, 469–482.

  • Fefferman, C. L., 2000: Existence and smoothness of the Navier–Stokes equation. Princeton University Department of Mathematics Tech. Rep. 6 pp.

  • Fertig, E. J., J. Harlim, and B. R. Hunt, 2007: A comparative study of 4D-VAR and a 4D ensemble Kalman filter: Perfect model simulations with Lorenz-96. Tellus, 59A, 96100.

    • Search Google Scholar
    • Export Citation
  • Good, I. J., 1952: Rational decisions. J. Roy. Stat. Soc., 16A, 107114.

  • Hagedorn, R., and L. A. Smith, 2009: Communicating the value of probabilistic forecasts with weather roulette. Meteor. Appl., 16, 143155.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., 2006: Ensemble-based atmospheric data assimilation. Predictability of Weather and Climate, T. Palmer and R. Hagedorn, Eds., Cambridge University Press, 124–156.

  • Hamill, T. M., and J. S. Whitaker, 2005: Accounting for the error due to unresolved scales in ensemble data assimilation: A comparison of different approaches. Mon. Wea. Rev., 133, 31323147.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background-error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 27762790.

    • Search Google Scholar
    • Export Citation
  • Hansen, J. A., and L. A. Smith, 2001: Probabilistic noise reduction. Tellus, 53A, 585598.

  • Haramel, S. M., C. K. R. T. Jones, and J. V. Moloney, 1985: Global dynamical behavior of the optical field in a ring cavity. J. Opt. Soc. Amer., 2B, 552564.

    • Search Google Scholar
    • Export Citation
  • Ikeda, K., 1979: Multiple valued stationarity state and its instability of the transmitted light by a ring cavity system. Opt. Commun., 30, 257261.

    • Search Google Scholar
    • Export Citation
  • Judd, K., 2008: Forecasting with imperfect models, dynamically constrained inverse problems, and gradient descent algorithms. Physica D, 237, 216232.

    • Search Google Scholar
    • Export Citation
  • Judd, K., and L. A. Smith, 2001: Indistinguishable states I: The perfect model scenario. Physica D, 151, 125141.

  • Judd, K., and L. A. Smith, 2004: Indistinguishable states II: The imperfect model scenario. Physica D, 196, 224242.

  • Judd, K., C. A. Reynolds, T. E. Rosmond, and L. A. Smith, 2008: The geometry of model error. J. Atmos. Sci., 65, 17491772.

  • Kennedy, M., and A. O’Hagan, 2001: Bayesian calibration of computer models. J. Roy. Stat. Soc., 63B, 425464.

  • Leeuwen, P. J. V., 2010: Nonlinear data assimilation in geosciences: An extremely efficient particle filter. Quart. J. Roy. Meteor. Soc., 136, 19911999.

    • Search Google Scholar
    • Export Citation
  • Leutbecher, M., and T. N. Palmer, 2008: Ensemble forecasting. J. Comput. Phys., 227, 35153539.

  • Lorenc, A. C., 1986: Analysis methods for numerical weather prediction. Quart. J. Roy. Meteor. Soc., 112, 11771194.

  • Lorenz, E. N., 1995: Predictability: A problem partly solved. Proc. Seminar on Predictability, Shinfield Park, United Kingdom, ECMWF, 40–58.

  • Lu, C. G., and G. L. Browning, 1998: The impact of observational and model errors on four-dimensional variational data assimilation. J. Atmos. Sci., 55, 9951011.

    • Search Google Scholar
    • Export Citation
  • Miller, R. N., M. Ghil, and F. Gauthiez, 1994: Advanced data assimilation in strongly nonlinear dynamical systems. J. Atmos. Sci., 51, 10371056.

    • Search Google Scholar
    • Export Citation
  • Mitchell, H. L., and P. L. Houtekamer, 2000: An adaptive ensemble Kalman filter. Mon. Wea. Rev., 128, 416433.

  • Mitchell, L., and G. A. Gottwald, 2012: Data assimilation in slow–fast systems using homogenized climate models. J. Atmos. Sci., 69, 13591377.

    • Search Google Scholar
    • Export Citation
  • Orrell, D., L. A. Smith, T. Palmer, and J. Barkmeijer, 2001: Model error in weather forecasting. Nonlinear Processes Geophys., 8, 357371.

    • Search Google Scholar
    • Export Citation
  • Penland, C., 2003: A stochastic approach to nonlinear dynamics: A review. Bull. Amer. Meteor. Soc., 84, 921925.

  • Pires, C., R. Vautard, and O. Talagrand, 1996: On extending the limits of variational assimilation in nonlinear chaotic systems. Tellus, 48A, 96121.

    • Search Google Scholar
    • Export Citation
  • Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133, 11551174.

    • Search Google Scholar
    • Export Citation
  • Roulston, M. S., and L. A. Smith, 2002: Evaluating probabilistic forecasts using information theory. Mon. Wea. Rev., 130, 16531660.

  • Sasaki, Y., 1970: Some basic formalisms on numerical variational analysis. Mon. Wea. Rev., 98, 875883.

  • Smith, L. A., 1992: Identification and prediction of low dimensional dynamics. Physica D, 58, 5076.

  • Smith, L. A., 1996: Accountability and error in ensemble forecasting. Proc. Seminar on Predictability, Shinfield Park, United Kingdom, ECMWF, 351–368.

  • Smith, L. A., 2002: What might we learn from climate forecasts? Proc. Natl. Acad. Sci. USA, 4, 24872492.

  • Smith, L. A., M. C. Cuellar, H. Du, and K. Judd, 2010: Exploiting dynamical coherence: A geometric approach to parameter estimation in nonlinear models. Phys. Lett., 374A, 26182623.

    • Search Google Scholar
    • Export Citation
  • Talagrand, O., and P. Courtier, 1987: Variational assimilation of meteorological observations with the adjoint vorticity equation. I: Theory. Quart. J. Roy. Meteor. Soc., 113, 13111328.

    • Search Google Scholar
    • Export Citation
  • Zupanski, D., 1997: A general weak constraint applicable to operational 4DVAR data assimilation systems. Mon. Wea. Rev., 123, 11121127.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 80 24 7
PDF Downloads 41 14 2

Pseudo-Orbit Data Assimilation. Part II: Assimilation with Imperfect Models

Hailiang DuCentre for the Analysis of Time Series, London School of Economics and Political Science, London, United Kingdom

Search for other papers by Hailiang Du in
Current site
Google Scholar
PubMed
Close
and
Leonard A. SmithCentre for the Analysis of Time Series, London School of Economics and Political Science, London, United Kingdom

Search for other papers by Leonard A. Smith in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Data assimilation and state estimation for nonlinear models is a challenging task mathematically. Performing this task in real time, as in operational weather forecasting, is even more challenging as the models are imperfect: the mathematical system that generated the observations (if such a thing exists) is not a member of the available model class (i.e., the set of mathematical structures admitted as potential models). To the extent that traditional approaches address structural model error at all, most fail to produce consistent treatments. This results in questionable estimates both of the model state and of its uncertainty. A promising alternative approach is proposed to produce more consistent estimates of the model state and to estimate the (state dependent) model error simultaneously. This alternative consists of pseudo-orbit data assimilation with a stopping criterion. It is argued to be more efficient and more coherent than one alternative variational approach [a version of weak-constraint four-dimensional variational data assimilation (4DVAR)]. Results that demonstrate the pseudo-orbit data assimilation approach can also outperform an ensemble Kalman filter approach are presented. Both comparisons are made in the context of the 18-dimensional Lorenz96 flow and the two-dimensional Ikeda map. Many challenges remain outside the perfect model scenario, both in defining the goals of data assimilation and in achieving high-quality state estimation. The pseudo-orbit data assimilation approach provides a new tool for approaching this open problem.

Corresponding author address: Hailiang Du, Centre for the Analysis of Time Series, London School of Economics and Political Science, Houghton Street, London WC2A 2AE, United Kingdom. E-mail: h.l.du@lse.ac.uk

Abstract

Data assimilation and state estimation for nonlinear models is a challenging task mathematically. Performing this task in real time, as in operational weather forecasting, is even more challenging as the models are imperfect: the mathematical system that generated the observations (if such a thing exists) is not a member of the available model class (i.e., the set of mathematical structures admitted as potential models). To the extent that traditional approaches address structural model error at all, most fail to produce consistent treatments. This results in questionable estimates both of the model state and of its uncertainty. A promising alternative approach is proposed to produce more consistent estimates of the model state and to estimate the (state dependent) model error simultaneously. This alternative consists of pseudo-orbit data assimilation with a stopping criterion. It is argued to be more efficient and more coherent than one alternative variational approach [a version of weak-constraint four-dimensional variational data assimilation (4DVAR)]. Results that demonstrate the pseudo-orbit data assimilation approach can also outperform an ensemble Kalman filter approach are presented. Both comparisons are made in the context of the 18-dimensional Lorenz96 flow and the two-dimensional Ikeda map. Many challenges remain outside the perfect model scenario, both in defining the goals of data assimilation and in achieving high-quality state estimation. The pseudo-orbit data assimilation approach provides a new tool for approaching this open problem.

Corresponding author address: Hailiang Du, Centre for the Analysis of Time Series, London School of Economics and Political Science, Houghton Street, London WC2A 2AE, United Kingdom. E-mail: h.l.du@lse.ac.uk
Save