Challenges for Inline Observation Error Estimation in the Presence of Misspecified Background Uncertainty

Andrew Walsworth aDepartment of Atmospheric and Oceanic Science, University of Maryland, College Park, College Park, Maryland

Search for other papers by Andrew Walsworth in
Current site
Google Scholar
PubMed
Close
,
Jonathan Poterjoy aDepartment of Atmospheric and Oceanic Science, University of Maryland, College Park, College Park, Maryland

Search for other papers by Jonathan Poterjoy in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0003-3927-4642
, and
Elizabeth Satterfield bMarine Meteorology Division, U.S. Naval Research Laboratory, Monterey, California

Search for other papers by Elizabeth Satterfield in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

For data assimilation to provide faithful state estimates for dynamical models, specifications of observation uncertainty need to be as accurate as possible. Innovation-based methods based on Desroziers diagnostics, are commonly used to estimate observation uncertainty, but such methods can depend greatly on the prescribed background uncertainty. For ensemble data assimilation, this uncertainty comes from statistics calculated from ensemble forecasts, which require inflation and localization to address under sampling. In this work, we use an ensemble Kalman filter (EnKF) with a low-dimensional Lorenz model to investigate the interplay between the Desroziers method and inflation. Two inflation techniques are used for this purpose: 1) a rigorously tuned fixed multiplicative scheme and 2) an adaptive state-space scheme. We document how inaccuracies in observation uncertainty affect errors in EnKF posteriors and study the combined impacts of misspecified initial observation uncertainty, sampling error, and model error on Desroziers estimates. We find that whether observation uncertainty is over- or underestimated greatly affects the stability of data assimilation and the accuracy of Desroziers estimates and that preference should be given to initial overestimates. Inline estimates of Desroziers tend to remove the dependence between ensemble spread–skill and the initially prescribed observation error. In addition, we find that the inclusion of model error introduces spurious correlations in observation uncertainty estimates. Further, we note that the adaptive inflation scheme is less robust than fixed inflation at mitigating multiple sources of error. Last, sampling error strongly exacerbates existing sources of error and greatly degrades EnKF estimates, which translates into biased Desroziers estimates of observation error covariance.

Significance Statement

To generate accurate predictions of various components of the Earth system, numerical models require an accurate specification of state variables at our current time. This step adopts a probabilistic consideration of our current state estimate versus information provided from environmental measurements of the true state. Various strategies exist for estimating uncertainty in observations within this framework, but are sensitive to a host of assumptions, which are investigated in this study.

© 2023 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jonathan Poterjoy, poterjoy@umd.edu

Abstract

For data assimilation to provide faithful state estimates for dynamical models, specifications of observation uncertainty need to be as accurate as possible. Innovation-based methods based on Desroziers diagnostics, are commonly used to estimate observation uncertainty, but such methods can depend greatly on the prescribed background uncertainty. For ensemble data assimilation, this uncertainty comes from statistics calculated from ensemble forecasts, which require inflation and localization to address under sampling. In this work, we use an ensemble Kalman filter (EnKF) with a low-dimensional Lorenz model to investigate the interplay between the Desroziers method and inflation. Two inflation techniques are used for this purpose: 1) a rigorously tuned fixed multiplicative scheme and 2) an adaptive state-space scheme. We document how inaccuracies in observation uncertainty affect errors in EnKF posteriors and study the combined impacts of misspecified initial observation uncertainty, sampling error, and model error on Desroziers estimates. We find that whether observation uncertainty is over- or underestimated greatly affects the stability of data assimilation and the accuracy of Desroziers estimates and that preference should be given to initial overestimates. Inline estimates of Desroziers tend to remove the dependence between ensemble spread–skill and the initially prescribed observation error. In addition, we find that the inclusion of model error introduces spurious correlations in observation uncertainty estimates. Further, we note that the adaptive inflation scheme is less robust than fixed inflation at mitigating multiple sources of error. Last, sampling error strongly exacerbates existing sources of error and greatly degrades EnKF estimates, which translates into biased Desroziers estimates of observation error covariance.

Significance Statement

To generate accurate predictions of various components of the Earth system, numerical models require an accurate specification of state variables at our current time. This step adopts a probabilistic consideration of our current state estimate versus information provided from environmental measurements of the true state. Various strategies exist for estimating uncertainty in observations within this framework, but are sensitive to a host of assumptions, which are investigated in this study.

© 2023 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Jonathan Poterjoy, poterjoy@umd.edu
Save
  • Anderson, J. L., 2007: An adaptive covariance inflation error correction algorithm for ensemble filters. Tellus, 59A, 210224, https://doi.org/10.1111/j.1600-0870.2006.00216.x.

    • Search Google Scholar
    • Export Citation
  • Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 27412758, https://doi.org/10.1175/1520-0493(1999)127<2741:AMCIOT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Berry, T., and T. Sauer, 2013: Adaptive ensemble Kalman filtering of non-linear systems. Tellus, 65A, 20331, https://doi.org/10.3402/tellusa.v65i0.20331.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2000: Adaptive sampling with the ensemble transform Kalman filter. Mon. Wea. Rev., 129, 420436, https://doi.org/10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Bormann, N., and P. Bauer, 2010: Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. I: Methods and application to ATOVS data. Quart. J. Roy. Meteor. Soc., 136, 10361050, https://doi.org/10.1002/qj.616.

    • Search Google Scholar
    • Export Citation
  • Bormann, N., A. Collard, and P. Bauer, 2010: Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. II: Application to AIRS and IASI data. Quart. J. Roy. Meteor. Soc., 136, 10511063, https://doi.org/10.1002/qj.615.

    • Search Google Scholar
    • Export Citation
  • Bowler, N. E., 2020: Revised GNSS-RO observation uncertainties in the Met Office NWP system. Quart. J. Roy. Meteor. Soc., 146, 22742296, https://doi.org/10.1002/qj.3791.

    • Search Google Scholar
    • Export Citation
  • Campbell, W. F., E. A. Satterfield, B. Ruston, and N. L. Baker, 2017: Accounting for correlated observation error in a dual-formulation 4D variational data assimilation system. Mon. Wea. Rev., 145, 10191032, https://doi.org/10.1175/MWR-D-16-0240.1.

    • Search Google Scholar
    • Export Citation
  • Cocucci, T. J., M. Pulido, M. Lucini, and P. Tandeo, 2020: Model error covariance estimation in particle and ensemble Kalman filters using an online expectation–maximization algorithm. Quart. J. Roy. Meteor. Soc., 147, 526543, https://doi.org/10.1002/qj.3931.

    • Search Google Scholar
    • Export Citation
  • Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 33853396, https://doi.org/10.1256/qj.05.108.

    • Search Google Scholar
    • Export Citation
  • Dreano, D., P. Tandeo, M. Pulido, B. Ait-El-Fquih, T. Chonavel, and I. Hoteit, 2017: Estimating model-error covariances in nonlinear state-space models using Kalman smoothing and the expectation–maximization algorithm. Quart. J. Roy. Meteor. Soc., 143, 18771885, https://doi.org/10.1002/qj.3048.

    • Search Google Scholar
    • Export Citation
  • Gharamti, M., 2018: Enhanced adaptive inflation algorithm for ensemble filters. Mon. Wea. Rev., 146, 623640, https://doi.org/10.1175/MWR-D-17-0187.1.

    • Search Google Scholar
    • Export Citation
  • Gottwald, G. A., and A. J. Majda, 2013: A mechanism for catastrophic filter divergence in data assimilation for sparse observation networks. Nonlinear Processes Geophys., 20, 705712, https://doi.org/10.5194/npg-20-705-2013.

    • Search Google Scholar
    • Export Citation
  • Hamill, T. H., and J. S. Whitaker, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 27762790, https://doi.org/10.1175/1520-0493(2001)129<2776:DDFOBE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Hotta, D., E. Kalnay, Y. Ota, and T. Miyoshi, 2017: EFSR: Ensemble forecast sensitivity to observation error covariance. Mon. Wea. Rev., 145, 50155031, https://doi.org/10.1175/MWR-D-17-0122.1.

    • Search Google Scholar
    • Export Citation
  • Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123137, https://doi.org/10.1175/1520-0493(2001)129<0123:ASEKFF>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Ide, K., M. Ghil, and A. Lorenc, 1997: Unified notation for data assimilation: Operational, sequential and variational. J. Meteor. Soc. Japan, 75, 181189, https://doi.org/10.2151/jmsj1965.75.1B_181.

    • Search Google Scholar
    • Export Citation
  • Jazwinski, A. H., 1970: Stochastic Processes and Filtering Theory. Academic Press, 376 pp.

  • Karspeck, A. R., 2016: An ensemble approach for the estimation of observational error illustrated for a nominal 1° global ocean model. Mon. Wea. Rev., 144, 17131728, https://doi.org/10.1175/MWR-D-14-00336.1.

    • Search Google Scholar
    • Export Citation
  • Li, H., E. Kalnay, and T. Miyoshi, 2009: Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 523533, https://doi.org/10.1002/qj.371.

    • Search Google Scholar
    • Export Citation
  • Lorenc, A. C., 1986: Analysis methods for numerical weather prediction. Quart. J. Roy. Meteor. Soc., 112, 11771194, https://doi.org/10.1002/qj.49711247414.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E., 2005: Designing chaotic models. J. Atmos. Sci., 62, 15741587, https://doi.org/10.1175/JAS3430.1.

  • Miyoshi, T., E. Kalnay, and H. Li, 2013: Estimating and including observation error correlations in data assimilation. Inverse Probl. Sci. Eng., 21, 387398, https://doi.org/10.1080/17415977.2012.712527.

    • Search Google Scholar
    • Export Citation
  • Poterjoy, J., F. Zhang, and Y. Weng, 2014: The effects of sampling errors on the EnKF assimilation of inner-core hurricane observations. Mon. Wea. Rev., 142, 16091630, https://doi.org/10.1175/MWR-D-13-00305.1.

    • Search Google Scholar
    • Export Citation
  • Satterfield, E. A., and Coauthors, 2021: Statistical parameter estimation for observation error modelling: Application to meteor radars. Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications, S. K. Park and L. Xu, Eds., Vol. 4, Springer, 185–213, https://doi.org/10.1007/978-3-030-77722-7_8.

  • Stewart, L. M., S. L. Dance, and N. K. Nichols, 2008: Correlated observation errors in data assimilation. Int. J. Numer. Methods Fluids, 56, 15211527, https://doi.org/10.1002/fld.1636.

    • Search Google Scholar
    • Export Citation
  • Tandeo, P., P. Ailliot, M. Bocquet, A. Carrassi, T. Miyoshi, M. Pulido, and Y. Zhen, 2020: A review of innovation-based methods to jointly estimate model and observation error covariance matrices in ensemble data assimilation. Mon. Wea. Rev., 148, 39733994, https://doi.org/10.1175/MWR-D-19-0240.1.

    • Search Google Scholar
    • Export Citation
  • Waller, J. A., S. L. Dance, and N. K. Nichols, 2016: Theoretical insight into diagnosing observation error correlations using observation-minus-background and observation-minus-analysis statistics. Quart. J. Roy. Meteor. Soc., 142, 418431, https://doi.org/10.1002/qj.2661.

    • Search Google Scholar
    • Export Citation
  • Wu, P. Y., S. C. Yang, C. C. Tsai, and H. W. Cheng, 2020: Convective-scale sampling error and its impact on the ensemble radar data assimilation system: A case study of a heavy rainfall event on 16 June 2008 in Taiwan. Mon. Wea. Rev., 148, 36313652, https://doi.org/10.1175/MWR-D-19-0319.1.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 387 387 12
Full Text Views 151 151 1
PDF Downloads 134 134 1