Comparing Two Approaches for Assessing Observation Impact

Ricardo Todling Global Modeling and Assimilation Office, NASA Goddard Space Flight Center, Greenbelt, Maryland

Search for other papers by Ricardo Todling in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Langland and Baker introduced an approach to assess the impact of observations on the forecasts. In that approach, a state-space aspect of the forecast is defined and a procedure is derived ultimately relating changes in the aspect with changes in the observing system. Some features of the state-space approach are to be noted: the typical choice of forecast aspect is rather subjective and leads to incomplete assessment of the observing system, it requires availability of a verification state that is in practice correlated with the forecast, and it involves the adjoint operator of the entire data assimilation system and is thus constrained by the validity of this operator. This article revisits the topic of observation impacts from the perspective of estimation theory. An observation-space metric is used to allow inferring observation impact on the forecasts without the limitations just mentioned. Using differences of observation-minus-forecast residuals obtained from consecutive forecasts leads to the following advantages: (i) it suggests a rather natural choice of forecast aspect that directly links to the data assimilation procedure, (ii) it avoids introducing undesirable correlations in the forecast aspect since verification is done against the observations, and (iii) it does not involve linearization and use of adjoints. The observation-space approach has the additional advantage of being nearly cost free and very simple to implement. In its simplest form it reduces to evaluating the statistics of observation-minus-background and observation-minus-analysis residuals with traditional methods. Illustrations comparing the approaches are given using the NASA Goddard Earth Observing System.

Corresponding author address: Dr. Ricardo Todling, Global Modeling and Assimilation Office, NASA GSFC, Code 610.1, Greenbelt, MD 20771. E-mail: ricardo.todling@nasa.gov

Abstract

Langland and Baker introduced an approach to assess the impact of observations on the forecasts. In that approach, a state-space aspect of the forecast is defined and a procedure is derived ultimately relating changes in the aspect with changes in the observing system. Some features of the state-space approach are to be noted: the typical choice of forecast aspect is rather subjective and leads to incomplete assessment of the observing system, it requires availability of a verification state that is in practice correlated with the forecast, and it involves the adjoint operator of the entire data assimilation system and is thus constrained by the validity of this operator. This article revisits the topic of observation impacts from the perspective of estimation theory. An observation-space metric is used to allow inferring observation impact on the forecasts without the limitations just mentioned. Using differences of observation-minus-forecast residuals obtained from consecutive forecasts leads to the following advantages: (i) it suggests a rather natural choice of forecast aspect that directly links to the data assimilation procedure, (ii) it avoids introducing undesirable correlations in the forecast aspect since verification is done against the observations, and (iii) it does not involve linearization and use of adjoints. The observation-space approach has the additional advantage of being nearly cost free and very simple to implement. In its simplest form it reduces to evaluating the statistics of observation-minus-background and observation-minus-analysis residuals with traditional methods. Illustrations comparing the approaches are given using the NASA Goddard Earth Observing System.

Corresponding author address: Dr. Ricardo Todling, Global Modeling and Assimilation Office, NASA GSFC, Code 610.1, Greenbelt, MD 20771. E-mail: ricardo.todling@nasa.gov
Save
  • Anderson, B. D. O., and J. B. Moore, 1979: Optimal Filtering. Prentice-Hall, 357 pp.

  • Bloom, S. C., L. L. Takacs, A. M. da Silva, and D. Ledvina, 1996: Data assimilation using incremental analysis updates. Mon. Wea. Rev., 124, 12561271.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., C. Cardinali, G. Kelly, and J.-N. Thépaut, 2007: The value of observations. II: The value of observations located in singular-vector-based target areas. Quart. J. Roy. Meteor. Soc., 133, 18171832.

    • Search Google Scholar
    • Export Citation
  • Cardinali, C., R. Buizza, G. Kelly, M. Shapiro, and J.-N. Thépaut, 2007: The value of observations. III: Influence of weather regimes on targeting. Quart. J. Roy. Meteor. Soc., 133, 18331842.

    • Search Google Scholar
    • Export Citation
  • Chapnik, B., G. Desroziers, F. Rabier, and O. Talagrand, 2006: Diagnosis and tuning of observational error in a quasi-operational data assimilation setting. Quart. J. Roy. Meteor. Soc., 132, 543565.

    • Search Google Scholar
    • Export Citation
  • Chou, M.-D., and M. J. Suarez, 1999: A shortwave radiation parameterization for atmospheric studies. Tech. Rep. NASA-TM-104606, Vol. 15, NASA, 40 pp.

  • Cohn, S. E., 1997: An introduction to estimation theory. J. Meteor. Soc. Japan, 75, 257288.

  • Cohn, S. E., N. Sivakumaran, and R. Todling, 1994: A fixed-lag Kalman smoother for retrospective data assimilation. Mon. Wea. Rev., 122, 28382867.

    • Search Google Scholar
    • Export Citation
  • Cohn, S. E., A. da Silva, J. Guo, M. Sienkiewicz, and D. Lamich, 1998: Assessing the effects of data selection with the DAO physical-space statistical analysis system. Mon. Wea. Rev., 126, 29132926.

    • Search Google Scholar
    • Export Citation
  • Collins, N., and Coauthors, 2005: Design and implementation of components in the Earth System Modeling Framework. Int. J. High Perform. Comput. Appl., 19, 341350.

    • Search Google Scholar
    • Export Citation
  • Courtier, P., and O. Talagrand, 1987: Variational assimilation of meteorological observations with the adjoint vorticity equation. I: Theory. Quart. J. Roy. Meteor. Soc., 113, 13111328.

    • Search Google Scholar
    • Export Citation
  • Courtier, P., J.-N. Thépaut, and A. Hollingsworth, 1994: A strategy for operational implementation of 4D-Var, using an incremental approach. Quart. J. Roy. Meteor. Soc., 120, 13671387.

    • Search Google Scholar
    • Export Citation
  • Cucurull, L., 2010: Improvement in the use of an operational constellation of GPS radio occultation receivers in weather forecasting. Wea. Forecasting, 25, 749767.

    • Search Google Scholar
    • Export Citation
  • Daescu, D. N., and R. Todling, 2009: Adjoint estimation of the variation in model functional output due to the assimilation of data. Mon. Wea. Rev., 137, 17051716.

    • Search Google Scholar
    • Export Citation
  • Daley, R., 1992: The lagged-innovation covariance: A performance diagnostic for data assimilation. Mon. Wea. Rev., 120, 178196.

  • Dee, D. P., and A. M. da Silva, 1999: Maximum-likelihood estimation of forecast and observation error covariance parameters. Part I: Methodology. Mon. Wea. Rev., 127, 18221834.

    • Search Google Scholar
    • Export Citation
  • Derber, J. C., and A. Rosati, 1989: A global oceanic data assimilation system. J. Phys. Oceanogr., 19, 13331347.

  • Derber, J. C., and W.-S. Wu, 1998: The use of TOVS could-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, 22872299.

    • Search Google Scholar
    • Export Citation
  • Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005a: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 33853396.

    • Search Google Scholar
    • Export Citation
  • Desroziers, G., P. Brousseau, and B. Chapnik, 2005b: Use of randomization to diagnose the impact of observations on analyses and forecasts. Quart. J. Roy. Meteor. Soc., 131, 28212837.

    • Search Google Scholar
    • Export Citation
  • Ehrendorfer, M., 2007: A review of issues in ensemble-based Kalman filtering. Meteor. Z., 16, 795818.

  • Errico, R. M., 2007: Interpretation of an adjoint-derived observational impact measure. Tellus, 59A, 273276.

  • Errico, R. M., R. Gelaro, E. Novakovskaia, and R. Todling, 2007: General characteristics of stratospheric singular vectors. Meteor. Z., 16, 621634.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., and Y. Zhu, 2009: Examination of observation impacts derived from observing system experiments (OSEs) and adjoint models. Tellus, 61A, 179193.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., Y. Zhu, and R. M. Errico, 2007: Examination of various-order adjoint-based approximations of observation impact. Meteor. Z., 16, 685692.

    • Search Google Scholar
    • Export Citation
  • Gelaro, R., R. H. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX observation impact intercomparison experiment. Mon. Wea. Rev., 138, 40094025.

    • Search Google Scholar
    • Export Citation
  • Giering, R., T. Kaminski, R. Todling, R. Errico, R. Gelaro, and N. Winslow, 2005: Generating tangent linear and adjoint versions of NASA/GMAO’s FORTRAN-90 global weather forecast model. Lecture Notes in Computational Science and Engineering, H. M. Bücker et al., Eds., Vol. 50, Springer, 275–284.

  • Hollingsworth, A., and P. Lönnberg, 1989: The verification of objective analyses: Diagnostics of analysis system performance. Meteor. Atmos. Phys., 40, 327.

    • Search Google Scholar
    • Export Citation
  • Jazwinski, A. H., 1970: Stochastic Processes and Filtering Theory. Academic Press, 376 pp.

  • Kailath, T., 1968: An innovations control approach to least square estimation—Part I: Linear filtering in additive white noise. IEEE Trans. Automat. Contrib., AC-13, 646655.

    • Search Google Scholar
    • Export Citation
  • Kelly, G., J.-N. Thépaut, R. Buizza, and C. Cardinali, 2007: The value of observations. I: Data denial experiments for the Atlantic and the Pacific. Quart. J. Roy. Meteor. Soc., 133, 18031815.

    • Search Google Scholar
    • Export Citation
  • Kleespies, T. J., P. van Delst, L. M. McMillin, and J. Derber, 2004: Atmospheric transmittance of an absorbing gas. 6. OPTRAN status report and introduction to the NESDIS/NCEP Community Radiative Transfer Model. Appl. Opt., 43, 31033109.

    • Search Google Scholar
    • Export Citation
  • Koster, R. D., M. J. Suarez, A. Ducharne, M. Stieglitz, and P. Kumar, 2000: A catchment-based approach to modeling land surface processes in a general circulation model: 1. Model structure. J. Geophys. Res., 105 (D20), 24 80924 822.

    • Search Google Scholar
    • Export Citation
  • Langland, R. H., 2005: Observation impact during the North Atlantic TReC-2003. Mon. Wea. Rev., 133, 22972309.

  • Langland, R. H., and N. L. Baker, 2004: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56A, 189201.

    • Search Google Scholar
    • Export Citation
  • Lewis, J. M., K. D. Raeder, and R. M. Errico, 2001: Vapor flux associated with return flow over the Gulf of Mexico: A sensitivity study using adjoint modeling. Tellus, 53A, 7493.

    • Search Google Scholar
    • Export Citation
  • Lewis, J. M., S. Lakshmivarahan, and S. K. Dhall, 2006: Dynamic Data Assimilation: A Least Squares Approach. Encyclopedia of Mathematics and Its Applications Series, Vol. 104, Cambridge University Press, 647 pp.

  • Lin, S.-J., 2004: A “vertically Lagrangian” finite-volume dynamical core for global models. Mon. Wea. Rev., 132, 22932307.

  • Liu, J., and E. Kalnay, 2008: Estimation of observation impact without adjoint model in an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 134, 13271335.

    • Search Google Scholar
    • Export Citation
  • Liu, J., E. Kalnay, T. Miyoshi, and C. Cardinali, 2009: Analysis sensitivity calculation in an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 18421851.

    • Search Google Scholar
    • Export Citation
  • Lupu, C., P. Gauthier, and S. Laroche, 2011: Evaluation of the impact of observations on analyses in 3D- and 4D-Var based on information content. Mon. Wea. Rev., 139, 726737.

    • Search Google Scholar
    • Export Citation
  • Ménard, R., S. E. Cohn, L.-P. Chang, and P. M. Lyster, 2000: Assimilation of stratospheric chemical tracer observations using a Kalman filter. Part I: Formulation. Mon. Wea. Rev., 128, 26542671.

    • Search Google Scholar
    • Export Citation
  • Moorthi, S., and M. Suarez, 1992: Relaxed Arakawa–Schubert: A parameterization of moist convection for general circulation models. Mon. Wea. Rev., 120, 9781002.

    • Search Google Scholar
    • Export Citation
  • Rabier, F., and Coauthors, 2008: An update on THORPEX-related research in data assimilation and observing strategies. Nonlinear Processes Geophys., 15, 8194.

    • Search Google Scholar
    • Export Citation
  • Rienecker, M. M., and Coauthors, 2008: The GEOS-5 Data Assimilation System—Documentation of versions 5.0.1, 5.1.0, and 5.2.0. Tech. Rep. NASA-TM-104606, Technical Report Series on Global Modeling and Data Assimilation, Vol. 27, NASA, 101 pp.

  • Stappers, R. J. J., and J. Barkmeijer, 2011: Optimal linearization trajectories. Ninth Int. Workshop on Adjoint Model Applications in Dynamic Meteorology, Cefalu, Italy, NASA, 31 pp. [Available online at http://gmao.gsfc.nasa.gov/events/adjoint_workshop-9/presentations/Barkmeijer.pdf.]

  • Stieglitz, M., A. Ducharne, R. Koster, and M. Suarez, 2001: The impact of detailed snow physics on the simulation of snow cover and subsurface thermodynamics at continental scales. J. Hydrometeor., 2, 228242.

    • Search Google Scholar
    • Export Citation
  • Todling, R., 2009: An approach to assess observation impact based on observation-minus-forecast residuals. Proc. ECMWF Workshop on Diagnostics of Data Assimilation System Performance, Reading, United Kingdom, ECMWF, 199–202. [Available online at http://www.ecmwf.int/publications/library/do/references/list/20091231.]

  • Todling, R., and Y. Trémolet, 2008: The GMAO 4DVAR and its adjoint tools. 16th Int. TOVS Study Conf. (ITSC-16), Angra dos Reis, Brazil, International TOVS Working Group. [Available online at http://cimss.ssec.wisc.edu/itwg/itsc/itsc16/proceedings/A30_Todling.pdf.]

  • Trémolet, Y., 2007: First-order and higher-order approximations of observation impact. Meteor. Z., 16, 693694.

  • Trémolet, Y., 2008: Computation of observation sensitivity and observation impact in incremental variational data assimilation. Tellus, 60A, 964978.

    • Search Google Scholar
    • Export Citation
  • Wu, W.-S., R. J. Purser, and D. F. Parrish, 2002: Three-dimensional variational analysis with spacially inhomogeneous covariances. Mon. Wea. Rev., 130, 29052916.

    • Search Google Scholar
    • Export Citation
  • Zhu, Y., and R. Gelaro, 2008: Observation sensitivity calculations using the adjoint of the Gridpoint Statistical Interpolation (GSI) analysis system. Mon. Wea. Rev., 136, 335351.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 428 94 17
PDF Downloads 341 78 19