Editorial Type:
Article
Article Type:
Research Article

A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts

Jeffrey L. Anderson Geophysical Fluid Dynamics Laboratory, Princeton University, Princeton, New Jersey

Search for other papers by Jeffrey L. Anderson in
Current site
Google Scholar
PubMed Close
and
Stephen L. Anderson Metron, Inc., Reston, Virginia

Search for other papers by Stephen L. Anderson in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Dec 1999
DOI:
https://doi.org/10.1175/1520-0493(1999)127<2741:AMCIOT>2.0.CO;2
Page(s):
2741–2758
Restricted access

Abstract

Knowledge of the probability distribution of initial conditions is central to almost all practical studies of predictability and to improvements in stochastic prediction of the atmosphere. Traditionally, data assimilation for atmospheric predictability or prediction experiments has attempted to find a single “best” estimate of the initial state. Additional information about the initial condition probability distribution is then obtained primarily through heuristic techniques that attempt to generate representative perturbations around the best estimate. However, a classical theory for generating an estimate of the complete probability distribution of an initial state given a set of observations exists. This nonlinear filtering theory can be applied to unify the data assimilation and ensemble generation problem and to produce superior estimates of the probability distribution of the initial state of the atmosphere (or ocean) on regional or global scales. A Monte Carlo implementation of the fully nonlinear filter has been developed and applied to several low-order models. The method is able to produce assimilations with small ensemble mean errors while also providing random samples of the initial condition probability distribution. The Monte Carlo method can be applied in models that traditionally require the application of initialization techniques without any explicit initialization. Initial application to larger models is promising, but a number of challenges remain before the method can be extended to large realistic forecast models.

Corresponding author address: Dr. Jeffrey L. Anderson, Geophysical Fluid Dynamics Laboratory, Princeton University, P.O. Box 308, Princeton, NJ 08542.

Email: jla@gfdl.gov

Abstract

Knowledge of the probability distribution of initial conditions is central to almost all practical studies of predictability and to improvements in stochastic prediction of the atmosphere. Traditionally, data assimilation for atmospheric predictability or prediction experiments has attempted to find a single “best” estimate of the initial state. Additional information about the initial condition probability distribution is then obtained primarily through heuristic techniques that attempt to generate representative perturbations around the best estimate. However, a classical theory for generating an estimate of the complete probability distribution of an initial state given a set of observations exists. This nonlinear filtering theory can be applied to unify the data assimilation and ensemble generation problem and to produce superior estimates of the probability distribution of the initial state of the atmosphere (or ocean) on regional or global scales. A Monte Carlo implementation of the fully nonlinear filter has been developed and applied to several low-order models. The method is able to produce assimilations with small ensemble mean errors while also providing random samples of the initial condition probability distribution. The Monte Carlo method can be applied in models that traditionally require the application of initialization techniques without any explicit initialization. Initial application to larger models is promising, but a number of challenges remain before the method can be extended to large realistic forecast models.

Corresponding author address: Dr. Jeffrey L. Anderson, Geophysical Fluid Dynamics Laboratory, Princeton University, P.O. Box 308, Princeton, NJ 08542.

Email: jla@gfdl.gov

Save
Cover Monthly Weather Review
Monthly Weather Review

  • Anderson, J. L., 1994: Selection of initial conditions for ensemble forecasts in a simple perfect model framework. J. Atmos. Sci.,53, 22–36.

  • ——, 1996a: Selection of initial conditions for ensemble forecasts in a simple perfect model framework. J. Atmos. Sci.,53, 22–36.

  • ——, 1996b: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate,9, 1518–1530.

  • ——, 1997: The impact of dynamical constraints on the selection of initial conditions for ensemble predictions: Low-order perfect model results. Mon. Wea. Rev.,125, 2969–2983.

  • ——, and V. Hubeny, 1997: A reexamination of methods for evaluating the predictability of the atmosphere. Non-linear Proc. Geosci.,4, 157–166.

  • Barker, T. W., 1991: The relationship between spread and forecast error in extended range forecasts. J. Climate,4, 733–742.

  • Brankovic, C., T. N. Palmer, F. Molteni, S. Tibaldi, and U. Cubasch, 1990: Extended-range predictions with ECMWF models: Time-lagged ensemble forecasting. Quart. J. Roy. Meteor. Soc.,116, 867–912.

  • Broomhead, D. S., R. Indik, A. C. Newell, and D. A. Rand, 1991: Local adaptive Galerkin bases for large-dimensional dynamical systems. Nonlinearity,4, 159–197.

  • Buizza, R., J. Tribbia, F. Molteni, and T. Palmer, 1993: Computation of optimal unstable structures for a numerical weather prediction model. Tellus,45A, 388–407.

  • ——, T. Petroliagis, T. Barkmeijer, M. Hamrud, A. Hollingsworth, A. Simmons, and N. Wedi, 1998: Impact of model resolution and ensemble size on the performance of an ensemble prediction system. Quart. J. Roy. Meteor. Soc.,124, 1935–1960.

  • Cramer, H., 1996: Mathematical Methods of Statistics. Princeton University Press, 575 pp.

  • Ehrendorfer, M., 1994: The Liouville equation and its potential usefullness for the prediction of forecast skill. Part I: Theory. Mon. Wea. Rev.,122, 703–713.

  • Epstein, E. S., 1969: Stochastic dynamic prediction. Tellus,21, 739–759.

  • Errico, R. M., and T. Vukicevic, 1992: Sensitivity analysis using an adjoint of the PSU/NCAR mesoscale model. Mon. Wea. Rev.,120, 1644–1660.

  • Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res.,99, 10 143–10 162.

  • ——, 1997: Advanced data assimilation for strongly nonlinear dynamics. Mon. Wea. Rev.,125, 1342–1354.

  • ——, and P. J. van Leeuwen, 1996: Assimilation of Geosat altimeter data for the Agulhas Current using the ensemble Kalman filter with a quasigeostrophic model. Mon. Wea. Rev.,124, 85–96.

  • Fukunaga, K., 1972: Introduction to Statistical Pattern Recognition. Academic Press, 369 pp.

  • Gleeson, T. A., 1970: Statistical-dynamical prediction. J. Appl. Meteor.,9, 333–344.

  • Harrison, M. S. J., D. S. Richardson, K. Robertson, and A. Woodcock, 1995: Medium-range ensembles using both the ECMSF T63 and unified models—An initial report. UKMO Tech. Rep. 153, 25 pp. [Available from United Kingdom Meteorological Office, London Rd., Bracknell, Berkshire RG12 2S2 United Kingdom.].

  • Henderson, H. W., and R. Wells, 1988: Obtaining attractor dimensions from meteorological time series. Advances in Geophysics, Vol. 30, Academic Press, 205–236.

  • Hoffman, R. N., and E. Kalnay, 1983: Lagged average forecasting, an alternative to Monte Carlo forecasting. Tellus,35a, 100–118.

  • Houtekamer, P. L., 1993: Global and local skill forecasts. Mon. Wea. Rev.,121, 1834–1846.

  • ——, and J. Derone, 1995: Methods for ensemble prediction. Mon. Wea. Rev.,123, 2181–2196.

  • ——, L. Lefaivre, J. Derome, H. Ritchie, and H. L. Mitchell, 1996:A system simulation approach to ensemble prediction. Mon. Wea. Rev.,124, 1225–1242.

  • Jazwinski, A. H., 1970: Stochastic Processes and Filtering Theory. Academic Press, 376 pp.

  • Kalnay, E., and A. Dalcher, 1987: Forecasting forecast skill. Mon. Wea. Rev.,115, 349–356.

  • Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev.,102, 409–418.

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci.,20, 130–141.

  • ——, 1980: Attractor sets and quasi-geostrophic equilibrium. J. Atmos. Sci.,37, 1685–1699.

  • ——, and V. Krishnamurthy, 1987: On the nonexistence of a slow manifold. J. Atmos. Sci.,44, 2940–2950.

  • Miller, R. N., M. Ghil, and F. Gauthiez, 1994: Advanced data assimilation in strongly nonlinear dynamical systems. J. Atmos. Sci.,51, 1037–1056.

  • Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc.,122, 73–120.

  • Moritz, R. E., and A. Sutera, 1981: The predictability problem: Effects of stochastic perturbations in multiequilibrium systems. Advances in Geophysics, Vol. 23, Academic Press, 345–383.

  • Murphy, J. M., 1988: The impact of ensemble forecasts on predictability. Quart. J. Roy. Meteor. Soc.,114, 463–493.

  • ——, 1990: Assessment of the practical utility of extended range ensemble forecasts. Quart. J. Roy. Meteor. Soc.,116, 89–125.

  • Palmer, T. N., 1993: Extended-range atmospheric prediction and the Lorenz model. Bull. Amer. Meteor. Soc.,74, 49–66.

  • Parrish, D. F., and J. C. Derber, 1992: The National Meteorological Center’s spectral statistical interpolation analysis system. Mon. Wea. Rev.,120, 1747–1763.

  • Silverman, B. W., 1986: Density Estimation for Statistics and Data Analysis. Chapman and Hall, 175 pp.

  • Tracton, S., and E. Kalnay, 1993: Operational ensemble forecasting at NMC: Practical aspects. Wea. Forecasting,8, 379–398.

  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc.,74, 2317–2330.

  • ——, and ——, 1996: Ensemble forecasting at NMC and the breeding method. NMC Office Note 407, 34 pp. [Available from NMC Office, 5200 Auth Rd., Camp Springs, MD 20746.].

  • ——, ——, S. M. Tracton, R. Wobus, J. Irwin, 1997: A synoptic evaluation of the NCEP ensemble. Wea. Forecasting,12, 140–153.

  • Van Leeuwen, P. J., and G. Evensen, 1996: Data assimilation and inverse methods in terms of a probabilistic formulation. Mon. Wea. Rev.,124, 2898–2912.

  • Vautard, R., and B. Legras, 1986: Invariant manifolds, quasi-geostrophy and initialization. J. Atmos. Sci.,43, 565–584.

  • Warn, T., and R. Menard, 1986: Nonlinear balance and gravity-inertial wave saturation in a simple atmospheric model. Tellus,38A, 285–294.

  • Wobus, R. L., and E. Kalnay, 1995: Three years of operational prediction of forecast skill at NMC. Mon. Wea. Rev.,123, 2132–2148.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1510 601 41
PDF Downloads 1129 354 32