• Anderson, T. W., , and L. A. Goodman, 1957: Statistical Inference about Markov Chains. Ann. Math. Stat., 28 , 89110.

  • Baldwin, M. P., and Coauthors, 2001: The quasi-biennial oscillation. Rev. Geophys., 39 , 179229.

  • Cencini, M., , G. Lacorata, , A. Vulpiani, , and E. Zambianchi, 1999: Mixing in a meandering jet: A Markovian approximation. J. Phys. Oceanogr., 29 , 25782594.

    • Search Google Scholar
    • Export Citation
  • Crommelin, D. T., 2004: Observed nondiffusive dynamics in large-scale atmospheric flow. J. Atmos. Sci., 61 , 23842396.

  • Dall’Amico, M., 2005: Data-based master equations for the stratosphere. Ph.D. thesis, Ludwig-Maximilians-Universität of Munich, Germany, 71 pp. [Available online at http://edoc.ub.uni-muenchen.de/archive/00003890/.].

  • Dall’Amico, M., , and J. Egger, 2007: Empirical master equations. Part II: Application to stratospheric QBO, solar cycle, and northern annular mode. J. Atmos. Sci., 64 , 29963015.

    • Search Google Scholar
    • Export Citation
  • Durran, D. R., 1998: Numerical Methods for Wave Equations in Geophysical Fluid Dynamics. Springer, 465 pp.

  • Egger, J., 2001: Master equations for climatic parameter sets. Climate Dyn., 17 , 169177.

  • Egger, J., 2002: Master equations for Himalayan valley winds. Stochastic Dyn., 2 , 381394.

  • Egger, J., , and T. Jònsson, 2002: Dynamic models for Icelandic meteorological data sets. Tellus, 54A , 113.

  • Egger, J., , and M. Dall’Amico, 2007: Empirical master equations: Numerics. Meteor. Z., 16 , 139147.

  • Fraedrich, K., 1988: El Niño/Southern Oscillation predictability. Mon. Wea. Rev., 116 , 10011012.

  • Gardiner, C. W., 1983: Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences. Springer, 442 pp.

  • Gradišek, J., , S. Siegert, , R. Friedrich, , and I. Grabec, 2000: Analysis of time series from stochastic processes. Phys. Rev. E, 62 , 31463155.

    • Search Google Scholar
    • Export Citation
  • Kaplan, D., , and L. Glass, 1995: Understanding Nonlinear Dynamics. Springer, 420 pp.

  • Kloeden, P. E., , E. Platen, , and H. Schurz, 1997: Numerical Solution of SDE through Computer Experiments. 2d ed. Springer-Verlag, 292 pp.

  • Labitzke, K. G., , and H. van Loon, 1999: The Stratosphere, Phenomena, History, and Relevance. Springer, 179 pp.

  • Levy, P., 1948: Processus stochastiques et mouvement brownien. Gauthier-Villars, 365 pp.

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20 , 130141.

  • Mesinger, F., , and A. Arakawa, 1976: Numerical methods used in atmospheric models. Global Atmospheric Research Programme (GARP) Publication Series 17, World Meteorological Organization, 64 pp.

  • Nicolis, C., 1990: Chaotic dynamics, Markov processes and climate predictability. Tellus, 42A , 401412.

  • Nicolis, C., , W. Ebeling, , and C. Baraldi, 1997: Markov processes, dynamic entropies and the statistical prediction of mesoscale weather regimes. Tellus, 49A , 108118.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., 1993: Extended range atmospheric prediction and the Lorenz model. Bull. Amer. Meteor. Soc., 74 , 4965.

  • Pasmanter, R. A., , and A. Timmermann, 2002: Cyclic Markov chains with an application to an intermediate ENSO model. Nonlinear Proc. Geophys., 9 , 114.

    • Search Google Scholar
    • Export Citation
  • Press, W. H., , S. A. Teukolsky, , W. T. Vetterling, , and B. P. Flannery, 1999: Numerical Recipes in Fortran 77. The Art of Scientific Computing. Volume 1 of Fortran Numerical Recipes. 2d ed. Cambridge University Press, 933 pp.

    • Search Google Scholar
    • Export Citation
  • Siegert, S., , R. Friedrich, , and J. Peinke, 1998: Analysis of data sets of stochastic systems. Phys. Lett. A, 243 , 275280.

  • Spekat, A., , B. Heller-Schulze, , and M. Lutz, 1983: Über Großwetter und Markov-Ketten (“Großwetter” circulation analysed by means of Markov chains). Meteor. Rundsch., 36 , 243248.

    • Search Google Scholar
    • Export Citation
  • Thompson, D. W. J., , and J. M. Wallace, 2000: Annular modes in the extratropical circulation. Part I: Month-to-month variability. J. Climate, 13 , 10001016.

    • Search Google Scholar
    • Export Citation
  • Thuburn, J., 2005: Climate sensitivities via a Fokker–Planck adjoint approach. Quart. J. Roy. Meteor. Soc., 131 , 7392.

  • Uppala, S. M., and Coauthors, 2005: The ERA-40 re-analysis. Quart. J. Roy. Meteor. Soc., 131 , 29613012.

  • Vautard, R., , K. C. Mo, , and M. Ghil, 1990: Statistical significance test for transition matrices of atmospheric Markov chains. J. Atmos. Sci., 47 , 19261931.

    • Search Google Scholar
    • Export Citation
  • von Storch, H., , and F. W. Zwiers, 1999: Statistical Analysis in Climate Research. Cambridge University Press, 484 pp.

  • Zwanzig, R., 2001: Nonequilibrium Statistical Mechanics. Oxford University Press, 222 pp.

  • View in gallery

    The probability density function f(tn) at time tn. The transition coefficient Wi2 gives the probability that the variable q leaves grid interval 2 to enter i at the next time step.

  • View in gallery

    Time series of the x component of the Lorenz model with white Gaussian noise; Eqs. (7). The dashed time series is obtained by decreasing the initial x value by only 10−3 (the white noise forcing is identical).

  • View in gallery

    Observed state density, defined with Eq. (4) and integrated along the y axis, for a time series of length Δt  = 51 200 and sampling interval Dt = 0.020, obtained from Eqs. (7) by extending the integration that lead to the solid time series in Fig. 2. The starting point of the time series is marked by a circled dot. The isoline contour interval is 0.5 × 10−3, and every third isoline is bold. (a) The grid size used to partition the phase space, Dq = 5.00, is too coarse to resolve the “holes” in the attractor wings; these may be seen in (b), where the grid size Dq = 2.50; in (c) the grid size Dq = 1.25.

  • View in gallery

    The ratio Rw defined with Eq. (6) is shown as a function of the time series length, Δt , in nondimensional time units; the time resolution Dt = 0.020. Points mark the results obtained with a grid size Dq = 2.50; circles correspond to Dq = 1.25. The time series are generated by extending the numerical integration of Eqs. (7) which lead to the solid time series in Fig. 2. There is no stationary estimate of the transition coefficients for Dq = 1.25 and Δt  = 1.00.

  • View in gallery

    (a) The position at t  = 0, t  = 0.2, t  = 0.4, and t  = 0.6 of a cloud of 8000 points according to Eqs. (7); the position at t  = 0.8 is shown in (b). (c) The normalized density of ensemble members (points) for t  = 0.8, obtained from (b) after discretizing the phase space with Dq = 2.50 and integrating along the y axis; the isoline contour interval is 10−3.

  • View in gallery

    PDF forecasts for time t  = 0.8 as delivered by EMEs derived from time series of varying lengths; the grid size Dq = 2.50; the time resolution Dt = 0.020. The star marks the sharp initial condition (see also Fig. 5a). The PDF has been integrated along the y axis. The isoline contour interval is 10−3, and every third isoline is bold. The reference density of ensemble members is shown in Fig. 5c.

  • View in gallery

    Skill S for the EME forecasts shown in Fig. 6 (and others), shown as a function of the time series length; the grid size Dq = 2.50; the time resolution Dt = 0.020. The reference density of ensemble members is shown in Fig. 5c.

  • View in gallery

    The convergence coefficient Cw is shown as a function of the length of the longer time series, which is twice as long as the shorter one; the time resolution Dt = 0.020. Points mark the results obtained with a grid size Dq = 2.50; circles correspond to Dq = 1.25.

  • View in gallery

    PDF forecasts for time t  = 0.8 as delivered by EMEs derived from time series of varying time resolutions; the grid size Dq = 2.50; the time series length Δt  = 3200. The star marks the sharp initial condition (see also Fig. 5a). The PDF has been integrated along the y axis. (a) Dt = 0.004; (b) Dt = 0.020 (same as Fig. 6d); (c) Dt = 0.100. The isoline contour interval is 10−3, and every third isoline is bold. The reference density of ensemble members is shown in Fig. 5c.

  • View in gallery

    Influence of the time step, Dt, on an EME. The phase plane is partitioned into squares of unit grid size. The dash-dotted lines indicate parts of a single illustrative trajectory, the arrows show the direction of motion, and bold points indicate sampled observations. The dashed closed curves describe the domain of a cloud of states at two different times, t 0 (initial time) and (a later time). The PDF is initially (t  = t 0) set equal to 1/Dq = 1 in cell (2, 2); PDF values predicted by the EME are written in all cells where the PDF is nonzero (italics are used to ditinguish among successive time steps). (a) The time resolution Dt = (t 0)/3, whereas in (b) Dt = (t 0). The PDF smears faster in (a).

  • View in gallery

    Correlation functions rxx, rzz, and ryx as observed (solid), and as reproduced by EMEs derived from time series of varying time resolutions: dotted for Dt = 0.100, dash-dotted for Dt = 0.020, and dashed for Dt = 0.004; the grid size Dq = 2.50; the time series length Δt  = 3200.

  • View in gallery

    (left) PDF forecasts for time t  = 0.8 as delivered by an EME derived from a time series of the x and z components of the Lorenz model with stochastic forcing; the grid size Dq = 2.50; the time series length Δt  = 3200; the time resolution Dt = 0.100. The star marks the sharp initial condition. The isoline contour interval is 10−3, and every third isoline is bold. The reference density of ensemble members is shown in Fig. 5c. The corresponding forecast, obtained with a three-dimensional EME is shown in Fig. 9c. (right) The autocorrelation function of the first component, rxx, as observed (solid) and as reproduced by the EME (dotted).

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 19 19 1
PDF Downloads 4 4 0

Empirical Master Equations. Part I: Numerical Properties

View More View Less
  • 1 Meteorological Institute of the Ludwig-Maximilians-Universität of Munich, Munich, Germany
© Get Permissions
Full access

Abstract

In the atmospheric sciences, master equations are mainly used in a discrete time approximation to provide forecasts of the probability density function in a discretized phase space spanned by a few climate variables. The coefficients of an empirical master equation (EME) are estimated from the relative frequencies of transitions observed in time series of the variables. The quality of an EME depends on, among other things, the length and time resolution of the available time series. In this part of the paper, these dependencies are studied on the basis of data from the three-component Lorenz model with additional white noise forcing. Thus, time series of almost any length and time resolution can be generated easily, and probability density forecasts can be compared directly with the evolution of an ensemble of points. Useful results are obtained by partitioning the phase space into several hundred cells of equal grid size. The authors find that a threshold length of the time series exists beyond which improvements in the performance of the EME are hard to detect. It is even more surprising that the performance deteriorates with reduction of the time step. This is due to an increase in numerical diffusion. The choice of the dimensionality and the selection of the variables of the EME are very important. The results of this part of the paper provide useful guidelines for any application of the EME in the atmospheric sciences and elsewhere. The second part of the paper illustrates the usefulness of these guidelines through applications to stratospheric dynamics.

* Current affiliation: Walker Institute for Climate System Research, Department of Meteorology, University of Reading, Reading, United Kingdom

Corresponding author address: Dr. M. Dall’Amico, Walker Institute for Climate System Research, Department of Meteorology, University of Reading, P.O. Box 243, Earley Gate, Reading RG6 6BB, United Kingdom. Email: m.dallamico@reading.ac.uk

Abstract

In the atmospheric sciences, master equations are mainly used in a discrete time approximation to provide forecasts of the probability density function in a discretized phase space spanned by a few climate variables. The coefficients of an empirical master equation (EME) are estimated from the relative frequencies of transitions observed in time series of the variables. The quality of an EME depends on, among other things, the length and time resolution of the available time series. In this part of the paper, these dependencies are studied on the basis of data from the three-component Lorenz model with additional white noise forcing. Thus, time series of almost any length and time resolution can be generated easily, and probability density forecasts can be compared directly with the evolution of an ensemble of points. Useful results are obtained by partitioning the phase space into several hundred cells of equal grid size. The authors find that a threshold length of the time series exists beyond which improvements in the performance of the EME are hard to detect. It is even more surprising that the performance deteriorates with reduction of the time step. This is due to an increase in numerical diffusion. The choice of the dimensionality and the selection of the variables of the EME are very important. The results of this part of the paper provide useful guidelines for any application of the EME in the atmospheric sciences and elsewhere. The second part of the paper illustrates the usefulness of these guidelines through applications to stratospheric dynamics.

* Current affiliation: Walker Institute for Climate System Research, Department of Meteorology, University of Reading, Reading, United Kingdom

Corresponding author address: Dr. M. Dall’Amico, Walker Institute for Climate System Research, Department of Meteorology, University of Reading, P.O. Box 243, Earley Gate, Reading RG6 6BB, United Kingdom. Email: m.dallamico@reading.ac.uk

1. Introduction

In climate research it is common practice to fit a statistic or stochastic model to time series of observed variables. A familiar example is the projection of data on a regression model (e.g., von Storch and Zwiers 1999). The master equation (e.g., Gardiner 1983, 8–11) is a prognostic equation for the probability density function (PDF) among discrete states of a system. A discrete time approximation of the master equation is used in this paper in discretized phase spaces spanned by climate variables. The coefficients of a discrete time master equation are probabilities for cell transitions. These probabilities can be estimated from a time series of the variables (e.g., Egger 2001), hence the attribute empirical. The empirical master equation (EME) is described in section 2. The PDF forecasts given by an EME can be used for making probabilistic predictions. Another use of the EME is for studying the processes underlying the variable set. For example, Egger (2001) studied inter alia the evolution of the mean position, that is, a trajectory in phase space, from various initial conditions. This approach is applied in the second part of this paper (Dall’Amico and Egger 2007, hereafter Part II) for studying the relationship between the variables. Another example is offered by Pasmanter and Timmermann (2002), where the entropy production is derived from the coefficients of EMEs for assessing predictability time scales.

The EME is not new in the atmospheric sciences. For instance, Spekat et al. (1983) analyzed the zonal, mixed, and meridional weather regimes from a centennial time series on the basis of an empirical first order Markov model; the latter is closely related to the EME used in this paper (see section 2). Fraedrich (1988) applied inter alia a Markov chain model to the problem of estimating predictability time scales from annual time series of ENSO. Egger (2001) derived master equations from time series of the equatorial components of the global angular momentum of the atmosphere and related torques in order to analyze dynamics in the phase plane of two variables at any one time. Pasmanter and Timmermann (2002) applied the theory of cyclic Markov chains to the ENSO predictability problem; Crommelin (2004) studied the issue of atmospheric circulation regimes in Northern Hemisphere winter using a similar Markov model.

EMEs are numerical structures whose numerical properties are not yet well known. Some of the factors influencing the quality of an EME depend on choices made by the user such as the number and the choice of the variables, and the type and degree of phase space partition. Only rules of thumb are available to assess an adequate level of partitioning. Other factors are predetermined, as for instance the accessible climate time series and computer resources. The length and resolution of the available time series greatly influence the quality of an EME. The number of variables must be small because long data records are needed to correctly estimate the coefficients of an EME, and any addition to the number of variables involved dramatically increases the amount of data required (see also Crommelin 2004, section 2). This paper presents for the first time three-dimensional EMEs, yet in many applications the studied system can be expected to have more dimensions.1 It is the purpose of this part of the paper to address these problems by taking output from the Lorenz (1963) model with additional white noise forcing as a data basis (section 3). Systematic variations of grid size, time series length, sampling interval, and the number of variables are conducted in section 4 in order to study their effect on how well the EME reproduces the dynamics of the studied system. The conclusions are outlined in section 5. The results of this part of the paper provide guidelines for the application of this methodology to any problem in (and beyond) the atmospheric sciences. Real data of limited length will be considered in Part II, were these guidelines are applied to EMEs for the quasi-biennial oscillation of equatorial stratospheric wind (see the review by Baldwin et al. 2001), the 11-yr solar cycle (e.g., Labitzke and van Loon 1999), and the northern annular mode (Thompson and Wallace 2000). This two-part paper is based on work presented in Dall’Amico’s doctoral thesis (Dall’Amico 2005).

2. EMEs

For the sake of simplicity, let us first limit our considerations to the case of a single stochastic variable q. We consider partitions of that region of the q axis where the PDF is nonzero into grid intervals of equal size Dq (see Fig. 1). (Other approaches are found in literature.)2 This region extends between qmin = 0 and qmax = imaxDq. The center of grid interval (i) is located at (i) = qmin + (i − ½)Dq. Let us assume that observations are available at discrete points in time tn = t 0 + n Dt, where Dt is the sampling time interval; time is also considered as a discrete variable. Any state for which q satisfies qmin + (i − 1)Dqq < qmin + (i)Dq is located in (i). The PDF is defined in the following way: fi(tn)Dq is the probability that the state variable q is in grid interval (i) at time tn. It is normalized so that Σimaxi=1fi(tn)Dq = 1 for every n. Obviously 0 ≤ fi(tn) ≤ 1/Dq. The discrete time master equation describes the dynamics in this discretized version of the phase space (see Zwanzig 2001, 61–63):
i1520-0469-64-9-2981-e1
where the transition coefficient W ii gives the probability that the state variable q leaves grid interval (i) to enter (i′) at the next time step (see Fig. 1). Gains (losses) in probability density by transitions are described by the second (third) term on the right. Boundary conditions are not needed provided the domain of the solution contains all available observations and these adequately represent all those states which the system may possibly occupy. Equation (1), though linear in f, may capture nonlinear relationships between the variables. It can be written in a more compact form:
i1520-0469-64-9-2981-e2
where Dii = WiiδiiΣpWpi. Since ΣiDii = 0 probability is conserved. Further, transition probabilities cannot be negative, so all off-diagonal terms of 𝗗 will be positive or zero. Thus, according to standard matrix theorems, 𝗗 will have at least one zero eigenvalue and all other eigenvalues will have negative real parts describing an approach to equilibrium. Matrix 𝗗 may have more than one equilibrium state so that different initial states may lead to different stationary solutions at infinite time (Zwanzig 2001). From Eq. (2) follows:
i1520-0469-64-9-2981-e3
which shows the equivalence between a discrete time master equation in the form of Eq. (1) and a first-order Markov chain description in the form of Eq. (3). In this paper, we assume that the underlying systems are ergodic, so there is only one equilibrium state (see Spekat et al. 1983). The eigenvector associated with the biggest eigenvalue of matrix 𝗪; that is, the unit eigenvalue3 represents the climatological equilibrium distribution f. The PDF will tend, for large n, to this climatological mean (Spekat et al. 1983). (The value of the second biggest eigenvalue of 𝗪 gives information on how fast the system tends to f.) This equilibrium distribution is expected, at least for adequately long data records, to be very similar to the observed state density ρ:
i1520-0469-64-9-2981-e4
where Ni is the number of events when an observation falls in (i); hence Σimaxi=1ρiDq = 1. Stationarity may be guaranteed by checking that the time series does not end in a cell from where no transition is observed.
In an EME, the transition coefficients are estimated from the relative frequencies of transitions found in a time series obtained from observations or numerical model runs:
i1520-0469-64-9-2981-e5
where Mii is the number of transitions from (i) to (i′) observed in the time series. Conservation of probability requires ΣiW ii = 1 so that ΣiMii = Ni. Therefore Ni is not increased when a time series ends in (i). Transition probabilities have been estimated from data, for instance, by Spekat et al. (1983), Nicolis et al. (1997), Egger (2001, 2002), Pasmanter and Timmermann (2002), and Crommelin (2004).
The statistical estimate in Eq. (5) is more reliable the longer the time series is. The frequencies of transition from grid interval (i) toward (i′) are expected to be binomially distributed (Spekat et al. 1983). Out of Ni transitions from (i), Ni · W ii are expected to result in transitions to (i′), implying that Ni · W ii is the expected value from Ni independent binomial Bernoulli trials where at each trial the probability of “success;” that is, transition to (i′) is W ii and the probability of “no success” 1 − W ii (Spekat et al. 1983). For the central limit theorem, the distribution will tend, for large sample sizes, to a normal distribution whose 90% confidence interval has a half-width, recall Eq. (5), of cii = 1.645W ii(1 − W ii)/Ni. In this paper, partitions of 102 to 103 and more cells are considered, so even 106 transitions are conceivable; an individual consideration of the significance level of just a few selected transitions4 is senseless. Therefore use is made here of a weighed ratio averaged over the whole phase space:
i1520-0469-64-9-2981-e6
where ρi, recall Eq. (4), acts as a weight. The ratio Rw can be interpreted as a noise-to-signal ratio. A value of Rw of the order unity would suggest that the half-widths of the confidence intervals of the transition coefficients’ estimates are of the same magnitude as the estimates themselves and their statistical significance is low.

Depending on the system, transition coefficients may depend on time. Seasonal dependence was introduced for instance by Pasmanter and Timmermann (2002), who estimated transition matrices for each month of the year on the basis of a 640-yr-long ENSO model run. However, the amount of data available in observational records is in most cases insufficient to introduce such a time dependence.

The quality of an EME can be expected to depend on the statistical character of the data. It is advantageous if they stem from a Markovian system. In particular, perfect EMEs (an example is given in the appendix) require the system to be Markovian. Further, it is known that discretization, as well as projection onto a lower-dimensional phase space, is a source of non-Markovian effects. A formal test of the Markovian property should lead to the nonrejection of the hypothesis of a first-order Markov chain with a second-order Markov chain as an alternative hypothesis5 (e.g., Anderson and Goodman 1957). In this paper, we apply the approach suggested by Cencini et al. (1999), Egger (2001), and Pasmanter and Timmermann (2002), that is, to compare correlation functions as delivered by the EME to those obtained directly from the data (i.e., the sample correlation functions). This is a sensible way of testing whether the EME correctly reproduces the behavior of the studied system. The sample cross-covariance function for a nonnegative time lag τ between variables qκ and qη can be estimated from a time series as
i1520-0469-64-9-2981-eq1
(the primes indicate that anomalies from the mean are being considered). For negative lags, the sum goes from t 0τ to tN. Covariance functions as given by an EME arise from spatial contributions. This paper focuses on three-dimensional systems. The indices of the cubic cells be i1, i2, and i3, with (i1, i2, i3) ≡ (i). The contribution to the total estimate of the cross-covariance function for variables qκ and qη at time lag τ from an EME run starting with unit probability in cell (i01, i02, i03) ≡ (i0) is
i1520-0469-64-9-2981-eq2
where lags τ are nonnegative, λ refers to the anomaly of the λth coordinate of the center of the particular cell (recall Fig. 1), ρi0, and g is the consequent conditional probability to be calculated from the EME run with g(i0, t  = 0) = 1/Dq3 and g = 0 elsewhere, so that
i1520-0469-64-9-2981-eq3
Here η(i0, t  = 0; i, t  = τ) = Σiη g(i0, t  = 0; i, t  = τ)Dq3 is the anomaly of the mean qη coordinate at lag τ for the PDF run that had started with unit probability in (i0). The contributions sum up to the total estimate of the covariance function, sEME[qκ(t ), qη(t  + τ)] = Σi0[sqκqη(τ)]i0. Covariances for negative lags may be obtained from sEME[qκ(t ), qη(t  + τ)] = sEME[qη(t ), qκ(t τ)].

3. The Lorenz model with additional white noise forcing

The synthetic time series needed for studying the numerical properties of the EME are generated by numerical integration of the Lorenz (1963) model equations with additional white noise forcing:
i1520-0469-64-9-2981-e7
where variables x, y, z, and time are nondimensional, the Prandtl number (Pr), r, and b are parameters and α is a constant. A Runge–Kutta method of the fourth order with time step DtRK = 0.001 is used.6 New values for ξ are continuously generated throughout the numerical integration of the model equations. The hth value ξ assumes is ξh = γh/DtRK (Levy 1948, 40–42), where γh is the hth value output by a random number generator with Gaussian deviate, 〈γh〉 = 0, and 〈γhγj〉 = δhj [see the Box–Muller algorithm in Press et al. (1999)]. Hence 〈ξh〉 = 0 and 〈ξh ξj〉 = δhj/DtRK, which is the numerical approximation to a Dirac function. The evolution of a cloud of points is not altered by reducing Dtrk by a factor of 10. Time series are then created from the integration run by sampling variable values every time interval DtDtrk; represents the sampling interval of an atmospheric observation and becomes the time step of the EME.

The Lorenz model displays chaotic dynamics for a suitable choice of the parameters. We choose the standard parameter set Pr = 10, r = 28, and b = 8/3 (e.g., Lorenz 1963; Palmer 1993; Kaplan and Glass 1995). The state vector q = (q1, q2, q3) = (x, y, z) evolves around the famous Lorenz attractor with its two butterfly-wing-shaped lobes. A trajectory far from the Lorenz attractor rapidly approaches the attractor, whereas trajectories near the attractor show sensitive dependence on initial conditions.

The Lorenz model has often been taken as a paradigm of large-scale atmospheric circulation (e.g., Palmer 1993). Though only three-dimensional, it reflects many of the properties of the full climate system (Thuburn 2005). It is a shortcoming of the model in comparison to the atmosphere that its diffusivity in phase space is fairly low. White noise in inserted partly for this reason. Moreover, the divergence of the Lorenz model without stochastic forcing is, with · (, , ż) = −Rr − 1 − b, negative and constant. This means that the phase space occupied by the trajectories is shrinking continuously onto the Lorenz attractor. The additional white noise forcing acts against this frictional contraction, leading to a stationary stochastic system (see von Storch and Zwiers 1999, 1–2), as needed for EMEs with time-constant transition coefficients. A noise amplitude α = 2.5 is chosen, such that a modest diffusion of the states is obtained (see Figs. 5a,b), which does not drastically alter character and shape of the trajectories (see Fig. 2).

Three time scales are associated with the model. The one describing the evolution of a trajectory about the (weakly) unstable fixed point at the center of each attractor wing is t win ≈ 0.7. The residence time, t res, in a wing varies approximately between 1 and 10 time units. These time scales can be seen in Fig. 2 (solid line), where the evolution of the x component from an arbitrary initial state is shown as a function of time. The dependence on initial conditions is revealed by the evolution of the dashed line, where the white noise forcing is identical but the initial conditions are slightly different from those of the solid line. There exists also a diffusive time scale, t difL/α, due to the stochastic forcing (L ≈ 20 is the diameter of a wing), with t win < t dif.

The model’s PDF evolves according to the Fokker–Planck equation
i1520-0469-64-9-2981-e8
Thuburn (2005) discussed the numerical solution of the steady Fokker–Planck equation in an application to a stochastic version of the Lorenz model. The use of Fokker–Planck equations in similar contexts to those where EMEs are applied is discussed in section 4c.

Both the choice of the Lorenz model and that of a rather weak white noise forcing are quite challenging. The attractor has a complex shape and a relatively fine grid size is required in order to resolve its wings. The demands on the quality of the EME would have been less stringent if a system with a more trivial attractor had been chosen or, as often seen in literature, if a stronger noise had been used [as in e.g., Gradišek et al. (2000) and Thuburn (2005), where Fokker–Planck equations are considered]. With weak white noise, a cloud of points smears quite slowly. In contrast, as α grows, the diffusion of a cloud of points due to the noise term becomes comparable to the numerical diffusion acting on the PDF forecast by an EME derived from a long time series (see section 4), leading to better predictive skill when comparing the two.

4. Results

The impact of the choice of the main numerical parameters on the quality of the EME is discussed in the following subsections. The transition coefficients of each EME are estimated from a single time series whose length is indicated as Δt .

a. Grid size

The choice of the grid size depends on the problem at hand. At best, a few guidelines can be formulated. The data of a “long” time series beginning near the attractor are included in a parallelepiped with a volume of about 50 · 60 · 50. This volume is the domain of solution. The grid size is set initially depending on the features that have to be resolved. In Fig. 3 is shown how the observed state density for a time series of length Δt  = 51200 varies depending on the grid size chosen to partition the phase space. With a grid size Dq = 5.00, the above parallelepiped is partitioned into 1200 cells (see Fig. 3a, where −30 ≤ y < 30). Almost 250 of these cells intersect the attractor. A relatively fine grid size is needed because of the complicated structure of the attractor. As may be seen in Fig. 3a, the choice of Dq = 5.00 does not resolve the “holes” in each of the butterfly wings. They are resolved with Dq = 2.50 (Fig. 3b) and the overall picture contains more details when Dq = 1.25 (Fig. 3c). However, the required computer cost grows dramatically by reducing the grid size. With Dq = 2.50, approximately 1000 cells intersect the attractor, and with Dq = 1.25 almost 4750 cells. Moreover, as the number of cells representing the domain of the solution grows by decreasing Dq, since transitions to and from all cells have to be considered, the size of the array representing the transition matrix, 𝗪, grows with the square of the number of cells. The time needed for computing correlation functions increases with approximately the eighth power of the inverse of the grid size for this three-dimensional case. Therefore the grid size is set at Dq = 2.50.

Figure 4 reports the ratios Rw for grid sizes Dq = 2.50 and Dq = 1.25 as a function of the length of the time series used to estimate 𝗪 for an arbitrary time resolution; the time series must be about eight times longer for the Rw values for Dq = 0.125 to be as low as those for Dq = 2.50. While deriving an EME from an observational time series, the grid size might have to be adjusted according to the ratio Rw and to the correlation functions delivered by the EME. Correlation functions as delivered by EMEs confirm the choice Dq = 2.50 in terms of quality and of needed computer resources also with respect to other grid sizes (not shown).

b. Time series length

The length of the time series affects the value of the ratio Rw, as may be seen in Fig. 4. The values of Rw in Fig. 4 are quite high for short time series even for the chosen grid size, Dq = 2.50. For Δt  < 200, for instance, Rw > 0.5, which means that the statistical significance of the transition coefficients’ estimates is low.

By affecting the estimate of the transition coefficients, the time series length can also have an impact on the PDF forecasts. As a case study, PDF forecasts from a particular initial condition, which are given by EMEs derived from time series of different lengths, are compared to the evolution of an ensemble of points. The latter is obtained by integration of the Eqs. (7) and is shown for t  = 0, 0.2, 0.4, 0.6 in Fig. 5a and for t  = 0.8 in Fig. 5b. These points are initially located in a cell of the phase space discretized with Dq = 2.50. This cell includes part of the attractor. Palmer (1993) pointed out that there are portions of the Lorenz attractor where trajectories depart fairly slowly. In other cases, an ensemble of points reaches the splitting region of the attractor and adjacent trajectories diverge toward the two different wings, so that there are regions of the attractor that are relatively more sensitive to initial conditions. The cloud in Fig. 5a moves partly through the splitting region between t  = 0.3 and t  = 0.4, and only very few points are located on the left-hand side of the attractor at t  = 0.8. Losses in prediction skill are expected if the PDF evolves occupying the splitting region imprecisely. Initial conditions as in Fig. 4 of Palmer (1993), which either evolve far away or directly through the splitting region, represent an easier task for the EME. The one in Fig. 5a is challenging and shows how the estimation of 𝗪 from a very short time series may lead to very poor forecasts. The forecasts are considered at time t  = 0.8. This time is close to the minimum time generally needed for nearby initial conditions to diverge toward the two wings of the attractor (see Palmer 1993). The reference density of ensemble members, μ, that is, the relative frequency of points per cell integrated along the y axis, is shown for t  = 0.8 in Fig. 5c. To compare the PDF with μ, the latter is normalized accordingly: ΣiμiDq3 = 1. In Fig. 5c there is a well-marked head on the right wing of the attractor and a long thinly populated tail. The PDF forecasts in Fig. 6 are compared to Fig. 5c. These forecasts are given by EMEs derived from time series of different lengths. The time series are obtained by extending the integration shown by the solid line in Fig. 2. No forecast is possible if the time series is shorter than 50 time units since no observation falls into the starting cell. The forecast shown in Fig. 6a is of poor quality. The forecasts shown in Figs. 6d,e,f are of better quality and almost identical. In these forecasts, the PDF is higher on the right wing of the attractor, just as the density of ensemble members in Fig. 5c; the PDF on the left wing is a result of the numerical diffusion (see below), which leads to nonzero PDF for time t  = 0.4 (see Fig. 5a) on a wider region, a part of which evolves to the left attractor wing.

The predictive skill of the PDF forecasts given by the EMEs is quantified by introducing a forecast skill score S. At any given time, S is a weighted measure of the mean square deviation between μ (Fig. 5c) and f (Fig. 6), the weight being μ (Gardiner 1983, p. 40). Moreover, S is normalized with the outcome in the case where the two patterns are fully apart (S = 1):
i1520-0469-64-9-2981-e9
A perfect forecast scores zero. Values of S for the PDF forecasts for t  = 0.8 shown in Fig. 6 (and others) are shown in Fig. 7 as a function of the time series length. A value S > 1 may occur if the PDF largely overestimates the point density in some cells. This can be the case when very short time series are used. The scarcity of data implies that the small number of observed transitions carry higher PDF values. This could also result in a better skill measure than that obtained with a longer time series, as is the case for the lower S values corresponding to Δt  = 100 and Δt  = 200 (see also the forecast corresponding to Δt  = 200 in Fig. 6b). For Δt  = 50 (Fig. 6a) however, higher PDF values are erroneously carried to the left side of the attractor. It is interesting that the skill hardly improves for time series lengths Δt  > 103, which is a threshold length for the chosen grid size Dq = 2.50 and for the time resolution Dt = 0.020. For these time series lengths, Rw is of order 0.1 (or lower). If the time series is obtained from observations, no skill may be computed as just one realization occurs in reality; however, Rw may still be calculated.
The results in Fig. 7 imply that a substantial prolongation of the times series does not automatically lead to an ever-increasing forecast quality. One will at most achieve a correct estimate of the transition coefficients relative to the chosen grid size and the used time resolution. To further investigate this point, we consider the coefficient
i1520-0469-64-9-2981-e10
which is the normalized vectorial distance between two matrices 𝗪(l) and 𝗪(s) estimated on the base of the longer and the shorter of two time series, respectively. The coefficient Cw is shown in Fig. 8 as a function of the longer time series length when this has twice the length of the shorter. As the time series length goes to infinity, Cw tends to zero, suggesting that 𝗪 converges. The weight ρi(l) reduces the effect of outliers in very long time series and at the borders of the attractor, which influence parts of 𝗪 that are not relevant for the evolution of the PDF. (Calculations carried out without weighting do not lead to convergence.) The results in Fig. 8 confirm that the extension of time series to extreme lengths does not produce a noticeable improvement in the estimate of 𝗪. As an effect of using a finer grid size, Dq = 1.25, corresponding Cw values in Fig. 8 are higher; the same results from the use of a coarser time resolution (not shown).

c. Time resolution

The transition coefficients, and consequently their estimate after Eq. (5), depend on the time step. On one hand, a fine time resolution is desirable in order to improve the statistical significance of the transition coefficients. The ratio Rw generally decreases with increasing ΣiNi as is the case for a long time series and a fine time resolution. On the other hand, the computing time increases by reducing the time step and, since the time scales of the investigated phenomena are usually known, Dt should not be unnecessarily fine. The time resolution also sets a limit to the highest systems’ frequency that an EME will be able to reproduce.

Figure 9 shows PDF forecasts obtained with different time steps. Figure 9b shows for reference the forecast obtained with a time series of length Δt  = 3200 and time resolution Dt = 0.020 (same as Fig. 6d), that is, longer than the threshold length discussed in section 4b. Figure 9a shows the PDF forecast delivered by the EME when the sampling frequency of the same time series as for Fig. 9b is increased by a factor of 5, that is, Dt = 0.004. In contrast, for Fig. 9c, the time resolution is coarser than in Fig. 9b by a factor of 5, Dt = 0.100. The best forecast of the three is Fig. 9c, where the PDF, integrated along the y axis, exceeds a value of 0.0100 on the right wing of the attractor. Surprisingly, this forecast is the one obtained with the coarsest time resolution, Dt = 0.100. The skill scores S computed for the forecasts in Fig. 9 confirm this. For the finest time resolution the skill is the worst at 0.83 (Fig. 9a), and improves to 0.75 with a time resolution Dt = 0.020 (Fig. 9b); the skill is best at 0.48 for the coarsest time resolution (Fig. 9c).

The two drawings in Fig. 10 represent a simple example, which helps one understand the surprisingly rapid smearing of the PDF observed in connection with a finer time resolution. In Fig. 10b, the time step is 3 times larger than in Fig. 10a. The PDF values written in Fig. 10 are predicted by this illustrative EME on the base of the few observations available. It is easy to follow the evolution of the PDF and to realize that the PDF smears much faster in the case with shorter Dt (Fig. 10a). Given good estimates of the transition coefficients, the EME makes best forecasts, whatever the initial condition, over one single time step no matter how large Dt. (The PDF forecast is almost perfect if the cloud of states initially coincides with a cell.) For the previous case study (Fig. 9), Dt = 0.8 is the best time resolution since only one time step is needed for prediction. After any time step, the PDF is spread evenly over each cell. This spreading is unavoidable and implies diffusion. This is a specific example of numerical diffusion. While Fig. 10 is illustrative, exact statements about the numerical spreading of PDFs can be made for simple systems; in the appendix, an explicit solution to the problem of numerical spreading is given for an advective case.

Figure 11 shows some correlation functions7 estimated directly from the time series (solid lines) and delivered by the EMEs that gave the forecasts in Fig. 9. The transition coefficients of these EMEs are estimated from time series differing only in their sampling interval whereas the starting point, the evolution, and the length are the same. Figure 11 shows that all EMEs approximate the decay of the correlation functions extremely well. The EME based on the time series with the coarsest time resolution does the best job in reproducing the sample correlations, confirming the results seen in the previous case study. This finding contrasts with standard results where the numerical diffusion is reduced for smaller time steps (Mesinger and Arakawa 1976). Given the low diffusion in the time series due to the model’s chaotic nature and due to white noise forcing, numerical diffusion prevails unless a much finer grid size is used at which the diffusion in the data dominates the advective transport (see Durran 1998, p. 139). As the time step grows, the ratio between the scale of diffusion in the data and the grid size increases, thus weakening the impact of numerical diffusion. As can be seen in Fig. 11, the decorrelation time for the time series in question is much longer than the time resolutions considered (Dt = 0.004, 0.020, and 0.100). This makes numerical diffusion the sole explanation for the result that the PDF forecasts delivered by the EME improve by increasing the time step. A reduction to numerical diffusion can also be sought by reducing the grid size. However, besides implying a growth of the ratio Rw, a grid size reduction also causes a rapid increase of the requirements for the computing resources.

The encouraging results in Figs. 9c and 11 (dashed–dotted lines) have important practical implications. With Dq = 2.50, about ζ = 103 cells intersect the region of the phase space occupied by the observations and up to ζ2 = 106 transitions are conceivable. Yet the EME was derived from a time series of length Δt  = 3200 and time resolution Dt = 0.100, that is, of only 32 × 103 data points. For the EME in question, Rw = 0.29. Transitions occur mainly toward the cells along the main direction of movement (not backward or in any transverse direction), and in this case merely 3.2 × 103 transitions can actually take place. For an atmospheric application where the region occupied by the observations has a much simpler structure than the attractor of Eqs. (7), the grid size can be initially set to a value that gives a few hundred cells. The grid size can then be adjusted depending on the ratio Rw for the considered time step. The adherence of the correlation functions may also suggest a change of the numerical parameters. This approach is adopted in Part II with positive results.

The EME used here is discrete in time and phase space. The Fokker–Planck equation, on the other hand, is a partial differential equation that can also be used to predict the PDF at least under favorable circumstances (Zwanzig 2001). In practice, the Fokker–Planck equation must be solved numerically, whereby it is transformed into a finite difference equation (e.g., Thuburn 2005). The drift and diffusion terms of a Fokker–Planck equation may also be estimated from data (Siegert et al. 1998). This technique was applied, for example, by Egger and Jònsson (2002) to meteorological observations in the Icelandic region. An empirical discrete Fokker–Planck equation can be seen as a particular case of an EME. Master equations are much more general than Fokker–Planck equations (Zwanzig 2001). In a discretized version of the phase space, the coefficients within the parentheses in Eq. (8) can also be estimated from a time series of the variables. Gradišek et al. (2000) applied this technique to time series of various origin, among these some stemming from the Lorenz (1963) model with additional stochastic forcings. They found that the maximum acceptable time step needed for the estimates of the drift and diffusion coefficients to converge is shorter than the time step required for the integration of the corresponding differential equations. Moreover, the integration of the Fokker–Planck equation is constrained by the Courant–Friedrichs–Levy criterion and its numerical implementation involves normally only neighboring cells. Large time steps can be used only with the EME, which considers transitions to any cell in phase space. Within a time Dt = 0.100, for instance, the state vector may jump to a cell separated from the previous one by up to 12 cells of grid size Dq = 2.50.

d. Dimension of the EME

In a more realistic situation, the EME does not contain as many variables as the investigated system. In practice, EMEs with, say, three variables are applied to systems with many variables. To assess the effect of considering a reduced set of variables, a two-dimensional EME is derived from a projection on the (x, z) plane of the same time series, which led to the prediction in Fig. 9c. In this case, variable y does not appear in the EME. The left panel of Fig. 12 shows the forecast delivered by such an EME starting from the same initial condition as in Fig. 9c. The partition of the x and z axes is unchanged. The value of Rw is now 0.19, whereas Rw was 0.29 in the three-dimensional case. Such a reduction should not surprise since the same number of data points as in the three-dimensional case is now used to estimate a much lower number of transitions. The forecast in the left panel of Fig. 12, where the PDF is about equally distributed on either attractor wing, is clearly worse than the corresponding three-dimensional one (Fig. 9c). The evolution of the autocorrelation function of the x component, rxx, is shown in the right panel of Fig. 12 and beyond about 0.3 time units it is not as good as the one delivered by the three-dimensional EME. In the case of an observational time series, the introduction of another variable should be considered.8 While deriving EMEs from atmospheric datasets, however, it is practically impossible to consider a complete set of variables. In the example of Fig. 12, the low value of Rw might have induced optimism, yet this EME does not quite a good job in reproducing the dynamics of the studied system. An Rw value below, say, 0.4, suggests that the amount of data available is adequate for the number of transitions to be estimated. For Rw values between 0.4 and 0.6, much caution in the study of the EME is recommended if the user decides not to increase the grid size. Values above 0.6 should lead to the use of a coarser grid size. Whatever the Rw value, caution is recommended in evaluating results obtained from an EME if its correlation functions suggest that the behavior of the system is not adequately reproduced. An atmospheric application requiring a five-dimensional rather than a three-dimensional EME is discussed in section 3b of Part II.

5. Conclusions

EMEs are constructed directly from data and provide a model of the phase space dynamics of a system. They may capture nonlinear behavior. A study of the numerical properties of EMEs has been the object of this part of the paper. This is the first time that grid size, time series length, time step, and phase space dimensionality have been studied systematically. The time series needed for the numerical study have been generated by numerical integration of the equations of the Lorenz model with additional white noise forcing. Thus, time series with the desired characteristics could be generated easily. The choice of the dynamical model has been quite challenging. The attractor of this model has a fairly complicated structure so that a fine grid size is needed in order for the EME to replicate the motion in phase space. Moreover, the model’s diffusivity due to its chaotic nature and due to the white noise forcing is fairly low, making numerical diffusion a dominant factor. The significance of the estimates of the transition coefficients of the EME has been assessed in terms of a weighted averaged ratio, Rw, between the half-width of the confidence intervals of the transition coefficient estimates and the estimates themselves. The adherence of correlation functions as delivered by the EME to those estimated from the data has been used as a test of how well the EME reproduces the dynamics of the system.

We find that:

  1. The grid size choice is a compromise between desired resolution in phase space on one side, and available data and computer resources on the other. A moderately fine grid size has delivered very encouraging results.
  2. The estimate of the transition coefficients improves with growing time series length. Moreover, we find that, for a given grid size and time resolution, there is a threshold time series length beyond which the forecast skill does not improve; this threshold length appears to be moderate. For the Lorenz model with additional white noise forcing, we find a threshold of order 103 time units. An approach considering the transition coefficient matrix 𝗪 as a multidimensional vector shows that 𝗪 converges with growing time series length.
  3. Surprisingly, EMEs derived from time series with coarser time resolutions show better forecast skills and deliver better correlation functions. This is due to a decrease in the numerical diffusion acting on the PDF forecasts as the time step grows. Best forecasts are obtained with a single time step.
  4. All in all, encouraging results are obtained considering partitions of the attractor into about 1000 cells with a time series consisting of only 3.2 × 104 data points. This result supports the applicability of the method to atmospheric time series. For a three-dimensional atmospheric application, we recommend to begin by considering partitions of the “data cloud” into several hundred cells. The numerical parameters can be adjusted according to the corresponding value of the ratio Rw and, finally, to the quality of the correlation functions delivered by the EME.
  5. The example chosen is unusual in that the EME has the same number of variables as the Lorenz model. An EME derived for time series of only the x and z components leads, despite a low Rw value, to poor PDF predictions. The adherence of the correlation functions deteriorates with respect to the three-dimensional case. This result demonstrates the importance of the number of variables and calls for caution in cases when the correlation functions delivered by the EME strongly disagree with the ones obtained from the data. This disagreement can occur particularly when high dimensional systems are studied, suggesting, if feasible, to increase the dimension of the EME.

In Part II, EMEs are derived from time series obtained from the ERA-40 re-analysis (Uppala et al. 2005) and observations. These time series are limited in time and hence the existence of a threshold length may not be tested. However, the remaining results of this numerical study, including the surprising role of time resolution, are confirmed.

Acknowledgments

Financial support was provided by the German Ministry of Education and Research and the German Aerospace Center within KLIMESTO, a project of the German Climate Research Program, Contract 01LD0033. See Part II for more thorough acknowledgments.

REFERENCES

  • Anderson, T. W., , and L. A. Goodman, 1957: Statistical Inference about Markov Chains. Ann. Math. Stat., 28 , 89110.

  • Baldwin, M. P., and Coauthors, 2001: The quasi-biennial oscillation. Rev. Geophys., 39 , 179229.

  • Cencini, M., , G. Lacorata, , A. Vulpiani, , and E. Zambianchi, 1999: Mixing in a meandering jet: A Markovian approximation. J. Phys. Oceanogr., 29 , 25782594.

    • Search Google Scholar
    • Export Citation
  • Crommelin, D. T., 2004: Observed nondiffusive dynamics in large-scale atmospheric flow. J. Atmos. Sci., 61 , 23842396.

  • Dall’Amico, M., 2005: Data-based master equations for the stratosphere. Ph.D. thesis, Ludwig-Maximilians-Universität of Munich, Germany, 71 pp. [Available online at http://edoc.ub.uni-muenchen.de/archive/00003890/.].

  • Dall’Amico, M., , and J. Egger, 2007: Empirical master equations. Part II: Application to stratospheric QBO, solar cycle, and northern annular mode. J. Atmos. Sci., 64 , 29963015.

    • Search Google Scholar
    • Export Citation
  • Durran, D. R., 1998: Numerical Methods for Wave Equations in Geophysical Fluid Dynamics. Springer, 465 pp.

  • Egger, J., 2001: Master equations for climatic parameter sets. Climate Dyn., 17 , 169177.

  • Egger, J., 2002: Master equations for Himalayan valley winds. Stochastic Dyn., 2 , 381394.

  • Egger, J., , and T. Jònsson, 2002: Dynamic models for Icelandic meteorological data sets. Tellus, 54A , 113.

  • Egger, J., , and M. Dall’Amico, 2007: Empirical master equations: Numerics. Meteor. Z., 16 , 139147.

  • Fraedrich, K., 1988: El Niño/Southern Oscillation predictability. Mon. Wea. Rev., 116 , 10011012.

  • Gardiner, C. W., 1983: Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences. Springer, 442 pp.

  • Gradišek, J., , S. Siegert, , R. Friedrich, , and I. Grabec, 2000: Analysis of time series from stochastic processes. Phys. Rev. E, 62 , 31463155.

    • Search Google Scholar
    • Export Citation
  • Kaplan, D., , and L. Glass, 1995: Understanding Nonlinear Dynamics. Springer, 420 pp.

  • Kloeden, P. E., , E. Platen, , and H. Schurz, 1997: Numerical Solution of SDE through Computer Experiments. 2d ed. Springer-Verlag, 292 pp.

  • Labitzke, K. G., , and H. van Loon, 1999: The Stratosphere, Phenomena, History, and Relevance. Springer, 179 pp.

  • Levy, P., 1948: Processus stochastiques et mouvement brownien. Gauthier-Villars, 365 pp.

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20 , 130141.

  • Mesinger, F., , and A. Arakawa, 1976: Numerical methods used in atmospheric models. Global Atmospheric Research Programme (GARP) Publication Series 17, World Meteorological Organization, 64 pp.

  • Nicolis, C., 1990: Chaotic dynamics, Markov processes and climate predictability. Tellus, 42A , 401412.

  • Nicolis, C., , W. Ebeling, , and C. Baraldi, 1997: Markov processes, dynamic entropies and the statistical prediction of mesoscale weather regimes. Tellus, 49A , 108118.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., 1993: Extended range atmospheric prediction and the Lorenz model. Bull. Amer. Meteor. Soc., 74 , 4965.

  • Pasmanter, R. A., , and A. Timmermann, 2002: Cyclic Markov chains with an application to an intermediate ENSO model. Nonlinear Proc. Geophys., 9 , 114.

    • Search Google Scholar
    • Export Citation
  • Press, W. H., , S. A. Teukolsky, , W. T. Vetterling, , and B. P. Flannery, 1999: Numerical Recipes in Fortran 77. The Art of Scientific Computing. Volume 1 of Fortran Numerical Recipes. 2d ed. Cambridge University Press, 933 pp.

    • Search Google Scholar
    • Export Citation
  • Siegert, S., , R. Friedrich, , and J. Peinke, 1998: Analysis of data sets of stochastic systems. Phys. Lett. A, 243 , 275280.

  • Spekat, A., , B. Heller-Schulze, , and M. Lutz, 1983: Über Großwetter und Markov-Ketten (“Großwetter” circulation analysed by means of Markov chains). Meteor. Rundsch., 36 , 243248.

    • Search Google Scholar
    • Export Citation
  • Thompson, D. W. J., , and J. M. Wallace, 2000: Annular modes in the extratropical circulation. Part I: Month-to-month variability. J. Climate, 13 , 10001016.

    • Search Google Scholar
    • Export Citation
  • Thuburn, J., 2005: Climate sensitivities via a Fokker–Planck adjoint approach. Quart. J. Roy. Meteor. Soc., 131 , 7392.

  • Uppala, S. M., and Coauthors, 2005: The ERA-40 re-analysis. Quart. J. Roy. Meteor. Soc., 131 , 29613012.

  • Vautard, R., , K. C. Mo, , and M. Ghil, 1990: Statistical significance test for transition matrices of atmospheric Markov chains. J. Atmos. Sci., 47 , 19261931.

    • Search Google Scholar
    • Export Citation
  • von Storch, H., , and F. W. Zwiers, 1999: Statistical Analysis in Climate Research. Cambridge University Press, 484 pp.

  • Zwanzig, R., 2001: Nonequilibrium Statistical Mechanics. Oxford University Press, 222 pp.

APPENDIX

Numerical Diffusion in an Advecting System

The increase with time step of numerical diffusion in Eq. (1) is studied with a system where the PDF is governed by the advection equation
i1520-0469-64-9-2981-ea1
with constant U0. It is a standard procedure to test numerical methods on the basis of the advection equation (e.g., Durran 1998). Appropriate boundary conditions can be chosen so that equilibrium solutions to Eq. (A1) are obtained.
The discretization of Fig. 1 is chosen, and the transition coefficients are calculated by following the advection of probability density. Assuming the initial distribution fJ(t 0) ≡ f0J = 1/Dq for a selected grid interval (J), let = UoDt be the distance covered by the mean flow during a time step, Dt. Thus, the states in grid interval (J − 1)Dq ≤ (J)Dq are moved to (J − 1)Dq + q ≤ (J)Dq + . This new range spans across two grid intervals (I − 1) and (I), where I is the integer part of J + 1 + (/Dq). Thus
i1520-0469-64-9-2981-ea2
and, therefore,
i1520-0469-64-9-2981-ea3
so that Eq. (1) is available with exact transition coefficients.
Given an initial distribution f0J = 1/Dq for a specific value of J, the analytic solution to Eq. (A1) is
i1520-0469-64-9-2981-ea4
Equation (1) gives Eq. (A2) after one time step. By applying again Eq. (A3), further evolution is obtained:
i1520-0469-64-9-2981-eqa1
At = t 0 + n Dt we obtain a binomial distribution with
i1520-0469-64-9-2981-ea5

The analytic solution, Eq. (A4), does not show any spread, whereas the solution of the master equation is spread out over n + 1 grid intervals at time = t 0 + n Dt. Thus, if is kept constant by reducing n and correspondingly increasing Dt, the spread decreases. There is, however, the complicating effect that Eq. (A5) depends also on . Changes of the time step affect . If, in particular, is a multiple of Dq, the situation is singular in that the master equation reproduces the analytic solution Eq. (A4). In that sense, a slight reduction of may lead to a reduction of numerical spread as comes close to a multiple of Dq. We may overcome this complication by looking just at differences (IJ)Dq. If Dt is altered such that this difference remains the same, the spread of Eq. (A5) decreases with increasing Dt. The extension of Eq. (A1) with small diffusive terms would not affect the result that numerical diffusion is reduced by increasing the time step Dt; it would sure complicate the treatment, yet removing this singularity. For the Λ-dimensional case, numerical diffusion can be expected to grow approximately with Λ(Dq2/Dt) [see Egger and Dall’Amico (2007), where the study in this appendix is extended and idealized two-dimensional flow configurations are also studied].

Fig. 1.
Fig. 1.

The probability density function f(tn) at time tn. The transition coefficient Wi2 gives the probability that the variable q leaves grid interval 2 to enter i at the next time step.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 2.
Fig. 2.

Time series of the x component of the Lorenz model with white Gaussian noise; Eqs. (7). The dashed time series is obtained by decreasing the initial x value by only 10−3 (the white noise forcing is identical).

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 3.
Fig. 3.

Observed state density, defined with Eq. (4) and integrated along the y axis, for a time series of length Δt  = 51 200 and sampling interval Dt = 0.020, obtained from Eqs. (7) by extending the integration that lead to the solid time series in Fig. 2. The starting point of the time series is marked by a circled dot. The isoline contour interval is 0.5 × 10−3, and every third isoline is bold. (a) The grid size used to partition the phase space, Dq = 5.00, is too coarse to resolve the “holes” in the attractor wings; these may be seen in (b), where the grid size Dq = 2.50; in (c) the grid size Dq = 1.25.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 4.
Fig. 4.

The ratio Rw defined with Eq. (6) is shown as a function of the time series length, Δt , in nondimensional time units; the time resolution Dt = 0.020. Points mark the results obtained with a grid size Dq = 2.50; circles correspond to Dq = 1.25. The time series are generated by extending the numerical integration of Eqs. (7) which lead to the solid time series in Fig. 2. There is no stationary estimate of the transition coefficients for Dq = 1.25 and Δt  = 1.00.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 5.
Fig. 5.

(a) The position at t  = 0, t  = 0.2, t  = 0.4, and t  = 0.6 of a cloud of 8000 points according to Eqs. (7); the position at t  = 0.8 is shown in (b). (c) The normalized density of ensemble members (points) for t  = 0.8, obtained from (b) after discretizing the phase space with Dq = 2.50 and integrating along the y axis; the isoline contour interval is 10−3.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 6.
Fig. 6.

PDF forecasts for time t  = 0.8 as delivered by EMEs derived from time series of varying lengths; the grid size Dq = 2.50; the time resolution Dt = 0.020. The star marks the sharp initial condition (see also Fig. 5a). The PDF has been integrated along the y axis. The isoline contour interval is 10−3, and every third isoline is bold. The reference density of ensemble members is shown in Fig. 5c.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 7.
Fig. 7.

Skill S for the EME forecasts shown in Fig. 6 (and others), shown as a function of the time series length; the grid size Dq = 2.50; the time resolution Dt = 0.020. The reference density of ensemble members is shown in Fig. 5c.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 8.
Fig. 8.

The convergence coefficient Cw is shown as a function of the length of the longer time series, which is twice as long as the shorter one; the time resolution Dt = 0.020. Points mark the results obtained with a grid size Dq = 2.50; circles correspond to Dq = 1.25.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 9.
Fig. 9.

PDF forecasts for time t  = 0.8 as delivered by EMEs derived from time series of varying time resolutions; the grid size Dq = 2.50; the time series length Δt  = 3200. The star marks the sharp initial condition (see also Fig. 5a). The PDF has been integrated along the y axis. (a) Dt = 0.004; (b) Dt = 0.020 (same as Fig. 6d); (c) Dt = 0.100. The isoline contour interval is 10−3, and every third isoline is bold. The reference density of ensemble members is shown in Fig. 5c.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 10.
Fig. 10.

Influence of the time step, Dt, on an EME. The phase plane is partitioned into squares of unit grid size. The dash-dotted lines indicate parts of a single illustrative trajectory, the arrows show the direction of motion, and bold points indicate sampled observations. The dashed closed curves describe the domain of a cloud of states at two different times, t 0 (initial time) and (a later time). The PDF is initially (t  = t 0) set equal to 1/Dq = 1 in cell (2, 2); PDF values predicted by the EME are written in all cells where the PDF is nonzero (italics are used to ditinguish among successive time steps). (a) The time resolution Dt = (t 0)/3, whereas in (b) Dt = (t 0). The PDF smears faster in (a).

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 11.
Fig. 11.

Correlation functions rxx, rzz, and ryx as observed (solid), and as reproduced by EMEs derived from time series of varying time resolutions: dotted for Dt = 0.100, dash-dotted for Dt = 0.020, and dashed for Dt = 0.004; the grid size Dq = 2.50; the time series length Δt  = 3200.

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

Fig. 12.
Fig. 12.

(left) PDF forecasts for time t  = 0.8 as delivered by an EME derived from a time series of the x and z components of the Lorenz model with stochastic forcing; the grid size Dq = 2.50; the time series length Δt  = 3200; the time resolution Dt = 0.100. The star marks the sharp initial condition. The isoline contour interval is 10−3, and every third isoline is bold. The reference density of ensemble members is shown in Fig. 5c. The corresponding forecast, obtained with a three-dimensional EME is shown in Fig. 9c. (right) The autocorrelation function of the first component, rxx, as observed (solid) and as reproduced by the EME (dotted).

Citation: Journal of the Atmospheric Sciences 64, 9; 10.1175/JAS3992.1

1

With the evolution of computing power, the implementation of four- or five-dimensional EMEs will eventually become feasible, provided that appropriately long time series are available.

2

Nicolis (1990) used the concept of Markov partitions to discretize the third component of the Lorenz (1963) model into two grid intervals. Pasmanter and Timmermann (2002) made use of equipartitions (i.e., each cell contains the same number of observations) to discretize one- and two-dimensional phase planes. Our approach of considering equal sized grid intervals has been used by Egger (2001), and, similarly, Thuburn (2005) partitioned into equal sized cubical cells the phase space spanned by the variables of the Lorenz (1963) model.

3

See also the discussion on stochastic matrices and the Perron–Frobenius theorem in Pasmanter and Timmermann (2002).

4

Crommelin (2004) introduced an approach to statistical significance applicable to equipartitions into, say, up to 10 cells, whenever one is interested in a few meaningful state transitions. Vautard et al. (1990) introduced a significance test based on Monte Carlo simulations while examining a few atmospheric circulation pattern transitions.

5

A second-order Markov chain involves a number of coefficients of order (i1max · i2max · . . . · iΛmax)3, where iλmax is the total number of grid intervals along the λth axis, and Λ is the number of variables. Such a hypothesis test is impractical with the computing resources we have. (In the papers cited in section 1, such hypothesis tests have been carried out by only a few authors in situations where the total number of cells was up to 5.) If the test is computationally feasible, the limited amount of data generally available in atmospheric applications would not allow the rejection of the hypothesis because of the high number of cells we plan to work with. Even if the hypothesis is rejected, the questions remain open on whether a first-order Markov chain description may still bring some insight in the underlying processes and whether a higher-order Markov chain description represents a feasible way to deal with the available data. For these reasons, we introduce another type of test of the quality of the EME.

6

If nonadditive noise terms were present, care should be taken in the choice of the numerical scheme (e.g., Kloeden et al. 1997).

7

Correlation functions were not introduced in section 4b because both the ones directly estimated from a time series and the ones delivered by an EME derived from the same time series improve with increasing time series length. Correlation functions estimated from a time series of length Δt  = 3200 negligibly improve if a substantially longer time series is used. A time series of length Δt  = 3200 occupies densely and uniformly the portion of the phase space where the attractor is located [not shown, compare Fig. 3b with Fig. 3.8 in Dall’Amico (2005)]. Correlation functions are derived from this whole portion.

8

Kaplan and Glass (1995, 308–311) mention how time-lag embedding enables the reconstruction of the geometry of a chaotic system from a time series even if only one of the variables is measured.

Save