• Arnold, L., 1998: Random Dynamical Systems. Springer, 586 pp.

  • Buizza, R., , J. Tribbia, , F. Molteni, , and T. Palmer, 1993: Computation of optimal unstable structures for numerical weather prediction models. Tellus, 45A, 388407.

    • Search Google Scholar
    • Export Citation
  • Cessi, P., , and W. R. Young, 1992: Multiple equilibria in two-dimensional thermohaline circulation. J. Fluid Mech., 241, 291309.

  • Cheung, K. K. W., 2001: Ensemble forecasting of tropical cyclone motion: Comparison between regional bred modes and random perturbations. Meteor. Atmos. Phys., 78, 2335.

    • Search Google Scholar
    • Export Citation
  • Corazza, M., and Coauthors, 2003: Use of the breeding technique to estimate the structure of the analysis errors of the day. Nonlinear Processes Geophys., 10, 233243.

    • Search Google Scholar
    • Export Citation
  • Deremble, B., , F. D’Andrea, , and M. Ghil, 2009: Fixed points, stable manifolds, weather regimes, and their predictability. Chaos, 19, 043109. doi:10.1063/1.3230497.

    • Search Google Scholar
    • Export Citation
  • Evans, E., , N. Bhatti, , J. Kinney, , L. Oann, , M. Peña, , S. Yang, , and E. Kalnay, 2004: RISE undergraduates find that regime changes in Lorenz’s model are predictable. Bull. Amer. Meteor. Soc., 85, 520524.

    • Search Google Scholar
    • Export Citation
  • Eyink, G., 2005: Statistical hydrodynamics of the thermohaline circulation in a two dimensional model. Tellus, 57A, 100115.

  • Foias, C., , G. R. Sell, , and R. Temam, 1988: Inertial manifolds for nonlinear evolutionary equations. J. Differ. Equations, 73, 309353.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., , and A. E. Raftery, 2005: Atmospheric science—Weather forecasting with ensemble methods. Science, 310, 248249.

  • Hallerberg, S., , D. Pazo, , J. M. Lopez, , and M. A. Rodriguez, 2010: Logarithmic bred vectors in spatiotemporal chaos: Structure and growth. Phys. Rev. E, 81, 066204, doi:10.1103/PhysRevE.81.066204.

    • Search Google Scholar
    • Export Citation
  • Hansen, J. A., , and L. A. Smith, 2000: The role of operational constraints in selecting supplementary observations. J. Atmos. Sci., 57, 28592871.

    • Search Google Scholar
    • Export Citation
  • Johnson, R. A., , K. J. Palmer, , and G. R. Sell, 1987: Ergodic properties of linear dynamical systems. SIAM J. Math. Anal., 18, 133.

  • Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, 341 pp.

  • LaSalle, J. P., , and S. Lefschetz, 1961: Stability by Lyapunov’s Direct Method with Applications. Academic Press, 134 pp.

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130141.

  • Lyapunov A. M., 1992: The general problem of the stability of motion (in Russian). Int. J. Control, 55, 521–790.

  • Mandelbrot, B. B., 1977: Fractals: Form, Chance, and Dimension (in French). W. H. Freeman and Co., 365 pp.

  • Mu, M., , and Z. N. Jiang, 2008: A new approach to the generation of initial perturbations for ensemble prediction: Conditional nonlinear optimal perturbation. Chin. Sci. Bull., 53, 20622068.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., , R. Gelaro, , J. Barkmeijer, , and R. Buizza, 1998: Singular vectors, metrics and adaptive observations. J. Atmos. Sci., 55, 633653.

    • Search Google Scholar
    • Export Citation
  • Pliss, V. A., , and G. R. Sell, 1999: Robustness of exponential dichotomies in infinite-dimensional dynamical systems. J. Dyn. Differ. Equations, 11, 471513.

    • Search Google Scholar
    • Export Citation
  • Pliss, V. A., , and G. R. Sell, 2001: Perturbations of normally hyperbolic manifolds with applications to the Navier–Stokes equations. J. Differ. Equations, 169, 396492.

    • Search Google Scholar
    • Export Citation
  • Primo, C., , M. A. Rodriguez, , and J. M. Gutierrez, 2008: Logarithmic bred vectors. A new ensemble method with adjustable spread and calibration time. J. Geophys. Res., 113, D05116, doi:10.1029/2007JD008998.

    • Search Google Scholar
    • Export Citation
  • Riviére, O., , G. Lapeyre, , and O. Talagrand, 2008: Nonlinear generalization of singular vectors: Behavior in a baroclinic unstable flow. J. Atmos. Sci., 65, 18961911.

    • Search Google Scholar
    • Export Citation
  • Sacker, R. J., , and G. R. Sell, 1976: Existence of dichotomies and invariant splittings for linear differential systems. II. J. Differ. Equations, 22, 478496.

    • Search Google Scholar
    • Export Citation
  • Sacker, R. J., , and G. R. Sell, 1977: Lifting properties in skew-product flows with applications to differential equations. Mem. Amer. Math. Soc., 11 (190), 1–67.

    • Search Google Scholar
    • Export Citation
  • Sacker, R. J., , and G. R. Sell, 1978: A spectral theory for linear differential systems. J. Differ. Equations, 27, 320358.

  • Sacker, R. J., , and G. R. Sell, 1980: The spectrum of an invariant submanifold. J. Differ. Equations, 38, 135160.

  • Sell, G. R., , and Y. You, 2002: Dynamics of Evolutionary Equations. Springer, 670 pp.

  • Toth, Z., , and E. Kalnay, 1993: Ensemble forecasting at NCEP: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 23172330.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., , and E. Kalnay, 1997: Ensemble forecasting at NCEP: The breeding method. Mon. Wea. Rev., 125, 32973318.

  • Wang, X. G., , and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 11401158.

    • Search Google Scholar
    • Export Citation
  • Wei, M. Z., , and Z. Toth, 2003: A new measure of ensemble performance: Perturbation versus error correlation analysis (PECA). Mon. Wea. Rev., 131, 15491565.

    • Search Google Scholar
    • Export Citation
  • Wolfe, C. L., , and R. M. Samelson, 2007: An efficient method for recovering Lyapunov vectors from singular vectors. Tellus, 59A, 355366.

    • Search Google Scholar
    • Export Citation
  • View in gallery

    EBVs, for times 24–30, taken at 0.1 time intervals, for Lorenz63 [viewed from three different angles, (a)–(c)]. Parameters and conditions are those used to generate Table 1.

  • View in gallery

    Close-up views of Fig. 1a: (a) zoomed in 8 times; (b) zoomed in 32 times; (c) zoomed in 60 times. Note the preservation of structure even as we zoom in.

  • View in gallery

    (top to bottom) The three components of BV and of the finite-time Lyapunov vectors, as a function of time for the Lorenz63 system. (For parameters and initial conditions, see text.) The vectors are nearly coincidental over the whole time span. The perturbation size is 1 in all of the components. For our choice of initial conditions the vectors exhibit more temporal regularity at earlier times. This transient behavior disappears at t = 13, approximately.

  • View in gallery

    Plot depicting the evolution of the 2-norm of an ensemble of 98 EBV(t). Parameters and initial conditions are the same as in Fig. 3. Initially, in the interval [0, 13] we note a very fast decay of some of the ensemble members, leading to a sorting in size, beyond that time. As described in the text, this outcome is one of the most distinguishing features of the EBVs, when compared to an ensemble of individual BV outcomes.

  • View in gallery

    (a) Forcing function fxx (44) used in the CY92 model; the numerical solution to CY92 is shown in (b). See Fig. 6 for the BV algorithm and finite-time Lyapunov vector algorithm outcomes.

  • View in gallery

    Comparison of the finite-time Lyapunov vectors and BVs. Effect of amplitude of perturbation on the CY92 model, with initial conditions Y0 = cos(x). The initial perturbation was ϵsin(x). Comparison of the finite-time Lyapunov vector and the BV outcomes at t = 70: ϵ = (a) 0.25 and (b) 0.025.

  • View in gallery

    (a) The jth simulation corresponding to a sine wave perturbation at t = 70 after the superposition of six BV simulations per (46). (b) Cross section of the EBVs at t = 70 run with an ensemble of perturbations [see (46)]. We note that unlike the BV case in (a), the EBV has settled into a structure that clearly reflects the dominant EBV ensemble member. (c) Cross section of the same EBVs, at an earlier time, t = 5. Even for short times, the EBV already shows an amplitude-ordering of its vectors. (d) Superposition of all EBVs, as a function of time and space.

  • View in gallery

    Wavenumber dependence of BVs and finite-time Lyapunov vectors: CY92 model, cosine initial condition, and nonsymmetric forcing (see Fig. 5). Comparison of the finite-time Lyapunov vectors and the BVs for sine wave perturbations of size ϵj = 0.25. The perturbations are the individual sine waves in (46): j = (a) 1, (b) 2, (c) 3, (d) 4, (e) 5, and (f) 6.

  • View in gallery

    The BVs at t = 19.025, corresponding to ϵj = ϵ, for all j. ϵ = (a) 0.6, (b) 0.8, and (c) 1.2. The outcomes are very sensitive to nonlinearity. There is a resulting ambiguity in structure of the perturbation field.

  • View in gallery

    The EBV with ϵ = (a) 0.6, (b) 0.8, and (c) 1.2. Compare outcomes to Fig. 9. The vectors are shown in their original scales.

  • View in gallery

    The EBV outcomes shown in Fig. 10, with each vector rescaled to an l2 norm of 1 to aid in visual comparison. Perturbation amplitudes ϵ = (a) 0.6, (b) 0.8, and (c) 1.2. Note that as the perturbation amplitude increases, the resemblance between the BV and the EBV outcomes is lost, except for one of the vectors.

  • View in gallery

    The (a) L2 norm, (b) L1 norm, and (c) L norm of a sample BV as a function of time, for the system in (58). The initial conditions are the same in all cases, and the amplitude of the perturbations was 0.15.

  • View in gallery

    (a) A sample BV, as a function of time. The calculation diverges and eventually fails (not shown); (b) 2-norm+of the numerically approximated M(Yn + , δt) − M(Yn, δt) as a function of time, for the linear system . Here, is an upper-triangular Jordan-block matrix of dimension 5, where has a single eigenvalue −1, which is repeated on the main diagonal, while the superdiagonal has entries 1. (c) 2-norm of the numerically approximated M(Yn + , δt) − M(Yn, δt), well-conditioned case.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 52 52 2
PDF Downloads 44 44 1

Ensemble Dynamics and Bred Vectors

View More View Less
  • 1 Institute for Mathematics and its Applications, University of Minnesota, Minneapolis, Minnesota
  • | 2 Department of Mathematics, The Pennsylvania State University, University Park, Pennsylvania
  • | 3 Department of Mathematics and Department of Physics, The University of Arizona, Tucson, Arizona
  • | 4 School of Mathematics, University of Minnesota, Minneapolis, Minnesota
© Get Permissions
Full access

Abstract

The new concept of an ensemble bred vector (EBV) algorithm is introduced to assess the sensitivity of model outputs to changes in initial conditions for weather forecasting. The new algorithm is based on collective dynamics in essential ways. As such, it keeps important geometric features that are lost in the earlier bred vector (BV) algorithm. By construction, the EBV algorithm produces one or more dominant vectors and is less prone to spurious results than the BV algorithm. It retains the attractive features of the BV algorithm with regard to being able to handle legacy codes, with minimal additional coding.

The performance of the EBV algorithm is investigated by comparing it to the BV algorithm as well as the finite-time Lyapunov vectors. With the help of a continuous-time adaptation of these algorithms, a theoretical justification is given to the observed fact that the vectors produced by the BV, EBV algorithms, and the finite-time Lyapunov vectors are similar for small amplitudes. The continuum theory establishes the relationship between the two algorithms and general directional derivatives.

Numerical comparisons of BV and EBV for the three-equation Lorenz model and for a forced, dissipative partial differential equation of Cahn–Hilliard type that arises in modeling the thermohaline circulation demonstrate that the EBV yields a size-ordered description of the perturbation field and is more robust than the BV in the higher nonlinear regime. The EBV yields insight into the fractal structure of the Lorenz attractor and of the inertial manifold for the Cahn–Hilliard-type partial differential equation.

Current affiliation: Department of Mathematics, Indiana University, Bloomington, Indiana.

Corresponding author address: Juan M. Restrepo, Department of Mathematics, The University of Arizona, 617 N. Santa Rita Ave., P.O. Box 210089, Tucson, AZ 85721. E-mail: restrepo@math.arizona.edu

Abstract

The new concept of an ensemble bred vector (EBV) algorithm is introduced to assess the sensitivity of model outputs to changes in initial conditions for weather forecasting. The new algorithm is based on collective dynamics in essential ways. As such, it keeps important geometric features that are lost in the earlier bred vector (BV) algorithm. By construction, the EBV algorithm produces one or more dominant vectors and is less prone to spurious results than the BV algorithm. It retains the attractive features of the BV algorithm with regard to being able to handle legacy codes, with minimal additional coding.

The performance of the EBV algorithm is investigated by comparing it to the BV algorithm as well as the finite-time Lyapunov vectors. With the help of a continuous-time adaptation of these algorithms, a theoretical justification is given to the observed fact that the vectors produced by the BV, EBV algorithms, and the finite-time Lyapunov vectors are similar for small amplitudes. The continuum theory establishes the relationship between the two algorithms and general directional derivatives.

Numerical comparisons of BV and EBV for the three-equation Lorenz model and for a forced, dissipative partial differential equation of Cahn–Hilliard type that arises in modeling the thermohaline circulation demonstrate that the EBV yields a size-ordered description of the perturbation field and is more robust than the BV in the higher nonlinear regime. The EBV yields insight into the fractal structure of the Lorenz attractor and of the inertial manifold for the Cahn–Hilliard-type partial differential equation.

Current affiliation: Department of Mathematics, Indiana University, Bloomington, Indiana.

Corresponding author address: Juan M. Restrepo, Department of Mathematics, The University of Arizona, 617 N. Santa Rita Ave., P.O. Box 210089, Tucson, AZ 85721. E-mail: restrepo@math.arizona.edu

1. Introduction

Central to weather prediction is the analysis of the sensitivity of a physical or computer-coded model to initial conditions. Model sensitivity to parameters is also important in model intercomparison. Such sensitivity is studied in order to obtain a better understanding of the role played by these parameters in model outcomes.

Sensitivity and predictability are often intertwined in the context of weather prediction and have been the subject of extensive research (see Buizza et al. 1993 and references therein). These are not exclusively weather-related issues, and thus geophysical fluid dynamics will often mine other physical, computational, and mathematical disciplines for ideas with which to assess dynamic sensitivity. Practical sensitivity methodologies must contend with the evolution and dynamics of highly coupled, complex, high-dimensional systems, riddled with subscale parameterizations and empirical relations, which are the norm in large-scale climate and meteorology models.

A tool used in the study of sensitivity analysis is the bred vector (BV) algorithm. It is proposed for use in forward sensitivity of weather and climate models. While in section 2a we present a brief survey of some of the applications of this algorithm in various sensitivity analyses, in this article we focus on the issue of the maximal growth of errors due to small changes in the initial conditions.

The concept of the BV algorithm we use is based on the theory first introduced in Toth and Kalnay (1993). In addition to the BV notion, we present here a new variant, which we call the ensemble bred vector (EBV) algorithm. The definitions of both the BV and EBV algorithms are presented in section 2.

In the BV algorithm, one follows an initial condition of the time-discrete nonlinear system, along with cloud, which describes a family of nearby solutions. (Since this algorithm is used to sample the error space, an ensemble of initial perturbations is bred simultaneously.) The perturbations at the initial time are fixed with a common small amplitude ϵ. After each cycle, the outcome of the perturbations is rescaled to the same amplitude ϵ. For the BV algorithm, the rescaling of each perturbation is independent of the others, and there is no mechanism to use the rescaling to compare the dynamics of nearby perturbations.

The new variation that we propose here, the EBV algorithm, differs from the BV algorithm in the rescaling rule. In particular, for the EBV those perturbations that are not the same size as the largest perturbation play a reduced role after the rescaling. Thus the rescaling used in the EBV algorithm serves us better in separating various levels of the dominant dynamics. In short, the EBV algorithm offers better insight into the relative behavior of nearby trajectories. Therefore, even when the initial perturbations of the two algorithms are based on the same cloud, the EBV algorithm is linked to the ensemble dynamics of the underlying nonlinear model, hence its name. We make use of both the BV and the EBV algorithms. In fact, one of our major goals is to present an in-depth comparison of the two algorithms, as they are used, or can be used, in describing the underlying dynamics of the model.

We show that in some metrics, the BV and EBV algorithms are comparable, with the EBV being more accurate and faster (see, e.g., Table 1 and Figs. 6, 7, snd 8). For other issues, especially those involving the longtime dynamics within the global attractor, the BV algorithm has a shortcoming, which limits its use (see section 4). The EBV, instead, leads to useful and interesting insight into the dynamics of the model, as is shown in the Figs. 1, 2, and 4.

Table 1.

Maximum and minimum values of D(T, δt) with T = 2, for two values of δt. See (42).

Table 1.
Fig. 1.
Fig. 1.

EBVs, for times 24–30, taken at 0.1 time intervals, for Lorenz63 [viewed from three different angles, (a)–(c)]. Parameters and conditions are those used to generate Table 1.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

Fig. 2.
Fig. 2.

Close-up views of Fig. 1a: (a) zoomed in 8 times; (b) zoomed in 32 times; (c) zoomed in 60 times. Note the preservation of structure even as we zoom in.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

This brings up the question: For a given model of sensitivity with respect to initial conditions, how does one determine the direction vector that results in the maximal increase in the error due to a small perturbation in the given direction initially in the long-time limit? This is where the Lyapunov vectors enter the scene. What one needs is a red vector, which is the Lyapunov vector x0 with ‖x0‖ = 1 and with the property that the corresponding strong Lyapunov exponent λ1 is the maximal Lyapunov exponent for the model. (See section 2e for the definition and more details. It should be noted that the Lyapunov exponents for the model require integration over 0 ≤ t < ∞, or over the real line .)

While the existence of a red vector is a matter of concern, we do present a rigorous proof of this property—under a reasonable assumption—in section 2e.

We use either the EBV or the BV algorithm to approximate the solutions of the tangent linear equation. In section 3 we show that either algorithm is a good approximation. For these algorithms one can integrate only over a finite interval 0 ≤ tT, where 0 < T < ∞. However, it is only under exceptional circumstances, for example, time-periodic or autonomous problems, that a finite-time integration will approximate well an infinite time average. For any hope for success in using a finite-time integration, we require that the model problem satisfy two properties:

  1. There is an attractor for the solutions.
  2. The initial condition is chosen to be on, or near, .

What can be expected with such a finite-time approximation, when the initial conditions are near the attractor ? In terms of the calculated time, one expects first to be in a transient state. Then, after a while, it is hoped that some meaningful information about the long-time dynamics of the model is gained. We include in this manuscript several studies of such approximations.

It is very important to note that, in order to better understand the maximal growth of errors due to small changes in the initial conditions, the dynamical information contained in the attractor of the model needs to be exploited. In particular, one needs to complete two steps:

  1. Locate the red vector x0, a unit vector, and the associated Lyapunov exponent λ1. This then determines the red vector solution R(x0, t), with x0 being the initial condition, for −∞ < t < ∞, for the model equation.
  2. Find a good approximation of the time evolution R(x0, t) on an appropriate finite-time interval, 0 ≤ tT < ∞.

Once the red vector x0 is known, then ϵx0 is the initial condition for the bred vector sequence (see section 2b). Due to the results derived here in section 3, either EBV(t) or BV(t) is a good finite-time approximation of R(x0, t), for 0 ≤ tT < ∞. (However, there are several advantages to the EBV, as we elucidate throughout the paper.) This takes care of step 2. Consequently, the problem boils down to a search for the red vector, which is addressed below in section 2e. Fortunately for us, there is a wealth of related mathematical information in Johnson et al. (1987, hereafter referred to as JPS87). As we show, by using JPS87, we are able to describe the mathematical process of finding the red vector. By using this citation, with the theory of the EBV(t) algorithm, this leads to a good solution for our sensitivity problem.

It was observed by Toth and Kalnay (1993, 1997) in several experimental runs that BVs resemble the leading finite-time Lyapunov vectors. To make a more quantitative comparison between Lyapunov vectors and BVs, in section 3 we study the continuum limits (as the basic time-step size for rescaling, and ϵ, go to 0) of the BV and the EBV algorithms and show direct connections between these limits and specific solutions of the continuous-time tangent linear equations. Section 3 also contains a further discussion of some of the desirable and interesting features of the new EBV. For instance, there is a natural ordering of the ensemble members of an EBV, and as we show, it is possible to observe perturbations with smaller sizes than the dominant one, but with very strong growth (see the spearlike behavior in Fig. 4).

We consider two models to exemplify the features of the new EBV. The first is the familiar Lorenz63 model introduced by Lorenz (1963). It has a well-known global attractor. The second is a nonlinear forced and dissipative partial differential equation of the Cahn–Hilliard type. This equation is a variation of a model proposed by Cessi and Young (1992) of the oceanic thermohaline circulation. We denote the equation associated with this model as CY92 in this study. (In fact, we impose periodic boundary conditions instead of the more physical zero-flux, zero-stress conditions at the poles.)

It turns out CY92 is a good example of the typical climate-related model dynamics. However, CY92 is special, since it has an inertial manifold. Consequently, the longtime dynamics of this partial differential equation is completely contained in the attractor of a finite-dimensional ordinary differential equation (ODE). In applications, the BV algorithm and its variants are in fact discrete-time algorithms based on finite-dimensional approximations of weather models, obtained either by mode projection, as in the Lorenz (1963) system, or by spatial discretization of partial differential equations (PDEs), as in the CY92 model, described below. As we show, the EBV yields insights into the structure of the attractor of the Lorenz63 model and of the inertial manifold to CY92.

Our numerical examples also highlight that the BV algorithm is sensitive to the amplitude and frequency content of the initial perturbation. In contrast, the outcome of the EBV algorithm shows a clear hierarchy among its members, and the first few members already generate an unambiguous characterization of the perturbation field at both large and small amplitudes. In the nonlinear regime, the EBV will be shown in section 4 to be less likely to produce spurious results than the BV. In section 5 we address implementation issues of the EBV.

2. BV and EBV algorithms; finite-time Lyapunov vectors

In this section we present the definitions and methodology for computing the two bred vector algorithms, the BV and the EBV. We also review the basic theory of Lyapunov vectors, Lyapunov exponents, and their finite-time counterparts. We compare these different tools below. However, first we include here a brief survey of some of the applications and theoretical issues that have been noted in the implementation of the BV algorithm.

a. Brief survey of applications of the BV algorithm

For the BV algorithm, we use the one originally proposed by Toth and Kalnay (1993). BV is purely algorithmic. It is “equation-free,” and thus with additional minimal computer coding it can handle legacy code representing even extremely complex models. Most alternatives for obtaining estimates of forward sensitivity will involve nontrivial additional coding. For example, in order to obtain the finite-time Lyapunov vectors and exponents, one needs to derive and make use of the tangent linear model. (From the beginning, it was realized that there existed some close connections between the BV algorithm and the tangent linear equations. As we show in section 3, there is a rigorous mathematical foundation for these connections.) Singular value decomposition methods, which can offer complementary information to Lyapunov-vector-inspired methods, also require a tangent linear model. Deremble et al. (2009) use this approach to study regime predictability in some reduced weather models. Other examples are Buizza et al. (1993) and Palmer et al. (1998). In another direction, Wolfe and Samelson (2007) propose the use of the multiplicative ergodic theorem (MET) and the finite-time singular vectors to approximate the Lyapunov vectors.

The BV algorithm is a finite-time, forward sensitivity methodology that, in addition to being useful in characterizing model sensitivity to initial conditions, has been proposed as a means to produce a reduced-rank representation of the background error in data assimilation and forecast error-covariance approximations (see, e.g., Corazza et al. (2003).

Several articles in the literature have addressed applications of the BV algorithm in weather modeling. See, in particular, Toth and Kalnay (1993, 1997) and Kalnay (2003), and references therein. For a comparison of the BV algorithm and other methods, such as Monte Carlo perturbed observations, we refer, for example, to Cheung (2001), Gneiting and Raftery (2005), Hansen and Smith (2000), and Wei and Toth (2003). For an application of the BV algorithm to ensemble Kalman filters, see Wang and Bishop (2003). Primo et al. (2008) and Hallerberg et al. (2010) have treated applications based on variations in the algorithm, where, for example, the rescaling is done by using a geometric mean.

As already discussed, several alternatives to the BV algorithm have been proposed and employed in the literature. BVs have been viewed as nonlinear analogs of finite-time Lyapunov vectors. Similarly, nonlinear analogs of singular vectors have been proposed, for instance, conditional nonlinear optimal perturbations proposed by Mu and Jiang (2008) and nonlinear singular vectors proposed by Riviére et al. (2008), although they entail a computationally expensive optimization.

Both the BV and EBV algorithms arise in the time-discretization of a continuous-time dynamical system. We consider the following initial value problem:
e1
where t represents time and G = G(y) is a map that has at least a bounded gradient. Since our main applications involve autonomous differential equations, we assume that G does not explicitly depend on time. The basic theory we present here has a routine extension to nonautonomous problems.

The solution vector y = y(t) can live in a finite- or infinite-dimensional normed linear space. In the former case, (1) is an (autonomous) system of ordinary differential equations, while in the latter case, (1) is an (autonomous) system of partial differential equations, modeling a time-dependent, spatially extended system. For systems of partial differential equations, we assume that either periodic boundary conditions or nonflux boundary conditions are prescribed. See, for example, Sell and You (2002). The use of other boundary conditions may lead to a related theory, but we do not address this issue here.

If the system contains evolution partial differential equations, the system, along with the boundary conditions, are discretized in space or projected onto a finite-dimensional space compatible with the boundary conditions. Consequently, we usually assume that (1) is a system of ordinary differential equations of dimension K, which may be large. Since most large-scale weather and climate circulation models presently use explicit-in-time integrators, we focus on numerical models of this type.

b. Bred vector algorithms: BV and EBV

We let y = y(t) denote a given (continuous time) solution of (1). We then turn to an approximate solution Yn = Y(tn), which is defined on the time grid: tn, for n = 0, 1, 2, … , where tn+1 = tn + δtn, for n = 0, 1, 2, … . We set t0 = 0. We assume that δtn = δt is positive and small, and that it does not depend on n, for n ≥ 0.

The initial value problem, which is approximated using an explicit numerical integration scheme, leads to consideration of the difference equation
e2
where Y(tn) is a solution of the discrete problem (2), and it may be viewed as an approximation of the continuous-time solution y(t), at t = tn. Likewise, the initial condition Y0 is an approximation of the initial condition y0. The points Yn are in the K-dimensional Euclidean space , and M = M(Y, δt) is the discrete-time solution operator on generated by the ordinary differential equation (1). As noted above, we assume that the discrete-time problem (2) has an attractor and that Y0 is on, or near, .
At this point it is convenient to introduce the related concepts of a cloud (at t = 0) and a family of initial perturbations. A cloud (at t = 0) is a family of tangent vectors (ι) at Y0 in that depends on ι, where and is a finite index set. The main requirement we impose is that
eq1
where ϵ is small, positive, and fixed. The collection of all terms in , for is called a family of initial perturbations. Notice that this collection lies on a sphere of radius ϵ in with center at (Y0, 0). Both the cloud and the perturbations evolve in time, via the BV or the EBV algorithms, which are defined below. As we now note, the definitions of the two algorithms differ only in the rescaling rule.

We begin by recalling the BV algorithm as given by Toth and Kalnay (1993, 1997). For n ≥ 0, we assume that the base point Yn and the perturbation vector δYn(ι) are known. For the (n + 1)st step we use the following:

  1. Yn+1 denotes the (n + 1) base point, and it is determined by (2);
  2. (ι), the (n + 1) perturbation vector, is given by
    e3
    e4
    where Rn+1 is a rescaling rule.

The time evolution of the cloud is (ι), where and n ≥ 0.

One rescaling rule, proposed by Toth and Kalnay (1993, 1997), consists of rescaling the nth perturbation vector to the previous one by
e5
Equivalently,
e6
so that Rn+1 = Rn+1(Y0) depends on the initial base point Y0, as well as the initial perturbation vector (ι). An alternate rescaling rule consists in rescaling periodically, at tmk, where k is an integer k ≥ 2, and m = 1, 2, … . In this case, rule (6) is used when n = mk, and
e7
We do not use this alternate rule in this paper. [For a discussion of rescaling time and regime predictability in some reduced models, see Deremble et al. (2009, and references therein)].
For the ensemble bred vector algorithm, instead of using Rn+1 as in (3) and (4), we use a uniform scaling , which is the same for all . In particular, we replace (3) and (4) with
e8
for all where
e9

Similarly, when (8) and (9) hold, we use (ι), for and n ≥ 0, to denote the time evolution of the cloud. [Alternatively, a periodic rescaling rule utilizing (7) above can be employed.] The time step used to compute the base trajectory and the time intervals between normalizations need not be the same. This is the case for both the BV algorithm as well as EBV.

The crucial difference between the BV and EBV algorithms is that, even when the BV is run concurrently over an ensemble of initial data, the outcome of the algorithm for each given datum does not depend on the other members of the ensemble. In contrast, the evolution of the ensemble members is interdependent in the EBV. Nevertheless, by construction, the EBV should exhibit similar behavior to the BV at small amplitudes. Indeed, one of the design principles for the EBV algorithm is to reinforce this aspect by providing it with a built-in acceleration mechanism.

Both the BV algorithm and EBV outcomes, on the other hand, depend on the choice of vector norm used to define the rescaling rule in either (6) or (9). While all norms are equivalent in the (finite) K-dimensional space where we seek solutions, in practice the constants appearing in the equivalence between different finite-dimensional norms generally strongly depend on the dimension K and eventually blow up as K becomes infinite. Hence, the choice of norm used can have an impact on the implementability and performance of these algorithms. In Riviére et al. (2008), it was suggested that the outcome of the BV algorithm may depend strongly on the choice of norm. This is actually a consequence of the nonnegligible nonlinear effects in the system. In section 5, we elaborate further on the issue of norm dependence.

In addition to the “renormalization time step” δt, there is another time step, the “integration time step,”, we denote by Δt. For example, the new time step is encountered when moving from the continuous-time problem (1) to the discrete-time problem (2). For the most part, we treat Δt and δt as being equal in the calculations described in this article. However, we always require that Δtδt.

Separation of scales

In nonlinear systems, it is possible that a perturbation will not grow very rapidly in the zone of perturbations with size near ϵ, but instead a scalar multiple of this perturbation can be dominant among the smaller perturbations and might grow quite fast there. It is also possible that the dominant vectors in different amplitude zones will be close in structure but very different in growth characteristics. These theoretical considerations are realized in the Lorenz63 system quite strikingly. See Fig. 4 and its discussion, especially the recurring patterns of “spears” therein. This can be seen very easily from the following relation:
eq2
where is defined similarly to (9), V(t) denotes an EBV(t) member, and denotes the same vector right before the rescaling. In a nonlinear system, this ratio can be larger than 1, for some range of perturbation sizes smaller than ϵ. This is the main mechanism that creates the vector zones of different magnitude with zonal growth characteristics, or a separation of scales.

Another case in point is presented in section 4 for the CY92 model. Even at late times and small amplitudes, we can see perturbations surviving the rescaling strategy, and they resemble what one would get from the usual BV algorithm at those sizes (and the finite-time Lyapunov vectors) very closely, whereas the dominating perturbation of size ϵ resembles a particular BV of size ϵ. This example actually illustrates more. There are members of the ensemble in small magnitudes that are slow in aligning with the dominant directions.

c. Lyapunov vectors

In section 4, we apply the BV and EBV algorithms to two models consisting of systems of (nonlinear) autonomous ODEs of the form (1). (The second model arises from the discretization of a PDE.) In each model, there is a compact, global attractor , which is a subset of , and is invariant for the time evolution of the system. [We refer the reader to section 2 of Sell and You (2002) for more information on attractors and global attractors.] We will let θ denote a typical point in the attractor , and we will let θ · t = y(t) denote the unique solution of (1) that satisfies y(0) = θ. Since is invariant, , for all t.

To study the sensitivity with respect to initial conditions on the attractor, the tangent linear model is used, which is defined as
e10a
e10b
where is the Jacobian matrix of G. Hence, is the linearization of (1) along the solution θ · t. We observe that, even if G does not explicitly depend on time, (10a) is generally nonautonomous, since θ · t changes with time.
We let U(θ, t) denote the solution operator of (10a), which takes the initial data to the solution at time t, so that U(θ, t)x0 is the solution of the initial value problem for (10). Such an operator is well defined by the uniqueness of solutions to the problem (10). Uniqueness of solutions also readily implies the cocycle identity:
e11
and all τ, t.
Next we consider a family of mappings Π = Π(t), which are defined for t by the relation
e12
We note that Π(t) maps into itself, for each t; it is jointly continuous in (t, θ, x0); it satisfies Π(0)(θ, x0) = (θ, x0), i.e., Π(0) = I, the identity operator]; as well as the evolution property:
e13

By using the discrete-time dynamics, where t and τ are restricted to satisfy t = n × δt and τ = m × δt, the notation and the theory of dynamical systems extends readily to the discrete-time problems of interest herein. Note that the θ component of Π does not depend on the x0 component. Thus Π is called a skew product flow. Since Π is linear in x0, it is sometimes called a linear skew product flow. In summary, the tangent linear equation over the attractor generates a linear skew product flow. [For more information on the theory of skew product flows in the context of nonautonomous dynamics, see the multiple works of Sacker and Sell (e.g., Sacker and Sell 1977, 1978, 1980).] The dynamics of Π are crucial for understanding the sensitivity and predictability of the underlying model.

In his opus magnum, which was published in Ukraine in 1892, Lyapunov presented his theory of stability for finite-dimensional ordinary differential equations. This work includes his study of the nonautonomous linear problem (10) [see Lyapunov (1992); yes, 100 years later]. One of Lyapunov’s goals was to develop an analog of the well-known eigenvalue–eigenvector theory, for the solutions of the autonomous problem, to the study of solutions of general nonautonomous (10).

The approach developed by Lyapunov begins with the four Lyapunov relations of exponential growth:
eq3
where x0 ≠ 0. Lyapunov was interested, as are we, in the case where these four limits are equal, and
e14
where x0 ≠ 0. The linearity of U(θ, t)x0 implies that sλ(θ, x0) = λ(θ, sx0), for s ≠ 0. Hence it can be assumed, as we do, that x0 is a unit vector, that is, ‖x0‖ = 1. When (14) holds, then λ(θ, x0) is a strong Lyapunov exponent, and the unit vector x0 is an (associate) Lyapunov vector. The Lyapunov spectrum, LYΣ, is the collection of all such λ(θ, x0), with and ‖x0‖ = 1. For example, if is an autonomous matrix, then the Lyapunov spectrum consists of all real numbers λ that satisfy λ = Reν, where ν is an eigenvalue of .
A Lyapunov vector is not an isolated vector; rather it spawns a line of Lyapunov vectors (through the origin) in . That is to say, a Lyapunov vector is a point in , the (K − 1)-dimensional projective space. For a given vector υ ≠ 0 in , we use [υ] ∈ to denote the unique line in that contains υ. (Note that [υ] = [−υ].) Conversely, when a line [u] in is mapped to a vector u, we require that the pre-image u lie on the line [u] and that ‖u‖ = 1. It should be noted that is a metric space. The projective metric dpr(u1, u2) is defined for nonzero vectors u1 and u2 in by
e15
where s1 and s2 are real numbers that satisfy ||s1u1|| = ||s2u2|| = 1. This metric is used for measuring the distance between the lines [u1] and [u2] in .
Since the solution operator U(θ, t) of the linear problem (10) maps lines in onto lines, this operator can be used to define a related projective flow Σ(θ, t) on by means of the relation
eq4
Using this, an equivalent flow [Π] on is obtained, where
e16
[compare with (12)]. Additional information about the dynamics on the projective flow [Π(t)] is obtained by using the Lyapunov vectors, as is noted below.

d. The finite-time Lyapunov vectors

Next we turn our attention to the question of finding good finite-time approximations of these Lyapunov vectors.

We begin by constructing a piecewise autonomous approximation of the tangent linear equation for (1). To this end, we replace (10) by
e17a
e17b
e17c
where , and y(tn) is the value of an exact solution of (1) at t = tn. The solution of this system at the grid points tn is, explicitly and recursively, given by
e18
Consequently, YN, the solution at time T = Nδt, is given by
e19
where
eq5
We now define the approximation of the Lyapunov vector associated with the largest Lyapunov exponent λ1, at the time t = Tthe finite-time Lyapunov vector, as the direction of steepest ascent for the matrix (T). For example, if an explicit Euler scheme had been used, then (T) would be replaced by its Eulerian approximation W(T), where
eq6
The finite-time Lyapunov vector is the singular vector corresponding to the largest singular value of (T).

In practice (see, e.g., section 4b for the case of the CY92 model), the finite-time Lyapunov vector is computed by directly solving a discrete approximation of the LTM and rescaling the output (the rescaling can be done at arbitrary intervals of time, since the problem is linear).

e. The search for the red vector

One of the main contributions found in JPS87 is an in-depth study of the interactions between two major theories of the longtime dynamics of nonautonomous, linear differential systems:

  • exponential dichotomies (and continuous foliations) and
  • the MET and ergodic measures.

We view JPS87 as a toolkit to be used in the analysis of the dynamics of related linear systems: It is this united theory, as we now show, that forms the mathematical foundations of the theory and applications of bred vectors. We begin with the first aspect: exponential dichotomies and continuous foliations.

Consider the family of shifted semiflows,
eq7
and the associate skew-product flows,
eq8
Let be a compact invariant set in . As noted in Sacker and Sell (1978), the skew-product flow Πλ is said to have an exponential dichotomy over , provided that there exist projectors Pλ and Qλ and constants K0 ≥ 1 and α > 0, such that Pλ(θ) + Qλ(θ) = I, and such that, for all θ and u, one has
e20
When there is an exponential dichotomy and (20) is valid, then the stable and unstable linear spaces, λ and λ—which are respectively the ranges of the projectors Qλ and Pλ—satisfy important dynamical properties that describe the exponential growth rate of selected solutions. For example, the dichotomic inequality above is equivalent to
e21
Let SSΣ denote the Sacker–Sell (SS) spectrum for Π, which is defined as the collection of all λ such that Πλ does not have an exponential dichotomy over .
The spectral theorem in Sacker and Sell (1978) describes the continuous foliation and other properties of the flow Π over . More precisely, there is an integer , where 1 ≤ K, such that SSΣ is the union of closed, bounded intervals; that is,
eq10
Also, there is a continuous foliation
eq11
where is a linearly independent, continuous family of subspaces of with , for θ. As is shown below, the rightmost interval [a1, b1] plays a special role in the study of bred vectors.

For 1 ≤ k, we let [ak, bk] denote the kth spectral interval and let Vk(θ) denote the corresponding subspace given by the continuous foliation. We next explore the important connections between the exponential growth rates of solutions with initial conditions in Vk(θ) and the kth interval [ak, bk]. Among other things, we will encounter the monotonicity property and the strictly monotone property.

Let ν and λ be real numbers that satisfy ν < λ, and both Uν(θ, t) and Uλ(θ, t) have exponential dichotomies over . Then the monotonicity property holds:
e22
The strictly monotone property is a consequence of the observation that the following three statements are equivalent:
  • ,
  • ,
  • there is a spectral interval [ak, bk] in the interval (ν, λ).
This brings us to a basic property. Let and a0 satisfy and b1 < a0. Next we fix ν and λ so that
eq12
Then neither ν nor λ lie in the Sacker–Sell spectrum SSΣ. Furthermore, the interval (ν, λ) contains the spectral interval [ak, bk]. By the strictly monotone property, the space is larger than , while is larger than . Moreover, as is shown in Sacker and Sell (1978),
e23
It is a consequence of the relations (21) and (23) that if the initial condition (θ, x0) satisfies x0Vk(θ), then the solution U(θ, t)x0 satisfies
eq13
eq14
(Note that K0 depends on the choice of ν and λ.)

It should be noted that all the terms used above, including Pλ(θ) and Qλ(θ), vary continuously in θ. Furthermore, the exponential dichotomy is robust, in the sense that it varies continuously under small perturbations. Small changes in the model result in a related exponential dichotomy with small changes in Pλ, Qλ, K0, and α (see Pliss and Sell 1999).

Moreover, it is shown in Sacker and Sell (1976) that if λ ∈ SSΣ, then there is a (θ, x0) ∈ , such that ||x0|| ≠ 0, and the solution Uλ(θ, t)x0 satisfies
e24
What becomes apparent shortly is that the red vector must be in the space V1(θ). Furthermore, since λ = b1 is in SSΣ, the pair (θ, x0), that arises in (24) for this choice of λ, is a candidate for the “red vector” designation. A red vector must satisfy (24), but the converse need not be true (more on this later).

As noted above, the second tool to be used in the theory of Lyapunov exponents–vectors is the multiplicative ergodic theorem and the ergodic measures on . Found in JPS87 is a study of the links between the LYΣ and the SSΣ. While the SSΣ leads to a continuous foliation, as noted above, the MET leads to a “measurable” refinement of this continuous splitting, as is noted in Arnold (1998, 177–179, remark 4.2.9). The latter reference is noteworthy because it contains various extensions of the MET to problems not originally envisioned in the pioneering works of an earlier generation.

The multiplicative ergodic theorem: Let be a (nonempty) compact, invariant subset of the attractor , and let μ be an ergodic measure on with μ() = 1. Then there is an invariant set in , with μ() = 1, and there is a k, with , such that the following hold:

  1. There is a measurable foliation
    eq15
    where {Wk(θ), … , W1(θ)} is a linearly independent, measurable family of subspaces of with dim Wj(θ) = mj ≥ 1, for 1 ≤ jk, (m1 + ⋯ + mk) = K, and all θ.
  2. There are real numbers λj, for 1 ≤ jk, that are the strong Lyapunov exponents λ(θ, x0) = λj, for all (θ, x0) ∈ × Wj(θ), with ||x0|| = 1, and one has λk < … < λ1.
  3. The ergodic spectrum LYΣ(μ), which depends on the ergodic measure μ, is this collection {λ1, … λk}. The Lyapunov spectrum LYΣ, used below, is the union
    eq16
    over all ergodic measures μ.
  4. For each j, the measurable vector bundle
    eq17
    is an invariant set for the projective flow. Furthermore, because of the exponential separation between these vector bundles, the bundle × Wj(θ), with j = 1, is an attractor for the projective flow.

Connections between the SSΣ and LYΣ: As is shown in JPS87, the following relations hold:

  1. LYΣ ⊂ SSΣ. (Note that the last inequality alone does not prove the existence of a red vector. More on this below.)
  2. For each i with , there is an ergodic measure μ with support in the spectral interval [ai, bi].
  3. Assume that the invariant set is “dynamically connected”; that is, cannot be written as the union of two disjoint, nonempty, closed, invariant sets. Then the following holds: Since b1 is the largest value in SSΣ, there is an ergodic measure μ with the property that λ = b1 is a strong Lyapunov exponent. It then follows that λ1 = b1 is a Lyapunov exponent for some ergodic measure μ on , and consequently there is a unit vector x1 in W1(θ) with the property that x1 is a red vector.
  4. The previous item is valid for any of the endpoints {a1, … , ak; b1bk}, but the related ergodic measures may differ. Since b1 is the largest value of the compact set SSΣ, it follows from the identity λ1 = b1 that there does exist a red vector under the conditions stated above.

It can happen, as in the case with the CY92 model, that there is a unique red vector. Furthermore, in this case, as noted in section 4, one has dimV1(θ) = 1. Hence W1(θ) = V1(θ), and there is a unique red vector in the projective flow. Moreover, due to the exponential dichotomies occurring in the CY92 model, the red vector is robust, and it varies continuously with small changes in the model (see Pliss and Sell 2001).

On the other hand, when dimW1(θ) = 1, there is always a unique red vector. If in addition, one has dimV1(θ) ≥ 2, then the red vector may be only measurable and not continuous. In short, the red vector need not be robust.

3. Continuum limits of the BV and EBV algorithms

We now elucidate further the relationships between the BV and EBV algorithms and the dynamics of the underlying system. In particular, we now formalize the connections between these algorithms and the solutions of the linear tangent equation, that is, the finite-time Lyapunov vectors. We are interested in the behavior as the step size δt goes to 0. To accomplish these aims we revert to a continuum formulation and, for simplicity, assume that the vector y(t) in (1) is a member of the Euclidean space . Since energy norms are used in many geophysical fluid mechanics problems, we will take the usual l2 norm. We denote the norm and inner product by ||·|| and (·,·), respectively, with ||·||2 = (·,·).

The notation in this section differs from what has been set in the rest of the paper in minor ways, like the letters representing the functions. This is to emphasize that, unlike what one computes in practice, the dynamical systems here are continuous in time. However, everything is clearly explained to avoid ambiguities without cluttering the presentation with technical details.

We stress that the term continuum limit refers to the rescaling time (sometimes called a cycle), not to the numerical integration time step. In a numerical context, this corresponds to a strategy where both integration time steps and the rescaling times are small. While this has no importance for linear systems, it leads to different outcomes when applied to nonlinear systems, even for quite small perturbation amplitudes.

We first recall the system (1):
e25
Our first goal below is to obtain the formula
e26
where for all tt0,
eq18
and Gy(δy, t) = G(y + δy) − G(y), t0tT. The vector W(t; ϵ) corresponds to a continuously rescaled bred vector V(t) at amplitude ϵ and with initial perturbation V(t0) = ϵW0 (W has the same direction as V, but it has amplitude 1). We streamline the presentation by skipping some of the technical details in derivation, and we assume at the outset that G in (25) has the necessary differentiability properties to make all the mathematical steps rigorous.
Let y and y + δy be two solutions of (25) that satisfy the initial conditions y(t0) = y0 and (y + δy)(t0) = y0 + δy0. Then δy(t) is a solution of
e27
Even if G is autonomous, the resulting (27) for δy is nonautonomous. Integrating (27) for tt0, we obtain
e28
We next define ϵ = ‖δy(t0)‖, and a family of vectors υ(t) on a sphere centered at zero with radius ϵ in by
eq19
Assuming that δy is bounded away from 0, υ(t) obeys the evolution equation:
e29
which makes explicit the fundamental dependence of the time evolution of υ on the norm. Taking the scalar product of (27) with δy(t), we also have
eq20
Substituting this relation in (29) gives, after some simplifications,
e30
We integrate (30) independently on successive intervals
eq21
where T = t0 + Nδt, and we denote the solution of (30) on the interval Ik = [t0 + (k − 1)δt, t0 + kδt) by υk(t). This is a system of N integral equations with N free parameters (the integration constants)
e31
where δyk satisfies (28) with the initial condition at t = t0 + (k − 1)δt, that is, the left-hand boundary of the interval Ik. We observe that, at t = t0 + (k − 1)δt, υk(t) is an approximation of the discrete bred vector with short rescaling time step δt. Hence, we take it as the basic approximation for the continuously rescaled bred vector we seek. Note that the integrand in (31) is simply the component of the vector perpendicular to υk(t) in at time t.
We take
e32
assuming the limits exists, and extend the domain of definition of υk by setting υk(t) = 0 if tIk. Similarly, we extend the domain of definition of δyk. We then let VN = υ1 + ⋯ + υN, and note that VN is continuous in time on [t0, t0 + Nδt] (extended to t = t0 + Nδt by continuity) and piecewise differentiable. We define δYN analogously as the sum of extended δyk’s. Then
eq22
exists and is continuous in time, for instance, when G has the necessary differentiability properties so that the sequence VN, possibly after passing to a subsequence, converges uniformly on compact time intervals, as N → ∞ (or, equivalently, as δt → 0). Since VN coincides with δYN at every grid point (by the initial conditions imposed on δyk, k = 0, 1, 2, … , N − 1, and the definition of δYN), the difference between VN and δYN goes to zero as N → ∞ in an appropriate norm that is allowed by how smooth G is. When G is continuously differentiable, for example, this convergence would be uniform on compact time intervals. Thus, by passing to the limit N → ∞ in (31), we obtain the following representation formula for V:
e33
Under the regularity assumptions above on G, the integrand is continuous, and hence V is a mild solution of an associated differential equation. Therefore, we can bootstrap and prove the further regularity of V. Note that, by construction,
eq23
Therefore, (33) is equivalent to (26), if we take δy0 = ϵW0.
An immediate consequence of (26) is the following. When G is sufficiently regular, W(t; ϵ) converges uniformly on [t0, T] as ϵ → 0. Let us denote the pointwise limit by W(t). Then, by (26), and the observation
eq24
where the right-hand side is the directional derivative of G in the W(t) direction, it follows that W(t) satisfies the differential equation
e34a
where
eq25
and
e34b
We recall the linear tangent equation for the problem (25):
e35a
e35b
where DyG is the Jacobian matrix of the map G (y), and y(t) is the solution of (25). As in section 2, U(t, s) denotes the solution operator of (35), taking the solution at time s to the solution and time t. With a slight abuse of notation, we write U(t) := U(t, t0), so that in particular the solution X(t) of (35) at time t is simply U(t)W0. We recall also that U is a semiflow; that is,
e36
What is the relation between (34) and (35)? It can be easily seen that rescaling X(t) to unit length and restarting the integration of (35) with X(t)/‖X(t)‖ at any tm ∈ [t0, T] does not alter the value of X(t)/‖X(t)‖ for ttm. By induction, this follows for any finite number of rescaling–restarting cycles. This is because of the linearity of (35). We leave the details to the reader. By a direct approximation argument, or by a similar argument that led to (34), we obtain
eq26
Thus, we have just established that the continuously rescaled bred vector W(t; ϵ), that is, the solution of (26), converges to the corresponding solution (rescaled to size 1) of the tangent linear (35) with the same initial data, that is, to the solution of (34), as ϵ → 0 uniformly on the compact set [t0, T], assuming that G is smooth enough. Since the solutions of (34) have constant magnitude 1, it is the analog of (26) for the linear tangent model in terms of continuous rescaling.

While continuous rescaling is inconsequential for linear equations, this is not the case for the nonlinear ones. However, when the initial amplitude is very small, the results above indicate that one might be able to rescale less often and still get comparable results. This is simply because it takes longer for nonlinear effects to start to dominate the picture.

a. Continuum limit for EBV

We briefly discuss the continuum limit for the EBV algorithm. Writing a system of differential equations for the EBV algorithm is slightly more involved compared to the BV algorithm. The perturbations δy(ι) in the ensemble that grow fastest at the largest amplitude (before rescaling) at time t will obey (26) as long as they are the fastest growing. All the other perturbations will satisfy the following differential equation:
e37
where , and h(t) is the maximum growth rate of the norm among all ensemble members with the maximum amplitude ϵ at time t. Assuming enough smoothness on h(t), (37) can be obtained by taking the time derivative of the vector where . (We give here an outline of the derivation.)

An EBV member δy(t) at time t well inside the ball of radius ϵ, which is the maximum amplitude for perturbations, evolves to become an EBV member at time t + Δt as given by the product of a vector function and a scalar function. This is dictated by the discrete-time algorithm. The vector function starts as δy(t) at time t and evolves according to (27) as Δt → 0. The scalar function is the maximum amplitude of a time-dependent set of vectors. For a smooth enough G, at t the amplitudes of these vectors are the same and equal to the maximum amplitude within a small margin of error. As the size of the time step approaches zero, for a finite ensemble, this margin of error vanishes. We emphasize that this is a local-in-time description that follows closely the discrete formulation and is valid only on infinitesimal time elements. The rate of change of the EBV member under consideration (in the limit Δt → 0) is thus given by (37), which is the result of the product rule and observations in this outline. It then can be shown that as ε approaches zero for a fixed time t, all ensemble members align with the solutions of the linear tangent equation. The analysis is very similar to the BV case, but some steps are (only slightly) more involved.

b. Tuning the maximum amplitude

We note that the convergence to the linear tangent equation is expected to be quicker for the EBV compared to the BV for most of the ensemble members. This is manifested in (26) and (37), and verified numerically in the next section. The acceleration is due to the inherent size ordering in the EBV algorithm: information on the scales where linear tangent equation dominates tends to be preserved in a robust manner against the changes in the parameter ϵ. On the other hand, at finite amplitude away from 0, the rescaling rule in the BV algorithm might create or prolong the life of certain instabilities due to the nonlinear effects. The algorithm output needs to be examined independently to check whether these vectors are in fact relevant to the dynamics or just artifacts of the rescaling strategy. We believe that the EBV algorithm is more resistant and robust in this regard. The parameter ϵ is tuned under different considerations in the EBV and BV algorithms.

4. Applications of the Bred vector algorithms

We compare the BV and the EBV on two problems: The Lorenz equations, or Lorenz63 (see Lorenz 1963), and CY92, a dissipative and forced nonlinear partial differential equation that arises in modeling the thermohaline circulation (see Cessi and Young 1992). CY92 is a Cahn–Hilliard equation, and it is shown to have an inertial manifold. We also have occasion to compare the BV and EBV results to the finite-time Lyapunov vector outcomes.

Throughout we use an explicit fourth-order Runge–Kutta time-marching scheme for the calculation of the base solutions as well as for the calculations of the BV, EBV, and finite-time Lyapunov vectors.

a. The Lorenz63 model

The finite-dimensional, nonlinear Lorenz63 model has often been used as a benchmark for testing sensitivity and, in particular, as a test problem for BV (see, e.g., Evans et al. 2004).

Let X = (X1, X2, X3) ∈ and let
e38
is a 3 × 3 matrix-valued function. The Lorenz model is described by the solutions of the nonlinear system:
e39
The associated tangent linear model is the skew product system
e40
where U = (U1, U2, U3) ∈ . The U equation in (40), which is a linear equation, is of special interest to us.
We use the notation for the tangent linear model introduced in section 2c. With the initial condition θ = X0, we let θ · t = S(θ, t) = X(t) denote the solution of the nonlinear (39) that satisfies S(θ, 0) = θ. In this study we set r = 28, b = 8/3, and σ = 10. It is well known that for these values of the parameters the Lorenz63 model has a chaotic global attractor. For the nonautonomous U equation, , we let
e41
where U0, and U(0) = U0.

We use the term “attractor” to refer to a compact, invariant set that attracts a neighborhood of itself. As a result, is Lyapunov stable, as a set. Consequently, for 0 < ϵ < 1, there is a family of ϵ neighborhoods, Nϵ, of , where each neighborhood is positively invariant and . When we write that θ is near , we mean that θNϵ, for some small ϵ > 0. See Sell and You (2002, chapter 2) for a history of this concept and more information.

By using a somewhat different—but equivalent—formulation, Toth and Kalnay have suggested that the time evolution BV(t) is a good approximation of the tangent linear solution U(t), over bounded time intervals. As a consequence of the continuum limit theory in section 3, we see that this perceptive observation has a solid mathematical basis, provided that the time step δt is small and the perturbation amplitude is small, as well.

To find a numerical validation of the continuum limit theory described in section 3, we calculate the distance
e42
where dpr is the projective metric on [see (15)], and BV(t) is a BV at time t. For this calculation, we assume that both BV(t) and U(t) satisfy the same initial condition (y0, ) at t = 0 and that y0 = θ is near the attractor . We also use identical ensembles of perturbation vectors for both the BV and the EBV algorithms. We then fix T > 0, and we examine the distances D(T, δt) (in the projective space) for different choices of δt. Our goal is to show that D(T, δt) becomes smaller as δt gets smaller. The maximum and minimum values, for both the BV and the EBV algorithms, are reported in Table 1. For the calculations used to generate the data in Table 1, we had set the initial conditions for both BV(t) and U(t), at t = 0, so that X = 0.5688, Y = 0.4694, Z = 0.0119, and = (1, 1, 0). For the ensemble, we took the threefold Cartesian product of the set
eq27
in and projected this product onto the sphere of radius 0.1, centered at the origin, in . By eliminating the repetitions introduced with this projection, an ensemble of 584 distinct points in is obtained. (Note that the initial conditions are “near” the Lorenz attractor.) The trajectory for the nonlinear problem was computed with a step size dt = 0.0001, so that any observed variations are only due to the differences in the algorithms and the relative effects on the perturbations. In Table 1, we fixed T = 2 and made two choices for the rescaling time δt, namely δt = 0.004 and 0.001. The results compare the outcomes of the BV and the EBV algorithms. As noted in section 3, both algorithms EBV(t) and BV(t) approximate solutions of the tangent linear equation, as δt and ϵ go to 0. By using the projective metric the rates of convergence can be compared in terms of this metric.

The maximal values of D(T, δt), for both the BV and EBV algorithms, are essentially the same for both choices of δt, and the minimal values are essentially the same for δt = 0.004. When one moves from δt = 0.004 to δt = 0.001, both minimal values decrease, as expected. However, the drop in the minimal value for the EBV algorithm is substantially larger than the drop for the BV algorithm. By using the perturbation corresponding to the minimal drop in D(T, δt) for each of the algorithms, one arrives at the best approximation given by the given algorithm. Clearly the “min BV” and “min EBV” columns in Table 1 show that the EBV algorithm yields a better approximation than the BV algorithm.

One can, of course, use different initial conditions for the two solutions BV(t) and U(t) and/or larger perturbations. However, one cannot expect to replicate the results seen in Table 1 in that case. First, there is a transient phase, which ends when the two solutions are “near” the attractor. Even if this transient phase is short, there is still a problem: While the attractor is stable, as a set, the flow on the attractor is generally not Lyapunov stable. Essentially all pairs of nearby orbits may diverge over long time intervals.

An interesting outcome of the use of the EBV algorithm on Lorenz63 is that it exposes the fractal behavior of the Lorenz attractor. We computed a 584-member EBV, using the same parameter values, initial conditions, and base trajectory as was used in generating Table 1, for the δt = 0.001 case. We constructed plots by merging all the vectors between time 24 and 30, at intervals of dt = 0.1. In that time, at any given instant, one only observes a couple of members reaching highest amplitude. Three views of the EBVs (shown in Fig. 1) are obtained by rotating the same plot about the X3 = Z axis. Zooming into Fig. 1a by factors of 8, 32, and 60, respectively, we obtain Fig. 2. The fact that similar patterns are observed at several levels of magnification meets the requirement of “fractal behavior” that was introduced by Mandelbrot (1977).

Figure 3 shows the time evolution of a sample BV and the corresponding finite-time Lyapunov vector with the same initial perturbation. The results were obtained with a time step of δt = 0.005. The computations of the BVs and finite-time Lyapunov vectors were performed with a time interval equal to the dynamics time step. At t = 0, X = 0.1493, Y = 6.2575, and Z = 1.8407. The initial perturbation vector was (1, 1, 1). As expected, small to moderate-sized perturbations produce similar results in the finite-time Lyapunov calculation and the bred vector calculation.

Fig. 3.
Fig. 3.

(top to bottom) The three components of BV and of the finite-time Lyapunov vectors, as a function of time for the Lorenz63 system. (For parameters and initial conditions, see text.) The vectors are nearly coincidental over the whole time span. The perturbation size is 1 in all of the components. For our choice of initial conditions the vectors exhibit more temporal regularity at earlier times. This transient behavior disappears at t = 13, approximately.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

Figure 4 is devoted to the results of a calculation of EBV(t) alone, for the same case. The figure depicts a rescaled time evolution of the norm of 98 distinct ensemble members in the EBV calculation. The rescaling is done with respect to the usual Euclidean l2 norm. The initial perturbations are made to sample a perturbation sphere of amplitude 1 about the initial conditions. The figure highlights the rapid decay of many of the vectors and the eventual size-ordering that is inherent in the EBV algorithm. For 0 < t ≤ 13, the results in Fig. 4 describe the transient behavior and are rather chaotic. However, for t ≥ 13 a very interesting pattern evolves: we see that the largest member of the ensemble takes on the value 1, for all t ≥ 0, which is expected. (This corresponds to the Lyapunov vector with the largest exponent.) From the totality of the information on the ensemble, it is possible to extract the next two Lyapunov vectors, where the exponents satisfy λ3λ2λ1. However, this would require a deeper analysis, and it is deferred to future work. In fact, viewed as a whole, the graphs corresponding to those vectors with size smaller than 1 have very useful structural information: Notice the recurrent “spearlike” pattern, which occurs after t = 17. What is happening is that a group of “small” vectors in the attractor grow rapidly, as they go around the horn in the Lorenz attractor. We propose that the rationale for this behavior is that the nonlinear equations of motion temporarily overwhelm the uniform rescaling rule for these small vectors. For t ≥ 17, we are beyond the transient zone, and we would not expect such behavior to occur if the linear term X strongly dominates the nonlinear term N(X) in the vicinity of the attractor. This is an excellent illustration of the fact that the EBV algorithm preserves the role of the nonlinear terms in the equations of motion.

Fig. 4.
Fig. 4.

Plot depicting the evolution of the 2-norm of an ensemble of 98 EBV(t). Parameters and initial conditions are the same as in Fig. 3. Initially, in the interval [0, 13] we note a very fast decay of some of the ensemble members, leading to a sorting in size, beyond that time. As described in the text, this outcome is one of the most distinguishing features of the EBVs, when compared to an ensemble of individual BV outcomes.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

This feature of the EBV cannot be replicated by BV, even a BV with an ensemble of perturbations. The reason for this is that the rescaling rule for the BV algorithm forces all the perturbations to have the same norm, for each t ≥ 0. Thus, Fig. 4 for the EBV would be replaced by a figure for the BV, where all of the perturbations are plotted on the top line only.

b. A Cahn–Hilliard equation

In their work on the thermohaline dynamics, Cessi and Young (1992) proposed a coupled model for the circulation, salinity, temperature, and density of the oceans, with atmospheric forcing. The crux of the model is the partial differential equation for salinity: The slow-time dynamics of the ocean salinity S(x, t), zonally averaged, and as a function of latitude x ∈ [−π, π] and time t ≥ 0, is described by
e43
The equation is subject to zero-flux and zero-stress boundary conditions at the poles; however, we will be considering periodic boundary conditions [for steady and periodic forcing f(x) as well as equilibrium solution sin(x) the conclusions that follow apply to the zero-flux case]. The positive parameter α affects the strength of the linear stability of the model. We fix α = 3.5 × 10−3, γ = 0.001, and . The forcing f(x) is a prescribed function that reflects balances of evaporation and precipitation of freshwater; it can be symmetric, about the equator (x = 0), but is more typically asymmetric (see Eyink 2005). The second derivative of the forcing function is chosen to be
e44
as shown in Fig. 5a. The initial condition chosen for this computation is S0(x) = cos(x). The base solution obtained numerically is displayed in Fig. 5b.
Fig. 5.
Fig. 5.

(a) Forcing function fxx (44) used in the CY92 model; the numerical solution to CY92 is shown in (b). See Fig. 6 for the BV algorithm and finite-time Lyapunov vector algorithm outcomes.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

A great deal is known and can be said about the mathematical structure of CY92 and its solutions. By letting S = ∂xu and f = ∂xF the CY92 can be related to the Cahn–Hilliard equation (CHE) with forcing: The general CHE, for u above, is
e45
where u = u(t, x) is a scalar field, ν > 0 is a constant, Δ is the Laplacian operator, Δ2 is the biharmonic operator, and g is a polynomial of degree 3: , with a3 > 0. The term F = F(x) is the forcing function. In general, the domain Ω may be an open bounded domain in the Euclidean space , with 1 ≤ m ≤ 3. However, we restrict our attention to the case where m = 1 and Ω is the interval Ω = (−π, π), with boundary Γ = {±π}. As we will see, the multiplicity of the largest Lyapunov exponent for this problem is 1.
For the analysis of the solutions of the CHE, discussed further in section 4c, the standard Sobolev spaces Hj = Hj(Ω), where j ≥ 0 is an integer and H0 = L2(Ω), are used. As usual, the inner product and norm on H0 is denoted by 〈·,·〉 = 〈·,·〉0 and ‖·‖ = ‖·‖0. For j ≥ 0 one uses
eq28
where
eq29
and
eq30
The finite-time Lyapunov vector algorithm for the CY92 model is derived by first rewriting the equation as
eq31
We opt here to first linearize and then discretize. To obtain the tangent linear equation, we need to find the linear map such that
eq32
where ‖·‖ is a norm. Typically, this will be the norm in the space where solutions live, such as the Sobolev space H1 for S, or dictated by physical considerations. In this infinite-dimensional model, different norms are not necessarily equivalent.
First, we note that
eq33
We let L be a finite-difference approximation to . The discrete tangent linear equation becomes the product of two matrices , where is the matrix whose entries are found by discretizing the operator
eq34
where is the identity matrix, and is the matrix corresponding to the discretization of αxx. The finite-time Lyapunov algorithm is obtained by solving
eq35
for t ∈ [0, T], subject to some initial vector perturbation δΘ(0) = s, where δΘ is the discretization of the perturbation δS. The perturbation is normalized to the norm of the initial perturbation at each δtdt, the time step of the explicit Runge–Kutta 4 time integration scheme employed here. Here, T = 70. In the simulations, δt = 0.01. We applied second-order centered finite differences in space and used 121 grid points, −π = x0, x1, … , x120 = π.

By construction the finite-time Lyapunov vectors are not amplitude sensitive; however, the BVs are. Thus different amplitude perturbations will yield different BVs, in the case of a general nonlinear problem. This outcome has important practical implications, if one would like to use BV to infer either the structure in the field or the degree of sensitivity of the outcomes to perturbations in initial conditions. A challenging problem could thus arise in the context of large-scale simulations: what is considered a large structural change or a highly sensitive outcome is physics-dependent, perhaps even difficult to surmise quantitatively; the physics in question may not be fully understood and thus a reasonable perturbation amplitude is simply guessed.

We ran the finite-time Lyapunov vector and the BV algorithms using the same initial condition, forcing, and perturbation. First, we examine the effect of the size of the amplitude of the perturbation on the outcomes: we do so by keeping the shape of the perturbation fixed, changing only the overall amplitude. Figures 6a and 6b are plots of the final BV and finite-time Lyapunov vector, corresponding to 0.25- and 0.025-sized perturbations in the l2-norm, respectively. When the perturbations are small the BVs and the finite-time Lyapunov vectors are consistent qualitatively and, nearly so, quantitatively. The outcomes shown here are typical of the general case; that is, the qualitative and quantitative disagreement grows with an increase in the amplitude of perturbations.

Fig. 6.
Fig. 6.

Comparison of the finite-time Lyapunov vectors and BVs. Effect of amplitude of perturbation on the CY92 model, with initial conditions Y0 = cos(x). The initial perturbation was ϵsin(x). Comparison of the finite-time Lyapunov vector and the BV outcomes at t = 70: ϵ = (a) 0.25 and (b) 0.025.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

The shape or spectral content of the perturbation mattered as well. The spectrum of the perturbation is clearly important when projecting onto spectral bases for a reduced representation. To illustrate this, we use the CY92 model, with the same forcing as before and same initial condition. We examine monochromatic sine wave perturbations with wavenumber j = 1, 2, … , 6 of the form
e46
where all of the perturbation amplitudes remain the same, ϵj = 0.25. The choice of the phase is inconsequential: to show this we chose noncommensurate phases among the sine wave components. In Fig. 7a we show the BVs associated with each of the perturbations (j = 1, 2, … , 6), at t = 70. Figure 7b shows the EBVs at t = 70. Even at t = 5, shown in Fig. 7c, we already see an amplitude-ordered structure in the EBV. The space–time plot of all EBVs is plotted in Fig. 7d; we note the very short transient phase, lasting till about t = 4, followed by a structure that is clearly dominated by the largest member of the EBV. The EBV calculation was run with identical parameters to those used in Fig. 7a, with the ensemble consisting of the same initial perturbations given in (46). In the BV case we are getting outcomes that do not have the reductive appearance of the EBV. On the other hand, there is no ambiguity to the prevailing ensemble member in the EBV case. This ensemble member has a clear correspondence in structure to the path and its perturbation field. The BVs should eventually agree with the EBVs, nevertheless. It is safe to assume that this will happen only after a very long time, longer than the time interval that might be suggested by the base solution.
Fig. 7.
Fig. 7.

(a) The jth simulation corresponding to a sine wave perturbation at t = 70 after the superposition of six BV simulations per (46). (b) Cross section of the EBVs at t = 70 run with an ensemble of perturbations [see (46)]. We note that unlike the BV case in (a), the EBV has settled into a structure that clearly reflects the dominant EBV ensemble member. (c) Cross section of the same EBVs, at an earlier time, t = 5. Even for short times, the EBV already shows an amplitude-ordering of its vectors. (d) Superposition of all EBVs, as a function of time and space.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

We also compared the BV, for each of the perturbations in (46), to the outcomes of the finite-time Lyapunov vector calculation. Figure 8 compares the BVs and finite-time Lyapunov vectors for each j wavenumber perturbation. The amplitudes are set to ϵj = 0.25, for j = 1, 2, … , 6. The structure of the finite-time Lyapunov vectors, to within a sign, is qualitatively the same, regardless of the wavenumber of the perturbation. The BVs are not. If the perturbation was made considerably smaller the differences between the BVs and the finite-time Lyapunov vectors would become small, as expected: As shown in section 3, the BVs and the finite-time Lyapunov vectors must be similar to each other, provided the perturbations are small enough. For larger amplitudes, the BVs might show more structure, since nonlinear effects could play a role in the structure of the BVs. Apparently, ϵ = 0.25 is already in the range of large perturbations of the CY92 about the solution chosen. It was not clear whether a chosen perturbation is large or small, based solely on the CY92 model itself; it was only clear to us after a comparison of the outcomes of the finite-time Lyapunov case and the BV case.

Fig. 8.
Fig. 8.

Wavenumber dependence of BVs and finite-time Lyapunov vectors: CY92 model, cosine initial condition, and nonsymmetric forcing (see Fig. 5). Comparison of the finite-time Lyapunov vectors and the BVs for sine wave perturbations of size ϵj = 0.25. The perturbations are the individual sine waves in (46): j = (a) 1, (b) 2, (c) 3, (d) 4, (e) 5, and (f) 6.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

c. Cahn–Hilliard dynamics and the existence of an attractor

The biharmonic operator Bu = Δ2u on Ω and the linear biharmonic equation,
e47
play a basic role in the study of the CHE. We assume for now that the operator B satisfies either the nonflux boundary conditions,
e48
or the periodic boundary conditions (see Sell and You 2002). Hereafter we assume that the forcing function F in (45) satisfies , for some j ≥ 0. That is to say,
eq36
Thus, by integrating (45), one observes that any solution u = u(t, x) satisfies
e49
One seeks solutions of the CHE (45) in the Sobolev space . A mild solution u = u(t) of the initial value problem is given by
e50
We denote the maximally defined solutions of (50) by u(t) = S(t)u0. With , this mild solution is uniquely determined, with , for 0 ≤ t < T(u0), where 0 < T(u0) ≤ ∞.
Because of (49), we see that, for j ≥ 2, the spaces
e51
are positively invariant spaces for the solutions of (45). Thus we focus only on solutions S(t)u0 that satisfy . We also denote the collection of stationary solutions of the CHE by
eq37
and .

1) The CHE with F ≡ 0

The basic problem with forcing F ≡ 0 is of special interest. Equation (42) becomes
e52
To study the solutions of (52), the Landau–Ginsburg functional is used:
e53
In addition, the Landau–Ginsburg functional satisfies
e54
where K(u) = −νΔu + g(u). Furthermore, there exist positive constants f0 and C0 such that −f0G(s) ≤ C0s4 + f0 and
e55
Since J(u0) is bounded below [see (55)] and decreasing along orbits [see (54)], it follows that J[S(t)u0] is a Lyapunov function (see LaSalle and Lefschetz 1961). This implies that the mild solution S(t)u0 is defined for all t > 0; that is, T(u0) = ∞, for every . It is fact that, whenever F ≡ 0 and , then ω(u0), the omega limit set of the solution S(t)u0 is a nonempty, compact, connected invariant set in Q0.

Let us now return to the CHE with forcing (45). By integrating (45), where and , as well, it follows that , for all t ≥ 0.

2) The global attractor

Assume that B satisfies the boundary conditions BC; that is, either the nonflux condition (48) holds, or the periodic boundary conditions hold. Let S(t) be the semiflow generated by the solution operator on . Then the following hold:

  1. S(t) has a nonempty, compact global attractor in , and attracts all bounded sets in . The attractor depends continuously on the forcing function .
  2. When , then the attractor is a compact, invariant set in V4r, for each r with 0 ≤ r ≤ 1.
  3. The set Q0 is nonempty, compact, and invariant with .
  4. Lastly, there is an inertial manifold for the solutions of the infinite-dimensional system (45) (see Foias et al. 1988 and Sell and You 2002). One finds this manifold by using the orthogonal projection onto the lowest N nodes, that is to say, into Span{ek: kN}, where {ek} are the eigenfunctions for the biharmonic operator B. One then makes a change of variables u = υ + w, where , and . The (υ, w) system,
    e56
    is equivalent to the CHE (42). It is then shown that, for N large, the variable w is enslaved to the variable υ in some neighborhood of . That is to say, w = Φ(υ), for a suitable function Φ. It turns out the longtime dynamics of (45) is equivalent to the longtime dynamics of the finite-dimensional ordinary differential equation:
    e57
    See Sell and You (2002, and references therein) for more details.

d. BV and EBV in the higher nonlinear regime

For sufficiently small perturbations and short renormalizing time intervals the finite-time Lyapunov vector, the BV algorithm, and the EBV are similar in outcomes. As we depart from these conditions, we see significant differences in the outcomes of the three methods. The BV outcomes are most sensitive to the amplitude and the frequency of the perturbations. Regardless of the amplitude of the perturbations, the EBV outcomes are structurally unambiguous and robust.

We revisit the CY92 simulations of BV and EBV, using the same forcing and initial conditions used section 4b but set the time-scale parameter α = 0.01, increase the spatial resolution to 320 points, and set the integration time step at δt = 0.000 015. The perturbation field is the same as in (46); however, we will increase the size of the perturbation of each of the components. For ϵj = 0.6, 0.8, and 1.2, with j = 1, 2, … , 6, in (46) we obtain Fig. 9, which shows all BVs at t = 19.025. We expect a simple structure in the perturbation field, and thus the BV should reflect this. The EBV results appear in Fig. 10. Comparison of Figs. 9 and 10 shows how the BV outcomes look qualitatively different from their EBV counterparts as we increase the initial amplitude of the ensemble. The comparison is further aided by reference to Fig. 11, in which each ensemble member of the EBV, at t = 19.025, has been rescaled to 1 in amplitude. The BV outcomes shown in Fig. 9 do not yield the structural clarity that the EBV ensemble displays in Fig. 10; this is a natural consequence of the size-ordering inherent in the EBV algorithm.

Fig. 9.
Fig. 9.

The BVs at t = 19.025, corresponding to ϵj = ϵ, for all j. ϵ = (a) 0.6, (b) 0.8, and (c) 1.2. The outcomes are very sensitive to nonlinearity. There is a resulting ambiguity in structure of the perturbation field.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

Fig. 10.
Fig. 10.

The EBV with ϵ = (a) 0.6, (b) 0.8, and (c) 1.2. Compare outcomes to Fig. 9. The vectors are shown in their original scales.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

Fig. 11.
Fig. 11.

The EBV outcomes shown in Fig. 10, with each vector rescaled to an l2 norm of 1 to aid in visual comparison. Perturbation amplitudes ϵ = (a) 0.6, (b) 0.8, and (c) 1.2. Note that as the perturbation amplitude increases, the resemblance between the BV and the EBV outcomes is lost, except for one of the vectors.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

In reference to Figs. 9c and 10c we see the similarity between one and only one of the BV vectors and the largest EBV. A striking structural feature in the EBV results shown in Fig. 10 is that there are only three significantly different shapes, among all of the six vectors (see Fig. 11, where the vectors have been rescaled to have size 1 in L2). For relative comparison, see Fig. 9.

The sensitivity of the outcomes to the size of initial perturbations is significantly different in the EBV and the BV outcomes. There are shape variations among the BVs as the amplitude of the perturbation is increased, as evidenced in Fig. 9. This degree of sensitivity is not as prevalent in the EBV outcomes, shown in Fig. 11. The BV algorithm does not distinguish between different perturbation scales. A large perturbation BV calculation will thus not yield information concerning smaller scales. In contrast, see Fig. 11c, for the EBV case, where small-scale information is evident. (See also Fig. 10c.)

5. Implementation issues

Like the BV algorithm, the EBV algorithm is capable of dealing with legacy code. The important difference between implementing a code that does an ensemble of BV and the EBV is that while the former can be run concurrently, in the EBV the ensemble members require normalization to each other. In terms of coding, this is a minor issue, if the state variable dimension of the underlying model is moderate. For very large problems communication becomes an issue, but not at all unfamiliar in concurrent or hybrid computing. Any additional computational issues borne by the EBV are well outweighed by the higher informational content of the EBV over the BV. Moreover, the EBV is much more robust under the nonlinear effects, as illustrated in section 4, than BV. This last aspect is very important, and it should be studied in the context of more chaotic equations in fluid mechanics, meteorology, and geophysics.

The numerical outcome of the BV and EBV algorithms depends on the choice of norm used for rescaling. This is alluded to in Riviére et al. (2008), but the reason for this dependence turns out to be easily explained and can be significant if nonlinear effects are not negligible. The dependence of the outcomes on the norm lies in the fact that it is not possible, in the general case, to scale out the norm in the algorithm if the underlying dynamics are nonlinear. To illustrate the norm dependence, we consider a simple two-dimensional system,
e58
subject to the (same) initial conditions [X1(0), X2(0)] = (0.8, −1). Figure 12 shows the outcomes of the BV algorithm for three different choices of norms, starting from perturbations of size no greater than 0.15. The norms we employ are all standard (finite) Lp norms. The resulting BVs reflect the characters of these norms. That different outcomes are obtained is not surprising: The shape of the unit sphere changes depending on which Lp norm is used.
Fig. 12.
Fig. 12.

The (a) L2 norm, (b) L1 norm, and (c) L norm of a sample BV as a function of time, for the system in (58). The initial conditions are the same in all cases, and the amplitude of the perturbations was 0.15.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

A more weighty consideration related to norms concerns physical and theoretical considerations. Conservation laws provide guidance for the most appropriate norms for the given dynamics (e.g., in Rayleigh–Bérnard convection, the temperature enjoys a maximum principle, while the velocity does not), but other considerations will play a role (e.g., a sup-norm may be an obvious choice in determining the location of severe weather events). It will not be uncommon, therefore, that a mixture of norms may be necessary in multiphysics problems; the fact that the choice of norm in the EBV–BV affects the results is in fact a good thing, not a bad one.

To end this section we wish to highlight a practical consideration that may be not be familiar to practitioners: implementing BV or EBV algorithms computationally. The specific issue is the impact of finite precision computing on the outcomes. It is easy to show that these algorithms have high numerical sensitivity. To illustrate how this plays out we consider the problem of calculating the vectors associated with
e59
Here is a square matrix of constants. For linear problems BV and EBV (and the finite-time Lyapunov) algorithms must yield the same vectors. We choose to illustrate computationally numerical ill-conditioning on a problem that exhibits transient growth due to the nonnormal structure of the matrix . We emphasize, however, that the numerical sensitivity we will be highlighting in this example does not hinge on the nature of the dynamics but rather on the algorithmic form of the BV and EBV themselves.

We take to be an upper-triangular Jordan-block matrix of dimension 5, where has a single eigenvalue λ, which is repeated on the main diagonal, while the diagonal directly above the main diagonal has nonzero entries. For simplicity we assume these to have the same value ρ. We assume further that λ = −1 and ρ = 1. The general solution operator contains “generalized” eigensolutions with growth rates tket, where 0 ≤ k ≤ 4. [In this case, the eigenvalue has (algebraic) multiplicity 5.] For this linear problem we expect to see, provided we take t sufficiently long to forget the transient behavior, a very trivial outcome to BV, or EBV.

In Fig. 13 we summarize the results of the BV calculation on this system. We employed an integration time step of 0.001, a perturbation initially of magnitude 0.01. We performed the calculation in double precision, using an explicit Runge–Kutta 4, but payed little attention to how numerical sensitivity was handled. In Fig. 13a we show the BV, that is, , as a function of time. The calculation of BV, using (3), clearly diverged, shortly after about t = 30 (and thus is not shown in Fig. 13a). However, there were indications that something was not right even before it became obvious that the solution was wrong: In Fig. 13b we show the 2-norm of M(Yn + , δt) − M(Yn, δt). It monotonically increases, even though all the factors tket, for 0 ≤ k ≤ 4 in the solution operator, decay for t > 4.

Fig. 13.
Fig. 13.

(a) A sample BV, as a function of time. The calculation diverges and eventually fails (not shown); (b) 2-norm+of the numerically approximated M(Yn + , δt) − M(Yn, δt) as a function of time, for the linear system . Here, is an upper-triangular Jordan-block matrix of dimension 5, where has a single eigenvalue −1, which is repeated on the main diagonal, while the superdiagonal has entries 1. (c) 2-norm of the numerically approximated M(Yn + , δt) − M(Yn, δt), well-conditioned case.

Citation: Monthly Weather Review 140, 7; 10.1175/MWR-D-10-05054.1

The outcome is not related to choosing a rescaling time to be too long: In fact, in this computation the rescaling is performed at each computational step. Moreover, the time step chosen was sufficiently small to guarantee asymptotic stability in the numerical integrator. A smaller time step would have been able to ameliorate to a certain extent the numerical sensitivity of the difference M(Yn + , δt) − M(Yn, δt); however, in very large-scale problems this might not be practical or even possible. Figure 13c shows the 2-norm of M(Yn + , δt) − M(Yn, δt), using a numerically well-conditioned implementation of the BV algorithm. The strategy used to obtain a well-conditioned outcome was to normalize the elements in the required subtraction before computing their difference. The well-conditioned calculation was capable of qualitatively good results. However, the strategy adapted here to increase the numerical stability was by no means generally applicable to all problems, nor was it optimal.

6. Concluding remarks

The main thrust of our work is to propose an ensemble-based vector-breeding algorithm, the ensemble bred vector (EBV) algorithm. It is based on the bred vector (BV) algorithm introduced by Toth and Kalnay (1993, 1997). We compare the EBV to the BV algorithm and the finite-time Lyapunov vector algorithms. In the EBV, an ensemble of initial perturbations is bred concurrently and then rescaled by the size of the largest member of the bred ensemble. The uniform normalization of all the ensemble members after each cycle is the distinctive trait of the EBV algorithm, which leads to some profound differences when compared to the BV algorithm.

As expected, when initial perturbations are sufficiently small, the EBV, the BV, and the finite-time Lyapunov vector algorithms lead to similar results. We gave a theoretical justification of this phenomenon by looking at the corresponding time-continuum analogs of the BV and EBV algorithms. We rescale frequently: The algorithms and results are formulated, for simplicity, assuming that the rescaling is done after every discrete time step.

In section 3, we develop a solid mathematical basis for both the BV and EBV algorithms and show that each algorithm results in good approximations of the solutions of the tangent linear model, when the step size δt is small. As is seen in Table 1, the EBV algorithm has a substantial advantage over the (classical) BV algorithm, in the sense that the drop in the minimal EBV error is substantially better than the corresponding drop in the minimal BV error.

In the study of the Lorenz attractor, the classical BV algorithm has a shortcoming that limits the ability to use this algorithm to study the dynamics inside the attractor. In particular, , for all n ≥ 0 and all ι [see Eq. (5)]. The BV algorithm maps the cloud into the surface of a two-dimensional sphere. The third dimension is lost, and with it so are the fractal patterns seen in Figs. 1 and 2. Also, the many patterns seen in Fig. 4 are lost because the BV rescaling would produce the single line at height ϵ as output.

Last, in Figs. 9, 10, and 11, we examine a series of related test problems that depart from the tangent linear model. The point here is to get a comparison of the performance of the two algorithms, as one moves further into nonlinear regime. Once again, one sees a significant advantage of the EBV over the BV.

The theoretical aspects for the EBV and BV algorithms are presented in section 2e. As is noted there, our application to the basic issue of the sensitivity with respect to errors in the initial conditions relies heavily on JPS87.

In conclusion, for the applications described in this article, we believe that the new EBV algorithm has been shown to be superior to the traditional BV algorithm. Finally, we ask: Is the EBV algorithm the “last” word on modifications of the classical BV? Probably not. However, we do expect that the EBV will serve as a good starting point for new theories of bred vectors.

Acknowledgments

JMR was partially supported by NSF Grant DMS-0335360. ALM was partially supported by NSF Grants DMS-0708902, DMS-1009713, and DMS-1009714. NB, ALM, JMR, and GRS wish to thank the Institute for Mathematics and its Applications (IMA) for their support and hospitality. JR and NB wish to acknowledge the Gulf of Mexico Research Institute for support. Research at the IMA is supported by the National Science Foundation and the University of Minnesota. GRS acknowledges his appreciation of Robert Sacker for related suggestions he made at an early stage in the development of this article. We express our sincere gratitude to the referees, who made an in-depth reading of the original manuscript and offered several helpful suggestions that led to improvements in the paper.

REFERENCES

  • Arnold, L., 1998: Random Dynamical Systems. Springer, 586 pp.

  • Buizza, R., , J. Tribbia, , F. Molteni, , and T. Palmer, 1993: Computation of optimal unstable structures for numerical weather prediction models. Tellus, 45A, 388407.

    • Search Google Scholar
    • Export Citation
  • Cessi, P., , and W. R. Young, 1992: Multiple equilibria in two-dimensional thermohaline circulation. J. Fluid Mech., 241, 291309.

  • Cheung, K. K. W., 2001: Ensemble forecasting of tropical cyclone motion: Comparison between regional bred modes and random perturbations. Meteor. Atmos. Phys., 78, 2335.

    • Search Google Scholar
    • Export Citation
  • Corazza, M., and Coauthors, 2003: Use of the breeding technique to estimate the structure of the analysis errors of the day. Nonlinear Processes Geophys., 10, 233243.

    • Search Google Scholar
    • Export Citation
  • Deremble, B., , F. D’Andrea, , and M. Ghil, 2009: Fixed points, stable manifolds, weather regimes, and their predictability. Chaos, 19, 043109. doi:10.1063/1.3230497.

    • Search Google Scholar
    • Export Citation
  • Evans, E., , N. Bhatti, , J. Kinney, , L. Oann, , M. Peña, , S. Yang, , and E. Kalnay, 2004: RISE undergraduates find that regime changes in Lorenz’s model are predictable. Bull. Amer. Meteor. Soc., 85, 520524.

    • Search Google Scholar
    • Export Citation
  • Eyink, G., 2005: Statistical hydrodynamics of the thermohaline circulation in a two dimensional model. Tellus, 57A, 100115.

  • Foias, C., , G. R. Sell, , and R. Temam, 1988: Inertial manifolds for nonlinear evolutionary equations. J. Differ. Equations, 73, 309353.

    • Search Google Scholar
    • Export Citation
  • Gneiting, T., , and A. E. Raftery, 2005: Atmospheric science—Weather forecasting with ensemble methods. Science, 310, 248249.

  • Hallerberg, S., , D. Pazo, , J. M. Lopez, , and M. A. Rodriguez, 2010: Logarithmic bred vectors in spatiotemporal chaos: Structure and growth. Phys. Rev. E, 81, 066204, doi:10.1103/PhysRevE.81.066204.

    • Search Google Scholar
    • Export Citation
  • Hansen, J. A., , and L. A. Smith, 2000: The role of operational constraints in selecting supplementary observations. J. Atmos. Sci., 57, 28592871.

    • Search Google Scholar
    • Export Citation
  • Johnson, R. A., , K. J. Palmer, , and G. R. Sell, 1987: Ergodic properties of linear dynamical systems. SIAM J. Math. Anal., 18, 133.

  • Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, 341 pp.

  • LaSalle, J. P., , and S. Lefschetz, 1961: Stability by Lyapunov’s Direct Method with Applications. Academic Press, 134 pp.

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130141.

  • Lyapunov A. M., 1992: The general problem of the stability of motion (in Russian). Int. J. Control, 55, 521–790.

  • Mandelbrot, B. B., 1977: Fractals: Form, Chance, and Dimension (in French). W. H. Freeman and Co., 365 pp.

  • Mu, M., , and Z. N. Jiang, 2008: A new approach to the generation of initial perturbations for ensemble prediction: Conditional nonlinear optimal perturbation. Chin. Sci. Bull., 53, 20622068.

    • Search Google Scholar
    • Export Citation
  • Palmer, T. N., , R. Gelaro, , J. Barkmeijer, , and R. Buizza, 1998: Singular vectors, metrics and adaptive observations. J. Atmos. Sci., 55, 633653.

    • Search Google Scholar
    • Export Citation
  • Pliss, V. A., , and G. R. Sell, 1999: Robustness of exponential dichotomies in infinite-dimensional dynamical systems. J. Dyn. Differ. Equations, 11, 471513.

    • Search Google Scholar
    • Export Citation
  • Pliss, V. A., , and G. R. Sell, 2001: Perturbations of normally hyperbolic manifolds with applications to the Navier–Stokes equations. J. Differ. Equations, 169, 396492.

    • Search Google Scholar
    • Export Citation
  • Primo, C., , M. A. Rodriguez, , and J. M. Gutierrez, 2008: Logarithmic bred vectors. A new ensemble method with adjustable spread and calibration time. J. Geophys. Res., 113, D05116, doi:10.1029/2007JD008998.

    • Search Google Scholar
    • Export Citation
  • Riviére, O., , G. Lapeyre, , and O. Talagrand, 2008: Nonlinear generalization of singular vectors: Behavior in a baroclinic unstable flow. J. Atmos. Sci., 65, 18961911.

    • Search Google Scholar
    • Export Citation
  • Sacker, R. J., , and G. R. Sell, 1976: Existence of dichotomies and invariant splittings for linear differential systems. II. J. Differ. Equations, 22, 478496.

    • Search Google Scholar
    • Export Citation
  • Sacker, R. J., , and G. R. Sell, 1977: Lifting properties in skew-product flows with applications to differential equations. Mem. Amer. Math. Soc., 11 (190), 1–67.

    • Search Google Scholar
    • Export Citation
  • Sacker, R. J., , and G. R. Sell, 1978: A spectral theory for linear differential systems. J. Differ. Equations, 27, 320358.

  • Sacker, R. J., , and G. R. Sell, 1980: The spectrum of an invariant submanifold. J. Differ. Equations, 38, 135160.

  • Sell, G. R., , and Y. You, 2002: Dynamics of Evolutionary Equations. Springer, 670 pp.

  • Toth, Z., , and E. Kalnay, 1993: Ensemble forecasting at NCEP: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 23172330.

    • Search Google Scholar
    • Export Citation
  • Toth, Z., , and E. Kalnay, 1997: Ensemble forecasting at NCEP: The breeding method. Mon. Wea. Rev., 125, 32973318.

  • Wang, X. G., , and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 11401158.

    • Search Google Scholar
    • Export Citation
  • Wei, M. Z., , and Z. Toth, 2003: A new measure of ensemble performance: Perturbation versus error correlation analysis (PECA). Mon. Wea. Rev., 131, 15491565.

    • Search Google Scholar
    • Export Citation
  • Wolfe, C. L., , and R. M. Samelson, 2007: An efficient method for recovering Lyapunov vectors from singular vectors. Tellus, 59A, 355366.

    • Search Google Scholar
    • Export Citation
Save