• Alligood, K., T. Sauer, and J. Yorke, 1997: Chaos: An Introduction to Dynamical Systems. Springer-Verlag, 603 pp.

  • Bergot, T., G. Hello, and A. Joly, 1999: Adaptive observations: A feasibility study. Mon. Wea. Rev., 127 , 743765.

  • Bishop, C. H., and Z. Toth, 1999: Ensemble transformation and adaptive observations. J. Atmos. Sci., 56 , 17481765.

  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129 , 420436.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., C. A. Reynolds, and H. K. Tippett, 2003: Optimization of the fixed global observing network in a simple model. J. Atmos. Sci., 60 , 14711489.

    • Search Google Scholar
    • Export Citation
  • Bretherton, C. S., M. Widmann, V. P. Dymnikov, J. M. Wallace, and I. Bladé, 1999: The effective number of spatial degrees of freedom of a time-varying field. J. Climate, 12 , 19902009.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., and T. N. Palmer, 1995: The singular-vector structure of the atmospheric general circulation. J. Atmos. Sci., 52 , 14341456.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., J. Tribbia, F. Molteni, and T. N. Palmer, 1993: Computation of optimal unstable structures for a numerical weather prediction model. Tellus, 45A , 388407.

    • Search Google Scholar
    • Export Citation
  • Chang, E. K. M., 1999: Characteristics of wave packets in the upper troposphere. Part II: Seasonal and hemispheric variations. J. Atmos. Sci., 56 , 17291747.

    • Search Google Scholar
    • Export Citation
  • Chang, E. K. M., 2000: Wave packets and life cycles of troughs in the upper troposphere: Examples from the Southern Hemisphere summer season of 1984/85. Mon. Wea. Rev., 128 , 2550.

    • Search Google Scholar
    • Export Citation
  • Chang, E. K. M., and D. B. Yu, 1999: Characteristics of wave packets in the upper troposphere. Part I: Northern Hemisphere winter. J. Atmos. Sci., 56 , 17081728.

    • Search Google Scholar
    • Export Citation
  • Errico, R. M., 2000: Interpretations of the total energy and rotational energy norms applied to determination of singular vectors. Quart. J. Roy. Meteor. Soc., 126 , 15811600.

    • Search Google Scholar
    • Export Citation
  • Errico, R. M., T. Vukicevic, and K. Reader, 1993: Comparison of initial and lateral boundary condition sensitivity for a limited-areas model. Tellus, 45 , 539577.

    • Search Google Scholar
    • Export Citation
  • Farrell, B. F., 1985: Transient growth of damped baroclinic waves. J. Atmos. Sci., 42 , 27182727.

  • Farrell, B. F., 1989: Transient development in confluent and diffluent flow. J. Atmos. Sci., 46 , 32793288.

  • Gelaro, R., R. H. Langland, G. D. Rohaly, and T. E. Rossmond, 1999: An assessment of the singular-vector approach to target observations using the FASTEX dataset. Quart. J. Roy. Meteor. Soc., 125 , 32993328.

    • Search Google Scholar
    • Export Citation
  • Golub, G. H., and C. F. Van Loan, 1989: Matrix Computations. 2d ed. The Johns Hopkins University Press, 642 pp.

  • Iyengar, G., Z. Toth, E. Kalnay, and J. S. Woolen, 1996: Are the bred vectors representative of analysis errors? Preprints, 11th Conf. on Numerical Weather Prediction, Norfolk, VA, Amer. Meteor. Soc., 64–65.

  • Kaplan, J. L., and J. A. Yorke, 1979: Chaotic behavior of multidimensional difference equations. Functional Differential Equations and Approximations of Fixed Points: Proceedings, Bonn, July 1978, Vol. 730H.-O. Peitgen and H.-O. Walther, Eds., Springer-Verlag, 204–227.

  • Lee, S., and I. M. Held, 1993: Baroclinic wave packets in models and observations. J. Atmos. Sci., 50 , 14131428.

  • Leutbecher, M., 2003: A reduced rank estimate of forecast error variance changes due to intermittent modifications of the observing network. J. Atmos. Sci., 60 , 729742.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55 , 399414.

    • Search Google Scholar
    • Export Citation
  • Majumdar, S. J., C. H. Bishop, B. J. Etherton, I. Szunyogh, and Z. Toth, 2001: Can an ensemble Kalman filter predict the reduction in forecast error variance produced by targeted observations? Quart. J. Roy. Meteor. Soc., 127 , 28032820.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122 , 73119.

    • Search Google Scholar
    • Export Citation
  • Orlanski, I., 2003: Bifurcation in eddy life cycles: Implications for storm track variability. J. Atmos. Sci., 60 , 9931023.

  • Orlanski, I., and J. Katzfey, 1991: The life cycle of a cyclone wave in the Southern Hemisphere. Part I: Eddy energy budget. J. Atmos. Sci., 48 , 19721998.

    • Search Google Scholar
    • Export Citation
  • Orlanski, I., and E. Chang, 1993: Ageostrophic geopotential fluxes and upstream development of baroclinic waves. J. Atmos. Sci., 50 , 212225.

    • Search Google Scholar
    • Export Citation
  • Ott, E., 2002: Chaos in Dynamical Systems. 2d ed. Cambridge University Press, 490 pp.

  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A , 415428.

  • Palmer, T. N., R. Gelaro, J. Barkmeijer, and R. Buizza, 1998: Singular vectors, metrics, and adaptive observations. J. Atmos. Sci., 55 , 633653.

    • Search Google Scholar
    • Export Citation
  • Patil, D. J., B. R. Hunt, E. Kalnay, J. A. Yorke, and E. Ott, 2001: Local low-dimensionality of atmospheric dynamics. Phys. Rev. Lett., 86 , 58785881.

    • Search Google Scholar
    • Export Citation
  • Rabier, F., P. Courtier, and O. Talagrand, 1992: An application of adjoint models to sensitivity analysis. Beitr. Phys. Atmos., 65 , 177192.

    • Search Google Scholar
    • Export Citation
  • Rabier, F., E. Klinker, P. Courtier, and A. Hollingsworth, 1996: Sensitivity of forecast errors to initial conditions. Quart. J. Roy. Meteor. Soc., 122 , 121150.

    • Search Google Scholar
    • Export Citation
  • Snyder, C., 1999: Error growth in flows with finite-amplitude waves or coherent structures. J. Atmos. Sci., 56 , 500506.

  • Szunyogh, I., Z. Toth, R. E. Morss, S. J. Majumdar, B. J. Etherton, and C. H. Bishop, 2000: The effect of targeted dropsonde observations during the 1999 winter storm reconnaissance program. Mon. Wea. Rev., 128 , 35203537.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., G. Gyarmati, and D. Devenyi, 2002a: Using observed data for testing the statistical consistency of initial ensemble perturbations. Idöjárás, 105 , 293303.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., Z. Toth, A. V. Zimin, S. J. Majumdar, and A. Persson, 2002b: Propagation of the effect of targeted observations: The 2000 winter storm reconnaissance program. Mon. Wea. Rev., 130 , 11441165.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., E. J. Kostelich, G. Gyarmati, B. R. Hunt, E. Kalnay, E. Ott, D. J. Patil, and J. A. Yorke, 2004: A local ensemble Kalman filter for the NCEP GFS model. Preprints, Symp. on 50th Anniversary of Numerical Weather Prediction, College Park, MD, Amer. Meteor. Soc., J2.4.

  • Talagrand, O., 1981: A study of the dynamics of four-dimensional data assimilation. Tellus, 33 , 4360.

  • Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125 , 32973319.

  • Toth, Z., I. Szunyogh, C. Bishop, S. Majumdar, R. Morss, and S. Lord, 2001: The use of targeted observations in operational numerical weather forecasting. Proc. Fifth Symp. on Integrated Observing Systems, Albequerque, NM, Amer. Meteor. Soc., 185–190.

  • Toth, Z., and Coauthors, 2002: Adaptive observations at NCEP: Past, present, and future. Preprints, Symp. on Observations, Data Assimilation, and Probabilistic Prediction, Orlando, FL, Amer. Meteor. Soc., 185–190.

  • Vannitsem, S., 2001: Toward a phase-space cartography of the short- and medium-range predictability of weather regimes. Tellus, 53A , 5673.

    • Search Google Scholar
    • Export Citation
  • Vannitsem, S., and C. Nicolis, 1997: Lyapunov vectors and error growth patterns in a T21L3 quasigeostrophic model. J. Atmos. Sci., 54 , 347361.

    • Search Google Scholar
    • Export Citation
  • Vannitsem, S., and C. Nicolis, 1998: Dynamics of fine-scale versus averaged observables in a T21L3 quasi-geostrophic model. Quart. J. Roy. Meteor. Soc., 124 , 22012226.

    • Search Google Scholar
    • Export Citation
  • Zimin, A. V., I. Szunyogh, D. J. Patil, B. R. Hunt, and E. Ott, 2003: Extracting envelopes of Rossby wave packets. Mon. Wea. Rev., 131 , 10111017.

    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    E dimension (shades) for a 30-pair ensemble and geopotential height for the control (contour lines) at the 50-kPa level for a 24-h forecast started from 0000 UTC on 7 Feb 2000. Contours are shown between 4800 and 5800 gpm, with 100-gpm intervals.

  • View in gallery
    Fig. 2.

    Mean E dimension over the NH midlatitudes (solid lines) and SH mid-latitudes (dotted lines) vs forecast hour for 72-h forecasts started from 0000 UTC on 7 Feb 2000 (no marker), 0000 UTC on 23 Jan 2000 (diamonds), and 0000 UTC on 1 Feb 2000 (circles). A somewhat weaker transient is found in the Tropics (not shown), but there the E dimension is significantly higher (> 16).

  • View in gallery
    Fig. 3.

    E dimension for a 30-pair ensemble (shades) for a 24-h forecast started from 0000 UTC on 7 Feb 2000 using various local regions sizes: (a) 3 × 3, (b) 5 × 5, (c) 7 × 7, and (d) 9 × 9 grid points.

  • View in gallery
    Fig. 4.

    Dependence of the E dimension in the NH midlatitudes on ensemble size. Shown are the maximum and minimum E dimensions averaged over 45 days every 24 h from 0000 UTC on 15 Jan 2000 to 0000 UTC on 28 Feb 2000. Error bars indicate one standard deviation. Also shown is the E dimension for an ensemble of Gaussian random noise.

  • View in gallery
    Fig. 5.

    E dimension in a representative low-dimensional region computed for ensemble sizes 15, 30, 65, 95, 125, and 150 pairs of bred vectors. Note the difference in the color schemes between the upper-left panel and the other panels.

  • View in gallery
    Fig. 6.

    Distribution of the local singular values in areas of low, medium, and high E dimension on a given day (0000 UTC on 9 Feb 2000). (top) First the singular value spectrum (normalized by the largest singular value) was computed for each local region; then the normalized spectra were averaged over the globe. (bottom) The percentage-explained variance obtained by a similar averaging. Error bars indicate the standard deviation over all local regions.

  • View in gallery
    Fig. 7.

    E dimension (color shades) and baroclinic energy conversion (thick contour lines). The contour interval is 300 m2 s−2 day−1, and values smaller than 600 m2 s−2 day−1 are not shown. The geopotential height of the control forecast at the 30-kPa level (thin contour lines) is shown between 8400 and 9200 gpm, with 100-gpm intervals. Results are in the domain 30°−60°N, 120°E−150°W and are shown every 3 h from 0200 UTC on 7 Feb 2000. The low-dimensional region of interest is marked by an arrow.

  • View in gallery
    Fig. 8.

    Time evolution of (top) the average E dimension and (bottom) the average energy conversion terms over a moving 5 × 5 grid point box (see text for further details) starting at 0200 UTC on 7 Feb 2000. Shown are the baroclinic energy conversion (solid line), energy transport (dashed line), divergence of the ageostrophic geopotential flux (dashed–dotted line), and barotropic energy conversion (dotted line), in m2 s−2 day−1.

  • View in gallery
    Fig. 9.

    E dimension (color shades) and barotropic energy conversion (thick contour lines). The contour interval is 150 m2 s−2 day−1, and values between −150 and 150 m2 s−2 day−1 are not shown. The geopotential height of the control forecast at the 30-kPa level (thin contour lines) is shown between 8400 and 9200 gpm, with 100-gpm intervals. Results are in the domain 30°−55°N, 95°−175°E and are shown every 5 h from 0100 UTC on 7 Feb 2000. The low-dimensional region of interest is marked by an arrow.

  • View in gallery
    Fig. 10.

    E dimension (color shades) and divergence of the ageostrophic geopotential flux (orange contour lines) with contour interval 200 m2 s−2 day−1 (values between −400 and 400 m2 s−2 day−1 are not shown), ageostrophic geopotential flux at the 30-kPa level (arrows; values smaller than 10 000 m3 s−3 are not shown), and wave packet envelope (yellow contours) contour interval 1.5 m s−1 (values smaller than 13 m s−1 are not shown).The geopotential height of the control forecast at the 30-kPa level (thin black contour lines) is shown between 8400 and 9200 gpm, with 100-gpm intervals. Results are in the domain 30°−50°N, 100°−155°W and are shown every 7 h from 1100 UTC on 7 Feb 2000.

  • View in gallery
    Fig. 11.

    E dimension (color shades) and wave packet envelope (thick contour lines) for a 48-h forecast lead time started from 0000 UTC on 7 Feb 2000. The contour interval is 2 m s−1, and values smaller than 16 m s−1 are not shown.

  • View in gallery
    Fig. 12.

    E dimension (color shades), barotropic energy conversion (thick black contour lines), divergence of the ageostrophic geopotential flux (white contour lines), and the energy transport (red contour lines in top panels, blue contour lines in bottom), with contour interval 200 m2 s−2 day−1 (values smaller than 400 m2 s−2 day−1 are not shown); the ageostrophic geopotential flux (arrows) at the 30-kPa level (values smaller than 5000 m3 s−3 are not shown); and the geopotential heights of the control forecast at the 30-kPa level (thin black contours) are shown between 8400 and 9200 gpm, with 100-gpm intervals, every 11 h from 0100 UTC on 7 Feb 2000.

  • View in gallery
    Fig. 13.

    E dimension (color shades) and divergence of the ageostrophic geopotential flux (orange contour lines), with contour interval 200 m2 s−2 day−1 (values smaller than 400 m2 s−2 day−1 are not shown); the ageostrophic geopotential flux (arrows; values smaller than 10 000 m3 s−3 are not shown); and the geopotential height of the control forecast at the 50-kPa level (thin black contour lines), with 100-gpm intervals, every 10 h from 0400 UTC on 1 Feb 2000.

  • View in gallery
    Fig. 14.

    E dimension (color shades), baroclinic energy conversion (thick black contour lines) and barotropic energy conversion (orange contour lines), both with contour interval 200 m2 s−2 day−1 (values smaller than 200 m2 s−2 day−1 are not shown), and the geopotential height of the control forecast at the 50-kPa level (thin black contour lines) with 100-gpm intervals, every 10 h from 0400 UTC on 1 Feb 2000.

  • View in gallery
    Fig. 15.

    Dropsonde locations (red crosses) and E dimension (color shades) for the WSR00-targeted observations field program.

  • View in gallery
    Fig. 16.

    Surface pressure forecast improvement in kPa (color shades) and the geopotential height at the 50-kPa level of the original analyses, shown between 4800 and 5850 gpm, with contour intervals of 75 gpm. (top) Targeted observation locations are marked with red crosses and (bottom) blue curves represent the 48-h verification regions. (left) The localized E dimension (thick black contour lines) with contour intervals of (top) 0.5 and (middle), (bottom) 0.4; high values are not shown. (right) The global E dimension (thick black contour lines) with contour intervals of (top) 0.5 and (middle), (bottom) 0.4; high values are not shown.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 130 55 8
PDF Downloads 88 44 2

Mechanisms for the Development of Locally Low-Dimensional Atmospheric Dynamics

Michael OczkowskiDepartment of Physics and Astronomy, Francis Marion University, Florence, South Carolina

Search for other papers by Michael Oczkowski in
Current site
Google Scholar
PubMed
Close
,
Istvan SzunyoghDepartment of Meteorology, and Institute for Physical Science and Technology, University of Maryland, College Park, College Park, Maryland

Search for other papers by Istvan Szunyogh in
Current site
Google Scholar
PubMed
Close
, and
D. J. PatilInstitute for Physical Science and Technology, University of Maryland, College Park, College Park, Maryland

Search for other papers by D. J. Patil in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

The complexity of atmospheric instabilities is investigated by a combination of numerical experiments and diagnostic tools that do not require the assumption of linear error dynamics. These tools include the well-established analysis of the local energetics of the atmospheric flow and the recently introduced ensemble dimension (E dimension). The E dimension is a local measure that varies in both space and time and quantifies the distribution of the variance between phase space directions for an ensemble of nonlinear model solutions over a geographically localized region. The E dimension is maximal, that is, equal to the number of ensemble members (k), when the variance is equally distributed between k phase space directions. The more unevenly distributed the variance, the lower the E dimension.

Numerical experiments with the state-of-the-art operational Global Forecast System (GFS) of the National Centers for Environmental Prediction (NCEP) at a reduced resolution are carried out to investigate the spatiotemporal evolution of the E dimension. This evolution is characterized by an initial transient phase in which coherent regions of low dimensionality develop through a rapid local decay of the E dimension. The typical duration of the transient is between 12 and 48 h depending on the flow; after the initial transient, the E dimension gradually increases with time.

The main goal of this study is to identify processes that contribute to transient local low-dimensional behavior. Case studies are presented to show that local baroclinic and barotropic instabilities, downstream development of upper-tropospheric wave packets, phase shifts of finite amplitude waves, anticyclonic wave breaking, and some combinations of these processes can all play crucial roles in lowering the E dimension.

The practical implication of the results is that a wide range of synoptic-scale weather events may exist whose prediction can be significantly improved in the short and early medium range by enhancing the prediction of only a few local phase space directions. This potential is demonstrated by a reexamination of the targeted weather observations missions from the 2000 Winter Storm Reconnaissance (WSR00) program.

Corresponding author address: Istvan Szunyogh, Institute for Physical Science and Technology, University of Maryland, College Park, College Park, MD 20742-2431. Email: szunyogh@ipst.umd.edu

Abstract

The complexity of atmospheric instabilities is investigated by a combination of numerical experiments and diagnostic tools that do not require the assumption of linear error dynamics. These tools include the well-established analysis of the local energetics of the atmospheric flow and the recently introduced ensemble dimension (E dimension). The E dimension is a local measure that varies in both space and time and quantifies the distribution of the variance between phase space directions for an ensemble of nonlinear model solutions over a geographically localized region. The E dimension is maximal, that is, equal to the number of ensemble members (k), when the variance is equally distributed between k phase space directions. The more unevenly distributed the variance, the lower the E dimension.

Numerical experiments with the state-of-the-art operational Global Forecast System (GFS) of the National Centers for Environmental Prediction (NCEP) at a reduced resolution are carried out to investigate the spatiotemporal evolution of the E dimension. This evolution is characterized by an initial transient phase in which coherent regions of low dimensionality develop through a rapid local decay of the E dimension. The typical duration of the transient is between 12 and 48 h depending on the flow; after the initial transient, the E dimension gradually increases with time.

The main goal of this study is to identify processes that contribute to transient local low-dimensional behavior. Case studies are presented to show that local baroclinic and barotropic instabilities, downstream development of upper-tropospheric wave packets, phase shifts of finite amplitude waves, anticyclonic wave breaking, and some combinations of these processes can all play crucial roles in lowering the E dimension.

The practical implication of the results is that a wide range of synoptic-scale weather events may exist whose prediction can be significantly improved in the short and early medium range by enhancing the prediction of only a few local phase space directions. This potential is demonstrated by a reexamination of the targeted weather observations missions from the 2000 Winter Storm Reconnaissance (WSR00) program.

Corresponding author address: Istvan Szunyogh, Institute for Physical Science and Technology, University of Maryland, College Park, College Park, MD 20742-2431. Email: szunyogh@ipst.umd.edu

1. Introduction

The complexity of a nonlinear dynamical system is typically characterized by estimating the dimensionality of its dynamics (e.g., Ott 2002; Alligood et al. 1997). In dynamical systems theory, the most frequently used estimate, which has been successfully applied to low-dimensional analogues of the atmosphere (Vannitsem and Nicolis 1997; Lorenz and Emanuel 1998), is the Lyapunov or Kaplan–Yorke (1979) dimension. However, for a high-dimensional, spatiotemporal dynamical system, such as a realistic dynamical model of the atmosphere, computation of the Lyapunov dimension is not feasible. Thus, quantifying the dimensionality of atmospheric dynamics requires a different approach.

A dimension definition that was specifically designed for large degree-of-freedom, spatiotemporally chaotic dynamical systems, such as the weather prediction models, is the Bred Vector Dimension (BV dimension) introduced by Patil et al. (2001). This measure estimates the dimensionality based on a principal component analysis of an ensemble of model forecasts over localized geographical regions. A critical distinction between the BV-dimension and traditional definitions of dimensions that are global and independent of space and time is that the BV-dimension is a local quantity that varies in both space and time. We note that this local measure is called the BV-dimension only for historical reasons, as it was first applied to an ensemble of bred vectors. Since it can be used for any ensemble of model solutions, regardless of the ensemble generation technique, we prefer to call it the ensemble dimension (E dimension).

The main goal of this paper is to identify processes of the atmospheric dynamics that can lead to extremely low-dimensional behavior in highly localized geographical regions. This low-dimensional behavior, first observed by Patil et al. (2001), is a transient phenomenon that occurs in the first few days of the evolution of the ensemble. The motivation for this project is more than scientific curiosity. Several important studies (and also some operationally implemented techniques) are based on the implicit assumption that errors in the shortand early medium-range (1–4 days) forecasts evolve in a low-dimensional space. For instance, the general success of the now operational targeted weather observation programs (Szunyogh et al. 2000, 2002b; Toth et al. 2001, 2002) is a strong indication that low-dimensional regions may play a crucial role in the predictability of localized severe weather events such as extratropical winter storms. These operational programs use a small (10–25 member) ensemble of numerical weather forecasts to identify the optimal geographical region (of about 500–1000-km radius) for collecting a small number (10–20) of extra dropsonde observations with the goal of improving the prediction of a particular forecast event (Bishop and Toth 1999; Bishop et al. 2001; Majumdar et al. 2001). This strategy would not be effective if local low-dimensionality did not exist.

The notion that local low-dimensionality can play an important role in atmospheric predictability is not new. For instance, the leading (right) singular vectors of the matrix that represents the tangent linear map of the nonlinear trajectories of the atmospheric models have a long history of use in studying atmospheric instabilities (e.g., Farrell 1985, 1989). The singular vectors of a model based on the primitive equations are geographically localized structures, and only a few of them occur in the same region at a given time (e.g., Buizza et al. 1993; Buizza and Palmer 1995). The success of using singular vectors or the closely related sensitivity gradient (e.g., Rabier et al. 1992; Errico et al. 1993) in operational ensemble forecasting (Molteni et al. 1996), finding the origin of unusually large localized forecast errors (e.g., Rabier et al. 1996), and determining the optimal locations of targeted weather observations (e.g., Bergot et al. 1999; Gelaro et al. 1999; Palmer et al. 1998; Leutbecher 2003) provides further evidence that the local dimensionality of the atmospheric flow can be low. Furthermore, Vannitsem and Nicolis (1998) and Vannitsem (2001) argued that the dominant Lyapunov vector (a single perturbation pattern) can efficiently characterize the short-term predictability of the quasigeostrophic flow in the Northern Hemisphere midlatitudes.

The study presented here is based on diagnostic tools that do not require theassumption of linear error dynamics. In addition to the E dimension, our diagnostics include wave packet analysis based on the Hilbert transform (Zimin et al. 2003) and the eddy kinetic energy balance equation (Orlanski and Katzfey 1991; Orlanski and Chang 1993; Chang 2000). These tools allow us to identify processes in which complex interactions between atmospheric instabilities play an important role and lead to the development of local low-dimensionality in nontrivial ways. Our results indicate that in a low-dimensional region, the dominant uncertainties in the state estimate may remain confined to a low-dimensional vector space for several days. Furthermore, there exists a wide variety of dynamical processes at the synoptic and large scales that can lead to the development of coherent low-dimensional regions with a typical life span from 1 to 3 days long. On the practical side, these results imply that improving the analysis in a low-dimensional region also improves the forecasts at least as long as the low-dimensionality prevails. The potential for such forecast improvements can be exploited by improving the analysis techniques, targeting weather observations, and/or optimizing the fixed global observing network (Bishop et al. 2003).

We detail our study in the following format: Section 2 gives a short description of the E dimension, section 3 explains its application to a state-of-the-art global weather prediction model [the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS)], and section 4 provides examples for the development of local low-dimensionality in the atmosphere. These examples show a number of scenarios involving such atmospheric processes as local baroclinic andbarotropic instabilities, downstream development (e.g., Orlanski and Chang 1993), uncertainty in the phase of finite amplitude waves generated by earlier instabilities (Snyder 1999), and anticyclonic wave breaking in the exit region of the Pacific jet (Orlanski 2003). The latter example is especially interesting, since strong low-dimensionality in the region of the breaking wave can explain an unexpected result of the 2000 Winter Storm Reconnaissance  targeted observation program (Szunyogh et al. 2002b): the largest forecast improvement during the field program was a much-improved prediction of an anticyclonic wave breaking (also analyzed in this paper). Finally, our conclusions are summarized in section 5.

2. E dimension

The E dimension measures the effective dimension spanned by a k-member set of ensemble perturbations in a local geographical region at a given time. The E dimension can characterize the effective number of dominant directions in the vector space spanned by the ensemble perturbations by giving more weight to directions that explain larger portions of the total ensemble variance. This is a unique feature of the E dimension, since the most frequently used measure, the ε rank (e.g., Golub and Van Loan 1989), gives equal weight to all directions that explain more variance than an arbitrarily chosen small ε.

In what follows, we first outline the procedure for computing the E dimension and then provide a more detailed description:

  1. The ensemble perturbations are defined as the difference between the ensemble members and the ensemble mean.

  2. At each grid point, a local region is defined that includes all grid points in a local neighborhood.

  3. For each local region and for each ensemble member, a local vector is formed consisting of all dynamical variables of the ensemble perturbation within the local region.

  4. The local vectors in each local region are combined to form a matrix whose columns contain the local vectors for all ensemble members.

  5. A singular value decomposition is performed on this matrix, resulting in a set of singular values.

  6. The E dimension is computed as a statistic on the singular values [see Eq. (1) below].

a. Definitions and basic properties

The k local vectors form the columns of a V × k matrix, 𝗕, where V is the number of components (grid point variables) in each local region. The k × k covariance matrix of 𝗕 is 𝗖 = 𝗕T 𝗕, where 𝗕Tis the transpose of 𝗕. Since the covariance matrix is nonnegative definite and symmetric, its k eigen values λi, i = 1 . . . k, are nonnegative (λi ≥ 0), and its eigenvectors, after multiplying by 𝗕 and normalizing, form an orthonormal set of vectors υi that span the column space of B. The singular values of 𝗕 are σi = λi, and we order them such that σ1σ2 ≥ . . . ≥ σk. The total variance in the ensemble of k local vectors is equal to the sum of σ2i = λi over all k singular values, since σ2i is the variance in the direction vi. (The directions vi, i = 1, 2, . . . , k represent orthogonal patterns in the ensemble of perturbations.)

By determining the number of directions that represent most of the variance in the ensemble of perturbations, we can define an effective dimension spanned by the k local vectors. For example, if two out of five singular values were zero and the remaining three were all equal, then the subspace spanned by the k local vectors would be three-dimensional. However, the smallest singular values of 𝗕 (and any matrix based on experimental data) are typically nonzero, though some of them can be extremely small. One option is to use thresholding, whereby the dimension is defined by either the number of singular values that are larger than ε (hereafter referred to as dε) or the number of principal components that explain 1 − ε percent of the variance in the ensemble (hereafter referred to as d). These measures equally weight each phase space direction represented by the nonzero singular values, regardless of the variance explained by that direction. This approach is not suitable for us, since we are looking for a measure that gives larger weight to the directions that explain a larger portion of the variance. Instead, we use the E dimension,
i1520-0469-62-4-1135-e1
which returns a (typically) noninteger value between 1 and k. The E dimension is a continuous function of the singular values that yields the value N when each vector explains 100/N percent of the variance of the ensemble. It is the simplest function we could construct that meets that criterion.

The response of the E dimension to changes in the distribution of the variance amongst the various directions of the phase space is illustrated by several simple cases in the appendix. We emphasize three key features: 1) if the ensemble consists of small perturbations purely due to noise or numerical error, the E dimension will be near k, representing equal distribution of the variance; 2) when dynamical instabilities lead to the development of dominant directions of uncertainty, the E dimension becomes substantially lower and insensitive to small changes in the singular values (σi) due to noise or numerical error; and 3) the E dimension does not depend on the total ensemble variance (spread), that is, an increasing ensemble spread does not change the E dimension unless the increase of the variance is unevenly distributed between the different phase space directions vi, i = 1, 2, . . . , k (e.g., when the variance is growing in a single phase space direction).

Low dimensionality is a property that eventually breaks down with increasing forecast time. This occurs as a result of the increasing number of active degrees of freedom that contribute to the local error dynamics. While improving forecasts in a low-dimensional region requires the better prediction of only a few important degrees of freedom, reducing the errors in a high-dimensional region requires improved prediction of many degrees of freedom. This, we believe, makes reducing the forecast errors increasingly harder as dimensionality increases. The case when high dimensionality is due to the complexity of the error dynamics, and not due to noise, can be recognized by the larger amplitude of the ensemble perturbations (Patil et al. 2001).

We note that potential applications of a statistic formally identical to Eq. (1) were discussed by Bretherton et al. (1999). They considered its use as a measure “of the effective number of spatial degrees of freedom (ESDOF), or number of independently varying spatial patterns, of a time-varying field of data.” We use the formula as a spatiotemporally varying measure of the effective number of spatial degrees of freedom in a set of ensemble perturbations. Our strategy is motivated by the belief that high-dimensionality is a fundamental property of the atmospheric flow; thus searching for a limited number of independently varying global spatial patterns is not practical when synoptic- and smaller-scale motions are also considered.

3. Application to the NCEP global model

We investigate the relationship between atmospheric dynamics and the E dimension using a T-62, 28-level version of the operational NCEP GFS. For each case investigated here, our dataset consists of 1) a control forecast started from an operational NCEP analysis and 2) a 30-pair ensemble of continuously evolved forecasts. The ensemble initial conditions were generated by a slightly modified version of the operationally implemented version of the breeding technique described in Iyengar et al. (1996) and Toth and Kalnay (1997). The modifications, which included increasing the rescaling frequency from 24 to 6 h and reducing the magnitude of the initial perturbations by a factor of 2, were introduced to better represent the magnitude and probability distribution of the analysis uncertainty (Szunyogh et al. 2002a). We emphasize that the breeding algorithm was used only to generate the initial ensemble perturbations, and the computations of the E dimension are based on continuously evolved ensemble members.

When calculating the E dimension, the elements of the local vectors are calculated using an energy rescaling in order to transform the different physical variables to comparable quantities of the same physical dimension. The energy rescaling is defined so that the square of the Euclidian norm of the vector formed by the different variables has energy dimension. The transformations are defined by
i1520-0469-62-4-1135-e2
where Ul, Vl, and Tl are, respectively, the perturbations from the main solution of the zonal and meriodinal wind velocities and temperatures at level l; Ps is the surface pressure; Pl is the pressure at level l; Dl = Pl+1Pl/Ps; Cp = 1004 J K−1 kg−1; Tr = 273 K; and Rd = 287 J mol−1 K−1. The total energy norm of Eq. (2) was first derived by Talagrand (1981) and studied in detail by Errico (2000). We note that this quantity is also used at the European Centre for Medium-Range Weather Forecasts (ECMWF) to generate initial ensemble perturbations (Buizza et al. 1993). The transformed variables are defined by UlDl,VlDl, TlCp/TrDl, and RdTr ln(Ps)

For the calculations of the E dimension that follow, we choose a box size of 5 × 5 grid points, so that it covers an area of about 1100 km × 1100 km in the midlatitudes. Since the model variables are output at 21 vertical levels (100, 97.5, 95, 92.5, 90, 85, 80, 75, 70, 65, 60, 55, 50, 45, 40, 35, 30, 25, 20, 15, and 10 kPa), each local vector is 5 × 5 × (3 × 20 + 1) = 1525 dimensional, and the matrix 𝗕 has size 1525 × 30. Note that we use positive/negative (p+/p) pairs of perturbations to define the local vectors by ½(p+p).

Figure 1 shows a typical map of the E dimension in the Northern Hemisphere for a 30-pair ensemble of 24-h forecasts. In this figure, and throughout the paper, the E dimension is colored with “hot” colors for low values and “cool” colors for high values. Note that many of the lowest-dimensional areas correspond to troughs, ridges, and waves in the atmospheric flow. This observation motivates us to examine the relationship between local atmospheric instabilities and local low-dimensionality.

Figure 2 illustrates the transient nature of local low-dimensionality. The areal mean of the E dimension decreases only in the initial phase of model integration, reaching a minimum, between 12 and 48 h depending on the atmospheric flow; then it gradually increases with time. The difference between the evolution of the E dimension on the different days is considerable in both hemispheres.

a. Dependence on the local region size

Our goal is to capture changes in the complexity of the atmospheric state at the synoptic scales. Since the scale of interest is only loosely defined, it is important to show that the low-dimensional regions are not sensitive to the choice of the local region size within a reasonable range. Figure 3 shows the E dimension for different local region sizes between about 600 km by 600 km (3 by 3 grid points) and 2000 km by 2000 km (9 by 9 grid points). We find that the results are only weakly sensitive to the size of the region. Although the selection of a larger local region size slightly increases the E dimension, the spatial extent of the low-dimensional regions and the locations of the low-dimensional regions are not affected by the choice of the local region size.

b. Dependence on the number of ensemble members

When the ensemble size is increased, the E dimension is also expected to increase until the number of ensemble members becomes sufficiently large to capture all dynamically active phase space directions in the local region. Figure 4 shows the time average of the minimum and maximum E dimension in a time series of maps for an increasing number of ensemble members. While the dimension scales linearly with the ensemble size for pure noise, both the maximum and minimum start saturating at relatively small ensemble sizes for the bred vectors. The saturation is much faster for the minimum than for the maximum and the 30-member ensemble, the size selected for our further investigations, provides a good approximation of the minimum dimension. This feature is also well illustrated by Fig. 5, which shows the effect of the ensemble size on a selected low-dimensional region. Although the value of the E dimension is somewhat higher for the larger ensembles, the location of the low-dimensional region is not affected by the ensemble size. Figure 4 also shows that a 30-member ensemble is not sufficient to estimate the E dimension in the higher-dimensional regions.

The behavior of the low-dimensional regions can be explained by the unique shape of the singular value spectrum that characterizes them. Figure 6 shows the normalized singular value spectra averaged over all low, middle, and high (≤ 6, 6−12, and ≥ 12, respectively) E dimensional regions for an ensemble of bred vectors on a typical day (upper panel). The steep initial decay of the spectra for the low-dimensional regions shows that the ensemble variance is primarily contained in a few directions of the local phase space. Even more striking is the result for the fraction-explainedensemble variance, defined by
i1520-0469-62-4-1135-e3
for the same regions (lower panel). To explain 95% of the variance (f = 0.95), the required number of singular values for low, medium, and high E dimensional regions are, respectively, 4, 8, and 15. Thus for the low-dimensional regions, only 4 out of a maximum 30 directions in the phase space are required to explain nearly all of the ensemble variance.

c. Local energetics

To identify the various mechanisms that affect the evolution of the atmospheric flow, all variables of the control forecast are partitioned into time mean and transient (eddy) components. The time mean is computed using a 1-month moving mean of the four daily operational analyses of NCEP, with the moving mean centered at the beginning of the forecast time of interest. The transient eddy components are determined by the deviation of the actual forecast values from their moving means. These transient components are then used to compute the terms in the local eddy kinetic energy balance equations (Orlanski and Katzfey 1991).

These energy tendencies have been extensively studied by Chang (2000) in relation to baroclinic life cycles, and we will use them to identify the energy transfer processes that lead to locally low-dimensional behavior. The local energetics are computed as vertically averaged quantities in pressure coordinates resulting in the equation
i1520-0469-62-4-1135-e4
where pb is the surface pressure (typically 100 kPa), pt is the pressure at the top of the vertical column (typically 10 kPa), and [ ] denotes a surface integral taken either on the top (t) or bottom (b) pressure surface. Lowercase terms and those with a subscript e indicate eddy components; a subscript m and an overbar indicate a time mean component; and a subscript a indicates an ageostrophic component. All vectors in bold are two-dimensional unless they have a subscript 3, in which case they are three-dimensional.

We choose to neglect both the surface fluxes (which are small) and the residue terms in our analysis, since the focus of this work is not a detailed study of the eddy kinetic energy balance. The remaining terms are the energy transport (advection of the eddy kinetic energy), divergence of the ageostrophic geopotential flux, baroclinic energy conversion, and barotropic energy conversion, respectively. Our experience is that the sum of these terms tends to overestimate the growth of eddy kinetic energy found by direct calculation, consistent with the observation (Chang 2000) that the residue term is dissipative in nature and thus is primarily negative.

Finally, we note that one has to be careful when interpreting the diagnostics based on the local energetics. Snyder (1999) showed a series of intriguing examples in which the perturbation growth was associated with a growing uncertainty in the phase of a propagating finite amplitude wave or coherent structure. Although this mechanism is very different from the idealized baroclinic and barotropic instabilities, in which an infinitesimal initial perturbation to a steady parallel flow starts growing, it is always associated with barotropic and baroclinic energy conversions. Since our eddies (defined by the deviation from the time mean flow) frequently have finite amplitudes that generate low-dimensional regions associated with differences in the phase of the waves, the mechanism described by Snyder (1999) certainly plays an important role in our examples.

4. Examples of locally low-dimensional behavior

a. Example 1: Pure baroclinic instability

The initial evolution of the atmospheric flow on 7 February 2000 at 30°–45°N, 120°W–180° shows a nearly textbook example of pure local baroclinic instability. A strongly localized region of baroclinic energy conversion (Fig. 7) has developed just to the east of Japan, at the entrance to the Pacific storm track. The center of the baroclinic energy conversion in the vicinity of a weak trough coincides with a developing low-dimensional region. As time evolves, we see that the peak of the baroclinic energy conversion and the local minimum E dimension continue to propagate eastward together at a speed of about 15° day−1. In addition, the trough deepens significantly as a result of the strong baroclinic energy conversion, with the vertically averaged eddy kinetic energy growing by over 220% during this time.

We can also examine the local energy conversion terms by computing them in a moving box centered on a local minimum of the E dimension. This box and the box used for computing the E dimension have the same size: 5 by 5 grid points. In Fig. 8, we follow such a box centered on the local minimum of the E dimension and compute the average local energy conversion per grid point. Note that since the box is moving, additional terms would be required for the closure of the local energy equations (Chang 2000), but we opt to neglect those terms since, unlike the case in Chang (2000), our box is not moving at a constant speed, making computation of the extra terms impractical. We find that the baroclinic energy conversion term, already by far the dominant term at hour 2, continues growing very rapidly until hour 12, after which it remains at a large value until a sharp decay at hour 20. Concurrently, the average E dimension in this local area drops continuously from 9.6 to 7 (Fig. 8). Note that the sharp decrease in the baroclinic energy conversion at 20-h forecast lead time is partially a result of the chosen box slowing down relative to the peak of the energy conversion. Thus, while there is a small drop over the entirety of the low-dimensional area, the box itself shows a large decrease in the baroclinic energy conversion. This example illustrates that a simple local baroclinic energy conversion process can generate locally low-dimensional behavior, even in a complex weather prediction model based on the primitive equations and using a state-of-the-art physical parameterization package.

b. Example 2: Upper-tropospheric wave packet in the entrance to the Pacific storm track

Our second example occurred in a time window that overlaps with the one used in the previous example. For this example, however, we focus on an area located farther to the west, centered at about 48°N, 112°E. This is the region where upper-tropospheric wave packets propagating through Asia reach the entrance of the Pacific storm track (Chang and Yu 1999). In our case, waves propagating along the northern Asian waveguide (through Siberia) shape the flow of the selected area.

The propagation of the wave packets is visualized by following the wave packet envelopes. To compute the envelopes, we use a method based on the Hilbert transform, presented in Zimin et al. (2003), which is more robust than the method of complex demodulation most frequently used in atmospheric studies (Lee and Held 1993). In the algorithm based on the Hilbert transform, 1) a one-dimensional signal is transformed into Fourier space; 2) the Fourier coefficients in a window of the relevant positive wave numbers are retained, while 3) the coefficients associated with the remaining positive wave numbers and all negative ones are set to zero; and 4) the inverse Fourier transform is applied to the filtered Fourier spectrum, and the envelope is defined as twice the modulus of the resulting signal.

In the initial stage, barotropic energy conversion leads to the development of a secondary low-dimensional area east of the entrance to the Pacific storm track (Fig. 9). Here, the E dimension becomes low where the barotropic energy conversion has a local peak. As the low-dimensional region moves into the Pacific storm track region, the lowest-dimensional spot begins to extend into regions east of the peak of barotropic energy conversion. This is due to a combination of the transfer of eddy kinetic energy (wave packet propagation) and the divergence of the ageostrophic geopotential flux becoming the dominant factor in generating eddy kinetic energy and reducing the local dimension (Fig. 10). Here, the picture of a process emerges, in which the low-dimensional region extends to the east with the wave packet (transfer of eddy kinetic energy) while the additional effect of the divergence of ageostrophic fluxes determines the location of the absolute local minimum of the dimension. Although this process starts much earlier than hour 21, that is the time when it becomes evident, leading to a rapid decrease of the E dimension in the area, where the center of the wave packet overlaps with the divergence of the ageostrophic geopotential fluxes. The decrease of the dimension is even more remarkable if we consider that, in our example, the initially positive barotropic energy conversion term becomes negative in the region of decreasing dimension.

Another interesting feature of this case is that the newly developed low-dimensional region merges into the low-dimensional region developed earlier by the pure baroclinic energy conversion (studied in our first example). The result is a low-dimensional region that occupies a large area in the North Pacific and dominates the atmospheric flow even at such long forecast times as 56 h after the time when the secondary minimum started developing (Fig. 11). This can occur because the eastward propagation of the waves toward the North American continent is blocked by the large-scale flow. This leads to the development of a quasi-static wave pattern in the absence of processes that could remove the eddy kinetic energy accumulated earlier. This is an example in which the large-scale flow plays an implicit role in the development of low-dimensional atmospheric structures.

c. Example 3: Upper-tropospheric wave packet propagation in the Southern Hemisphere summer

The Southern Hemisphere storm track is the simplest among all storm tracks: it is essentially parallel to the latitude circles, and the wave amplitudes are zonal and nearly symmetric (Chang 1999). We found that this flow configuration is extremely favorable for the development of large, long-lived regions of very low E dimension. Here we present an example for the evolution of such a low-dimensional area (Fig. 12). This example is similar to example 2 in that the same three terms—ageostrophic geopotential fluxes, eddy kinetic energy transport, and barotropic energy conversion—play dominant roles. The barotropic energy conversion in this case is a nearly stationary feature centered at about 55°S, 165°E at the beginning of the period depicted in Fig. 12. This persistent energy conversion process is the result of a continuous propagation of eddy kinetic energy through a deformation zone associated with a quasi-stationary long wave. The eddy kinetic energy propagates from the upstream edge of the long wave, where the transport of eddy kinetic energy and the divergence of ageostrophic geopotential fluxes lead to the initial development of a weak trough and the associated local minimum of the E dimension. As time evolves, the combined effects of the continuous downstream propagation of eddy kinetic energy and steady barotropic energy conversion lead to the development of a new trough and an associated low-dimensional region at the downstream edge of the long-wave pattern.

d. Example 4: Anticyclonic wave breaking in the exit region of the Pacific storm track

This example is the evolution of the atmospheric flow in the northeast Pacific starting from 0000 UTC on 1 February 2000 (Figs. 13 and 14). The main focus is on the low-dimensional regions associated with a pair of cyclonic waves with an eddy anticyclonic center at about 40°N, 160°W at hour 4. This flow configuration is somewhat reminiscent of the Anticyclonic Vortex Control (AVC) of Orlanski (2003), his Fig. 10a), in which the anticyclonic wave breaking is due to a modest baroclinic forcing. There are important differences, however, between the idealized AVC and our example. Most importantly, in our case the background flow is not zonal, and the two cyclonic vortices are in very different stages of their evolutions.

On the one hand, the western cyclonic wave is a classic case of downstream baroclinic development, in which eddy kinetic energy is first generated only by the divergence of the ageostrophic geopotential fluxes (at hour 4; Fig. 13), triggering baroclinic energy conversion (at hour 14; Fig. 14), which later becomes the sole source of eddy kinetic energy (at hour 34; Fig. 14). These energy conversion processes generate local low-dimensional behavior.

On the other hand, the development of the eastern cyclonic wave is governed by the combination of a strong barotropic energy conversion and a weak divergence of the ageostrophic geopotential fluxes. As time evolves, the energy conversion by the transient features weakens, and the main axis of the low-dimensional region becomes aligned with the axis of the deepening large-scale trough. During this process, the dimension significantly decreases. A comparison of the time-evolving flow with the 1-month time mean shows that they become increasingly similar to each other (especially on the eastern side of the trough). This explains the diminishing role of the transient features and also shows that synoptic-scale transient features can generate locally low-dimensional behavior in the slowly varying large-scale components of the flow.

e. Example 5: Targeted observations

A comparison of maps depicting the E dimension with results from the WSR00 targeted weather observations program revealed a strikingly close relationship between local low-dimensionality and the synoptic features sampled by observations during WSR00. Nearly all locations chosen for enhanced sampling were in regions of low-dimensionality (Fig. 15). (On one occasion, 0000 UTC on 13 February 2000, the local region was beyond the reach of the reconnaissance plane deployed in Hawaii.) This finding shows that the ensemble transform Kalman filter (ETKF; Bishop et al. 2001; Majumdar et al. 2001), which was used to determine the optimal dropsonde locations during WSR00, tends to select regions of local low-dimensionality. The fact that targeted data were always collected in regions of important synoptic-scale features and that these data led to a substantial overall forecast improvement (Szunyogh et al. 2002b) supports the view that regions of local low-dimensionality tend to play an important role in shaping the weather.

We select one of the WSR00 flight missions for further examination. The region of interest and the starting date are the same as in example 4, but this time we follow the evolution of the flow for a longer time (48 h). Targeted observations for this case were collected during a ferry flight from northern California to Hawaii. The changes in the forecast quality, shown by color shades in Fig. 16, are estimated by comparing two forecasts with the NCEP GFS to the operational NCEP analyses. One of the forecasts is initiated by assimilating all observations (including the targeted observations), while the other is initiated by assimilating all but the targeted observations. The forecast improvement due to the targeted data is defined by the difference between the absolute values of the surface pressure errors in the two forecasts.

On the initial leg of the flight, observed data were collected in a region of negative and then positive barotropic energy conversion (comparing the upper-most right panel in Figs. 14 and 16). Near the point where the flight track turned from the northeast–southwest to south direction, the dropsondes sampled a region of baroclinic energy conversion (upper-most left panels in Figs. 14 and 16). The relationship between forecast improvements and the regions of local low-dimensionality is apparent (Fig. 16). In this figure, the right-hand side panels are based on the global ensemble perturbations (used in our previous four examples), while the left-hand side panels are based on ensemble perturbations initially confined to the targeted region. These local initial perturbations were created by setting the perturbation amplitudes to zero outside the targeted regions in the global initial perturbations. To avoid nonphysical effects at the localization boundaries, the perturbations were smoothed by a distance-dependent Gaussian filter (provided by J. Purser). Finally, a new forecast ensemble was recreated by running forecasts from the localized initial perturbations.

It can be seen by comparing the right- and left-hand side panels of Fig. 16 that the locally low-dimensional behavior in the prediction of the trough is mainly due to the locally low-dimensional behavior of the targeted region. Although the E dimensions are somewhat larger for the global initial perturbations, indicating that initial perturbations from outside the targeted region tend to increase the dimension, the locations of the lowest-dimensional regions are the same in the two ensembles. This demonstrates that a large portion of the dominant forecast error patterns can evolve confined to locally low-dimensional subspaces of the state space, even for seemingly complicated forecast situations that involve complex interactions between transient features and the large-scale flow.

5. Conclusions

This paper is the first application of the E dimension, introduced recently by Patil et al. (2001), to analyze the local complexity of atmospheric processes. We presented a number of scenarios for the development of locally low E dimensional atmospheric dynamics. These scenarios included pure baroclinic instabilities, complex processes involving baroclinic and barotropic instabilities, the divergence of ageostrophic geopotential fluxes, upper-tropospheric wave packet propagation, and interaction between transient and slowly varying components of the atmospheric flow. While local low-dimensionality is usually the result of complex processes, on most occasions, the decrease of the E dimension is closely related to a single term of the eddy kinetic energy equation. When the largest term on the right-hand side of Eq. (4) is larger than 40 m2 s2 h1, the E dimension decreases in 68% of all cases, while for a threshold of 60 m2 s2 h1 the E dimension decreases in 77% of the cases and for a threshold of 80 m2 s2 h1 the E dimension decreases in 85% of the cases.

In this study, the E dimension was always applied to nonlinearly evolving finite amplitude ensemble perturbations. For these finite amplitude ensembles local low-dimensionality is a transient property. One may speculate, however, about the possibility of computing the E dimension for linearly evolving infinitesimal perturbations. In this case, the E dimension would characterize the dimensionality of the spaces tangent to the manifold upon which the atmospheric state evolves, and similarly to the Lyapunov (Kaplan–Yorke 1979) dimension, it would measure an inherent property of the atmospheric (model) dynamics. Although our perturbations are not infinitesimal, their initial amplitudes are typically small, and we speculate that the low-dimensional behavior we observe may initially reflect local low-dimensionality of the manifold upon which the state evolves. Our main result is the nontrivial finding that local low-dimensionality of the error dynamics can prevail for an extended time, even if the amplitude of the perturbations is finite and complex nonlinear processes are involved.

The practical implication of our results is that there may be a wide range of weather events whose prediction can be enhanced by improving the initial conditions. This potential can be exploited by simply improving the initial conditions in the region of the primary instability. This can be achieved by using linear techniques to find the location where improvement in the initial conditions is most needed. The successful application of linear techniques to find the location of targeted weather observations, mentioned in the introduction, confirms this conclusion. A technique that can find the relationships between a chain of low-dimensional events may allow us to target the forecast effects of observations for even longer forecast times. The ETKF technique, based on the analysis of an ensemble of nonlinear forecasts, may well be just such a scheme. Exploring the relationship between ETKF results and the E dimension will be a subject of our future research.

Finally, we note that ensemble Kalman filter data assimilation schemes are expected to have an advantage over traditional optimal interpolation (OI) and three-dimensional variational data assimilation (3DVAR) schemes in estimating the atmospheric state in the low-dimensional regions. This belief is one of our main motivations to develop the local ensemble Kalman filter (LEKF) data assimilation scheme (Ott et al. 2004) that seeks the best estimate of the atmospheric state in local low-dimensional regions. Preliminary results with the LEKF, implemented on the same version of the NCEP GFS that is used in the present study, are very promising; under the perfect model scenario, a 40-member ensemble can accurately track the state of the model (Szunyogh et al. 2004), and the state estimates provided by the LEKF are particularly accurate in the midlatitudes where local low dimensionality occurs most frequently.

Acknowledgments

The authors thank J. Purser for his Gaussian filter; our colleagues, B. Hunt, E. Ott, J. Yorke, E. Kalnay, and A. Zimin, for useful discussions; Hugh van den Dool for calling the paper Bretherton et al. (1999) to our attention; and S. Vannitsem and one anonymous referee for their insightful comments. This work was supported by a James S. McDonnell 21st Century Research Award, the Office of Naval Research (Physics), the Army Research Office, and by a THORPEX grant from the National Oceanic and Atmospheric Administration.

REFERENCES

  • Alligood, K., T. Sauer, and J. Yorke, 1997: Chaos: An Introduction to Dynamical Systems. Springer-Verlag, 603 pp.

  • Bergot, T., G. Hello, and A. Joly, 1999: Adaptive observations: A feasibility study. Mon. Wea. Rev., 127 , 743765.

  • Bishop, C. H., and Z. Toth, 1999: Ensemble transformation and adaptive observations. J. Atmos. Sci., 56 , 17481765.

  • Bishop, C. H., B. J. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Wea. Rev., 129 , 420436.

    • Search Google Scholar
    • Export Citation
  • Bishop, C. H., C. A. Reynolds, and H. K. Tippett, 2003: Optimization of the fixed global observing network in a simple model. J. Atmos. Sci., 60 , 14711489.

    • Search Google Scholar
    • Export Citation
  • Bretherton, C. S., M. Widmann, V. P. Dymnikov, J. M. Wallace, and I. Bladé, 1999: The effective number of spatial degrees of freedom of a time-varying field. J. Climate, 12 , 19902009.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., and T. N. Palmer, 1995: The singular-vector structure of the atmospheric general circulation. J. Atmos. Sci., 52 , 14341456.

    • Search Google Scholar
    • Export Citation
  • Buizza, R., J. Tribbia, F. Molteni, and T. N. Palmer, 1993: Computation of optimal unstable structures for a numerical weather prediction model. Tellus, 45A , 388407.

    • Search Google Scholar
    • Export Citation
  • Chang, E. K. M., 1999: Characteristics of wave packets in the upper troposphere. Part II: Seasonal and hemispheric variations. J. Atmos. Sci., 56 , 17291747.

    • Search Google Scholar
    • Export Citation
  • Chang, E. K. M., 2000: Wave packets and life cycles of troughs in the upper troposphere: Examples from the Southern Hemisphere summer season of 1984/85. Mon. Wea. Rev., 128 , 2550.

    • Search Google Scholar
    • Export Citation
  • Chang, E. K. M., and D. B. Yu, 1999: Characteristics of wave packets in the upper troposphere. Part I: Northern Hemisphere winter. J. Atmos. Sci., 56 , 17081728.

    • Search Google Scholar
    • Export Citation
  • Errico, R. M., 2000: Interpretations of the total energy and rotational energy norms applied to determination of singular vectors. Quart. J. Roy. Meteor. Soc., 126 , 15811600.

    • Search Google Scholar
    • Export Citation
  • Errico, R. M., T. Vukicevic, and K. Reader, 1993: Comparison of initial and lateral boundary condition sensitivity for a limited-areas model. Tellus, 45 , 539577.

    • Search Google Scholar
    • Export Citation
  • Farrell, B. F., 1985: Transient growth of damped baroclinic waves. J. Atmos. Sci., 42 , 27182727.

  • Farrell, B. F., 1989: Transient development in confluent and diffluent flow. J. Atmos. Sci., 46 , 32793288.

  • Gelaro, R., R. H. Langland, G. D. Rohaly, and T. E. Rossmond, 1999: An assessment of the singular-vector approach to target observations using the FASTEX dataset. Quart. J. Roy. Meteor. Soc., 125 , 32993328.

    • Search Google Scholar
    • Export Citation
  • Golub, G. H., and C. F. Van Loan, 1989: Matrix Computations. 2d ed. The Johns Hopkins University Press, 642 pp.

  • Iyengar, G., Z. Toth, E. Kalnay, and J. S. Woolen, 1996: Are the bred vectors representative of analysis errors? Preprints, 11th Conf. on Numerical Weather Prediction, Norfolk, VA, Amer. Meteor. Soc., 64–65.

  • Kaplan, J. L., and J. A. Yorke, 1979: Chaotic behavior of multidimensional difference equations. Functional Differential Equations and Approximations of Fixed Points: Proceedings, Bonn, July 1978, Vol. 730H.-O. Peitgen and H.-O. Walther, Eds., Springer-Verlag, 204–227.

  • Lee, S., and I. M. Held, 1993: Baroclinic wave packets in models and observations. J. Atmos. Sci., 50 , 14131428.

  • Leutbecher, M., 2003: A reduced rank estimate of forecast error variance changes due to intermittent modifications of the observing network. J. Atmos. Sci., 60 , 729742.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55 , 399414.

    • Search Google Scholar
    • Export Citation
  • Majumdar, S. J., C. H. Bishop, B. J. Etherton, I. Szunyogh, and Z. Toth, 2001: Can an ensemble Kalman filter predict the reduction in forecast error variance produced by targeted observations? Quart. J. Roy. Meteor. Soc., 127 , 28032820.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122 , 73119.

    • Search Google Scholar
    • Export Citation
  • Orlanski, I., 2003: Bifurcation in eddy life cycles: Implications for storm track variability. J. Atmos. Sci., 60 , 9931023.

  • Orlanski, I., and J. Katzfey, 1991: The life cycle of a cyclone wave in the Southern Hemisphere. Part I: Eddy energy budget. J. Atmos. Sci., 48 , 19721998.

    • Search Google Scholar
    • Export Citation
  • Orlanski, I., and E. Chang, 1993: Ageostrophic geopotential fluxes and upstream development of baroclinic waves. J. Atmos. Sci., 50 , 212225.

    • Search Google Scholar
    • Export Citation
  • Ott, E., 2002: Chaos in Dynamical Systems. 2d ed. Cambridge University Press, 490 pp.

  • Ott, E., and Coauthors, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A , 415428.

  • Palmer, T. N., R. Gelaro, J. Barkmeijer, and R. Buizza, 1998: Singular vectors, metrics, and adaptive observations. J. Atmos. Sci., 55 , 633653.

    • Search Google Scholar
    • Export Citation
  • Patil, D. J., B. R. Hunt, E. Kalnay, J. A. Yorke, and E. Ott, 2001: Local low-dimensionality of atmospheric dynamics. Phys. Rev. Lett., 86 , 58785881.

    • Search Google Scholar
    • Export Citation
  • Rabier, F., P. Courtier, and O. Talagrand, 1992: An application of adjoint models to sensitivity analysis. Beitr. Phys. Atmos., 65 , 177192.

    • Search Google Scholar
    • Export Citation
  • Rabier, F., E. Klinker, P. Courtier, and A. Hollingsworth, 1996: Sensitivity of forecast errors to initial conditions. Quart. J. Roy. Meteor. Soc., 122 , 121150.

    • Search Google Scholar
    • Export Citation
  • Snyder, C., 1999: Error growth in flows with finite-amplitude waves or coherent structures. J. Atmos. Sci., 56 , 500506.

  • Szunyogh, I., Z. Toth, R. E. Morss, S. J. Majumdar, B. J. Etherton, and C. H. Bishop, 2000: The effect of targeted dropsonde observations during the 1999 winter storm reconnaissance program. Mon. Wea. Rev., 128 , 35203537.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., G. Gyarmati, and D. Devenyi, 2002a: Using observed data for testing the statistical consistency of initial ensemble perturbations. Idöjárás, 105 , 293303.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., Z. Toth, A. V. Zimin, S. J. Majumdar, and A. Persson, 2002b: Propagation of the effect of targeted observations: The 2000 winter storm reconnaissance program. Mon. Wea. Rev., 130 , 11441165.

    • Search Google Scholar
    • Export Citation
  • Szunyogh, I., E. J. Kostelich, G. Gyarmati, B. R. Hunt, E. Kalnay, E. Ott, D. J. Patil, and J. A. Yorke, 2004: A local ensemble Kalman filter for the NCEP GFS model. Preprints, Symp. on 50th Anniversary of Numerical Weather Prediction, College Park, MD, Amer. Meteor. Soc., J2.4.

  • Talagrand, O., 1981: A study of the dynamics of four-dimensional data assimilation. Tellus, 33 , 4360.

  • Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125 , 32973319.

  • Toth, Z., I. Szunyogh, C. Bishop, S. Majumdar, R. Morss, and S. Lord, 2001: The use of targeted observations in operational numerical weather forecasting. Proc. Fifth Symp. on Integrated Observing Systems, Albequerque, NM, Amer. Meteor. Soc., 185–190.

  • Toth, Z., and Coauthors, 2002: Adaptive observations at NCEP: Past, present, and future. Preprints, Symp. on Observations, Data Assimilation, and Probabilistic Prediction, Orlando, FL, Amer. Meteor. Soc., 185–190.

  • Vannitsem, S., 2001: Toward a phase-space cartography of the short- and medium-range predictability of weather regimes. Tellus, 53A , 5673.

    • Search Google Scholar
    • Export Citation
  • Vannitsem, S., and C. Nicolis, 1997: Lyapunov vectors and error growth patterns in a T21L3 quasigeostrophic model. J. Atmos. Sci., 54 , 347361.

    • Search Google Scholar
    • Export Citation
  • Vannitsem, S., and C. Nicolis, 1998: Dynamics of fine-scale versus averaged observables in a T21L3 quasi-geostrophic model. Quart. J. Roy. Meteor. Soc., 124 , 22012226.

    • Search Google Scholar
    • Export Citation
  • Zimin, A. V., I. Szunyogh, D. J. Patil, B. R. Hunt, and E. Ott, 2003: Extracting envelopes of Rossby wave packets. Mon. Wea. Rev., 131 , 10111017.

    • Search Google Scholar
    • Export Citation

APPENDIX

Illustrations of the E Dimension on Simple Examples

Assume that the ensemble has five members, k = 5. If all five local vectors were identical (representing one dominant direction), then the singular values would be (5, 0, 0, 0, 0).This would yield E dimension 1. If the total variance in the ensemble was equally distributed between two orthogonal directions υ1 and υ2, then the singular values would be (5/2, 5/2, 0, 0, 0), and our statistic would yield E = 2. If the local vectors again lay in the same two-dimensional subspace spanned by unit vectors υ1 and υ2, but the variance was not equally distributed between the two directions, then the E dimension would be a noninteger between 1 and 2. For instance, if all five local vectors had unit length, four of them pointed to the same direction, while the remaining one pointed to an orthogonal direction, the singular values would be (2, 1, 0, 0, 0), and E = 1.8. While the dimension of the vector space spanned by the local vectors is 2, the E dimension is smaller, reflecting the degree of dominance of one direction over the other. We can also consider examples in which the local vectors are not of unit length. For instance, when the local vectors span a two-dimensional space, we might have singular values such as (5, 1, 0, 0, 0), which would yield E ≈ 1.4, indicating an even stronger dominance of one direction over the other.

Finally, Table A1 provides examples to illustrate the relationship between the E dimension and the dimensions based on thresholding the singular values or the explained variance. These examples show clearly that the distribution of the variance between the directions is best characterized by the E dimension.

Fig. 1.
Fig. 1.

E dimension (shades) for a 30-pair ensemble and geopotential height for the control (contour lines) at the 50-kPa level for a 24-h forecast started from 0000 UTC on 7 Feb 2000. Contours are shown between 4800 and 5800 gpm, with 100-gpm intervals.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 2.
Fig. 2.

Mean E dimension over the NH midlatitudes (solid lines) and SH mid-latitudes (dotted lines) vs forecast hour for 72-h forecasts started from 0000 UTC on 7 Feb 2000 (no marker), 0000 UTC on 23 Jan 2000 (diamonds), and 0000 UTC on 1 Feb 2000 (circles). A somewhat weaker transient is found in the Tropics (not shown), but there the E dimension is significantly higher (> 16).

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 3.
Fig. 3.

E dimension for a 30-pair ensemble (shades) for a 24-h forecast started from 0000 UTC on 7 Feb 2000 using various local regions sizes: (a) 3 × 3, (b) 5 × 5, (c) 7 × 7, and (d) 9 × 9 grid points.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 4.
Fig. 4.

Dependence of the E dimension in the NH midlatitudes on ensemble size. Shown are the maximum and minimum E dimensions averaged over 45 days every 24 h from 0000 UTC on 15 Jan 2000 to 0000 UTC on 28 Feb 2000. Error bars indicate one standard deviation. Also shown is the E dimension for an ensemble of Gaussian random noise.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 5.
Fig. 5.

E dimension in a representative low-dimensional region computed for ensemble sizes 15, 30, 65, 95, 125, and 150 pairs of bred vectors. Note the difference in the color schemes between the upper-left panel and the other panels.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 6.
Fig. 6.

Distribution of the local singular values in areas of low, medium, and high E dimension on a given day (0000 UTC on 9 Feb 2000). (top) First the singular value spectrum (normalized by the largest singular value) was computed for each local region; then the normalized spectra were averaged over the globe. (bottom) The percentage-explained variance obtained by a similar averaging. Error bars indicate the standard deviation over all local regions.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 7.
Fig. 7.

E dimension (color shades) and baroclinic energy conversion (thick contour lines). The contour interval is 300 m2 s−2 day−1, and values smaller than 600 m2 s−2 day−1 are not shown. The geopotential height of the control forecast at the 30-kPa level (thin contour lines) is shown between 8400 and 9200 gpm, with 100-gpm intervals. Results are in the domain 30°−60°N, 120°E−150°W and are shown every 3 h from 0200 UTC on 7 Feb 2000. The low-dimensional region of interest is marked by an arrow.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 8.
Fig. 8.

Time evolution of (top) the average E dimension and (bottom) the average energy conversion terms over a moving 5 × 5 grid point box (see text for further details) starting at 0200 UTC on 7 Feb 2000. Shown are the baroclinic energy conversion (solid line), energy transport (dashed line), divergence of the ageostrophic geopotential flux (dashed–dotted line), and barotropic energy conversion (dotted line), in m2 s−2 day−1.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 9.
Fig. 9.

E dimension (color shades) and barotropic energy conversion (thick contour lines). The contour interval is 150 m2 s−2 day−1, and values between −150 and 150 m2 s−2 day−1 are not shown. The geopotential height of the control forecast at the 30-kPa level (thin contour lines) is shown between 8400 and 9200 gpm, with 100-gpm intervals. Results are in the domain 30°−55°N, 95°−175°E and are shown every 5 h from 0100 UTC on 7 Feb 2000. The low-dimensional region of interest is marked by an arrow.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 10.
Fig. 10.

E dimension (color shades) and divergence of the ageostrophic geopotential flux (orange contour lines) with contour interval 200 m2 s−2 day−1 (values between −400 and 400 m2 s−2 day−1 are not shown), ageostrophic geopotential flux at the 30-kPa level (arrows; values smaller than 10 000 m3 s−3 are not shown), and wave packet envelope (yellow contours) contour interval 1.5 m s−1 (values smaller than 13 m s−1 are not shown).The geopotential height of the control forecast at the 30-kPa level (thin black contour lines) is shown between 8400 and 9200 gpm, with 100-gpm intervals. Results are in the domain 30°−50°N, 100°−155°W and are shown every 7 h from 1100 UTC on 7 Feb 2000.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 11.
Fig. 11.

E dimension (color shades) and wave packet envelope (thick contour lines) for a 48-h forecast lead time started from 0000 UTC on 7 Feb 2000. The contour interval is 2 m s−1, and values smaller than 16 m s−1 are not shown.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 12.
Fig. 12.

E dimension (color shades), barotropic energy conversion (thick black contour lines), divergence of the ageostrophic geopotential flux (white contour lines), and the energy transport (red contour lines in top panels, blue contour lines in bottom), with contour interval 200 m2 s−2 day−1 (values smaller than 400 m2 s−2 day−1 are not shown); the ageostrophic geopotential flux (arrows) at the 30-kPa level (values smaller than 5000 m3 s−3 are not shown); and the geopotential heights of the control forecast at the 30-kPa level (thin black contours) are shown between 8400 and 9200 gpm, with 100-gpm intervals, every 11 h from 0100 UTC on 7 Feb 2000.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 13.
Fig. 13.

E dimension (color shades) and divergence of the ageostrophic geopotential flux (orange contour lines), with contour interval 200 m2 s−2 day−1 (values smaller than 400 m2 s−2 day−1 are not shown); the ageostrophic geopotential flux (arrows; values smaller than 10 000 m3 s−3 are not shown); and the geopotential height of the control forecast at the 50-kPa level (thin black contour lines), with 100-gpm intervals, every 10 h from 0400 UTC on 1 Feb 2000.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 14.
Fig. 14.

E dimension (color shades), baroclinic energy conversion (thick black contour lines) and barotropic energy conversion (orange contour lines), both with contour interval 200 m2 s−2 day−1 (values smaller than 200 m2 s−2 day−1 are not shown), and the geopotential height of the control forecast at the 50-kPa level (thin black contour lines) with 100-gpm intervals, every 10 h from 0400 UTC on 1 Feb 2000.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 15.
Fig. 15.

Dropsonde locations (red crosses) and E dimension (color shades) for the WSR00-targeted observations field program.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Fig. 16.
Fig. 16.

Surface pressure forecast improvement in kPa (color shades) and the geopotential height at the 50-kPa level of the original analyses, shown between 4800 and 5850 gpm, with contour intervals of 75 gpm. (top) Targeted observation locations are marked with red crosses and (bottom) blue curves represent the 48-h verification regions. (left) The localized E dimension (thick black contour lines) with contour intervals of (top) 0.5 and (middle), (bottom) 0.4; high values are not shown. (right) The global E dimension (thick black contour lines) with contour intervals of (top) 0.5 and (middle), (bottom) 0.4; high values are not shown.

Citation: Journal of the Atmospheric Sciences 62, 4; 10.1175/JAS3403.1

Table A1. Simple examples of the E dimension; a dimension based on thresholding of the singular values, dε (at a value of 0.01); and a dimension based on thresholding of the explained variance, d (at a value of 95%). In this example, the total number of nonzero singular values (k) is arbitrary, but the singular values are subject to the constraint ∑ki=1σ2i = 1.

i1520-0469-62-4-1135-ta01
Save