Multilevel Regression Modeling of Nonlinear Processes: Derivation and Applications to Climatic Variability

S. Kravtsov Department of Atmospheric and Oceanic Sciences, and Institute of Geophysics and Planetary Physics, University of California, Los Angeles, Los Angeles, California

Search for other papers by S. Kravtsov in
Current site
Google Scholar
PubMed
Close
,
D. Kondrashov Department of Atmospheric and Oceanic Sciences, and Institute of Geophysics and Planetary Physics, University of California, Los Angeles, Los Angeles, California

Search for other papers by D. Kondrashov in
Current site
Google Scholar
PubMed
Close
, and
M. Ghil Department of Atmospheric and Oceanic Sciences, and Institute of Geophysics and Planetary Physics, University of California, Los Angeles, Los Angeles, California

Search for other papers by M. Ghil in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Predictive models are constructed to best describe an observed field’s statistics within a given class of nonlinear dynamics driven by a spatially coherent noise that is white in time. For linear dynamics, such inverse stochastic models are obtained by multiple linear regression (MLR). Nonlinear dynamics, when more appropriate, is accommodated by applying multiple polynomial regression (MPR) instead; the resulting model uses polynomial predictors, but the dependence on the regression parameters is linear in both MPR and MLR.

The basic concepts are illustrated using the Lorenz convection model, the classical double-well problem, and a three-well problem in two space dimensions. Given a data sample that is long enough, MPR successfully reconstructs the model coefficients in the former two cases, while the resulting inverse model captures the three-regime structure of the system’s probability density function (PDF) in the latter case.

A novel multilevel generalization of the classic regression procedure is introduced next. In this generalization, the residual stochastic forcing at a given level is subsequently modeled as a function of variables at this level and all the preceding ones. The number of levels is determined so that the lag-0 covariance of the residual forcing converges to a constant matrix, while its lag-1 covariance vanishes.

This method has been applied to the output of a three-layer, quasigeostrophic model and to the analysis of Northern Hemisphere wintertime geopotential height anomalies. In both cases, the inverse model simulations reproduce well the multiregime structure of the PDF constructed in the subspace spanned by the dataset’s leading empirical orthogonal functions, as well as the detailed spectrum of the dataset’s temporal evolution. These encouraging results are interpreted in terms of the modeled low-frequency flow’s feedback on the statistics of the subgrid-scale processes.

* Additional affiliation: Départment Terre–Atmosphère–Océan and Laboratoire de Météorologie Dynamique/IPSL, Ecole Normale Supérieure, Paris, France

Corresponding author address: Dr. Sergey Kravtsov, Department of Atmospheric and Oceanic Sciences, University of California, Los Angeles, 405 Hilgard Ave., Los Angeles, CA 90095-1565. Email: sergey@atmos.ucla.edu

Abstract

Predictive models are constructed to best describe an observed field’s statistics within a given class of nonlinear dynamics driven by a spatially coherent noise that is white in time. For linear dynamics, such inverse stochastic models are obtained by multiple linear regression (MLR). Nonlinear dynamics, when more appropriate, is accommodated by applying multiple polynomial regression (MPR) instead; the resulting model uses polynomial predictors, but the dependence on the regression parameters is linear in both MPR and MLR.

The basic concepts are illustrated using the Lorenz convection model, the classical double-well problem, and a three-well problem in two space dimensions. Given a data sample that is long enough, MPR successfully reconstructs the model coefficients in the former two cases, while the resulting inverse model captures the three-regime structure of the system’s probability density function (PDF) in the latter case.

A novel multilevel generalization of the classic regression procedure is introduced next. In this generalization, the residual stochastic forcing at a given level is subsequently modeled as a function of variables at this level and all the preceding ones. The number of levels is determined so that the lag-0 covariance of the residual forcing converges to a constant matrix, while its lag-1 covariance vanishes.

This method has been applied to the output of a three-layer, quasigeostrophic model and to the analysis of Northern Hemisphere wintertime geopotential height anomalies. In both cases, the inverse model simulations reproduce well the multiregime structure of the PDF constructed in the subspace spanned by the dataset’s leading empirical orthogonal functions, as well as the detailed spectrum of the dataset’s temporal evolution. These encouraging results are interpreted in terms of the modeled low-frequency flow’s feedback on the statistics of the subgrid-scale processes.

* Additional affiliation: Départment Terre–Atmosphère–Océan and Laboratoire de Météorologie Dynamique/IPSL, Ecole Normale Supérieure, Paris, France

Corresponding author address: Dr. Sergey Kravtsov, Department of Atmospheric and Oceanic Sciences, University of California, Los Angeles, 405 Hilgard Ave., Los Angeles, CA 90095-1565. Email: sergey@atmos.ucla.edu

Save
  • Allen, R. M., and L. A. Smith, 1996: Monte Carlo SSA: Detecting irregular oscillations in the presence of colored noise. J. Climate, 9 , 33733404.

    • Search Google Scholar
    • Export Citation
  • Berloff, P., and J. C. McWilliams, 2002: Material transport in oceanic gyres. Part II: Hierarchy of stochastic models. J. Phys. Oceanogr., 32 , 797830.

    • Search Google Scholar
    • Export Citation
  • Box, G. E. P., G. M. Jenkins, and G. C. Reinsel, 1994: Time Series Analysis, Forecasting and Control. 3d ed. Prentice Hall, 592 pp.

  • Cheng, X. H., and J. M. Wallace, 1993: Analysis of the Northern Hemisphere wintertime 500-hPa height field: Spatial patterns. J. Atmos. Sci., 50 , 26742696.

    • Search Google Scholar
    • Export Citation
  • Da Costa, E., and R. Vautard, 1997: A qualitative realistic low-order model of the extratropical low-frequency variability built from long records of potential vorticity. J. Atmos. Sci., 54 , 10641084.

    • Search Google Scholar
    • Export Citation
  • D’Andrea, F., 2002: Extratropical low-frequency variability as a low-dimensional problem. Part II: Stationarity and stability of large-scale equilibria. Quart. J. Roy. Meteor. Soc., 128 , 10591073.

    • Search Google Scholar
    • Export Citation
  • D’Andrea, F., and R. Vautard, 2001: Extratropical low-frequency variability as a low-dimensional problem. Part I: A simplified model. Quart. J. Roy. Meteor. Soc., 127 , 13571374.

    • Search Google Scholar
    • Export Citation
  • DelSole, T., 1996: Can quasigeostrophic turbulence be modeled stochastically? J. Atmos. Sci., 53 , 16171633.

  • DelSole, T., 2000: A fundamental limitation of Markov models. J. Atmos. Sci., 57 , 21582168.

  • Deser, C., 2000: On the teleconnectivity of the “Arctic Oscillation.”. Geophys. Res. Lett., 27 , 779782.

  • Dettinger, M. D., M. Ghil, C. M. Strong, W. Weibel, and P. Yiou, 1995: Software expedites singular-spectrum analysis of noisy time series. Eos, Trans. Amer. Geophys. Union, 76 .12, 14, 21.

    • Search Google Scholar
    • Export Citation
  • Dijkstra, H. A., and M. Ghil, 2005: Low-frequency variability of the large-scale ocean circulation: A dynamical systems approach. Rev. Geophys, in press.

    • Search Google Scholar
    • Export Citation
  • Farrell, B. F., and P. J. Ioannou, 1993: Stochastic forcing of the linearized Navier–Stokes equations. Phys. Fluids, A5 , 26002609.

  • Farrell, B. F., and P. J. Ioannou, 1995: Stochastic dynamics of the midlatitude atmospheric jet. J. Atmos. Sci., 52 , 16421656.

  • Franzke, C., A. J. Majda, and E. Vanden-Eijnden, 2005: Low-order stochastic mode reduction for a realistic barotropic model climate. J. Atmos. Sci., 62 , 17221745.

    • Search Google Scholar
    • Export Citation
  • Ghil, M., and S. Childress, 1987: Topics in Geophysical Fluid Dynamics: Atmospheric Dynamics, Dynamo Theory and Climate Dynamics. Springer-Verlag, 485 pp.

    • Search Google Scholar
    • Export Citation
  • Ghil, M., and K. C. Mo, 1991: Intraseasonal oscillations in the global atmosphere. Part I: Northern Hemisphere and Tropics. J. Atmos. Sci., 48 , 752779.

    • Search Google Scholar
    • Export Citation
  • Ghil, M., and A. W. Robertson, 2000: Solving problems with GCMs: General circulation models and their role in the climate modeling hierarchy. General Circulation Model Development: Past, Present and Future, D. Randall, Ed., Academic Press, 285–325.

    • Search Google Scholar
    • Export Citation
  • Ghil, M., and Coauthors, 2002: Advanced spectral methods for climatic time series. Rev. Geophys., 40 .1003, doi:10.1029/2000RG000092.

  • Hand, D., H. Mannila, and P. Smyth, 2001: Principles of Data Mining. MIT Press, 546 pp.

  • Hannachi, A., 1997: Low-frequency variability in a GCM: Three-dimensional flow regimes and their dynamics. J. Climate, 10 , 13571379.

  • Hannachi, A., and A. O’Neill, 2001: Atmospheric multiple equilibria and non-Gaussian behavior in model simulations. Quart. J. Roy. Meteor. Soc., 127 , 939958.

    • Search Google Scholar
    • Export Citation
  • Hasselmann, K., 1999: Climate change: Linear and nonlinear signature. Nature, 398 , 755756.

  • Höskuldsson, A., 1996: Prediction Methods in Science and Technology. Thor Publishing.

  • Johnson, S. D., D. S. Battisti, and E. S. Sarachik, 2000: Empirically derived Markov models and prediction of tropical Pacific sea surface temperature anomalies. J. Climate, 13 , 317.

    • Search Google Scholar
    • Export Citation
  • Jolliffe, I. T., 2002: Principal Component Analysis. 2d ed. Springer, 487 pp.

  • Kimoto, M., and M. Ghil, 1993a: Multiple flow regimes in the Northern Hemisphere winter. Part I: Methodology and hemispheric regimes. J. Atmos. Sci., 50 , 26252643.

    • Search Google Scholar
    • Export Citation
  • Kimoto, M., and M. Ghil, 1993b: Multiple flow regimes in the Northern Hemisphere winter. Part II: Sectorial regimes and preferred transitions. J. Atmos. Sci., 50 , 26452673.

    • Search Google Scholar
    • Export Citation
  • Kondrashov, D., K. Ide, and M. Ghil, 2004: Weather regimes and preferred transition paths in a three-level quasigesotrophic model. J. Atmos. Sci., 61 , 568587.

    • Search Google Scholar
    • Export Citation
  • Kondrashov, D., S. Kravtsov, A. W. Robertson, and M. Ghil, 2005: A hierarchy of data-based ENSO models. J. Climate, 18 , 44254444.

  • Legras, B., and M. Ghil, 1985: Persistent anomalies, blocking and variations in atmospheric predictability. J. Atmos. Sci., 42 , 433471.

    • Search Google Scholar
    • Export Citation
  • Lorenz, E. N., 1963a: Deterministic nonperiodic flow. J. Atmos. Sci., 20 , 130141.

  • Lorenz, E. N., 1963b: The mechanics of vacillation. J. Atmos. Sci., 20 , 448464.

  • Majda, A. J., I. Timofeyev, and E. Vanden-Eijnden, 1999: Models for stochastic climate prediction. Proc. Natl. Acad. Sci. USA, 96 , 1468714691.

    • Search Google Scholar
    • Export Citation
  • Majda, A. J., I. Timofeyev, and E. Vanden-Eijnden, 2001: A mathematical framework for stochastic climate models. Commun. Pure Appl. Math., 54 , 891974.

    • Search Google Scholar
    • Export Citation
  • Majda, A. J., I. Timofeyev, and E. Vanden-Eijnden, 2002: A priori test of a stochastic mode reduction strategy. Physica D, 170 , 206252.

    • Search Google Scholar
    • Export Citation
  • Majda, A. J., I. Timofeyev, and E. Vanden-Eijnden, 2003: Systematic strategies for stochastic mode reduction in climate. J. Atmos. Sci., 60 , 17051722.

    • Search Google Scholar
    • Export Citation
  • Marshall, J., and F. Molteni, 1993: Toward a dynamical understanding of atmospheric weather regimes. J. Atmos. Sci., 50 , 17921818.

  • McCullagh, P., and J. A. Nelder, 1989: Generalized Linear Models. Chapman and Hall, 511 pp.

  • Miller, R. N., M. Ghil, and F. Gauthiez, 1994: Advanced data assimilation in strongly nonlinear dynamical systems. J. Atmos. Sci., 51 , 10371056.

    • Search Google Scholar
    • Export Citation
  • Mo, K., and M. Ghil, 1988: Cluster analysis of multiple planetary flow regimes. J. Geophys. Res., 93D , 1092710952.

  • Molteni, F., 1996a: On the dynamics of planetary flow regimes. Part I: The role of high-frequency transients. J. Atmos. Sci., 53 , 19501971.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., 1996b: On the dynamics of planetary flow regimes. Part II: Results from a hierarchy of orographically forced models. J. Atmos. Sci., 53 , 19721992.

    • Search Google Scholar
    • Export Citation
  • Molteni, F., 2002: Weather regimes and multiple equilibria. Encyclopedia of Atmospheric Science, J. R. Holton, J. Curry, and J. Pyle, Eds., Academic Press, 2577–2585.

    • Search Google Scholar
    • Export Citation
  • Mundt, M. D., and J. E. Hart, 1994: Secondary instability, EOF reduction, and the transition to baroclinic chaos. Physica D, 78 , 6592.

    • Search Google Scholar
    • Export Citation
  • Penland, C., 1989: Random forcing and forecasting using principal oscillation pattern analysis. Mon. Wea. Rev., 117 , 21652185.

  • Penland, C., 1996: A stochastic model of Indo-Pacific sea-surface temperature anomalies. Physica D, 98 , 534558.

  • Penland, C., and M. Ghil, 1993: Forecasting Northern Hemisphere 700-mb geopotential height anomalies using empirical normal modes. Mon. Wea. Rev., 121 , 23552372.

    • Search Google Scholar
    • Export Citation
  • Penland, C., and P. D. Sardeshmukh, 1995: The optimal growth of tropical sea surface temperature anomalies. J. Climate, 8 , 19992024.

  • Penland, C., and L. Matrosova, 1998: Prediction of tropical Atlantic sea-surface temperatures using linear inverse modeling. J. Climate, 11 , 483496.

    • Search Google Scholar
    • Export Citation
  • Preisendorfer, R. W., 1988: Principal Component Analysis in Meteorology and Oceanography. Elsevier, 425 pp.

  • Press, W. H., S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, 1994: Numerical Recipes. 2d ed. Cambridge University Press, 994 pp.

    • Search Google Scholar
    • Export Citation
  • Randall, D., 2000: General Circulation Model Development: Past, Present and Future. Academic Press, 781 pp.

  • Richman, M. B., 1986: Rotation of principal components. J. Climatol., 6 , 293335.

  • Rinne, J., and V. Karhila, 1975: A spectral barotropic model in horizontal empirical orthogonal functions. Quart. J. Roy. Meteor. Soc., 101 , 365382.

    • Search Google Scholar
    • Export Citation
  • Schubert, S. D., 1985: A statistical-dynamical study of empirically determined modes of atmospheric variability. J. Atmos. Sci., 42 , 317.

    • Search Google Scholar
    • Export Citation
  • Selten, F. M., 1995: An efficient description of the dynamics of the barotropic flow. J. Atmos. Sci., 52 , 915936.

  • Selten, F. M., 1997: Baroclinic empirical orthogonal functions as basis functions in an atmospheric model. J. Atmos. Sci., 54 , 21002114.

    • Search Google Scholar
    • Export Citation
  • Sirovich, L., and J. D. Rodriguez, 1987: Coherent structures and chaos—A model problem. Phys. Lett., 120 , 211214.

  • Smyth, P., K. Ide, and M. Ghil, 1999: Multiple regimes in Northern Hemisphere height fields via mixture model clustering. J. Atmos. Sci., 56 , 37043723.

    • Search Google Scholar
    • Export Citation
  • Vautard, R., and B. Legras, 1988: On the source of midlatitude low-frequency variability. Part II: Nonlinear equilibration of weather regimes. J. Atmos. Sci., 45 , 28452867.

    • Search Google Scholar
    • Export Citation
  • Vautard, R., and M. Ghil, 1989: Singular spectrum analysis in nonlinear dynamics, with applications to paleoclimatic time series. Physica D, 35 , 395424.

    • Search Google Scholar
    • Export Citation
  • Von Mises, R., 1964: Mathematical Theory of Probability and Statistics. Academic Press, 694 pp.

  • Wallace, J. M., 2000: North Atlantic Oscillation/annular mode: Two paradigms—One phenomenon. Quart. J. Roy. Meteor. Soc., 126 , 791805.

    • Search Google Scholar
    • Export Citation
  • Wetherill, G. B., 1986: Regression Analysis with Applications. Chapman and Hall, 311 pp.

  • Winkler, C. R., M. Newman, and P. D. Sardeshmukh, 2001: A linear model of wintertime low-frequency variability. Part I: Formulation and forecast skill. J. Climate, 14 , 44744494.

    • Search Google Scholar
    • Export Citation
  • Wold, S., A. Ruhe, H. Wold, and W. J. Dunn III, 1984: The collinearity problem in linear regression: The Partial Least Square approach to generalized inverses. SIAM J. Sci. Stat. Comput., 5 , 735743.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 945 241 14
PDF Downloads 628 178 9