1. Introduction

Principal component analysis (PCA), also known as empirical orthogonal function (EOF) analysis, is a powerful technique for the objective characterization of low-dimensional linear structure in multivariate datasets. Consequently, it has enjoyed broad application in meteorology and oceanography (for a review, see von Storch and Zwiers 1999). However, the constraint that only linear structure can be detected is a strong one, and in general it leads to suboptimal low-dimensional approximations. PCA can be viewed as one of many feature extraction methods, which in general are concerned with the characterization of low-dimensional structure in multivariate datasets. Kramer (1991) demonstrated that if PCA is formulated as a variational problem for the detection of low-dimensional linear structure, then it has a natural extension to the more general problem of characterizing low-dimensional nonlinear structure and a straightforward implementation using feed-forward neural networks. He denoted this generalization nonlinear principal component analysis (NLPCA). Monahan (2000a) discussed the application of NLPCA to climatological datasets and demonstrated that the method was able to produce more representative one- and two-dimensional approximations of the Lorenz attractor, a dataset with substantial nonlinear low-dimensional structure, than was PCA. In particular, the one- and two-dimensional NLPCA approximations explained, respectively, 76% and 99.5% of the variance in the Lorenz data, as compared to 60% and 95% for the PCA approximation. While NLPCA has been applied in a number of different fields [see Monahan (2000a) for an overview], it has not yet been systematically applied to climatological datasets, apart from a single unpublished report by Sengupta and Boyle (1995), the results of which were equivocal.

In this paper, we apply NLPCA to two geophysical datasets: tropical Pacific Ocean sea surface temperature (SST) and tropical Indo–Pacific sea level pressure (SLP). Section 2 provides a brief overview of NLPCA, and section 3 describes the methodology used to construct the NLPCA approximations. Sections 4 and 5 describe the results of the NLPCA of sea surface temperature and sea level pressure, respectively, and a summary and conclusions are given in Section 6.

2. Nonlinear principal component analysis

As is always the case when mathematical models are generalized, certain features of the original model survive the generalization, and other features are lost. An important characteristic of PCA is that it partitions variance, in the sense that if X̂(t_n) is the PCA approximation to X(t_n), then

View Expanded

which is to say that the total variance of X(t_n) is the sum of the total variance of X̂(t_n), with the total variance of the residuals X(t_n) − X̂(t_n). This partition is important, as it allows one to describe a PCA approximation as describing a given fraction of the variance in the system. Considering a dataset sampled from the Lorenz system, Monahan (2000a) found empirically that NLPCA also partitions variance in this sense. While we are not aware of a proof of this result, it is found again in all examples considered in this paper. Consequently, we are comfortable according it the status of an empirical fact.

A feature that does not survive the generalization is the additive nature of a P-dimensional PCA approximation. The P-dimensional PCA approximation to X(t_n) may be found by first finding the 1D PCA approximation X̂⁽¹⁾(t_n), subtracting this from the original data to form the residual R⁽¹⁾(t_n) = X(t_n) − X̂⁽¹⁾(t_n), determining the 1D PCA approximation of R⁽¹⁾(t_n), X̂⁽²⁾(t_n), and iterating this procedure until P successive 1D approximations have been found. The resulting P-dimensional approximation will be the sum of the X̂⁽ⁱ⁾(t_n). Alternatively, the optimal P-dimensional approximation can be determined all at once. With PCA, these two approaches are equivalent, because any linear function, g, of P variables (u₁, . . . , u_P) has the additive structure

View Expanded

With NLPCA, however, the two approaches are distinct, because a nonlinear function f cannot usually be decomposed as a generalized additive model:

fu₁u₂u_Pf₁u₁f₂u₂f_Pu_P(8)

for some f₁, f₂, . . . , f_P. Note that this is unrelated to the fact that NLPCA approximations are found to partition variance. We shall refer to the first, iterative approach as a modal analysis and to the second as nonmodal. Naturally enough, in a modal analysis, each 1D approximation will be referred to as a mode and ordered in terms of decreasing fraction of variance explained. We will compare both the modal and nonmodal approaches in this paper. Theoretically, the nonmodal P-dimensional approximation should be superior to the modal approximation, because it will be drawn from a broader class of functions, although the modal analysis is more amenable to interpretation. Of course, a general P-dimensional analysis could involve both modal and nonmodal decompositions at various stages; such mixed modal/nonmodal analyses will not be considered here.

3. Model building

Neural networks are powerful function approximation tools, and the avoidance of overfitting is a primary issue in their implementation (Finnoff et al. 1993; Yuval 2000). A model is said to be overfit if its performance on the data used to determine the model parameters (the training data) is good, but is poor on other data. To avoid overfitting, we have used a simple technique called early stopping, as in Monahan (2000a). Because the neural network is nonlinear in the model parameters, these must be determined iteratively, in a process referred to as training. In early stopping, the training is terminated before the error function is minimized, according to a well-defined stopping criterion. In essence, the idea behind early stopping is that the training is allowed to continue sufficiently long to fit the structure underlying the data, but not long enough to fit the noise. The term “noise” refers to any variability that cannot be robustly characterized by the approximation being constructed. The strategy we employed was to hold aside a fraction (in our case, 20%) of randomly selected data points in a validation set not used to train the network. While network training proceeded, the network performance on the validation set was monitored, and training was stopped when this performance began to degrade, or after 5000 iterations, whichever came first. Note that other methods exist for robustly minimizing the error function (Yuval 2000); early stopping was used because of its relative transparency and ease of implementation.

The early stopping algorithm described above confers on the training results a degree of sensitivity to the network parameters used to initialize the iterative training procedure; this is exacerbated by the possible existence of multiple minima in the error function (5). To address this problem, an ensemble of at least 20 training runs starting from different, randomly chosen, initial parameter values was carried out, for each analysis performed. The training results from this ensemble were examined, and those members of the ensemble for which the final error over the validation set was greater than that over the training set were discarded. We shall refer to the remaining members of the ensemble as candidate models.

The number of neurons L in the encoding and decoding layers determines the class of functions that s_f and f can take. As L increases, there is more flexibility in the forms of s_f and f, but the model also has more parameters, implying both that the error surface becomes more complicated and that the parameters are less constrained by data. Consequently, if L is large, the scatter among the candidate models can be large, as measured by the normalized mean square distance (NMSD). The NMSD between approximations X̂^{1}(t_n) and X̂^{2}(t_n) is defined as

View Expanded

This statistic was introduced in Monahan (2000a), in which it was found that NLPCA approximations for which the NMSD was less than about 2% were essentially indistinguishable. In the end, the number L of neurons used in the encoding and decoding layers was the maximum such that the NMSD between NLPCA approximations to X(t_n) in the candidate model set was less than 5%. This threshold value of NMSD was chosen on the basis of experience and intuition, and not on the basis of any rigorous sampling theory for this test statistic (which we do not have). In other words, for any given analysis, the value of L used in the NLPCA network is the largest that produces a robust set of candidate models. The early stopping technique ensures that the NLPCA approximation is robust with respect to the introduction of new data, and the existence of a set of similar candidate models (as measured by NMSD) ensures that the approximation is robust with respect to the initial parameter values used in the training.

Finally, once L was determined and a set of candidate models obtained, the model selected as “the” NLPCA approximation was the one with the highest fraction of explained variance (FEV),

View Expanded

which is a meaningful statistic because NLPCA partitions variance as described in Eq. (6). The FEV differed little between candidate models.

4. Tropical Pacific sea surface temperature

We consider first a dataset composed of monthly averaged National Oceanic and Atmospheric Administration sea surface temperatures (SST) for the tropical Pacific Ocean. The data are on a 2° × 2° grid from 19°S to 19°N and from 125°E to 69°W, and span the period from January 1950 to December 1998 (Smith et al. 1996). A climatological annual cycle was calculated by averaging the data for each calendar month, and monthly SST anomalies (SSTAs) were defined relative to this annual cycle. To render the NLPCA problem tractable, the dataset was preprocessed by projecting it on the space of its first 10 EOF modes {e_k}¹⁰_k=1, in which 91.4% of the total variance is contained. By doing so, we take advantage of the data compression aspect of PCA, which is distinct from feature extraction, for which we shall use NLPCA. Such preprocessing of data to reduce the problem to a manageable size is common in rotated PCA (Barnston and Livezey 1987) and in statistical forecasting (Barnston 1994; Tangang et al. 1998). The first three EOF spatial patterns of SSTA are displayed in Fig. 2; these explain, respectively, 57.6%, 10.9%, and 6.8% of the total SST variance.

A scatterplot of the two leading principal component time series is shown by the solid dots in Fig. 3a. The time series corresponding to these two PCA modes are uncorrelated, but they are clearly not independent. Indeed, Fig. 3a indicates that both strongly positive and negative values of the first PCA time series are associated with negative values of the second PCA time series. Physically, this describes the fact that the strongest positive SST anomalies during an average El Niño event lie closer to the eastern boundary of the Pacific than do the coldest anomalies during an average La Niña event. We shall return to this point later.

We consider first a modal NLPCA decomposition of this SSTA data. Mode 1, found using a network with L = 4 nodes in the encoding and decoding layers and a single neuron in the bottleneck layer, explains 69.1% of the variance in the 10-dimensional EOF space and, thus, 63.3% of the variance in the total SSTA data, as compared to 57.6% explained by the 1D PCA approximation. Four candidate models were obtained from an ensemble of 20; these models differed with an NMSD of at most 1%. Projections of the first NLPCA mode onto the subspaces spanned by the SSTA EOFs (e₁, e₂), (e₂, e₃), (e₁, e₃), and (e₁, e₂, e₃) are shown by the open circles in Figs. 3a–d, respectively. All four projections are shown because it is difficult to understand the structure of the NLPCA approximation from a single projection alone. This is particularly evident in Fig. 3b, the curve, viewed edge-on, appears to be self-intersecting, when in fact the other projections demonstrate that this is not the case. Figure 3a indicates that NLPCA mode 1 characterizes the structure discussed in the previous paragraph; NLPCA mode 1 is primarily a mixture of PCA modes 1 and 2. Associated with this mode is the standardized time series

View Expanded

corresponding to the (standardized) output of the single neuron in the bottleneck layer. A plot of α₁(t_n) appears in Fig. 4. This time series bears a strong resemblance to the Niño-3.4 time series (defined as the average SSTA over a box from 7°S to 7°N, and from 119° to 171°W), also displayed in Fig. 4; the correlation coefficient between the two series is 0.88.

In contrast to PCA, no single spatial pattern is associated with any given NLPCA mode. The approximation X̂, however, corresponds to a sequence of patterns that can be visualized cinematographically. This cinematographic interpretation is implicit in traditional PCA; the 1D PCA approximation X̂(t_n) = [e₁ · X(t_n)]e₁ describes the evolution in time of a standing oscillation. This oscillation has a fixed spatial structure with an amplitude that varies in time. The more general approximation X̂(t_n) = (f&thinsp∘ s_f)[X(t_n)], with s_f and f nonlinear, is not so constrained, and can characterize more complex lower-dimensional structures. There is no reason in general to expect the optimal 1D approximation to a spatial field to be a standing oscillation, but standing oscillations are the only such approximations that traditional PCA can produce. The power of NLPCA lies in its ability to characterize more general lower-dimensional structure.

Figure 5 displays maps corresponding to the first NLPCA mode X̂⁽¹⁾, which corresponds to values of the time series α₁ = −3.5, −1.5, −0.75, −0.25, 0.25, 0.75, 1.5, 3.5. These values were chosen to provide a representative sample of spatial structures associated with the NLPCA approximation. Clearly, NLPCA mode 1 describes the evolution of average ENSO events. This should be contrasted with PCA mode 1, which describes only the standing oscillation associated with ENSO variability. This difference between NLPCA and PCA modes 1 results from the spatial asymmetry between the average warm and cold phases of ENSO. In particular, warm events described by NLPCA mode 1 display the strongest anomalies near the Peruvian coast, whereas the cold events are strongest near 150°W. This asymmetry in the evolution of NLPCA mode 1 arises because NLPCA mode 1 mixes PCA modes 1 and 2; for both El Niño and La Niña events, the PCA mode 2 spatial map (Fig. 2b) enters into the NLPCA mode-1 approximation with the same (negative) sign.

This spatial asymmetry between average El Niño and average La Niña events is manifest in a composite study. Figure 6a is a composite of November–December–January (NDJ) averaged SSTA for those years in which the NDJ Niño-3.4 index is greater than one standard deviation above the long-term mean; Figure 6b is the same for those NDJ for which Niño-3.4 is less than one standard deviation below the long-term mean. This averaging period was used for the composites because NDJ displays the largest variance of all 3-month seasons. These two maps correspond to the SSTA patterns of an average El Niño and an average La Niña, respectively. Note that, consistent with the maps corresponding to the 1D NLPCA approximation (Fig. 5), the largest SST anomalies tend to be located in the central Pacific during average La Niña events and centered in the eastern Pacific during average El Niño events. This asymmetry in the composite fields was previously noted by Hoerling et al. (1997).

The symmetric component of the composite El Niño and La Niña maps, as defined in appendix B, is displayed in Fig. 6c. This map, which in a rough sense characterizes the pattern in the composites that is related nonlinearly to the Niño-3.4 time series, bears a strong resemblance to SST EOF mode 2 (Fig. 2b). In fact, the spatial correlation between the two maps is −0.90. The antisymmetric component of the composite (not shown) bears a strong resemblance to EOF 1; the pattern correlation between these two is 0.975. Thus, PCA mode 1 may be interpreted as characterizing the component of average ENSO behavior that is antisymmetric between El Niño and La Niña events. By mixing EOF modes 1 and 2, NLPCA mode 1 is able to characterize the spatial asymmetry between average El Niño and average La Niña events. The bias of SST toward warm anomalies in the eastern part of the basin and toward cold anomalies in the western part is also evident in the study of Burgers and Stephenson (1999), who calculate the skewness of the observed SSTA distribution. It is interesting to note the striking similarity between their map of the spatial distribution of skewness (their Fig. 3a) and the symmetric component of the SSTA composite displayed in Fig. 6c.

A final comparison of the 1D NLPCA and 1D PCA approximations is given in Figs. 7a and 7b, which show, respectively, maps of the pointwise correlation between the original SSTA data and the 1D PCA approximation, and of the pointwise correlation between SSTA and the 1D NLPCA approximation. The two approximations are equally well correlated with the original data over the eastern-central half of the Pacific Ocean, except near the Ecuadorian coast, where the NLPCA correlations are somewhat higher than those of the PCA approximation. In the western Pacific, and in particular in the neighborhood of the EOF mode 1 zero line, the 1D NLPCA approximation displays a greater fidelity to the original data, as measured by the pointwise correlation, than does the 1D PCA approximation.

We now consider SSTA NLPCA mode 2, which was calculated using a network containing L = 3 neurons in the encoding and decoding layers. Figure 8 displays mode 2 projected onto the subspaces spanned by SSTA EOFs (e₁, e₂), (e₂, e₃), (e₁, e₃), and (e₁, e₂, e₃). The five candidate models from an ensemble of 20 trials differed from each other with an NMSD that was always less than 4%. NLPCA mode 2 explains 11.1% of the variance in the original SSTA data. The associated standardized time series, α₂(t_n), is shown in Fig. 9. Interestingly, the correlation coefficient between α₁(t_n) and α₂(t_n) is −0.06; the two time series are uncorrelated. Figure 10 displays maps of SSTA NLPCA mode 2, X̂⁽²⁾, corresponding to α₂ = −4, −1, −0.1, 0, 0.1, 0.2, 0.3, 0.4, 1. These values of α₂ were selected to produce a representative sample of maps describing NLPCA mode 2. When α₂ is strongly negative, SSTA NLPCA mode 2 is characterized by negative anomalies in the central and western Pacific and positive anomalies in the eastern Pacific. As α₂ increases through zero, the anomalies decrease in magnitude, while the positive anomalies in the eastern part of the basin become increasingly concentrated in the equatorial region. Eventually, the region of positive SSTA breaks off from the coast of South America and migrates into the central Pacific. As α₂ increases further, the SSTA pattern becomes the opposite of that for α₂ near zero, with positive anomalies in the central and western Pacific and negative anomalies in the east. Finally, for α₂, near the extreme positive part of its range, SSTA NLPCA mode 2 is characterized by negative anomalies along the equator, extending to the date line, with positive anomalies throughout the rest of the basin. Because the anomalies are often concentrated along the equator, it is reasonable to associate this mode with some aspects of ENSO variability not captured by mode 1. Indeed, it is interesting to note from Fig. 9 that NLPCA mode 2 is more active in the later part of the record than in the earlier. The two strong minima in α₂(t_n) coincide with the decay phases of the large El Niño events of 1982–83 and 1997–98, describing the lingering patches of warm water in the eastern tropical Pacific observed during these periods. Two of the three weaker minima in the post-1977 period are associated with the peaks of the La Niña events of 1984–85 and 1988–89. This is consistent with the fact that the cold anomalies during La Niñas in this later period are somewhat stronger, and more concentrated in the central Pacific Ocean and weaker in the eastern Pacific Ocean, than La Niña events in the earlier part of the record, as indicated by a composite analysis (not shown).

A number of studies have noted a shift in ENSO variability in 1977 (e.g., Wang 1995). The apparent nonstationarity of α₂(t_n) is consistent with a shift at this time, although the precise timing of the shift in α₂(t_n) is not obvious. The 1977 shift is, in fact, manifest in the SSTA NLPCA mode 1 time series α₁ (Fig. 4); the time series is biased toward negative extrema before 1977 and toward positive extrema after 1977. However, it should be noted that this nonstationarity in variance may simply reflect the inclusion, from November 1981 onwards, of satellite data in the Smith et al. dataset (Smith et al. 1996).

Thus, the first mode of the modal NLPCA decomposition of tropical Pacific SSTA describes the average variability associated with the ENSO phenomenon and nicely characterizes the asymmetry in spatial structure between average El Niño and average La Niña events. The second mode characterizes some differences in evolution between individual ENSO events, and in particular, displays a nonstationarity consistent with the observed 1977 “regime shift” in ENSO variability.

Plots of a 2D nonmodal NLPCA approximation of the SSTA data (i.e., using two neurons in the bottleneck layer), projected in the subspaces spanned by SSTA EOFs (e₁, e₂), (e₂, e₃), (e₁, e₃), and (e₁, e₂, e₃), are shown in Figs. 11a–d. The associated network used L = 6 neurons in the encoding and decoding layers, and the NMSD between candidate models (8 out of an ensemble of 20) varied between 1% and 3%. The 2D nonmodal NLPCA approximation explains 79.0% of the variance in the truncated dataset, and thus 72.2% of the variance of the original data. The time series corresponding to the output of the bottleneck layers (not shown), denoted (β₁, β₂)(t_n) = s_f[X(t_n)], are highly correlated with each other (r = −0.835) and with the Niño-3.4 index (r₁ = −0.879 and r₂ = 0.889, respectively). Because the 2D nonmodal NLPCA depends on two parameters, β₁ and β₂, it is difficult to visualize the results using a sequence of maps as we did with the modal NLPCA in Figs. 5 and 10. Figures 12a and 12b display maps of the pointwise correlation coefficient between the SSTA data and the 2D PCA and 2D nonmodal NLPCA approximations, respectively. The 2D nonmodal NLPCA approximation produces higher correlations than the 2D PCA approximation in the central equatorial, western, and southeastern Pacific, and slightly lower correlations in the eastern equatorial Pacific.

It is worth considering the time series β₁(t_n) and β₂(t_n) in more detail. As was pointed out by Malthouse (1998), the parameterization s_f[X(t_n)] of the P-dimensional surface determined by NLPCA is only defined up to an arbitrary homeomorphism (i.e., a continuous, one-to-one, and onto function with continuous inverse). That is, for an arbitrary homeomorphism g: ℜ^P → ℜ^P, the time series g{s_f[X(t_n)]} is also an acceptable parameterization of the surface, because f ∘ s_f = (f ∘ g⁻¹) ∘(g ∘ s_f).; In particular, for any homeomorphism g: ℜ² → ℜ², [g₁(t_n), g₂(t_n)] = g[β₁(t_n), β₂(t_n)] parameterizes the surface found by 2D nonmodal NLPCA. Which parameterization is determined by the NLPCA algorithm is a matter of chance. This degeneracy complicates the interpretation of the time series determined by nonmodal NLPCA. In particular, the time series β_i(t_n) may not be independent, or they may not even be uncorrelated.

Determining a set of P independent variables γ_i parameterizing the surface from the set of P time series β_i(t_n) determined empirically by NLPCA is a problem of feature extraction in the space of the variables parameterizing the surface. Thus, PCA or modal NLPCA can be used to calculate the γ_i(t_n). In the case at hand, inspection of the scatterplot of β₁(t_n) with β₂(t_n) (not shown) indicated that PCA was appropriate for separation of the correlated time series β₁(t_n) and β₂(t_n) into two uncorrelated time series γ₁(t_n) and γ₂(t_n). The homeomorphism g is thus simply a linear function. The first PCA mode explained 92.7% of the variance in β space, and the associated time series (not shown) describes ENSO variability. Its correlation coefficient with the Niño-3.4 time series is 0.92 and with α₁(t_n) is 0.87. The second PCA mode explains the remaining 7.3% of the variance in β space, and the associated time series γ₂(t_n) (not shown) is rather similar to α₂(t_n); the two time series have a correlation of 0.7. In particular γ₂(t_n) displays the same difference in activity between the pre-1977 and the post-1977 periods as does α₂(t_n), with the same prominent peaks appearing in both time series. The parameterization [β₁(t_n), β₂(t_n)] thus contains essentially the same information as the two time series α₁(t_n) and α₂(t_n). This extra step of processing required to allow interpretation of the time series produced by nonmodal NLPCA indicates a distinct disadvantage of nonmodal NLPCA as compared to modal NLPCA.

The 2D modal NLPCA approximation to the SSTA data (not shown), obtained by adding the leading two 1D NLPCA approximations, strongly resembles the 2D nonmodal NLPCA approximation displayed in Fig. 11. Because of the variance-partitioning property of NLPCA, the fraction of variance explained by this approximation is the sum of the fractions explained by the two 1D modal approximations, that is, 74.4% (slightly greater than that obtained with the 2D nonmodal approximation). This approximation differs in detail from the nonmodal approximation displayed in Fig. 11, but the two agree broadly in their general features. Figure 12c displays a map of the pointwise correlation coefficient between the 2D modal NLPCA approximation and the original SSTA data; correlations are somewhat higher than those of the 2D nonmodal approximation in the western Pacific Ocean and somewhat lower in the eastern equatorial Pacific, but by and large the differences between the two correlation maps are small.

Note that, while in principle we would expect a 2D nonmodal NLPCA approximation to be better able to characterize general low-dimensional structure than a 2D modal NLPCA approximation, in the case of SSTA we find that the former explains 72.2% of the variance, while the latter explains 74.4%. In fact, the model corresponding to the modal NLPCA had 13% more free parameters than that corresponding to the nonmodal NLPCA. We suspect that this allowed the modal model more flexibility than the nonmodal, leading to the slight improvement in the fraction of variance explained.

Thus, we have seen that in both 1D and 2D, and for both modal and nonmodal approaches, NLPCA produces approximations of greater fidelity to the tropical Pacific SSTA data than does PCA. In particular both the 1D PCA and 1D NLPCA approximations describe“average” ENSO variability, but the 1D modal NLPCA approximation was able to characterize the spatial asymmetry between average El Niño and La Niña events in a fashion that 1D PCA cannot. Note that there are considerable differences between the structure and evolution of individual El Niño and La Niña events. No claim is made that any of the 1D or 2D SSTA NLPCA approximations captures the full range of this variability. Indeed, Penland and Sardeshmukh (1995) find that an embedding space of 15 PCA modes is needed for their linear inverse model SSTA forecasts. As an N-dimensional manifold requires a Cartesian embedding space of at most 2N + 1 dimensions, to prevent spurious self-intersections, it would seem that a manifold of a dimensionality of at least 7 is required to capture all aspects of ENSO variability. This study presents low-dimensional estimates of this presumed ENSO attractor.

5. Tropical Indo–Pacific sea level pressure

As a second application of NLPCA to a dataset of climatological significance, we consider the Comprehensive Ocean–Atmosphere Data Set monthly averaged see level pressure (SLP) over the tropical Indo–Pacific area (Woodruff et al. 1987), on a 2° × 2° grid from 27°S to 19°N, and from 31°E to 67°W, covering the period from January 1950 to December 1998. The annual cycle was removed in the same fashion as the SST data to produce sea level pressure anomalies (SLPAs). The SLPA field was then smoothed in time using a 3-month running mean filter and in space with a 1–2–1 filter in each spatial direction. The resulting smoothed SLPA field was then projected onto the space spanned by its 10 leading EOF modes, which together explain 60% of the total variance in the data.

Figure 13 displays the 1D NLPCA approximation of the SLPA data projected on the subspaces spanned by SLPA EOFs (e₁, e₂), (e₂, e₃), (e₁, e₃), and (e₁, e₂, e₃). This curve was obtained using a network with L = 2 neurons in the encoding and decoding layers; the NMSD between the eight candidate models from an ensemble of 50 ranged between 0.1% and 0.3%. The 1D NLPCA approximation explains 27.0% of the total variance in the SLPA data; this is only a slight improvement over the fraction of variance explained by the 1D PCA approximation. Figure 14 displays plots of α₁(t_n), the standardized time series associated with the 1D NLPCA approximation, and the Southern Oscillation index (SOI), calculated by subtracting the SLPA at Darwin, Australia (12°S, 131°E) from that at Tahiti (17°S, 149°W), and then applying a 3-month running average smoother. The two time series bear a strong resemblance to each other on interannual and longer timescales; their correlation coefficient is 0.7. The 1D modal NLPCA approximation thus seems to describe ENSO variability in the SLPA field. This association is reinforced by inspection of the sequence of maps of X̂⁽¹⁾ for α₁ = −3, −2, −1, −0.5, 0, 0.5, 1, 2 (Fig. 15). This sequence of spatial patterns describes the east–west SLPA dipole associated with average Southern Oscillation variability. Figure 16 displays composites of SLPA averaged over those December–January–February (DJF) periods in which the SOI was less than 1 standard deviation (Fig. 16a) below the long-term average or was more than 1 standard deviation above the long-term average (Fig. 16b). This 3-month period was selected because, of all 3-month seasons in the record, it displayed the greatest variance. Figures 16a and 16b represent average El Niño and La Niña patterns, respectively. Clearly, the maps in Fig. 15 for α₁ < 0 correspond to the El Niño composite and those for α₁ > 0 correspond to the La Niña composite. Comparison of Figs. 15 and 16 indicates that the 1D NLPCA approximation characterizes an asymmetry between average El Niño and average La Niña events, particularly in the eastern half of the domain. As was the case with SSTA, maps of pointwise correlation between the data and the 1D PCA and 1D NLPCA approximations (not shown) indicate that, particularly around the nodal line of the 1D PCA approximation, the 1D NLPCA approximation displays a higher degree of fidelity to the original data.

In calculating the second mode of the modal NLPCA decomposition of the SLPA data, it was determined that only for l = 1 neuron in the encoding and decoding layers could robust results be obtained. Neural network–based NLPCA can only find nonlinear structure if there are two or more neurons in the encoding and decoding layers. In other words, the optimal 1D characterization of the residual data, obtained by subtracting from the original SLPA data the 1D NLPCA approximation, is a straight line. The spatial pattern and associated time series of this mode (not shown), which explains 15.9% of the variance of the original data set, strongly resemble those of SLPA PCA mode 2. The correlation between the two time series is 0.96, and the pattern correlation between the two spatial patterns is 0.93. Thus, with the dataset available, no robust lower-dimensional nonlinear structures could be found in these residuals. Furthermore, the results of a 2D nonmodal NLPCA of the SLPA data did not yield particularly interesting results.

We conclude that, apart from a weakly nonlinear 1D NLPCA approximation corresponding to ENSO variability and characterizing a slight asymmetry between average El Niño and average La Niña events, the robust low-dimensional structure of this SLPA data is linear.

6. Summary and conclusions

Traditional PCA is a powerful tool for the detection and characterization of low-dimensional linear structure in multivariate datasets. However, when a dataset contains substantial nonlinear low-dimensional structure, the results of PCA are suboptimal. We have investigated the application of a nonlinear generalization of PCA, denoted NLPCA, to two datasets of climatic significance: tropical Pacific SST and tropical Indo–Pacific SLP.

Considering first the SST data, it was found that a 1D NLPCA approximation, which we denote NLPCA mode 1, explains 63.3% of the total variance in the SST field, in contrast to 57.6% for the first PCA mode. That this 1D approximation describes average ENSO variability is clear upon inspection of both the time series α₁(t_n) (Fig. 4a) and the sequence of spatial maps (Fig. 5) characterizing the approximation. PCA mode 1 also characterizes average ENSO variability, but only its linear component, so it is unable to describe the asymmetry in spatial pattern between average warm and cold events manifest in the 1D NLPCA approximation and in a composite analysis. NLPCA improves on PCA by allowing low-dimensional approximations to have a structure other than that of simple standing oscillations. While both the 1D NLPCA approximation and the composite analysis describe the asymmetry between average warm and cold ENSO events, NLPCA has the advantages that it does not require the a priori specification of a time series over which to composite and that it provides a full 1D approximation to the data, in contrast to the 0D approximation produced by the composite analysis. SSTA NLPCA mode 2 explains 11.1% of the SSTA variance and characterizes some aspects of ENSO variability not described by SSTA NLPCA mode 1. In particular, a striking temporal nonstationarity in the variability of this mode is consistent with the difference in variability between pre-1977 and post-1977 La Niña events discussed in a number of recent studies (e.g., Wang 1995). A 2D nonmodal NLPCA approximation is found to explain 72.2% of the total SSTA variance, in contrast to 68.5% for a 2D PCA approximation and 74.4% for a modal 2D NLPCA approximation. It is difficult to visualize a 2D nonmodal NLPCA approximation, but pointwise correlation maps comparing the spatial performance of the 2D PCA approximation, the 2D nonmodal NLPCA approximation, and the 2D modal NLPCA approximation (Fig. 12) indicate that the nonlinear approximations are not too different, one from the other, and that they characterize variability in the western part of the Pacific basin better than the linear approximation does. However, as the time series of variables parameterizing the surface produced by nonmodal NLPCA are only determined up to an arbitrary homeomorphism, complicating their interpretation, a secondary feature extraction problem using PCA had to be carried out to recapture from the nonmodal analysis the same information present in the time series produced by a modal analysis. This indicates a significant disadvantage of nonmodal NLPCA as compared to modal. Taken together, the results presented in section 4 demonstrate that the tropical Pacific dataset considered contains nonlinear lower-dimensional structure that can robustly be characterized by NLPCA.

In section 5, it was seen that the tropical Indo–Pacific SLPA dataset was primarily characterized by linear low-dimensional structure. A 1D NLPCA approximation was found to explain 27% of the total variance, in contrast to 24.2% for the 1D PCA approximation. This mode describes average ENS0 variability in the SLPA field and characterizes the asymmetry in spatial pattern between average El Niño and average La Niña events. No robust nonlinear second NLPCA mode could be found, indicating that the SLPA data are dominated by linear structure.

This analysis has been a preliminary investigation of the application of NLPCA to climatic datasets. Because of its ease of implementation, early stopping with an ensemble of models was used to ensure the robustness of the results of NLPCA to the introduction of new data and to the initial parameter values used in network training. Other, more sophisticated, regularization techniques involving the modification of the cost function can be implemented to automate the determination of model parameters such as L (the number of neurons in the encoding and decoding layers) and to eliminate the necessity of considering an ensemble of candidate models (Yuval 2000). Such modifications of the model building procedure will presumably be implemented in future applications of NLPCA to the analysis of climate data.

The operation of feature extraction plays a significant role in the statistical analysis of climatic datasets. Generally, there is no a priori reason to expect that any low-dimensional structure underlying a multivariate dataset should be linear, and thus no reason to expect traditional PCA to provide an optimal characterization of this lower-dimensional structure. NLPCA provides a natural generalization of PCA to the nonlinear feature extraction problem. As we have demonstrated using tropical Pacific SST anomaly data, NLPCA can provide insight into the structure of a dataset that PCA could not. Because of its complexity we do not feel that NLPCA will replace simpler traditional approaches such as PCA, but that it will serve as another useful method in the climate statistician’s tool box.