1. Introduction

Suppose that each station within a geographical domain has a data record (a time series). Our goal is to group (cluster) these series into disjoint regions (climatic zones) from this data. Two series from the same region should be more similar (in general) than series from different regions. Essentially, the problem involves clustering of time series.

Clustering is the standard method used to define climate zones. Stooksbury and Michaels (1990), Fovell and Fovell (1993), Gong and Richman (1995), DeGaetano (1996, 2001), Fovell (1997), Bunkers et al. (1996), Yao (1997), Gerstengarbe et al. (1999), Böhm et al. (2001), Steinbach et al. (2003), Unal et al. (2003), and Vrac and Naveau (2007) use clustering methods to construct climate zones. Clustering methods are also useful for classifying weather conditions into different synoptic regimes (Kalkstein et al. 1987; Davis and Walker 1992; Michelangeli et al. 1995; Kidson 2000; Stephenson et al. 2004; Straus et al. 2007). Complementing this climate work, the problem of clustering autocorrelated zero mean series has been recently considered in the statistics literature. In particular, Coates and Diggle (1986), Kakizawa et al. (1998), and Lund et al. (2009) show how to test whether two zero mean time series have the same autocovariance structure. Of these three references, only Kakizawa et al. (1998) consider more than two series. Zero mean series with differing autocorrelations often arise in financial and speech settings. Indeed, speech series can effectively be discriminated through their autocovariance structures. Unfortunately, speech classification methods are not directly applicable to our climate clustering problem, where the first moment (or mean) is the most prominent series feature. On the other hand, the second moment (autocovariances) also provides information: two stations that are truly similar should have similar means and autocovariances. As we show later, ignoring autocorrelations results in a suboptimal method.

Another important practical issue lies with seasonality. Most climate series possess seasonal structures. Seasonality has been handled in various ways by previous authors, but our treatment is general in that our model allows for seasonal means, variances, and autocorrelations. Differences in the means of two series, even if only seasonal, are a useful quantity to discriminate from. Because of this, we do not advocate clustering anomalies where the mean (seasonal or nonseasonal) has been removed. This issue will be revisited in section 5 below.

In this paper, we develop a clustering method that examines the means and autocovariances of the series in tandem. Such a method is based on the one-step-ahead prediction errors for general time series models. In fact, since the scaled one-step-ahead prediction errors of Gaussian time series are independent and identically distributed (IID), the proposed method is essentially an optimal clustering method for IID data. The methods can handle covariate and/or trend components if desired.

The rest of this paper proceeds as follows: Section 2 proposes a distance that measures how far apart two time series are. This distance accounts for the mean and autocovariances (in tandem) in a natural way. As a by-product of the analysis, a statistical test for whether two series should serve as reference stations for one another is obtained. Section 3 discusses how to cluster a collection of series into a prespecified number of regions from the section 2 distances. Section 4 provides a short simulation study of the reference station test aspects of this study. Section 5 illustrates the techniques by clustering 292 temperature records from the state of Colorado. Section 6 closes with conclusions and comments.

2. A new distance

Clustering techniques require a distance measuring the similarity between any two time series. In this section, our goal is to develop such a distance that accounts for means and autocovariances simultaneously. Our attention here is on two series X = {X_t} and Y = {Y_t} representing two distinct stations in the domain of study. The distance between X and Y, denoted by d(X, Y), should be small when the two series are similar and large when they are dissimilar. Any distance used should obey the general properties of distances. From Mardia et al. (1979), a function d is a distance if it is

1. symmetric, where d(X, Y) = d(Y, X);

2. nonnegative, where d(X, Y) ≥ 0; and

3. identification marking, where d(X, X) = 0.

   For many distances, the following properties also hold:

4. definiteness, where d(X, Y) = 0 if and only if X = Y; and

5. the triangle inequality, where d(X, Y) ≤ d(X, Z) + d(Z, Y).

The distance that we propose for two series of length N has the form

View Expanded

where a hat indicates a one-step-ahead prediction. This distance is computed under a statistical null hypothesis that X and Y have the same first two moments: E[X_t] = E[Y_t] for all t and cov(X_t, X_s) = cov(Y_t, Y_s) for all t and s. Specifically, the same model form must be fitted to both X and Y.

For the other quantities in (2.1), X̂_t = P(X_t|1, X₁, … , X_{t − 1}) is the best linear prediction of X_t from a constant (this allows mean effects into the prediction) and the “observed past” X₁, … , X_{t − 1} (this allows for second-order, or autocovariance, effects) and υ_t = E[(X_t − X̂_t)²] is its squared prediction error. The divisor υ_t^1/2 is needed to account for the possibility of seasonal variances; for example, one-step-ahead temperature predictions have larger errors during winter months than in summer. Scaling by υ_t^1/2 ensures that each observation contributes equally to the distance. An example is given below that shows how to compute X̂_t and υ_t for monthly temperature series.

The key fact driving the above comes from time series: regardless of the mean and correlation structures of {X_t}, the scaled error sequence {S_t} defined for each fixed t via S_t = (X_t − X̂_t)/υ_t^1/2 is zero mean white noise with a unit variance. This means that S_t and S_s are uncorrelated when t ≠ s, E[S_t] ≡ 0, and var (S_t) ≡ 1. Brockwell and Davis (1991, chapter 5) is a good reference for time series prediction theory. By examining the one-step-ahead prediction residuals, we are essentially transforming a clustering problem with a time-varying mean and autocorrelated errors into one with zero mean and uncorrelated errors (and the latter is a well-studied problem of statistics; see Hartigan 1975).

The distance in (2.1) obeys all five properties of distances listed above. This is not the only distance metric that can be used; in fact, Moeckel and Murray (1997), Maharaj (2000), Boets et al. (2005), and Bengtsson and Cavanaugh (2008) provide alternative choices. One can link the distance in (2.1) to likelihood ratio tests for Gaussian series, but we will not pursue such aspects here. A very convenient facet of the distance in (2.1) is that it scales into a chi-squared test to assess the equality of two time series (whether or not they can serve as references for one another). We now elaborate on this test.

For Gaussian {X_t}, the distribution of S_t is standard normal (exactly) for each t. In cases where unknown regression (mean) and time series (second moment) parameters are estimated, {S_t} is approximately zero mean unit variance Gaussian white noise, the approximation becoming better as N increases. But more can be said. Under the null hypothesis that the two series have the same dynamics (i.e., the two series are able to serve as references for one another), d(X, Y) is distributed as χ_N²/N: a chi-squared random variable with N degrees of freedom divided by N. If the null hypothesis is false, then d(X, Y) will be larger than can be statistically explained by a χ_N²/N random variate when the same model is fitted to both X and Y. This distributional statement is not overly sensitive to the Gaussian assumption for large N. Hence, one rejects that X and Y are good reference stations for one another at confidence level (1 − α) × 100% if

View Expanded

where is the percentile of the chi-squared distribution with N degrees of freedom that cuts off an upper tail area of α. For example, when α = 0.05 and N = 40, = 26.509. For N > 40, typically the practical case, Devore and Berk (2007) suggest the approximation

View Expanded

where z_α is the αth critical value from the standard normal distribution (e.g., z_α = 1.645 when α = 0.05).

Our next task is to fit good time series models to our series of interest and show how to compute d(X, Y) from these time series fits. This is tedious but essential. Periodicities and covariate effects will be considered. Let T denote the period of the series (e.g., T = 12 for monthly data and T = 365 for daily data). Our seasonal notation takes X_nT+ν as the observation during the νth season of cycle n in X. Here, ν is a seasonality suffix satisfying 1 ≤ ν ≤ T. For example, with monthly data, ν = 1 refers to January and ν = 10 to October.

Our rudimentary model for X is

View Expanded

Here, f is a trend function, μ_ν is the mean (detrended) for season ν, and {ϵ_t} is a zero mean periodically stationary time series of errors (see Lund et al. 1995 for an overview of periodic time series in climate modeling). The trend function f may contain a covariate component consisting of perhaps El Niño–Southern Oscillation (ENSO) or North Atlantic Oscillation (NAO), a simple linear trend component, or all of the above. For example, the model

View Expanded

with T = 12 contains a time-homogeneous simple linear trend, a location parameter μ_ν representing the month ν temperature (detrended), and a covariate effect for the North Atlantic Oscillation. In this case, f (t) = αt + κ NAO_t. While choosing the appropriate regression function is an art, we comment that the same functional form must be fitted to all series within the domain.

Once the regression model form is chosen, one must estimate all regression and time series parameters in the model for both stations. From these estimates, the distance between the two stations can be computed. Computation of the quantities needed to evaluate the distance is discussed in the appendix.

As a simple example of the above methods, we consider 54 yr of monthly temperatures from Athens and Atlanta, Georgia. Both stations lie in the Piedmont region of north Georgia and are approximately 75 miles apart. There are N = 648 observations from both stations.

Empirical computations do not reveal a trend in these series. As we do not consider covariates, the trend function fitted is simply f (t) ≡ 0. The time series model fitted to the errors is a first-order periodic autoregression with mean μ_ν, autoregressive parameter ϕ_ν, and white noise variance σ_ν² during season ν for 1 ≤ ν ≤ T. This model is explained further in the appendix here or in Lund et al. (1995).

The distance computed between Athens and Atlanta is d = 0.075. Using the chi-squared distribution in (2.2), we obtain a p value of approximately 1.000, strongly suggesting that Athens and Atlanta are very good reference stations for one another. Figure 1 plots monthly estimates of μ_ν, ϕ_ν, and σ_ν² for each station. As these parameters are virtually identical, we conclude that that Athens and Atlanta indeed have similar means and autocovariances. One could use this information to calibrate a new gauge or fill in missing data.

This completes our description of the distance function behind the clustering. Our next section shows how to cluster the series into zones given all their pairwise distances.

3. Clustering the stations

This section discusses how to take the section 2 distances and define a fixed number of zones (say K) out of M total stations. Here, the zones will be referred to as clusters and are simply subsets of stations. While many different clustering methods exist, we will focus on hierarchical agglomerative methods for simplicity as these perform reasonably well on a variety of climate data structures. Other clustering techniques, many of which are prevalent in synoptic classification, include Ward’s minimum variance, centroid, and K means (see Gong and Richman 1995 and Wilks 2006 for overviews).

Henceforth, we consider three widely used agglomerative clustering methods: nearest-neighbor single linkage, furthest-neighbor complete linkage, and average linkage. These methods all start with each station serving as its own cluster. Next, the two stations having the smallest distance are identified and combined into a cluster of two stations. At this point, there are M − 1 clusters. At each successive step in the algorithm, the two clusters having the smallest distance between them are merged into a single cluster (this between-cluster distance is clarified below). The step at which there are K clusters is the grouping that we seek.

These three methods differ only in how they define the distance between two clusters. The distance between the clusters C₁ and C₂, denoted by d(C₁, C₂), uses the distance in section 2 in the following ways. Single linkage methods define cluster distances as the minimum distance between any two stations in the respective clusters:

View Expanded

where X_i is the series from any station in cluster C₁ and X_j is the series from any station in cluster C₂. Complete linkage methods use the maximum distance

View Expanded

and average linkage methods compromise the former two by taking the average distances between all pairs of stations in the cluster:

View Expanded

Here, #(C_i) is the number of stations in cluster C_i. These three methods share common properties (e.g., the optimal clusters do not change under monotone transformations of the distance measure), but each method has nuances in dealing with ties and in algorithmic implementation (Mardia et al. 1979 is a good reference). Single linkage methods have been the least popular and average linkage methods the most popular (Wilks 2006). However, as there is no a priori reason to prefer any of them, we try all three methods. Once distances between all pairs of stations are calculated and stored in a matrix, clustering algorithms can be quickly run. We refer the reader to Hartigan (1975) for algorithmic details.

4. A simulation study

This short section presents a simulation study of the reference station test proposed in section 2. Here, we will generate synthetic series with known statistical properties. Because the series are simulated, this allows us to preclude changepoint effects from influencing the results. To elaborate, station relocation times or instrumentation changes can induce level shifts into climatic time series and produce spurious results if applied to the unadjusted (raw) series (see Lund and Reeves 2002 for examples).

Our simulations will examine monthly data (T = 12) over a century of operation (c = 100). Each simulation is drawn from a first-order periodic autoregressive model with a periodic mean; specifically, the seasonal means satisfy

View Expanded

and the seasonal autoregressive parameters are [see (A.1) for clarification]

View Expanded

For simplicity, we do not consider trend or covariate effects; that is, f (t) ≡ 0. Also, the seasonal white noise variances are taken as unity: σ_ν = 1 for each season ν ∈ {1, … , T}. We will consider five different parameter sets in (4.1) and (4.2):

View Expanded

Here, case I entails a situation where the average annual temperature is 60°, average temperatures fluctuate between 30° and 90°, and there is moderate positive correlation in the error process {ϵ_t}. Case II is similar to case I except the seasonal mean cycle is less pronounced. Case III has the same dynamics as case I except that the station is located in a 1° colder setting (in each and every month). Case IV uses the same seasonal means as case I but takes heavier (stronger) autocorrelations in the errors. Case V is like case IV except that there is no seasonal cycle in the autocorrelations. In truth, each of the five cases are different; hence, an ideal test would reject the notion that any station could serve as a reference for any other station. Figure 2 plots a synthetic draw from each of the five cases. Visually, it is difficult to distinguish some of these series from one another.

Table 1 below summarizes results from 10 000 simulated series from each station. On each of the 10 000 replications, series from the five stations are first generated, one each from cases 1–5, respectively. We then compare the distances between each pair of series (there are 10 comparisons to make) with α = 0.05. Table 1 shows the sample proportion of times (out of 10 000) that the test in (2.2) rejects that the stations can be used as references for one another.

The sample proportions show that cases I, II, and III are easily differentiated. The test has worked well (perfectly in fact) when one station’s seasonal mean differs. Differences in autocovariances are harder to detect. This said, case I and case IV, where true differences lied only with the autocovariances, were correctly differentiated 78% of the time. Case I and case V are very close in statistical structure, with the only difference being the slight oscillation in ϕ_ν in case I. The test correctly discriminates between these two cases only 5.3% of the time.

We also studied the so-called type I error (also called a false alarm rate) of our reference station test. Here, the 10 000 pairs of series from case I were generated and were compared with a 95% test. The proportion of times the test rejected that the station pairs could serve as references for one another was 0.0317, slightly (but not too far) below the 0.05 level it should be. The reason that the proportion is slightly lower than 5% lies with time series parameter estimation. As the sample sizes increases, these proportions will move closer to 5%. For cases II–V, the type I errors are 0.0304, 0.0299, 0.0219, and 0.0292, respectively, which are diagonal elements of Table 1.

It is instructive to compare our results to those obtained with the simple squared Euclidean distance

View Expanded

used by Fovell and Fovell (1993) and DeGaetano (1996). The distance in (4.4) is similar to that in (2.1) except that it neglects the seasonal means and autocovariances in the data. Table 2 summarizes results from 10 000 simulations with the same specifications as in cases I–V above. One sees very large rejection probabilities, which are desirable, on the off-diagonal entries. However, the type I errors on the diagonals pertain to cases where the series are simulated from the same dynamics; here, the rejection rates should be close to 5%. Hence, the test based on squared Euclidean distances is essentially equivalent to a test that simply rejects station equality all of the time (which is unacceptable).

With the reference station test performing well, we now move to the task of clustering a geographic region with our new distance. Before this, we make a comment. It is possible for two stations to be in the same cluster and not be good references for one another. Indeed, clustering simply minimizes the within-cluster heterogeneity; in extreme cases, one could reject that every pair of stations within a cluster are adequate references for one another.

5. Colorado climate zones

This section examines how clustering techniques based on (2.1) work in a setting with several distinct climatic zones: the state of Colorado. Colorado is composed of high plains in its eastern regions, mountains in its center and western parts, and canyon lands, mesas, and steppes on its western flanks. As noted in Bengtsson and Cavanaugh (2008), establishing climate zones in Colorado is difficult because of its widely varying topography and elevations. An accurate clustering of Colorado would help delineate agricultural and forestation zones—plants and animals that thrive at one station should thrive at other stations within the same zone. Thornthwaite (1931) lists many other reasons to pursue a rigorous definition of climate zones.

Monthly maximum and minimum temperatures for 292 cooperative Colorado weather stations were downloaded from the Web site http://www.image.ucar.edu/Data/US.monthly.met/FullData.shtml#precip. At each station, the monthly average temperature is the simple average of the monthly maximum and minimum temperatures. The data span 1895–1997. This dataset contains 270 more stations than the 22 analyzed in Bengtsson and Cavanaugh (2008).

Some of the 292 stations appear to have trends. For simplicity, no covariate effects are considered. Hence, the trend model is f (t) = αt at each station. First-order periodic autoregressive error models, as discussed in the appendix, are fitted as the error component at each station.

In many applications, the number of zones (clusters) is a priori determined or is estimated via an ad hoc “stopping rule” such as the pseudo-F criterion (Calinski and Harabasz 1974). As our advance lies with improving the distance between different time series, we set the number of clusters as six, keep this parameter fixed, and focus on the differences between our method and squared Euclidean distance methods. The differences between the methods for other cluster numbers are similar.

Figures 3 –5 graphically portray the results by plotting the cluster number over each station’s location. The two panels compare the distance in (2.1) (top panel) to the squared Euclidean distance (bottom panel). The three figures report clusterings for single linkage, complete linkage, and average linkage methods, respectively.

The single linkage plots in Fig. 3 are problematic because one cluster contains all but five of the stations. This is attributed to the fact that single linkage methods use the minimum of distances between two stations in the respective clusters as the within-cluster distance: when a cluster contains many stations, it is likely that some station in the cluster will match the “station in question” well. Several of the stations are so different from all others that the clustering algorithm isolates them as “singleton” clusters. This drawback applies to both the new and squared Euclidean distances. These five different stations were omitted and the clustering was redone, but the single linkage clustering simply found five new singleton clusters.

The complete and average linkage clusterings in Figs. 4 and 5 are more appealing in that they are less degenerate. The overall patterns in these two figures are very similar, but there are also some differences. The patterns in Figs. 4 and 5 clearly identify the eastern plains and the San Juan, San De Cristo, Gore, and Front Mountain Ranges in the state’s center and southwestern locales, and the scattered canyonlands in the west. The grasslands in the southeastern corner of the state (Baca County in particular) are also apparent. There are subtle differences between the results from the two distances. For example, the two average linkage plots in Fig. 5 for Larimer County in the north-central part of the state differ. This county has five different zones under our distance, but only four zones under a squared Euclidean distance. The station responsible for this discrepancy is the Willow Park station in Rocky Mountain National Park (labeled WP in the graphic), which resides at the extreme elevation of 10 702 feet. Hence, there is reason to believe that the additional zone is warranted [i.e., the distance in (2.1) is preferable]. In Fig. 4, Mineral County in the southwest part of the state sees disagreement. Here, squared Euclidean distances imply that the six stations in the county all belong to the same zone while the distance in (2.1) prefers two zones. Given that the stations in this county include the Wolf Creek Ski Area (Colorado’s snowiest ski resort, labeled WC in the graphic) and Wagon Wheel Gap (a town at a much lower elevation, labeled WW in the graphic) two zones for this county seem more realistic. Whether to use the complete or average linkage clusterings is always debatable. Unless there is a good reason, we suggest the average linkage clustering simply because averages are more stable than maximums.

To explore the differences noted above in Larimer County further, we will examine the Longs Peak (LP in Fig. 5) and Willow Park stations. The Longs Peak station resides at 9005 feet, some 1697 feet below the Willow Park station. For average linkage methods, the squared Euclidean distance puts these stations in the same zone while the new distance in (2.1) puts them in different zones (see Fig. 5). The reference station comparison test in section 2 for these two stations has d = 3.214, with a p value of approximately zero. Hence, statistical evidence suggests that different zones are warranted.

It is worth comparing the results to conventional methods. Figure 6 shows the resulting clustering if a sample mean is subtracted from the series before analysis (this is one common definition of an anomaly). Specifically, Fig. 6 shows an average linkage clustering with a Euclidean distance applied to the “grand mean adjusted series” {X_t − X}, where X is the station average of all series values. No trend or covariates were considered here. Observe that a significant amount of structure is lost: the clustering places many mountainous stations in the same zone as plains stations. Figure 7 shows an average linkage Euclidean distance clustering of the monthly adjusted anomalies {X_{nT + ν} − μ̂_ν}. This clustering places almost all stations in the same zone and hence appears suboptimal. Figures 5 –7 show why one does not want to cluster anomalies—removing the mean has resulted in a loss of information.

The results have data collection implications. For instance, Fig. 8 depicts relationships between 9 stations in Larimer County. Specifically, a line is drawn to connect any pair of stations that pass as being equivalent at the 95% level with the previously discussed chi-squared test. The graphic shows that station 8, which is the previously encountered Willow Park station, is different than all other stations in the county. On the other extreme, station 2 tests as equivalent to stations 1, 3, 4, 5, and 9. Because station 2 is connected to the most other stations, this station seems to record the most redundant data and would be the best candidate to omit if one was forced to discontinue data collection at some station within the county.

6. Conclusions and comments

The methods here enable one to more accurately define climate zones. By considering the seasonal mean cycle and autocovariances of the data from stations being clustered, improvements were made to techniques that employ squared Euclidean distances. As a by-product of the methods, a statistical test for whether two stations can serve as references for one another is obtained.

We close with a comment. First, the methods here are univariate in that only temperatures were considered. Multivariate extensions of the methods are currently being studied and could improve the resulting clusterings further. The necessary arguments for such an endeavor are reasonably straightforward for Gaussian data. However, as precipitations are decisively non-Gaussian, several nuances may need to be addressed to obtain a method that performs optimally. Even in multivariate Gaussian settings, we do not recommend standardizing each component series by subtracting a mean and dividing by a standard deviation. Indeed, one of the components may be a better discriminator than the others; mean and covariance information can be destroyed in such a transformation. Rather, multivariate extensions should simply use multivariate versions of the periodic time series models. This topic is an active area of current statistical research.