a. Relative changepoint testing

Conrad and Pollak (1962) state that “a climatological series is relatively homogeneous with respect to a synchronous series at another place if the temperature differences (or precipitation ratios) of pairs of homologous averages constitute a series of random numbers” (i.e., white noise). The assumption is that similar variations in climate occur at nearby locations because of the spatial correlation inherent to meteorological fields (Livezey and Chen 1983). A statistically significant and persistent violation of relative homogeneity is presumed to be artificial or, at least, to have origins other than the background variations in weather and climate. Relative homogeneity testing is therefore conducted primarily to distinguish artificial breaks from real climate variability, although it may also improve the power of detecting artificial shifts. The reason is that when two temperature series {X_t} and {Y_t} are highly correlated [i.e., Corr (X_t, Y_t) = ρ > ½], the variance of their differences will be lower relative to the original series.

To carry out relative homogeneity testing, a reference series is commonly constructed by averaging values from locations near the target site whose observations are in question (Karl and Williams 1987; Alexandersson and Moberg 1997; Vincent 1998). Unfortunately, the homogeneity of the reference series cannot be taken for granted because undocumented changepoints may be present in any one of the averaged series (Hanssen-Bauer and Førland 1994; MW05). Strategies for reducing changepoint attribution errors have included assessing the homogeneity of the reference series itself (McCarthy et al. 2008) and building a reference from previously adjusted series (González-Rouco et al. 2001). Unfortunately, conducting a separate assessment of reference series homogeneity fails to exploit the enhanced sensitivity of relative homogeneity testing, and many small-amplitude changepoints may go undetected in the reference series only to be later attributed to the target series. Similar problems may arise when adjusted data are used to build a reference series because artifacts from the original imperfect reference series can be transferred to the adjusted data themselves.

Alternatively, relative homogeneity testing can be implemented via a pairwise comparison of individual climate series (Jones et al. 1986; Slonosky et al. 1999; Menne and Duchon 2001; Caussinus and Mestre 2004). In pairwise testing, the cause of undocumented changepoints can be traced more directly, that is, without first testing the reference series or assuming it is homogeneous. Unfortunately, implementing pairwise testing has usually required a manual review of the results. For example, Jones et al. (1986) conducted an arduous station-by-station homogenization by manually determining the cause of changepoints in paired difference series. Caussinus and Mestre (2004) computed the locations of changepoints in difference series automatically, but still deferred to an analyst to attribute the cause. In contrast, an automated approach was developed for the pairwise algorithm, as described in section 3.

b. Distinction between documented and undocumented changepoints

In the absence of station history records, the date of inhomogeneity must be treated as an unknown parameter. In such cases, a systematic search through all values in a series is required to identify the dates of statistically significant discontinuities. The systematic nature of the search necessitates the use of a more conservative set of critical values relative to the standard values that are appropriate for testing the significance of known changes to observation practice (Lund and Reeves 2002). This means that tests for undocumented changepoints are less sensitive than comparable tests for documented changes. It follows that to maximize the power of changepoint detection, station histories should be exploited whenever possible.

The strategy used by the pairwise algorithm is to first identify all evidence of changepoints using the less sensitive tests for undocumented changepoints. Where possible, the results are then combined with information about documented changes whose impact may go undetected by these less sensitive tests. An important benefit of this approach is that all possible changepoints are identified before estimates of their magnitude are made.

c. Resolving multiple undocumented changepoints

While the issue of accurately resolving multiple undocumented changepoints remains an active area of statistical research (R07), two approaches are in operative use. The first, more common approach uses a recursive testing procedure (e.g., Vincent 1998) to overcome the “at most one changepoint” assumption behind most hypothesis tests for undocumented changepoints. The second approach relies on a penalty function to constrain the number of changepoints resolved through an optimization routine used to maximize the contrast between sequential mean levels of a series (e.g., Caussinus and Mestre 2004).

A recursive testing approach is used in the pairwise algorithm for the following two reasons: First, the approach is associated with a low probability of false changepoint detection without requiring an analyst to interpret the results (cf. Caussinus and Mestre 2004). Second, MW05 noted that when the recursive hypothesis test method is carried out using a semihierarchical splitting algorithm (Hawkins 1976), the power of changepoint detection can be comparable to that of optimal algorithms.

Recursive testing is based on a hierarchic, binary segmentation of the test series whereby a series is split at the location where the test statistic reaches a maximum, that is, the point at which the separation between the mean before and after the breakpoint is greatest. Then, the subsequences on either side of the first split are likewise evaluated, and the process is repeated recursively until the magnitude of the statistic does not exceed the chosen significance level in any remaining subsequences (or the sample size in a segment is too small to test). A semihierarchic implementation of this method means that each splitting step is followed by a merging step to test whether a split chosen at an earlier stage has lost its importance after subsequent breakpoints are identified, thereby more closely approximating an optimal solution.

d. Impact of local, unrepresentative trends

Ideally, a changepoint detection method would differentiate trend changes from step changes. In practice, however, many of the commonly used tests for undocumented changepoints are not robust to the presence of trends in the test data because they are based solely on comparing the means of two sequential intervals. Use of such tests in the presence of trends can lead to falsely detected step changes as well as to inaccurate estimates of the magnitude of a shift when it occurs within a general trend (DeGaetano 2006; Pielke et al. 2007). Conversely, methods that directly account for both step changes and trend changes (e.g., Vincent 1998; Lund and Reeves 2002; Wang 2003) are characterized by much lower powers of detection than the simpler difference in means tests.

While no one test clearly outperforms others under all circumstances, the standard normal homogeneity test (SNHT; Alexandersson 1986) has been shown to have superior accuracy in identifying the position of a step change under a wide variety of step and trend inhomogeneity scenarios relative to other commonly used methods (DeGaetano 2006; R07). For this reason, the pairwise algorithm uses the SNHT along with a verification process that identifies the form of the apparent changepoint (e.g., step change, step change within a trend, etc.). In fact, the pairwise testing procedure is similar to the Vincent (1998) and R07 forward and backward regression methods, respectively, but is more easily adaptable to a recursive testing approach for resolving multiple undocumented changepoints, and at the same time retains the higher power of detection of the SNHT.

a. Selection of neighbors and formulation of difference series

The pairwise algorithm starts by finding the 100 nearest neighbors for each temperature station within a network of stations. These neighboring stations are then ranked according to their correlation with the target. The first differences of the monthly anomalies are used to calculate the correlation coefficients [i.e., Corr (X_t − X_{t − 1}, Y_t − Y_{t − 1})] in order to minimize the impact of artificial shifts in determining the correlation (Peterson et al. 1998b). A series must simply be positively correlated with the target series to be eligible as a neighbor. Eligible neighbors could also be restricted to those series for whom ρ ≥ ½. This restriction effectively occurs in practice for monthly temperature values from the U.S. Cooperative Observer Network where more than 99.5% of the monthly temperature series from the 100 nearest neighbors are correlated at this level (and ρ ≥ 0.8 in 90% of cases).

From all eligible neighbors, the set used for the pairwise analysis is selected using a two-step process. First, an account is made of the years and months for which both the target and its 40 most highly correlated neighbors report monthly mean maximum and minimum temperature data. Then, beginning with the 41st most highly correlated neighbor, the algorithm assesses whether an additional neighbor adds any data for the years and months that have fewer than seven viable neighbors. If the neighbor in question provides records for such data-sparse periods, it replaces the least correlated of the original 40 with the new neighbor provided that the addition does not remove data for other data-sparse periods. This process ensures that, whenever possible, at least seven neighbors are available at all times during the target station’s period of record (the rationale for attempting to make at least seven target–neighbor comparisons is provided in section 4).

Next, time series of differences {D_t} are formed between all target–neighbor monthly temperature series. To illustrate this, take two monthly series {X_t} and {Y_t}, that is, a target and one of its correlated neighbors. Following Lund et al. (2007), these two series can be represented as

View Expanded

and

View Expanded

where μ represents the monthly mean anomaly at the specific series, T = 12 represents the months in the annual cycle, ν ∈ {1, … , 12} is the monthly index, m = the year (or annual cycle) number, and the ε_t terms denote mean zero error terms at time t for the two series. The δ_t terms represent shift factors cause by station changes, which are thought to be step functions. Following Lu and Lund (2007), these shift factors are of the form

View Expanded

where n = the total number of values common to {X_t} and {Y_t}, and Δ and c represent the size and time of a shift, respectively. Because the timing of the level shifts is often unknown in climate networks, the goal of the pairwise algorithm is to reveal the shift times {c₁, c₂, … , c_{k − 1}} not only for {X_t} and {Y_t}, but for all of the series in the network no matter how complete the station metadata. Once the timing of the shifts is known, their magnitudes {Δ₁, Δ₂, … , Δ_{k − 1}} can be estimated.

Differencing the {X_t} and {Y_t} yields the {D_t} series, which has the form

View Expanded

In reality, it is unrealistic to assume that β^X and β^Y are stationary in t given the nature of multidecadal variations in climate series; however, it may be that β^X ≈ β^Y in general. This assumption is evaluated further in subsequent steps because if β^X ≠ β^Y, then a local, unrepresentative trend (i.e., creeping inhomogeneity) is present in {X_t} and/or {Y_t}. At present, the periodicity in (4) is considered to be negligible, especially since {X_t} and {Y_t} are first deseasonalized.

Figure 1 provides an example {D_t} series formed between mean monthly maximum temperature anomalies from Chula Vista, California, and nine highly correlated neighbor series. The reduction in the variance of the {D_t} series relative to the original target series is clearly evident. The variety of overlapping periods and relative shifts between the records from Chula Vista and its neighbors is common in surface temperature records.

b. Identification of undocumented changepoints

After all difference series have been formed, the SNHT is used to identify undocumented changepoints in each {D_t} using the semihierarchical splitting algorithm and a 5% significance level (α = 0.05). The SNHT evaluates the null hypothesis (H₀) that a {D_t} series has a constant mean against the alternative hypothesis (H_A) that there is an undocumented step change on date c. To account for the possibility of multiple changepoints, the difference series is assumed to consist of K segments, each bounded by two changepoints (c_{k − 1} and c_k). In the pairwise algorithm, SNHT takes the form

View Expanded

View Expanded

where N(μ, σ²) refers to a random normal variable with a mean μ and variance σ², and μ₁ ≠ μ₂. For convenience we define c₀ = 1 and c_k = n, the total number of values in the {D_t} series. The unsubscripted c in (6) refers to the assignment of an undocumented changepoint between two previously established changepoints (c_{k − 1} and c_k) as the semihierarchal splitting algorithm iterates through the succession of splitting and merging steps, ultimately converging on a solution of K segments bounded by K − 1 shifts.

c. Classification of breakpoints identified by the SNHT test

The result of step b is a set of K − 1 apparent changepoints for each {D_t} series. Because the SNHT assumes that each series is of the form

View Expanded

the next step determines whether this piecewise stationary model is justified for each changepoint. The determination is made by fitting a hierarchy of potential models for all segments centered on each kth breakpoint. The five models (M1–M5) are described in Table 1 (after R07). The model that minimizes the Bayesian information criterion (BIC; Schwarz 1978) is selected as the best representation for each changepoint.

Procedurally, the BIC is calculated by fitting M1–M5 to every segment c_{k − 1} + 1 to c_{k + 1} for all k = 1, K. The BIC is defined as

View Expanded

where p is the number of parameters required to fit the model, n′ is the number of data points in the segment from c_{k − 1} + 1 to c_{k + 1}, and L is the likelihood of the model in question. For the models listed in Table 1,

View Expanded

where SSE refers to the sum of squared errors for the particular model fit.

In some cases, one or more of the original K − 1 changepoints may be eliminated from the solution for a particular {D_t} series. For example, if the true model between the values of c_{k − 1} + 1 and c_{k + 1} is a constant increasing trend (M2), the SNHT may have identified an apparent jump in the middle of the trend interval, whereas the BIC is likely to be lower for M2 than for any of the other four models. In such a case, the false changepoint time is removed from and K is decremented. Alternatively, the use of the BIC may determine that the {D_t} segment between c_{k − 1} + 1 and c_{k + 1} more appropriately follows M4 (step change within a constant trend) or M5 (a step change separated by different trends). If so, there is evidence of a relative trend between the two series, and the magnitude of the step change Δ required in subsequent steps e and f should be calculated using the higher dimension models (M4 and M5) to avoid calculating a biased estimate of the step.

d. Attribution of shifts in the difference series

Given that breaks in a difference series will be induced by discontinuities in either {X_t} or {Y_t}, the next step is to identify the series responsible for a particular discontinuity. To begin, an array of change dates by station is formed, and all of the changepoint dates detected in the {D_t} series are temporarily assigned to both {X_t} and {Y_t}. Specifically, a count is incremented for the date of shift each time a station is implicated by a break in one of its difference series. The resulting array of change dates by station is then “unconfounded” by systematically identifying those stations that are common to numerous difference series with the same date of change. More specifically, the station/date with the highest overall changepoint count is identified. This station is then tagged as the “culprit” or “perpetrator,” that is, as the cause of the breaks on the date with the highest breakpoint count. The corresponding count on that particular change date is then decremented for all of the perpetrator’s neighbors, and the process is repeated using the updated shift–date tallies. The procedure continues recursively until no station/shift date count is greater than one for any station/date in the period of record.

e. Assignment of undocumented changepoint dates

Although undocumented shifts are assigned to a perpetrating series in step d, the date of an undocumented changepoint returned by the SNHT is subject to sampling variability. As illustrated in Fig. 2, the degree of this sampling variability is a function of the magnitude of changepoint, with larger changepoints associated with more precise estimates of the date of change. This means that testing a group of target–neighbor difference series often leads to a range of undocumented changepoint dates clustered around the time of the actual change. The simulations summarized in Fig. 2 were used to estimate the confidence limits of a changepoint date as a function of the magnitude of step change.

To determine which change dates likely refer to the same discontinuity, an interim estimate of step-change magnitude is therefore necessary. The estimate is calculated using the most appropriate change model (M3, M4, or M5) according to the BIC, which is used to determine the range of uncertainty for a particular undocumented shift date. The cluster of dates falling within overlapping confidence limits is then conflated to a single date at the target in one of two ways: 1) it is assigned to the date of a known event in the target station’s history that occurs within the confidence limits for a shift of that magnitude, or 2) it is assigned to the most common changepoint date that falls within these simulated confidence limits, which means that the discontinuity appears to be truly undocumented.

f. Calculation of adjustments

Steps a–e are necessary simply to identify undocumented changepoints in all temperature series. In many applications, however, station histories also may be available, which might provide additional information regarding possible discontinuities. When available, the dates of documented events should be combined with evidence of undocumented changepoints because the impact of documented events may be too subtle for the tests for undocumented shifts to detect. Potential adjustments can then be calculated for all undocumented and documented shifts at the same time.

Adjustments are determined by calculating multiple estimates of Δ using segments from neighboring series that are homogeneous for at least 24 months before and after the target changepoint. (When two changepoints occur within 24 months in the target series, an adjustment is made for their combined effect.) The range of pairwise estimates for a particular step change is considered to be a measure of the confidence with which the magnitude of the discontinuity can be estimated. As in step e, the step model found to be most appropriate (i.e., M3, M4, or M5) according to the BIC can be used to calculate a final estimate of the shift for each relevant {D_t} segment to avoid biased estimates of Δ when a relative trend is also present. At least three separate pairwise estimates of step-change magnitude are required for each target changepoint because the distribution of estimates is used to determine the significance of the adjustment (when fewer than 3 estimates are available, the shift is considered “unadjustable”). Moreover, because the distribution of step-change estimates is not necessarily symmetric, the median estimate is used to adjust the target series.

The consistency of the pairwise estimates for Δ is determined by comparing the median estimate to either the 5th percentile (median > 0) or to the 95th percentile (median < 0) of all estimates, subject to an initial outlier check. Because fewer than 20 estimates may be available for any given changepoint, a multiple of the difference between the median and the first quartile (Q₁) or between the median and third quartile (Q₃) serves as an estimate of the 5th or 95th percentile, respectively. A factor of 2.5 is used because it approximates a one-tailed test at the 5% (α = 0.05) significance level (assuming independent estimates). When the median and the tail of the distribution closest to zero are of the same sign (i.e., median − Q1 × 2.5 or median + Q3 × 2.5), the step change is considered to be significant, and an adjustment is made to the target series. This approach is similar to the Tukey (box plot) outlier test (Tukey 1977), but allows for asymmetry in the distribution of estimates. Alternatively, one could simply use the median Δ estimate when all estimates are of the same sign. Both approaches appear to yield comparable results.

g. Example of changepoint detection and adjustment

Application of the pairwise algorithm to the group of series shown in Fig. 1 revealed two significant changepoints in Chula Vista maximum temperatures, both of which were associated with documented station moves, first on 1 January 1982 and then again on 25 April 1985. Difference series between the pairwise-adjusted mean monthly maximum temperatures for Chula Vista and its neighbors are shown in Fig. 3, which suggests that the algorithm has removed the major step inhomogeneities from all series in the group.

a. Evaluation under monthly temperature simulations

The performance of the pairwise algorithm was first evaluated using two different sets of simulated monthly temperature anomalies. One set was comprised of series with step changes, while the second set contained series with both trend and step inhomogeneities. Both sets consisted of 1000 groups of 21 correlated “red noise” series generated as in MW05. The average correlation between each series within a group was about 0.7. For all series the mean (μ) was zero and the standard deviation (σ) was one; the number of values in each series (n) was equal to 1200, the equivalent of 100 yr of monthly means.

A random number of step changes was imposed on each series at random dates. The number of steps per series varied symmetrically about a peak frequency of 5, with as few as 0 and as many as 10. The magnitude of each step change was also assigned randomly by sampling from the standard normal distribution, which means that about two-thirds of the imposed steps were equal to one σ or less. As discussed in MW05, the standard normal distribution is a good proxy for the distribution of known impacts to U.S. temperature series (Karl and Williams 1987). All imposed step changes were treated as undocumented, and 10 neighbors were identified by the pairwise algorithm for all 21 series in the groups.

In the “monthly steps and trends” simulations, a trend inhomogeneity was added to roughly 60% of the simulated series. The magnitude of this trend was varied randomly from 0.001σ month⁻¹ up to about 0.18σ month⁻¹, while the trend interval varied randomly from 2 months up to the full period of record. Usually the trend inhomogeneity did not initiate with a step change, although steps frequently occurred randomly within the intervals of a creeping inhomogeneity. In total, about 25% of all series segments were characterized by a trend.

Figure 4 illustrates the impact of random step-only shifts on one group of simulated series. Prior to imposing step changes, the true trend in each series was zero. After imposing shifts, the trends ranged from −7.62σ century⁻¹ to +4.34σ century⁻¹. The pairwise algorithm correctly identified 34 of the 43 imposed step changes. Of the nine shifts not identified, six had a magnitude of less than 0.3σ, which is below the sensitivity of most tests for undocumented changepoints (DeGaetano 2006; Ducré-Robitaille et al. 2003). Furthermore, the largest undetected changepoint (+0.696σ) was preceded 10 time steps earlier by another undetected changepoint of −0.451σ; that is, the two changepoints essentially masked one another. The overall effectiveness of the pairwise adjustments is evident in Fig. 5, which depicts the 10 series after homogenization by the pairwise algorithm. Note that changepoints have been adjusted relative to the latest mean level in each series, the convention in climate data homogenization. In general, the adjusted series all have trends much closer to the true “climate” trend of zero.

Table 2 more generally summarizes the detection skill of the pairwise approach for both the step-only and the step-/trend-change scenarios. The hit rate (the ratio of the number of changepoints correctly identified relative to the total number imposed) is roughly 67% for both scenarios. The false-alarm rate (FAR; the ratio of falsely detected changepoints to the total number detected) is 6.77% for the step-only scenario (only slightly higher than the expected type-I error rate at the α = 0.05 significance level) and 19.65% for the step/trend scenario. The increase in false alarms when trend inhomogeneities are present occurs for two main reasons. First, the beginning or end of a trend inhomogeneity is often identified as a step change by the pairwise algorithm. Second, short interval trends of about 24 months or less tend to be virtually indistinguishable from step changes and are therefore adjusted as an abrupt change. Indeed, the largest magnitude false alarms under the steps-and-trend inhomogeneity simulations result from short interval, but large magnitude trend inhomogeneities that are approximated by a step change.

Histograms indicating the magnitude of hits, misses, and false alarms for the step-only and step/trend simulations are shown in Figs. 6 and 7, respectively. In both cases, changes in excess of 0.5σ are readily detected, and most misses are generally less than 0.5σ. The number of false alarms is also generally small, suggesting that they will have little impact on the homogenized trends for the simulated series.

Regarding the series trends, two measures of error are provided in Table 2. The first is the root-mean-square error (RMSE) for a trend calculated using the unadjusted series and the second is the RMSE for trends calculated using the adjusted series. As shown in the table, the pairwise homogenization process greatly reduces the error associated with the calculation of the true background climate trend. Table 2 also indicates that the RMSE for changepoint estimates in series with trends is about as good as in the series with no trend inhomogeneities, which suggests that the model identification is reasonably successful at identifying step changes that occur within local trends. A more thorough assessment of changepoint-type identification is provided in section 4c.

b. Pairwise versus reference series changepoint detection skill

The use of a reference series is the most widely employed approach to relative changepoint detection, and MW05 evaluated the implications of such an approach for undocumented changepoint detection. The pairwise approach was therefore evaluated using the same simulations and scenarios as in MW05 to directly compare its skill of undocumented changepoint detection against the reference series approach. Table 3 depicts the seven scenarios evaluated in MW05. Each case was comprised of 1000 groups of six correlated series (one target and five neighbors) with n = 100 values. Of the three reference series formulations evaluated by MW05, the one based on a correlated weighted average of the five neighbors (Alexandersson and Moberg 1997) is compared here. As in the pairwise algorithm, the SNHT was used to test the target-minus-weighted-average reference {D_t} series (α = 0.05). All changepoints detected in the {D_t} series were attributed to the target series to test the consequences of assuming reference series homogeneity.

Table 4 summarizes the pairwise and reference series detection skill for the MW05 target series. Two statistics are presented for each case: the FAR (previously described) and the correct changepoint (CRC) power statistic (R07), which is the percentage of time that either (a) the changepoint date in the target series was selected within ±2 time steps of the correct date or (b) the target was correctly identified as homogeneous. Basically, the CRC is synonymous with hit rate except that it also credits the number of times that the target series was successfully identified as homogeneous.

In general, the pairwise algorithm has a much higher success rate in identifying homogeneous target series than the reference series approach as indicated by the higher CRC percentages for cases 1, 3, and 5. This is true when the neighbor series are themselves homogeneous as in cases 1 and 5, but especially when all the neighbors have changepoints as in case 3, which cause numerous inhomogeneities in the reference series. More generally, Table 4 indicates the degree to which the pairwise approach limits the number of false alarms whenever the neighboring series are impacted by undocumented changepoints as evidenced by the low FAR for cases 4 and 6 relative to the reference series approach.

As shown in Fig. 8, the pairwise hit rate meets or exceeds that of the reference series approach when there are at least seven viable neighbors available at all times during a target station’s history. (This is the foundation for the number of neighbors selected for comparison as described in section 3a.) The relatively steep increase in the power of detection as the number of comparisons increases illustrates an advantage of pairwise testing, namely, that there are multiple chances to detect a changepoint in any particular target series. If the SNHT misses a changepoint in one target–neighbor difference series, or if it misidentifies the date, there are a number of additional chances to test for the same undocumented break. The chances are not completely independent, however, because any two {D_t} series with a common target will have an expected correlation of 0.5 (Menne and Duchon 2001). Moreover, the power of pairwise detection can be further improved by increasing the sample size between changepoints, which can be achieved by testing serial monthly values rather than annual or seasonal averages. This accounts for the higher hit rate in the “monthly” simulations, that is, 67% (Table 2) compared to the rate of a little less than 50% shown in Fig. 8 when 10 neighbors are available.

c. Skill in identifying the type of changepoint

The magnitude of a step change will not be accurately estimated if the type of changepoint has been misidentified. Consequently, the skill of the pairwise algorithm in classifying changepoint type was assessed for the range of models in Table 1. As in section 4b, a set of 1000 groups of target and neighbor series with n = 100 values were used for each scenario. In this case different magnitudes of trend and step parameters, that is, c, Δ, β, β₁, and β₂, were imposed on the target series as shown in Table 5; the five neighbor series, in contrast, were always homogeneous (M1). The magnitudes of the parameters imposed on the target series were the same as those used by R07, although only a portion of the results are summarized here.

A comparison of the CRC’s in Table 5 for the Δ = 1σ simulations indicates that the pairwise algorithm correctly identified more than 85% of these step changes regardless of whether the target series followed M3, M4, or M5. Moreover, the algorithm also correctly identified more than 85% of the M2 (constant trend) target series as homogeneous (no steps). On the other hand, there is more variability in the skill of classifying the type of changepoint as indicated by the correct type percentages shown in bold. The percentages indicate that the algorithm had somewhat less success in classifying M4- and M5-type changepoints relative to M3-type changepoints and series that follow M2.

Under the M2 scenarios, the pairwise algorithm correctly classified more than 85% of the {D_t} series when β was greater than or equal to 0.01 (a slope yielding a change of 1σ in 100 time steps), but less than 50% when β = 0.005 (a change of 0.5σ in 100 time steps). The reason for the difference is that the BIC does not always distinguish a sloped line from a flat line when β is small. This kind of misclassification, however, does not impact the CRC because there is no step change assigned to the target. On the other hand, when β is larger, the SNHT tends to partition the {D_t} trend into one or more step-type changes. The BIC correctly reclassifies most of these breaks as M2, but also cannot always distinguish a trend (M2) from a step change (M3, M4, or M5). Consequently, the pairwise algorithm classifies only 91% of M2 target series as homogeneous (no step) when β = 0.01 and 86.9% when β = 0.02. The impact of this type of misclassification is to inadvertently remove some of the unique target series trend as a step adjustment, thereby bringing the target series more in line with the regional background climate trend captured by the neighbors (DeGaetano 2006; Pielke et al. 2007).

For target series under M3 (step change with no trend), the overall power of detection is a function of the magnitude of the step, as shown in previous investigations (e.g., DeGaetano 2006). In the pairwise algorithm, most (> 88%) of the {D_t} series with a step change of 1σ or greater were correctly identified as M3, and the CRC exceeds 90% in such cases. On the other hand, many (about 45%) of the 0.5σ magnitude step changes are misclassified as a trend change (M2).

When the target series follows M4 (step change within a constant trend), the pairwise CRC varies between 85% and 90% for the 1σ step changes, close the M3 rate. However, in the M4 simulations, the algorithm frequently (about 80% of the time when β = 0.005) misclassifies the {D_t} series as M3, especially when β is small. This type of misclassification also leads to a biased estimate of the magnitude of the jump by aliasing the unique target trend on to the estimate of the step change. Much like a false alarm when the target follows M2, the biased estimate would bring the adjusted target more in agreement with the background trend captured by the neighbors (DeGaetano 2006; Pielke et al. 2007).

Under M5, the target series has a step change within a trend change, but there is also a change in trend coincident with the step. In this scenario, the CRCs are comparable to the M4 simulations, but in this case, the pairwise algorithm tended to misclassify the {D_t} series as M3 or M4 in roughly equal proportions. Consequently, some of the target series trends would be aliased onto the estimate of the M5 step changes, as in the case of the M4 target series simulations.

Overall, the results in Table 5 are consistent with the changepoint-type identification capabilities of the generalized methods investigated by R07, namely, that it is more challenging to classify M4- and M5-type changepoints. As shown in R07, the lower identification skill occurs even when changepoint tests specifically designed for these types of change are used, that is, Wang (2003) for M4 and Lund and Reeves (2002) for M5. Nevertheless, from Table 5 and results (not shown) based on directly testing a target series as in R07, it appears that the pairwise approach (SNHT plus BIC) has comparable skill at model identification compared to the methods evaluated by R07. The advantage of the pairwise approach is that the SNHT’s superior power of detection is exploited.

The skill of identifying changepoint type, like the power of detection, can also be improved by increasing the sample size of the test series, that is, by testing serial monthly series. For example, the percentage of correctly identified M4 difference series is about 70% at β = 0.02 when n = 240 and c = 120 versus 50% for n = 100 and c = 50. Similarly, when β₁ = 0.01 and β₂ = 0.03 under M5, the percentage of series correctly identified increases to 75% for n = 240 versus about 50% for n = 100. In addition, the skill of changepoint detection and identification increases with increasing correlation between series, which reduces the variance of the {D_t} series. As noted by DeGaetano (2006), the correlation between temperature series in the United States is typically higher than in the simulations used here.