Editorial Type:
Article
Article Type:
Research Article

A Comparison of Climate Signal Trend Detection Uncertainty Analysis Methods

Nipa Phojanamongkolkij Systems Engineering and Engineering Methods Branch, NASA Langley Research Center, Hampton, Virginia

Search for other papers by Nipa Phojanamongkolkij in
Current site
Google Scholar
PubMed Close
,
Seiji Kato Climate Science Branch, NASA Langley Research Center, Hampton, Virginia

Search for other papers by Seiji Kato in
Current site
Google Scholar
PubMed Close
,
Bruce A. Wielicki Climate Science Branch, NASA Langley Research Center, Hampton, Virginia

Search for other papers by Bruce A. Wielicki in
Current site
Google Scholar
PubMed Close
,
Patrick C. Taylor Climate Science Branch, NASA Langley Research Center, Hampton, Virginia

Search for other papers by Patrick C. Taylor in
Current site
Google Scholar
PubMed Close
, and
Martin G. Mlynczak Climate Science Branch, NASA Langley Research Center, Hampton, Virginia

Search for other papers by Martin G. Mlynczak in
Current site
Google Scholar
PubMed Close
Print Publication:
01 May 2014
DOI:
https://doi.org/10.1175/JCLI-D-13-00400.1
Page(s):
3363–3376
Full access

Abstract

Two climate signal trend analysis methods are the focus of this paper. The uncertainty of trend estimate from these two methods is investigated using Monte Carlo simulation. Several theoretically and randomly generated series of white noise, first-order autoregressive and second-order autoregressive, are explored. The choice of method that is most appropriate for the time series of interest depends upon the autocorrelation structure of the series. If the structure has its autocorrelation coefficients decreased with increasing lags (i.e., an exponential decay pattern), then the method of Weatherhead et al. is adequate. If the structure exhibits a decreasing sinusoid pattern of coefficient with lags (or a damped sinusoid pattern) or a mixture of both exponential decay and damped sinusoid patterns, then the method of Leroy et al. is recommended. The two methods are then applied to the time series of monthly and globally averaged top-of-the-atmosphere (TOA) irradiances for the reflected solar shortwave and emitted longwave regions, using radiance observations made by Clouds and the Earth’s Radiant Energy System (CERES) instruments during March 2000 through June 2011. Examination of the autocorrelation structures indicates that the reflected shortwave region has an exponential decay pattern, while the longwave region has a mixture of exponential decay and damped sinusoid patterns. Therefore, it is recommended that the method of Weatherhead et al. is used for the series of reflected shortwave irradiances and that the method of Leroy et al. is used for the series of emitted longwave irradiances.

Denotes Open Access content.

Corresponding author address: Nipa Phojanamongkolkij, NASA Langley Research Center, MS 131, Hampton, VA 23681-2199. E-mail: nipa.phojanamongkolkij@nasa.gov

Abstract

Two climate signal trend analysis methods are the focus of this paper. The uncertainty of trend estimate from these two methods is investigated using Monte Carlo simulation. Several theoretically and randomly generated series of white noise, first-order autoregressive and second-order autoregressive, are explored. The choice of method that is most appropriate for the time series of interest depends upon the autocorrelation structure of the series. If the structure has its autocorrelation coefficients decreased with increasing lags (i.e., an exponential decay pattern), then the method of Weatherhead et al. is adequate. If the structure exhibits a decreasing sinusoid pattern of coefficient with lags (or a damped sinusoid pattern) or a mixture of both exponential decay and damped sinusoid patterns, then the method of Leroy et al. is recommended. The two methods are then applied to the time series of monthly and globally averaged top-of-the-atmosphere (TOA) irradiances for the reflected solar shortwave and emitted longwave regions, using radiance observations made by Clouds and the Earth’s Radiant Energy System (CERES) instruments during March 2000 through June 2011. Examination of the autocorrelation structures indicates that the reflected shortwave region has an exponential decay pattern, while the longwave region has a mixture of exponential decay and damped sinusoid patterns. Therefore, it is recommended that the method of Weatherhead et al. is used for the series of reflected shortwave irradiances and that the method of Leroy et al. is used for the series of emitted longwave irradiances.

Denotes Open Access content.

Corresponding author address: Nipa Phojanamongkolkij, NASA Langley Research Center, MS 131, Hampton, VA 23681-2199. E-mail: nipa.phojanamongkolkij@nasa.gov
Keywords: Climate records; Forecasting techniques; Statistical forecasting; Model comparison

1. Introduction

Detection of long-term trends in various types of atmospheric data has been studied extensively in the past 30 years (e.g., Hill et al. 1977; Reinsel et al. 1989; Tiao et al. 1990; Weatherhead et al. 1998; Von Storch and Zwiers 1999; Leroy et al. 2008a,b). There are many factors that affect the detection, including the magnitude of the trend to be detected, the temporal length of available observations, the magnitude of the variability and autocorrelation of the noise in the data, and uncertainty in the observations. During the past decade, two trend detection methods have been widely used: the method of Weatherhead et al. (1998, hereafter W98) and the method of Leroy et al. (2008b, hereafter L08). It was not clear if the uncertainty of climate trends and the time to detect them are sensitive to the method used and under what circumstances that one method is preferred over the other. That was the motivation for this study. The W98 method is detailed in section 1a. Section 1b details the L08 method. Comparison of both methods is given in section 1c.

a. The W98 method

Let Yt be the measurement of interest. The trend model assumes linearity, as expressed in , where α is a constant term, St is the seasonal component, Xt = t/12 is the linear trend function, ω is the magnitude of the trend per year, Nt is the unexplained noise of the measurement data, and t = 1, 2, 3, … , T months. The noise Nt can be obtained from the deseasonalized and detrended time series monthly data. A deseasonalized series is obtained by subtracting the monthly-mean value from each month. For example, the average of all January months is subtracted from each individual January month, with the same process used for all months in the year. This process removes the bulk of Earth’s very large and repeatable seasonal cycle from the climate record. Then the trend is removed from the deseasonalized series, generating the Nt deseasonalized and detrended time series.

The W98 method assumes the noise Nt follows a first-order autoregressive model (AR1). The uncertainty of trend estimate given by the W98 is
e1

The estimate of the trend per year is , is the variance of the trend estimate, n = T/12 is the number of years of time series data, is the variance of the deseasonalized and detrended Nt monthly time series, and is the lag-1 autocorrelation coefficient.

b. The L08 method

The trend model for the L08 is also linear, as expressed in , where Yi is the measurement, α is a constant term, ti is the linear trend variable, m is the magnitude of the trend per month, Ni is the unexplained noise of the measurement data, and i = 1, 2, 3, … , T months. The uncertainty of trend estimate given by the L08 is
e2
Term is the length of time series, and dt is the interval between observations. In this study dt = 1, and therefore reduces to T.

The L08 takes theoretical autocorrelation at all lags (i.e., from −∞ to +∞) into consideration to obtain the uncertainty of the trend estimate [Eq. (2)]. However, we do not know the theoretical autocorrelation , but can estimate it by the sample autocorrelation from the available time series data. Hassani (2010) has shown that . We also know the properties of and . Therefore, it is not possible to estimate with in practice because . Consequently, a criterion is needed for which sample autocorrelation to consider.

c. Method comparison

The W98 has its uncertainty for trend estimate per year [Eq. (1)], while the L08 has its uncertainty in a per month basis [Eq. (2)]. To make the W98 into the uncertainty for trend estimate per month and assume the trend is reasonably linear with time, we take the variance property of , and therefore Eq. (1) becomes
eq1
e3

The obvious difference in the two uncertainties [Eqs. (2) and (3)] is the last term on the right-hand side of the equations, which indicates how the correlation structure is handled by the two methods. We define decorrelation time τ as the time needed for data to be uncorrelated (Von Storch and Zwiers 1999). The decorrelation time for W98 and L08 was set as and , respectively. Therefore, we focus on comparing terms and when comparing the two methods.

The W98 assumes the noise Nt follows the AR1 model, which requires only one estimated model parameter (i.e., the first lag sample autocorrelation coefficient ). As such, it is simple to use. However, any departure of Nt from the AR1 model will likely cause the W98 to perform poorly. The L08 does not assume any autoregressive model or any correlation structure for the Nt series. It takes into account theoretical autocorrelation coefficients at all lags in computing the uncertainty of trend estimate. It is therefore theoretically versatile. However, all theoretical coefficients are not known in practice, and rather are estimated by their corresponding sample autocorrelation coefficients. Mathematically it is not possible to include sample coefficients at all lags because . Therefore, it is essential to have a criterion on which sample coefficients to include in the estimation of the L08’s uncertainty. Table 1 summarizes the equations for the uncertainty of trend estimate and the advantages and disadvantages of both methods.

Table 1.

Method comparison summary.

Table 1.

The organization of the paper is as follows: Section 2 describes and compares the two methods using the Monte Carlo simulation with theoretically and randomly constructed time series. The findings from this section are applied to the reflected shortwave and emitted longwave irradiances at the top of the atmosphere (TOA), as estimated during March 2000 through June 2011 from observed Clouds and the Earth’s Radiant Energy System (CERES) instruments on the National Aeronautics and Space Administration (NASA) Terra satellite. This application is detailed in section 3. Section 4 concludes the study.

2. Monte Carlo simulation study

In section 2a, the Monte Carlo simulation settings are described. Simulation results are given in section 2b.

a. Simulation settings

We compare the W98 and L08 methods with the theoretically and randomly generated first- and second-order autoregressive (i.e., AR1 and AR2) time series. The AR1 series follows , and , where is a randomly generated observation at time t, is the model parameter, and is serially uncorrelated normally distributed random data with a mean of zero and standard deviation of one (or white noise) at time t, or . The Nt is a deseasonalized and detrended time series as described in section 1. Note that the white noise series can be thought of as AR1 with . For an AR1 series to be stationary, it is required that (Montgomery et al. 2008).

The AR2 series follows , and , where and are parameters of the model. For an AR2 series to be stationary, we consider two roots m1 and m2 of the polynomial . There are three possible cases for the two roots. In case 1, if both roots are real, distinct, and |m1| and |m2| < 1, then the autocorrelation function (ACF) exhibits a mixture of two exponential decay patterns. In case 2, if both roots are complex conjugates in the form of and satisfy , then the ACF has a damped sinusoid pattern. In case 3, if there is one real root with m1 = m2, then the ACF shows a single exponential decay pattern. We study all three cases of the AR2 series. We limit our simulation study to AR2 for simplicity because it already captures phenomena that are likely to complicate the W98.

Table 2 outlines the time series data that are used with the W98 and L08 methods in the first three columns. The sensitivity of both methods to record length (for 10, 20, …, 100 years) is included in this study. The exploration matrix is a 13 time series by two methods by 10 record lengths (13 × 2 × 10) matrix. The column “series number” will be used to facilitate all output figures. Figure 1 shows in detail how the computer simulation is done. The outer loop is repeated 13 times, one for each time series. The middle loop is repeated M times, one for each simulation. The determination of the number of simulations M will be given later in this section. For each simulation, the inner loop computes the values by the two methods and is repeated for 10 different record lengths of interest. Also produced for each time series is the theoretical value, which does not depend on record length. We choose to compare the terms rather than the uncertainties of trend estimate because the uncertainties are functions of both record length T and term [Eqs. (2) and (3)]. Given significantly large T, the two methods could provide similar and very small uncertainties, but the similarity is primarily attributable to the effect of large T. Therefore, comparing the with respect to the theoretical value using two methods is adequate to get an insight into their performances independently from the record length T.

Table 2.

Exploration matrix for the W98 and L08 methods. The record lengths for time series data are 10, 20, … , 100 yr.

Table 2.
Fig. 1.
Fig. 1.

Computer simulation framework.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

The theoretical value for AR1 from Eq. (3) is
e4
According to W98, , where is the variance of serially uncorrelated normally distributed random data, which in this study is one. Given that , Eq. (4) becomes
e5
The theoretical value for the AR2 is [from Eq. (A12) in the appendix]
e6

Parameters of the AR1 model in Table 2 are chosen to be 0.0, 0.3, 0.6, and 0.9, representing white noise, low-, medium-, and high-correlated series. For each AR1 series, we first randomly generate a data point sampled from a Norm(0, 1) distribution. This first sample represents a data at time t = 0, or . The next data point N1 is obtained from . The process is repeated until 1200 data points (from N1 to N1200) are obtained, representing a 100-yr monthly series. The ACFs of all chosen 100-yr monthly AR1 models are given in Fig. 2. Sample autocorrelation coefficients up to 10 log10(T) lags, where T = 1200, are computed and shown in the ACFs (Venables and Ripley 2002). Any autocorrelations that are outside the dashed lines are considered statistically significant within a 95% confidence level. For the white noise series (Fig. 2a), only lag 0 is significant, which is the property of the serially uncorrelated series. Figures 2b to 2d give ACFs exhibiting exponentially decay patterns. As parameter increases, its ACF pattern decays slower as lags increase.

Fig. 2.
Fig. 2.

The ACF of all 100-yr monthly AR1 models with the series number from Table 2.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

Parameters and of the AR2 model are chosen to represent the three cases of the stationary AR2 series. For each AR2 series, we first randomly generate two data points N−1 and N0 sampled from a Norm(0, 1) distribution (i.e., and ). The next data point N1 is obtained from . The process is repeated until 1200 data points (from N1 to N1200) are obtained, representing a 100-yr monthly series. The ACF of the first case has a mixture of two exponential decay patterns (Figs. 3a–c). The ACF of the second case exhibits a damped sinusoid pattern (Figs. 3d–f). The ACF of the third case has a single exponential decay pattern (Figs. 3g–i).

Fig. 3.
Fig. 3.

The ACF of all 100-yr monthly AR2 series with the series number from Table 2.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

We determine the number of simulations (M in Fig. 1) needed for all chosen time series using the variance reduction technique (Ross 2006). The average of the simulation solutions has to be an unbiased estimator of its corresponding truth. This average value changes as the number of simulations M increases. We evaluate the sensitivity of using different numbers of simulations to simulation error (the standard deviation of the average). We would expect simulation error to decrease as the number of simulations increases. Simulation error generally can be calculated from
e7
The U is the value of our simulated variable of interest, which is in this study. The value is the average of U from M simulations. Figure 4 shows the sensitivity of the number of simulations to simulation errors (as percent of the ) for all 100-yr monthly series given in Table 2 using the W98 method. The error converges to the minimum value at a different number of simulations depending on the series. For our study, we can therefore use 10 000 simulations for all series to guarantee convergence to the minimum percent error.
Fig. 4.
Fig. 4.

Sensitivity of the number of simulations to the simulation errors (as percent of its simulated mean) for all 100-yr monthly series (i.e., AR1 and AR2 series) with the W98 method.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

For the L08 method, the criterion for estimating the decorrelation time is to compute the sample autocorrelation coefficients up to 10 log10(T) lags, then only sum all coefficients of statistically significant lags to get the estimate. Similar to Fig. 4, the sensitivity of the number of simulations to the percent of simulation errors for all 100-yr monthly series using the L08 method is shown in Fig. 5. For all series, we therefore use 10 000 simulations to guarantee convergence to the minimum percent error. We notice that similar sensitivity plots (not shown) can be obtained for all 10-yr monthly series using the two methods. Therefore, we set M (from Fig. 1) to 10 000 simulations for the exploration matrix in Table 2.

Fig. 5.
Fig. 5.

As in Fig. 4, but for the L08 method.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

b. Simulation results

We ran the simulation study according to Fig. 1 using M = 10 000 simulations for all 13 series. We draw our conclusion based on comparing each method’s distributions (from 10 000 samples) with respect to the corresponding theoretical values. The simulated performances obtained from the W98 and L08 methods vary depending upon the series. Generally, there are five distinctive performances of the median with respect to their corresponding theoretical values. Examples of these five distinctive performances are given in Fig. 6. In this figure, box plots of simulated values at different record lengths as estimated from the two methods are shown. The corresponding theoretical values are given by the dark horizontal gray lines. The bottom and top of the box are the first (Q1) and third (Q3) quartiles, and the band inside the box is the second (Q2) quartile (or the median). The interquartile range (IQR) is defined as the difference between Q3 and Q1 (i.e., Q3 − Q1). The low and high ends of the whiskers represent Q1 − 1.5IQR and Q3 + 1.5IQR, respectively. Any values outside the whiskers represent extreme values. The sample equation is
eq2
with the numerator being the sum of squares of standard normal deviations, and the denominator is a constant. This sum of squares of deviations follows the chi-square distribution with T − 1 degrees of freedom (Montgomery and Runger 1994). We can conclude that the distribution of simulated values follows the chi-square distribution with T degrees of freedom. The tail regions of the distributions are possible but less probable.
Fig. 6.
Fig. 6.

The box plot of simulated values lengths as estimated from the two methods for the selective monthly series given in Table 2 at different record for (a) AR1_0, (b) AR1_3, (c) AR2_1c, (d) AR2_2c, (e) AR2_3a, and (f) AR2_3c. The corresponding theoretical values are given by the dark gray horizontal lines in all figures. The vertical ranges of the figures are not extended out to show the extreme values in order to facilitate readability of more critical information such as the quartile and median values.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

To focus on more critical information such as the quartile and median values, the vertical ranges of the figures are not extended out to the extreme values, which are usually on the tail regions of the distributions. These figures are intended to show the decreasing trend of ranges of distributions as record lengths increase. Figures 6a and 6b are for AR1_0 and AR1_3, respectively. All other AR1 series (AR1_1 and AR1_2) perform similarly to these two figures. The AR2_1c and AR2_3b series perform similarly and can be represented in Fig. 6c. All the AR2 series with damped sinusoid ACFs (AR2_2a, AR2_2b, and AR2_2c) perform similarly as shown in Fig. 6d. The AR2_3a perform differently from other series and is shown in Fig. 6e. The AR2_1a, AR2_1b, and AR2_3c series perform similarly as in Fig. 6f.

Table 3 summarizes the simulated performances obtained from the W98 and L08 methods for these five distinctive performances of the median value. The performance of the distribution ranges and tail regions does not vary with series, but rather vary depending upon the methods. For the AR2 with damped sinusoid ACFs, the L08 gives reasonable medians, while the W98 gives significantly higher medians than the theoretical values. Ranges of the distributions and tail regions from both methods are smaller as record lengths increase. Ranges from the W98 are smaller than those from the L08. The L08’s values occasionally are negative because the sum of all sample autocorrelation coefficients of statistically significant lags for the τ estimate is dominated by the negative coefficients. The likelihood (as the percent of total 10 000 simulations) of the L08’s negative τ value is shown in Fig. 7. As the record length increases, the likelihood becomes smaller. For all series, except the series with damped sinusoid ACFs, the L08 has a likelihood of no more than 3% for the 10-yr record length to give negative τ values. For the series with damped sinusoid ACFs (Fig. 7c), the L08 has higher likelihood (as high as 28%) for the 10-yr record length.

Table 3.

Monte Carlo performance comparison for the W98 and L08 methods.

Table 3.
Fig. 7.
Fig. 7.

The likelihood of negative τ values from the L08 for all simulated series, based on the percent total of the 10 000 computer simulations: (a) AR1 series; (b) AR2 series with ACF exhibiting a mixture of two exponential decay patterns (case 1, AR2_1); (c) AR2 series with ACF exhibiting damped sinusoid pattern (case 2, AR2_2); and (d) AR2 series with ACF exhibiting a single exponential decay pattern (case 3, AR2_3).

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

Thus far, we have used as our metric of interest. This metric is not a physical unit that is matter scientifically. As such, we further compute percent differences for uncertainty of trend estimate between the simulated medians and the theoretical (truth) for the monthly series as given in Table 2 at selective record lengths of 10, 30, 50, and 100 yr (Fig. 8). General observation for the L08 method is that as record lengths increase, percent differences from the truth decrease for all series. This observation can also be made for all AR1 series, including white noise, estimated by the W98 method (Fig. 8a). For AR2 series estimated by the W98 method (Figs. 8b–d), this observation is true only for the AR2_1a and AR2_2b. For all AR1 and white noise series, the W98 method gives smaller percent differences than those from the L08 (Fig. 8a). The maximum percent difference from the theoretical (truth) across all record lengths for the white noise monthly series is 1% from the W98 and 2% from the L08. The AR1 series has a maximum difference of 26% using the W98 and 38% using the L08. Therefore, the W98 is preferred over the L08 for all AR1 and white noise series. For AR2 series (Figs. 8b,d), the L08 gives smaller percent differences than those from the W98 for record lengths of more than 10 yr but gives larger differences for a record length of 10 yr. Maximum differences are 17% from the W98 and 26% from the L08 for AR2 series with a mixture of two exponential decay ACF patterns (case 1 in Fig. 8b). Maximum differences are 22% from the W98 and 20% from the L08 for the AR2 series with single exponential decay ACF pattern (case 3 in Fig. 8d). For the AR2 series with damped sinusoid ACFs (AR2 case 2 in Fig. 8c), the W98 gives a maximum of 73% differences for the AR2 series, while the L08 gives a maximum of 30%.

Fig. 8.
Fig. 8.

Percent differences for uncertainty of trend estimate between the simulated median and the theoretical (truth) for the monthly series as given in Table 2 at record lengths of 10, 30, 50, and 100 yr: (a) AR1 series; (b) AR2 series with ACF exhibiting a mixture of two exponential decay patterns (case 1, AR2_1); (c) AR2 series with ACF exhibiting damped sinusoid pattern (case 2, AR2_2); and (d) AR2 series with ACF exhibiting a single exponential decay pattern (case 3, AR2_3).

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

3. Trend analysis for Clouds and the Earth’s Radiant Energy System

CERES instruments measure broadband radiances in three channels: the shortwave (wavelength from 0.2 to 5 μm), the longwave (wavelength from 5 to 50 μm), and the total (wavelength from 0.2 to 50 μm). Observed radiances are converted to TOA irradiances using angular distribution models (ADMs) (Loeb et al. 2001, 2005). A detailed description of CERES and its instruments is provided in Wielicki et al. (1996).

The 136 months of reflected shortwave and emitted longwave irradiances at TOA estimated from observed CERES broadband radiances from March 2000 through June 2011 [Energy Balanced and Filled (EBAF) edition 2.6r; Loeb et al. 2009] are used in this study. The reflected shortwave and emitted longwave irradiances monthly series are globally averaged and deseasonalized. No statistically significant trend was found in either deseasonalized series. This is an indication that the current record length may be insufficient to detect any irradiance change. Figure 9 shows time series plots for deseasonalized and detrended global average reflected shortwave and longwave data. The standard deviations for these series are 0.51 and 0.45 W m−2 for shortwave and longwave, respectively. These values are consistent with those reported in Kato (2009).

Fig. 9.
Fig. 9.

Time series plots of monthly deseasonalized and detrended global averages for (a) reflected shortwave irradiances and (b) emitted longwave irradiances.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

Figure 10 gives autocorrelation functions (Figs. 10a,b) for both monthly series. Figures 10c and 10d provide autocorrelation functions for both monthly series if they were to behave as the perfect AR1 series, using the estimated lag-1 sample autocorrelation coefficient from the shortwave (Fig. 10c) and longwave (Fig. 10d). The ACFs of both series (Figs. 10a,b) do not exhibit purely damped sinusoid.

Fig. 10.
Fig. 10.

Autocorrelation functions of deseasonalized and detrended monthly series for (a) reflected shortwave irradiances and (b) emitted longwave irradiances. Autocorrelation functions if the (c) reflected shortwave and (d) emitted longwave irradiances were a perfect AR1 time series using lag-1 sample autocorrelation coefficients estimated from the corresponding series.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

Using Eqs. (3) and (2), the uncertainties of trend estimate for the shortwave series are 1.27 and 1.48 mW m−2 for the W98 and L08, respectively. For the longwave series, the uncertainties of trend estimate for the longwave are 1.40 and 2.31 mW m−2 for the W98 and L08, respectively. Uncertainty of trend estimate for the longwave series is higher than that for the shortwave series. By examining Figs. 10a and 10b, the longwave irradiance series has more significant lags (at a 95% confidence level) than the shortwave series. Consequently, the longwave series takes a longer time to have monthly data uncorrelated (i.e., larger decorrelation time), and, in turn, its estimated uncertainty is higher than that of the shortwave series.

The shortwave ACF looks similar to the perfect AR1 (Figs. 10a,c). From section 2, the simulations of the AR1 series for 10-yr record length (Fig. 8a) have shown that the W98 method produces better uncertainty (in terms of smaller percent differences from the theoretical value) than that from the L08 method. Therefore, we recommend using the W98 method for the shortwave series. The longwave ACF does not exhibit a purely exponential decay nor damped sinusoid pattern, but rather a mixture of both patterns, making the exponential decay slower than a purely exponential decay pattern. This behavior can be observed by comparing Figs. 10b and 10d. The W98, which is based on exponential decay of the ACF pattern similar to Fig. 10d, underestimates the uncertainty of trend estimate for the longwave series with its ACF as in Fig. 10b, while the L08 provides a more reasonable value. Therefore, we recommend using the L08 method for the longwave series.

Another metric of interest is the number of years n needed to detect a trend (per year) of magnitude with 90% testing power,1 as estimated by the W98:
e8
The number of years estimated by the L08 is similar to Eq. (8), with replacing . Figure 11 graphs the estimates of time (in years) to detect the various ratios. As the trend magnitude (or the ratio) gets smaller, the time to detect the smaller magnitude takes longer. For the reflected shortwave (Fig. 11a), there is a slight difference between the W98 and L08 to detect various ratios, with a maximum difference of about 6 yr at the ratio of 0.01. For the longwave (Fig. 11b), there is a noticeable difference between the two methods to detect various ratios, with a maximum difference of about 14 yr at the ratio of 0.01.
Fig. 11.
Fig. 11.

Number of years required to detect various ratios with 90% testing power (the power to detect correctly 90% of the time a trend that is at least twice as large as its uncertainty) by the W98 and L08 methods: (a) the reflected shortwave irradiances and (b) the emitted longwave irradiances.

Citation: Journal of Climate 27, 9; 10.1175/JCLI-D-13-00400.1

As an example, based on our earlier recommendation of using the W98 to estimate the uncertainty for the reflected shortwave irradiances, if the trend per year is about 5.06 mW m−2 [i.e., of 0.51 W m−2 and the ratio of 0.01], it will take about 52.3 yr to detect it using the W98 method. For the emitted longwave irradiances, based on our earlier recommendation of using the L08, if the trend per year is about 4.49 mW m−2 [i.e., of 0.45 W m−2 and the ratio of 0.01], it will take about 84.4 yr to detect it using the L08 method.

4. Conclusions

In this paper, two trend analysis methods, the methods of W98 and L08, are compared. The two methods are based on linear regression models with covariance structures of the time series data properly accounted for under different underlying assumptions. The W98 assumes that the time series data follow the first-order autoregressive model (AR1), where its autocorrelation function (ACF) decays exponentially with lags. In practice, only the lag-1 sample autocorrelation coefficient is used to estimate the uncertainty of the trend estimate making it simple to use. However, with any departure from the exponential decay pattern, the W98 is expected to perform poorly.

The L08 method, on the other hand, does not assume any autoregressive model or any correlation structure. It uses theoretical autocorrelation coefficients at all lags (i.e., , where is the lag μ autocorrelation coefficient) in the analysis making it ideally versatile. In practice, the theoretical coefficients of the time series are not known, but can be estimated by its sample autocorrelation coefficients from the available time series data of length T months. Taking sample coefficients at all available lags in estimating L08’s uncertainty of the trend will provide zero uncertainty; specifically, is zero. Therefore, it is necessary to set up the criterion to determine which sample autocorrelation coefficients to include in the computation. The criterion used in this study is to include only the statistically significant coefficients up to 10 log10(T) lags in the summation.

Because the term is directly related to the uncertainty of trend estimate [Eqs. (2) and (3)] from the two methods, comparing this term estimated by the two methods is adequate in a Monte Carlo simulation study. The comparison is performed by the simulation study with the theoretically and randomly generated theoretical AR1 time series with of 0.0, 0.3, 0.6, and 0.9, representing white noise, low-, medium-, and high-correlated series. The study also extends to the theoretically and randomly generated second-order autoregressive (AR2) series. We limit our simulation study to AR2 for simplicity because it already captures phenomena that are likely to complicate the W98.

For the AR2 with damped sinusoid ACFs, the L08 gives reasonable medians, while the W98 gives significantly higher medians than the theoretical values. The ranges of distributions and tail regions of the distributions from both methods are smaller as record lengths increase. Ranges are smaller when the W98 is used. The L08 occasionally gives a negative τ value, which, in turn, makes a negative value. For the shorter record lengths, the L08’s tail regions are negative in value; however, the negative tail regions become smaller as the record lengths increase.

From the Monte Carlo simulation study, the white noise monthly series has a maximum percent difference from the theoretical (truth) of 1% and 2% from the W98 and L08, respectively. The AR1 series has a maximum difference of 26% using the W98 and 38% using the L08. The AR2 series with a mixture of two exponential decay ACF patterns has a maximum difference of 17% using the W98 and of 26% using the L08. The AR2 series with single exponential decay ACF pattern has a maximum difference of 22% using the W98 and of 20% using the L08. The AR2 series with the damped sinusoid ACF pattern has a maximum of 73% using the W98 and 30% using the L08.

Both methods are then applied to the time series of deseasonalized anomalies derived from monthly and globally averaged reflected shortwave and emitted longwave irradiances at the top of the atmosphere (TOA), as estimated from CERES broadband radiances measurements during March 2000 through June 2011. Examination of the autocorrelation structure shows that the reflected shortwave series has an exponential decay pattern, while the longwave series has a mixture of exponential decay and damped sinusoid patterns. Therefore, we recommend using the W98 with the reflected shortwave series and the L08 with the longwave series for trend analysis. The uncertainty of trend estimate for the shortwave series is approximately 1.27 mW m−2. Its month-to-month standard deviation is 0.51 W m−2, and if the trend per year is 100 times smaller than this standard deviation (i.e., trend per year of approximately 0.0051 W m−2), it will take about 52.3 yr to detect it. For the longwave series, the uncertainty of the trend estimate is about 2.31 mW m−2. Its month-to-month standard deviation is 0.45 W m−2, and if the trend per year is 100 times smaller than this standard deviation (i.e., trend per year of approximately 0.0045 W m−2), it will take about 84.4 yr to detect it. Note that while the results in this paper are applied to radiation data as an example, they apply in general to all climate change time series.

One potential for future study is to explore other more accurate methods for estimating for the L08. One suggested method is to fit an autoregressive moving average (ARMA) model and use its model parameter estimates for estimation. Some bias correction methods (Lee and Lund 2004) could be used to substantially increase the accuracy of the estimation.

We have shown that estimates of climate trend uncertainties can be sensitive to the methods used. The tests performed in this paper can be applied to any climate time series to determine if the results are sensitive to the method used, either the W98 or L08. If the results are sensitive, then the choice of the method should be based on the physical nature shown in the autocorrelation structure of the climate time series.

Acknowledgments

We thank Dr. Stephen Leroy for useful discussions and all reviewers’ comments for valuable insights. We also thank Ms. Amber Richards and Dr. Joe A. Walker for proof reading the manuscript. This work is supported by the NASA CLARREO project.

APPENDIX

Derivation for the AR2 Theoretical Value Given in Section 2a

The derivation of uncertainty of trend estimate for the AR2 model is an extension of the AR1’s derivation as in Tiao et al. (1990). Let be the measurement of interest. The linear trend model assumes
ea1
where is a constant term, is the seasonal component, is the linear trend function, is the magnitude of the trend per year, is the unexplained noise of the measurement data, and months. It is assumed that the noise follows an AR2 model with
ea2
where and are parameters of the model. From Eq. (A1), we can obtain
ea3
and
ea4
From Eqs. (A1), (A3), and (A4),
ea5
Replacing the last term on the right-hand side with [from Eq. (A2)] and in Eq. (A5), we get
ea6
Letting , , , and , Eq. (A6) becomes
ea7
The estimate of the regression coefficient can be obtained essentially by ordinary least squares calculations applied to Eq. (A7). Therefore,
eq3
Because and , therefore . As such,
ea8
And from simple linear regression of Eq. (A7),
ea9
Because , Eq. (A9) becomes
ea10
Because is trend per year, the variance of trend per month can be computed from
ea11
Comparing Eq. (A11) with Eqs. (2) or (3), we get
ea12

REFERENCES

  • Hassani, H., 2010: A note on the sum of the sample autocorrelation function. Physica A, 389, 1601–1606, doi:10.1016/j.physa.2009.12.050.

    • Search Google Scholar
    • Export Citation

  • Hill, W. J., P. N. Sheldon, and J. J. Tiede, 1977: Analyzing worldwide total ozone for trends. Geophys. Res. Lett., 4, 21–24, doi:10.1029/GL004i001p00021.

    • Search Google Scholar
    • Export Citation

  • Kato, S., 2009: Interannual variability of the global radiation budget. J. Climate, 22, 4893–4907, doi:10.1175/2009JCLI2795.1.

  • Lee, J., and R. Lund, 2004: Revisiting simple linear regression with autocorrelated errors. Biometrika, 91, 240–245, doi:10.1093/biomet/91.1.240.

    • Search Google Scholar
    • Export Citation

  • Leroy, S. S., J. G. Anderson, J. Dykema, and R. Goody, 2008a: Testing climate models using thermal infrared spectra. J. Climate, 21, 1863–1875, doi:10.1175/2007JCLI2061.1.

    • Search Google Scholar
    • Export Citation

  • Leroy, S. S., J. G. Anderson, and G. Ohring, 2008b: Climate signal detection times and constraints on climate benchmark accuracy requirements. J. Climate, 21, 841–846, doi:10.1175/2007JCLI1946.1.

    • Search Google Scholar
    • Export Citation

  • Loeb, N. G., K. J. Priestley, D. P. Kratz, E. B. Geier, R. N. Green, B. A. Wielicki, P. O. R. Hinton, and S. K. Nolan, 2001: Determination of unfiltered radiances from the Clouds and the Earth’s Radiant Energy System (CERES) instrument. J. Appl. Meteor., 40, 822–835, doi:10.1175/1520-0450(2001)040<0822:DOURFT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Loeb, N. G., S. Kato, K. Loukachine, and N. M. Smith, 2005: Angular distribution models for top-of-atmosphere radiative flux estimation from the Clouds and the Earth’s Radiant Energy System instrument on the Terra satellite. Part I: Methodology. J. Atmos. Oceanic Technol., 22, 338–351, doi:10.1175/JTECH1712.1.

    • Search Google Scholar
    • Export Citation

  • Loeb, N. G., B. A. Wielicki, D. R. Doelling, G. L. Smith, D. K. Keyes, S. Kato, N. Manalo-Smith, and T. Wong, 2009: Toward optimal closure of the earth’s top-of-atmosphere radiation budget. J. Climate, 22, 748–766, doi:10.1175/2008JCLI2637.1.

    • Search Google Scholar
    • Export Citation

  • Montgomery, D. C., and G. C. Runger, 1994: Applied Statistics and Probability for Engineers. John Wiley and Sons, 895 pp.

  • Montgomery, D. C., C. L. Jennings, and M. Kulahci, 2008: Introduction to Time Series Analysis and Forecasting. John Wiley and Sons, 441 pp.

    • Search Google Scholar
    • Export Citation

  • Reinsel, G. C., G. C. Tiao, J. J. DeLuisi, S. Basu, and K. Carriere, 1989: Trend analysis of aerosol-corrected Umkehr ozone profile data through 1987. J. Geophys. Res., 94, 16 373–16 386, doi:10.1029/JD094iD13p16373.

    • Search Google Scholar
    • Export Citation

  • Ross, S. M., 2006: Simulation. 4th ed. Elsevier, 298 pp.

  • Tiao, G. C., and Coauthors, 1990: Effects of autocorrelation and temporal sampling schemes on estimates of trend and spatial correlation. J. Geophys. Res., 95, 20 507–20 517, doi:10.1029/JD095iD12p20507.

    • Search Google Scholar
    • Export Citation

  • Venables, W. N., and B. D. Ripley, 2002: Modern Applied Statistics with S. 4th ed. Springer-Verlag, 495 pp.

  • Von Storch, H., and F. W. Zwiers, 1999: Statistical Analysis in Climate Research. Cambridge University Press, 484 pp.

  • Weatherhead, E. C., and Coauthors, 1998: Factors affecting the detection of trends: Statistical considerations and applications to environmental data. J. Geophys. Res., 103, 17 149–17 161, doi:10.1029/98JD00995.

    • Search Google Scholar
    • Export Citation

  • Wielicki, B. A., B. R. Barkstrom, E. F. Harrison, R. B. Lee III, G. L. Smith, and J. E. Cooper, 1996: Clouds and the Earth’s Radiant Energy System (CERES): An Earth Observing System experiment. Bull. Amer. Meteor. Soc., 77, 853–868, doi:10.1175/1520-0477(1996)077<0853:CATERE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
1

The testing power is the power to detect correctly 90% of the time a trend that is at least twice as large as its uncertainty (i.e., > 2).

Save
Cover Journal of Climate
Journal of Climate

  • Hassani, H., 2010: A note on the sum of the sample autocorrelation function. Physica A, 389, 1601–1606, doi:10.1016/j.physa.2009.12.050.

    • Search Google Scholar
    • Export Citation

  • Hill, W. J., P. N. Sheldon, and J. J. Tiede, 1977: Analyzing worldwide total ozone for trends. Geophys. Res. Lett., 4, 21–24, doi:10.1029/GL004i001p00021.

    • Search Google Scholar
    • Export Citation

  • Kato, S., 2009: Interannual variability of the global radiation budget. J. Climate, 22, 4893–4907, doi:10.1175/2009JCLI2795.1.

  • Lee, J., and R. Lund, 2004: Revisiting simple linear regression with autocorrelated errors. Biometrika, 91, 240–245, doi:10.1093/biomet/91.1.240.

    • Search Google Scholar
    • Export Citation

  • Leroy, S. S., J. G. Anderson, J. Dykema, and R. Goody, 2008a: Testing climate models using thermal infrared spectra. J. Climate, 21, 1863–1875, doi:10.1175/2007JCLI2061.1.

    • Search Google Scholar
    • Export Citation

  • Leroy, S. S., J. G. Anderson, and G. Ohring, 2008b: Climate signal detection times and constraints on climate benchmark accuracy requirements. J. Climate, 21, 841–846, doi:10.1175/2007JCLI1946.1.

    • Search Google Scholar
    • Export Citation

  • Loeb, N. G., K. J. Priestley, D. P. Kratz, E. B. Geier, R. N. Green, B. A. Wielicki, P. O. R. Hinton, and S. K. Nolan, 2001: Determination of unfiltered radiances from the Clouds and the Earth’s Radiant Energy System (CERES) instrument. J. Appl. Meteor., 40, 822–835, doi:10.1175/1520-0450(2001)040<0822:DOURFT>2.0.CO;2.

    • Search Google Scholar
    • Export Citation

  • Loeb, N. G., S. Kato, K. Loukachine, and N. M. Smith, 2005: Angular distribution models for top-of-atmosphere radiative flux estimation from the Clouds and the Earth’s Radiant Energy System instrument on the Terra satellite. Part I: Methodology. J. Atmos. Oceanic Technol., 22, 338–351, doi:10.1175/JTECH1712.1.

    • Search Google Scholar
    • Export Citation

  • Loeb, N. G., B. A. Wielicki, D. R. Doelling, G. L. Smith, D. K. Keyes, S. Kato, N. Manalo-Smith, and T. Wong, 2009: Toward optimal closure of the earth’s top-of-atmosphere radiation budget. J. Climate, 22, 748–766, doi:10.1175/2008JCLI2637.1.

    • Search Google Scholar
    • Export Citation

  • Montgomery, D. C., and G. C. Runger, 1994: Applied Statistics and Probability for Engineers. John Wiley and Sons, 895 pp.

  • Montgomery, D. C., C. L. Jennings, and M. Kulahci, 2008: Introduction to Time Series Analysis and Forecasting. John Wiley and Sons, 441 pp.

    • Search Google Scholar
    • Export Citation

  • Reinsel, G. C., G. C. Tiao, J. J. DeLuisi, S. Basu, and K. Carriere, 1989: Trend analysis of aerosol-corrected Umkehr ozone profile data through 1987. J. Geophys. Res., 94, 16 373–16 386, doi:10.1029/JD094iD13p16373.

    • Search Google Scholar
    • Export Citation

  • Ross, S. M., 2006: Simulation. 4th ed. Elsevier, 298 pp.

  • Tiao, G. C., and Coauthors, 1990: Effects of autocorrelation and temporal sampling schemes on estimates of trend and spatial correlation. J. Geophys. Res., 95, 20 507–20 517, doi:10.1029/JD095iD12p20507.

    • Search Google Scholar
    • Export Citation

  • Venables, W. N., and B. D. Ripley, 2002: Modern Applied Statistics with S. 4th ed. Springer-Verlag, 495 pp.

  • Von Storch, H., and F. W. Zwiers, 1999: Statistical Analysis in Climate Research. Cambridge University Press, 484 pp.

  • Weatherhead, E. C., and Coauthors, 1998: Factors affecting the detection of trends: Statistical considerations and applications to environmental data. J. Geophys. Res., 103, 17 149–17 161, doi:10.1029/98JD00995.

    • Search Google Scholar
    • Export Citation

  • Wielicki, B. A., B. R. Barkstrom, E. F. Harrison, R. B. Lee III, G. L. Smith, and J. E. Cooper, 1996: Clouds and the Earth’s Radiant Energy System (CERES): An Earth Observing System experiment. Bull. Amer. Meteor. Soc., 77, 853–868, doi:10.1175/1520-0477(1996)077<0853:CATERE>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • <sc>Fig</sc>. 1.
    View in gallery
    Fig. 1.

    Computer simulation framework.

  • <sc>Fig</sc>. 2.
    View in gallery
    Fig. 2.

    The ACF of all 100-yr monthly AR1 models with the series number from Table 2.

  • <sc>Fig</sc>. 3.
    View in gallery
    Fig. 3.

    The ACF of all 100-yr monthly AR2 series with the series number from Table 2.

  • <sc>Fig</sc>. 4.
    View in gallery
    Fig. 4.

    Sensitivity of the number of simulations to the simulation errors (as percent of its simulated mean) for all 100-yr monthly series (i.e., AR1 and AR2 series) with the W98 method.

  • <sc>Fig</sc>. 5.
    View in gallery
    Fig. 5.

    As in Fig. 4, but for the L08 method.

  • <sc>Fig</sc>. 6.
    View in gallery
    Fig. 6.

    The box plot of simulated values lengths as estimated from the two methods for the selective monthly series given in Table 2 at different record for (a) AR1_0, (b) AR1_3, (c) AR2_1c, (d) AR2_2c, (e) AR2_3a, and (f) AR2_3c. The corresponding theoretical values are given by the dark gray horizontal lines in all figures. The vertical ranges of the figures are not extended out to show the extreme values in order to facilitate readability of more critical information such as the quartile and median values.

  • <sc>Fig</sc>. 7.
    View in gallery
    Fig. 7.

    The likelihood of negative τ values from the L08 for all simulated series, based on the percent total of the 10 000 computer simulations: (a) AR1 series; (b) AR2 series with ACF exhibiting a mixture of two exponential decay patterns (case 1, AR2_1); (c) AR2 series with ACF exhibiting damped sinusoid pattern (case 2, AR2_2); and (d) AR2 series with ACF exhibiting a single exponential decay pattern (case 3, AR2_3).

  • <sc>Fig</sc>. 8.
    View in gallery
    Fig. 8.

    Percent differences for uncertainty of trend estimate between the simulated median and the theoretical (truth) for the monthly series as given in Table 2 at record lengths of 10, 30, 50, and 100 yr: (a) AR1 series; (b) AR2 series with ACF exhibiting a mixture of two exponential decay patterns (case 1, AR2_1); (c) AR2 series with ACF exhibiting damped sinusoid pattern (case 2, AR2_2); and (d) AR2 series with ACF exhibiting a single exponential decay pattern (case 3, AR2_3).

  • <sc>Fig</sc>. 9.
    View in gallery
    Fig. 9.

    Time series plots of monthly deseasonalized and detrended global averages for (a) reflected shortwave irradiances and (b) emitted longwave irradiances.

  • <sc>Fig</sc>. 10.
    View in gallery
    Fig. 10.

    Autocorrelation functions of deseasonalized and detrended monthly series for (a) reflected shortwave irradiances and (b) emitted longwave irradiances. Autocorrelation functions if the (c) reflected shortwave and (d) emitted longwave irradiances were a perfect AR1 time series using lag-1 sample autocorrelation coefficients estimated from the corresponding series.

  • <sc>Fig</sc>. 11.
    View in gallery
    Fig. 11.

    Number of years required to detect various ratios with 90% testing power (the power to detect correctly 90% of the time a trend that is at least twice as large as its uncertainty) by the W98 and L08 methods: (a) the reflected shortwave irradiances and (b) the emitted longwave irradiances.

Fig. 1.

Computer simulation framework.
Fig. 2.

The ACF of all 100-yr monthly AR1 models with the series number from Table 2.
Fig. 3.

The ACF of all 100-yr monthly AR2 series with the series number from Table 2.
Fig. 4.

Sensitivity of the number of simulations to the simulation errors (as percent of its simulated mean) for all 100-yr monthly series (i.e., AR1 and AR2 series) with the W98 method.
Fig. 5.

As in Fig. 4, but for the L08 method.
Fig. 6.

The box plot of simulated values lengths as estimated from the two methods for the selective monthly series given in Table 2 at different record for (a) AR1_0, (b) AR1_3, (c) AR2_1c, (d) AR2_2c, (e) AR2_3a, and (f) AR2_3c. The corresponding theoretical values are given by the dark gray horizontal lines in all figures. The vertical ranges of the figures are not extended out to show the extreme values in order to facilitate readability of more critical information such as the quartile and median values.
Fig. 7.

The likelihood of negative τ values from the L08 for all simulated series, based on the percent total of the 10 000 computer simulations: (a) AR1 series; (b) AR2 series with ACF exhibiting a mixture of two exponential decay patterns (case 1, AR2_1); (c) AR2 series with ACF exhibiting damped sinusoid pattern (case 2, AR2_2); and (d) AR2 series with ACF exhibiting a single exponential decay pattern (case 3, AR2_3).
Fig. 8.

Percent differences for uncertainty of trend estimate between the simulated median and the theoretical (truth) for the monthly series as given in Table 2 at record lengths of 10, 30, 50, and 100 yr: (a) AR1 series; (b) AR2 series with ACF exhibiting a mixture of two exponential decay patterns (case 1, AR2_1); (c) AR2 series with ACF exhibiting damped sinusoid pattern (case 2, AR2_2); and (d) AR2 series with ACF exhibiting a single exponential decay pattern (case 3, AR2_3).
Fig. 9.

Time series plots of monthly deseasonalized and detrended global averages for (a) reflected shortwave irradiances and (b) emitted longwave irradiances.
Fig. 10.

Autocorrelation functions of deseasonalized and detrended monthly series for (a) reflected shortwave irradiances and (b) emitted longwave irradiances. Autocorrelation functions if the (c) reflected shortwave and (d) emitted longwave irradiances were a perfect AR1 time series using lag-1 sample autocorrelation coefficients estimated from the corresponding series.
Fig. 11.

Number of years required to detect various ratios with 90% testing power (the power to detect correctly 90% of the time a trend that is at least twice as large as its uncertainty) by the W98 and L08 methods: (a) the reflected shortwave irradiances and (b) the emitted longwave irradiances.
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 927 508 101
PDF Downloads 287 36 5