1. Introduction

Tropical climate prediction has in recent decades been pursued by both empirical methods and numerical modeling (reviews in Hastenrath 1985, 330–352; 1986; 1990; 1995a; 1995b; 347–373; 2002; Barnston et al. 1994, 1999; Palmer and Anderson 1994; Carson 1998; Latif et al. 1998; Anderson et al. 1999; Goddard et al. 2001). The seasonal forecasting of equatorial Pacific sea surface temperature (SST) offered the opportunity to compare the performance of the two approaches. Accordingly, Barnston et al. (1994) found that numerical modeling did not surpass the performance of empirically based prediction. Half a decade later, Barnston et al. (1999) confirmed the earlier assessment and expressed reservation on whether numerical modeling would ever outperform empirical methods. Consistent with this are the perceptions conveyed in Anderson et al. (1999), Webster et al. (1998), and Bamzai and Shukla (1999). Performance comparisons for key tropical regions are in order.

An early target of opportunity in tropical climate prediction has been the classic problem of northeast Brazil (Nordeste) droughts. An earlier paper (Greischar and Hastenrath 2000) presented a verification of performance of decade-long real-time empirically based forecasting for the Nordeste at the University of Wisconsin. Another article (Folland et al. 2001) verified the performance of the sustained real-time forecasting by the Hadley Centre of the U.K. Met Office. Comparison between various empirical exercises is compromised by the different predictands used. In addition to empirical methods, the paper included 4 yr of numerical model results. For this short period numerical modeling did not surpass the performance of the empirically based forecasts.

A numerical modeling experiment at the International Research Institute for Climate Prediction (IRI) of Columbia University provides data over a 3-decade-long period suitable for a performance comparison with the empirical approach at the University of Wisconsin. This is the objective of the present paper. Sections 2 and 3 describe the empirical and numerical methods, section 4 presents the comparison of performance, section 5 compares two numerical modeling experiments, and a synthesis is offered in section 6.

2. Empirical forecasting

Details of the data and methods used in our empirical forecasting have been presented in earlier papers (Hastenrath and Greischar 1993b; Greischar and Hastenrath 2000), and a short summary suffices here. The predictand is an index of March–June (MJ) rainfall in the northern Nordeste, based on a network of 27 rain gauges (see Fig. 1). Figure 2 illustrates the concentration of rainfall activity during these months. This dimensionless index is the all-station average of normalized departures. For the average of the 27 stations and with the reference period 1912–56, the index corresponds to a mean MJ rainfall total of 500 mm and standard deviation of 200 mm. The predictors contain information through the end of January. Four indices are of interest here, namely, an index of preseason October–January rainfall at 27 stations, January indices of the fields of meridional wind component and of SST in the tropical Atlantic, and of equatorial Pacific SST (PT; see Fig. 1). This PT index (Hastenrath and Greischar 1993b) has been calculated as described by Wright (1984). These same four exindes served as input to stepwise multiple regression (SR) and neural networking (NN). Relationships were developed during the 1921–57 period that was used as the dependent portion of the record or “training period,” 1958– 89 served as the independent portion of the record or “verification period,” and real-time forecasting continued to 2000 (although for that last year rain gauge measurements were no longer received for verfication). Thus, the years 1958–99 are available for verification of forecast performance.

3. Numerical modeling

For the numerical modeling experiment, the predictand was the MJ rainfall averaged over three grid points (see Fig. 1) obtained from the dataset of New et al. (1999), which was produced from precipitation observations and given in millimeters per day. This index is hereafter denoted as XO.

The computer model used was ECHAM4.5, the resolution was T42L19 (a truncated series of spherical harmonics with triangular truncation at wavenumber 42, with 19 irregularly spaced levels; Roeckner et al. 1996), and the experiment comprised an ensemble of 12 runs. A description for a similar experiment with a different model is given by Goddard and Mason (2002). Global SST information through February served as input to model the MJ rainfall. The values produced from the numerical modeling are denoted by XM. The model run covers the years 1968–2002.

In addition to this prognostic modeling using information through the end of February (“persisted SST anomalies”; Goddard and Mason 2002), another diagnostic experiment was available with the same resolution and with 24 ensemble members covering the years 1950–98. It used information even beyond the end of February (“simulation”; Goddard and Mason 2002). The Nordeste rainfall values produced from this experiment are denoted by XE. Section 5 compares the experience from the prognostic (persisted SST anomalies; XM) versus the diagnostic (simulation; XE) modeling.

4. Verification of performance

From the preceding sections 2 and 3, one recognizes the 32 yr (1968–99) as being common to both the empirical prediction and the numerical modeling experiment. The index series of the observed MJ and XO, as well as that of the calculated SR, NN, and XM, were converted into units of millimeters of rainfall over the entire MJ season. Time series plots of the five rainfall series are displayed in Fig. 3, and Table 1 shows correlations between these rainfall series and also the PT index (Fig. 1). Figure 3a illustrates the overall consistency between the observed series MJ and XO correlated at +0.96. The calculated series SR, NN, and XM exhibit some differences. The means and standard deviations of the five precipitation series over the 1968–99 period are presented in Table 2. The means and standard deviations of the observed series MJ and XO are similar. The calculated series SR and NN also have similar means and somewhat smaller standard deviations, whereas the mean of XM is distinctly smaller than that of the observed XO and especially MJ. The verification of forecast performance and comparison of the empirical and numerical modeling approaches are further presented in Fig. 4 and Table 3. In comparing the approaches it must be recalled that conditions were more demanding for the empirical forecasting, which used input information through the end of January only, whereas the numerical modeling experiment had the privilege of input information through February.

Following common practice, four statistics are used to measure the forecast skill (Nicholls 1984): the correlation coefficient (corr) between the calculated (clc) and observed (obs), the root-mean-square error (rmse), the absolute error (abse), and the bias (bias). The last three statistics are compiled as follows:

View Expanded

where the summation extends over the n forecast years. Results are presented in Table 3.

Figures 4a,b illustrate for stepwise multiple regression and neural networking similar scatter and no prevalence of positive or negative differences. By contrast, Fig. 4c shows for the numerical modeling larger scatter and overwhelmingly negative differences. The pictorial display in Fig. 4 is complemented by the quantitative assessment in Table 3. For the empirical approach correlation is high and there is little bias. By contrast, for the numerical modeling correlation is weaker, errors are larger, and bias is negative and large. Known bias in numerical modeling can be readily accounted for. Sources of bias in SST-driven numerical modeling have been examined by Goddard and Mason (2002).

As the numerical model is driven exclusively by SST, it is interesting to examine whether it might perform better during Pacific warm years. In the average over the five warmest (1973, 1983, 1992, 1995, 1998) as compared to the five coldest years (1971, 1974, 1976, 1989, 1999) in Fig. 4d, the numerical model gives about the same differences against what is observed (DM, Fig. 4c).

5. Comparison of prognostic and diagnostic models

Figure 3d illustrates the time series of rainfall values obtained from the two numerical modeling experiments, Figs. 5 and 4c show their differences DE = XE − XO and DM = XM − XO against the observed XO, and Tables 2 and 3 present quantitative evaluations of the agreement between modeled and observed values. For the modeled XE and XM, Fig. 3d and Table 2 show similar mean and standard deviation. However, not surprisingly, Figs. 5 and 4c and Table 3 bear out better agreement with the observed XO for the diagnostic XE than for the prognostic XM. For XE as compared to XM, the scatter is somewhat larger, the errors smaller, and the negative bias still large (Table 3 and Figs. 5 and 4c), a matter considered by Goddard and Mason (2002). Most remarkable in Table 3, the correlation with XO reaches an extremely high value, not seen in forecasting proper.

Diagnostic considerations are also in order regarding the January versus February lead time of forecasts. The numerical modeling experiment at forecasting reported in sections 3 and 4 enjoyed the privilege of input information through February, by which time the SST anomaly pattern in the tropical Atlantic is known to be much better developed than in January (Curtis and Hastenrath 1995). Empirical analyses based on earlier work (Hastenrath and Greischar 1993b; not detailed here) show that SST input through February captures a much larger portion of the variance in Nordeste rainfall that SST information to the end of January. This insight is pertinent to the February versus January SST input to numerical modeling.

6. Conclusions

Tropical climate prediction is receiving increased attention. Empirical and numerical modeling approaches need to be cultivated in tandem, as suggested before (Hastenrath 1995a, 2002), considering that this would be mutually fruitful for diagnostic understanding. Expectations for an eventual superior performance of numerical modeling persist (discussions in Barnston et al. 1994, 1999; Anderson et al. 1999), and systematic comparisons are desirable. Comparisons of performance of climate forecast methods require consistency in the predictand and a long common reference period. Suitable targets have accordingly been rare. The classic problem of the recurrent droughts of Brazil's Nordeste has been a target of opportunity for the development of both empirical and numerical modeling approaches. The report on the sustained real-time forecasting for the Nordeste by the Hadley Centre of the Met Office (Folland et al. 2001) included a comparison with 4 yr of predictions from numerical modeling; these did not surpass the performance of the empirical approach.

The empirical methods developed at the University of Wisconsin have been documented before (Hastenrath and Greischar 1993a; Greischar and Hastenrath 2000). The predictand is the March–June rainfall at a network of 27 stations in the northern Nordeste. Predictors are the preseason (October–January) rainfall in the Nordeste, as well as January indices of the fields of meridional wind component and SST in the tropical Atlantic and equatorial Pacific. These serve as input to stepwise multiple regression and neural networking, and results have been reported earlier (Greischar and Hastenrath 2000). A fortunate opportunity for comparison arose from a recent experiment with a numerical model (ECHAM4.5) conducted at the International Institute for Climate Prediction, Columbia University. This used global SST information through February to predict the March–June rainfall at three grid points in the Nordeste. The predictands for the empirical and numerical model forecast are correlated at +0.96, and the independent portion of record of the empirical prediction shares with the numerical modeling the common period 1968–99.

For numerical modeling, stepwise multiple regression, and neural networking, predicted versus observed rainfall over the 32-yr period have been evaluated in terms of correlation, root-mean-square error, absolute error, and bias. Brazil's Nordeste may be among the more predictable regions of the Tropics. Performance is indeed high for both the numerical modeling and empirical approaches. Numerical modeling accounts for 46% of the variance, although it yields moderate errors and strong negative bias. Statistically not significant, there is some indication for the numerical model to perform better during Pacific warm years. For the empirical methods, errors and bias are small; stepwise multiple regression captures 53% and neural networking 67% of the variance of observed rainfall. Overall, this evaluation complements earlier assessments for other targets (Barnston et al. 1994, 1999; Anderson et al. 1999; Webster et al. 1998), reporting that numerical modeling does not exceed the performance of empirically based climate prediction. Further systematic appraisals for key tropical regions are in order.