Forecast Calibration and Combination: A Simple Bayesian Approach for ENSO

1. Introduction

The El Niño–Southern Oscillation (ENSO) is an important large-scale ocean–atmosphere coupled phenomenon that has large impacts on the climate of many regions around the world (Horel and Wallace 1981; Stoeckenius 1981; Ropelewski and Halpert 1986, 1987, 1989). Since the strong El Niño episode in 1982/83, many efforts have been made to produce routine forecasts of tropical Pacific sea surface temperatures (SST). Long-lead forecasts several months in advance help local governments and industries plan their actions prior to the occurrence of the phenomenon (Patt 2000).

ENSO forecasts are currently produced using either physically derived dynamical climate models or empirical (statistical) relationships based on historical data. For a comprehensive review of ENSO forecasting studies developed during the last two decades see Mason and Mimmack (2002). The comparative skill of these two approaches is a subject of much debate (Berliner et al. 2000b). Recent forecast comparisons suggest that empirical models perform at least as well as dynamical coupled models (Barnston et al. 1999; Anderson et al. 1999). Some studies argue that empirical models perform better (e.g., Landsea and Knaff 2000), while other studies claim that dynamical climate models can give better ENSO forecasts (e.g., Trenberth 1998).

For both medium-range and seasonal forecasts, it is common practice to use the ensemble technique to cope with the probabilistic nature of the forecasts (e.g. Stockdale et al. 1998; Taylor and Buizza 2003; Palmer et al. 2004). However, using only model produced forecast information ignores all prior (historical) knowledge and is prone to model systematic errors. At this point it is worth stressing the distinction between climate model outputs and observed climate/weather. Climate model outputs should not be treated as observed climate because they contain model structural and parametric errors, which should be corrected by calibration against observations.

Given these two distinct approaches to forecasting, it is natural to ask whether combining them may produce a forecast with more skill than either forecast considered separately. Thompson (1977) was one of the first to show that a simple linear combination of two independent 24-h weather predictions, obtained by minimizing the mean-square error of the combined forecast, could reduce the forecast error variance by about 20%. Fraedrich and Leslie (1987) also noted that by linearly combining stochastic short-range forecasts with dynamical model weather predictions it was possible to obtain significantly better prediction skill. Fraedrich and Smith (1989) then extended this approach to seasonal forecasts with lead times of up to 3 months. They linearly combined an empirical forecast with a deterministic model forecast for predicting tropical Pacific SST anomalies. It was shown that by minimizing the combined forecast mean-square error considerable improvement in skill can be obtained. More recently, Metzger et al. (2004) have extended the Fraedrich and Smith (1989) combination scheme to predict Niño-3 index (5°N–5°S, 90°–150°W) anomalies for lead times up to 24 months. They found that the linear combination of empirical and deterministic forecasts can provide improvement in prediction skill if the predictions of individual schemes are independent and of comparable skill. However, only modest skill improvements were found. Krishnamurti et al. (1999, 2000a,b, 2001), Pavan and Doblas-Reyes (2000), and Stefanova and Krishnamurti (2002) have introduced the multimodel method for combining dynamical weather and climate forecasts. The multimodel method linearly combines ensemble forecasts from different models by minimizing the mean-square error of the combined forecast. It has been demonstrated that the multimodel invariably outperforms any of the individual models.

From this brief review, it is clear that there is still a need for more research into how to produce well-calibrated combined forecasts. The aim of this study is to introduce a simple Bayesian approach and to demonstrate it by using monthly Niño-3.4 index (5°N–5°S, 120°–170°W) forecasts at a 5-month lead time. One particular advantage of this method is that it merges valuable past (historical) information with coupled model ensemble forecasts to produce better quality probability estimates of the mean forecast value and its respective uncertainty.

The Bayesian approach has been discussed for decision making in applied meteorology by Epstein (1962) and for statistical inference and prediction in climatology by Epstein (1985). It has also been successfully used in other areas such as hydrology (e.g., Krzysztofowicz 1983; Krzysztofowicz and Herr 2001) and recently in climate studies (e.g., Berliner et al. 2000a,b; Rajagopalan et al. 2002). As pointed out by Mason and Mimmack (2002), ENSO forecasts are usually issued in deterministic terms and very little attention has been directed to careful estimation of forecast uncertainty. This study treats ENSO forecasts in probabilistic terms, with particular attention directed to the estimation of prediction uncertainty. For this particular application, Niño-3.4 index interval forecasts are used to summarize the mean and the variance of the predicted normal distribution.

Section 2 introduces the empirical and coupled model ensemble forecasts of the Niño-3.4 index used in this study. Section 3 describes the Bayesian method used to combine the forecasts, and section 4 presents results of the combined forecasts. Section 5 concludes the article with a summary and a discussion of possible future areas for research.

2. Empirical and coupled model ensemble forecasts of ENSO

These methods here will be demonstrated using 5-month lead forecasts of the December mean Niño-3.4 index starting from conditions at the end of the preceding July. Empirical and coupled model ensemble forecasts available over the T = 13-yr period (1987–99) have been used. This short record is typical of the length of datasets produced by most of the world's climate prediction centers. Details concerning datasets and forecast lead times are given in appendix A. Figure 1 shows the historical (1950–2001) December Niño-3.4 index time series. The largest El Niño (1972, 1982, and 1997) and La Niña (1970, 1973, 1988, and 1998) events can be clearly seen.

a. Empirical forecast of ENSO

1) The empirical model

The simplest 5-month lead empirical model for forecasting the December mean Niño-3.4 index uses linear regression with the preceding July mean Niño-3.4 index historical time series as the linear predictor. That is, θ_t = β_o + β₁ψ_t + ε_t, where θ_t and ψ_t are the December and July Niño-3.4 monthly mean values, respectively; β_o and β₁ are the intercept and slope parameters, respectively; ε_t is a “normal” (Gaussian) random variable with zero mean and variance σ²_o [i.e., ε_t ∼ N(0, σ²_o)]; and t is the year being forecast. This model can be written more explicitly in probabilistic notation as

θ_tψ_tNμ_otσ²_o(1)

with the mean given by

μ_otβ_oβ₁ψ_t(2)

that is, a linear function of the predictor ψ_t. The standard statistical symbol | denotes “given” (conditional upon) and ∼ denotes “is distributed as.”

Figure 2 shows a scatterplot of the December versus the preceding July Niño-3.4 index for the period 1950–2001 (N = 52 observations). The linear regression fit is indicated in Fig. 2 as a solid line. A large amount of the total variance of December is explained by the preceding July Niño-3.4 index (R² = 0.76). This emphasizes the importance of persistence for forecasting the Niño-3.4 index.

2) Empirical model cross validation

To avoid artificial skill, the empirical model has been evaluated using a cross-validation “leave one out” method (Wilks 1995, his section 6.3.6). To produce a forecast for time t, only data at other times (years) different than t have been used to estimate model parameters and errors.

Figure 3a shows empirical forecasts for the target period 1987–99 (thick line), observed values (thin line), and the December climatological mean of 26.5°C (short-dashed line). The 95% prediction interval (P.I.) for θ_t given ψ_t is also shown (gray area surrounded by long-dashed lines). The 95% prediction interval is defined by

μ̂_otσ̂_ot(3)

where μ̂_ot = β̂_o + β̂₁ψ_t is the Niño-3.4 index predicted mean for a particular December and σ̂_ot is the predicted standard deviation given by

View Expanded

where n = N − 1 is the total number of years used in the cross validation, ψ_t = 1/n Σ_i≠t ψ_i is the long-term climatological mean of the July Niño-3.4 index, S²_t = 1/n Σ_i≠t [ψ_i − ψ_t]², and σ̂_o = [1/(n − 2) Σ_i≠t (θ_i − μ̂_oi)²]^1/2 is the estimated empirical model standard deviation (see Draper and Smith 1998, their section 3.1).

Equations (3) and (4) show that the smallest prediction interval is obtained when the predictor equals its mean value ψ_t = ψ_t. On the other hand, by moving away from ψ_t in either direction the prediction interval increases. The greater distance a particular July Niño-3.4 index (ψ_t) is from the climatological mean value (ψ_t), the larger is the extrapolation error made when predicting the following December Niño-3.4 index (θ_t). However, the use of Eq. (4) compared to σ̂_ot = σ̂_o leads to only small changes in practice in the prediction interval, because the S²_t term in the denominator is proportional to the sum of n terms of the same magnitude as the term (ψ_t − ψ_t)². The most precise predictions are obtained for the July Niño-3.4 index values in the “middle” of the observed range of ψ_t, while for more extreme values farther away from the climatological mean, predictions are less precise.

Figure 3a shows that the empirical forecast prediction interval does not vary much from year to year, indicating stability of estimates such as σ̂_o. This simple model provides good forecasts, especially for the 1988 and 1998 La Niña episodes and for the 1997 El Niño episode. Out of the 13 yr the model has only once (in 1987) forecast the Niño-3.4 index outside the 95% P.I. Measures of forecast skill and uncertainty will be discussed in more detail in section 4.

Figure 3b shows the time series of the standardized forecast errors

View Expanded

where μ̂_ot is the forecasted mean, θ_t is the observed value, and σ̂_ot is the prediction standard deviation at time t. If this empirical model is appropriate, the standardized forecast errors should be distributed as independent normally distributed random variables with zero mean and unit variance. This appears to be the case from Fig. 3b. Although some slight sign of serial correlation may suggest the need of future model extensions, the standardized forecast errors appear to have constant variance and are well centered on zero with no obvious large outliers. The periods 1988–90 and 1997–98 have small standardized errors, while 1987, the period 1991–96, and 1999 have larger standardized errors. The largest standardized forecast error occurred in 1987.

3. Bayesian method for combining forecasts

The Bayesian method is a consistent probabilistic approach that can be used for combining historical (climatological) information (θ) with dynamical model ensemble mean forecasts (X). The Bayesian method is firmly based on rigorous probability theory and so can provide well-calibrated probability forecasts.

With no access to a coupled model ensemble mean forecast X, the only possible probabilistic assessment about the observable variable θ has to be based on the assumption that future values of θ will behave like they did in the past. For example, the probability distribution of θ can be estimated by using the climatological probability density function p(θ) estimated from historical observations. In Bayesian theory, p(θ), is known as the prior distribution and encapsulates prior knowledge about likely possible values of θ—from past experience not all values of θ were found to occur equally likely. A more informative prior is the empirical model defined in section 2a.

However, when a particular ensemble mean forecast X = x is known for the future, it is then possible to update the prior p(θ) to obtain the conditional posterior distribution p(θ | X = x). In other words, this is the probability distribution of θ given that the forecast X = x is known. Conditioning on forecasts helps to reduce the uncertainty about future values of θ (Jolliffe and Stephenson 2003, their chapter 9). This procedure is illustrated schematically in Fig. 5. The normal prior probability density (short-dashed line) when combined with a normal likelihood probability density (dashed line) yields a normal posterior probability density (solid line). The posterior distribution p(θ | X = x) is found from the prior p(θ) by making use of Bayes' theorem:

View Expanded

where θ_t is the observable variable at time t and x is a particular value of ensemble mean forecast at time t. Note that both the posterior distribution and the likelihood function are considered to be functions of θ_t. Finally, p(X_t = x) does not depend on θ_t and therefore only plays the role of a normalizing constant (Lee 1997).

The likelihood p(X | θ) of obtaining an ensemble mean forecast X given observations θ is an essential ingredient in the Bayesian updating procedure that can be estimated by stratifying past ensemble mean forecasts (hindcasts) on past observations. The likelihood provides a convenient summary of the calibration and resolution of past forecasts (Jolliffe and Stephenson 2003).

The Bayesian approach has several important advantages over approaches that rely solely on sampling ensembles of coupled model forecasts (e.g., Stockdale et al. 1998; Taylor and Buizza 2003). First, the Bayesian approach appropriately incorporates prior information about the distribution contained in historical observations (i.e., combination). Second, the likelihood estimation provides a natural way of correcting for biases in the model forecasts that often occur in coupled model systems (i.e., calibration). Third, the resulting well-calibrated posterior distribution allows one to generate an arbitrarily large sample (a megaensemble) of possible climate realizations, of use for example in scenario studies of risk and forecast value (Jolliffe and Stephenson 2003, their chapter 8). It should be noted that, even for perfect forecasts, ensembles of model forecasts are not realizations of real climate—climate forecasts are variables in model space not in observation space. Climate model forecasts are generally not perfectly calibrated (although some models may produce well-calibrated raw forecasts) and contain uncorrected forecast errors. Ensemble forecast variances, for example, are likely to either underestimate or overestimate posterior uncertainties. In summary, ensemble spread does not generally explain all the forecast uncertainty and ensemble relative frequency does not perfectly estimate the probability of climate.

The Bayesian method has three main steps: (i) choice of the prior distribution, (ii) modeling of the likelihood function, and (iii) determination of the posterior distribution. For simplicity, it has been assumed in this study of Niño-3.4 that both prior and likelihood distributions are normal (Gaussian). The Niño-3.4 index has already been demonstrated to be well approximated by the normal distribution (e.g., Burgers and Stephenson 1999; Hannachi et al. 2003).

b. Modeling of the likelihood function

Figure 6 shows a scatterplot of raw coupled model ensemble forecasts versus the observed December Niño-3.4 index for the period 1987–99. Ensemble means are depicted using large open circles. The dashed line is what one expects for perfect forecasts in which the forecast values are identical to the observed values. The likelihood p(X_t | θ_t) is modeled by performing a weighted linear regression between the ensemble mean forecasts (X_t) and matching observations (θ_t):

X_tθ_tNαβθ_tγV_t(8)

where α and β are the intercept and slope parameters, respectively. Regression weights are given by w_t = V⁻¹_t, where V_t is the sample variance of the ensemble mean estimated from V = s²_X/m, where m is the number of ensemble forecasts (m = 9 for our forecasting example). Forecasts with larger ensemble spread have more uncertain ensemble means and so must be given less weight in the regression.

For independent ensemble forecasts the variance of the ensemble mean forecast in the likelihood model would be given by V_t (see Clarke and Cooke 1992, their section 10.3). However, if the ensemble members are not independent, the variance differs from V_t. A simple way to ensure consistency is to allow scaling of the ensemble variance V_t by a factor γ in Eq. (8). Ideally γ should be equal to one but in practice here γ is larger that one. In the case of a perfect model, but not independent ensemble members, γ can be interpreted as m/m′, where m is the number of ensemble members and m′ is the effective number of independent forecasts. The dependency factor γ is obtained as a weighted mean of the square regression residuals:

View Expanded

where n is the length of the time series and w_t = V⁻¹_t. Since the expectation of the ensemble mean is modeled by linear regression (α + βθ_t), it follows that the estimated γ will encompass the errors in this linear assumption.

The solid line in Fig. 6 is the best-fit linear weighted regression between raw ensemble mean values X_t and observations θ_t, corresponding to estimates for the whole period of α̂ = 6.24°C, β̂ = 0.75, and γ̂ = 7.05. It can be clearly seen that the raw coupled model ensemble forecast is biased. These values and Fig. 6 indicate that (i) the variance in Niño-3.4 explained by the coupled model is underestimated [i.e., Var(X_t) < Var(θ̂_t) because β < 1]; (ii) the coupled model generally underestimates the mean SST in the Niño-3.4 region [solid line generally below dashed line in Fig. (6)]; and (iii) either there are not enough independent ensemble members (m′ = m/γ̂ = 1.3) or the error in the coupled model ensemble forecasts cannot be removed by a linear regression.

To avoid introducing artificial skill, both prior and likelihood distribution parameters are estimated using cross-validation by leaving out the year being forecast. The mean cross-validated likelihood estimated parameters are α̂ = 6.27(1.44) [°C]; β̂ = 0.75(0.05); and γ̂ = 7.05(0.18), where the values in parentheses are the mean of the standard errors obtained for each of the cross-validated estimates.

c. Determination of the posterior distribution

From Bayes' theorem [Eq. (6)] it can be shown that for a normal prior distribution θ ∼ N(μ_ot, σ²_ot) and normal likelihood X_t | θ_t ∼ N(α + βθ_t, γV_t), the posterior distribution is also normal (Lee 1997). The resulting normal posterior distribution is given by

θ_tX_tNμ_tσ²_t(10)

with the mean μ_t and the variance σ²_t equal to

View Expanded

A derivation of Eqs. (11) and (12) is presented in appendix B. The inverse of the variance is known in statistics as the precision. Equation (11) states that the precision of the posterior distribution (1/σ²_t) is exactly equal to the precision of the prior distribution (1/σ²_ot) plus the precision of the ensemble system (β²/γV_t). Perfectly accurate unbiased forecasts would have precision 1/V_t. However, forecasts are not perfectly accurate and unbiased and so the precision is instead given by the term β²/γV_t.

Equation (12) gives the posterior combined mean (μ_t) as the precision weighted sum of the prior empirical mean (μ_ot) and the raw coupled model ensemble mean (X_t). Note that the precision of the prior distribution and the precision of the ensemble system are weights for the prior mean and raw ensemble mean, respectively. The mean bias of the ensemble system is corrected when the difference between X_t and α is divided by the rescaling factor β (term in brackets). Note, however, that the role of the prior diminishes with the increase of the sample size m so that the posterior distribution is increasingly dominated by the likelihood and not very much affected by the prior.

4. Results

Figure 7a shows the mean of the combined forecast (thick line), observations (thin line), the 95% P.I. (gray shaded surrounded by long-dashed line) and the December climatological mean of 26.5°C (short-dashed line). Comparison of this forecast with the empirical forecast alone (Fig. 3a) and raw coupled model ensemble forecast alone (Fig. 4a) shows that the combined forecasts are in closer agreement with the observations. The 95% P.I.s are also reduced compared to those of the empirical forecasts indicating a reduction in forecast uncertainty due to combination with raw coupled model forecasts. Unlike the raw coupled model forecasts, only one forecast year (1994) falls outside the 95% P.I., indicating that the forecasts are better calibrated than the raw coupled model forecasts. However, it is worth mentioning that a similar effect could be obtained by crudely removing the mean bias from the raw coupled model forecasts and rescaling the averaged ensemble spread to match the error variance.

Figure 7b shows the combined forecast standardized errors. The smallest errors were found within the period 1987–93 and in 1995 and 1998. The largest errors were in 1994, 1996, 1997, and 1999. It can be seen that these errors are evenly distributed and centered on zero.

Figure 8 shows plots of the standardized forecast error versus forecast values for the three types of forecasts presented so far. Figure 8b shows that the raw coupled model ensemble forecast is negatively biased. The standardized errors for the empirical forecast (Fig. 8a) and for the combined forecast (Fig. 8c) are evenly spread around the zero line. Note also that the combined forecast does not show dependency on forecast values. However, this is not the case for the raw coupled model ensemble forecast, in which larger forecast values are associated with larger standardized forecast errors.

Table 1 gives some deterministic verification scores and a measure of forecast uncertainty of seven different forecasts of the December Niño-3.4 index for the period 1987–99. All the forecasts were produced using the cross-validation leave one out method and Table 1 summarizes the skill of these forecasts in the short 13-yr sample period.

• The climatological forecast is given by the historical Niño-3.4 index December mean value (θ) of 26.5°C and the historical December standard deviation (s_θ) of 1.19°C.

• The empirical forecast is given by μ̂_ot and σ̂_ot, as defined in section 2a.

• The raw coupled model ensemble forecast is given by X_t and s_X, as defined in section 2b.

• The bias-corrected forecast is given by X^′_t = X_t − X + θ and s_X, where X_t is the raw ensemble mean forecast at time t, and X and θ are the time means of the raw ensemble mean forecast and the observed mean values over the forecast period 1987–99, respectively. This is a special case of a Bayesian forecast with uniform prior (defined below) and simplified likelihood [β = 1 and γ = m in Eq. (8)]. Simplified likelihood models the ensemble mean bias as a constant (α) and the sample variance of the ensemble forecast as mV_t = s²_X.

• The combined forecast with uniform prior is given by (X_t − α)/β and γV_t/β². It is obtained by setting σ⁻²_ot to zero in Eqs. (11) and (12), that is, all values of the index are equally likely. This prior characterizes a “no-previous-information” reference case. The combined forecast with uniform prior can be seen as a Bayesian bias correction in the raw ensemble mean and it is useful for comparison with the bias-corrected forecast.

• The combined forecast with climatological prior is given by μ_oc and σ_oc. It is obtained when the December normal climatological distribution [i.e., N(θ, s²_θ)] is used as the prior distribution.

• The combined forecast is given by μ̂_t and σ̂_t, as defined in section 3b.

The MSE and mean absolute error (MAE) have been used as verification scores for the forecast means. The MAE skill score given by SS = 1 − (MAE/MAE_c), where MAE_c is the climatological MAE, was used to measure forecast skill. The reason for using this score instead of the MSE skill score is because the MAE skill score provides a more resistant measure for small samples (Jolliffe and Stephenson 2003). Forecast uncertainty was summarized by the time mean of the predicted forecast standard deviations over the forecast period 1987–99.

The climatological forecast is the most uncertain and imprecise forecast with the largest MSE and MAE errors and the largest prediction uncertainty (Table 1). The raw coupled model ensemble forecast has (coincidentally) the same MSE as the empirical forecast, and a slightly larger MAE than the empirical forecast. Note that although these two models have similar MSE and MAE their uncertainty estimates are quite different. The width of the 95% P.I. in Fig. 4a, which is proportional to the mean uncertainty shown in Table 1, shows that the coupled model uncertainty is unrealistically underestimated and fails to cover the range of observations. The bias-corrected coupled model forecast has smaller MSE and MAE than the empirical forecast, and a greater skill score than the raw coupled model ensemble forecast. The uniform prior forecast has smaller MSE and MAE than the bias-corrected forecast, a slightly better skill score than the bias-corrected forecast and a much greater skill score than the raw coupled model ensemble forecast. The uniform prior has also smaller errors than the empirical forecast, and a greater skill score than the empirical forecast. These results suggest that the use of prior information helps to improve forecast skill. It also has a larger forecast uncertainty that is between the uncertainty of the raw coupled model ensemble forecast and the empirical forecast. The combined forecast with climatological prior has slightly smaller MSE and MAE than the combined forecast with uniform prior and greater skill scores than the bias-corrected forecast and the raw coupled model ensemble forecast, indicating that the use of climatological prior information helped to improve even more forecast skill. The combined forecast with climatological prior also has smaller errors than the empirical forecast, and a greater skill score than the empirical forecast. It also has greater forecast uncertainty, which is only slightly smaller than the uniform prior forecast uncertainty. The combined forecast has the smallest values of MSE and MAE of all the forecasts. It also shows an impressive improvement of 23% in skill when compared to the raw coupled model forecasts, indicating that the use of a more informative prior led to additional improvement in forecast skill. Additionally, it provides a much better and more realistic uncertainty estimate compared to the other forecasts.

Table 2 summarizes the standardized forecast errors. The mean standardized forecast error shows that the raw coupled model forecast is negatively biased, with the largest mean error of all the forecasts. The climatological forecast, the combined forecast with uniform prior, and the combined forecast with climatological prior have the smallest mean errors, indicating that these forecasts are well calibrated. The raw coupled model ensemble and the bias-corrected ensemble forecasts have the largest and most unrealistic variances of the standardized forecast errors. All forecasts have variances larger than one suggesting that the prediction uncertainty of the forecasts is being underestimated.

Because these scores are based on only a small sample of forecasts, one might worry that the benefits of using the Bayesian approach are due to chance sampling. However, similar conclusions as here were obtained when the same methodology was applied to three other versions of the ECMWF seasonal forecasting system, one of which had a much longer record of 44 yr (Coelho et al., 2003). Additional analyses of the robustness of the obtained results have been performed by splitting the 44-yr record into 3 samples of 13 forecasts each. It has been found that Bayesian combined forecasts generally provide better and more reliable forecasts than raw coupled model and empirical forecasts.

5. Conclusions

A Bayesian approach for calibrating and combining empirical and raw coupled model ensemble forecasts has been presented. The combined 5-month lead forecast of the Niño-3.4 index has been shown to have greater forecast skill than either of the forecasts individually. This indicates that both empirical and raw coupled model ensemble forecasts contain mutually useful information. In other words, neither forecast is sufficient for the other forecast and so increased forecast skill can be obtained by combining both types of forecast. In order to produce improved interval forecasts of the Niño-3.4 index, empirical and coupled model forecasts should be combined together. The combined forecast also provides a more reliable prediction error estimate because it is based on a well-founded calibration approach that incorporates valuable historical information.

Good quality forecasts are expected to have both small prediction errors (good accuracy) and reliable forecast uncertainty estimates. It has been shown that, although the ECMWF raw coupled model ensemble forecast is able to simulate the interannual variability of the Niño-3.4 index reasonably well 5 months in advance, it underestimates both the mean SST value in the Niño-3.4 region and forecast uncertainty. The simple empirical model, on the other hand, provides more skillful forecasts compared to the raw coupled model ensemble forecast. These forecasts are less biased and present larger and more reliable uncertainty estimates. When the Bayesian approach was used to combine these two forecasts together, more skillful forecasts were obtained having more accuracy and reliability.

It is important to stress that both the prior and the likelihood model used in this study are simple. More sophisticated regression models could easily produce greater improvements in forecast skill, yet this is not the ultimate aim of this pilot study. It should be noted that some of the forecast errors/uncertainty derive from the modeling assumption used here (e.g., normal–normal model). Our approach does not fully incorporate uncertainty in the likelihood model parameter estimates that could be treated using a hierarchical Bayesian approach (see Berliner et al. 2000b). This methodology also needs to be developed in order to combine ensemble forecasts from different coupled models (multimodel approach).