This study assesses the skill of multimodel forecasts of 10-m wind speed, significant wave height, and mean wave period in the North Atlantic for the winter months. The 10-m winds from four North American multimodel ensemble models and three European Multimodel Seasonal-to-Interannual Prediction project (EUROSIP) models are used to force WAVEWATCH III experiments. Ten ensembles are used for each model. All three variables can be predicted using December initial conditions. The spatial maps of rank probability skill score are explained by the impact of the North Atlantic Oscillation (NAO) on the large-scale wind–wave relationship. Two winter case studies are investigated to understand the relationship between large-scale environmental conditions such as sea surface temperature, geopotential height at 500 hPa, and zonal wind at 200 hPa to the NAO and the wind–wave climate. The very strong negative NAO in 2008/09 was not well forecast by any of the ensembles while most models correctly predicted the sign of the event. This led to a poor forecast of the surface wind and waves. A Monte Carlo model combination analysis is applied to understand how many models are needed for a skillful multimodel forecast. While the grand multimodel ensemble provides robust skill, in some cases skill improves once some models are not included.
Accurate seasonal forecasts of wind and waves in the North Atlantic will provide benefits to coastal land management (Tsimplis et al. 2005), marine vessel routing (Coll et al. 2013; Heij and Knapp 2015; Bell and Kirtman 2018), renewable energy (Clark et al. 2017; Torralba et al. 2017), and oil and gas activities (Colman et al. 2011).
The variability of the winter North Atlantic wind–wave climate is dominated by a small number of modes. Previous studies have shown the North Atlantic Oscillation (NAO) to have the dominant effect on wind–wave variability and, to a lesser degree, the east Atlantic pattern (Bacon and Carter 1991; Wolf and Woolf 2006; Dodet et al. 2010; Stopa and Cheung 2014; Martínez-Asensio et al. 2016; Castelle et al. 2017; Zubiate et al. 2017). The NAO is the term given to the north–south oscillation of mass that results in variability of the latitude of the North Atlantic midlatitude storm track, which is most prominent in winter (Stephenson et al. 2003). It is traditionally measured using mean sea level pressure (MSLP) fluctuations between Iceland and the Azores. During the NAO’s positive (negative) phase the storm track takes more of a northerly (southerly) track.
It is important to understand the dynamics of the NAO to understand sources of predictability. The self-maintaining nature of the North Atlantic storm track is discussed in Hoskins and Valdes (1990) and Ambaum and Novak (2014). However, we are more interested in how the storm track behaves overall during the winter months. Brayshaw et al. (2009) noted that midlatitude orography and the land–sea temperature contrast on the east coast of North America play a role in determining the southwest–northeast tilt of the upper-tropospheric jet. In addition, sea surface temperature (SST) gradients in the North Atlantic act to perturb the jet stream. Gradients in the western North Atlantic, such as the Gulf Stream, strengthen the jet downstream, while gradients in the North Atlantic Current weaken the jet (Brayshaw et al. 2009; O’Reilly et al. 2017b).
In addition, the NAO is influenced by remote SSTs, such as those in the tropical Pacific. The mechanisms by which ENSO influences the NAO remain unclear and are a topic of debate. Some studies look at the stratospheric pathway, where during warm events (El Niños) the polar vortex weakens and the frequency of sudden stratospheric warming events increases (Butler et al. 2017). These events descend from the stratosphere, weakening the upper-level westerlies and the resultant NAO is negative (Brönnimann et al. 2007; Bell et al. 2009; Hansen et al. 2017). Kim et al. (2012) found the seasonal forecast model CFSv2 simulates this El Niño–negative NAO response better than the seasonal forecast model ECMWF System 4. Others argue for a tropospheric pathway via the role of quasi-stationary waves kicked off from diabatic heating in the tropical Pacific, which are enhanced by eddy–mean flow interactions in the extratropics (Graf and Zanchettin 2012; Jiménez-Esteve and Domeisen 2018). Butler et al. (2014) separate these two pathways and find the stratospheric pathway mostly influences the North Atlantic, whereas the tropospheric pathway mostly influences North America. It is also possible El Niño–Southern Oscillation (ENSO) SSTs can modulate North Atlantic north tropical SSTs (Alexander et al. 2002), which then in turn affect the NAO. Zhang et al. (2018) found that the ENSO–NAO relationship is highly nonlinear and can depend on the type of ENSO event. Finally, SSTs in the South Atlantic are also thought to play a role in the NAO via a barotropic response to upper-level convergence over the Caribbean (Robertson et al. 2000).
Despite seasonal forecasts often struggling in the extratropics due to a low signal-to-noise ratio (Eade et al. 2014), recent studies have found the NAO to be predictable on seasonal time scales (Scaife et al. 2014; Siegert et al. 2016; Yang et al. 2015; Dunstone et al. 2016; Jha et al. 2016; Athanasiadis et al. 2017; Wang et al. 2017). This is in part due to the drivers of seasonal NAO variability being predictable on seasonal time scales. For example, tropical Pacific SSTs are often at the crux of a skillful seasonal forecast (Shukla 1998) and its impact on the NAO is captured in the Global Seasonal Forecast System version 5 (GloSea5) model [see Fig. 2 in Scaife et al. (2014)], which has good skill at forecasting the NAO. Atlantic SST has also shown to be predictable on seasonal time scales (Rodwell and Folland 2002), such as in the North Atlantic subpolar gyre.
While it is important to understand and predict the physical mechanisms forcing the NAO (Scaife et al. 2016), the role of ensembles and multimodel forecasts increases the skill of forecasting the NAO. Figure 3 in Scaife et al. (2014) shows that increasing the number of ensembles increases the correlation of the ensemble-mean forecast with the observed NAO. This skill limit is determined by the quotient of how well the ensemble members correlate with the observations over the correlation between pairs of ensembles members. Sardeshmukh et al. (2000) and Eade et al. (2014) also note how reliable predictions may be achieved using a large ensemble size to reduce noise. Multimodel forecasts have shown their benefit over a single model forecast in recent years (Palmer et al. 2004; Kirtman et al. 2014), especially in their superiority of probabilistic verification metrics. The skill increase in multimodel forecasts comes from error cancellation of model biases. Doblas-Reyes et al. (2003) found a multimodel forecast of the NAO was superior to individual models using European models from the DEMETER project (Palmer et al. 2004); however, NAO skill is marginal in these old-generation models. Athanasiadis et al. (2017) investigated the NAO and Arctic Oscillation (AO) using CFSv2, GloSea5, and CMCC seasonal forecasts. They found an increase in the correlation coefficient with the observed NAO in part coming from the increased ensemble size. Ehsan et al. (2017) found good skill in predicting winter precipitation in two subtropical regions using NMME and the ECMWF System 4 models.
In this study we make use of a grand multimodel ensemble (GMME) to investigate the skill of seasonal forecasts of wind and waves in the North Atlantic. The GMME is a comprehensive multimodel study using models in the North American Multimodel Ensemble (NMME) and models in the European Multimodel Seasonal-to-Interannual Prediction project (EUROSIP). EUROSIP is designed to provide multimodel forecasts from independent coupled seasonal forecasting systems (see https://www.ecmwf.int/en/forecasts/documentation-and-support/long-range/seasonal-forecast-documentation/eurosip-user-guide/multi-model for more information). While, previous studies have focused on seasonal forecasting of surface winds, to our knowledge, this is the first study looking at seasonal ocean wave forecasts using a multimodel ensemble.
This paper is structured as follows. The datasets, model experiments, and skill metrics are described in section 2. The spatial skill levels of the forecasts are shown in section 3. Section 4 shows the relationship between wind and waves and MSLP as well as quantifying how well the models forecast year-to-year variability. Section 5 shows the relationship of the NAO to large-scale environmental conditions, and section 6 discusses its impact on local surface winds and waves. Examples of where the multimodel ensemble performed well and poorly are given in section 7. Section 8 shows forecast error growth with increasing lead time, and section 9 quantifies the skill improvement of the multimodel forecasts. Finally, the discussion is given in section 10, and the conclusions are presented in section 11.
a. Seasonal forecast models
The seasonal forecast models used in this study are shown in Table 1. It is worth noting that CFSv2 and GloSea5 are lagged ensemble starts, whereas all other models initialize on the first of every month. Both CFSv2 and GloSea5 have high ocean resolutions: 0.5° and 0.25°, respectively. Whereas the other models have ocean resolutions of approximately 1°. A higher ocean resolution is advantageous as it provides more robust sea surface temperature gradients, which are important for accurately simulating synoptic atmospheric variability (Kirtman et al. 2012; Parfitt et al. 2017). The models with a resolved stratosphere are GloSea5, System 4, and System 5. Ten ensembles are used for each model throughout this study to ensure a fair comparison and to remove any bias caused by the lagged starts of CFSv2 and GloSea5. All datasets have been interpolated to 1°. The data are averaged over the winter months of December–February (DJF) for analysis after forcing the wave model with high-frequency 6-hourly wind data. The time period of interest is from winter 1992/93 to winter 2009/10. While there are issues with using a small sample size (Siegert et al. 2016; O’Reilly et al. 2017a), it is the only time period with overlapping data for all seven models. The environmental variables of SST, MSLP, geopotential height at 500 hPa (), and zonal wind at 200 hPa () are investigated in relation to the NAO. Unfortunately, data for sea ice, stratosphere diagnostics, and ocean heat content diagnostics were not available for all seven models and therefore are not included in this study.
b. Ocean wave model
The 10-m winds from the seasonal forecast models are used to force WaveWatchIII experiments. The 4.18 version (Tolman et al. 2014) is used with Ardhuin et al. (2010) wind input and dissipation source terms. The experiments are integrated over a large Atlantic domain (100°W–30°E, 70°S–80°N) to ensure swell waves are captured. ETOPO1 (Amante and Eakins 2009) is used for the bathymetry, and islands smaller than 1° are included at obstruction points defined by Chawla and Tolman (2008). In this study the wave variables of significant wave height ( = 4), where E is the total variance given as the sum over all frequency and direction bins: , and mean wave period ( = = 2), are investigated.
c. Reanalysis data
Model forecast results are compared to reanalysis data, which are used as close to the observed values and have the advantage of spatial homogeneity. The European Centre for Medium-Range Weather Forecasts interim reanalysis (ERA-Interim; Dee et al. 2011) is used as previous studies have shown, compared to other reanalyses, that ERA-Interim has the smallest bias for 10-m winds (Stopa and Cheung 2014). The horizontal resolution of ERA-Interim is ~80 km; however, the data are interpolated to 1°. Even though ERA-Interim is coupled to a wave model, it uses a different wave model than that used in this study (Wave Assimilation Model; Bidlot et al. 2007). Therefore, the ERA-Interim winds are used as input to the WaveWatchIII model and compared to the forecasts. We have opted not to bias correct the winds, such as is done in Durrant et al. (2013, 2014), because of the complexity of a multimodel approach. Using the unbiased winds from all models and ERA-Interim allows for a fair comparison. The surface winds provide mechanic energy into the ocean, which generates wind waves (Simmonds and Keay 2002). The wind waves develop into swell waves once they propagate away from the energy source.
d. Forecast skill assessment
The quality of the forecasts is assessed using the probabilistic verification metric of rank probability skill score (RPSS). The RPSS measures the probabilistic skill relative to a climatological forecast and a value greater than zero is said to be better than using climatology as the predictor (Weigel et al. 2007). The systematic error is removed from the models by subtracting the observed climatology from the predictor climatology before the predictions and observations are split into terciles. This is discussed fully in Bell and Kirtman (2018).
3. Spatial probabilistic skill assessment
a. 10-m wind speed
The ability of the models to predict the DJF 10-m wind speed using December initial conditions is shown in Fig. 1. RPSS values greater than zero, in red, show where the forecast models have skill. Figs. 1a–g show the skill for the individual models in alphabetical order as they are presented in Table 1. The models have varying levels of skill, and the patterns are slightly different for each model. The west subtropical region around 30°N is the most predictable region as all models tend to be able to forecast this region. Skill becomes noisier in the extratropics; however, some models such as CCSM4, CFSv2, and GloSea5 are skillful around the British Isles. CFSv2 and GloSea5 have high levels of skill across the entire North Atlantic, as shown by the average RPSS value given at the top right in each panel. RPSS is noisier in the other models, and the spatial average RPSS value is close to zero. Figure 1h shows the RPSS for the GMME. The spatial pattern is smooth, and the spatially averaged RPSS value is greater than any of the individual models.
b. Significant wave height
The RPSS of forecast (Fig. 2) is mostly similar to the RPSS of the 10-m wind speed (Fig. 1), but there are spatial differences. Skill in the 10-m wind speed is related to skill in downstream of the winds such as in the southwest and northeast regions. This is due to wave growth along the direction of the winds and is observed in other basins (e.g., Bell and Kirtman 2018). Skill can be observed in three distinct regions: the western tropical North Atlantic, the subtropical North Atlantic, and the northeast North Atlantic. CFSv2 and GloSea5 have the highest levels of forecast skill compared to other models. The GMME has a similar level of skill to GloSea5 when the RPSS is averaged over all grid points in the spatial domain.
c. Mean wave period
Figure 3 shows the RPSS of the forecast mean wave period . Not only is related to the 10-m wind speed but to the wind direction and fetch length as well. The three regions of predictability have a southwest–northeast tilt, which reflects the mean climate. CFSv2 and GloSea5 are the most skillful models across the entire basin but System 5 has large regional RPSS values in the middle of and to the northeast of the basin. The regional skill in forecasting 10-m wind speed in CCSM4 does not translate into a skillful forecast, most likely because of limited skill in the wind–wave generation regions (the west coast of Africa and the U.S. East Coast). The winds in these regions are important as errors in wind direction and fetch length here will lead to errors in downstream. Similar to Fig. 2, the GMME forecast of is as skillful as the best model, GloSea5.
d. Signal-to-noise ratio
The RPSS results should be looked at in context with the signal-to-noise ratio (StN ratio; e.g., Kumar 2009) to provide an understanding of how confident a forecast is. We use the methodology of Athanasiadis et al. (2017) and define signal as the standard deviation of the ensemble means and noise as the standard deviation of the ensemble members. This calculation yields a range between zero and one with zero indicating regions that have large noise and therefore large ensemble spread. We show the StN ratio for 10-m wind speed (Fig. 4a), (Fig. 4b), and (Fig. 4c) from the GMME. It is interesting to note the regions of low StN ratio tend to occur downstream of the large-scale circulation, such as to the east of the Caribbean and to the west of western Europe. Therefore, forecast skill in the center of the basin, where the StN ratio is large can be viewed with a high level of confidence.
4. Temporal deterministic skill assessment
The spatial pattern of variability of winter wind and waves in the North Atlantic is largely explained by the first empirical orthogonal function (EOF1) in the spatial domain 100°W–30°E, 0°–80°N. We show the spatial loading pattern of EOF1 for 10-wind speed (Fig. 5a), (Fig. 5b), (Fig. 5c), and MSLP (Fig. 5d); in addition, the variances explained by EOF1 are 31%, 44%, 55%, and 59%, respectively. The spatial patterns look markedly similar for all panels. For MSLP and there is a dipole in the northeast North Atlantic that resembles that of the NAO. From hereon we refer to MSLP EOF1 and the NAO. For 10-wind speed the pattern is more tripolar, as there is some variability in the tropics. Finally, EOF1 of shows a northeast–southwest tripole, along the direction of the 10-m wind. There is a strong case to be made here that a skillful forecast of surface wind and waves largely depends on a skillful forecast of the NAO due to similar patterns of variability. For the temporal analysis we use the first principal component (PC1) related to EOF1 calculated using the methodology of Dawson (2016). The advantage of using a PC instead of a point-based analysis is that it will take into account structural differences between the models.
a. 10-m wind speed time series
The red lines in Figs. 6a–g are ensemble-mean forecast PC1s for the individual models using December initial conditions. Additional information about the model’s PC1 is given in the top-left corner of the panels. Figure 6h is the GMME PC1 forecast. The correlation coefficient, p value of the correlation coefficient, and mean absolute error are given in the top-right corner of all panels. The absolute error for each year is given below the panel. The PC1 is well predicted in CFSv2, GloSea5, and System 5, with correlation coefficient values of 0.72, 0.77, and 0.7, respectively. The GMME forecast has a correlation coefficient of 0.71 and is similar to the superior models. The other models, excluding System 4, have correlation values between 0.44 and 0.53. System 4 has a low correlation coefficient value and is largely a results of two years when it failed to capture the sign of the forecast: 1998 and 2009.
b. Significant wave-height time series
The results of year-to-year forecasting of (Fig. 7) are similar to that of 10-m wind speed but with slightly lower skill. For example, the best models—CFSv2, GloSea5, and System 5—have correlation coefficient values between 0.51 and 0.76. This is not surprising as the observed PC1 results are very similar for and 10-m wind speed, such as the large negative event in 2009. For the GMME (Fig. 7h) the forecasts are generally good. However, the 1997 forecast predicted a negative pattern of EOF1, whereas a neutral pattern was observed.
c. Mean wave period time series
Most of the models that did not accurately forecast the time series of 10-m wind speed (and therefore ) did not forecast the time series of (Figs. 8a–c,f). The 10-m wind speed alone does not explain the results of the variability. It is the large-scale wind climate that controls the wind direction and fetch length. Therefore, local and remote regions require accurate wind forecasts. The models that do accurately forecast the time series of 10-m wind speed also capture the variability of to a strong degree, such as GloSea5, which has a correlation coefficient of 0.84. For the models that have poor forecasts for the 10-m wind speed the resultant is worse. In these cases (CanCM3, CCSM4, and System 4) there is error propagation arising from differences in wind speed, wind direction, and fetch length.
d. NAO time series
The NAO is well predicted in CFSv2 and GloSea5, with correlation coefficient values of 0.74 and 0.64, respectively. CCSM4 is shown to predict the NAO well with a correlation coefficient value of 0.64; however, wave forecast skill in CCSM4 is limited to the northeast North Atlantic, where the NAO is most dominant. The GMME forecast has a correlation coefficient of 0.69 and is only behind CFSv2. The results are similar to those of the wind and wave variables discussed above. This demonstrates there is a strong relationship between the NAO, surface winds, and surface waves.
Table 2 shows the ranking of all eight models in terms of yearly absolute error. The last column is the sum of the ranks. Table 2 highlights the importance of using multimodel forecasts, as the “best” model varies from year to year. CFSv2 has the best ranking out of all the models, followed by GMME, GloSea5, and CCSM4. System 4 and System 5 have similar rankings, followed by CanCM3 and CanCM4. The top row in Table 2 gives the sum of the models’ absolute error and defines years that are more predictable than others. This is discussed further in section 7.
5. NAO and large-scale environmental conditions
The NAO is influenced by a combination of teleconnections from large-scale environmental conditions as well as internal processes. The results in this section investigate the relationship between large-scale environmental conditions and the NAO. The results ensure that the forecasts are predicting the NAO for the right reasons. Variables considered include SST, , , and MSLP.
a. Sea surface temperature
The observed correlation coefficient of MSLP PC1 (the black line in Fig. 9) with SST is presented in Fig. 10a. Significant points at the 99th percentile value are shown via stippling. There is a strong dipole in the North Atlantic, as discussed in Brayshaw et al. (2009), that has a large influence on the NAO. There are however dipoles outside of the North Atlantic that are correlated with the NAO. Examples include the regions in the subtropical North Pacific and the South Atlantic. Figure 10b shows the same relationship using the GMME forecast and Fig. 10c is GMME minus the observations. The GMME captures the large-scale spatial relationship between SST and the NAO; however, there are some regional differences. For example, the GMME has a larger correlation with eastern tropical Pacific SSTs, whereas this relationship is more noisy in ERA-Interim. The GMME forecast correlation pattern in the North Atlantic is slightly weaker than observed, and few points are statistically significant.
b. Geopotential height at 500 hPa
The observed correlation of MSLP PC1 with is presented in Fig. 11a. There is a clear NAO pattern in the North Atlantic and to a lesser degree a signal in the North Pacific. Points in the Southern Ocean that are significant are likely to be related to the southern annular mode, which shows variability over a wide range of time scales (Simmonds and King 2004). The GMME forecast correlation of MSLP PC1 with is very good in the North Atlantic (Fig. 11b). Outside of the North Atlantic, the relationship is stronger in GMME than observed (Fig. 11c), as the GMME forecasts a broad hemispheric oscillation in both hemispheres.
c. Zonal wind at 200 hPa
The –NAO correlation coefficient is a noisier field than the previous environmental variables (Fig. 12). The observed pattern in the North Atlantic shows the largest signal in the NAO region but there are regions of alternating correlation coefficient signs that extend south throughout the Atlantic and east to central Europe (Fig. 12a). There is also a tripolar pattern in the North Pacific. The GMME forecasts of the –NAO correlation coefficient are much smoother than observed and the banding is too zonal (Fig. 12b). The pattern in the North Pacific is also too strong in the GMME forecasts.
d. Mean sea level pressure
The observed correlation coefficient of DJF MSLP with North Atlantic MSLP PC1 (Fig. 13a) shows an NAO pattern that is expected, given the construction of MSLP PC1. There is also a strong relationship with mean sea level pressure in the central to northeast Pacific. The region with the largest correlation coefficient is shifted farther north in the GMME forecast compared to the observations (Fig. 13c). In addition, the negative correlation coefficient in the western Pacific is much stronger in the GMME forecast. However, the differences are small in the regions with known physical relationships to the NAO: the North Atlantic and tropical eastern Pacific.
6. NAO and winds and waves in the North Atlantic
The impact of the NAO on the winter wind and wave climate in the North Atlantic is presented below for 10-m wind speed, , and .
a. 10-m wind speed
The NAO’s impact on 10-m wind speed is a tripolar pattern (Fig. 14a), as discussed in previous studies (e.g., Fig. 5a in Zubiate et al. 2017). It is encouraging that the GMME forecast captures the large-scale tripolar pattern. Some regional differences exist such as the GMME forecast showing a correlation of the opposite sign in the Gulf of Mexico compared to the observations. The GMME in addition has a much stronger correlation coefficient in the extratropics (Fig. 14c). It is worth noting that the spatial pattern of the NAO–10-m wind speed correlation is strongly related to the RPSS spatial pattern (Fig. 1 c), highlighting the importance of the NAO in predicting winter 10-m wind speed.
b. Significant wave height
The NAO influences in a northeast–southwest pattern that takes into account wave growth along the wind direction (Fig. 15a). The GMME correctly predicts this large-scale pattern (Fig. 15b) but the regional differences, such as stronger correlation in the tropics and extratropics, arise due to the wind forcing (cf. Fig. 15c with Fig. 14c).
c. Mean wave period
The NAO tripolar pattern for has an east–west pattern along the major axis of the jet stream (Fig. 16a). During NAO positive (negative) years the jet stream is stronger (weaker) and farther north (south). This results in longer- (shorter) period waves in the northeast part of the basin. The differences in the GMME forecast and the observations are small in the central part of the basin, the negative correlation area (Fig. 16c). Similar to 10-m wind speed and , the relationship is too strong in the tropics and extratropics.
7. Case studies
The advantage of a multimodel ensemble forecast is explored by looking at individual winter forecasts. Spatial anomalies and probability distributions are investigated for the environmental patterns discussed in section 5. These case studies are selected in terms of the best and worst forecasts using the sum of the models’ absolute error for each year (Table 2). Winter 2002/03 is chosen as a good forecast as all the models predicted the NAO. All model forecasts, apart from CanCM3, had an absolute error of 0.2 or less for this year for the NAO. Winter 2008/09 is chosen as a poor forecast as all models fail to capture the magnitude of the negative NAO event. The model sum absolute error for this winter is 10.4, of which CanCM3 and System 4 made large contributions to.
a. Winter 2002/03: A good multimodel ensemble forecast
The NAO during winter 2002/03 was close to zero. It is therefore surprising that this year was well forecast as models tend to struggle to predict near-normal conditions compared to above- or below-average conditions (van den Dool and Toth 1991). The SST in the North Atlantic was slightly warmer than average (Fig. 17a). The GMME forecast is closer to average in the North Atlantic, in particular, in the region where the correlation coefficient is greater than 0.5 for the MSLP PC1 and SST (the black contour in Fig. 17b and see Fig. 10b). The ensembles are averaged over this region and shown as a probability density function in Fig. 17c. The red circle is the observed value. The GMME forecast as shown in Fig. 17f is good, as most ensembles predict near-normal conditions, although two ensembles forecast large negative anomalies. Weaker in the region with large correlation with the NAO is correctly forecast in the GMME (Fig. 17h). The difference between the GMME forecast and observed values for MSLP is small in the North Atlantic (Fig. 17k). While the large-scale environmental conditions are well forecast for the neutral NAO winter, its impact on the wind and wave climate is not as well captured. Most ensembles predict near-normal conditions for the wind and waves (Figs. 17o,r,u); however, below-average conditions were observed.
b. Winter 2008/09: A poor multimodel ensemble forecast
The NAO during winter 2008/09 is the strongest negative NAO in the forecast period. There were large SST anomalies in the North Atlantic in the region, which is strongly related to the NAO Fig. 18a, which explains the large negative NAO. The GMME did well in predicting this pattern, but it struggled to forecast the magnitude of anomalies, as shown in the probability density function (Fig. 18c). The shows a clear negative NAO pattern (Fig. 18d). The GMME forecasts the Northern Hemisphere response (Fig. 18e), however, it does not capture the magnitude of the blocking event over the Labrador Sea, as well as the magnitude of the low in the central North Atlantic. Similarly, the GMME forecasts of and MSLP are of the right sign but are too weak (Figs. 18g–l). The wind and wave patterns are also too weak in the GMME forecasts although there are a few ensembles that do correctly capture the observed pattern (Figs. 18m–u).
8. Forecast error growth with lead time
Figure 19 shows the RPSS averaged over points in the North Atlantic where the correlation with MSLP PC1 is larger than 0.5 (see black contour in Fig. 17) for different lead-time forecasts. For lead-time zero (December initial conditions) all parameters are skillfully predicted with RPSS values greater than 0.1. The parameter that is most predictable is SST with an RPSS of 0.4, followed by MSLP. For November initial conditions predicting DJF, a lead time of 1 month, the RPSS drops for all parameters but the forecasts remain mostly still more accurate than when using a climatological forecast. The predictability of drops notably in the November initial conditions. Finally, using a lead time of 2 months, the October initial conditions forecasts show negligible skill in the North Atlantic for almost all parameters. The SST forecast skill drops markedly to being much worse than a climatological forecast. However, there is still a small predictable signal in the MSLP.
9. How many models are needed for a skillful multimodel forecast?
We show the RPSSs of all possible model combinations averaged over the region defined in Fig. 19 for 10-m wind speed (Fig. 20a), (Fig. 20b), (Fig. 20c), and MSLP (Fig. 20d). The maximum number of model combinations with seven models is 35 using two- and three-model combinations and the results are shown in the box-and-whisker plot. For all parameters the median skill for two models, denoted by the orange line, is double that of the median skill of the single models. The RPSS for the seven-model combination is marginally less than that of the best model (GloSea5) for 10-m wind speed (0.2 and 0.21). However, removing CanCM3, the only model with RPSS less than zero, gives an RPSS greater than any individual model, 0.215. The best forecast is given by combining CCSM4, CFSv2, and GloSea5. The relationship is similar for and . The best forecast for is obtained by combining CFSv2 and GloSea5, whereas the best forecast for is obtained by combining CanCM4, CFSv2, and GloSea5. The relationship for MSLP is different. The seven-model combination is better than any of the individual models. A combination of CCSM4, CFSv2, GloSea5, and System 5 provides the best forecast.
Seasonal forecasting of DJF winds and waves in the North Atlantic are shown to be skillful using December initial conditions with a combination of four NMME models and three EUROSIP models. The large multimodel ensemble dataset provides a comprehensive assessment of the uncertainty associated with seasonal prediction. There are two standout models in the assessment of individual models—CFSv2 and GloSea5; both of which have a high level of skill at predicting winds and waves in the North Atlantic as well as the year-to-year variability. The correlation coefficient of the ensemble mean forecast with the observed NAO is 0.74 for CFSv2 and 0.64 for GloSea5. Kim et al. (2012) found CFSv2 captures the expected winter atmospheric response to ENSO, moreso in the Pacific, but to a degree in the North Atlantic. In addition, they showed CFSv2 has good predictability skill for SST in the North Atlantic. GloSea5’s ability to forecast the NAO at various time scales is given in Scaife et al. (2014), Athanasiadis et al. (2017), and Clark et al. (2017). Scaife et al. (2014) determine GloSea5’s skillful forecast is due to the high-resolution ocean model (0.25°) as well as a resolved stratosphere. Further modeling studies may break down the role of a high-resolution ocean model versus a well-resolved stratosphere and their importance on the NAO.
The benefit of a multimodel ensemble forecast is shown in the spatial maps of RPSS (Figs. 1–3) and the Monte Carlo model combination analysis (Fig. 20). The results are in line with other studies that found multimodel ensemble forecasts are superior to individual model forecasts (Palmer et al. 2004; Kirtman et al. 2014 and references therein). The results in this case however showed one could be more selective in the model combinations. For example, selecting the three or four best models often gave a slightly improved forecast compared to the GMME. However, the best model combination is dependent on the variable of interest. Using all models available still provides robust skill. It is impossible to know a priori which models will yield the highest skill in combination for the operational forecasting. One approach to address this, which has been explored in O’Connor et al. (2017), is to use machine learning to weight models based on their spatial and temporal biases. In light of this, using all models is still recommended.
Correlation maps of the NAO with large-scale environmental conditions for observations and the seasonal forecasts help us to understand the model processes when predicting the NAO and its impact on surface winds and waves. However, mathematically the calculation merely yields an association, and the results must be interpreted by considering dynamical plausibility. For SST (Fig. 10), the dipole in the North Atlantic is expected although there is still debate regarding the role of the ocean versus atmospheric processes in forcing the NAO (Czaja et al. 2013). The SST dipole in the North Pacific is likely associated with the Arctic Oscillation (AO; Thompson and Wallace 1998). This pattern emerges as the AO and NAO are highly correlated and the physical mechanisms associated with annual modes for explaining the existence of the NAO are not well understood (Ambaum et al. 2001). One could undertake an idealized atmospheric general circulation model experiment with SST forcing in the North Pacific and climatology elsewhere to understand this further. For (Fig. 11), the pattern in the North Atlantic–European sector is well understood for its impact on the geostrophic winds.
The case study of winter 2002/03, when the GMME correctly predicted near-normal NAO conditions, was a central Pacific El Niño year (McPhaden 2004). The winter anomaly shows a wave train emanating from east of Hawaii (see Fig. 17d) into the North Atlantic. However, this did not project onto the NAO pattern and produced more of an east–west dipole in the North Atlantic. An average over the region related to the NAO therefore gives near-normal conditions. The model forecasts did not capture this wave train, and therefore the spatial structure is different. The poor NAO forecast during winter 2008/09 is well studied in Harada et al. (2010) and Ayarzagüena et al. (2011). A major stratospheric warming took place, but the external factors were unfavorable so the event was not well forecast. In this particular case it is likely that the atmosphere influenced the North Atlantic ocean beneath the storm track. It is worth mentioning that the majority of models correctly predicted the sign of the forecast. It was only CanCM3 and System 4 that predicted more near-normal conditions. The poor NAO forecast translated to a poor forecast of surface winds and waves in the North Atlantic with almost all ensembles not capturing the magnitude of the event.
Forecast skill, defined as RPSS of winds and waves in the North Atlantic, decreases to near zero when increasing the lead time to 2 months ahead of DJF. SST forecast skill drops markedly but MSLP remains predictable at a lead time of 2 months despite other local environmental conditions being difficult to forecast. There is potential here to increase the lead time for the NAO forecast and explore regions of predictability as was done in Dunstone et al. (2016).
Wintertime wind and wave variability is largely controlled by the NAO. Therefore, any skill in forecasting the wind and waves in the North Atlantic starts with assessing the forecasting skill of the NAO. The use of a grand multimodel ensemble (GMME) provides a large ensemble size that is advantageous for forecasting regions that areinherently noisy such as the extratropics. The GMME provides skillful forecasts of winds and waves in the North Atlantic during the winter months although it is limited to a 1-month lead time. The most predictable regions are around the British Isles, the Azores, and to the east of the Caribbean. The GMME also predicts the year-to-year winter variability of winds and waves to a high degree. An assessment of the forecasting large-scale environment conditions and their relationship to the NAO shows that the models capture the local response in the North Atlantic but tend to have a larger correlation in the tropical Pacific than expected. The forecast response of the NAO to surface winds and waves is well captured and explains the spatial pattern of the rank probability skill score: regions that have the strongest correlation with the NAO are more predictable. The NAO does not explain the entire pattern though and it is likely other factors are at play. These include other modes of variability such as the eastern Atlantic pattern as well as remote wind forcing controlling the fetch length. However, these are outside the scope of this study and will be investigated in a future manuscript. A look at two winter case studies shows how sea surface temperatures (SSTs) affect the atmospheric variability, which in turn affects ocean winds and waves. The NAO of 2002/03 was well forecast despite it being near normal as near-normal conditions are hard to forecast. The tropical Pacific SSTs were well forecast, but the local SSTs were not. This has the impact of reducing the skill of the winds and waves in the North Atlantic. The NAO forecast during winter 2008/09 was poor despite the models capturing the sign of the event. Error was propagated in the forecast from the poor NAO forecast to a poor surface wind forecast and a poor ocean wave forecast. A multimodel ensemble forecast provides robust skill and is often better than any individual model for probabilistic skill assessments. The optimum forecast can be produced by comparing all possible model combinations, and the best skill was found by combining three or four models, depending on the parameter of interest.
The authors acknowledge support from National Oceanic and Atmospheric Administration Award NA15OAR4320064 and Office of Naval Research Award N6833516C0087. The authors thank ECMWF Copernicus for providing the EUROSIP data, EarthSystemGrid for providing the NMME data, and the University of Miami Center for Computational Science for providing computer resources. We thank three anonymous reviewers for their constructive comments, which helped to improve the manuscript. We thank the developers of the following open-source python packages: Xarray (Hoyer and Hamman 2017), Dask, Cartopy, Matplotlib, and Seaborn.
Current affiliation: Royal Caribbean, Miami, Florida.