Global Extreme Heat Forecasting Using Neural Weather Models

a. Data, predictors, and targets

The targets y are constructed from the daily average of the standardized climatological anomalies of the temperature 2 m above the surface T_2m, which we denote ${\tilde{T}}_{2\mathrm{m}}$. We recognize that air temperature is a suboptimal indicator of heat-related illness (Xu et al. 2016; Heo et al. 2019), but it enables comparison with other models in the literature (Weyn et al. 2021; Wulff and Domeisen 2019; Lin et al. 2022). Each target vector y includes ${\tilde{T}}_{2\mathrm{m}}$ for a set of lead times τ = 1, …, τ_l days and for all tiles of a gnomonic equiangular grid of Earth’s surface (Ronchi et al. 1996). Thus, the target size d is the flattened length of the tensor ${\mathbf{y}}_{\text{cs}}\in {\mathrm{ℝ}}^{{\tau }_l\times f\times h\times w}$, where f = 6 is the number of faces of the cubed sphere and h = 48 and w = 48 are the number of meridional and zonal tiles of each face, respectively. The grid is shown in Fig. 1 for reference; the surface area of each tile is approximately 192² km². To assess the skill of the model from the short to the subseasonal range, we aim to predict temperature anomalies for the next τ_l = 28 days.

View Full Size

Depiction of the gnomonic cubed-sphere grid onto which predictors and targets are projected. The cubed sphere is composed of 6 faces with 48² = 2304 cells each.

Citation: Artificial Intelligence for the Earth Systems 2, 1; 10.1175/AIES-D-22-0035.1

• Download Figure
• Download figure as PowerPoint slide

The inputs x ∈ V ⊂ Ρ^q contain υ daily averaged surface fields on the cubed sphere, such that q is the flattened length of the tensor ${\mathbf{x}}_{\text{cs}}\in {\mathrm{ℝ}}^{\upsilon \times f\times h\times w}$. Here, υ = τ_pυ_t + υ_i is the number of input fields and υ_t and υ_i are the number of time-dependent and independent fields, respectively. For time-dependent fields, daily averages from the last τ_p = 7 days are included. We consider as time-dependent fields the geopotential height and potential vorticity at 300, 500 and 700 hPa, the temperature T_2m, the top net outgoing longwave radiative flux (OLR), and the volume of soil water. To these fields we append as auxiliary time-independent variables the latitude, longitude, topography, and a land–sea mask. We also include the present insolation and date as inputs. The full list of predictors is summarized in Table 1.

Table 1

Predictors used for the temperature anomaly forecasting task. Anomalies are standardized (std) with respect to climatology. Heights are specified as above ground level (a.g.) and below ground level (b.g.).

View Table

The choice of predictors is informed by studies linking extreme heat events to midtropospheric geopotential height (Teng et al. 2013; Mann et al. 2018), soil moisture (Vautard et al. 2007; Benson and Dirmeyer 2021) and large-scale phenomena with characteristic OLR signatures, like the Madden–Julian oscillation (MJO; Jacques-Coper et al. 2015; Maloney et al. 2019) or the boreal summer intraseasonal oscillation (Lin et al. 2022). In addition, we include T_2m to learn about transport processes such as advection; and its standardized anomaly to facilitate improving upon a persistence model. Both T_2m and ${\tilde{T}}_{2\mathrm{m}}$ capture the signature of large-scale oscillations such as El Niño–Southern Oscillation or the North Atlantic Oscillation (Ogi et al. 2003; Wang et al. 2011; Wright et al. 2014).

All data are daily averages from the ERA5 reanalysis product (Hersbach et al. 2020), downloaded at 2° × 2° resolution from the Copernicus Climate Data Store (CDS) and projected onto the cubed sphere following Ullrich and Taylor (2015), Ullrich et al. (2016). The climatology is computed for the time period 1979–2019. All days of the year, not only the summer days, are used to train the model. We do this to learn about physical processes that are season independent, like advection. When framed as a classification task, extreme heat prediction may require undersampling of nonextreme samples during training (Jacques-Dumas et al. 2022). Here, we make use of all available information with no explicit undersampling; class imbalance is dealt with through the use of custom losses as discussed in section 2b. In addition, we perform a sequential split of the data into training (1979–2012), validation (2013–16), and test sets (2017–21). The limited amount of historical data available means that the influence of longer modes of climate variability (e.g., the Pacific decadal oscillation) is unlikely to be robustly captured. Furthermore, a shift in the target distribution from training to testing sets is implicit with this split, due to climate change (White et al. 2022; Chan et al. 2020). This shift in the temperature distribution is, however, representative of situations in which a warning system might be used in practice, since both data-driven and NWP models are calibrated using historical observations.

b. Losses considered

Unless the optimal parameter vector θ^* is able to yield a perfect model [i.e., Ψ(θ^*, x) = y ∀ (x, y)], the optimum will depend on the definition of the loss. This is the case in the extreme heat prediction task, since the chaotic nature of the atmosphere precludes a perfect forecast of trajectories from inexact initial conditions (Lorenz 1969a,b; Slingo and Palmer 2011). For this reason, we can expect models that minimize generic losses to be suboptimal for the extreme prediction task.

In the following, we denote models trained with the loss (2) as HeatNet for (a, b) = (1, 0) and ExtNet for (a, b) = (0.5, 0.5). The model trained with the MSE loss, representative of general neural weather prediction systems (Rasp et al. 2020; Weyn et al. 2020, 2021), is denoted GenNet. We trained HeatNet, ExtNet, and GenNet models using a hyperparameter search over the learning rate, and the magnitude of L1 and L2 norm regularization. All models were trained until they started overfitting to the training set, evidenced by an increase in validation loss persistent over many epochs. Notably, our best HeatNet and ExtNet models were obtained through transfer learning, by retraining our best GenNet model for a few (<3) epochs on the custom exponential loss. This implies that any performance improvements of HeatNet or ExtNet with respect to generic models trained on the MSE loss can be realized efficiently through transfer learning from the original models. Our transfer learning methodology relies on early stopping to retain an inductive bias toward the GenNet parameters (Yosinski et al. 2014; Li et al. 2018). The implementation of other transfer learning techniques, like Bayesian regularization toward the original (i.e., prior) model parameters (Li et al. 2018; Inubushi and Goto 2020), may result in further skill improvements and will be explored in the future.

a. Receptive field

Because of the nonrecurrent nature of the architecture and the lead times considered, it is crucial to achieve a fully receptive field if we want to capture long-range dependencies and teleconnections (Espeholt et al. 2022). A fully receptive field is realized through two design characteristics of the proposed architecture, which is sketched in Fig. 2. The first one is the use of dilated convolutions, which rapidly increase the receptive field of any location on the cubed sphere as information traverses the network (Yu and Koltun 2016). The second one is the use of a UNet-type architecture (Ronneberger et al. 2015) with 3 resolution levels going from the data resolution to the synoptic scale: ∼200², ∼400², and ∼800² km². Coarser-resolution levels increase the receptive field proportionally to their downsampling rate, allowing one to achieve larger receptive fields with fewer layers.

View Full Size

Neural weather model architecture, modified from a UNet 3+ architecture (Huang et al. 2020). The number of layers of each encoder and decoder stack is as indicated in the schematic. Encoder convolutional layers have geometrically growing dilation factors r = 2^l, where l = 0, 1, … is the layer number within the stack from inputs to outputs, and decoder layers have dilation factors r = 8 and 16. The layers connecting same-level encoders and decoders have convolutions with 32 filters and dilation factor r = 4. All other layers have dilation factor r = 1, and all layers have convolutional kernels of size 3 × 3. Same-level layers implement 32, 64, and 128 filters in the first, second and third levels, respectively.

Citation: Artificial Intelligence for the Earth Systems 2, 1; 10.1175/AIES-D-22-0035.1

• Download Figure
• Download figure as PowerPoint slide

b. Encoder and decoder architecture

The architecture of our model is based on the UNet 3+ architecture (Huang et al. 2020) with a few modifications. All nonlinearities consist of parametric rectified linear units (PReLUs) that share parameters across all dimensions except the channels (He et al. 2015). We use dilated convolutions, as previously mentioned, with dilation factors r that increase geometrically with network depth at every resolution level. The first two levels have encoder and decoder stacks with two layers each, and the synoptic-scale level is composed of an encoder stack with four layers.

Each encoder layer l = 0, 1, … applies 2D 3 × 3 dilated convolutions with dilation factor r = 2^l and a PReLU nonlinearity. The decoder layers in the first two levels apply 2D 3 × 3 convolutions with dilation factors r = 2³ and 2⁴, respectively, and are each followed by a PReLU nonlinearity. In addition, we include a nonlinear skip connection at the finest resolution level to easily capture persistence. Downsampling between levels is performed using max-pooling. In the decoder, upsampling is followed by 2D 3 × 3 convolutions. The number of layers per level was obtained through cross-validation from a small set of architectures that achieved a full receptive field.

All layers at 200² km² are composed of 32 convolutional filters, and layers at 400²- and 800²-km² resolution apply 64 and 128 filters to their inputs, respectively. The skip connections between encoder and decoder stacks, as well as the upsampling layers, have 32 filters each. In each layer of the network, we use two independent convolutional kernels for each filter: one covering all four equatorial faces of the cubed sphere, and the other covering the polar faces. As shown in section 4, there is no discernible imprint of the cubed sphere edges on the model forecasts. In the end, the model architecture has about 1.8 million parameters, halfway between the complexity of the models used in Weyn et al. (2020, 2021).

a. Reference models considered

We assess the skill of HeatNet, ExtNet, and GenNet against persistence and the ECMWF S2S forecast system (Vitart et al. 2017). The ECMWF S2S system is an operational model that provides real-time 46-day forecasts 2 times per week. For dates in the test set (2017–21), real-time forecasts used ECMWF’s IFS cycles CY43R1, CY46R1, and CY47R2 (ECMWF 2016, 2019, 2020). The S2S system employed 91 vertical levels until May 2021, and 137 levels after that. All versions are coupled to an ocean model at 0.25° resolution with an interactive sea ice model and use a triangular-cubic-octahedral horizontal discretization with 16-km resolution for days 1–15 and 32 km after that (Malardel et al. 2016).

To allow comparison with our deep learning systems, the ECMWF forecasts are bilinearly interpolated to the same resolution as the input ERA5 data (2°), subtracting the same ERA5 mean climatology to produce climatological anomalies. Then, the results are mapped to be cubed sphere using the conservative remapping of Ullrich and Taylor (2015) and Ullrich et al. (2016); all skill metrics are computed on this grid. To assess potential errors due to the spherical harmonics truncation employed by ECMWF’s Meteorological Archival and Retrieval System (MARS), we downloaded the forecasts at 2° and 0.25° resolutions, and compared the forecasts after bilinearly interpolating the latter to the 2° grid. The root-mean-square difference between the forecasts is ∼10⁻³ K, much lower than typical forecast errors.

For all comparisons in this study, we employ both the real-time daily averaged ECMWF control and perturbed ensemble forecasts (Vitart et al. 2017). Model drift is removed from the real-time ECMWF forecasts using 660 reforecasts covering the past 20 years and initialized from ERA5 data (Vitart et al. 2017). Comparisons with the ECMWF control assess the skill of NWMs against a deterministic “best guess” physics-based forecast. The ECMWF S2S ensemble prediction system employs 50 additional ensemble members, perturbing both their initial conditions and model physics to capture forecast uncertainty (Buizza et al. 1999). Operational warning systems typically use perturbed ensembles, which have been shown to yield a higher economic value than high-resolution deterministic forecasts (Richardson 2000; Palmer 2017). For this reason, we include the ECMWF ensemble mean forecast for comparison. Information beyond the first moment of the ensemble statistics is also valuable (Molteni et al. 1996; Zhu et al. 2002; Palmer 2017). However, we limit our comparison to the ensemble mean in this study, since we only consider NWM point forecasts. Even though our models yield a single deterministic output, direct NWM forecasts more closely resemble an ensemble mean prediction than a physical trajectory of the system; this interpretation is supported by the results in sections 4b–f.

Two additional points should be considered when interpreting the relative skill of the ECMWF forecasts. First, the real-time ECMWF system is initialized from the operational IFS analysis, not ERA5, which leads to reduced accuracy at short lead times. Second, the native resolution of the ECMWF system is higher than the resolution of the NWMs. This is both an asset and a liability when evaluating pointwise objective scores; higher resolution reduces structural model errors, but inevitable errors in the timing and location of sharper resolved features can result in lower skill (Mass et al. 2002). Nevertheless, negative impacts of resolution on forecast skill are reduced in our study through the smoothing induced by bilinear interpolation and daily averaging (Accadia et al. 2003).

b. Forecast skill for summer over land

Although we train the NWMs using global data from all seasons, we evaluate here the performance of the forecast systems exclusively for summer over land, where heatwave prediction is most relevant. We define summer as the June–August trimester for the Northern Hemisphere and December–February for the Southern Hemisphere.

To assess model skill during increasingly hot summer days, we evaluate forecasts using two different temperature anomaly percentiles: the 75th (hot) and 95th (extremely hot) percentiles. Setting these thresholds allows assessing the forecast systems as binary classifiers. When evaluating regressive skill, conditioning on the target distribution can confront forecasters with the dilemma of overforecasting a rare event to improve their scores. There is no obvious way to avoid this problem when evaluating the regressive skill of deterministic forecasts at predicting extremes (Lerch et al. 2017). We verified that this dilemma is not a concern for the models we evaluate, after their global bias is subtracted, since they either underpredict extreme anomalies (NWMs), or are well-calibrated (ECMWF control); results conditioned on the union of forecast and target values, which account for false alarms, are included in appendix B.

The regressive skill of the models is characterized in this study through the debiased root-mean-square error (RMSE_d) and the centered anomaly correlation coefficient (AnCC) of standardized temperature anomalies. The RMSE_d is defined as the RMSE of forecasts with respect to targets after removing the global mean bias per lead time of forecasts with respect to targets in the entire test set. We choose to debias the forecasts to prevent forecast bias from positively affecting the skill metric, since the mean target above the temperature thresholds is nonzero (Lerch et al. 2017). (The subtracted bias is shown for all models in Fig. 5 for reference; it is clear that subtracting the bias prevents HeatNet from hedging.)

The skill of the different models over land is shown in Fig. 3 for the summers of 2017–21. The lead time is shown in a logarithmic scale to differentiate between three different time scales: the short range (<3 days), the medium range (3–10 days), and the extended or subseasonal range (11–28 days). In the short-range errors are dominated by the initialization, which is more precise for the NWMs, since ERA5 data are fed as predictors. The medium range is characterized by predictable trajectories of the atmospheric state, whereas forecasting a single physical trajectory in the extended range typically adds little value over climatology (Lorenz 1969b). Predictive power in the subseasonal range is associated with slower dynamical modes of the climate system, like the MJO or those arising from ocean–atmosphere interactions (Palmer 1993; Zhou et al. 2019).

View Full Size

Forecast metrics for different models during the summer months of 2017–21 and over land. Metrics are shown for forecasts conditioned on target standardized temperature anomalies being above the (a)–(d) 75th and (e)–(h) 95th percentiles. Shown are the (left) debiased root mean square error (RMSE_d), (left center) centered anomaly correlation coefficient (AnCC), (right center) equitable threat score (ETS) and (right) extremal dependence index (EDI). Uncertainty bands, shown for the NWMs as a reference, represent 1 standard deviation. Results are only shown for metrics with a robust uncertainty estimate; details may be found in appendix A.

Citation: Artificial Intelligence for the Earth Systems 2, 1; 10.1175/AIES-D-22-0035.1

• Download Figure
• Download figure as PowerPoint slide

The extreme-focused HeatNet and ExtNet outperform GenNet for both temperature thresholds and all metrics considered, highlighting the usefulness of the exponential loss (2) in the extreme prediction task. All NWMs maintain a higher anomaly correlation with the targets than persistence, but only the models trained on the exponential loss improve upon persistence in a mean-square-error sense during extremely hot days (${\tilde{T}}_{2\mathrm{m}}>95\text{th}\hspace{0.17em}\text{percentile}$). HeatNet, which is trained to emphasize positive extremes exclusively, yields forecasts with higher AnCC than the symmetric ExtNet during hot summer days. Although the AnCC difference indicates higher predictive potential relative to ExtNet in this task, its one-sided emphasis on heatwaves leads to a significant positive bias, as shown in Figs. 4 and 5. This bias is detrimental to the prediction of dynamic temperature anomalies, increasing the RMSE_d, and does not lead to significant classification skill improvements over ExtNet (Figs. 3g,h).

View Full Size

PDFs [here f()] of forecast global standardized temperature anomalies during the period 2017–21. Results are shown for all NWMs, the control and ensemble forecasts from the ECMWF S2S system, and the true target distribution (ERA5). Note that the PDFs are not centered about zero, indicating a prediction of the shift in climatology from the 1979–2019 mean.

Citation: Artificial Intelligence for the Earth Systems 2, 1; 10.1175/AIES-D-22-0035.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Forecast metrics for the general 2-m temperature T_2m prediction task, for different models as a function of lead time, showing the (a) RMSE, (b) AnCC, (c) Kullback–Leibler divergence D_KL of ${\tilde{T}}_{2\mathrm{m}}$ with respect to the targets, and (d) unconditional T_2m bias. All results are global and temporal averages over the period 2017–21. NWM names follow section 2b. Uncertainty is defined as in Fig. 3.

Citation: Artificial Intelligence for the Earth Systems 2, 1; 10.1175/AIES-D-22-0035.1

• Download Figure
• Download figure as PowerPoint slide

The skill of all models is comparable for day-ahead forecasting. In the medium range, the control and ensemble ECMWF forecasts remain superior, but their skill drops significantly faster than that of the NWMs beyond the first week. After the second week, the extreme-focused NWMs have higher regressive skill than the physics-based models. The RMSE_d skill of NWMs relative to the ECMWF system is significantly higher when considering all hot days (${\tilde{T}}_{2\mathrm{m}}>75\text{th}\hspace{0.17em}\text{percentile}$) than during extremely hot days (${\tilde{T}}_{2\mathrm{m}}>95\text{th}\hspace{0.17em}\text{percentile}$). Positive skill in the extended range is enabled by the use of the exponential loss, without which the NWM forecasts cannot improve upon the AnCC of the operational ensemble system.

The ECMWF ensemble mean forecast substantially improves upon the regressive skill of the control run in the extended range, with RMSE_d and AnCC metrics closer to ExtNet. However, higher regressive skill in the extended range does not translate into higher classification skill, as shown by the ETS and EDI diagnostics. As classifiers, HeatNet and ExtNet have slightly higher skill than the ECMWF models beyond the medium range for hot days (${\tilde{T}}_{2\mathrm{m}}>75\text{th}\hspace{0.17em}\text{percentile}$). During extremely hot days (${\tilde{T}}_{2\mathrm{m}}>95\text{th}\hspace{0.17em}\text{percentile}$), all NWMs fail to improve upon the ECMWF system as classifiers, although the use of the exponential loss significantly increases the skill of the NWMs for all regression and classification metrics considered.

We also analyzed interhemispheric differences in skill for summer over land using the same thresholds and found that all models have higher skill in the Northern Hemisphere. The interhemispheric contrast is higher for the NWMs than for the physics-based models; results are included in the online supplemental material.

c. Smoothing of forecasts with lead time

The contrast between regressive and classifying skill of NWMs and the ECMWF ensemble is due to a smoothing of their forecasts as lead time progresses, and the predictability of the targets diminishes. Here, we define smoothing as loss of sharpness, or loss of ability to predict events far from climatology. This smoothing is illustrated in Fig. 4 through the evolution of the forecast probability density functions (PDFs) with lead time for all models considered. Smoothing leads to a density concentration near the mean, as the probability of strong temperature anomalies decreases.

In the case of the ECMWF ensemble, lower predictability reduces the correlation between individual forecasts with lead time. This leads to a variance reduction in the ensemble mean distribution. Smoothing is also typical of data-driven methods, although in this case it is the result of forecast error minimization under uncertainty (e.g., Sønderby et al. 2020). While the PDF of individual (hindcast corrected) physics-based forecasts remains relatively constant, data-driven forecasts shift toward distributions closer to the target mean, with fewer extreme events.

Notably, this smoothing is slowed down through the use of the exponential loss (2), particularly in ExtNet. The use of the symmetric exponential loss increases the probability of significant deviations from climatology: ExtNet forecasts deviations above the 95th target percentile 14 days ahead 4.5 times more frequently than GenNet, and only 25% less frequently than the ECMWF ensemble mean. Minimizing the positive exponential loss also reduces forecast smoothing, but it leads to a positive bias and makes HeatNet forecasts of negative anomalies extremely unlikely. The deviation of the forecast distribution from the true target PDF is further quantified in Fig. 5 through the Kullback–Leibler (KL) divergence, which is an information-based measure of the difference between probability distributions (Kullback and Leibler 1951; Joyce 2011). The use of the symmetric exponential loss reduces the divergence of ExtNet to less than half of the GenNet divergence for all lead times, whereas the bias induced by the positive extreme loss results in a similar KL divergence when compared with GenNet.

Although ExtNet does not manage to capture the same sharpness as the ECMWF ensemble, it is closer in KL divergence to it than to the MSE-trained GenNet model, highlighting the effectiveness of the exponential loss (2) in retaining forecast sharpness for a given architecture. Interestingly, the difference in probability of strong positive anomaly forecasts between ExtNet and the ECMWF ensemble mean in the extended range is significantly smaller than the difference in negative anomaly probabilities (Fig. 4c), even though the loss used to train ExtNet is symmetric. This suggests that positive anomalies are easier to capture than negative anomalies given our predictors.

d. Global surface temperature prediction skill

To further assess the effect of the exponential loss (2) on the general temperature prediction problem, we include in Fig. 5 the RMSE and AnCC of T_2m (i.e., not standardized) for all dates in the test set, over both land and oceans. Note that the RMSE in this case is not debiased. Remarkably, ExtNet shows a very small reduction in forecast skill in the general temperature prediction problem with respect to GenNet. All NWMs beat persistence for all lead times and remain skillful with respect to the ECMWF control beyond the medium range; the ECMWF ensemble mean remains the most skillful model in the general prediction task. Although the RMSE of ExtNet forecasts converges to that of climatology after 3 weeks, the model can forecast strong deviations from climatology (as shown in Fig. 7 for an individual forecast at 23 days of lead time). Finally, the forecast biases of GenNet and ExtNet are similar in magnitude to those of the ECMWF model (Fig. 5d), even though the neural weather predictions are not bias-corrected by reforecasts. HeatNet does suffer from a significant positive bias, which explains its loss of skill with respect to ExtNet. From Figs. 3 to 5, it is evident that ExtNet provides the best compromise between extreme heat forecasting skill, forecast reliability and general prediction accuracy among the NWMs considered.

Figure 5a also allows comparison with other NWMs in the literature. Weyn et al. (2021) use a neural network with a simpler albeit similar architecture as a time integrator, forecasting fields 6 and 12 h into the future with each inference step. They show that only when taking an ensemble mean of such models can they beat the RMSE of the ECMWF S2S control forecast in the extended range. In contrast, producing all lead time predictions at once, a single ExtNet forecast is able to improve upon the ECMWF control forecast both in RMSE and AnCC in the extended range. This is consistent with studies comparing direct and iterative forecasting using NWMs, which show that the former configuration leads to enhanced regressive skill (Rasp et al. 2020). The similarities between the ensemble forecast of Weyn et al. (2021), the ECMWF ensemble, and our results, suggest that NWMs outputting longer lead times yield forecasts more similar to the ensemble mean of physics-driven forecasts than to a given physical trajectory. The similarities include the smoothing of forecasts with lead time, and the saturation of the RMSE in the extended range around the climatological error.

The results in Figs. 3–5 yield important insights into the questions that we posed in the introduction. NWMs trained on limited historical data can improve upon persistence in the prediction of out-of-sample rare events, in a regressive sense. As classifiers of extreme events, they only remain skillful with respect to persistence in the short range, due to their loss of sharpness with lead time. For our chosen architecture, positive regressive and classifier skill can only be achieved for extreme events when employing the exponential loss (2). Furthermore, training on the symmetric exponential loss, ExtNet is able to reduce the prediction error for extreme events and slow down the distributional shift with lead time, all while maintaining an unconditional regressive skill practically indistinguishable from models trained on the MSE. The extreme-focused models improve upon the ECMWF models in the prediction of rate events in a regressive sense after 2 weeks; in the medium range the physics-based models remain vastly superior. We now explore two specific heatwave events as forecast by the ECMWF model and the NWMs to illustrate the implications of these results.

e. Analysis of the 2017 western European heatwave

Sections 4b–d highlight the different ways in which uncertainty affects physics-based and NWM forecasts. These differences are further explored here at the regional scale by considering the western European heatwave of June 2017. The 2017 heatwave resulted in the hottest June on record in Spain and the Netherlands, and the second warmest in France and Switzerland. It was associated with northward warm air intrusions fostered by a subtropical ridge over western Europe, as shown by Sánchez-Benítez et al. (2018).

Forecasts of the standardized temperature anomaly on 20 June 2017 are shown in Fig. 6 for several lead times. The ECMWF S2S forecasts initialized 5 days prior accurately predicted the spatial anomaly patterns over western Europe, slightly overpredicting their magnitude over coastal regions and Morocco. In contrast, the control forecast initialized 15 days prior projected important negative temperature anomalies over most of western Europe, the opposite of what was observed. It also failed to predict the warm air intrusion from the Saharan coast. Only about 10 of the 50 ECMWF perturbed ensemble members predicted warm temperature anomalies over France and Spain, and a higher fraction predicted negative anomalies; forecasts from 22 of these members are shown in appendix C for reference. As a result, the ensemble mean forecast was close to climatology outside the Mediterranean Sea.

View Full Size

Daily averaged standardized 2-m temperature anomaly over Europe on 20 Jun 2017 from (a) the ERA5 reanalysis product and as forecast by (b),(c) the ECMWF S2S control (ECMWF Cont.), (d)–(f) the ECMWF perturbed ensemble mean (ECMWF Ens.), and (g)–(i) ExtNet. The lead time of the forecasts is given in parentheses in the title of each panel.

Citation: Artificial Intelligence for the Earth Systems 2, 1; 10.1175/AIES-D-22-0035.1

• Download Figure
• Download figure as PowerPoint slide

On the other hand, ExtNet robustly forecast the warm air intrusion for the same lead time (15 days), but not its northward penetration into France and the “Benelux” countries. At 5 days of lead time, ExtNet predicted positive temperature anomalies over Europe, although the forecast was too mild and inferior to the ECMWF forecasts. Overall, the NWM forecasts track well both the magnitude and patterns of temperature anomaly in the short range. In the medium and extended ranges, the forecasts match the temperature anomaly patterns well, but underestimate their magnitude. Figure 6 is consistent with our hypothesis that, contrary to physics-based models, forecasts by direct NWMs do not represent trajectories of the system. They are more closely related to the mean projection of an ensemble of physics-based forecasts, or NWMs acting as time integrators (Slingo and Palmer 2011; Weyn et al. 2021; Scher and Messori 2021).

f. Analysis of the 2021 western North American heatwave

To showcase the benefits of the symmetric exponential loss, we compare forecasts of the 2021 western North American (WNA) heatwave provided by ExtNet, GenNet, and the ECMWF ensemble in Fig. 7. We consider the WNA heatwave because its forecast using operational systems is well characterized in the literature (Lin et al. 2022). Several phenomena have been suggested as causes of the WNA heatwave. Lin et al. (2022) note the eastward propagation of a Rossby wave train from the tropical western Pacific that may have favored the formation of a heat dome over western North America. Mo et al. (2022) and Lin et al. (2022) also show that the heatwave was preceded by a strong atmospheric river transporting warm moist air from Southeast Asia into the region.

View Full Size

Daily averaged standardized 2-m temperature anomaly over North America on 26 Jun 2021 from (a) the ERA5 reanalysis product and as forecast by (b),(c) ECMWF Ens.; (d)–(f) ExtNet; and (g)–(i) GenNet. The lead time of the forecasts is given in parentheses in the title of each panel.

Citation: Artificial Intelligence for the Earth Systems 2, 1; 10.1175/AIES-D-22-0035.1

• Download Figure
• Download figure as PowerPoint slide

We focus on the heatwave onset, which took place 25–26 June 2021. The actual temperature anomaly on 26 June was characterized by heatwave conditions over Washington, Oregon, and British Columbia (Canada). Extreme temperature anomalies were also observed over the northeastern Pacific and the Labrador Sea (Fig. 7a). The ECMWF ensemble forecast from 21 June correctly predicted warm temperature anomalies over western North America. The forecast from 3 June, more than 3 weeks ahead, failed to predict positive anomalies over western North America or the Labrador Sea. This loss of predictive skill over land has been linked to the inability to forecast both the continental penetration of the atmospheric river (Mo et al. 2022), and the eastward shift of the atmospheric ridge over western Canada (Lin et al. 2022).

ExtNet forecast the anomaly pattern correctly with 2 days of lead time, but only predicted significant positive anomalies over Washington, Oregon, and the Labrador Sea in the forecast 5 days prior. Relative to GenNet, ExtNet provides significantly better and sharper forecasts for all lead times considered, confirming the results in sections 4b–d. GenNet underpredicted the extent of the heatwave over North America even in the short range, and failed to predict continental penetration 5 days prior to the event. At 23 days of lead time, the ExtNet forecast closely resembles that of the ECMWF ensemble, exhibiting an anomaly dipole over the eastern Pacific (Figs. 7c,f). The data-driven model exhibits better correlation with the target over the Labrador and Bering Seas at this lead time, highlighting the skill of the model in the extended range.

a. Feature attribution through integrated gradients

The attribution for each feature x⁽ⁱ⁾ is defined as the mean absolute value of its contribution to the model forecast y′ = Ψ(θ^*, x) with respect to a null-contribution baseline forecast ${\mathbf{y}}_b^{\prime }=\mathbf{\Psi }({\mathit{\theta }}^{*},{\mathbf{x}}_b)$. We formulate the baseline input x_b to be the feature vector on the linear path between x_min and x_max that results in a forecast closest to global climatology (y′ ≈ 0), where x_min and x_max are feature vectors constructed using the minimum and maximum values of the features found in the evaluation set, respectively.

b. Relevance of model inputs

We apply the integrated gradients methodology described in section 5a to ExtNet forecasts for summer over land during the 5-yr period from 2015 to 2019. Feature attributions are shown in Fig. 8 for the extreme and the general prediction task, and for lead times spanning the short, medium and extended range. The contributions from the most recent data, data from the previous 2 days, and data from the first 4 days of the week preceding the forecast are shown in different colors to quantify the relevance of past history as a predictor of future states.

View Full Size

Forecast feature attribution for summer over land, using data spanning the period 2015–19. Attribution is shown for (a),(c),(e) extreme (${\tilde{T}}_{2\mathrm{m}}>2$) and (b),(d),(f) general prediction, and for different lead times. Feature names follow the notation name/pressure level, where each name follows ECMWF notation: mtnlwrf is outgoing longwave radiation, pv is potential vorticity, z is geopotential, swvli is soil water volume at level i below the ground, and t2m is temperature 2 m above the surface. Pressure level 0 represents the land surface. The legend label −iD defines the contribution of data i days before forecast initialization.

Citation: Artificial Intelligence for the Earth Systems 2, 1; 10.1175/AIES-D-22-0035.1

• Download Figure
• Download figure as PowerPoint slide

We find that ${\tilde{T}}_{2\mathrm{m}}$ itself is the most important feature in all cases, while the relative relevance of other features increases with forecast lead time and depends strongly on the task considered (i.e., extreme or general prediction). In addition, the relevance of feature history robustly increases with lead time, suggesting that NWM forecasts in the extended range learn to rely on lower frequency signals. These shifts in relevance with lead time may be used to optimize the NWM architecture, for instance by pruning connections between short lead time forecasts and predictors more than a few days old.

In the extreme prediction task, the temperature anomaly, 700-hPa geopotential height anomaly, topography, and OLR are important predictors at all lead times. Soil moisture below 7 cm gains relevance with lead time, consistent with the characteristic low frequency of land–atmosphere coupling processes and the memory of root-zone soil moisture (Wu et al. 2002). Additional relevant predictors include the potential vorticity at 500 hPa, and the geopotential height anomaly at 300 and 500 hPa. The potential vorticity at 700 hPa and the surface soil moisture above 7 cm are irrelevant to ExtNet predictions at all time scales.

The most important predictors for extreme prediction are also the most important ones for the general forecasting task. However, the relative importance of the temperature anomaly is significantly greater in the general problem, dominating the total attribution. Soil moisture plays a much smaller role in this task relative to heatwave prediction, consistent with observations of a much stronger land–atmosphere coupling under extreme conditions (Orth and Seneviratne 2012; Benson and Dirmeyer 2021). Overall, Fig. 8 suggests that generic estimates of the relevance of alternative features for NWM forecasting may underestimate the contribution of such predictors to extreme event forecasting. For the auxiliary features, we find topography to be the most relevant predictor, followed by the land–sea mask. Finally, the global attribution decreases with lead time, consistent with the progressive loss of predictive information and forecast sharpness.