1. Introduction
Reliable precipitation forecasts are essential for decision-making in agriculture (Ingram et al. 2002), road transportation (Changnon 1996), aviation (Kulesa 2003), and construction (Senouci et al. 2018). In particular, short-term precipitation forecasts of up to 2 h (i.e., nowcasting) are important to enable water-related risk management. Short-term forecasting systems rely on nowcasts to provide decision-making support for taking preventative actions (Thirugnanam et al. 2020; De Luca 2013). Given that the frequency and intensity of heavy rainfall events are expected to increase due to climate change, the need for accurate precipitation nowcasting becomes ever more important (Attema et al. 2014).
Traditionally, nowcasts are made using numerical weather prediction (NWP) models or radar extrapolation–based models (Bauer et al. 2015). NWP models use physics-based simulation to forecast the weather. These models suffer from spinup problems, and they are therefore less skillful in precipitation nowcasting than radar extrapolation–based models such as spectral prognosis (S-PROG; Seed 2003). These models forecast precipitation by computing a motion field from recent radar data and advecting real-time observations along this trajectory. Optical flow methods are designed with the assumptions of Lagrangian persistence in mind, assuming that both the total rainfall intensity and the motion field remain constant (Germann and Zawadzki 2002). However, Lagrangian persistence does not always hold as motion, and intensity can vary with time, and thus, these assumptions can limit model performance.
More recently, various studies have successfully applied machine learning for weather nowcasting (e.g., Shi et al. 2017; van der Kooij 2021). Machine learning methods, such as deep learning (DL), take advantage of the large-scale availability of historical weather data. These models do not require the assumption of Lagrangian persistence. For example, Trebing et al. (2021) proposed the small attention U-Net (SmaAt-U-Net) model for precipitation nowcasting, which is a lightweight version of the U-Net architecture. Furthermore, RainNet (Ayzel et al. 2020) is based on the U-Net and segmentation neural network (SegNet) architectures. Several proposed deep learning models for nowcasting are based on the sequence-to-sequence long short-term memory (LSTM) framework (Sutskever et al. 2014). For example, Shi et al. (2015) proposed a convolutional LSTM (ConvLSTM) approach for precipitation nowcasting using radar images. This was further improved in the trajectory gated recurrent unit (TrajGRU) model (Shi et al. 2017). ConvLSTM and TrajGRU can capture spatiotemporal correlation better than a fully connected LSTM and optical flow–based methods such as S-PROG (Shi et al. 2017). MetNet (Sønderby et al. 2020) used spatial downsampling, temporal encoding, and axial attention–based aggregation for radar-based forecasting. MetNet outperformed NWP and optical flow models in terms of the F1 score, based on U.S. radar data.
In addition to novel deep learning models for precipitation nowcasting, other machine learning techniques have shown their usefulness in this field. Data augmentation and transfer learning, in particular, deserve mention as they help enhance deep learning models’ performance and generalization capabilities. Data augmentation plays a role in mitigating the challenges posed by limited training data and improving the robustness of DL classifiers. For example, U-Net-based models, widely used for precipitation nowcasting, have demonstrated improved results when combined with data augmentation strategies (Ronneberger et al. 2015; Falk et al. 2019; Shorten and Khoshgoftaar 2019). The training dataset can be expanded by applying rotations, flips, and scaling techniques, enabling the model to learn more diverse and representative features. Furthermore, transfer learning has emerged as a powerful approach to leverage pretrained models and adapt them to new domains or tasks. Transfer learning techniques in weather nowcasting models can facilitate generalization to different locations and enhance performance (Khorrami et al. 2021; Herruzo et al. 2021; Han et al. 2021). By incorporating data augmentation, models such as U-Net-based architectures can benefit from an expanded training dataset and learn more representative features. Transfer learning, on the other hand, enables models like ConvLSTM, TrajGRU, and MetNet to leverage preexisting knowledge and adapt it to new locations.
Although these deep learning approaches typically perform better than traditional approaches, they can also lead to forecasts that are smooth, blurry, or look unrealistic (Tran and Song 2019). Generative adversarial networks (GANs) have been shown to address this issue (Hayatbini et al. 2019; Jing et al. 2019; Tian et al. 2019). In the GAN framework, two DL models are pitted against each other in an adversarial way (Goodfellow et al. 2014). The generator creates synthetic samples, and the discriminator has to differentiate between real and synthetic samples. The generative model does not solely optimize the distance to a target image but also aims to minimize the penalty that occurs when the discriminator assesses the image as unrealistic. Eventually, this leads to more detailed and realistic forecasts. For example, Jing et al. (2019) proposed the adversarial extrapolation neural network (AENN) GAN model that outperformed various other nowcasting models. Further, Ravuri et al. (2021) proposed a deep generative model of radar (DGMR), based on a conditional generative model with one generator and two discriminators. One discriminator focused on spatial consistency and the other on temporal consistency. The model was evaluated quantitatively using several statistical metrics and qualitatively by a group of expert meteorologists in the United Kingdom. The model showed more fine-grained details and more realistic results than other traditional and DL methods.
Despite recent advancements, several gaps remain that require further research. First, while the issue of decreased performance (in terms of accuracy) for high-intensity rainfall and long lead times is not unique to deep learning methods, various studies have shown that deep learning approaches are also susceptible to decreased performance in these cases (Ayzel et al. 2020; Shi et al. 2015, 2017). Further, deep learning models typically forecast precipitation using limited or no physical constraints. However, weather-based features can contain much information and dependencies for precipitation (Bouget et al. 2021). Our study has investigated several potential extensions to the state-of-the-art DGMR model. First, the influence of different loss functions on forecast accuracy was explored, including an extended balanced loss function that assigns more weight to severe events. Second, the fusion of radar data and weather-based features has been explored to incorporate additional meteorological information into the modeling. Through a comparison with various benchmark methods, we provide qualitative and quantitative insights on how these experiments can further improve precipitation nowcasting performance.
2. Methods
a. Data
As a case study, all models have been trained and evaluated on precipitation radar data from the Royal Netherlands Meteorological Institute (KNMI). Data from the period of 2006–21 were obtained from operational radar stations located in Herwijnen and Den Helder, which provided full coverage of the Netherlands and surrounding areas in Belgium, Germany, and the North Sea. This research has used the real-time radar product with a 5-min temporal and 1 km × 1 km spatial resolution. To explore the potential of other meteorological variables for precipitation nowcasting, air temperature data were obtained from KNMI. In total, more than 45 automatic weather stations in the Netherlands provided the 1.5-m air temperature in degrees Celsius at 10-min intervals. The automatic weather stations are located on sea and land. The stations on sea are mostly on platforms with a higher altitude, and therefore, the measured temperature has been corrected for this higher altitude, by applying a moist-adiabatic lapse rate of 0.006 K m−1.
b. Preprocessing steps
As this study has used a real-time radar product, various additional preprocessing steps have been implemented to enhance data quality. First, a mean-field bias correction was applied to the data using automatic rain gauge stations. Further, extreme rainfall intensity values (e.g., caused by hail or clutter) were clipped at 100 mm h−1. The real-time product also contains various instances of clutter. Clutter consists of unwanted echoes (i.e., objects not part of the radar’s target), such as airplanes, birds, ships, or wind turbines. Although machine learning models could learn to handle clutter, it is important to remove it during preprocessing. First, machine learning models would have an unfair advantage compared to extrapolation-based methods. Second, clutter in the reference dataset will have a negative influence on both the model and the verification scores. Rainfall tends to gradually increase from low to high precipitation values and vice versa, while clutter suddenly increases and decreases in radar reflectivity, resulting in abnormally large gradients. The amount of clutter in the entire dataset was reduced by discarding single rain images that contained many pixels with a high gradient (Schreurs 2021). Finally, in order to make the models faster, radar images with very low precipitation intensities were discarded to avoid images that did not contain any rainfall at all. Images were labeled as “containing precipitation,” if the mean intensity per pixel was above 0.01 mm h−1. This led to the removal of 58.22% of samples (see Table 1). Note that only 4.18% of the samples contain heavy precipitation with a peak intensity higher than 10 mm h−1. All rainfall rates have been normalized using minimum–maximum normalization for further processing, resulting in values between 0 and 1.
Statistics of precipitation intensity using radar data (years 2006–21). Five-minute frames/sequences containing a specific amount of precipitation (mm h−1 is defined by peak value over the entire frame), expressed in percentages. Precipitation sequences of 30 min, containing six consecutive images of 5-min frames.
The radar images in the dataset have an original size of 765 × 700 pixels. Zero padding was used to increase the dimensions to 768 × 768 pixels. Bilinear interpolation reduced the image size by a factor of 3, obtaining a final precipitation image with a size of 256 × 256 pixels. A sequence of radar images is used to capture the evolution of precipitation. To obtain 30-min episodes of rain events, only sequences containing six consecutive 5-min frames, with precipitation, were retained (see Table 1). The 30-min sequences were used for precipitation nowcasting for the next 5, 10, 15, 30, 60, and 90 min. Forecasts were evaluated using the original radar images. Therefore, the model outputs were upsampled using bilinear interpolation with a factor of 3 and cropped to a size of 765 × 700 pixels.
When dealing with time series data, splitting the data randomly into training and test sets is not recommended, as data leakage may occur (Fu 2011). To prevent this, the dataset was split using consecutive years. The years from 2008 to 2020 were used as a training set, whereas 2006 was used as a validation set to optimize hyperparameters. The most recent year available at the time of this study (i.e., 2021) was used as a test set.
c. Deep generative model of radar
d. Model extensions
First, the impact of different loss functions on the ability to accurately forecast higher rainfall intensities was investigated. Importantly, the proportion of rainfall events at different intensity thresholds is imbalanced, with only a minority of the data capturing severe events (see Table 1). Standard loss functions, therefore, optimize the model mainly for moderate intensity events since samples are treated as equally important, and most samples fall in the moderate and moderate-heavy categories. However, high-impact events are typically of most importance for meteorological applications. Different loss functions could encourage the model to predict higher rainfall intensities more accurately. Note that this could also lead to (marginally) lower performance for lower rainfall intensities.
In a conditional GAN, the reconstruction loss
Besides different loss functions, the addition of weather-related features to the model was also investigated. Temperature data from KNMI were transformed to a 765 × 700 grid to align with the dimensions of the precipitation data. Temperature data were used from automatic weather stations on land and sea. For each automatic weather station, the temperature measurement was placed onto the grid, based on location. Empty grid cells were filled using inverse distance-weighted (IDW) interpolation (Lu and Wong 2008). The IDW method interpolated the pixel cell values by averaging the station data points in the neighborhood of each processing cell, using a 5-km block. Zero padding and bilinear interpolation reduced the size to 256 × 256 pixels, and the values were normalized. Finally, the precipitation and temperature images were stacked along the temporal axis, resulting in a generated input tensor with dimensions of 6 × 256 × 256 × 2 (i.e., two channels capturing six precipitation maps and six temperature maps). This way, the convolution filters take into account that rainfall and temperature are provided at the same location and time.
e. Implementation and calibration
The DGMR model with the extensions described in section 2d was implemented using Python and the TensorFlow Keras library (Abadi et al. 2016). Figure 1 presents a schematic overview of the implemented modeling pipeline. The code of our implementation, using KNMI data, is available through GitHub.1
Flowchart of the modeling process for DGMR and AENN GAN: 1) green blocks are data handling; 2) yellow blocks are distribution of the frames into the modeling process; 3) orange blocks are specifically related to the DGMR; 4) blue blocks are the three main components of the model (generator and two discriminators); 5) red blocks are output.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
Models were trained and evaluated on the CPU and graphical processing unit (GPU) cluster nodes of the Data Science department at Radboud University (i.e., 2x Intel Xeon Silver 4214 2.2 GHz, 128 GB, 7x GPU). Hyperparameters were iteratively changed to optimize forecast accuracy on the validation set and improve the stability of the models. The scaling parameter for the regularization term in DGMR was set to 21. A batch size of 16 was used, and the model was trained for 125 epochs using the Adam optimizer with a learning rate of 0.0005 for the generator and 0.000 15 for the discriminators. The discriminators were trained two times for every update step of the generator. (All hyperparameter values for the DGMR model are reported in Table B2 of appendix B.)
f. Benchmark models
Model performance in the different experiments was compared to various benchmark models. The selected benchmark models were the optical flow model S-PROG and the deep learning models U-Net, MetNet, and AENN GAN. A description of these models is provided below. As the focus of this study is on precipitation nowcasting in the Netherlands, all benchmark models were calibrated using the data described in section 2a. While data augmentation can be beneficial for specific scenarios and models (Shorten and Khoshgoftaar 2019), additional data augmentation is not included to compare with the DGMR (Ravuri et al. 2021) directly.
1) S-PROG
S-PROG (Seed 2003) is an optical flow–based extrapolation method. First, S-PROG converts radar rainfall rates (mm h−1) to radar reflectivity values (from R to dBR to dBZ). The motion field is constructed using the Lucas–Kanade method (Lucas and Kanade 1981), and semi-Lagrangian extrapolation is applied to the reflectivity field. An autoregressive (AR) AR(2) model was used to estimate the Lagrangian autocorrelation coefficients. The radar field is then decomposed into eight cascade levels, after which the forecasts are constructed using the sum of all the Lagrangian nowcasts at different cascade levels. Finally, values are converted back to rainfall rates (mm h−1) (Imhoff et al. 2020). As the Lagrangian persistence assumes that the rainfall intensity and motion field are stationary for predicting the evolution of the motion field, S-PROG does not need any training. In this study, the open-source implementation (i.e., Pysteps deterministic) and setup from Schreurs (2021) have been used.
2) U-Net
U-Net (Ronneberger et al. 2015) is an encoder–decoder model for image segmentation that can also be used for precipitation nowcasting (e.g., Agrawal et al. 2019; Ayzel et al. 2020; Nie et al. 2021). The standard U-Net model uses convolutional and transposed convolutional layers for the encoder and decoder, respectively. Skipping connections between corresponding layers in the encoder and decoder improves details in the output image. Ravuri et al. (2021) implemented an adapted version of the U-Net architecture by switching the convolutional layers for residual blocks, leading to a slight improvement for all rain intensities. Therefore, this adapted version of the U-Net has been used as a benchmark model. A batch size of 8 was used, and the model was trained for 300 epochs using the Adam optimizer with a learning rate of 0.0002 and default exponential decay rates.
3) MetNet
MetNet is a deep learning framework for precipitation nowcasting proposed by Sønderby et al. (2020). The model architecture is based on spatial downsampling, temporal encoding (using the ConvLSTM framework as encoder), and axial attention-based aggregation. Further, the network uses a separate loss term for each grid cell and produces a probabilistic precipitation map. MetNet uses different layers of input data for the network, including (i) radar precipitation data, (ii) Geostationary Operational Environmental Satellite-16 (GOES-16) data (providing data over the Americas), (iii) per-pixel elevation and position embeddings, and (iv) per-frame time embeddings. Not all of those inputs were easily accessible. Therefore, the model implementation in this study only used the radar data and no additional data (as in Ravuri et al. 2021). Although MetNet was evaluated by Sønderby et al. (2020) on a prediction lead time of up to 8 h, it was trained in our study with lead times of 5–90 min. A batch size of 8 was used, and the model was trained for 500 epochs using the Adam optimizer with a learning rate of 0.000 02 and a default decay rate of 0.
4) AENN GAN
The AENN GAN (Jing et al. 2019) uses a generator and two discriminators to address the issue of blurry forecasts. The generator uses an encoder–decoder architecture for sequence-to-sequence prediction (Sutskever et al. 2014). First, the conditional generator extracts features from the input images. The sequence in feature space is modeled using ConvLSTM layers to capture the temporal dynamics. Similar to the DGMR model, there is a spatial discriminator for generated single frames and a temporal discriminator for frame sequences. Previously, the AENN GAN was implemented using radar data from KNMI for precipitation nowcasting (Schreurs 2021). This implementation replaced the ConvLSTM layers with convolutional gated recurrent unit (ConvGRU) layers to further improve accuracy. Here, this adapted AENN GAN was used as a benchmark model. A batch size of 8 was used, and the model was trained for 100 epochs using the Adam optimizer with a learning rate of 0.0001 for the generator and discriminator.
g. Evaluation criteria
To evaluate the experiments and benchmark methods, forecasts were generated for lead times of 5, 10, 15, 30, 60, and 90 min. Model forecasts were resized and cropped to 765 × 700 pixel for comparison with the ground truth radar images. Our study assesses the models from both a qualitative and quantitative perspective. The qualitative assessment focuses on whether the forecast broadly captures the meteorological characteristics visible in the ground truth image, the level of details visible in the forecast, and whether the generated radar image and sequence look realistic. For the quantitative assessment, both continuous (i.e., MSE and MAE) and categorical metrics are used. The categorical metrics are used to analyze the performance of the models at different rainfall intensity levels (i.e., thresholds). Therefore, the forecasted and observed images are converted into binary maps based on a chosen precipitation threshold, indicating at each pixel location whether precipitation rates are higher than the threshold or not. Three rainfall intensity thresholds are selected for evaluation: 1, 10, and 20 mm h−1. The latter two thresholds capture high and very high rain-intensity events (see also Table 1). The highest precipitation threshold that we consider is 20 mm h−1, which is the 98th percentile of the dataset. The amount of cases with a threshold higher than 20 mm h−1 is not enough to verify the results in a meaningful way. The binary maps are used to compute various statistical metrics (Imhoff et al. 2022). Statistical metrics use a contingency table, with four possible outcomes: (i) true positives (TP), (ii) false positives (FP), (iii) true negatives (TN), and (iv) false negatives (FN). The critical success index (CSI, also known as threat score),
3. Results and discussion
In this section, we present and discuss the results of the proposed methods. Models are evaluated using a shorter maximum lead time than former related studies (Ravuri et al. 2021; Schreurs 2021). As previously discussed, the data from year 2021 are used as the test set; see appendix A for the behavior of the model toward the number of high-intensity events in the test set.
a. Benchmark comparison
Since the original DGMR model (Ravuri et al. 2021) has not been applied to radar data from KNMI, we compare the DGMR against four benchmarks (AENN GAN, S-PROG, U-Net, and MetNet) using those data. The nowcasting performances are evaluated using the MSE, MAE, and the categorical score CSI. Finally, the runtime performance on CPU and GPU is evaluated.
Figures 2a and 2b show that the DGMR, without extensions, performs better than the S-PROG in terms of MAE and MSE. Compared to the deep learning models, the S-PROG model performs better than the U-Net model for longer lead times, but worse than the other DL models. The MetNet model performs worse than the DGMR and similarly (in terms of MSE) or worse (in terms of MAE) than AENN GAN. Especially for the shorter lead times, the DGMR performs best.
(a) Performance of the different models, measured with MAE (mm h−1). (b) Performance of the different models, measured with MSE [(mm h−1)2].
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
Table 2 shows the CSI for the generative models without extensions and the benchmark models (aggregation over all lead times). Comparing the models without extensions against the benchmark models, we see that the DGMR performs best regarding the CSI for the 1 and 20 mm h−1 rainfall intensity threshold. For 10 mm h−1 rainfall intensity, the MetNet model performs best.
CSI for the baseline models and AENN GAN and DGMR using different losses and features; the best result (highest value) for each threshold is indicated in bold. Aggregation over all lead times is used.
When using the default loss functions, the results show that the DGMR scores higher on nowcasting for smaller lead times (i.e., below 30 min), in terms of MSE and MAE. The DGMR also scores better on higher rainfall intensities (i.e., above 10 mm h−1) in terms of the CSI. There are more wrongly predicted nowcasting samples when using the DGMR for higher rainfall intensities (i.e., FAR score of 0.219). This is consistent with the work of Ravuri et al. (2021) that the prediction of high rainfall intensities at longer lead times remains a problem. Using the default loss function, neither of the models do well for high rainfall intensities at longer lead times, but the DGMR model performs the best.
The inference speed is evaluated on the CPU and GPU. The reported time in Table 3 is the median execution speed time for the selected models across 10 samples. (For training times, see Table B1.) The execution speed of U-Net using the CPU/GPU is significantly shorter than for the other models, followed by the AENN GAN and DGMR. Except for MetNet, all the deep learning models are faster than the S-PROG model.
Median execution speed time for the selected models across 10 samples.
Besides the quantitative analyses above, radar nowcast images from the different models have also been assessed from a qualitative perspective. Figure 3 illustrates the model behavior and the difference of nowcasts by the different models for a rainy event, initialized at t0 = 1105 LT 1 May 2021, with a lead time of 30 min. Figure 3 shows that the DGMR outperforms the benchmark models. (See Fig. 9 for the model behavior of the DGMR, with lead times of 5, 10, 15, 30, 60, and 90 min.) All four models capture more or less the same rain pattern, but the AENN GAN model suffers from checkerboard artifacts. Further, MetNet and S-PROG produce more blurry results for higher lead times. The DGMR consists of more (trainable) parameters compared to the AENN GAN. This might be a reason for the increase in performance at several intensity thresholds. However, it also comes with a slight increase in computational requirements. Furthermore, additional examples of the prediction frames are provided in appendix D.
Nowcast and target value for different models for a rainy event, initialized at t0 = 1105:00 LT 1 May 2021 and lead time + 30 min (mm h−1).
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
b. Balanced loss function
In this subsection, the model performances of the DGMR and AENN GAN with four loss versions are compared: the RMSE loss, the Hinge loss [Eq. (4)], the balanced loss function [Eq. (6)], and the extended balanced loss function [Eq. (5)]. All four losses are reconstruction/regularization losses. We propose the extended balanced loss function with the goal of improving model performance for higher precipitation intensities. The extended balanced loss uses the Manhattan distance
As shown in Figs. 4a and 4b, the DGMR-BalancedExtended is somewhat better than the DGMR using the other losses in terms of MAE and MSE for the shorter lead times. Overall, the DGMR-BalancedExtended has the lowest errors in MAE and MSE. The DGMR-Balanced loss performs best for the lead time of 90 min. Thus, based on the MSE and MAE, the proposed extended balanced loss is, in particular, promising for short-term nowcasting, whereas the balanced loss from Ravuri et al. (2021) is preferable for lead times over 90 min.
Performance of DGMR using different loss functions, measured with (a) MAE (mm h−1) and (b) MSE [(mm h−1)2].
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
Table 4 and Fig. 5 show the CSI scores of the two models for different losses and thresholds of precipitation. Higher CSI scores indicate better performance, while the number of false negatives and false positives should be as low as possible. The DGMR-BalancedExtended performs best, followed by DGMR-Balanced; this is the case for all rainfall intensities. The AENN GAN-RMSE performs better than the AENN GAN-Hinge, but it should be noticed that the differences between the RMSE loss and the Hinge loss are often minimal in terms of the CSI and F1 (Tables 4 and 5). High rainfall events are detected more often, as confirmed by the higher value of the POD (Table 6), when using the extended balanced loss. Moreover, in the case of the extended balanced loss, there are also less false alarms for the highest rainfall intensity events, looking at the FAR (Table 7).
CSI for AENN GAN and DGMR using different extensions. The best results (highest values) are indicated in bold.
Performance diagram for AENN GAN and DGMR, using different extensions and rainfall intensity thresholds. The diagonal represents the bias ratio of 1; better CSI values lay close to the top right of the figure. Aggregation over all lead times is used.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
F1 score for AENN GAN and DGMR using different extensions. The best results (highest values) are indicated in bold.
POD for AENN GAN and DGMR using different extensions. The best results (highest values) are indicated in bold. Aggregation over all lead times is used.
FAR for AENN GAN and DGMR using different extensions. The best results (lowest values) are indicated in bold. Aggregation over all lead times is used.
Overall, the models using the extended balanced loss perform better in terms of the CSI, F1, and POD. Nevertheless, these models generally show a higher FAR, while a lower score would be preferred. Using the DGMR with the balanced loss shows the second-best results for the CSI, F1, and POD and the best result for the FAR differs per threshold.
A point to notice is the performance of the AENN GAN with the Hinge loss; the CSI and POD results show that this combination performs the worst. The Hinge loss together with the AENN GAN is making use of the
c. Temperature as an additional feature
This subsection evaluates the model performances of the AENN GAN and DGMR when a feature is added (using the default loss functions). The temperature maps are used as additional input features. The nowcasting performances are evaluated using the MSE, MAE, and the categorical scores CSI, F1, POD, and FAR.
The overall evaluation results for adding the temperature maps to the input are summarized in Figs. 6a and 6b, Table 4, and Fig. 5. The best performance is achieved by the DGMR, using the stacked vector with the temperature grid as the input for the model. For all lead times, adding the temperature feature substantially improves the performance in terms of MAE and MSE.
(a) Performance of DGMR with and without temperature data as additional input, measured with (a) MAE (mm h−1) and (b) MSE [(mm h−1)2].
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
See Table 4 and Fig. 5 for the CSI scores; the scores for a threshold of 1.0, 10.0, and 20 mm h−1 increase when using the temperature as an additional feature. This increment in the score is the case for both models. The F1 scores for the AENN GAN and DGMR are provided in Table 5, averaged over the lead times. As with the CSI score, including temperature as an additional feature improves F1 score over all thresholds.
High rainfall events are detected more often when using the temperature maps as an additional feature; this is confirmed by the higher value of the POD (Table 6) when using both features. A significant increase in the POD is seen when using the DGMR with the additional temperature data. The largest increase is seen when using the DGMR with the additional temperature data, for a threshold of 1.0 and 20.0 mm h−1. The DGMR with additional temperature data consistently outperforms the models with different loss functions. Looking at the FAR (Table 7), there are less false alarms for the threshold of 20.0 mm h−1, when using the temperature maps as additional features.
Overall, the models using the temperature data as an additional feature perform generally better regarding the CSI, F1, POD, and FAR. Using the AENN GAN with temperature data shows the best result regarding the FAR for high-intensity rainfall (Table 7); in this case, the amount of falsely predicted events is lowest. The addition of temperature data (DGMR) leads to a skill increase for all selected rainfall thresholds, with superior performance for lower lead times but an inferior performance for higher lead times. At a threshold of 20.0 mm h−1, the DGMR with temperature data shows a CSI that is almost 1.5 times the CSI of the corresponding AENN GAN model.
Finally, we evaluate the impact of the temperature on the prediction. Shapley (SHAP) values are an explanation method for deep learning models, using the kernel SHAP method from Molnar (2022). Shapley values (Lundberg and Lee 2017) are a mathematical concept derived from cooperative game theory. In the context of machine learning, Shapley values quantify the impact of each feature on the prediction outcome of a model. They provide a fair and interpretable way to attribute importance to features by considering their marginal contributions within all possible feature combinations. In our study, we employ Shapley values to assess the individual contribution of the temperature feature in the precipitation nowcasting process. Positive Shapley values indicate a feature’s positive impact on precipitation prediction, while negative values indicate the opposite. The magnitude of the value signifies the strength of the feature’s influence. In Fig. 7, we can see the dependency plot of the rainfall intensities from the DGMR using the temperature as an additional feature. The figure shows two distinct regimes: one with a smaller slope between the SHAP values and the precipitation intensities for lower temperatures and the other with a larger slope for higher temperatures. This can be explained by the mostly convective nature of precipitation in summer and nonconvective nature of precipitation in winter (Xu 2013). Using this information, adding new weather-related features to the models can give more insight into the underlying relations and an improved precipitation nowcast.
Dependency plot between SHAP values and rainfall intensities, showing the interaction with the (air) temperature (color scale). Blue colors indicate lower temperature values, and orange colors indicate higher temperature values.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
d. Combination of extensions
We analyze the radar nowcast images from the DGMR with extensions to illustrate differences in nowcasting. Figure 8 presents the model behavior and the differences of nowcasts using the different extensions for a rainy event at t0 = 1105 LT 1 May 2021, with a lead time of 30 min. All variations in the models capture similar motion characteristics of the rain. See Fig. 9 for the model behavior of the DGMR and the DGMR using both extensions, with lead times of 5, 10, 15, 30, 60, and 90 min. The DGMR is able to learn more from the temperature data for larger lead times. Looking at areas with high rainfall intensities, we can see that the extended balanced loss increases the intensities in rain fields. Using the temperature data improves the high rainfall intensities and seems to improve the motion and evolution of precipitation.
Nowcast and target value for different extensions, for a rainy event, initialized at t0 =1105:00 LT 1 May 2021 (mm h−1).
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
Nowcast and target value for different models, for a rainy event, initialized at t0 = 1105:00 LT 1 May 2021 (mm h−1).
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
The bias measures the ratio of the frequency of forecast events to the frequency of observed events. It indicates whether the forecast system tends to underestimate or overestimate events. In Fig. 5, according to the diagonal, the AENN GAN—without extensions for 1 mm h−1—has the best tendency to estimate the nowcasting (ideal bias of 1). The figure also shows that for all three intensities, the DGMR with temperature data (i.e., triangle left) has higher CSI scores (i.e., blue contours) than the other models. For the highest threshold, the second best is the DGMR with extended balanced loss. The extensions to the DGMR (extended balanced loss and temperature) gradually improve the CSI by increasing the probability of detection (y axis), while generally preserving the performance for the success ratio (x axis).
Next, results for running the generative networks with both extensions in the same experiment are described. Results of both models are compared with the results of the generative networks without any extensions, the generative networks with different loss functions, and the generative networks using the temperature data as an additional feature. Appendix C highlights the behavior of different loss function of the DGMR, using both extensions.
Incorporating the extended balanced loss helps to put more importance on high rainfall events. Including the physical features, and temperature data, helps improve the nowcasting by using the underlying relations between temperature and precipitation. The CSI, F1, and POD of the DGMR with both extensions show improvements compared to the DGMR and the DGMR with single extensions (Tables 4–7). For the shorter lead times, the improvement is the most significant.
A point to notice is the increase in CSI, F1, and POD when using the combination of extensions for the AENN GAN model. When using the combination, the AENN GAN performs (slightly) better than the standard DGMR. Looking at the AENN GAN for the threshold of 20 mm h−1, CSI shows a more than fourfold increase when using extended balanced loss compared to the AENN GAN without any extensions. Using the combination of extensions shows an additional increase of around 18%. For the AENN GAN, there is an increase in the F1 score when using the combination of extensions, for all rainfall rates. From the POD scores, it appears that for higher rainfall (i.e., 20 mm h−1), there is a substantial increase in the probability of detection. The FAR score indicates the lowest number of high rainfall events falsely predicted when using the AENN GAN with the additional temperature data. However, the mixture of extensions is responsible for a slight increase in the FAR, but still lower compared to only using the extended balanced loss.
Looking at the DGMR for the threshold of 20 mm h−1, CSI shows an increase of around 35% when using extended balanced loss compared to the DGMR without any extensions. Using the combination of extensions shows an additional increase of around 8%. The DGMR with temperature data leads to a decrease of around 60% in terms of FAR for the threshold of 20 mm h−1 and a decrease of around 55% when using the combination of extensions. The decrease in the FAR is related to less falsely predicted high rainfall events.
4. Conclusions
In this study, we have compared several methods for nowcasting precipitation with a primary focus on higher rainfall intensities. In addition, we have proposed extensions to improve the performance of the deep learning approaches.
Using radar data from KNMI, deep learning models have been implemented for precipitation nowcasting in the Netherlands. The DGMR previously proposed in Ravuri et al. (2021) was compared to the AENN GAN and other benchmark models (i.e., S-PROG, U-Net, and MetNet). Precipitation has been forecast for 5-, 10-, 15-, 30-, 60-, and 90-min lead times. S-PROG was chosen as the benchmark of an extrapolation-based method. AENN GAN and DGMR were both able to outperform S-PROG. Although AENN GAN beats S-PROG, AENN GAN is not skillful at predicting precipitation at thresholds 10 and 20 mm h−1, for lead times of 15 min and longer. During the benchmark comparison, we have found that DGMR performs best for shorter lead times (i.e., below 30 min), whereas MetNet and AENN GAN perform best for longer lead times (i.e., above 30 min). Neither of the models, with the default loss function, do well for high rainfall intensities at longer lead times, but DGMR performs the best. We further proposed two extensions to improve the performance of the generative models: the extended balanced loss function and the addition of temperature data as a feature. Incorporating the extended balanced loss helps to put more importance on high rainfall events. Including the physical features, and temperature data, helps to improve the nowcasting by using the underlying relations between temperature and precipitation.
First, the model performance of DGMR with the proposed extended balanced loss function has been evaluated against RMSE loss, Hinge loss, and balanced loss. The extended balanced loss substantially impacts the performance of both AENN GAN and DGMR. We found that the performance of DGMR with the extended balanced loss function improves, particularly for shorter lead times (i.e., below 30 min), which can be relevant for short-term forecasting systems. The extended balanced loss increased the CSI, F1, POD, and the skillful lead times. However, there was an increase in false alarms (i.e., FAR). AENN GAN with the extended balanced loss leads to an increase in performance for all selected rainfall thresholds.
Second, temperature data were included as an additional feature in the DGMR and AENN GAN models. Including this additional feature further improves the DGMR performance, especially for the CSI, F1, and POD. Additionally, the temperature feature leads to a skill increase for all thresholds. Especially for the highest threshold (i.e., 20 mm h−1), the AENN GAN improves substantially when including the additional feature.
Using the combination of extensions leads to an overall improvement in performance for both the AENN GAN and DGMR models. The DGMR using the combination of extensions leads to the best-achieved results, and the prominent advantage is achieved for all rainfall intensity thresholds.
In all cases, with and without extensions, DGMR outperforms the other models. The DGMR, especially with the extensions, is skillful at nowcasting the precipitation for high rainfall intensities, up until 60 min. Neither the deep learning models nor the S-PROG model were skillful at predicting high rainfall intensities for larger lead times (i.e., above 60 min). The Shapley values confirm the distinct relationship between temperature and precipitation in summer versus winter due to (deep) convection in summer. The DGMR could, in the future, be extended with more weather features, for instance, the wind speed (Bouget et al. 2021).
Future work for (probabilistic) precipitation nowcasting of high-intensity events could explore 1) using GANs for the generation of high-intensity events, 2) incorporating more weather-related features, and 3) the uncertainty of the loss functions. Furthermore, transfer learning could be considered to study the generalization of the proposed method to regions with different rainfall characteristics.
Acknowledgments.
Open-source Python code bases, “Skillful Precipitation Nowcasting using Deep Generative Models of Radar” by DeepMind (https://github.com/google-deepmind/deepmind-research/tree/master/nowcasting) and “Nowcasting Precipitation using GANs” by Koert Schreurs (https://github.com/KoertS/precipitation-nowcasting-using-GANs), were used as the original models for the modifications with extensions (extended balanced loss and temperature data).
Data availability statement.
Real-time precipitation radar data were used from 2008 till 2021. The 1.5-m air temperature data were retrieved from 45 automatic weather stations. The data from 2018 onwards are publicly available on the KNMI data platform (https://dataplatform.knmi.nl). This dataset was used to train and validate the model.
APPENDIX A
Robustness of the Results to the Choice of the Test Set
This paper used data from the year 2021 as the test data for verifying the different models. The year 2021 contained more rainfall compared to some previous years. Especially, the months of July, August, and November captured most of the high-intensity events of that year. While the number of rainfall days was similar for 2021 compared to other years, the number of days with high-intensity precipitation was higher. Thus, using the extended balanced loss and 2021 as a test set can lead to biasing the results toward the high-intensity events. For that reason, the DGMR experiments are performed with two different test sets, one for the part of 2021 with more high-intensity events (July, August, October, and November; “Test set—Heavy”) and the other set containing the remaining months (“Test set—Light”).
Increasing the number of high-intensity events in the test data increases the number of high rainfall intensities that are correctly predicted (see Tables A1–A3). Overall, the DGMR using Test set—Heavy performs better in terms of CSI and POD, but the models show also a higher FAR (which should be lower for better performance), especially for the threshold of 20 mm h−1 and using the extended balanced loss.
CSI for DGMR using different extensions and test sets. Aggregation over all lead times is used.
POD for DGMR using different extensions and test sets. Aggregation over all lead times is used.
FAR for DGMR using different extensions and test sets. Aggregation over all lead times is used.
APPENDIX B
Training
Models were trained and evaluated on the 7 GPU nodes. See Table B1 for an overview of the training times for the deep learning models (without subset extractions).
Training time of AENN GAN and DGMR, using seven GPUs (time in h).
Hyperparameters settings are optimized using grid search. Table B2 presents the hyperparameter setting of the DGMR used in Ravuri et al. (2021) and the setting used in this paper after performing a grid search. The optimal chosen hyperparameters set is the set that could gain the highest performance on the validation set.
Hyperparameter settings for DGMR with both extensions, for the default case (DGMR; Ravuri et al. 2021) and settings used after applying grid search.
APPENDIX C
Comparison of Loss Functions
Figure C1 illustrates the generator, discriminator, and reconstruction losses of the DGMR (with both extensions). We observe that the training stabilizes around 125 epochs.
Performance of DGMR with both extensions, measured with three different loss functions.
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
APPENDIX D
Prediction Frames
Figures D1 and D2 present additional examples of prediction frames.
Nowcast and target value for different models, for a rainy event, initialized at t0 =1900:00 LT 16 Feb 2021 (mm h−1).
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
Nowcast and target value for different models, for a rainy event, initialized at t0 = 1000:00 LT 25 Feb 2021 (mm h−1).
Citation: Artificial Intelligence for the Earth Systems 2, 4; 10.1175/AIES-D-23-0017.1
REFERENCES
Abadi, M., and Coauthors, 2016: Tensorflow: A system for large-scale machine learning. Proc. 12th USENIX Symp. on Operating Systems Design and Implementation (OSDI’16), Savannah, GA, USENIX Association, 265–283, https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf.
Agrawal, S., L. Barrington, C. Bromberg, J. Burge, C. Gazen, and J. Hickey, 2019: Machine learning for precipitation nowcasting from radar images. arXiv, 1912.12132v1, https://doi.org/10.48550/arXiv.1912.12132.
Attema, J., and Coauthors, 2014: KNMI’14: Climate change scenarios for the 21st century—A Netherlands perspective. KNMI Scientific Rep. WR 2014-01, 120 pp.
Ayzel, G., T. Scheffer, and M. Heistermann, 2020: RainNet v1. 0: A convolutional neural network for radar-based precipitation nowcasting. Geosci. Model Dev., 13, 2631–2644, https://doi.org/10.5194/gmd-13-2631-2020.
Bauer, P., A. Thorpe, and G. Brunet, 2015: The quiet revolution of numerical weather prediction. Nature, 525, 47–55, https://doi.org/10.1038/nature14956.
Bouget, V., D. Béréziat, J. Brajard, A. A. Charantonis, and A. Filoche, 2021: Fusion of rain radar images and wind forecasts in a deep learning model applied to rain nowcasting. Remote Sens., 13, 246, https://doi.org/10.3390/rs13020246.
Changnon, S. A., 1996: Effects of summer precipitation on urban transportation. Climatic Change, 32, 481–494, https://doi.org/10.1007/BF00140357.
Choi, S., and Y. Kim, 2021: Rad-cGAN v1.0: Radar-based precipitation nowcasting model with conditional generative adversarial networks for multiple domains. Geosci. Model Dev., 15, 5967–5985, https://doi.org/10.5194/gmd-15-5967-2022.
De Luca, D. L., 2013: Rainfall Nowcasting Models for Early Warning Systems. Nova Science Publishers, 196 pp.
Falk, T., and Coauthors, 2019: U-Net: Deep learning for cell counting, detection, and morphometry. Nat. Methods, 16, 67–70, https://doi.org/10.1038/s41592-018-0261-2.
Fu, T.-C., 2011: A review on time series data mining. Eng. Appl. Artif. Intell., 24, 164–181, https://doi.org/10.1016/j.engappai.2010.09.007.
Germann, U., and I. Zawadzki, 2002: Scale-dependence of the predictability of precipitation from continental radar images. Part I: Description of the methodology. Mon. Wea. Rev., 130, 2859–2873, https://doi.org/10.1175/1520-0493(2002)130<2859:SDOTPO>2.0.CO;2.
Goodfellow, I., J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, 2014: Generative adversarial networks. NIPS’14: Proceedings of the 27th International Conference on Neural Information Processing Systems, MIT Press, 2672–2680, https://dl.acm.org/doi/10.5555/2969033.2969125.
Han, L., Y. Zhao, H. Chen, and V. Chandrasekar, 2021: Advancing radar nowcasting through deep transfer learning. IEEE Trans. Geosci. Remote Sens., 60, 4100609, https://doi.org/10.1109/TGRS.2021.3056470.
Hayatbini, N., and Coauthors, 2019: Conditional generative adversarial networks (cGANs) for near real-time precipitation estimation from multispectral GOES-16 satellite imageries—PERSIANN-cGAN. Remote Sens., 11, 2193, https://doi.org/10.3390/rs11192193.
Herruzo, P., A. Gruca, L. Lliso, X. Calbet, P. Rípodas, S. Hochreiter, M. Kopp, and D. P. Kreil, 2021: High-resolution multi-channel weather forecasting—First insights on transfer learning from the Weather4cast competitions 2021. 2021 IEEE Int. Conf. on Big Data (Big Data), Orlando, FL, IEEE, 5750–5757, https://doi.org/10.1109/BigData52589.2021.9672063.
Imhoff, R. O., C. C. Brauer, A. Overeem, A. H. Weerts, and R. Uijlenhoet, 2020: Spatial and temporal evaluation of radar rainfall nowcasting techniques on 1,533 events. Water Resour. Res., 56, e2019WR026723, https://doi.org/10.1029/2019WR026723.
Imhoff, R. O., C. C. Brauer, K. J. van Heeringen, R. Uijlenhoet, and A. H. Weerts, 2022: Large-sample evaluation of radar rainfall nowcasting for flood early warning. Water Resour. Res., 58, e2021WR031591, https://doi.org/10.1029/2021WR031591.
Ingram, K. T., M. C. Roncoli, and P. H. Kirshen, 2002: Opportunities and constraints for farmers of West Africa to use seasonal precipitation forecasts with Burkina Faso as a case study. Agric. Syst., 74, 331–349, https://doi.org/10.1016/S0308-521X(02)00044-6.
Jing, J. R., Q. Li, X. Y. Ding, N. L. Sun, R. Tang, and Y. L. Cai, 2019: AENN: A generative adversarial neural network for weather radar echo extrapolation. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., 42, 89–94, https://doi.org/10.5194/isprs-archives-XLII-3-W9-89-2019.
Khorrami, P., O. Simek, B. Cheung, M. Veillette, R. Dangovski, I. Rugina, M. Soljacic, and P. Agrawal, 2021: Adapting deep learning models to new meteorological contexts using transfer learning. 2021 IEEE Int. Conf. on Big Data (Big Data), Orlando, FL, IEEE, 4169–4177, https://doi.org/10.1109/BigData52589.2021.9671451.
Kulesa, G., 2003: Weather and aviation: How does weather affect the safety and operations of airports and aviation, and how does FAA work to manage weather-related effects? The Potential Impacts of Climate Change on Transportation: Workshop Summary and Proceedings, Transportation Research Board, 1–10, https://trid.trb.org/view/663829.
Lu, G. Y., and D. W. Wong, 2008: An adaptive inverse-distance weighting spatial interpolation technique. Comput. Geosci., 34, 1044–1055, https://doi.org/10.1016/j.cageo.2007.07.010.
Lucas, B. D., and T. Kanade, 1981: An iterative image registration technique with an application to stereo vision. IJCAI’81: Proceedings of the 7th International Joint Conference on Artificial Intelligence, Morgan Kaufmann, 674–679, https://dl.acm.org/doi/10.5555/1623264.1623280.
Lundberg, S. M., and S.-I. Lee, 2017: A unified approach to interpreting model predictions. NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., 4768–4777, https://dl.acm.org/doi/10.5555/3295222.3295230.
Marshall, J. S., and W. M. K. Palmer, 1948: The distribution of raindrops with size. J. Meteor., 5, 165–166, https://doi.org/10.1175/1520-0469(1948)005<0165:TDORWS>2.0.CO;2.
Miyato, T., T. Kataoka, M. Koyama, and Y. Yoshida, 2018: Spectral normalization for generative adversarial networks. arXiv, 1802.05957v1, https://doi.org/10.48550/arXiv.1802.05957.
Molnar, C., 2022: Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. 2nd ed. Molnar, 318 pp.
Nie, T., K. Deng, C. Shao, C. Zhao, K. Ren, and J. Song, 2021: Self-attention UNet model for radar based precipitation nowcasting. 2021 IEEE Sixth Int. Conf. on Data Science in Cyberspace (DSC), Shenzhen, China, IEEE, 493–499, https://doi.org/10.1109/DSC53577.2021.00079.
Ravuri, S., and Coauthors, 2021: Skilful precipitation nowcasting using deep generative models of radar. Nature, 597, 672–677, https://doi.org/10.1038/s41586-021-03854-z.
Ronneberger, O., P. Fischer, and T. Brox, 2015: U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, N. Navab et al., Eds., Springer International Publishing, 234–241, https://doi.org/10.1007/978-3-319-24574-4_28.
Schreurs, K., 2021: Precipitation nowcasting using generative adversarial networks. M.S. thesis, Dept. of Data Science, Radboud University, 55 pp.
Seed, A. W., 2003: A dynamic and spatial scaling approach to advection forecasting. J. Appl. Meteor., 42, 381–388, https://doi.org/10.1175/1520-0450(2003)042<0381:ADASSA>2.0.CO;2.
Senouci, A., M. Al-Abbasi, and N. N. Eldin, 2018: Impact of weather conditions on construction labour productivity in Qatar. Middle East J. Manage., 5, 34–49, https://doi.org/10.1504/MEJM.2018.088734.
Seoh, R., 2020: Qualitative analysis of Monte Carlo dropout. arXiv, 2007.01720v1, https://doi.org/10.48550/arXiv.2007.01720.
Shi, X., Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-C. Woo, 2015: Convolutional LSTM network: A machine learning approach for precipitation nowcasting. NIPS’15: Proceedings of the 28th International Conference on Neural Information Processing Systems, MIT Press, 802–810, https://dl.acm.org/doi/10.5555/2969239.2969329.
Shi, X., Z. Gao, L. Lausen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-C. Woo, 2017: Deep learning for precipitation nowcasting: A benchmark and a new model. NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., 5622–5632, https://dl.acm.org/doi/10.5555/3295222.3295313.
Shorten, C., and T. M. Khoshgoftaar, 2019: A survey on image data augmentation for deep learning. J. Big Data, 6, 60, https://doi.org/10.1186/s40537-019-0197-0.
Sønderby, C. K., and Coauthors, 2020: MetNet: A neural weather model for precipitation forecasting. arXiv, 2003.12140v2, https://doi.org/10.48550/arXiv.2003.12140.
Sutskever, I., O. Vinyals, and Q. V. Le, 2014: Sequence to sequence learning with neural networks. NIPS’14: Proceedings of the 27th International Conference on Neural Information Processing Systems, MIT Press, 3104–3112, https://dl.acm.org/doi/10.5555/2969033.2969173.
Thirugnanam, H., M. V. Ramesh, and V. P. Rangan, 2020: Enhancing the reliability of landslide early warning systems by machine learning. Landslides, 17, 2231–2246, https://doi.org/10.1007/s10346-020-01453-z.
Tian, L., X. Li, Y. Ye, P. Xie, and Y. Li, 2019: A generative adversarial gated recurrent unit model for precipitation nowcasting. IEEE Geosci. Remote Sens. Lett., 17, 601–605, https://doi.org/10.1109/LGRS.2019.2926776.
Tran, Q.-K., and S.-K. Song, 2019: Computer vision in precipitation nowcasting: Applying image quality assessment metrics for training deep neural networks. Atmosphere, 10, 244, https://doi.org/10.3390/atmos10050244.
Trebing, K., T. Staǹczyk, and S. Mehrkanoon, 2021: SmaAt-UNet: Precipitation nowcasting using a small attention-UNet architecture. Pattern Recognit. Lett., 145, 178–186, https://doi.org/10.1016/j.patrec.2021.01.036.
van der Kooij, E., 2021: Nowcasting heavy precipitation in the Netherlands: A deep learning approach. M.S. thesis, Dept. of Science, Delft University of Technology, 63 pp.
Xu, W., 2013: Precipitation and convective characteristics of summer deep convection over East Asia observed by TRMM. Mon. Wea. Rev., 141, 1577–1592, https://doi.org/10.1175/MWR-D-12-00177.1.
