Browse
Abstract
Seasonal hypoxia is a recurring threat to ecosystems and fisheries in the Chesapeake Bay. Hypoxia forecasting based on coupled hydrodynamic and biogeochemical models has proven useful for many stakeholders, as these models excel in accounting for the effects of physical forcing on oxygen supply, but may fall short in replicating the more complex biogeochemical processes that govern oxygen consumption. Satellite-derived reflectances could be used to indicate the presence of surface organic matter over the Bay. However, teasing apart the contribution of atmospheric and aquatic constituents from the signal received by the satellite is not straightforward. As a result, it is difficult to derive surface concentrations of organic matter from satellite data in a robust fashion. A potential solution to this complexity is to use deep learning to build end-to-end applications that do not require precise accounting of the satellite signal from the atmosphere or water, phytoplankton blooms, or sediment plumes. By training a deep neural network with data from a vast suite of variables that could potentially affect oxygen in the water column, improvement of short-term (daily) hypoxia forecast may be possible. Here, we predict oxygen concentrations using inputs that account for both physical and biogeochemical factors. The physical inputs include wind velocity reanalysis information, together with 3D outputs from an estuarine hydrodynamic model, including current velocity, water temperature, and salinity. Satellite-derived spectral reflectance data are used as a surrogate for the biogeochemical factors. These input fields are time series of weekly statistics calculated from daily information, starting 8 weeks before each oxygen observation was collected. To accommodate this input data structure, we adopted a model architecture of long short-term memory networks with eight time steps. At each time step, a set of convolutional neural networks are used to extract information from the inputs. Ablation and cross-validation tests suggest that among all input features, the strongest predictor is the 3D temperature field, with which the new model can outperform the state-of-the-art by ∼20% in terms of median absolute error. Our approach represents a novel application of deep learning to address a complex water management challenge.
Significance Statement
This study presents a novel approach that combines deep learning and hydrodynamic model outputs to improve the accuracy of hypoxia forecasts in the Chesapeake Bay. By training a deep neural network with both physical and biogeochemical information as input features, the model accurately predicts oxygen concentration at any depth in the water column 1 day in advance. This approach has the potential to benefit stakeholders and inform adaptation measures during the recurring threat of hypoxia in the Chesapeake Bay. The success of this study suggests the potential for similar applications of deep learning to address complex water management challenges. Further research could investigate the application of this approach to different forecast lead times and other regions and ecosystem types.
Abstract
Seasonal hypoxia is a recurring threat to ecosystems and fisheries in the Chesapeake Bay. Hypoxia forecasting based on coupled hydrodynamic and biogeochemical models has proven useful for many stakeholders, as these models excel in accounting for the effects of physical forcing on oxygen supply, but may fall short in replicating the more complex biogeochemical processes that govern oxygen consumption. Satellite-derived reflectances could be used to indicate the presence of surface organic matter over the Bay. However, teasing apart the contribution of atmospheric and aquatic constituents from the signal received by the satellite is not straightforward. As a result, it is difficult to derive surface concentrations of organic matter from satellite data in a robust fashion. A potential solution to this complexity is to use deep learning to build end-to-end applications that do not require precise accounting of the satellite signal from the atmosphere or water, phytoplankton blooms, or sediment plumes. By training a deep neural network with data from a vast suite of variables that could potentially affect oxygen in the water column, improvement of short-term (daily) hypoxia forecast may be possible. Here, we predict oxygen concentrations using inputs that account for both physical and biogeochemical factors. The physical inputs include wind velocity reanalysis information, together with 3D outputs from an estuarine hydrodynamic model, including current velocity, water temperature, and salinity. Satellite-derived spectral reflectance data are used as a surrogate for the biogeochemical factors. These input fields are time series of weekly statistics calculated from daily information, starting 8 weeks before each oxygen observation was collected. To accommodate this input data structure, we adopted a model architecture of long short-term memory networks with eight time steps. At each time step, a set of convolutional neural networks are used to extract information from the inputs. Ablation and cross-validation tests suggest that among all input features, the strongest predictor is the 3D temperature field, with which the new model can outperform the state-of-the-art by ∼20% in terms of median absolute error. Our approach represents a novel application of deep learning to address a complex water management challenge.
Significance Statement
This study presents a novel approach that combines deep learning and hydrodynamic model outputs to improve the accuracy of hypoxia forecasts in the Chesapeake Bay. By training a deep neural network with both physical and biogeochemical information as input features, the model accurately predicts oxygen concentration at any depth in the water column 1 day in advance. This approach has the potential to benefit stakeholders and inform adaptation measures during the recurring threat of hypoxia in the Chesapeake Bay. The success of this study suggests the potential for similar applications of deep learning to address complex water management challenges. Further research could investigate the application of this approach to different forecast lead times and other regions and ecosystem types.
Abstract
Explainable artificial intelligence (XAI) methods shed light on the predictions of machine learning algorithms. Several different approaches exist and have already been applied in climate science. However, usually missing ground truth explanations complicate their evaluation and comparison, subsequently impeding the choice of the XAI method. Therefore, in this work, we introduce XAI evaluation in the climate context and discuss different desired explanation properties, namely, robustness, faithfulness, randomization, complexity, and localization. To this end, we chose previous work as a case study where the decade of annual-mean temperature maps is predicted. After training both a multilayer perceptron (MLP) and a convolutional neural network (CNN), multiple XAI methods are applied and their skill scores in reference to a random uniform explanation are calculated for each property. Independent of the network, we find that XAI methods such as Integrated Gradients, layerwise relevance propagation, and input times gradients exhibit considerable robustness, faithfulness, and complexity while sacrificing randomization performance. Sensitivity methods, gradient, SmoothGrad, NoiseGrad, and FusionGrad, match the robustness skill but sacrifice faithfulness and complexity for the randomization skill. We find architecture-dependent performance differences regarding robustness, complexity, and localization skills of different XAI methods, highlighting the necessity for research task-specific evaluation. Overall, our work offers an overview of different evaluation properties in the climate science context and shows how to compare and benchmark different explanation methods, assessing their suitability based on strengths and weaknesses, for the specific research problem at hand. By that, we aim to support climate researchers in the selection of a suitable XAI method.
Significance Statement
Explainable artificial intelligence (XAI) helps to understand the reasoning behind the prediction of a neural network. XAI methods have been applied in climate science to validate networks and provide new insight into physical processes. However, the increasing number of XAI methods can overwhelm practitioners, making it difficult to choose an explanation method. Since XAI methods’ results can vary, uninformed choices might cause misleading conclusions about the network decision. In this work, we introduce XAI evaluation to compare and assess the performance of explanation methods based on five desirable properties. We demonstrate that XAI evaluation reveals the strengths and weaknesses of different XAI methods. Thus, our work provides climate researchers with the tools to compare, analyze, and subsequently choose explanation methods.
Abstract
Explainable artificial intelligence (XAI) methods shed light on the predictions of machine learning algorithms. Several different approaches exist and have already been applied in climate science. However, usually missing ground truth explanations complicate their evaluation and comparison, subsequently impeding the choice of the XAI method. Therefore, in this work, we introduce XAI evaluation in the climate context and discuss different desired explanation properties, namely, robustness, faithfulness, randomization, complexity, and localization. To this end, we chose previous work as a case study where the decade of annual-mean temperature maps is predicted. After training both a multilayer perceptron (MLP) and a convolutional neural network (CNN), multiple XAI methods are applied and their skill scores in reference to a random uniform explanation are calculated for each property. Independent of the network, we find that XAI methods such as Integrated Gradients, layerwise relevance propagation, and input times gradients exhibit considerable robustness, faithfulness, and complexity while sacrificing randomization performance. Sensitivity methods, gradient, SmoothGrad, NoiseGrad, and FusionGrad, match the robustness skill but sacrifice faithfulness and complexity for the randomization skill. We find architecture-dependent performance differences regarding robustness, complexity, and localization skills of different XAI methods, highlighting the necessity for research task-specific evaluation. Overall, our work offers an overview of different evaluation properties in the climate science context and shows how to compare and benchmark different explanation methods, assessing their suitability based on strengths and weaknesses, for the specific research problem at hand. By that, we aim to support climate researchers in the selection of a suitable XAI method.
Significance Statement
Explainable artificial intelligence (XAI) helps to understand the reasoning behind the prediction of a neural network. XAI methods have been applied in climate science to validate networks and provide new insight into physical processes. However, the increasing number of XAI methods can overwhelm practitioners, making it difficult to choose an explanation method. Since XAI methods’ results can vary, uninformed choices might cause misleading conclusions about the network decision. In this work, we introduce XAI evaluation to compare and assess the performance of explanation methods based on five desirable properties. We demonstrate that XAI evaluation reveals the strengths and weaknesses of different XAI methods. Thus, our work provides climate researchers with the tools to compare, analyze, and subsequently choose explanation methods.
Abstract
Wind gusts are often associated with severe hazards and can cause structural and environmental damages, making gust prediction a crucial element of weather forecasting services. In this study, we explored the utilization of machine learning (ML) algorithms integrated with numerical weather prediction outputs from the Weather Research and Forecasting (WRF) Model, to align the estimation of wind gust potential with observed gusts. We have used two ML algorithms, namely, random forest (RF) and extreme gradient boosting (XGB), along with two statistical techniques: generalized linear model with identity link function (GLM-Identity) and generalized linear model with log link function (GLM-Log), to predict storm wind gusts for the northeast (NE) United States. We used 61 simulated extratropical and tropical storms that occurred between 2005 and 2020 to develop and validate the ML and statistical models. To assess the ML model performance, we compared our results with postprocessed gust potential from WRF. Our findings showed that ML models, especially XGB, performed significantly better than statistical models and Unified Post Processor for the WRF (WRF-UPP) Model and were able to better align predicted with observed gusts across all storms. The ML models faced challenges capturing the upper tail of the gust distribution, and the learning curves suggested that XGB was more effective than RF in generating better predictions with fewer storms.
Abstract
Wind gusts are often associated with severe hazards and can cause structural and environmental damages, making gust prediction a crucial element of weather forecasting services. In this study, we explored the utilization of machine learning (ML) algorithms integrated with numerical weather prediction outputs from the Weather Research and Forecasting (WRF) Model, to align the estimation of wind gust potential with observed gusts. We have used two ML algorithms, namely, random forest (RF) and extreme gradient boosting (XGB), along with two statistical techniques: generalized linear model with identity link function (GLM-Identity) and generalized linear model with log link function (GLM-Log), to predict storm wind gusts for the northeast (NE) United States. We used 61 simulated extratropical and tropical storms that occurred between 2005 and 2020 to develop and validate the ML and statistical models. To assess the ML model performance, we compared our results with postprocessed gust potential from WRF. Our findings showed that ML models, especially XGB, performed significantly better than statistical models and Unified Post Processor for the WRF (WRF-UPP) Model and were able to better align predicted with observed gusts across all storms. The ML models faced challenges capturing the upper tail of the gust distribution, and the learning curves suggested that XGB was more effective than RF in generating better predictions with fewer storms.
Abstract
Detection and tracking of tropical cyclones (TCs) in numerical weather prediction model outputs is essential for many applications, such as forecast guidance and real-time monitoring of events. While this task has been automated in the 1990s with heuristic models, relying on a set of empirical rules and thresholds, the recent success of machine learning methods to detect objects in images opens new perspectives. This paper introduces and evaluates the capacity of a convolutional neural network based on the U-Net architecture to detect the TC wind structure, including maximum wind speed area and hurricane-force wind speed area, in the outputs of the convective-scale AROME model. A dataset of 400 AROME forecasts over the West Indies domain has been entirely hand-labeled by experts, following a rigorous process to reduce heterogeneities. The U-Net performs well on a wide variety of TC intensities and shapes, with an average intersection-over-union metric of around 0.8. Its performances, however, strongly depend on the TC strength, and the detection of weak cyclones is more challenging since their structure is less well defined. The U-Net also significantly outperforms an operational heuristic detection model, with a significant gain for weak TCs, while running much faster. In the last part, the capacity of the U-Net to generalize on slightly different data is demonstrated in the context of a domain change and a resolution increase. In both cases, the pretrained U-Net achieves similar performances as the original dataset.
Abstract
Detection and tracking of tropical cyclones (TCs) in numerical weather prediction model outputs is essential for many applications, such as forecast guidance and real-time monitoring of events. While this task has been automated in the 1990s with heuristic models, relying on a set of empirical rules and thresholds, the recent success of machine learning methods to detect objects in images opens new perspectives. This paper introduces and evaluates the capacity of a convolutional neural network based on the U-Net architecture to detect the TC wind structure, including maximum wind speed area and hurricane-force wind speed area, in the outputs of the convective-scale AROME model. A dataset of 400 AROME forecasts over the West Indies domain has been entirely hand-labeled by experts, following a rigorous process to reduce heterogeneities. The U-Net performs well on a wide variety of TC intensities and shapes, with an average intersection-over-union metric of around 0.8. Its performances, however, strongly depend on the TC strength, and the detection of weak cyclones is more challenging since their structure is less well defined. The U-Net also significantly outperforms an operational heuristic detection model, with a significant gain for weak TCs, while running much faster. In the last part, the capacity of the U-Net to generalize on slightly different data is demonstrated in the context of a domain change and a resolution increase. In both cases, the pretrained U-Net achieves similar performances as the original dataset.
Abstract
CloudSat’s Cloud Profiling Radar is a valuable tool for remotely monitoring high-latitude snowfall, but its ability to observe hydrometeor activity near the Earth’s surface is limited by a radar blind zone caused by ground clutter contamination. This study presents the development of a deeply supervised U-Net-style convolutional neural network to predict cold season reflectivity profiles within the blind zone at two Arctic locations. The network learns to predict the presence and intensity of near-surface hydrometeors by coupling latent features encoded in blind zone-aloft clouds with additional context from collocated atmospheric state variables (i.e., temperature, specific humidity, and wind speed). Results show that the U-Net predictions outperform traditional linear extrapolation methods, with low mean absolute error, a 38% higher Sørensen–Dice coefficient, and vertical reflectivity distributions 60% closer to observed values. The U-Net is also able to detect the presence of near-surface cloud with a critical success index (CSI) of 72% and cases of shallow cumuliform snowfall and virga with 18% higher CSI values compared to linear methods. An explainability analysis shows that reflectivity information throughout the scene, especially at cloud edges and at the 1.2-km blind zone threshold, along with atmospheric state variables near the tropopause, are the most significant contributors to model skill. This surface-trained generative inpainting technique has the potential to enhance current and future remote sensing precipitation missions by providing a better understanding of the nonlinear relationship between blind zone reflectivity values and the surrounding atmospheric state.
Significance Statement
Snowfall is a critical contributor to the global water–energy budget, with important connections to water resource management, flood mitigation, and ecosystem sustainability. However, traditional spaceborne remote monitoring of snowfall faces challenges due to a near-surface radar blind zone, which masks a portion of the atmosphere. In this study, a deep learning model was developed to fill in missing data across these regions using surface radar and atmospheric state variables. The model accurately predicts reflectivity, with significant improvements over conventional methods. This innovative approach enhances our understanding of reflectivity patterns and atmospheric interactions, bolstering advances in remote snowfall prediction.
Abstract
CloudSat’s Cloud Profiling Radar is a valuable tool for remotely monitoring high-latitude snowfall, but its ability to observe hydrometeor activity near the Earth’s surface is limited by a radar blind zone caused by ground clutter contamination. This study presents the development of a deeply supervised U-Net-style convolutional neural network to predict cold season reflectivity profiles within the blind zone at two Arctic locations. The network learns to predict the presence and intensity of near-surface hydrometeors by coupling latent features encoded in blind zone-aloft clouds with additional context from collocated atmospheric state variables (i.e., temperature, specific humidity, and wind speed). Results show that the U-Net predictions outperform traditional linear extrapolation methods, with low mean absolute error, a 38% higher Sørensen–Dice coefficient, and vertical reflectivity distributions 60% closer to observed values. The U-Net is also able to detect the presence of near-surface cloud with a critical success index (CSI) of 72% and cases of shallow cumuliform snowfall and virga with 18% higher CSI values compared to linear methods. An explainability analysis shows that reflectivity information throughout the scene, especially at cloud edges and at the 1.2-km blind zone threshold, along with atmospheric state variables near the tropopause, are the most significant contributors to model skill. This surface-trained generative inpainting technique has the potential to enhance current and future remote sensing precipitation missions by providing a better understanding of the nonlinear relationship between blind zone reflectivity values and the surrounding atmospheric state.
Significance Statement
Snowfall is a critical contributor to the global water–energy budget, with important connections to water resource management, flood mitigation, and ecosystem sustainability. However, traditional spaceborne remote monitoring of snowfall faces challenges due to a near-surface radar blind zone, which masks a portion of the atmosphere. In this study, a deep learning model was developed to fill in missing data across these regions using surface radar and atmospheric state variables. The model accurately predicts reflectivity, with significant improvements over conventional methods. This innovative approach enhances our understanding of reflectivity patterns and atmospheric interactions, bolstering advances in remote snowfall prediction.
Abstract
We present a method to downscale idealized geophysical fluid simulations using generative models based on diffusion maps. By analyzing the Fourier spectra of fields drawn from different data distributions, we show how a diffusion bridge can be used as a transformation between a low-resolution and a high-resolution dataset, allowing for new sample generation of high-resolution fields given specific low-resolution features. The ability to generate new samples allows for the computation of any statistic of interest, without any additional calibration or training. Our unsupervised setup is also designed to downscale fields without access to paired training data; this flexibility allows for the combination of multiple source and target domains without additional training. We demonstrate that the method enhances resolution and corrects context-dependent biases in geophysical fluid simulations, including in extreme events. We anticipate that the same method can be used to downscale the output of climate simulations, including temperature and precipitation fields, without needing to train a new model for each application and providing a significant computational cost savings.
Significance Statement
The purpose of this study is to apply recent advances in generative machine learning technologies to obtain higher-resolution geophysical fluid dynamics model output at lower cost compared with direct simulation while preserving important statistical properties of the high-resolution data. This is important because while high-resolution climate model output is required by many applications, it is also computationally expensive to obtain.
Abstract
We present a method to downscale idealized geophysical fluid simulations using generative models based on diffusion maps. By analyzing the Fourier spectra of fields drawn from different data distributions, we show how a diffusion bridge can be used as a transformation between a low-resolution and a high-resolution dataset, allowing for new sample generation of high-resolution fields given specific low-resolution features. The ability to generate new samples allows for the computation of any statistic of interest, without any additional calibration or training. Our unsupervised setup is also designed to downscale fields without access to paired training data; this flexibility allows for the combination of multiple source and target domains without additional training. We demonstrate that the method enhances resolution and corrects context-dependent biases in geophysical fluid simulations, including in extreme events. We anticipate that the same method can be used to downscale the output of climate simulations, including temperature and precipitation fields, without needing to train a new model for each application and providing a significant computational cost savings.
Significance Statement
The purpose of this study is to apply recent advances in generative machine learning technologies to obtain higher-resolution geophysical fluid dynamics model output at lower cost compared with direct simulation while preserving important statistical properties of the high-resolution data. This is important because while high-resolution climate model output is required by many applications, it is also computationally expensive to obtain.
Abstract
The different phases of the Coupled Model Intercomparison Project (CMIP) provide ensembles of past, present, and future climate simulations crucial for climate change impact and adaptation activities. These ensembles are produced using multiple global climate models (GCMs) from different modeling centers with some shared building blocks and interdependencies. Applications typically follow the “model democracy” approach which might have significant implications in the resulting products (e.g., large bias and low spread). Thus, quantifying model similarity within ensembles is crucial for interpreting model agreement and multimodel uncertainty in climate change studies. The classical methods used for assessing GCM similarity can be classified into two groups. The a priori approach relies on expert knowledge about the components of these models, while the a posteriori approach seeks similarity in the GCMs’ output variables and is thus data-driven. In this study, we apply probabilistic network models (PNMs), a well-established machine learning technique, as a new a posteriori method to measure intermodel similarities. The proposed methodology is applied to surface temperature fields of the historical experiments from the CMIP5 multimodel ensemble and different reanalysis gridded datasets. PNMs are able to learn the complex spatial dependency structures present in climate data, including teleconnections operating on multiple spatial scales, characteristic of the underlying GCM. A distance metric building on the resulting PNMs is applied to characterize GCM model dependencies. The results of this approach are in line with those obtained with more traditional methods but have further explanatory potential building on probabilistic model querying.
Significance Statement
The present study proposes the use of probabilistic network models (PNMs) to quantify model similarity within ensembles of global climate models (GCMs). This is crucial for interpreting model agreement and multimodel uncertainty in climate change studies. When applied to climate data (gridded global surface temperature in this study), PNMs encode the relevant spatial dependencies (local and remote connections). Similarities among the PNMs resulting from different GCMs can be quantified and are shown to capture similar GCM formulations reported in previous studies. Differently to other machine learning methods previously applied to this problem, PNMs are fully explainable (allowing probabilistic querying) and are applicable to high-dimensional gridded raw data.
Abstract
The different phases of the Coupled Model Intercomparison Project (CMIP) provide ensembles of past, present, and future climate simulations crucial for climate change impact and adaptation activities. These ensembles are produced using multiple global climate models (GCMs) from different modeling centers with some shared building blocks and interdependencies. Applications typically follow the “model democracy” approach which might have significant implications in the resulting products (e.g., large bias and low spread). Thus, quantifying model similarity within ensembles is crucial for interpreting model agreement and multimodel uncertainty in climate change studies. The classical methods used for assessing GCM similarity can be classified into two groups. The a priori approach relies on expert knowledge about the components of these models, while the a posteriori approach seeks similarity in the GCMs’ output variables and is thus data-driven. In this study, we apply probabilistic network models (PNMs), a well-established machine learning technique, as a new a posteriori method to measure intermodel similarities. The proposed methodology is applied to surface temperature fields of the historical experiments from the CMIP5 multimodel ensemble and different reanalysis gridded datasets. PNMs are able to learn the complex spatial dependency structures present in climate data, including teleconnections operating on multiple spatial scales, characteristic of the underlying GCM. A distance metric building on the resulting PNMs is applied to characterize GCM model dependencies. The results of this approach are in line with those obtained with more traditional methods but have further explanatory potential building on probabilistic model querying.
Significance Statement
The present study proposes the use of probabilistic network models (PNMs) to quantify model similarity within ensembles of global climate models (GCMs). This is crucial for interpreting model agreement and multimodel uncertainty in climate change studies. When applied to climate data (gridded global surface temperature in this study), PNMs encode the relevant spatial dependencies (local and remote connections). Similarities among the PNMs resulting from different GCMs can be quantified and are shown to capture similar GCM formulations reported in previous studies. Differently to other machine learning methods previously applied to this problem, PNMs are fully explainable (allowing probabilistic querying) and are applicable to high-dimensional gridded raw data.
Abstract
An ensemble postprocessing method is developed for the probabilistic prediction of severe weather (tornadoes, hail, and wind gusts) over the conterminous United States (CONUS). The method combines conditional generative adversarial networks (CGANs), a type of deep generative model, with a convolutional neural network (CNN) to postprocess convection-allowing model (CAM) forecasts. The CGANs are designed to create synthetic ensemble members from deterministic CAM forecasts, and their outputs are processed by the CNN to estimate the probability of severe weather. The method is tested using High-Resolution Rapid Refresh (HRRR) 1–24-h forecasts as inputs and Storm Prediction Center (SPC) severe weather reports as targets. The method produced skillful predictions with up to 20% Brier skill score (BSS) increases compared to other neural-network-based reference methods using a testing dataset of HRRR forecasts in 2021. For the evaluation of uncertainty quantification, the method is overconfident but produces meaningful ensemble spreads that can distinguish good and bad forecasts. The quality of CGAN outputs is also evaluated. Results show that the CGAN outputs behave similarly to a numerical ensemble; they preserved the intervariable correlations and the contribution of influential predictors as in the original HRRR forecasts. This work provides a novel approach to postprocess CAM output using neural networks that can be applied to severe weather prediction.
Significance Statement
We use a new machine learning (ML) technique to generate probabilistic forecasts of convective weather hazards, such as tornadoes and hailstorms, with the output from high-resolution numerical weather model forecasts. The new ML system generates an ensemble of synthetic forecast fields from a single forecast, which are then used to train ML models for convective hazard prediction. Using this ML-generated ensemble for training leads to improvements of 10%–20% in severe weather forecast skills compared to using other ML algorithms that use only output from the single forecast. This work is unique in that it explores the use of ML methods for producing synthetic forecasts of convective storm events and using these to train ML systems for high-impact convective weather prediction.
Abstract
An ensemble postprocessing method is developed for the probabilistic prediction of severe weather (tornadoes, hail, and wind gusts) over the conterminous United States (CONUS). The method combines conditional generative adversarial networks (CGANs), a type of deep generative model, with a convolutional neural network (CNN) to postprocess convection-allowing model (CAM) forecasts. The CGANs are designed to create synthetic ensemble members from deterministic CAM forecasts, and their outputs are processed by the CNN to estimate the probability of severe weather. The method is tested using High-Resolution Rapid Refresh (HRRR) 1–24-h forecasts as inputs and Storm Prediction Center (SPC) severe weather reports as targets. The method produced skillful predictions with up to 20% Brier skill score (BSS) increases compared to other neural-network-based reference methods using a testing dataset of HRRR forecasts in 2021. For the evaluation of uncertainty quantification, the method is overconfident but produces meaningful ensemble spreads that can distinguish good and bad forecasts. The quality of CGAN outputs is also evaluated. Results show that the CGAN outputs behave similarly to a numerical ensemble; they preserved the intervariable correlations and the contribution of influential predictors as in the original HRRR forecasts. This work provides a novel approach to postprocess CAM output using neural networks that can be applied to severe weather prediction.
Significance Statement
We use a new machine learning (ML) technique to generate probabilistic forecasts of convective weather hazards, such as tornadoes and hailstorms, with the output from high-resolution numerical weather model forecasts. The new ML system generates an ensemble of synthetic forecast fields from a single forecast, which are then used to train ML models for convective hazard prediction. Using this ML-generated ensemble for training leads to improvements of 10%–20% in severe weather forecast skills compared to using other ML algorithms that use only output from the single forecast. This work is unique in that it explores the use of ML methods for producing synthetic forecasts of convective storm events and using these to train ML systems for high-impact convective weather prediction.
Abstract
Weather radar data are critical for nowcasting and an integral component of numerical weather prediction models. While weather radar data provide valuable information at high resolution, their ground-based nature limits their availability, which impedes large-scale applications. In contrast, meteorological satellites cover larger domains but with coarser resolution. However, with the rapid advancements in data-driven methodologies and modern sensors aboard geostationary satellites, new opportunities are emerging to bridge the gap between ground- and space-based observations, ultimately leading to more skillful weather prediction with high accuracy. Here, we present a transformer-based model for nowcasting ground-based radar image sequences using satellite data up to 2-h lead time. Trained on a dataset reflecting severe weather conditions, the model predicts radar fields occurring under different weather phenomena and shows robustness against rapidly growing/decaying fields and complex field structures. Model interpretation reveals that the infrared channel centered at 10.3 μm (C13) contains skillful information for all weather conditions, while lightning data have the highest relative feature importance in severe weather conditions, particularly in shorter lead times. The model can support precipitation nowcasting across large domains without an explicit need for radar towers, enhance numerical weather prediction and hydrological models, and provide radar proxy for data-scarce regions. Moreover, the open-source framework facilitates progress toward operational data-driven nowcasting.
Significance Statement
Ground-based weather radar data are essential for nowcasting, but data availability limitations hamper usage of radar data across large domains. We present a machine learning model, rooted in transformer architecture, that performs nowcasting of radar data using high-resolution geostationary satellite retrievals, for lead times of up to 2 h. Our model captures the spatiotemporal dynamics of radar fields from satellite data and offers accurate forecasts. Analysis indicates that the infrared channel centered at 10.3 μm provides useful information for nowcasting radar fields under various weather conditions. However, lightning activity exhibits the highest forecasting skill for severe weather at short lead times. Our findings show the potential of transformer-based models for nowcasting severe weather.
Abstract
Weather radar data are critical for nowcasting and an integral component of numerical weather prediction models. While weather radar data provide valuable information at high resolution, their ground-based nature limits their availability, which impedes large-scale applications. In contrast, meteorological satellites cover larger domains but with coarser resolution. However, with the rapid advancements in data-driven methodologies and modern sensors aboard geostationary satellites, new opportunities are emerging to bridge the gap between ground- and space-based observations, ultimately leading to more skillful weather prediction with high accuracy. Here, we present a transformer-based model for nowcasting ground-based radar image sequences using satellite data up to 2-h lead time. Trained on a dataset reflecting severe weather conditions, the model predicts radar fields occurring under different weather phenomena and shows robustness against rapidly growing/decaying fields and complex field structures. Model interpretation reveals that the infrared channel centered at 10.3 μm (C13) contains skillful information for all weather conditions, while lightning data have the highest relative feature importance in severe weather conditions, particularly in shorter lead times. The model can support precipitation nowcasting across large domains without an explicit need for radar towers, enhance numerical weather prediction and hydrological models, and provide radar proxy for data-scarce regions. Moreover, the open-source framework facilitates progress toward operational data-driven nowcasting.
Significance Statement
Ground-based weather radar data are essential for nowcasting, but data availability limitations hamper usage of radar data across large domains. We present a machine learning model, rooted in transformer architecture, that performs nowcasting of radar data using high-resolution geostationary satellite retrievals, for lead times of up to 2 h. Our model captures the spatiotemporal dynamics of radar fields from satellite data and offers accurate forecasts. Analysis indicates that the infrared channel centered at 10.3 μm provides useful information for nowcasting radar fields under various weather conditions. However, lightning activity exhibits the highest forecasting skill for severe weather at short lead times. Our findings show the potential of transformer-based models for nowcasting severe weather.
